128 lines
4.6 KiB
Markdown
128 lines
4.6 KiB
Markdown
# aic-kube MaaS Orchestration #
|
|
|
|
## Inventory/Design Schema ##
|
|
|
|
Would be nice to see this get out of Git-managed flat-files and into
|
|
an API accessible datastore that allows more fine-grained queries
|
|
and supports inheritance/composition natively
|
|
|
|
### Requirements ###
|
|
|
|
1. Use profile inheritance to minimize data repitition
|
|
2. Future-proof to support configuration items (CIs) we consider static today
|
|
3. Support composition to allow federated source of truth
|
|
|
|
### Components ###
|
|
|
|
#### bootstrap_seed.yaml ####
|
|
|
|
Probably needs renamed, but I see this as being the Formation data
|
|
that specifies explicit values - hostnames, IP address assignments,
|
|
VLAN IDs, etc...
|
|
|
|
#### bootstrap_hwdefinition.yaml ####
|
|
|
|
Looks to be mainly focused on initial OOB control and getting a server
|
|
into PXE mode. Should be able to leave as-is and adopt existing
|
|
Apollo logic for IPMI.
|
|
|
|
#### bootstrap_profiles.yaml ####
|
|
|
|
Currently doesn't exist, but I think it makes sense to separate the
|
|
data that describes the design (like general profiles) from the data
|
|
that ties the design to explicit assignments (i.e. Formation data)
|
|
|
|
### Questions/Issues ###
|
|
|
|
* Need DNS specs in YAML - where to include it, possibly per network
|
|
making it effective when that network is chosen as "primary"
|
|
* Can set comma-delimited list of DNS servers per subnet
|
|
* Consider YAML pointers and anchors to diminish how much inheritance
|
|
has to be calculated in code
|
|
* Does not appear to support key subtraction
|
|
* No way to merge hashes based on a 'primary' key
|
|
* Scott thinks best to keep with managing inheritance in code. Can minimize
|
|
magic via comments in YAML
|
|
* Best way of specifying hardware paths (eth0 may not be consistent across boots)
|
|
* PCI bus address
|
|
* For NIC lshw should tie a PCI address to a MAC (serial number) and logical
|
|
name (eth0)
|
|
* MaaS API is keyed to logical name
|
|
* SCSI bus address
|
|
* How to specify custom hardware drivers
|
|
* OOB config - /etc/maas/drivers.yaml specify the suite of drivers available
|
|
to MaaS outside what is built into the streams. MaaS will automatically
|
|
consume a driver when it detects hardware that needs it. This config is not
|
|
API accessible.
|
|
* Expound on storage, at least specifying primary boot drive
|
|
|
|
## Orchestrator Service Fabric ##
|
|
|
|
Consider Go over Python considering performance differences, concurrency support
|
|
and ease of deploying static binaries.
|
|
|
|
### Requirements ###
|
|
|
|
1. Make interfaces at both ends pluggable
|
|
2. Support metadata API for node introspection
|
|
3. Support future addition of network and storage orchestration
|
|
|
|
### Services ###
|
|
|
|
#### Design Consumer ####
|
|
|
|
Pluggable service to ingest a inventory/design specification, convert it to a standard
|
|
internal representaion, and persist it to the Design State API. Initial implementation
|
|
is the consumer of AIC YAML schema.
|
|
|
|
#### Design State API ####
|
|
|
|
API for querying and updating the current design specification and persisted orchestration status.
|
|
CRUD support of CIs that are not bootstrap-related, but can be used by other automation.
|
|
|
|
#### Control API ####
|
|
|
|
User-approachable API for initiating orchestration actions or accessing other internal
|
|
APIs
|
|
|
|
#### Infrastructure Orchestrator ####
|
|
|
|
Handle validation of complete design, ordering and managing downstream API calls for hardware
|
|
provisioning/bootstrapping
|
|
|
|
#### Server Driver ####
|
|
|
|
Pluggable provisioner for server bootstrapping. Initial implementation is MaaS client.
|
|
|
|
#### Network Driver ####
|
|
|
|
Pluggable provisioner for network provisioning. Initial implementation is Noop.
|
|
|
|
#### Introspection API ####
|
|
|
|
API for bootstrapping nodes to load self data. Possibly pluggable as this is basically an
|
|
authenticated bridge to the Design State API
|
|
|
|
### Questions ###
|
|
|
|
1. Should this drive the MaaS configuration or only drive node deployment?
|
|
2. What do we need commissioning scripts for?
|
|
* Firmware updates
|
|
* Hardware RAID configs
|
|
* Hardware manifest validation (as designed/as built)
|
|
3.
|
|
|
|
## Resources ##
|
|
|
|
* [MaaS Node Deployment](https://insights.ubuntu.com/2016/01/23/maas-setup-deploying-openstack-on-maas-1-9-with-juju/)
|
|
* [MaaS Network Provisioning](https://insights.ubuntu.com/2016/01/31/nodes-networking-deploying-openstack-on-maas-1-9-with-juju/)
|
|
* [Custom Curtin Scripts](http://caribou.kamikamamak.com/2015/06/26/custom-partitioning-with-maas-and-curtin-2/)
|
|
* [curtin_userdata for bootstrapping K8](https://gist.github.com/aric49/4d424929346e7fbb3ddbd87c7a49ba67)
|
|
* [YAML anchors and pointers](http://blog.daemonl.com/2016/02/yaml.html)
|
|
* [YAML Wikipedia](https://en.wikipedia.org/wiki/YAML)
|
|
* [MaaS 3rd Party Drivers](http://people.canonical.com/~jlane/third-party-drivers.html)
|
|
* [MaaS API Spec](https://maas.ubuntu.com/docs2.0/api.html)
|
|
* [VMware Go IPMI](https://github.com/vmware/goipmi)
|
|
|
|
|