Readthedocs failed to render Drydock exceptions with error:
> WARNING: autodoc: failed to import exception xxx from module
> 'drydock_provisioner'; the following exception was raised: No module
> named 'drydock_provisioner'
Trying to add Drydock requirements to the installed requirements list,
so that Readthedocs has all modules, including those needed for the
Drydock itself.
Unify docs building by utilizing Zuul docs-on-readthedocs template job.
Cosmetic readability changes:
1. combined all Makefile .PHONY targets into one
2. merged multiple LABEL instructions in Dockerfile into one
Change-Id: I6a9b47cffc66d739968fa886c51e25b1e09ef124
- Update the maasdriver to support two concurrent
rack controllers when configuring networking.
- Identify a baremetal node from both the MAAS node and
rack controller collections
- Relax validation to only require at least one healthy
rack controller
Change-Id: I04beb56a8212b65061840021b13b412fbb37ae81
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes
Change-Id: I64fb508a3ff72258d82c795193e918f2945deb29
This patch set changes Drydock's exceptions documentation
(contained underneath operators guide) because it isn't rendering
correctly as a list table on RTD (the autoexception information
is missing) [0].
The easy fix is to change the tabularized view (list table)
into basically a series of autoexception classes which sufficiently
captures the level of detail required, anyway.
Note that running `tox -e docs` locally and opening the resulting
index.html page appears to work -- but not when hosted on RTD.
[0] https://airship-drydock.readthedocs.io/en/latest/exceptions/exceptions.html
Change-Id: Ie4bc01f1fe6aee9b9a58fd956f5b350df44bef51
- Clean up the docs and code around bootactions to support the
baclient work
- Update the Jinja2 environment in the bootaction rendering
- Refactor authentication of bootaction signal API calls
Change-Id: Ic64f0c7ee09a487be750188953013f1ed3cd99cb
Add additional tunables to how SQLalchemy creates connection pools
Also start including static doc assets to they can be published
Change-Id: I268dc265a6b6cf1a200b235a5f99e65e89a95637
- Add a validator for bootactions to warn if a node doesn't
have at least one
- Add a validator for bootactions to error if a package
version specifier is invalid
- Unit tests for the validation
Change-Id: I61d8aa3831791af0484498e6fe9f7c1c83dbf540
- Support a list of debian packages as a bootaction asset
- Add unit testing for parsing the additional bootaction information
- Add __eq__ and __hash__ for DocumentReference to allow checking
equality and list presence
Change-Id: I0ca42baf7aae6dc2e52efd5b311d0632e069dd79
- New diagrams and documents for developer overview
- Update conf.py for docs to work w/ readthedocs.io
- Add policy and config gen to `make docs`
- Update zuul-linter to support checked in images
- Last fix to document publishing
Change-Id: I4faa1b87032ae5b0e786aa0fd998f809124b7987
- Add validations that OOB configs for nodes are valid for the
oob type defined
- Add documentation for using Drydock/MAAS to deploy libvirt VMs
- Add logic to update the MAAS node power parameters to allow power
control of libvirt VMs
Change-Id: Ia7d5fbd1659636d46cf1790fe3fc66ca6c6fee89
- Changed the expected input from site_design to design_ref.
- Converts the design_ref json/dict into a SiteDesign object.
Change-Id: Ie1f6738fec64749cde9d2843d9373175006794c0
- Added builddata, subtaskerrors, and layers as options to get_task.
- Added get_nodes_for_filter to call POST /nodefilter.
- Updated endpoint for POST /nodes to POST /nodefilter.
- Made node_filter optional in POST /nodefilter.
Change-Id: I456a6e9991d03af3d375c448f5cbf07a21e91f1d
Hostnames with underscores caused a deployment failure.
Update to use a double underscore as a delimiter and provide
a design validator to check that hostnames do not contain it.
Closes #78
Change-Id: Ib148aed5cffe7fd8bc08441eaef8a45af6601bdd
- Added a new query param 'layers=x' to /tasks/{task_id}
where x is a number.
This param allows the API to drill down to all the subtasks.
If x is -1 it will reeturn all subtasks, otherwise it will return
x layers of subtasks.
- Added a new query param 'subtaskerrors=true' to /tasks/{task_id}.
If true then any errors from subtasks will be included in the
subtask_errors element in the response.
- Added a POST to /nodes that requires node_filter and site_design
in the body. Then uses those parameters to return a list of nodes.
Change-Id: I9b4c06dd1c9de2b0500092aa4b4bfacac08eac54
- Add hugepages and cpu_sets stanzas to HardwareProfile as the
size and count of hugepages and the exact CPUs to pin for SRIOV
are dependent on hardware.
- Add sriov stanza to a node interface to specify vf_count and
trustedmode. These will be passthrough values as Drydock doesn't
configure SRIOV.
- Add sriov information to the bootaction context so it can be written
to disk on a deployed node if needed
- Allow an interface configuration to be skipped when an interface has
no defined network_link for things like SR-IOV interfaces.
- Add kernel parameter reference support to access hardware profile
information
- Add unit tests
- Update topology documentation for usage of HardwareProfile
and kernel parameter references
Change-Id: Iefd326f5c6fad19dbd21300ee249019a3dfd4848
- Add list of build data to detailed task response when builddata=true
specifed in query string
- Add new endpoint for /nodes/nodename/builddata to retrieve
build data for a particular node
- Update docs for new API capabilities
- Testing all around
Change-Id: If0fcd2962d4389789af45ad1fbe61d226ac6a403
For Bootactions that use the template pipeline
segment, add the `action.design_ref` field to the
context for Jinja2.
- Add design_ref to the render_assets method signature
- Update calls to include the contextual design_ref
- Add unit test for design_ref rendering
Change-Id: I81017e050b6f1c0a3e66ee824ecb8ffd154e45dd
Route domains will allow multiple L3 networks
to self organize static routes. This allows additions
of L3 networks with manually updating previously deployed
networks.
- Fix a YamlIngester error with labels on NetworkLinks
- Update the CLI --block option
- Add routedomains attribue to Networks in schema and object model
- Add routedomain documentation
- Add unit test for routedomain route generation
- Add unit test coverage reporting
Change-Id: I059d2eae6da84c4f9ad909f0287432e6cf0970d0
- Implement boot action rendering and API
- Reorganize DB integration tests and add a tox -e postgres entrypoint
- Add boot action unit tests
- Add node filter unit test
- Add boot action context creation to deployment workflow
- Fix regression bug in MaaS Machines model
- Downgrade to Python 3.5 due to CICD limitations
Change-Id: I6c8f100cbe209f9b1c6c6ff1285365d89343ae2a
This design is for the framework of deploying actions and assets to
nodes being deployed by Drydock. Included is an API for nodes to
report back the result status of any running boot actions.
Change-Id: I2becd6af33d0202d0da73bab01f21b010fc3ea60
Refactor orchestrator to break large
monolithic functions into small functions
per action.
- Update orchestrator to match new statemgmt API
- Pull most code out of __init__.py files
- Create action classes for Orchestrator actions
- Create action classes for Driver actions
- Orchestrator consumes tasks from database queue
- Additional encapsulation of task functionality into Task class
- Create shared integration test fixtures
- Fix Sphinx entrypoint so package install works
- Disable bootdata API until BootAction implementation
- Bring codebase into PEP8 compliance
- Update documentation reflect code changes
- Mark SQL #nosec for bandit
Change-Id: Id9a7bdedcdd5bbf07aeabbdb52db0f0b71f1e4a4
Refactor the task model and add database persistence for tasks.
- Document task schema
- Use Alembic for database creation
- Use SQLalchemy for database abstraction
- Update Task status and result enumerations
Change-Id: I247b88f293144a0bdf891958e19711d975c729ba
This PS adds tooling and automation to automatically generate
Drydock's documentation into feature-rich HTML pages that can
be hosted.
To run the documentation job, simply execute:
tox -e docs
A future PS should add warning_is_error to 'build_sphinx' in
setup.py once the import warnings are addressed.
Change-Id: I91a3c585b2c27096e7fde12d180638d1ae4bdb81
Implement interface bonding via the MaaS
API
- Documentation on writing topology definition of networking and host
network attachment
- Adjust topology YAML schema for interface definition
- Add MaaS API support for create_bond
- Fix some bugs from Gerrit #377818
- Update MaaS API client to support multi-select options
Change-Id: I1c42300ede3f67595ebc8029b0f375622b459254
- Run codebase through YAPF for formatting
- Add tox configuration for yapf and pep8
- Fix some non-YAPF pep8 failures
- Enhance verify_site for better MaaS-integration testing
- Create initial basic functional test
Change-Id: Ie5b5275d7795693a6551764362aee916b99b3e56
Move sample config to etc/drydock
Update docs to generate a config with tox
Update configuration for Keystone
- Add config generation to tox.ini
- Fix default in bootdata config
- Add keystone dependencies
- Add config generator config
- Move sample config to a skeleton etc/drydock tree
Use PasteDeploy for WSGI integration
Using keystonemiddleware outside of a PasteDeploy
pipeline is deprecated. Move Drydock to use PasteDeploy
and integrate with keystonemiddleware
Update Falcon context object
Add keystone identity fields to context object
Clean up context marker field
Fix AuthMiddleware for keystone
Update falcon middleware to harvest headers injected
by keystonemiddleware
Fix context middleware
Update context middleware to enforce
a UUID-formatted external context marker
Lock keystonemiddleware version
Lock keystonemiddleware version to the Newton release
Sample drydock.conf with keystone
This drydock.conf file is known to integrate successfully
with Keystone via keystonemiddleware and the password plugin
Add .dockerignore
Stop adding .tox environment to docker images
Integrate with oslo.policy
Add oslo.policy 1.9.0 to requirements (Newton release)
Add tox job to generate sample policy.yaml
Create DrydockPolicy as facade for RBAC
Inject policy engine into API init
Create a DrydockPolicy instance and inject it into
the Drydock API resources.
Remove per-resource authorization
Update Drydock context and auth middleware
Update Drydock context to use keystone IDs instead of names as required
by oslo.policy
Update AuthMiddleware to capture headers when request provides
a service token
Add RBAC for /designs API
Add RBAC enforcement for GET and POST of
/api/v1.0/designs endpoint
Refactor check_policy
Refactor check_policy into the base class
Enforce RBAC for /designs/id endpoint
Enforce RBAC on /designs/id/parts endpoint
Enforce RBAC on /designs/id/parts/kind
Enforce RBAC on /designs/id/parts/kinds/
Enforce RBAC on /tasks/ endpoints
Create unit tests
- New unit tests for DrydockPolicy
- New unit tests for AuthMiddleware w/ Keystone integration
Address impacting keystonemiddleware bug
Use v4.9.1 to address https://bugs.launchpad.net/keystonemiddleware/+bug/1653646
Add oslo_config fixtures for unit testing
API base class fixes
Fix an import error in API resource base class
More graceful error handling in drydock_client
Create shared function for checking API response status codes
Create client errors for auth
Create specific Exceptions for Unauthorized
and Forbidden responses
Ignore generated sample configs
Lock iso8601 version
oslo.versionedobjects appears to be impcompatible with
iso8601 0.1.12 on Python 3.2+
Update docs for Keystone
Note Keystone as a external depdendency and
add notes on correctly configuring Drydock for
Keystone integration
Add keystoneauth1 to list_opts
Explicitly pull keystoneauth password plugin
options when generating a config template
Update reference config for keystone
Update the reference config template
for Keystone integration
Add keystoneauth1 to requirements
Need to directly include keystoneauth1 so that
oslo_config options can be pulled from it
Update config doc for keystoneauth1
Use the keystoneauth1 generated configuration options
for the configuration docs
Remove auth options
Force dependence on Keystone as the only authentication
backend
Clean up imports
Fix how falcon modules are imported
Default to empty role list
Move param extraction
Enforce RBAC before starting to parse parameters
Implement DocumentedRuleDefault
Use DocumentedRuleDefault for policy defaults at request
of @tlam. Requires v 1.21.1 of oslo_policy, which is tied
to the Pike openstack release.
Change sample output filenames
Update filenames to follow Openstack convention
Fix tests to use hex formatted IDs
Openstack resource IDs are not hyphenated, so update
unit tests to reflect this
Fix formating and whitespace
Refactor a few small items for code review
Update keystone integration to be more
robust with Newton codebase
Centralize policy_engine reference to
support a decorator-based model
RBAC enforcement decorator
Add units tests for decorator-based
RBAC and the tasks API
Minor refactoring and format changes
Change-Id: I35f90b0c88ec577fda1077814f5eac5c0ffb41e9