81066ae98f
This PS updates python modules and code to match Airflow 2.6.2 as well as deploys new Airflow: - bionic py36 gates were removed - python code corrected to match new modules versions - selection of python modules versions was performed based on airflow-2.6.2 constraints - airskiff deploy pipeline was aligned with latest in treasuremap v1.9 - shipyard chart was corrected to match new airflow cli, configuration items and their default values - added new celery configuration items and their values - updated airflow runtime logging config - disabled deprecation and future python warnings in airflow images - added celery to the list of airflow providers - adjusted airflow runtime scripts to match new cli - shipyard SQL queries to airflow DB were adjusted to match new SQL schema of the db - shipyard_airflow and shipyard_client unit tests were updated to match new DB structure and new cli - airflow db sync job is using db upgrade command - helm version uplifted to v3.12.2 Change-Id: Ife88e53ce0dd8dc77bf267de1f5e6b8361ca76fd |
||
|---|---|---|
| .. | ||
| alembic | ||
| etc/shipyard | ||
| generator | ||
| shipyard_airflow | ||
| tests | ||
| .coveragerc | ||
| README.rst | ||
| alembic.ini | ||
| entrypoint.sh | ||
| requirements-direct.txt | ||
| requirements-frozen.txt | ||
| requirements.txt | ||
| setup.cfg | ||
| setup.py | ||
| test-requirements.txt | ||
| tox.ini | ||
README.rst
Shipyard
Shipyard is the directed acyclic graph controller for Kubernetes and OpenStack control plane life cycle management, and a component of the Airship Undercloud Platform (UCP).
Shipyard provides the entrypoint for the following aspects of the control plane established by the Airship:
<dt>
Designs and Secrets
</dt>
<dd>
Site designs, including the configuration of bare metal host
nodes, network design, operating systems, Kubernetes nodes,
Armada manifests, Helm charts, and any other descriptors that
define the build out of a group of servers enter the Airship via
Shipyard. Secrets, such as passwords and certificates use the
same mechanism. <br />
The designs and secrets are stored in Airship's Deckhand,
providing for version history and secure storage among other
document-based conveniences.
</dd>
<dt>
Actions
</dt>
<dd>
Interaction with the site's control plane is done via
invocation of actions in Shipyard. Each action is backed by
a workflow implemented as a directed acyclic graph (DAG) that
runs using Apache Airflow. Shipyard provides a mechanism to
monitor and control the execution of the workflow.
</dd>Find more documentation for Shipyard on Read the Docs
Integration Points:
OpenStack Identity
(Keystone) provides authentication and support for role based
authorization
Apache Airflow provides the framework and automation of workflows provided by Shipyard
PostgreSQL is used to persist information to correlate workflows with users and history of workflow commands
Deckhand supplies storage and management of site designs and secrets
Drydock is orchestrated by Shipyard to perform bare metal node provisioning
Promenade is indirectly orchestrated by Shipyard to configure and join Kubernetes nodes
Armada is orchestrated by Shipyard to deploy and test Kubernetes workloads
Apache Airflow provides the framework and automation of workflows provided by Shipyard
PostgreSQL is used to persist information to correlate workflows with users and history of workflow commands
Deckhand supplies storage and management of site designs and secrets
Drydock is orchestrated by Shipyard to perform bare metal node provisioning
Promenade is indirectly orchestrated by Shipyard to configure and join Kubernetes nodes
Armada is orchestrated by Shipyard to deploy and test Kubernetes workloads