airship
/
shipyard
Code Issues Proposed changes
A cluster lifecycle orchestrator for Airship.
476 Commits 3 Branches 0 Tags 8.1 MiB
Python 95.7%
Shell 3.1%
Smarty 0.8%
Makefile 0.4%
Go to file
Bryan Strassner e59fb314c1 Set ULID of action on DAG request 
Sets the run_id for a DAG invoked in Airflow to the same ULID assigned
to it in Shipyard. While this was already happening as a parameter to
the DAG being invoked, by making it the run_id, further correlation is
possible, at a level that both Shipyard and the Airflow framework are
aware.

As part of making this change, fragility was uncovered in the
rest_api_plugin that expedited the need to switch to the built-in, but
experimental airflow API to trigger a dag (one of two API endpoints
provided - this is important later in this story). In any case, the 3rd
party rest_api_plugin was removed.

As a result of the rest_api_plugin being removed:
1) the simpleton helm test to check the api of airflow was also removed
(it used the version endpoint of this plugin). As the built-in api
provides no version endpoint or similarly accessible-without-being-stateful
endpoint, the helm test had no new place to look for something to call.
2) Some clean up of exclusions and documentation was possible - test
coverage, security exclusions, left over documentation remnants

Change-Id: I0b68496a8500408b776b4acc12888aa017c4c7d2
 		2018-08-10 10:23:30 -05:00
charts/shipyard Set ULID of action on DAG request 2018-08-10 10:23:30 -05:00
docs Set ULID of action on DAG request 2018-08-10 10:23:30 -05:00
etc/shipyard Refactor shipyard to UCP target layout 2018-04-24 16:47:13 -05:00
images Update Dockerfile to allow override of FROM variable 2018-07-17 16:35:42 -05:00
src/bin Set ULID of action on DAG request 2018-08-10 10:23:30 -05:00
tools Add no_proxy to docker base command 2018-08-01 11:21:44 -05:00
.dockerignore [refactor] logging refactor + redaction filter 2018-06-01 17:14:15 -05:00
.editorconfig Cleanup dockerfile and add editorconfig 2018-02-16 13:44:15 -06:00
.gitignore Workflow to support deployment groups 2018-06-20 09:55:15 -05:00
.gitreview Update .gitreview for openstack infra 2018-05-17 19:26:55 +01:00
.zuul.yaml fix(gate): fix airship-shipyard-ubuntu job 2018-07-06 04:17:13 +00:00
LICENSE Add Apache 2.0 LICENSE file 2018-05-14 13:46:28 +00:00
Makefile Update Dockerfile to allow override of FROM variable 2018-07-17 16:35:42 -05:00
README.rst Update Deckhand commit 2018-06-01 21:16:08 +01:00
requirements.readthedocs.txt Refactor shipyard to UCP target layout 2018-04-24 16:47:13 -05:00
tox.ini fix tox python3 overrides 2018-07-27 13:13:02 +07:00

README.rst

Shipyard

Shipyard adopts the Falcon web framework and uses Apache Airflow as the backend engine to programmatically author, schedule and monitor workflows.

The current workflow is as follows:

  1. Initial region/site data will be passed to Shipyard from either a human operator or Jenkins
  2. The data (in YAML format) will be sent to Deckhand for validation and storage
  3. Shipyard will make use of the post-processed data from DeckHand to interact with Drydock.
  4. Drydock will interact with Promenade to provision and deploy bare metal nodes using Ubuntu MAAS and a resilient Kubernetes cluster will be created at the end of the process
  5. Once the Kubernetes clusters are up and validated to be working properly, Shipyard will interact with Armada to deploy OpenStack using OpenStack Helm
  6. Once the OpenStack cluster is deployed, Shipyard will trigger a workflow to perform basic sanity health checks on the cluster

Note: This project, along with the tools used within are community-based and open sourced.

Mission

The goal for Shipyard is to provide a customizable framework for operators and developers alike. This framework will enable end-users to orchestrate and deploy a fully functional container-based Cloud.

Getting Started

This project is under development at the moment. We encourage anyone who is interested in Shipyard to review our documentation.

Bugs

If you find a bug, please feel free to create a Storyboard issue.