This PS adds default values for chart values and resolves some issues
in python code that utilizes these values:
validation_connect_timeout: 20
validation_read_timeout: 300
deckhand_client_connect_timeout: 20
deckhand_client_read_timeout: 300
drydock_client_connect_timeout: 20
drydock_client_read_timeout: 300
Change-Id: Ic5b1920257859239613a3ce77134e6b05bd7e9dd
This PS is created to :
- roll back apache-airflow back to 1.10.5
- change default chart values from bionic to ubuntu_focal
- save freshly generated config and policy files samples
- in focal Dockerfile postgres client version is pubped up to v15
- change airflow docker image requirements from limited to fully frozen for shipyard-airflow project
- adjusted requirements-direct.txt for shipyard_airflow
- adjusted requirements-direct.txt for shipyard_client
- regenerated requirements-frozen for both projects
- fixed unit tests after upgrading click module
- gen_all tox profile processing has been moved over to py8 gate because it requires focal zuul node to run
- upgraded airskiff gate playbook to include latest treasuremap patchset with updated airskiff site admada manifests
Change-Id: I47e44f5cfa19b2649697e7cc5a31557a6f4fcfea
- upgraded Airflow to 1.10.15 -
https://airflow.apache.org/docs/apache-airflow/1.10.15/changelog.html
- disabled xenial, bionic and opensuse images build gates
- added focal image build gate
- added focal zuul build node
- adjusted Makefile for focal
- added bindep.txt to utilize bindep zuul base role for zuul build node
pre-setup
- added focal Dockerfile
- implemented freeze requirements.txt approach like in other Airship
projects
- removed specific requirements.txt for airflow in favor of using
requirements-frozen.txt from shipyard_airflow project when building
airflow docker image
- fixed docker image publishing to Quay
- replaces deprecated LOG.warn with new LOG.warning call
- replaced deprecated body attribute in responce wiht responce.text
attribute
- update of falcon module deprecated .API call - replaced wiht
falcon.App call
- deprecated routing.create_http_method_map method replaced with
routing.map_http_methods
- re-formatted code tabulations based on yapf recommendations
- replaced deprecated protocol attribute in Pytest create_environ() with
http_version attribute
- replaced deprecated app attribute in Pytest create_environ() with
root_path attribute
- fixed airflow CLI commands to match 1.10.15 version
- updated zuul gates to work on focal nodes and added focal specific
node setup items by adding appriate ansible tasks and roles
- uplifted Helm to 3.9.4
- uplifted stable HTK commit id
- updated tox.in to with with tox v4
- uplifted dependences references to other Airship projects
- common python dependences were syncronized with other Airship
projects(Promenade, Deckhand, Armada, Drydock)
- fixed airskiff deployment gate
- fixed genconfig* profiles in shipyard-airflow tox.ini responsible for
maintanance of policy.yaml.sample and shipyard.conf.sample
Change-Id: I0c85187dc9bacf0849382563dd5ff7e9b2814c59
Added support to buid shipyard and airflow images using either a xenial
or Ubuntu bionic base image.
The default base image is set to bionic.
Change-Id: I6ad4d42dede081586b3ccea87a42e250979ac106
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.
Note that redirects will be set up as well but let's point now to the
new location.
For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html
Change-Id: I09b5d45fb5ca505b1f97d0a78cc082c1e0254edd
Follow up change for deployment-status ConfigMap [0] adding a few
things that were either missed or need to be added, especially unit
tests for the new DeploymentStatusOperator class
[0] https://review.opendev.org/#/c/660197/
Change-Id: Ieaf55dc7725ea92062fde7997a676726b2396253
Adding DISTRO parameter for makefile to invoke distribution specific
Dockerfile and build image accordingly.
Also qualifying image job name to distinguish ubuntu vs opensuse
check jobs and passing distro specific variable.
Similar logic for pushing distro specific images.
Added new document section for multiple distro support.
Change-Id: I215f8a107cab8770181c0472e0cc29053b2b978e
This PS adds funtionality to Shipyard to validate the existence of
the Pegleg-generated "deployment-version" document (Pegleg change id:
I7919b02d70c9797f689cdad85066d3953b978901) when a user runs create
configdocs. This validation only checks the presence of the document
(by name and schema) and does not care about the document's other
contents.
The severity of a failed validation is configurable through the
"validations" config section in shipyard.conf. The default severity
is "Skip", meaning the validation is not ran at all.
Note that with the default configuration of new validation, Shipyard
functionality should be unchanged.
Change-Id: I754617de81f628a24232e890b12b157ba6731c25
This PS adds funtionality to Shipyard to validate the existence of
the Pegleg-generated "deployment-version" document (Pegleg change id:
I7919b02d70c9797f689cdad85066d3953b978901). As implied, this new
validation only checks for the presence of the document (by name and
schema) and currently does not care about any of the document's
contents under "data".
The severity of a failed validation is configurable through the new
"validations" configuration section in shipyard.conf, and is
defaulted to skip the validation altogether. This means that by
default, this patch set does not alter the functionality of Shipyard
Note that with the default configuration of this new validation,
Shipyard functionality should be unchanged.
Change-Id: I5e7269066f769804710a0fd1f2c8d0aece0d3314
Currently, any document name or schema referenced in the Shipyard
code base is a hard-coded string. Often times, these strings are
repeated throughout the code. This patch set adds a new configuration
section to shipyard.conf to define document names and schemas so they
can then be referenced in the Shipyard code via the oslo
configuration object. This functionality will be important for
upcoming Shipyard features which will call for more documents to be
validated as well as some new Shipyard-created docs.
Change-Id: I34ae8cd578bab730d004c3d176e3817b5a45c89e
Moves the airflow web server container from its own pod into the
Shipyard pod. This removes exposed network surface area from the
Shipyard suite of software. Shipyard, after this change accesses the
Airflow API using localhost in the same k8s pod.
Change-Id: Ied4bd415a8d78c393b7256ead27a6a2176f4a2d6
Moves the airflow sceduler to a container in the airflow-worker
statefulset so that its version lifecycle matches that of the worker.
Leaves the stand-alone scheduler in place to support upgradability from
prior installations that included a standalone scheduler. New
installations are advised to turn off the scheduler template from
rendering using the values.yaml flag.
This is an attempt to make disruptive upgrades to airflow less impactful
to a "update_site" action from Shipyard.
Additionally this removes the template for airflow-flower, which is not in use.
Change-Id: I0608793ee6aba1eb3ce0f5e9567655287014a0ca
Adds an option to create configdocs as an empty colleciton. This is done
as an explicit flag (--empty-collection) on the command as opposed to
using empty files to prevent accidental emtpy file loads leading to
frustration.
Since this introduced a new flag value for the CLI, the CLIs using flag
values were updated to use the standard is_flag=True instead of the
flag_value=True or some other value when a boolean flag is expected.
Minor updates to CLI tests due to moving to responses 0.10.2
Depends-On: https://review.openstack.org/#/c/614421/
Change-Id: I489b0e1183335cbfbaa2014c1458a84dadf6bb0b
- Adds a query parameter 'cleartext-secrets' to get full raw documents.
- Adds CLI flag to get full raw documents.
Change-Id: If38974c8433c8360cc47ae1273720ad76e87a6fd
While iterating on the next steps of using notes, it became clear that
several changes to the output and access methods for notes needed
enhancements. This change introduces a new way to access a note's URL
information via a new API/CLI, while removing the resolution of URLs
from the existing note output. This supports the concept of "builddata"
coming back with sizes of 800kb or more - which really can never work
out inline in other data, especially in cases where there is
multiplicity of the information across many items.
New API: GET /notedetails/{id}
CLI: shipyard get notedetails/{id} and/or shipyard get notedetails {id}
Returns the resolution of the URL for a note, outputting the raw info as
the response (not structured in a JSON response).
The CLI will attempt to minimally format the response if it has inline
\n characters by replacing them will real newlines in the output (if the
output-format is set to either cli or format. Raw format will be
returned as-is.
The existing notes responses are changed to not include the resolution
of the URL information inline, but rather provide the text:
Details at notedetails/{id}
The CLI will interpret this and present:
- Info available with 'describe notedetails/09876543210987654321098765'
This is an attempt to inform the user to access the note details that
way - luckily the API and CLI align on the term notedetails, as the word
details works well enough in the singular form presented by the CLI and
the plural form used by the API.
The ID returned is the unique id of the note (ULID format).
Notes that have no URL will return a 404 response from the API (and
an appropriately formatted value from the CLI).
This approach solves an issue beyond the large inline values from URLs;
providing a means to NOT resolve the URLs except in a one-at-a-time way.
Long lists of notes will no longer have the risk of long waits nor
needing of parallelization of retrieval of URLs for notes.
This change introduces an API-side sorting of notes by timestamp,
providing a chronological presentation of the information that may or
may not match the ULID or insertion ordering of the notes.
Additional feedback from peers about the output of noted indicated that
the CLI formatting of notes in general was in need of visual tuning. As
such, this change introduces changes to the formatting of the output
of notes from the CLI:
- Notes for describing an item will be presented with a more specific
header, e.g.: Action Notes: or Step Notes: instead of simply Notes.
- Tables with notes will change the header from "Notes" to "Footnotes"
give the user a better marker that the notes follow the current
table.
- Table footnotes will be presented in a table format similar to
the following, with headings matching the kind of note being
produced.
Step Footnotes Note
(1) > blah blah blah
> yakkity yakkity
(2) > stuff stuff stuff stuff stuff stuff stuff
stuff stuff stuff
- Info available with 'describe notedetails/...
> things things things
Change-Id: I1680505d5c555b2293419179ade995b0e8484e6d
Enhance the Shipyard API and CLI to retrieve notes that have been
specified against actions and steps. Includes a new reusable parameter
for verbosity.
Change-Id: I1c7f47c0346ce783dacd62b8bbc1fd35a0bf285b
Adds the common/shared code to support notes in Shipyard such that this
component can be reused between the Shipyard API and the workflows.
Includes a single example of creating and retreiving notes as part of
action creation, that will likely be changed in a future change.
Note that this change doesn't include changes to the CLI to represent
notes outwardly, but the first example can be seen using the
--output-format=format option (shipyard get actions)
Change-Id: I2f87713eb74dae312912ff4c36e6ae30a569ea38
This patch modifies API behavior for
GET /v1.0/actions/{action_id}/steps/{step_id}/logs
such way:
- it returns the same status code as Airflow HTTP request returned
if Airflow responds with a status code of 400 or greater,
- it returns 500 error status code if an exception happens during
Airflow HTTP request (200 was before).
Warning: this change breaks API backward compatibility, now a client
could get 4xx or 5xx codes proxied from Airflow.
Change-Id: Ic5dceb3abc34415d21b4d8d4e71b4e5661a7363d
This commit introduces an action, `test_site`, that invokes Helm
tests for all deployed releases using the
`ArmadaTestReleasesOperator` introduced in [1]. This action supports
the ability to invoke Helm tests for a specific release using the
`release` parameter and cleanup resources if the `cleanup` parameter
is set to `true`.
[1] https://review.openstack.org/#/c/603236/
Depends-On: https://review.openstack.org/#/c/603236/
Change-Id: Ib5f38fe4b8a6516ee2afae62774ec84f1d2eb1ad
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes
Change-Id: I4b8cc6ddf491e35d600a83f5f82d7717108e31dd