This PS updates python modules and code to match Airflow 2.6.2 as well
as deploys new Airflow:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was performed based on
airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9
- shipyard chart was corrected to match new airflow cli, configuration
items and their default values
- added new celery configuration items and their values
- updated airflow runtime logging config
- disabled deprecation and future python warnings in airflow images
- added celery to the list of airflow providers
- adjusted airflow runtime scripts to match new cli
- shipyard SQL queries to airflow DB were adjusted to match new SQL
schema of the db
- shipyard_airflow and shipyard_client unit tests were updated to match
new DB structure and new cli
- airflow db sync job is using db upgrade command
- helm version uplifted to v3.12.2
Change-Id: Ife88e53ce0dd8dc77bf267de1f5e6b8361ca76fd
- Overrode uwsgi default configs to improve stability and performance.
- Increased mas number of worker processes to increase capacity and
performance.
- Enabled uwsgi cheaper subsystem to scale worker processes dynamically.
- Uplifted uwsgi to the latest release to bring bug fixes and
improvements since 2018.
Upgraded uwsgi to bring in bug fixes since 2018.
For background information for this change please see:
https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html
Change-Id: If067e9786e9dbbd39ef832dea6f51aa5523af4d7
Remove OSH Authors copyright
The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.
This change removes all references to this copyright by the
non-existent group and any blank lines underneath.
Change-Id: Ic8de1678a754ba466dbd8d12c4f078151a78a091
Update apiversion for ClusterRole, ClusterRoleBinding to rbac.authorization.k8s.io/v1
Update apiversion for deployment to apps/v1
Update apiversion for statefulset to apps/v1
Add selector match labels to deployment
This patch is similar to https://review.opendev.org/#/c/638276/
These changes are required to install shipyard helm chart on k8s 1.16.0
Change-Id: I7ac6fc060fbd6a5feea747ebbe8121c5a2eb4b6f
This PS adds pod anti-affinity to airflow/shipyard pods,
so that the scheduler can constrain pods against labels on other pods
running on the node. The default soft rule is in place so that if
the scheduler can’t satisfy the the requirement, the pod will still
be scheduled, and is overridable.
Change-Id: I67d0792a1f624044f8975c9540ab691f4e638b3f
- deployment-shipyard
This updates the shipyard chart to include the pod
security context on the pod template.
This also adds the container security context to set
allowPrivilegeEscalation to false and readOnlyRootFilesystem to true
Change-Id: Idb1b848847eaec2b6e24389c063b7ece2973c4dc
Moves the airflow web server container from its own pod into the
Shipyard pod. This removes exposed network surface area from the
Shipyard suite of software. Shipyard, after this change accesses the
Airflow API using localhost in the same k8s pod.
Change-Id: Ied4bd415a8d78c393b7256ead27a6a2176f4a2d6
This commit adds the Werkzeug ProfilerMiddleware to Shipyard API
requests. This option can be enabled using the
`conf.shipyard.base.profiler` option and should not be used in
production.
Change-Id: I293840d78baf670478047faad87fdcfe2f8af70e
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. This can be used, for example, to force an
artificial manifest change in CICD scenarios, for upgradability
testing purposes.
Change-Id: I75fc7516e0d32e7e0df0fecf1f9bec0e234adfcc
Provides different fields in the values.yaml to use for the node
selector keys/values used in deployments vs jobs.
Change-Id: I12d7c6257aea0ac00cd77cd3f6331a2b7380b589
Updated configurations to point to openstack-helm-infra
for reference to helm-toolkit as helm-toolkit has been
removed from the openstack-helm repo [0]
Also aligned with changes to the keystone user set up in
OSH using Helm ToolKit so as to get pass Helm Lint.
Updated Makefile targets to install helm dynamically
[0] https://review.openstack.org/#/c/558065/
Change-Id: I0a0813516f9ad176ff005b4693e6b933013a99fd
As part of ongoing effort to update the "application" and
"component" labels for the UCP components, there is a need
to align with the convention. We will update the label for
the shipyard API pod in this case.
Also updated helm_tk.sh to point to openstack-helm-infra for
reference to helm-toolkit as helm-toolkit has been removed
from the openstack-helm repo [0]
[0] https://review.openstack.org/#/c/558065/
Change-Id: I0b2acda47d87f8dda35fbf054e1c8d906b495061
All UCP Operators will inherit from the UcpBaseOperator [0]
This patch set will align the rest of the Operators, i.e. Armada,
Deckhand and Promenade Operators with the UcpBaseOperator
It also updates the name of the shipyard container to be
'shipyard-api' instead of 'shipyard'
[0] https://review.gerrithub.io/#/c/407736/
Change-Id: I516590c492e9bb5554161119dade278d74197374
This patch set does the following to enhance health/status checks
on the shipyard-api pod:
1) Add Liveness Probe
2) Update Readiness Probe
Change-Id: Ifab63a8724f29fb38124f43d475bb022807a4cce
- Support configured Postgres admin password
- Use secrets for database job environment setup
This patch set also updates a bunch of banners
Change-Id: I238cfd123b5aad31c9cb93864cff7641f719f3df
Updates to entry.sh to allow for multi-workers/threads
Updates Shipyard chart to allow parametrs to be configurable
Change-Id: I6ad9d198ac4df4c7c85dfcf5c04afd3c7966f0f0
This patch set removes the shipyard config, policy and paste.ini
template from the existing Shipyard Helm Chart. This is done to
align with the current approach in OpenStack Helm.
1) Remove shipyard config template
2) Remove shipyard policy.yaml template
3) Remove shipyard api-paste.ini template
4) Update related template files
There has also been a recent change to the Helm Toolkit which will
break the current implementation of the Shipyard Chart
The changes in Helm Toolkit were made to the 'images' definition
in values.yaml to facilitate adding the option to prefix image
name etc
This P.S. will also update the Shipyard Chart to align with the
recent changes in Helm Toolkit
Change-Id: Ie79fd9da2c9a577027dd0dddbcca6b7f7b3b4f6f
Sergiy Markin