This PS replaces old airskiff integration gate
with new kubeadm based airskiff integration gate.
The main goal of this gate is to test new deckhand
image and chart together with other Airship
components.
Change-Id: I4221b7be64e9a7e07964fa817d42c21a3f172db0
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1
Change-Id: If7191c8291e31488a8a5f26107981616e14ea12c
Upgrading htk to version 0.2.55, which deprecates the ingress class
annotation (kubernetes.io/ingress.class) with .spec.ingressClassName
https://review.opendev.org/c/openstack/openstack-helm-infra/+/891720
Change-Id: I573a926ab6fb07f10c0c4d9020746ba80e6d8dbd
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
This PS updates python modules and code to match Airflow 2.6.2:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfrmed based on
airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9
- postgresql image updated to 14.8
Change-Id: I65a1b86473ee3e988aae353b59fb5473d75851f9
This PS makes the following changes:
- uses deploy-k8s.sh from treasuremap
- makes sure the airskiff-deploy playbook is using 80Gb partition if
available
- adds available security updates to docker images
Change-Id: I0f330cb15ec32b12703f0bc6620b3f3c797a25bb
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.
Also the latest osh-infra commit was used with kubernetes 1.27.1 for
integration tests.
Change-Id: Ia2c951f27b96774b553e7c0c7c6809172312f753
- adjusted .gitignore to keep fresh egg-info and omit build artifacts
- fresh egg-info data is needed for promenade that depends on Deckhand
- restored deckhand-functional-uwsgi-py38 gate
- restored deckhand-integration-uwsgi-py38 gate
- made deckhand-airskiff-deployment gate voting ( treasuremap project
has been updated)
- removed bionic gates
- updated focal dockerfile
- added more binary deps into bindep.txt
- updated deckhand chart values to latest images - focal and wallaby
- fixed python code to compy with CVE's found by fresh version of bandit
- implemented pip freeze approach
- added tox -e freeze profile to manage it
- requirements-frozen.txt is now main file with requirements
- requirements-direct.txt is the file to control deps
- updated setup.cfg to adjust to newer version of setuptools
- fixed airskiff-deploy gate
- fixed docker-image-build playbook to restore Quay repo image publish
- updated other playbooks to include roles from zuul/base-jobs in order
to setup build hosts properly
- removed workaround with hardcoded dns resolver ip 10.96.0.10 as it
became obsolette due to recent fix in openstack-helm-infra
- adjusted tools/whitespace-linter.sh script
- tox.ini has been brought to compliance with tox4 requirements
- replaced str() calls with six.text_type() according to D325 Deckhand specific
commandment from Hacking.rst
- locked python-barbicanclient version with 5.2.0 because of breaking
changes in the upper versions
Change-Id: I1cd3c97e83569c4db7e958b3400bdd4b7ea5e668
update dockerfile for python deckhand install
add deckhand version to chart 1.0
add chart version 0.2.0
update all packages to latest in requirements.txt
update zuul jobs for focal and python 3.8
remove zuul job functional-uwsgi-py38 in favor of functional-docker-py38
update tox config
typecast to string in re.sub() function
add stestr to test-requirements.txt
add SQLAlchemy jsonpickle sphinx-rtd-theme stestr to requirements.txt
deprecated function: BarbicanException -> BarbicanClientException
fix mock import using unittest
fix import collections to collections.abc
fix for collections modules for older than python 3.10 versions.
deprecated function: json -> to_json
deprecated function: werkzeug.contrib.profiler ->
werkzeug.middleware.profiler
deprecated function: falcon.AIP -> falcon.App
deprecation warning: switch from resp.body to resp.text
rename fixtures to dh_fixtures because there is an imported module
fixtures
switch from stream.read to bounded_stream.read
deprecated function: falcon process_response needed additional parameter
deprecated function: falcon default_exception_handler changed parameter
order
move from MagicMock object to falcon test generated object to fix
incompatability with upgraded Falcon module.
Adjust gabbi tests to fix incompatability with upgraded DeepDiff module
update Makefile to execute ubuntu_focal
update HTK (helmtoolkit)
unpin barbican to pass integration tests
Use helm 3 in chart build.
`helm serve` is removed in helm 3 so this moves
to using local `file://` dependencies [0] instead.
Change-Id: I180416f480edea1b8968d80c993b3e1fcc95c08d
The following job is failing, seemingly due to changes in openstack-helm
and openstack-helm-infra:
deckhand-integration-docker-py36-ubuntu_bionic
For now, disable the job.
Change-Id: I55e441079a470d6f346d70857655f0a77ddc67c3
Update tox configuration and Zuul config to use Python 3.6, and make the
xenial and opensuse gates non-voting.
Change-Id: Iab98ad6cf269d57acf0598ae431b2f661673b234
- With Ubuntu bionic based deckhand docker image, uswsi crashes with
segmentation fault when it tries to load psycopg2 library, causing
the deckhand service become unavailable.
The root cause of this problem seems to be that uwsgi and psycopg2
binary wheels are built with different ssl libraries.
To address this issue, upgrading psycopg2 to the latest release to
the latest binary release.
- For opensuse 15.1 image build, python3-six was updated to address
package not found issue.
- Updated gating ansbile playbooks to address non-voting gate failures.
Change-Id: I7be920e16e6114eb2bdbc052a6761f29008baf81
- Added ability to build deckhand docker image, using either the
Ubuntu xenial or the Ubuntu bionic base image.
- Made the bionic base image the default base image for deckhand
docker image build.
Change-Id: I26657de34a233ee3223a7f93fc667e734ac9140b
Fix problems with integration test zuul jobs which were made non-voting
by [0] and restore them to voting status. These jobs exercise some
secret-management functions that test integration between Deckhand and
Barbican that doesn't happen in the Airskiff jobs, so they still have
some test value.
- Some integration test scripts were pointing to old versions of
openstack-helm scripts that are no longer maintained. This PS
updates those scripts to point to current OSH scripts.
- Uplift openstack-helm-infra pin in helm_tk.sh to a recent
version
- Add build-images to uwsgi test jobs to ensure airship_deckhand_path
is set
[0]: https://review.opendev.org/660738
Change-Id: If83dead126307a98b44ad5d8f367a972c48a30d0
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.
See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html
Change-Id: I13689a385bf237681bc96f3cc62fc17864f12659
This removes several of the the required-projects from the Airskiff
gate as the scripts for the gate can already clone down the necessary
repositories and the required-projects can overwrite any pins that may
have been included in the Airskiff scripts.
This requires a new nodeset as well, because the Airskiff gate does not
presently work with ubuntu-bionic
Change-Id: Idff16a450af1e3f9fe0ee49496fd7d72df69b595
Fixing incorrect secret name used in previous review to address
image publish feature
https://review.opendev.org/#/c/638301/
Change-Id: I9eadad315db271169303265ce82a9f08008116e1
Adding DISTRO parameter for makefile to invoke distribution specific
Dockerfile and build image accordingly.
Updated logic for existing jobs to have distro specific logic
for building and publishing images.
Added multiple distro specific document in operator section.
Change-Id: I415ab28b06ea17b21e76d28ccb3e284041c8072d
The deckhand gates run osh-infra scripts that now have a dependency
on Ubuntu Bionic. This updates the nodepool to match.
Change-Id: I433264f7aec7cbe42b6b0a57562c2959d6c82a33
Deckhand integration tests are currently broken, and this makes them
non-voting. The test harness is superflous now that the (also
non-voting) airskiff job is in place, and it should be considered as
follow-on work whether we should remove the integration tests in favor
of the airskiff integration job, or fix/maintain the current integration
test harness and/or tests themselves.
Change-Id: Ib3ce88693d80de6c9c79c278eed2e312fe3a0b6f
On April 19, OpenStack completed the migration from openstack.org to
opendev.org [0]. During this migration, Airship projects moved from the
"openstack" namespace to the "airship" namespace. This commit makes
preliminary updates to gate jobs to account for the migration and fixes
a broken integration gate.
[0] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005011.html
Change-Id: I9257b0adab21bcc7596a5898ac9160bc6d63c225
This commit introduces a non-voting job to lint Helm charts against the
latest version of Helm toolkit from OpenStack-Helm Infra and a gate for
a pinned version of Helm toolkit.
Change-Id: Ia98209745641db011469d54220777110b2f6ae3a
With this patch, if commit being merged modifies both code and non-code
files, we will first build and push new image, and then will run
tagging job (which will do no harm).
Change-Id: Id9e9e705f9f5f69b969003c444895e1e0676c0dd
Added container image tagging on quay.io with commit id on every merge,
when certain files but not the code are changed (docs, charts, etc.).
Previously we were building updated container images, adding tags and
uploading images onto quay.io only when the code has been changed. The
main reason for not re-building images for every single merged commit
was a long waiting time in Zuul `post` queue for a node (VM)
assignment for the job to run on.
With this change we will have image tag for every merged commit,
including documentation commits, and those tags would be added almost
immediately (within ~1 min), as job runs directly on Zuul and does not
wait in Zuul queue for a node (VM) assignment.
Change-Id: I9692947b74a16de0eb3e8100d30cd4310069abfe
This is a mechanically generated patch to add a unit test job running
under Python 3.6 as part of the python3-first goal.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I4d08c67fb07cc4efeb48b7f81db9f5a01e444575
Story: #2002586
This PS simply uses airship-deckhand-single-node instead of the
openstack-helm variety for the nodeset in .zuul.yaml for
consistency.
Change-Id: I0a881c89752dd24a48e918a0ffb4637d4a5e6483
To cut down on Zuul nodes, this patchset combines the integration
docker job with the airship-deckhand-ubuntu job together, because
the former essentially subsumes the latter: integration docker
job sets up the host, deploys docker, deploys k8s, generates
the Deckhand chart using Helm, builds the Deckhand image using
Docker, deploys Deckhand, PostgreSQL and other Keystone/Barbican
dependencies using OSH, then finally runs integration tests
against the deployment.
This, then, validates everything that airship-deckhand-ubuntu
did deployment-wise.
Also, this patchset switches the 2 uwsgi jobs over to voting: false
as the Docker jobs are already voting.
Change-Id: I10b15f5a45a32413dc904ee32bdbbb4e9cdf98a2
This patchset adds integration tests job to .zuul.yaml so that
they can be run against Zuul per each commit.
The number of jobs that are executed in the gate pipeline
have been pruned because only *-docker jobs are of critical
importance. We can safely skip over *-uwsgi jobs in order
to save on Zuul resources.
Change-Id: I98ce49163b2db64cbdf18fe00b82c99fecd4139d
This patch set adds a trigger on post to trigger a RTD job to start
building the documentation post-merge.
Change-Id: Icd0ad1670b0d239e601a4456f84cc46e2f28aaf2
Moving the whitespace linting job as part of PEP8 job to reduce the need to
create a VM just to find whitespaces in none *.py files. Python files'
whitespace violation is covered as part of PEP8 job.
Change-Id: I3bc6047c9ab65e8f55f91a949a6c05ef4e38ea16
Signed-off-by: Tin Lam <tin@irrational.io>
This patchset either expands on existing entries in irrelevant-files
or adds irrelevant-files to various jobs to save on resources
in Zuul.
Change-Id: I97c58069dce21a368d4267f15cd656c8d858ecc3
This patchset adds py27/35 postgresql unit tests to .zuul.yaml;
they use the pre-defined openstack-tox-py27|35 template defined
in project-config as the parent.
Change-Id: I572764417d98b6ab902fe2329cfb3b396ac38966
This adds a uwsgi integration test gate to .zuul.yaml so that
deploying Deckhand via uwsgi (in a more standalone fashion,
sans containerization) works as intended.
Change-Id: I933f4781cd72e2df309efcb0515441db7ab96895
This adds a uwsgi functional test check to .zuul.yaml so that
deploying Deckhand via uwsgi (in a more standalone fashion,
sans containerization) works as intended.
Change-Id: I931ab4d11719daca7665d3a25b00e353c707237e