This PS replaces old airskiff integration gate
with new kubeadm based airskiff integration gate.
The main goal of this gate is to test new Shipyard
and Airflow images and chart together with other
Airship components.
Change-Id: I7a2df36a89f92f720eabbc5783b935aeeebb3afc
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1
Change-Id: I7bbcbc8f9e6334100f47702f4546f0963d11b2ca
Upgrading htk to version 0.2.55, which deprecates the ingress class
annotation (kubernetes.io/ingress.class) with .spec.ingressClassName
https://review.opendev.org/c/openstack/openstack-helm-infra/+/891720
Change-Id: I392b9ebaebc696ae07a97b0a5d34c1a433b442ae
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
This PS updates python modules and code to match Airflow 2.6.2 as well
as deploys new Airflow:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was performed based on
airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9
- shipyard chart was corrected to match new airflow cli, configuration
items and their default values
- added new celery configuration items and their values
- updated airflow runtime logging config
- disabled deprecation and future python warnings in airflow images
- added celery to the list of airflow providers
- adjusted airflow runtime scripts to match new cli
- shipyard SQL queries to airflow DB were adjusted to match new SQL
schema of the db
- shipyard_airflow and shipyard_client unit tests were updated to match
new DB structure and new cli
- airflow db sync job is using db upgrade command
- helm version uplifted to v3.12.2
Change-Id: Ife88e53ce0dd8dc77bf267de1f5e6b8361ca76fd
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.
Change-Id: I4c4c7186e67c771f8249b2017b2507aac17760b7
- upgraded Airflow to 1.10.15 -
https://airflow.apache.org/docs/apache-airflow/1.10.15/changelog.html
- disabled xenial, bionic and opensuse images build gates
- added focal image build gate
- added focal zuul build node
- adjusted Makefile for focal
- added bindep.txt to utilize bindep zuul base role for zuul build node
pre-setup
- added focal Dockerfile
- implemented freeze requirements.txt approach like in other Airship
projects
- removed specific requirements.txt for airflow in favor of using
requirements-frozen.txt from shipyard_airflow project when building
airflow docker image
- fixed docker image publishing to Quay
- replaces deprecated LOG.warn with new LOG.warning call
- replaced deprecated body attribute in responce wiht responce.text
attribute
- update of falcon module deprecated .API call - replaced wiht
falcon.App call
- deprecated routing.create_http_method_map method replaced with
routing.map_http_methods
- re-formatted code tabulations based on yapf recommendations
- replaced deprecated protocol attribute in Pytest create_environ() with
http_version attribute
- replaced deprecated app attribute in Pytest create_environ() with
root_path attribute
- fixed airflow CLI commands to match 1.10.15 version
- updated zuul gates to work on focal nodes and added focal specific
node setup items by adding appriate ansible tasks and roles
- uplifted Helm to 3.9.4
- uplifted stable HTK commit id
- updated tox.in to with with tox v4
- uplifted dependences references to other Airship projects
- common python dependences were syncronized with other Airship
projects(Promenade, Deckhand, Armada, Drydock)
- fixed airskiff deployment gate
- fixed genconfig* profiles in shipyard-airflow tox.ini responsible for
maintanance of policy.yaml.sample and shipyard.conf.sample
Change-Id: I0c85187dc9bacf0849382563dd5ff7e9b2814c59
Update Zuul config and tox settings to not run python3.5 jobs, and to
make the xenial image jobs non-voting.
Change-Id: Icc9f71e60e6b48254917ea1d41ce9bc823cce512
* Install older version of pip<21.0 for ubuntu_xenial images
* Install setuptools via pip for ubuntu_xenial images
* Pin typing-extensions to 3.7.2 and apache-airflow to 1.10.5
* Move promenade Dependencies under UCP components in requirements.txt
* With apache-airflow=1.10.5, strip ANSI escape sequences
in test_deployment_group_manager.py
* Update tox.ini to support apache-airflow=1.10.5
* airskiff gate fixes
- Pin treasuremap to v1.9 branch
- Pin openstack-helm-infra to master
- Remove openstack client setup as it's not used
Change-Id: Iee4ce59fdceacb165120a69d11c44e6e47feaea8
Updated to pip to pip3 to address zuul gate issue.
Set the ensure_global_symlinks to true for zuul-jobs to set
symlink for tox path."
Change-Id: I1b0634ef18328bccc4d6929072f53db779d70ef1
Added support to buid shipyard and airflow images using either a xenial
or Ubuntu bionic base image.
The default base image is set to bionic.
Change-Id: I6ad4d42dede081586b3ccea87a42e250979ac106
This removes several of the the required-projects from the Airskiff
gate as the scripts for the gate can already clone down the necessary
repositories and the required-projects can overwrite any pins that may
have been included in the Airskiff scripts.
Change-Id: Ia70169bc89b493562e4a0db9d703a46fb2d2f87d
Adding DISTRO parameter for makefile to invoke distribution specific
Dockerfile and build image accordingly.
Also qualifying image job name to distinguish ubuntu vs opensuse
check jobs and passing distro specific variable.
Similar logic for pushing distro specific images.
Added new document section for multiple distro support.
Change-Id: I215f8a107cab8770181c0472e0cc29053b2b978e
This commit introduces a non-voting job to lint Helm charts against the
latest version of Helm toolkit from OpenStack-Helm Infra. This job
should serve as an indicator of when it's safe to advance the version of
Helm toolkit used by Airship.
Additionally, this commit modifies all Helm chart lint jobs to run on
each commit, regardless of the files modified by a change. This should
not introduce a noticeable difference in CI runtime, as these jobs
execute quicker than the tox jobs.
Change-Id: Ife3b285a2f9a160df9724e9dcb230243d06be3ab
This commit introduces a chart build gate that triggers when changes are
made to charts in the charts/ directory.
Change-Id: Iec1c6da785d26e2e7273b37fd1028f3bf47ec199
Currently, the airship-shipyard-ubuntu job deploys K8s using outdated
playbooks. The job fails due to a change [0] that introduced an AppArmor
requirement to Calico. This change removes the failing job, as its
success does not depend on changes in Shipyard.
In the future, jobs requiring K8s should be deployed using the new
Minikube gate scripts from the OpenStack-Helm infra repository. They
will be added in future changes once they are required. Additionally, a
future change will introduce a Helm linting gate for charts hosted in
the Shipyard repository.
[0] https://review.openstack.org/614805
[1] https://git.openstack.org/cgit/openstack/openstack-helm-infra/tree/tools/deployment/common/005-deploy-k8s.sh
Change-Id: I72aaff9050eab844ef0856794b059de14e2a4a28
1. Removed osh-infra-upgrade-host playbook from image building jobs,
as non-relevant
2. Removed osh-infra-deploy-docker playbook from image building jobs,
as docker-ce is deployed in run-image playbook
3. Removed osh-infra-build, osh-infra-deploy-k8s and security-bandit
playbooks - they are not referenced anywhere; bandit has already
been merged with pep8 job
Change-Id: I27f0fd8f346d0b52c55870b6005be525a1fec671
This patch set uses zuul templates for python35 and python36
because:
1) python35 template is maintained by project-config; don't need
to maintain separately in SY
2) python36 is now what SY will be building off in the Dockerfile
so it is better to also test this actively in CICD
Change-Id: I98230b51db69f1054d11428f48bd09dc2547f8b6
This change adds the global zuul pep8 tox job, which runs both
bandit and pep8 using tox. This also removes the two other airship
specific lint-pep8 and bandit zuul jobs since they are both covered
by the default openstack global one.
Change-Id: Iebea5b872f78762d6f401b574d53965b2e1c090b
This patch set adds tox tests to .zuul.yaml
Prometheus scrapers are not running in the airship-shipyard
and hence removed to prevent airship-shipyard-ubuntu job
from failing.
Change-Id: Ib805946ea76664b7448e02b4c7e4adb40c4dbaf2
This PS adds the skeleton for a set of zuul checks and gates for
Airship, using the framework from OpenStack-Helm.
Change-Id: I620f75dd0d450168f5781a870ef8f29d4fd3a0cb
Signed-off-by: Pete Birley <pete@port.direct>