Commit Graph

712 Commits

Author SHA1 Message Date
SPEARS, DUSTIN (ds443n) 7c1865b7db Update helm to v3.14.2
Change-Id: I6952261625d74a7a2205b34b3fe08b02afab6089
2024-03-15 14:04:42 -04:00
Sergiy Markin 03a55bbc31 Kubeadm based Airskiff gate
This PS replaces old airskiff integration gate
with new kubeadm based airskiff integration gate.
The main goal of this gate is to test new Armada
image and chart together with other Airship
components.

Change-Id: I2351b55fa136453a0a6a27a2875208c5e18e579e
2024-03-13 15:11:30 +00:00
Ruslan Aliev a3a1645169 Use designated image for chart test
Armada chart currently uses the same image for testing
as for deployment. The PS introduces flexible way to choose
the image for tests.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: If9bebd27cf710e91c49c8dcf8f247990bd5acfab
2024-03-08 15:21:45 -06:00
Sergiy Markin e85f0ccdeb Airflow stable 2.8.2
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1

Change-Id: I5c7fb6ae950ea0f906106fc2184d5c6eb7858f5a
2024-02-26 21:00:58 +00:00
Sergiy Markin b3b8d05fe7 Airflow stable 2.8.1
Change-Id: I46c97108d50304fd4e3417f6057d999c891b0661
2024-02-21 22:24:46 +00:00
Ruslan Aliev d9e2248172 Add configurable support of armada-operator for armada-api
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I76fb41062d152bf360a85d781c19ab5b204769b8
2024-02-12 11:09:18 -06:00
Ritchie, Frank (fr801x) 410d110c1a Update helm toolkit reference
Update helm toolkit reference

Update helm toolkit ref so that ingress will be created with a pathType
of Prefix.

https: //review.opendev.org/c/openstack/openstack-helm-infra/+/905757
Change-Id: I64c6ec08c6f454bd09c200174d8709afd6cda416
2024-01-23 17:27:24 -05:00
SPEARS, DUSTIN (ds443n) ea56249752 Update helm to v3.13.2
Change-Id: I6f235d1fba8d9c1c47a1dcf655e35b97e0382763
2023-11-15 09:27:52 -05:00
Anselme, Schubert (sa246v) 5df69442b0 Deprecating the Ingress Class Annotation
Upgrading htk to version 0.2.55, which deprecates the ingress class
annotation (kubernetes.io/ingress.class) with .spec.ingressClassName

https://review.opendev.org/c/openstack/openstack-helm-infra/+/891720

Change-Id: I304b3239f6d9f1951de03bd821c8ba4c0a5b9a60
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
2023-10-05 22:55:39 +00:00
Sergiy Markin 6595dd83ea Add docker image tag
This PS adds  airflow_2.6.2-${distro} tag to published image that was prevoulsly tagges as latest-${distro}.

Change-Id: I74d61110b235ab5618f2c4e45320cf51c90635ce
2023-08-25 20:02:56 +00:00
Sergiy Markin cdcdbd292d Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
  airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9

Change-Id: If6f57325339995216d2553c7a5ff56e7673b5acc
2023-08-19 00:37:37 +00:00
Sergiy Markin 25c3d621d6 Armada updates
This PS makes the following changes:

- makes sure the airskiff-deploy playbook is using 80Gb partition if
  available
- adds available security updates to docker images

Change-Id: I405743f9ae69194407bcc49a736b63e9993655c3
2023-08-18 21:53:29 +00:00
SPEARS, DUSTIN (ds443n) 4f43e2cc55 Update helm to 3.12.2
Also this PS updates airskiff deployment playbook based on the latest
treasuremap changes and makes sure the airskiff deployment uses freshly
built armada image.

Change-Id: I913f93d695e4068199e193a5839bf27e257a535e
2023-08-03 16:29:51 +00:00
Sergiy Markin 812546c875 Update armada image run test
This PS improves armada image run test by enriching it with config
generator. Also this test was removed from make images section of
Makefile. Build images zuul gate has make run_images task to test
the freshly built image now.

Change-Id: I87e089e5d268d248cda1dad1cd417694d166fdfb
2023-05-24 20:14:32 +00:00
Sergiy Markin 275dc4506f Updates to armada
- update github.com ssh host key
- fix bionic image upload to Quay.io
- fix armada_image_run.sh script
- fix armada config presense

Change-Id: I2af589baabb9e76e2e1a29ff266d6c3b7ae412e3
2023-05-22 21:37:16 +00:00
Sergiy Markin 966742ec74 Restored ubuntu_bionic image build
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integratinal
tests.

Also the latest osh-infra commit was used with kubernetes 1.27.1 or
integration tests.

Make images section of Makefile was enriched with run_armada step in
order to quicktest the freshly built images - this is vital for focal
and bionic images differences.

Change-Id: Ida43eebd4aa51fff6261ac9cf5f8259359bc99a9
2023-05-19 13:52:30 +00:00
Zuul fa0af775d5 Merge "Armada timeout adjustment" 2023-05-16 19:27:54 +00:00
Yuvraj Singh (ys542d) 3c3a20addb 111596: [CPDEFECT-4365] helm binary to v3.11.1
Change-Id: I078e995d5f1ef794b106e196d37f49fce519fcfd
2023-05-15 11:05:11 -07:00
Sergiy Markin 4be7fa44a7 Armada timeout adjustment
This PS set read/connection timeout to None for http requests.

Change-Id: Ic874485bb82a9c386224f5adc530e366b01ae24e
2023-05-11 21:15:22 +00:00
Sergiy Markin b778f9faf8 Update airskiff deployment gate
This PS updates armada to use new reliable airskiff site manifests for
testing armada.

Change-Id: I5c07f19755731b6df678aad0c066cfe588c79a80
2023-05-09 13:50:15 +00:00
Sergiy Markin 09f207f257 Re-add missed paste module
This PS adds Paste module that was lost during py38/focal upgrade. Also airskiff gate has been altered to use focal image of armada as bionic to cover ubuntu releases change. Airskiff in treasuremap is still pointed to bionic while here we want to test focal in airskiff.

Change-Id: I3cb1eb6232a332707a1440d33d8908ae83603487
2023-05-03 14:17:40 +00:00
Sergiy Markin a8d9b7214c Armada updates
This PS delivers the following updates:
- fixed sample config and policy files generation process

Change-Id: I6393abc6aac0d79539820dc5768a7094147190a6
2023-04-28 18:56:45 +00:00
Sergiy Markin a6dcf17d17 Removing egg-info folder
Change-Id: I532ccd9f281e975c050aeab6d9714efcb232ff5d
2023-04-26 22:17:11 +00:00
Sergiy Markin 21bdc838d1 Sync requirements with shipyard
This PS adjusts list of dependences needed to get syncronized with Shipyard project.

Change-Id: I65d9a3e621fdeb2579d15de3bb5da1049f4c8c39
2023-04-24 23:18:59 +00:00
Sergiy Markin 386a686e69 [focal] Python modules sync with Airship project
- armada-airskiff-deploy is voting gate again
- fixed falcon.API deprecation - -> falcon.App
- fixed collections.abc.defaultdict not found error
- fixed tox4 requirements
- implemented requirements-frozen.txt approach to make allike as other
  Airship projects
- uplifted docker version in the image building and publishing gate

Change-Id: I337ec07cd6d082acabd9ad65dd9eefb728a43b12
2023-04-21 23:49:14 +00:00
SPEARS, DUSTIN (ds443n) 0e9d828fe6 Update armada to focal
Update kubernetes client to v26.1.0
Updating armada to focal base image
Remove xenial and opensuse dockerfiles
Update tox python from py35 to py38
Add apparmor for docker build
Uplift HTK chart version 0.2.52
Bumping up some python dependencies to get in sync with shipyard
Added clear-firewall role for airskiff-deploy playbook

Change-Id: If06a3f60466702d05a21c24a7cb8041bed41507a
2023-03-20 11:30:37 -04:00
SPEARS, DUSTIN (ds443n) 099de8aaf4 Bump k8s client to v25.3.0
Bumping k8s client to v25.3.0
Cronjob batch v1beta1 no longer available in k8s 1.25
Update tox.ini file to be compatible with v4

Change-Id: Iac79c52c97c9ef1223ae8d502da1572ef8d068fa
2023-01-18 11:25:05 -05:00
Ruslan Aliev 379c88d619 Bump helm to v3.9.4 and k8s client to 24.2.0
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I55d7b34e7525fe6359157d0fa042332a91ac9c95
2022-09-19 11:21:50 -05:00
Ruslan Aliev ef0ac2f252 Bump helm version from 3.6.3 to 3.8.2
Also bump k8s dependency to v1.23.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I188788a41419184cc3dd9172f54ab3a290cabe83
2022-06-30 22:19:56 -05:00
Phil Sphicas 416eff4e52 Armada client: Enable TCP keepalives
This change enables TCP keepalives in the transport adapter used by the
Armada client.

Change-Id: I44d9fe0f29dfb1c00250a76452f4db610ea377ad
2022-03-14 13:40:23 -07:00
Phil Sphicas c5d39f27ca Create lock CRD as apiextensions.k8s.io/v1 object
Kubernetes v1.22 stopped serving the apiextensions.k8s.io/v1beta1 API
version of CustomResourceDefinition.

This change ensures that the locks.armada.process CRD is created using
the apiextensions.k8s.io/v1 API.

The kubernetes client package is also updated to take advantage of the
dynamic client.

Change-Id: Icd518ab5cbb78e8b15f63d19c51b5f5b9a67e995
2022-03-09 16:36:40 -08:00
francisy 790af88d61 Remove diff
Getting rid of logs that will show diff in results

Change-Id: I8994506f517928c1a945b794e72b2b7cbb960ab7
2021-12-09 10:47:50 -05:00
francisy 82a138f11b Remove Log Info
Passwords are shown in logs that need to be removed

Change-Id: I0be2d53c9e613ecb4b4f62c5164266fbaa596944
2021-12-08 13:32:17 -05:00
Phil Sphicas 7c432fdcb4 Update HTK stable commit (Ingress)
Update helm-toolkit to include updated Ingress templates that use
current API versions (i.e. networking.k8s.io/v1)

https://review.opendev.org/c/openstack/openstack-helm-infra/+/813115

Change-Id: I3cb054714f17d462550c89d15e3eb0ec2011177c
2021-10-18 11:49:58 -07:00
Sean Eagan 026a00a88e Attempt to fix Armada self-upgrade race condition
Change-Id: I1b84abb02fedfc788739de162d1e4938a008bc7d
2021-10-09 14:33:01 -05:00
Sean Eagan b5b519f66a Fix list_release_ids()
Change-Id: I8ad7c95a11e74858803d779e99afda6670a3f198
2021-10-07 22:56:11 -05:00
Sean Eagan 862c289e4b Fix dependency symlink idempotency
Previously, existing dependency symlinks were causing
FileExistsErrors. This simplifies the logic to ensure
they do get deleted before being recreated. It also
moves to erroring if a chart's `charts` directory is
not a directory, since this is something a chart author
ought to fix, since it needs to be a directory in order
to add dependency sub-directories to it.

Change-Id: I7cc1d50f2bf09602866ce2cf13a4d6c8f6e118ce
2021-10-05 12:59:48 -05:00
Sean Eagan 34ebac542b Helm 3: Fix Job labels
See the dependency below for details.

Depends-On: https://review.opendev.org/c/openstack/openstack-helm-infra/+/811826
Change-Id: Id1638ff32d134a598e9c1a73a4ea2532ca9785b8
2021-10-05 02:41:39 +00:00
Sean Eagan a5730f8db8 Remove Tiller
For now we leave the tiller status enpdpoint, until
Shipyard has had a release to stop depending on it [0].

[0]: https://review.opendev.org/c/airship/shipyard/+/802718

Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: If8a02d7118f6840fdbbe088b4086aee9a18ababb
2021-10-05 02:41:32 +00:00
Sean Eagan 2efb96eea0 charts: move to helm 3 preferred apis
- `helm.sh/hook: test-success` > `helm.sh/hook: test`

Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I5471b2825d24da5584d40902430fdf99ea54e529
2021-10-05 02:41:26 +00:00
Sean Eagan f3842f9fe1 Use helm 3 in chart build
`helm serve` is removed in helm 3 so this moves
to using local `file://` dependencies [0] instead.

[0]: https://helm.sh/docs/chart_best_practices/dependencies/#repository-urls

Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I2282e231591a89073e65d9db3f59e1baea707380
2021-10-05 02:41:21 +00:00
Sean Eagan 68747d0815 Use helm 3 CLI as backend
Helm 3 breaking changes (likely non-exhaustive):

- crd-install hook removed and replaced with crds directory in
  chart where all CRDs defined in it will be installed before
  any rendering of the chart
- test-failure hook annotation value removed, and test-success
  deprecated. Use test instead
- `--force` no longer handles recreating resources which
  cannot be updated due to e.g. immutability [0]
- `--recreate-pods` removed, use declarative approach instead [1]

[0]: https://github.com/helm/helm/issues/7082
[1]: https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments

Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I20ff40ba55197de3d37e5fd647e7d2524a53248f
2021-10-04 21:40:26 -05:00
Sean Eagan 4260e5335c Add Helm 3 CLI to images
Change-Id: Ibd0b585c458b55008f014f245a2781fad8d055f3
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
2021-10-04 18:20:24 -05:00
Sean Eagan 8c5e5c7d24 Remove unused commands
This removes release rollback/delete functionality. This functionality
was likely not being used and thus was likely not working.

This primary driver for this change is to ease introduction of Helm 3
support. Particularly to avoid having to make API changes related to
the namespacing of helm releases in Helm 3.

This also removes the swagger api documentation as it was not
maintained.

Change-Id: I7edb1c449d43690c87e5bb24726a9fcaf428c00b
2021-09-30 17:22:16 -05:00
Sean Eagan 58c0df5201 Extract pre-update actions out of tiller handler
This is a pre-requisite for Helm 3 integration, so that these
actions run regardless of whether we are going through the
tiller handler.

Change-Id: I97d7bcc823d11b527fcdaa7967fcab62af1c8161
2021-09-30 17:22:16 -05:00
Sean Eagan 5f1ffbbbbe Revert "Airship 2 support features"
This reverts commit c75898cd6a.

Airship 2 ended up using the Flux helm-controller instead:
https://github.com/fluxcd/helm-controller

So this is no longer needed. Removing it to get rid of tech
debt to ease introduction of Helm 3 support.

This retains the part of the commit which extracts the
chart download logic to its own handler as this is still useful.

Change-Id: Icb468be2d4916620fd78df250fd038ab58840182
2021-09-30 17:22:16 -05:00
Sean Eagan b37417253f Gate fixes
- Fix helm CLI download URL, the previous URL is no longer available,
  and is superceded by get.helm.sh
- Fix jsonschema dependency spec, 4.x alpha versions were causing the
  unit tests to fail for python 3.5
- Remove oslo.db dependency as it was not used and one of its recursive
  dependencies (tempita) was failing during setup

Change-Id: I03e8a738bb8aad620903119638210077d211770c
2021-09-30 17:21:33 -05:00
DeJaeger, Darren (dd118r) e9a2309e0a Changes for new version of falcon
Moving to falcon 3.0.0+ brings in some changes to the response
object. One of those is the deprecation of the "body" response
field. This PS attempts to get ahead and make the necessary
changes to no longer use deprecated fields.

Change-Id: Iac5d8cd2c658c337dfe7937db8124f3107d77e91
2021-06-24 15:02:42 +00:00
DeJaeger, Darren (dd118r) 9aadc14777 Armada improved logging, uplift dependency
This PS:

1) Looks to improve specific logging in Armada, so that
it's easier to debug deployment related issues
2) Uplifts the k8s Python dependency to 12.0.0
3) Enforces 'watch' timeouts more strictly, as the call to
the Kubernetes Python watch function seemed unreliable.
4) Adds a field selector to the 'watch' stream to look for
the DELETE action to have been completed on the specific
pod/job/cronjob, rather than looking across the whole
namespace or via labels. This will narrow what the watch
is looking at, making the logs less busy.

Change-Id: I1952b0db32fb0b56ffffcddeae0532beb5a27b67
2021-06-24 10:53:06 -04:00
anthony.bellino 973338590f Gate fix for py36
Pin cmd2 1.5.0

Change-Id: I1e20818f47b933dfc0dc78785fd3664d774c7263
2021-06-22 16:01:05 +00:00