Commit Graph

473 Commits

Author SHA1 Message Date
Sergiy Markin b1ca8d08aa Airflow stable 2.8.2
This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1

Change-Id: I1ab801527dacb7c667caf1d459d71f9df522cef6
2024-02-26 21:23:10 +00:00
Sergiy Markin 5470976d4a Airflow stable 2.8.1
Change-Id: I5e78501487e0e1cd3ec1be657f96b80ce951637c
2024-02-23 19:57:08 +00:00
Ruslan Aliev 74aa971c9d Bump promenade version to support crd printer columns
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ie702a148753ef8eebe741fac10c234b0b33fa550
2024-02-12 13:53:54 -06:00
SPEARS, DUSTIN (ds443n) 6f266ef8a4 Bump promenade changes
upgrade etcd to v3.5.11

https: //review.opendev.org/c/airship/promenade/+/907761
Change-Id: I2bc304d7d19306bcb3d5f5a49ab7cd06052ea082
2024-02-09 14:44:03 -05:00
Ruslan Aliev 9fde367ddb Bump promenade dependency
* add full support of operator-based bootstrap

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I092b90a58525fbc78293ca5e7ea29c4341ca62c5
2024-01-16 10:35:49 -06:00
Ruslan Aliev 747f687979 Bump promenade version (armada apply cli operator support)
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I3a0cf166c4e3c9abca163a44efdda14c3e16b2bc
2024-01-05 12:53:53 -06:00
Ruslan Aliev 3130df03df Bump promenade version [0600 perm for kubelet.service]
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I7a7ef9cbf7d9a3927d730999e0967e54d091a4ba
2023-12-12 17:31:29 +00:00
Sergiy Markin b0f823aec7 Uplift shipyard
This PS uplifts shipyard dependency

Change-Id: I07fc6ee98ff3af66713eb8eed41ce3b7f295e4ca
2023-08-31 20:46:16 +00:00
Sergiy Markin eb96649b61 Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was performed based on
  airflow-2.6.2 constraints
- a new pegleg validation gate was created based on airskiff deploy
  pipeline from treasuremap v1.9

Change-Id: Icc96deb0b745eaab7a19d83b00aad909dcd28c25
2023-08-30 16:07:28 +00:00
Sergiy Markin fd5b16778a Pegleg updates
This PS increases docker image build timeout and restored bionic image
publishing.

Change-Id: I127eaadc0c69b86cf180f5663d618ff85d3a29fa
2023-08-25 23:15:19 +00:00
Ruslan Aliev 85da464cec Add gettext package to docker images
Allows to use envsubst utility within pegleg container.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I7733854253f3d4a6f9367678d93da9d4056e9535
2023-06-21 10:25:22 -05:00
Ruslan Aliev 23d30dfe2b Bump promenade dependency
Fixes an issue in up.sh script.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I2ee50f1316120e01e39d12caca961b82f0950f62
2023-06-09 18:12:01 +00:00
Sergiy Markin 0f2ab241f9 Restored ubuntu_bionic image build
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.

Change-Id: Id31d97ced8732d823937fb1f218e7ad8760d735c
2023-06-07 21:02:28 +00:00
Sergiy Markin 03722e844c Uplift Airship clients
This PS uplofts latest version of Airship components clients.

Change-Id: I0a7e578fdfbf84b898b1b7db22b2631fdb39f321
2023-05-16 20:48:41 +00:00
Sergiy Markin 3352c3f59d Uplift shipyard
This PS uplifts fresh version of shipyard with airflow 1.10.5

Change-Id: I509b4e7ec15a7c5cfc41399ca74c41eb69c3aedc
2023-05-08 22:31:13 +00:00
Sergiy Markin c052d40277 Pegleg focal upgrade
This PS delivers focal version of Pegleg image and has the following updates:
- removed release-notes-jobs-python3 gate job because of incompartibility with Sphinx from current requirements
- added focal gate node and switched gates to use it
- added bindep.txt file into project root
- added bindep role into gate jobs
- added ubuntu_focal dockerfile for building focal pegleg image
- switched tox profiles to py38
- uplifted references to shipyard_client, promenade and deckhand projects
- resolved required dependencies conflicts by weakening constraints in Pipfile
- updated tox profile update-requirements for generate requirements.txt and test-requirements.txt
- generated new Pipfile.lock, requirements.txt and test-requirements.txt from Pipfile
- switched tox profiles to use requirements.txt and test-requirements.txt instead of pipenv because of upstream zuul nodes Pypi mirrorring issue
- updated reference to seaworthy site certificates in treasuremap repo
- fixed unit tests issues caused by pytest/mock updates and new openssl version
- fixed focal docker image publishing issue
- added multiprocessing into coverage tests running process
- made unit and coveraget tests more verbosive

Change-Id: I5c4c519dc725cfb8c7b4e14756347c9336028aff
2023-05-02 15:05:45 +00:00
Ruslan Aliev 770cf4256e Uplift k8s, promenade and shipyard dependencies
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I185cfa259c9351f8e25f223f60d9a01b331f66b6
2022-09-29 16:19:14 -05:00
Mosher, Jaymes (jm616v) 0669f57e16 Uplift promenade for haproxy.cfg typo
Uplift promenade to include:
https://review.opendev.org/c/airship/promenade/+/855432

Small typo. The extra trailing newline disappeared when I added
the comment.

Also removing dependency on gitdb, as it conflicts with gitdb2
which seems to be required.

Change-Id: I8fb9413bf3bf46a68b88635c76b9192e1f9f8b21
2022-09-15 13:11:43 -06:00
Mosher, Jaymes (jm616v) e2ba17a049 Uplift promenade to include haproxy.cfg fix
Uplift promenade to include the following patchset:

Ensure haproxy.cfg ends with newline to support Haproxy
versions >=2.3
https://review.opendev.org/c/airship/promenade/+/854466

Change-Id: Ifecfd093220226ece45fdb62ec6cb48c6b8732d0
2022-08-29 10:31:24 -06:00
Ruslan Aliev 6833800ecc Uplift k8s, promenade and shipyard dependencies
* disable python 35, leave only 36
* switch to bionic nodeset only
* update requirements.txt, Pipfile and lock

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ia3c4df9d1f39f0f2cebf8ba0d89aebc5eec6f674
2022-07-20 18:40:42 -05:00
Phil Sphicas cbc87967eb Uplift Promenade (genesis scripts again)
Small adjustments to the Helm wrapper script:
https://review.opendev.org/c/airship/promenade/+/833640
https://review.opendev.org/c/airship/promenade/+/834509

Change-Id: I5835cdc2c2bfed481d184c99215a5a93904f5c28
2022-03-22 07:49:00 -07:00
Phil Sphicas 1a7e2da9d7 Update treasuremap ref for cert tests
Unit tests are failing for check_pki_certs against airsloop, since those
certs are expiring in less than 60 days.

This change updates the reference to a commit with regenerated certs.

Change-Id: Ib1ea40a37eef14c2df44b8446fdbcfcd0e345a95
2022-03-22 07:49:00 -07:00
Phil Sphicas ad740d7aa2 Uplift Promenade (genesis scripts)
Uplift Promenade to include the following update:

827385: Use same helm wrapper script for all nodes
https://review.opendev.org/c/airship/promenade/+/827385

Change-Id: I5dd7f8785e02ed53dbefc24246be3ec52fb7e775
2022-02-03 14:02:34 -08:00
Phil Sphicas e9280d3c85 Enable Deckhand substitution enhancement
Uplift Deckhand to include source substring extraction feature.
https://review.opendev.org/c/airship/deckhand/+/824657

Change-Id: I51b872189c38a3df9240ce33c5cc88eba40235fb
2022-01-18 14:15:28 -08:00
francisy 212dd0826e Pegleg Update
Changing ref from Promenade changes

Change-Id: I2fe5cdc6e2961d0dd905844c1c183d07e55187c5
2022-01-10 22:36:45 +00:00
Maximilian Weiss 9e8216aa5e Fix some outdated Zuul dependency errors
Shpinx incorrectly claims it wants docutils >=0.18 but that is an error
and older versions fail with that, as mentioned here:

https://github.com/sphinx-doc/sphinx/issues/9841

Additionally, the repo URL for OpenSUSE 15.3 python has changed.

Change-Id: I9bee6cf3ad7aaba80a44f2bd2f917b16c776c0d7
2022-01-10 22:18:38 +00:00
Phil Sphicas 772d3a47a6 Uplift Promenade (validators, pod templates)
Uplift the embedded version of Promenade to produce a genesis bundle
with updated validation script and pod templates.

Relevant changes:
* Remove log-test pod if validation succeeds [0]
* Update tolerations and priority classes [1]

Full list of changes in Promenade since the last uplift:
* e4d9d99 Update charts to use stable Kubernetes APIs
* e14854b Update HTK stable commit (Ingress)
* 0890626 Update tolerations and priority classes
* e43b6f0 Remove log-test pod if validation succeeds
* 2f823c6 Helm 3: Fix Job labels

[0] https://review.opendev.org/c/airship/promenade/+/814471
[1] https://review.opendev.org/c/airship/promenade/+/814486

Change-Id: I19d790aca9d3f8f23c07e88d5bdb314686fe5528
2021-10-20 10:25:51 -07:00
Phil Sphicas 2dc409f8cd Make xenial image jobs non-voting
Update Zuul config to make the xenial image jobs non-voting.

Change-Id: Id27cb2aa695d0d7535ac1b1501c06ac1d7a82680
2021-10-20 10:25:51 -07:00
Sean Eagan 4cd2ee6dab Uplift promenade
Also fix a python dependency failure [0].

[0]: https://review.opendev.org/c/skyline/skyline-apiserver/+/808195

Depends-On: https://review.opendev.org/c/airship/promenade/+/812039

Change-Id: I5aa508bce18de011956ef1b649c2ed17e376be68
2021-10-06 15:26:54 -05:00
Phil Sphicas b2b780a4c0 Uplift Promenade and Deckhand
Promenade uplift to include /etc/etcd and /var/log/kubernetes are
created on the host, and Deckhand uplift to use jsonschema 3.2.0.

https://review.opendev.org/c/airship/promenade/+/804915
https://review.opendev.org/c/airship/deckhand/+/807812

Change-Id: Ie01fd645357cfe0a21142c2a14d5a0b5f5ffea4f
2021-09-08 17:21:31 +00:00
Thirunavukkarasu Palani 590ac7e20a Uplift Promenade to the latest commit
Updates Pegleg to include the latest Promenade updates.

List of Pegleg changes since the previous uplift:
* 7692b36 Kubelet warning fix
* 183b977 Fix deprecated warning in Promenade controller-manager chart
* 1401664 Fix deprecated warning in Promenade apiserver chart
* 9da1262 Add configMap to proxy chart

Change-Id: I3f36c1575de4c748edc1c640ba9b66b59ca3de26
2021-08-05 09:21:48 -07:00
anthony.bellino 86c52e4c8c Gate fix for pegleg
Remove default pip.conf as some packages are
not availablei from the mirror and cause
the image build jobs to fail.

Change-Id: I125360952190a4dc5cfb98647cd94dcf506f9b71
2021-08-03 17:21:53 -07:00
anthony.bellino 5acd80abcf Fix opensuse_15 image
Leap 15.3 changes for opensuse image build

Change-Id: I24952bf66f579a5b96ecff7b55fbc39877c93f7c
2021-05-14 13:42:02 +00:00
Phil Sphicas 1997595fc2 Uplift Promenade to latest commit (k8s 1.20)
Updates Pegleg to include the latest Promenade updates, including
changes to support Kubernetes v1.20.

Complete list of Pegleg changes since the previous uplift:
* 06254b3 CoreDNS: Uplift to 1.7.0
* ae6782b Kubernetes: Uplift to v1.20.5
* 5cf854e CoreDNS: Migrate Corefile to version 1.6.4
* 9533be3 Add required apiserver serviceaccount flags
* f3febea Gate stability improvements
* 300a399 apiserver(-webhook): Allow fileless kube-apiserver command_options

Change-Id: I61fb95e0d35bb10b6f22f4dd1cff79a6d5f92df4
2021-05-05 05:02:14 +00:00
Phil Sphicas 3cd44ad855 Uplift Promenade, update bootstrap-armada
Uplift the Promenade commit to pick up a change that disables the
kube-apiserver insecure-port in the Armada bootstrap pod. [0]

List of Promenade changes since the last uplift:
* fd9f3d6 Stop using kube-apiserver insecure-port
* ef66d10 Remove TaintBasedEvictions feature gate references
* c6b62ff apiserver(-webhook): Allow volume overrides
* 27f181a Add configmap-etc-hash to apiserver anchor
* a57158d Disable kubernetes-etcd anchor cleanup in gates

0: https://review.opendev.org/c/airship/promenade/+/784016

Change-Id: Idfb28343b7ab3a69b420f3a63ef0d42a5259a84f
2021-05-05 04:54:02 +00:00
Phil Sphicas 2a0519e397 Fix failing check_pki_certs test
Unit tests are failing (as they will annually) for check_pki_certs,
which is run against the airsloop site in the treasuremap repo. Similar
to the change last year [0], this updates the treasuremap ref to the tip
of the v1.9 branch, which includes new certs for airsloop [1], seaworthy
[2], and seaworthy-virt [3] sites.

Change-Id: Ibfcdde11a35ed2207e78c2d2f372ab2d2e1f2e63
0: https://review.opendev.org/c/airship/pegleg/+/729269
1: https://review.opendev.org/c/airship/treasuremap/+/789482
2: https://review.opendev.org/c/airship/treasuremap/+/789483
3: https://review.opendev.org/c/airship/treasuremap/+/789484
2021-05-04 16:26:33 +00:00
Zuul 753517c419 Merge "Cache fernet keys when encrypting or decrypting" 2021-02-22 17:13:53 +00:00
Phil Sphicas 8f68c2b4f2 Uplift promenade, remove hyperkube
Uplifts promenade to the latest verson. Notably, this removes support
for kubelet extraction from the hyperkube image (which has not been
possible since Kubernetes v1.16).

Complete list of promenade changes since the last uplift:
* 5bb5886 Uplift Kubernetes to v1.19.7
* 023e7d4 Uplift etcd to v3.4.13
* e2324e7 Remove remaining hyperkube references
* 5323ca2 Deploy with standalone kubernetes images
* c7e7294 Remove hyperkube extraction functionality
* 0307391 Update cleanup.sh developer script

Change-Id: I51766a1b9fb8bb2e86f60370625a2bb81fd9e8f6
2021-02-15 04:33:54 +00:00
Phil Sphicas ce50da79e7 Uplift promenade to support non-tar urls
Uplifts promenade to a version that supports specifying a direct url as
the source for a file in the HostSystem and Genesis schema.

Complete list of promenade changes since the last uplift:
* c9862e5 Allow url as source of file to be deployed to host
* d161528 Avoid calico-etcd crashloop
* 77c7624 Fix ubuntu_xenial build (use pip <21.0)
* 630e504 Update to container image repo k8s.gcr.io
* 5e70957 Merge "Makefile; clean should include .tox"
* 946a28d Use HostToContainer mountPropagation
* f29d6df Ignore upstream chart repos when installing Helm
* 2f2a872 Makefile; clean should include .tox
* 922e3b2 Uplift HTK for etcd backup/restore delete archive capability
* de9f841 kube-apiserver: disable http2

Change-Id: Ia054136956d0a6c3ac24ae1658085a62157427ea
2021-02-15 04:33:54 +00:00
Phil Sphicas f020bdbc3f Fix ubuntu_xenial build (use pip <21.0)
pip 21.0 requires Python >= 3.6. [0]
The latest official python3 package for xenial is 3.5.1-3 [1]

Until we stop building xenial images, ensure that an older pip version
is used.

0: https://pypi.org/project/pip/21.0/
1: https://packages.ubuntu.com/xenial/python3
Change-Id: I6a51ae5b9e3222ca404c7ccd7dea1209b20ce8fd
2021-02-15 04:33:54 +00:00
Phil Sphicas c7b903a6ea Cache fernet keys when encrypting or decrypting
The PEGLEG_PASSPHRASE and PEGLEG_SALT environment variables are used to
generate the fernet key for encryption and decryption of site documents,
and the global passphrase and salt are used to generate the fernet key
for global documents.

This change caches the resulting fernet keys to avoid recalculating them
for each document that needs to be encrypted or decrypted, resulting in
a small time savings.

Change-Id: I7b7e77a4740e7abb54efce2fcb3cca6d84a9d7d4
2021-01-19 15:26:11 +00:00
Alexander Hughes 452734fb72 Making certificate check more robust
Some operators may have externally managed certificates that do not
conform to the originally expected certificate pattern of

---BEGIN CERTIFICATE-----
foo
-----END CERTIFICATE-----

and may instead include additional information on the subject/issuer. In
these cases the current regex will fail to load certs that can be parsed
with the existing cfssl command. Addressing this by tightening up the
regex prior to trying to obtain certificate information.

Change-Id: Ief9993632718caa46b52761b49a97621f134ca53
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
2020-12-29 16:16:57 -05:00
francisy 8ad13a330e Edit cli.rst
Adding in note under environment variables that PROMENADE_ENCRYPTION_KEY environment variable is required

Change-Id: I52c24608b2ecf61cbeea693ccbae1d2dd081ac29
2020-11-16 15:12:12 +00:00
Rick Bartra 0b2ac8f953 Update pip package versions in preparation of pip 20.3
When pip is upgraded to 20.3, the pip dependency resolver is much more
strict and will no longer install a combination of packages that is mutually
inconsistent[0].

These changes account for the fact that Pegleg imports Shipyard, Promenade,
and Deckhand. Having said that, with pip 20.3, the pip packages amongst
those projects cannot conflict. A follow-up change may be needed if more
conflicts are found.

[0] https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020

Change-Id: Iedf7227c468d678430a5525a2d71d22ce210c557
2020-10-19 12:24:38 -04:00
Phil Sphicas 2600b16694 Uplift promenade to support non-hyperkube images
Uplift promenade to include templates without /hyperkube references,
allowing the use of standalone apiserver, controller, proxy, and
scheduler images.

Changes in promenade since previous uplift:
* d603386 Remove additional /hyperkube reference
* 8513b46 Update pip package versions in preparation of pip 20.3
* fb36579 kube-apiserver: use HTTP probes instead of exec
* be7b82e kube-apiserver: Allow probe customization
* 8c52be3 Remove /hyperkube prefix
* 9402535 Change helm-toolkit dependency version to ">= 0.1.0"
* b41c315 Accelerate YAML operations with LibYAML
* 14a95e6 Upgrade LibYAML version in container builds
* 6d7df80 Fix bug with block_literal_representer
* 9853fcf fix - pep8 and image build gates
* 9742790 Upgrade etcd to 3.4.3

Change-Id: I48628259f8db26a925f92ef859f82e7928ab786a
2020-09-30 03:24:26 +00:00
Phil Sphicas 0524e01779 Accelerate YAML operations with LibYAML
Patch PyYAML (via the pylibyaml library) to automatically enable the
LibYAML parser and emitter, which are faster than the Python versions.

https://pypi.org/project/pylibyaml/

Change-Id: Ic48d2234ca3107404d9f883ca6038a12ca06a408
2020-09-25 01:01:20 +00:00
Phil Sphicas 3ca39ef54a Include LibYAML in container builds
Updates Dockerfiles to build the LibYAML library, which can provide much
faster YAML parsing and emitting than the native Python library.

https://pyyaml.org/wiki/LibYAML

Change-Id: I4cd48d5d5b5dddc44c88e9e08e405db96359ea6f
2020-09-25 01:01:20 +00:00
Zuul c55dc2346e Merge "Fix bug with block_literal_representer" 2020-09-24 15:49:56 +00:00
Zuul bd02eec539 Merge "Ignore binary files when whitespace linting" 2020-09-24 15:49:55 +00:00
Zuul 02ec2ff43d Merge "Promenade uplift" 2020-09-24 15:42:35 +00:00