This PS bumps up Airflow version to the latest
2.8.2 and also bumps up openstack dependences to
Antelope 2023.1
Change-Id: Iae72c6da9406749cf157437495f31dc3b9f6ba2c
This PS updates python modules and code to match Airflow 2.6.2:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
airflow-2.6.2 constraints
Change-Id: I9c3e139b3437414a61af7e7c0b7d7e533fadefda
This PS adjusts list of dependences needed to get syncronized with Shipyard project:
- lock sphinx with 3.3.1 version for doc generation
- updated deckhand reference
- adjusted other python dependences
Change-Id: I5b0a60a2c0709a37d65cb8258bf8c79631c94f00
upgrades kubernetes client to v1.26.0
remove installation of containerd during genesis.sh to prevent containerd downgrade
update bitnami kubectl image to image with curl installed for readiness check
Change-Id: I3afd5a7e7211bae3f52263167a62a012da0619a0
add focal dockerfile
update zuul jobs for focal
update tox for tox4 changes
update all requirements to latest and match deckhand
update cfssl from R1.2 to v1.6.3
fixed local gates for focal
updated examples promenade manifests to run on focal
Change-Id: I2af4043784766d36588c6f738053ad66e7b89a90
Address changes and deprecations in Kubernetes v1.21=>v1.23
controller-manager:
* --authorization-kubeconfig and --authentication-kubeconfig must be set
* liveness/readiness probes must use HTTPS
* the default port has been changed to 10257
kubelet:
* --dynamic-config-dir has been deprecated, will not move to GA
* --cni-bin-dir has been deprecated, will be removed with dockershim
* --cni-conf-dir has been deprecated, will be removed with dockershim
* --network-plugin has been deprecated, will be removed with dockershim
https: //github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#deprecation
https: //kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/
https: //github.com/kubernetes/enhancements/tree/master/keps/sig-node/281-dynamic-kubelet-configuration
Change-Id: Ia996d7c14d81d1d8b8067f11c02ffb4ce90eb49a
Installing Deckhand req. can be problematic which also
installs pbr for the ubuntu_xenial image.
* Install requirements sequential for ubuntu_xenial image
* Install pbr before Deckhand
Change-Id: I647fa42beb749d4b0cbf3278509780777d82334b
The extraction of the monolithic hyperkube binary from its container
image to be used as kubelet was last relevant in Kubernetes 1.16. Since
then, the hyperkube image has been deprecated, the structure of the
image has been changed, and it has ultimately been eliminated in
Kubernetes 1.19.
This change cleans up promenade accordingly.
Reverts the following commits:
* 886007b New CLI option to extract hyperkube
* 32a6c15 hyperkube image in promenade init
* 955deed New source for hyperkube binary definition
Change-Id: Ib62ecdf1af13abe8202a4ba4f86c39b9042ed13f
When pip is upgraded to 20.3, the pip dependency resolver is much more
strict and will no longer install a combination of packages that is mutually
inconsistent[0].
These changes account for the fact that Shipyard imports Armada, Drydock,
Promenade, and Deckhand. Having said that, with pip 20.3, the pip
packages amongst those projects cannot conflict. A follow-up change may
be needed if more conflicts are found.
Change-Id: Ie6effbdae759158e19b0b0adb2bdac0396eab047
Patch PyYAML (via the pylibyaml library) to automatically enable the
LibYAML parser and emitter, which are faster than the Python versions.
https://pypi.org/project/pylibyaml/
Change-Id: Iad54bfd21083b24cad5429bd8ecf794a9ead513e
This ps makes following changes to upgrade kubernetes from v1.17.3
to v1.18.6.
- Updated all references to k8s images to 1.18.6
- Updated command options and api object and versions based on
k8s 1.18 release notes:
https://kubernetes.io/docs/setup/release/notes/
- Uplifted uwsgi to 2.0.19.1 to align with other airship
components, and to bring in fixes and improvements.
- Added build-essentials and python3-dev packages to pass the zull
gate, which was looking for a c compiler.
Change-Id: I1160d1e6e2f02a0524043641b9296ea39edb301e
Pegleg is a consumer of both Promenade and Deckhand, these projects
have conflicting versions of multiple Python requirements.
This patch:
- Uplifts Promenade to more recent versions of these dependencies to
match Deckhand.
- Applies `tox -e freeze` to uplift requirements-frozen.txt
Locally these changes can be tested to ensure compatability with
Pegleg by following these steps:
1. Checkout this patch locally
2. Clone Pegleg to local disc
3. Update Pegleg's Pipfile, specifically changing line 30:
promenade = {file = "/your/path/to/local/promenade"}
4. Run tox -e update-requirements from Pegleg
If these steps are successful you will see the following output:
update-requirements: commands succeeded
congratulations :)
Change-Id: Ifdc74c4f80f599058e8b56008e234324a6d89e49
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
The current Promenade image is vulnerable to several CVEs:
CVE-2019-3462
CVE-2018-16865
CVE-2018-16864
Which Ubuntu 16.04/18.04 addresses.
This patchset makes the following changes:
1. Adds new distro specific dockerfiles for xenial/bionic.
2. Updates gates to be specific about the ubuntu image being
checked.
3. Updates .zuul.yaml checks/gates/post jobs for xenial/bionic.
4. Updates build-image.sh docker build for specific dockerfile
specified in config.sh (IMAGE_PROMENADE_DISTRO).
Change-Id: I89e5297a3baa8c2d2c142e5e29932476fc628398
Upgrades requests 2.20->2.22 to fix a security vulnerability.
Upgrades PyYAML 3.12->~5.1 to fix incompatibility with the version used
by pegleg. This upgrade should not affect any yaml references currently
used in promenade.
Upgrades deckhand to match newer version used by pegleg.
Change-Id: Ie0ccfb36c4942ce0c782c3d2ffac70919c4c24f6
Pegleg Promenade Deckhand and Shipyard should all use same versions
of packages when able. Requests currently is giving a warning
in Pegleg:
ERROR: deckhand 0.0.1.dev657 has requirement urllib3==1.24, but
you'll have urllib3 1.24.3 which is incompatible.
Change-Id: Icdd5c687bb7072bd479107fa9a6f80a0fe1284e9
Uplift deckhand dependency to include support for v2 schemas [0].
[0]: https://review.opendev.org/#/c/666659/
Change-Id: I0e4cc96a03b9e58a946cb3570e9027e85507f970
Now it's possible to use hyperkube Docker image to extract hyperkube binary.
Use case for this feature is kubelet/kubectl delivery in one binary(hyperkube)
which is built into Docker image. Promenade will extract hyperkube from Docker image,
create symlinks for kubelet/kubectl pointed to hyperkube. To do so promenade container
need to be configured to use Docker on the host where this container will be created.
This is happening only for script generation for genesis node. Later when promenade
will be started as a service pod inside ucp cluster it will generate scripts for joining nodes
by using cached hyperkube from /tmp.
Old way to delivery kubelet from tarball is still supported.
Configuration for the new method.
Need to export environment variables to properly configure Docker in Docker.
Docker socket should be provided as a mounted file inside promenade.
Also need to set temporary permissions for this socket during the build scripts stage.
Example:
DOCKER_SOCK="/var/run/docker.sock"
sudo chmod o+rw $DOCKER_SOCK
export DOCKER_HOST="unix:/${DOCKER_SOCK}"
export PROMENADE_TMP="abs_path_tmp_dir_on_host"
export PROMENADE_TMP_LOCAL="tmp_dir_inside_container"
After genesis scripts generation Docker socket permission should be turned back:
sudo chmod o-rw $DOCKER_SOCK
Change-Id: Ida22ea934fc551fec34df162d8147c8b9e630330
A recent Promenade change [0] introduced user-context tracing support in
Promenade, which requires the passing of an additional parameter to the
Deckhand API client. The Deckhand client was updated to handle the
additional parameter [1], but the Deckhand version Promenade uses was
not updated. This change uplifts the version of Deckhand in
requirements.txt for Promenade to a version that supports the
aforementioned, additional, user-context parameter.
[0] https://review.openstack.org/634071
[1] https://review.openstack.org/634068
Change-Id: Iaea328b32f95262e9e81be957893b7a50ac295fa
This change adds the global zuul pep8 tox job, which runs both
bandit and pep8 using tox. This also removes the two other airship
specific lint-pep8 and bandit zuul jobs since they are both covered
by the default openstack global one.
Also cleaned up the tox.ini by moving the requirements into the
test-requirements.txt file.
Change-Id: Iccf6228ab9e6d621d3047994b3adc192d67273c9
- Uplift deckhand commit to include fix for document replacement
- Add pod status polling to genesis bootstrap
Change-Id: I32d22110749b334d1fbf19f910e41ab0b7ff3a16
Use the Deckhand engine module directly to manage local configuration
files during CLI usage.
Note: not doing document validation as DH currently requires schemas to
be sourced from the database. Simple schema validation in place.
- Layering/substitution
- Schema validation based on DataSchema documents in payload
- Add deckhand to requirements
A few tooling updates
- concatenate test & schema yaml files into a single file to avoid name
conflicts
- make nginx directory in build-scripts stage
Change-Id: I2d56244f01c58052f14331bc09fd5843d4c95292
This change includes several interconnected features:
* Migration to Deckhand-based configuration. This is integrated here,
because new configuration data were needed, so it would have been
wasted effort to either implement it in the old format or to update
the old configuration data to Dechkand format.
* Failing faster with stronger validation. Migration to Deckhand
configuration was a good opportunity to add schema validation, which
is a requirement in the near term anyway. Additionally, rendering
all templates up front adds an additional layer of "fail-fast".
* Separation of certificate generation and configuration assembly into
different commands. Combined with Deckhand substitution, this creates
a much clearer distinction between Promenade configuration and
deployable secrets.
* Migration of components to charts. This is a key step that will
enable support for dynamic node management. Additionally, this paves
the way for significant configurability in component deployment.
* Version of kubelet is configurable & controlled via download url.
* Restructuring templates to be more intuitive. Many of the templates
require changes or deletion due to the migration to charts.
* Installation of pre-configured useful tools on hosts, including calicoctl.
* DNS is now provided by coredns, which is highly configurable.
Change-Id: I9f2d8da6346f4308be5083a54764ce6035a2e10c
* remove old files
* sketch of non-bootkube genesis
* add basic chroot/bootstrap script
* cleanup kubectl/kubelet fetching
* fix cni bin asset path
* add non-pod asset loader
* add example ca
* refactor key gen/distribution
* flannel up on genesis
* refactor some code toward join
* WIP: last commit working on "self-hosted, helm-managed"
* first pass at consolidating config for vanilla deploy
* refactor cli a bit
* use provided cluster ca
* separate genesis and join scripts
* add basic etcd joining
* actually run the proxy everywhere
* update readme
* enable kubelet service
* add pki most places
* use consistent sa keypair
* use quay.io/attcomdev/promenade
* fix typo in n3
* tls everywhere in kubernetes
* tls for etcd
* remove currently unused files