Commit Graph

310 Commits

Author SHA1 Message Date
Sergiy Markin 1a0ea41f28 Airflow stable 2.8.1
Change-Id: I49093c3c5bbdf076ef9f2ffa39769f9a2ec55c5c
2024-02-22 21:09:21 +00:00
Ruslan Aliev b154334a20 Adjust kubectl get command for armada charts
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ic830568efb7589b46282ad3fd9293e4358929468
2024-02-08 21:03:58 -06:00
SPEARS, DUSTIN (ds443n) 7ce7301476 Update ETCD to v3.5.11
Since after v3.5.6 etcd-io switched to a
distroless base image. Etcd anchor pods
are now using etcd-utility and etcd is
running a sidecar for health checks.

Change-Id: I198dca1209097de4d60a53a7568f0c4790679599
2024-02-08 10:35:33 -05:00
Ruslan Aliev 910b06e4ba Add complete support for operator-based bootstrap
* operator logs is now streaming to pipeline and to pod
 * printing status of armada chart objects
 * adjust armada container cmd parameters to support both
   golang and python based images

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I6d8629a48c1b862db937ddc3cd68792220388b19
2024-01-10 10:27:07 -06:00
Ruslan Aliev 2dda3c505c Enable configurable support of armada apply operator mode
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ie5e582aa7b4a64a4fa3f3fabb8d65ded76e14340
2024-01-05 11:07:12 -06:00
Ruslan Aliev 6d90e785ff Change permissions to 0600 of kubelet.service.
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Id4ec3c301f4e0ed54168389ea23afc5b3043a6ff
2023-12-11 20:02:27 +00:00
Ruslan Aliev 29405cec00 Add configurable support for armada-operator
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I22cf48839ccfd62a6ed293080fd8b90a2f31a5f9
2023-11-20 17:18:15 -06:00
Anselme, Schubert (sa246v) a20e00e485
Deprecating the Ingress Class Annotation
Upgrading htk to version 0.2.55, which deprecates the ingress class
annotation (kubernetes.io/ingress.class) with .spec.ingressClassName

https://review.opendev.org/c/openstack/openstack-helm-infra/+/891720

Change-Id: Ibdec296ed9998d8fae22256aa1efa72933d53276
Signed-off-by: Anselme, Schubert (sa246v) <sa246v@att.com>
2023-10-06 09:57:59 -04:00
Sergiy Markin 69a74590e7 Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
  airflow-2.6.2 constraints

Change-Id: I9c3e139b3437414a61af7e7c0b7d7e533fadefda
2023-08-29 21:12:11 +00:00
Ruslan Aliev 16debd8174 Remove allow-downgrade and dist-upgrades parts from up.sh
These changes were not needed and have negative impact on
the node deployment process.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I134a2acdf831f1c1e2f475a09b2f1d4a85cf68bf
2023-06-08 18:57:29 -05:00
Sergiy Markin 45bcba068e Promenade timeout adjustment
This PS set read/connection timeout to None for http requests.

Change-Id: Ia2d8b59a5fa55bac24575149c8c1e8a67707c29d
2023-05-12 16:19:33 +00:00
Sergiy Markin a3fdc9e52c [focal] Fix requests.body attribute deprecation
This PS updates calls to body attribute of requests module with new text
attribute.

Change-Id: I8f69d18a0ac5a0065072642a58364584392bde37
2023-05-10 21:39:02 +00:00
Sergiy Markin 32ad8a96b0 [focal] Python modules sync with Airship project
- uplifted/downgraded some python modules
- fixed falcon.API deprecation - -> falcon.App
- uplifted deckhand reference for python deps
- fixed formatting style  using yapf linter
- added bindep role and bindep.txt file with required deps
- fixed quai docker image publishing
- re-enabled openstack-tox-py38 gate job

Change-Id: I0e248182efad75630721a1291bc86a5edc79c22a
2023-04-21 06:09:14 +00:00
az7961 b549359b9c Make sure kubernetes starts after containerd
Change-Id: If1627c29b5760bce029fc6e0458900bde2919bde
2023-04-18 11:46:09 -05:00
SPEARS, DUSTIN (ds443n) 27a8b0d798 k8s upgrade to 1.26.0
upgrades kubernetes client to v1.26.0
remove installation of containerd during genesis.sh to prevent containerd downgrade
update bitnami kubectl image to image with curl installed for readiness check

Change-Id: I3afd5a7e7211bae3f52263167a62a012da0619a0
2023-03-20 13:16:48 -04:00
Wahlstedt, Walter (ww229g) 8ce937a9f7 updates for focal
add focal dockerfile
update zuul jobs for focal
update tox for tox4 changes
update all requirements to latest and match deckhand
update cfssl from R1.2 to v1.6.3
fixed local gates for focal
updated examples promenade manifests to run on focal

Change-Id: I2af4043784766d36588c6f738053ad66e7b89a90
2023-02-27 12:11:07 -05:00
Mosher, Jaymes (jm616v) 7da16b3c0b Fix haproxy.cfg template
Fix small typo in previous patch:
https://review.opendev.org/c/airship/promenade/+/854466

The extra trailing newline was removed when I added the comment.

Change-Id: I45bd68ff09f0e099ca7a0c10eb8e26671bc315bf
2022-08-31 13:58:43 -06:00
Mosher, Jaymes (jm616v) 95a668541c Ensure haproxy.cfg ends with newline
Versions of Haproxy >=2.3 require the config file to end in a newline
or they'll exit with an error.

Change-Id: I9301ea679536b10ee5ad0d87d42c1655e5852616
2022-08-25 12:02:04 -06:00
Ruslan Aliev e207bbe966 k8s upgrade to v1.23.7
Address changes and deprecations in Kubernetes v1.21=>v1.23

controller-manager:
* --authorization-kubeconfig and --authentication-kubeconfig must be set
* liveness/readiness probes must use HTTPS
* the default port has been changed to 10257

kubelet:
* --dynamic-config-dir has been deprecated, will not move to GA
* --cni-bin-dir has been deprecated, will be removed with dockershim
* --cni-conf-dir has been deprecated, will be removed with dockershim
* --network-plugin has been deprecated, will be removed with dockershim

https: //github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#deprecation
https: //kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/
https: //github.com/kubernetes/enhancements/tree/master/keps/sig-node/281-dynamic-kubelet-configuration
Change-Id: Ia996d7c14d81d1d8b8067f11c02ffb4ce90eb49a
2022-06-29 00:21:45 -05:00
Phil Sphicas 363ceca069 Fix typo in Helm wrapper script
The previous update to the Helm wrapper script [0] was incorrect.

0: https://review.opendev.org/c/airship/promenade/+/833640
Change-Id: I14aa996357af9c7ca81dc68fc9fc06fe9461ce4f
2022-03-21 08:32:39 -07:00
Phil Sphicas 79cb856fab Update Helm wrapper script
Small updates to incorporate shellcheck recommendations.

Change-Id: I450706b404735f07eef8cf605303363030db16b3
2022-03-14 07:54:26 -07:00
Phil Sphicas bc24599267 Use same helm wrapper script for all nodes
The helm wrapper script should be the same for genesis and non-genesis
nodes. The one previously used by join nodes is removed.

Change-Id: I212127f258b9eba4fce776cb690060dc413061ca
2022-02-01 18:18:35 -08:00
Phil Sphicas 08906262fd Update tolerations and priority classes
* Give kube-proxy a blanket toleration
* Replace scheduler.alpha.kubernetes.io/critical-pod annotation with
    priorityClassName: system-node-critical

Change-Id: I810333913c09531eefa1ded014fe090d4cca7f7d
2021-10-18 11:33:54 -07:00
Phil Sphicas e43b6f0128 Remove log-test pod if validation succeeds
The validation function validate_kubectl_logs, which may be executed as
part of genesis or cluster join, creates a log-test pod and checks that
the 'kubectl logs' output is correct.

These completed pods don't really need to live in the cluster beyond the
initial deployment.

This change deletes the log-test pod if the validation is successful.

Change-Id: I6ae9c55f960ea70335d1fd79380c7119dc11a5e2
2021-10-18 10:03:12 -07:00
Sean Eagan 53d0ecb7f9 Remove Tiller
Depends-On: https://review.opendev.org/c/airship/armada/+/812047
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: Ic1eddda3639d629a94bb39f93bf48da54445469f
2021-10-06 13:16:34 +00:00
Sean Eagan 1017536b9f Move to helm 3 CLI in cluster
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I3c385547d30796aecb67aaa6482da34765052c47
2021-10-01 12:23:12 -05:00
Phil Sphicas bc14e9bbb9 Ensure /etc/etcd and /var/log/kubernetes exist
Create additional directories on the host, ensuring that they exist with
the appropriate permissions:
- /etc/etcd
- /var/log/kubernetes

Change-Id: I0b7bed19b849037cfcc812453731460563270278
2021-08-14 21:11:58 +00:00
Thirunavukkarasu Palani 5f964eced5 Kubelet warning fix
Warning: For remote container runtime,
--pod-infra-container-image is ignored in kubelet,
which should be set in that remote runtime instead

Change-Id: Iec2df4873857c0d74a267810ef215f246102c2f4
2021-07-27 18:58:46 +00:00
Thirunavukkarasu Palani 7692b36fe9 Fix deprecated warning in Promenade controller-manager chart
Deprecated warning:
1. Flag --address has been deprecated, see --bind-address instead.
2. Flag --port has been deprecated, see --secure-port instead.

Change-Id: Ie93e95ab755dd338ac31914d1a50e61e351b907e
2021-07-14 04:15:41 +00:00
ubuntu 183b977754 Fix deprecated warning in Promenade apiserver chart
Removed PersistentVolumeLabel from apiserver to fix below warning.
Deprecated warning:
1. PersistentVolumeLabel admission controller is deprecated.
   Please remove this controller from your configuration files and scripts.
2. insecure-port has been deprecated, This flag has no effect now
   and will be removed in v1.24.

Change-Id: Iaccff8467b5ed967fa41e85b38c27f7345cd97bb
2021-06-29 16:14:17 +00:00
Phil Sphicas 9533be32a1 Add required apiserver serviceaccount flags
In v1.20, TokenRequest and TokenRequestProjection become GA features,
and the following flags are required by the API server:
* --service-account-issuer
* --service-account-key-file
* --service-account-signing-key-file

This change ensures that the flags are set, and that the required keys
are in the right places.

Change-Id: I6606c5b1c9ff005d1943b424e3e7ad4d20b68408
2021-04-30 22:45:43 +00:00
Phil Sphicas fd9f3d6cec Stop using kube-apiserver insecure-port
The tiller container in the armada bootstrap pod relies on the insecure
port that kube-apiserver once listened on by default. The kube-apiserver
ability to serve on an insecure port, deprecated since v1.10, has been
removed in v1.20. [0]

This change updates the armada bootstrap pod to use the secure port
instead.

0: https://github.com/kubernetes/kubernetes/pull/95856

Change-Id: I6a37fa4e7f97c7aaa3cd0f61b56214483a7dc217
2021-04-21 21:52:29 +00:00
Phil Sphicas e2324e7db8 Remove remaining hyperkube references
This change eliminates all remaining references to hyperkube, as it is
no longer supported.

Change-Id: Id0a4c142b1dc76561f7d2c18fb76edfc5a60267a
2021-02-11 17:23:32 +00:00
Phil Sphicas c7e72942a9 Remove hyperkube extraction functionality
The extraction of the monolithic hyperkube binary from its container
image to be used as kubelet was last relevant in Kubernetes 1.16. Since
then, the hyperkube image has been deprecated, the structure of the
image has been changed, and it has ultimately been eliminated in
Kubernetes 1.19.

This change cleans up promenade accordingly.

Reverts the following commits:
* 886007b New CLI option to extract hyperkube
* 32a6c15 hyperkube image in promenade init
* 955deed New source for hyperkube binary definition

Change-Id: Ib62ecdf1af13abe8202a4ba4f86c39b9042ed13f
2021-02-11 17:23:32 +00:00
Phil Sphicas c9862e5749 Allow url as source of file to be deployed to host
This change allows the HostSystem and Genesis document to specify direct
URLs to files (for example, kubelet and kubectl) that are to be written
to the deployed hosts.

Change-Id: I1828d4a9e654537448631434b26b5becc4d2d717
2021-02-11 17:23:32 +00:00
Phil Sphicas d603386d49 Remove additional /hyperkube reference
The /hyperkube prefix isn't required and causes problems when using
non-hyperkube images elsewhere.

Related earlier change: https://review.opendev.org/#/c/754487/

Change-Id: I23918669bae4d9b7d41140b2c26d3176c45665ee
2020-09-29 23:27:27 +00:00
Rick Bartra 8513b46153 Update pip package versions in preparation of pip 20.3
When pip is upgraded to 20.3, the pip dependency resolver is much more
strict and will no longer install a combination of packages that is mutually
inconsistent[0].

These changes account for the fact that Shipyard imports Armada, Drydock,
Promenade, and Deckhand. Having said that, with pip 20.3, the pip
packages amongst those projects cannot conflict. A follow-up change may
be needed if more conflicts are found.

Change-Id: Ie6effbdae759158e19b0b0adb2bdac0396eab047
2020-09-28 16:34:04 -04:00
Phil Sphicas b41c3151fb Accelerate YAML operations with LibYAML
Patch PyYAML (via the pylibyaml library) to automatically enable the
LibYAML parser and emitter, which are faster than the Python versions.

https://pypi.org/project/pylibyaml/

Change-Id: Iad54bfd21083b24cad5429bd8ecf794a9ead513e
2020-09-24 04:27:29 +00:00
Phil Sphicas 6d7df808f3 Fix bug with block_literal_representer
When serializing a block literal, be explicit that we want to treat it
as a string, instead of relying on implicit conversion.

Change-Id: Ia79600ebc228d8417342a0703167f34703169d5a
2020-09-24 04:27:29 +00:00
Mahmoudi, Ahmad (am495p) c302a083a6 Upgrade k8s from v1.17.3 to v1.18.6
This ps makes following changes to upgrade kubernetes from v1.17.3
to v1.18.6.
  - Updated all references to k8s images to 1.18.6
  - Updated command options and api object and versions based on
    k8s 1.18 release notes:
      https://kubernetes.io/docs/setup/release/notes/
  - Uplifted uwsgi to 2.0.19.1 to align with other airship
    components, and to bring in fixes and improvements.
  - Added build-essentials and python3-dev packages to pass the zull
    gate, which was looking for a c compiler.

Change-Id: I1160d1e6e2f02a0524043641b9296ea39edb301e
2020-08-19 15:56:45 +00:00
Ahmad Mahmoudi 9f42b502f7 Updated resiliency gate
Updated resiliency gate script to consistently pass all gate stages,
using ubuntu bionic image for node deployment.

- Updated developer-onbording.rst with information on how to configure
  and run the resilency gate behind corporate proxy.
- Updated the gate scripts to use the proxy configuration.
- Updated up.sh to pull the hyperkube image as cache, to speed up and
  stabalize the initial kublet deployment of kubernetes cluster services.
- Updated and added sleeps and retries in some of gate stages and
  scripts to avoid gate failures due to transient environment issues.
- Updated the ubuntu base image for node deployments from xenial to\
  bionic base image.
- Added code in treadown-nodes stage to manually remove the etcd
  members: kubernetes and calico, since they still remain listed as
  etcd members on genesis node, even after genesis is torn down.

Change-Id: Ia11d66ab30ac7a07626d4f1d02a6da48155f862d
2020-07-21 22:45:23 +00:00
Phil Sphicas 6d72ff3d0c Improve HostSystem.yaml parsability
Some YAML parsers (e.g. libyaml) don't like : without a trailing space.
This adds whitespace to improve parser compatibility and readability.

Change-Id: I62230ab3caef4963b2b63a264396e7057530fd3f
2020-06-11 04:43:08 +00:00
John Lawrence 64fdac754a Enable NTP protocol access
Allow the Prometheus node exporter to access the NTP
metrics.

Change-Id: I623a84effc61cf496b5c524ddb2fabed8066a89a
2020-04-27 16:57:47 +00:00
Chris Wedgwood 81a941a055 add the ability to mask systemd units
Change-Id: I4f2a1ed3f5b2d4491784bef9b6e4c9b2f3896396
2020-02-03 14:28:05 -06:00
Chris Wedgwood 6db4c70e9e [genesis] avoid restarting kubelet in cleanup
The cleanup process runs concurrently with pods that are actively
using kubernetes endpoints.  In kubelet restart the endpoints are
often recreated breaking networking.

For now avoid the final restart.

Change-Id: I852721caa853315c6550e253cd3813ae49f00a4a
2020-01-30 18:22:11 -06:00
Chris Wedgwood b65930f205 Prevent creation of kubernetes service endpoint by bootstrap apiserver
If the kubernetes apiserver (in the bootstrap Armada pod) runs with the
reconciler enabled, the kubernetes endpoint can be created with an
invalid port which will not be corrected later.

Change-Id: I6d5fb86c6c4ffded9f42bda6e2ffbf2fbc13806f
2020-01-28 14:02:28 -06:00
Anderson, Craig (ca846m) 704e818eda Fix systemd-resolved race conditions
1. systemd-resolved should be removed/disabled before the symlink is
2. `domain` is redundant with the FQDN and replaced by `search`
3. correct resolv.conf EOL formatting issue

Change-Id: If7f8037c0623d9b1eb43171f09e492985a66b351
2020-01-25 10:15:04 -08:00
Zuul 59d6ac7636 Merge "Remove kubelet restart from join.sh" 2020-01-14 16:37:03 +00:00
Matt McEuen 153882b153 Remove kubelet restart from join.sh
The kubelet restart at the end of the join script appears
to be unnecessary, since the only action taken by the script
between that and the previous kubelet start is node labelling
(which doesn't require a kubelet restart).

In addition, the timing of this restart may be triggering
a kubernetes state synchronization bug, where a pod's status
isn't updated to reflect the readiness of all of its containers.

Change-Id: I480d1b345e5ddcce0cac961ff9c2b76526c5b76f
2020-01-14 08:32:09 -06:00
Yasin, Siraj (SY495P) ff0a7ccabb [apt-retry] - Added logic to retry apt update
When there is failure to fetch any of the apt urls, it skips and
continues. Due to which apt install fails in next step.
So added retry if apt fetch fails before proceeding to apt install.

Change-Id: I658024481b1be98d280cb1c9c4c2fb733a0d5697
2020-01-09 16:17:45 +00:00