Commit Graph

214 Commits

Author SHA1 Message Date
Ruslan Aliev a58678d5d2 Add configurable ETCD parameters to aux cluster
Bump k8s, calico, etcd, coredns and helm.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I77373c223c6ea723ee31fe51e6fb4a9e84be03f7
2024-04-18 13:22:17 -05:00
Ruslan Aliev b154334a20 Adjust kubectl get command for armada charts
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ic830568efb7589b46282ad3fd9293e4358929468
2024-02-08 21:03:58 -06:00
SPEARS, DUSTIN (ds443n) 7ce7301476 Update ETCD to v3.5.11
Since after v3.5.6 etcd-io switched to a
distroless base image. Etcd anchor pods
are now using etcd-utility and etcd is
running a sidecar for health checks.

Change-Id: I198dca1209097de4d60a53a7568f0c4790679599
2024-02-08 10:35:33 -05:00
Ruslan Aliev 910b06e4ba Add complete support for operator-based bootstrap
* operator logs is now streaming to pipeline and to pod
 * printing status of armada chart objects
 * adjust armada container cmd parameters to support both
   golang and python based images

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I6d8629a48c1b862db937ddc3cd68792220388b19
2024-01-10 10:27:07 -06:00
Ruslan Aliev 2dda3c505c Enable configurable support of armada apply operator mode
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ie5e582aa7b4a64a4fa3f3fabb8d65ded76e14340
2024-01-05 11:07:12 -06:00
Ruslan Aliev 6d90e785ff Change permissions to 0600 of kubelet.service.
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Id4ec3c301f4e0ed54168389ea23afc5b3043a6ff
2023-12-11 20:02:27 +00:00
Ruslan Aliev 29405cec00 Add configurable support for armada-operator
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I22cf48839ccfd62a6ed293080fd8b90a2f31a5f9
2023-11-20 17:18:15 -06:00
Sergiy Markin 69a74590e7 Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
  airflow-2.6.2 constraints

Change-Id: I9c3e139b3437414a61af7e7c0b7d7e533fadefda
2023-08-29 21:12:11 +00:00
Ruslan Aliev 16debd8174 Remove allow-downgrade and dist-upgrades parts from up.sh
These changes were not needed and have negative impact on
the node deployment process.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I134a2acdf831f1c1e2f475a09b2f1d4a85cf68bf
2023-06-08 18:57:29 -05:00
az7961 b549359b9c Make sure kubernetes starts after containerd
Change-Id: If1627c29b5760bce029fc6e0458900bde2919bde
2023-04-18 11:46:09 -05:00
Wahlstedt, Walter (ww229g) 8ce937a9f7 updates for focal
add focal dockerfile
update zuul jobs for focal
update tox for tox4 changes
update all requirements to latest and match deckhand
update cfssl from R1.2 to v1.6.3
fixed local gates for focal
updated examples promenade manifests to run on focal

Change-Id: I2af4043784766d36588c6f738053ad66e7b89a90
2023-02-27 12:11:07 -05:00
Mosher, Jaymes (jm616v) 7da16b3c0b Fix haproxy.cfg template
Fix small typo in previous patch:
https://review.opendev.org/c/airship/promenade/+/854466

The extra trailing newline was removed when I added the comment.

Change-Id: I45bd68ff09f0e099ca7a0c10eb8e26671bc315bf
2022-08-31 13:58:43 -06:00
Mosher, Jaymes (jm616v) 95a668541c Ensure haproxy.cfg ends with newline
Versions of Haproxy >=2.3 require the config file to end in a newline
or they'll exit with an error.

Change-Id: I9301ea679536b10ee5ad0d87d42c1655e5852616
2022-08-25 12:02:04 -06:00
Ruslan Aliev e207bbe966 k8s upgrade to v1.23.7
Address changes and deprecations in Kubernetes v1.21=>v1.23

controller-manager:
* --authorization-kubeconfig and --authentication-kubeconfig must be set
* liveness/readiness probes must use HTTPS
* the default port has been changed to 10257

kubelet:
* --dynamic-config-dir has been deprecated, will not move to GA
* --cni-bin-dir has been deprecated, will be removed with dockershim
* --cni-conf-dir has been deprecated, will be removed with dockershim
* --network-plugin has been deprecated, will be removed with dockershim

https: //github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#deprecation
https: //kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/
https: //github.com/kubernetes/enhancements/tree/master/keps/sig-node/281-dynamic-kubelet-configuration
Change-Id: Ia996d7c14d81d1d8b8067f11c02ffb4ce90eb49a
2022-06-29 00:21:45 -05:00
Phil Sphicas 363ceca069 Fix typo in Helm wrapper script
The previous update to the Helm wrapper script [0] was incorrect.

0: https://review.opendev.org/c/airship/promenade/+/833640
Change-Id: I14aa996357af9c7ca81dc68fc9fc06fe9461ce4f
2022-03-21 08:32:39 -07:00
Phil Sphicas 79cb856fab Update Helm wrapper script
Small updates to incorporate shellcheck recommendations.

Change-Id: I450706b404735f07eef8cf605303363030db16b3
2022-03-14 07:54:26 -07:00
Phil Sphicas bc24599267 Use same helm wrapper script for all nodes
The helm wrapper script should be the same for genesis and non-genesis
nodes. The one previously used by join nodes is removed.

Change-Id: I212127f258b9eba4fce776cb690060dc413061ca
2022-02-01 18:18:35 -08:00
Phil Sphicas 08906262fd Update tolerations and priority classes
* Give kube-proxy a blanket toleration
* Replace scheduler.alpha.kubernetes.io/critical-pod annotation with
    priorityClassName: system-node-critical

Change-Id: I810333913c09531eefa1ded014fe090d4cca7f7d
2021-10-18 11:33:54 -07:00
Phil Sphicas e43b6f0128 Remove log-test pod if validation succeeds
The validation function validate_kubectl_logs, which may be executed as
part of genesis or cluster join, creates a log-test pod and checks that
the 'kubectl logs' output is correct.

These completed pods don't really need to live in the cluster beyond the
initial deployment.

This change deletes the log-test pod if the validation is successful.

Change-Id: I6ae9c55f960ea70335d1fd79380c7119dc11a5e2
2021-10-18 10:03:12 -07:00
Sean Eagan 53d0ecb7f9 Remove Tiller
Depends-On: https://review.opendev.org/c/airship/armada/+/812047
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: Ic1eddda3639d629a94bb39f93bf48da54445469f
2021-10-06 13:16:34 +00:00
Sean Eagan 1017536b9f Move to helm 3 CLI in cluster
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I3c385547d30796aecb67aaa6482da34765052c47
2021-10-01 12:23:12 -05:00
Phil Sphicas bc14e9bbb9 Ensure /etc/etcd and /var/log/kubernetes exist
Create additional directories on the host, ensuring that they exist with
the appropriate permissions:
- /etc/etcd
- /var/log/kubernetes

Change-Id: I0b7bed19b849037cfcc812453731460563270278
2021-08-14 21:11:58 +00:00
Thirunavukkarasu Palani 5f964eced5 Kubelet warning fix
Warning: For remote container runtime,
--pod-infra-container-image is ignored in kubelet,
which should be set in that remote runtime instead

Change-Id: Iec2df4873857c0d74a267810ef215f246102c2f4
2021-07-27 18:58:46 +00:00
Thirunavukkarasu Palani 7692b36fe9 Fix deprecated warning in Promenade controller-manager chart
Deprecated warning:
1. Flag --address has been deprecated, see --bind-address instead.
2. Flag --port has been deprecated, see --secure-port instead.

Change-Id: Ie93e95ab755dd338ac31914d1a50e61e351b907e
2021-07-14 04:15:41 +00:00
ubuntu 183b977754 Fix deprecated warning in Promenade apiserver chart
Removed PersistentVolumeLabel from apiserver to fix below warning.
Deprecated warning:
1. PersistentVolumeLabel admission controller is deprecated.
   Please remove this controller from your configuration files and scripts.
2. insecure-port has been deprecated, This flag has no effect now
   and will be removed in v1.24.

Change-Id: Iaccff8467b5ed967fa41e85b38c27f7345cd97bb
2021-06-29 16:14:17 +00:00
Phil Sphicas 9533be32a1 Add required apiserver serviceaccount flags
In v1.20, TokenRequest and TokenRequestProjection become GA features,
and the following flags are required by the API server:
* --service-account-issuer
* --service-account-key-file
* --service-account-signing-key-file

This change ensures that the flags are set, and that the required keys
are in the right places.

Change-Id: I6606c5b1c9ff005d1943b424e3e7ad4d20b68408
2021-04-30 22:45:43 +00:00
Phil Sphicas fd9f3d6cec Stop using kube-apiserver insecure-port
The tiller container in the armada bootstrap pod relies on the insecure
port that kube-apiserver once listened on by default. The kube-apiserver
ability to serve on an insecure port, deprecated since v1.10, has been
removed in v1.20. [0]

This change updates the armada bootstrap pod to use the secure port
instead.

0: https://github.com/kubernetes/kubernetes/pull/95856

Change-Id: I6a37fa4e7f97c7aaa3cd0f61b56214483a7dc217
2021-04-21 21:52:29 +00:00
Phil Sphicas e2324e7db8 Remove remaining hyperkube references
This change eliminates all remaining references to hyperkube, as it is
no longer supported.

Change-Id: Id0a4c142b1dc76561f7d2c18fb76edfc5a60267a
2021-02-11 17:23:32 +00:00
Phil Sphicas d603386d49 Remove additional /hyperkube reference
The /hyperkube prefix isn't required and causes problems when using
non-hyperkube images elsewhere.

Related earlier change: https://review.opendev.org/#/c/754487/

Change-Id: I23918669bae4d9b7d41140b2c26d3176c45665ee
2020-09-29 23:27:27 +00:00
Mahmoudi, Ahmad (am495p) c302a083a6 Upgrade k8s from v1.17.3 to v1.18.6
This ps makes following changes to upgrade kubernetes from v1.17.3
to v1.18.6.
  - Updated all references to k8s images to 1.18.6
  - Updated command options and api object and versions based on
    k8s 1.18 release notes:
      https://kubernetes.io/docs/setup/release/notes/
  - Uplifted uwsgi to 2.0.19.1 to align with other airship
    components, and to bring in fixes and improvements.
  - Added build-essentials and python3-dev packages to pass the zull
    gate, which was looking for a c compiler.

Change-Id: I1160d1e6e2f02a0524043641b9296ea39edb301e
2020-08-19 15:56:45 +00:00
Ahmad Mahmoudi 9f42b502f7 Updated resiliency gate
Updated resiliency gate script to consistently pass all gate stages,
using ubuntu bionic image for node deployment.

- Updated developer-onbording.rst with information on how to configure
  and run the resilency gate behind corporate proxy.
- Updated the gate scripts to use the proxy configuration.
- Updated up.sh to pull the hyperkube image as cache, to speed up and
  stabalize the initial kublet deployment of kubernetes cluster services.
- Updated and added sleeps and retries in some of gate stages and
  scripts to avoid gate failures due to transient environment issues.
- Updated the ubuntu base image for node deployments from xenial to\
  bionic base image.
- Added code in treadown-nodes stage to manually remove the etcd
  members: kubernetes and calico, since they still remain listed as
  etcd members on genesis node, even after genesis is torn down.

Change-Id: Ia11d66ab30ac7a07626d4f1d02a6da48155f862d
2020-07-21 22:45:23 +00:00
John Lawrence 64fdac754a Enable NTP protocol access
Allow the Prometheus node exporter to access the NTP
metrics.

Change-Id: I623a84effc61cf496b5c524ddb2fabed8066a89a
2020-04-27 16:57:47 +00:00
Chris Wedgwood 81a941a055 add the ability to mask systemd units
Change-Id: I4f2a1ed3f5b2d4491784bef9b6e4c9b2f3896396
2020-02-03 14:28:05 -06:00
Chris Wedgwood 6db4c70e9e [genesis] avoid restarting kubelet in cleanup
The cleanup process runs concurrently with pods that are actively
using kubernetes endpoints.  In kubelet restart the endpoints are
often recreated breaking networking.

For now avoid the final restart.

Change-Id: I852721caa853315c6550e253cd3813ae49f00a4a
2020-01-30 18:22:11 -06:00
Chris Wedgwood b65930f205 Prevent creation of kubernetes service endpoint by bootstrap apiserver
If the kubernetes apiserver (in the bootstrap Armada pod) runs with the
reconciler enabled, the kubernetes endpoint can be created with an
invalid port which will not be corrected later.

Change-Id: I6d5fb86c6c4ffded9f42bda6e2ffbf2fbc13806f
2020-01-28 14:02:28 -06:00
Anderson, Craig (ca846m) 704e818eda Fix systemd-resolved race conditions
1. systemd-resolved should be removed/disabled before the symlink is
2. `domain` is redundant with the FQDN and replaced by `search`
3. correct resolv.conf EOL formatting issue

Change-Id: If7f8037c0623d9b1eb43171f09e492985a66b351
2020-01-25 10:15:04 -08:00
Zuul 59d6ac7636 Merge "Remove kubelet restart from join.sh" 2020-01-14 16:37:03 +00:00
Matt McEuen 153882b153 Remove kubelet restart from join.sh
The kubelet restart at the end of the join script appears
to be unnecessary, since the only action taken by the script
between that and the previous kubelet start is node labelling
(which doesn't require a kubelet restart).

In addition, the timing of this restart may be triggering
a kubernetes state synchronization bug, where a pod's status
isn't updated to reflect the readiness of all of its containers.

Change-Id: I480d1b345e5ddcce0cac961ff9c2b76526c5b76f
2020-01-14 08:32:09 -06:00
Yasin, Siraj (SY495P) ff0a7ccabb [apt-retry] - Added logic to retry apt update
When there is failure to fetch any of the apt urls, it skips and
continues. Due to which apt install fails in next step.
So added retry if apt fetch fails before proceeding to apt install.

Change-Id: I658024481b1be98d280cb1c9c4c2fb733a0d5697
2020-01-09 16:17:45 +00:00
Phil Sphicas 1a1c69c064 Fix: genesis.sh and join.sh rendering fix
Fixes a rendering issue with the previous HostSystem schema change when
common packages are omitted.

https://review.opendev.org/#/c/699162/

Change-Id: I629c652be1575351c8b33b141467f2839badc112
2019-12-18 08:39:23 -08:00
Phil Sphicas 04ddbcd2a4 promenade/HostSystem/v1 schema cleanup
When the genesis and join package source definitions were split, the
.common, .genesis, and .join subkeys were inserted directly under
.properties.packages instead of .properties.packages.properties),
causing anything under packages to erroneously pass.

This change implements the intended validation, allowing packages to be
defined under either under .packages.common, or .packages.genesis and
.packages.join. The expectation is that the genesis node will end up
with the union of what is defined under genesis and common. Required
packages (a runtime and socat) need to be defined in at least one of
those locations. Similarly, join nodes will have the union of join
packages and common packages.

Change-Id: I4a658eef6efbba53ba04b2d8b4ea4711ca0b1ab0
2019-12-16 20:36:10 -08:00
Ahmad Mahmoudi f5ad002690 Ubuntu 18.04, configure chrony
Added ntp server configuration to be used by chrony as the recommended
ntp service for ubuntu 18.04.
Since, chorny is not installed by default in ubuntu 18.04, also added
chrony apt package to be installed on airship nodes.

Change-Id: If1437a79cf89806043f62e2eac49c3b4b5eae2cd
2019-12-11 15:02:24 -06:00
Ahmad Mahmoudi d5afa2030c Disable systemd-resolved
- Disable systemd-resolved service to test using static
   /etc/resolv.conf instead.
 - Updted up.sh to install socat package only if it is
   defined for the specific role.

Change-Id: Ibbc874aec2585a32694e7b843f4c790d38bbb3dd
2019-12-04 02:54:12 -06:00
Doug Aaser 7e7d334794 Update coredns and etcd in containerd gate
This patchset updates coredns, calico, and etcd in the containerd gate
to bring it up to date with the promenade genesis gate
It also adds the abilty to set a proxy in the containerd daemon

Change-Id: I581b27206512a4b6d8ec3a3d4212946ca3265308
2019-12-02 16:53:49 +00:00
Egorov, Stanislav 66cb4d2367 containerd support
Introduced new name for the field to define package that has files
which will be used as runtime for UCP containers.

Prepared set of yaml files as an example of containerd usage.

Prepared zuul job to use containerd in simple deployment.

Change-Id: Ifc82a505d064c4f13efccfd92ffc336a510220bf
2019-11-20 16:31:30 -08:00
Sean Eagan 0b0edfd599 Dynamic kubelet config support
This adds support for dynamic kubelet config [0]. An
actual implementation of dynamically updating the kubelet
config will be added at a later time to take advantage of this.

[0]: https://kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/

Change-Id: I2c136cb5b69c9c51086d7c8d8fd6608008998323
2019-11-01 14:14:42 -05:00
Doug Aaser fd1ff8444d Migrate config to KubeletConfiguration
This patchset changes the way that kubelet receives it configuration
parameters so that we can enable [dynamic kubelet configuration][1] down
the line. Starting in Kubernetes v1.11 the configuration of some
parameters has been moved from command line arguments to a static
[configuration file][2].

[1] https://kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/
[2] https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/

Change-Id: Id406ae81fcf44ed0319513e5befc37fd4cff30e5
2019-10-30 18:22:20 +00:00
Kumar, Nishant (nk613n) b49805ae82 Chart changes to support k8s v1.16
This PS includes changes to support k8s 1.16, these
changes would work with existing kubernetes version
as well. A seperate change would be done to uplift
kubernetes to 1.16.

Hyperkube short aliases are removed in k8s 1.15
https://github.com/kubernetes/kubernetes/pull/76953

- Rename binaries of kubernetes components in promenade and
corresponding anchor helm charts
- Kubelet flag --allow-priveleged is deprecated in k8s 1.15 and
removed in 1.16. Remove the flag from kubelet template. This
fix will be backward compatible as long as psp are defined.

Change-Id: I751dd7c0281b0c00ac8f283c1df379e932fe4658
2019-10-25 13:59:22 +00:00
Scott Hussey b037267203 (promenade) Support genesis on 18.04
- Cleanup the genesis.sh template to support Ubuntu Bionic

Change-Id: I33d4731bbadfc3ec54b43606184a40d7597e9831
2019-10-01 03:28:28 -05:00
Sean Eagan e5c65f8cd9 Add configuration for tiller storage
This adds a parameter to the genesis schema
to configure the tiller storage [0] type. For backward
compatibility, by default the parameter is not passed
to tiller, thus relying on the upstream default, which
is 'configmap'.

[0]: https://helm.sh/docs/using_helm/#tiller-s-release-information

Change-Id: I045f8b57f695385b1a502a8f13f61a58d400784e
2019-09-23 14:43:45 -05:00