This PS updates python modules and code to match Airflow 2.6.2:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
airflow-2.6.2 constraints
Change-Id: I9c3e139b3437414a61af7e7c0b7d7e533fadefda
upgrades kubernetes client to v1.26.0
remove installation of containerd during genesis.sh to prevent containerd downgrade
update bitnami kubectl image to image with curl installed for readiness check
Change-Id: I3afd5a7e7211bae3f52263167a62a012da0619a0
Address changes and deprecations in Kubernetes v1.21=>v1.23
controller-manager:
* --authorization-kubeconfig and --authentication-kubeconfig must be set
* liveness/readiness probes must use HTTPS
* the default port has been changed to 10257
kubelet:
* --dynamic-config-dir has been deprecated, will not move to GA
* --cni-bin-dir has been deprecated, will be removed with dockershim
* --cni-conf-dir has been deprecated, will be removed with dockershim
* --network-plugin has been deprecated, will be removed with dockershim
https: //github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md#deprecation
https: //kubernetes.io/docs/tasks/administer-cluster/reconfigure-kubelet/
https: //github.com/kubernetes/enhancements/tree/master/keps/sig-node/281-dynamic-kubelet-configuration
Change-Id: Ia996d7c14d81d1d8b8067f11c02ffb4ce90eb49a
Update the anchor pods to use a regularly patched and updated kubectl
image that contains the necessary components (bash, jq, curl, etc.) in
addition to kubectl: https://hub.docker.com/r/bitnami/kubectl
Change-Id: Ia3e75dc334c3c1a88abfec10fb0367447e79a538
* Give kube-proxy a blanket toleration
* Replace scheduler.alpha.kubernetes.io/critical-pod annotation with
priorityClassName: system-node-critical
Change-Id: I810333913c09531eefa1ded014fe090d4cca7f7d
Replace all usages of the hyperkube image with standalone container
images for apiserver, controller, scheduler, and proxy.
Change-Id: I44392c7900a72edd35bc5afa1c50bec8e04f927f
gcr.io/google_containers/ no longer contains some of the image
versions we require, use the new location.
Change-Id: I8f9a976a35ca632d785dd4d05f2a55713bde8c3e
For any host mounts that include /var/lib/kubelet, use HostToContainer
mountPropagation, which avoids creating extra references to mounts in
other containers.
Affects the following resources:
* haproxy-anchor daemonset
* kubernetes-apiserver-anchor daemonset
* kubernetes-controller-manager-anchor daemonset
* kubernetes-scheduler-anchor daemonset
Change-Id: Ib7fb018c4c1916d00311a73f64f77a99b682d4c8
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0
Change-Id: Ifd2d7af1f2dabe9bbccd65551e0223dddff529dc
This ps makes following changes to upgrade kubernetes from v1.17.3
to v1.18.6.
- Updated all references to k8s images to 1.18.6
- Updated command options and api object and versions based on
k8s 1.18 release notes:
https://kubernetes.io/docs/setup/release/notes/
- Uplifted uwsgi to 2.0.19.1 to align with other airship
components, and to bring in fixes and improvements.
- Added build-essentials and python3-dev packages to pass the zull
gate, which was looking for a c compiler.
Change-Id: I1160d1e6e2f02a0524043641b9296ea39edb301e
This changes adds security context template at pod level to
set run as user value
This also adds security context template at container level to
set readOnly-fs flag
Change-Id: Iba720e687218987cfefe7a9f08630fb11e8eac12
This updates the coredns, haproxy and etcd chart to include the pod
security context on the pod template.
This also adds the container security context to set
readOnlyRootFilesystem flag
Change-Id: I9b5b0ea83acd4c5656577d8cbc684a5031ca0111
The pattern:
create, write, close, rename
is *not* robust in many circumstances. The data blocks from the write
are not always flushed/persisted before the rename (metadata) changes
meaning you can end up where the replacement file is corrupted
(usually has 0 bytes at the end).
Change-Id: Icdd2bb6f20330e5e94b3081f0d0b8a74417f60d4
The resource requests/limits were missing for the HAProxy main
container, although they are there for the init container. This patchset
adds the resource clause to the main container.
Change-Id: I0441dddfbee86da7a4fa6311f6b5e4eb274601bc
This removes an echo from the haproxy anchor, and renders
the related IDENTIFIER variable value directly inline instead.
The echo approach fails under some condition related to host
reboots, resulting in faulty input data for the script and
bad output haproxy configs, with e.g. "frontend -fe".
Change-Id: Id4e258b04290a8ce96b8b518a9c541ecedeee39e
This adds "set -u" (in addition to the existing -x) to the anchor
scripts. This should fix an issue seen occasionally in the haproxy
chart which is only explainable by the IDENTIFIER variable failing
to get set correctly.
All variables used in the anchor scripts ought to be defined, and
there's no need to rely on blank strings as defaults.
"set -e" was considered for this, but may have unintended side-effects:
-u should be safe and avoid the issue we've seen.
Change-Id: Idbc2f9f77d4754874999d5d83d322a17076c7392
This adds -e to the pre_stop scripts, so that they fail out if
any of their commands fail. This is required, since it's the only
way to communicate whether there is an issue during pre_hook
execution.
"The logs for a Hook handler are not exposed in Pod events.
If a handler fails for some reason, it broadcasts an event."
https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks
As an example, this issue was discovered when "touch /tmp/stop"
was failing silently due to a readOnlyRootFilesystem setting,
resulting in pods that would not successfully Terminate until
the grace period was exhausted.
Change-Id: Ic9a228230d944530e31ed61f4239fd434cbb6187
The pre-stop script tries to touch /tmp/stop,
however because of a rofs, fails with permission denied
resulting in the anchor pod getting stuck in Terminating.
This PS adds the mount path /tmp to the anchor container to
resolve the issue.
Change-Id: I3380e4a62b20ae8fdc5da1a72e6794e7cc357218
- Add validation in the anchor that backend IP addresses
and ports sourced from Kubernetes are valid looking strings.
Change-Id: I7539b633dc49efd7262a22c6f9ff040880d9724f
- Some reported cases that the haproxy config was corrupted during
node reboots. Attempt to add additional safeguards of coordination
between the anchor and the service pod.
- Support nulling out a default entry in the service list
- Add additional log statements in the anchor
Change-Id: Ie673c50e1037d5dff2b9f67b14032e188183a5d9
To be able to run with the nobody user, an init container
is used in the haproxy-anchor pod to change the ownership and
permissions of '/host/etc/promenade/haproxy'. Security conext
was included in 'etc/kubernetes/manifests/haproxy.yaml' and
'promenade/schemas/Genesis.yaml' schema was updated to included
run_as_user property for haproxy pod.
Change-Id: Id248face0be43c417284ceb781997634a9c4dd5e
- The anchor pod for haproxy writes to the host
disk and in order to manage file permissions
should run as root. Without this fix, the
haproxy chart is not resilient to node failure.
Change-Id: I9ea9b9a1a2a760be2b3ebb38bd45ead8aaefa034
This updates k8s chart to include the podsecurity context
on the pod template
This also adds the container security context to set
readOnlyRootFilesystem to true
Change-Id: Ic823232fbbb3b0967047d88de81f6a2ee83dcd3e
- When the anchor provides a new haproxy config file
to the running haproxy, add a reasonable check that
the new config is valid:
- Is it a valid config file per haproxy
- Does it contain the expected number of frontends
- Update helm version for linting to 2.14.1
Change-Id: I7a49deb372831c44f05c7baa870735c515519cb2
This version fixes manifest validation [0], so a couple invalid
manifests are fixed in this patchset as well.
[0]: 32d7f1a3fc
Change-Id: I0cbdf21cf016271bef2d8a541687ce3ab28081ce
Sergiy Markin