To avoid cycling the pods in the anchor daemonset too quickly, only
consider a kubernetes-apiserver-anchor pod ready if:
- it created the static manifest kubernetes-apiserver.yaml
- the kubernetes-apiserver pod on the same host is ready
Change-Id: I53dd1c044332946eeb965f07ae828910f00b04c6
The existing apiserver chart supports volume overrides for the anchor
daemonset, but not for the apiserver static pod itself. The feature to
allow volume overrides in the apiserver-webhook chart was never fully
implemented.
This changes allows volume overrides via values.yaml for both charts,
and provides a more complete audit example that includes mounting the
audit log destination as a host path volume.
Change-Id: I27ccf77671a190e8cb6b66d8a9b13c2cde6c9a45
There are several kubernetes bugs [0,1,2] involving connection problems
that seem related to the Go net/http2 library, where the stream state
and connection state can get out of sync. This can manifest as a kubelet
issue, where the node status gets stuck in a NotReady state, but can
also happen elsewhere.
In newer versions of the Go libraries some issues are fixed [3,4], but
the fixes are not present in k8s 1.18.
This change disables http2 in kube-apiserver and webhook-apiserver. This
should be sufficient to avoid the majority of the issues, as disabling
on one side of the connection is enough, and apiserver is generally
either the client or the server.
0: https://github.com/kubernetes/kubernetes/issues/87615
1: https://github.com/kubernetes/kubernetes/issues/80313
2: https://github.com/kubernetes/client-go/issues/374
3: https://github.com/golang/go/issues/40423
4: https://github.com/golang/go/issues/40201
Change-Id: Id693a7201acffccbc4b3db8f4e4b96290fd50288
The existing exec probes for apiserver rely on things that do not exist
in the official kubernetes release images (bash, socat).
This change modifies the apiserver to use HTTP probes of the recommended
liveness and readiness endpoints.[0]
Also sets `--anonymous-auth=true` (the default setting), as kubelet is
unable to provide a client certificate when performing the health check.
RBAC rules apply, but unauthenticated users will be able to access the
following endpoints:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: system:public-info-viewer
rules:
- nonResourceURLs:
- /healthz
- /livez
- /readyz
- /version
- /version/
verbs:
- get
0: https://v1-18.docs.kubernetes.io/docs/reference/using-api/health-checks/
Change-Id: I06d739c844fe85ec6cbf47d3bb69a39cd008ddd8
Uses the standard helm-toolkit macros for liveness and readiness probes,
allowing them to be enabled or disabled, and params to be overridden.
Change-Id: Ie9aef97f56f2205ada24f17e7cafabc5943ae097
This changes adds security context template at pod level to
set run as user value
This also adds security context template at container level to
set readOnly-fs flag
Change-Id: Iba720e687218987cfefe7a9f08630fb11e8eac12
This PS ensures there is a newline present between the cert and its
key when concatenating them together.
Change-Id: I72319c1a415d683f19ff8f96060eb39bbec34b75
Signed-off-by: Pete Birley <pete@port.direct>
- Support key rotation for the etcd encryption key in the
apiserver chart
- Remove configmap annotations from the apiserver anchor pods
as the pod is built to pickup changes in configmap contents
without restart.
- Also update the apiserver anchor DaemonSet to apps/v1 and
make required updates to support that update.
Change-Id: I2d18996bbe04bada9da2bce01a502550d3681c97
If we enable these tls settings, then having quotes around the values
will prevent apiserver from starting.
Change-Id: I39d1e5861262074ef0c50f22d0fae47d822f8319
This change accomplishes 2 primary things:
1. It generalizes work to enable the EventRateLimit admission plugin.
2. It restructures the anchor so that during an upgrade an "old" anchor
does not try to coordinate the injection of "new" data from
configmaps/secrets.
It also includes these ancillary changes:
* Clean up apiserver argument specification in the chart.
* De-duplicate and realign apiserver arguments in bootstrapping templates.
It has the side effects of:
* Adding a new field, ".apiserver.arguments" to the Genesis config,
which will be the preferred way to configure bootstrapping apiservers
going forward (in lieu of command_prefix).
Change-Id: I33cfe80ee8e29cd79e479a7985e3c098a2288fda
This avoids leaving zombies in cases where the processes don't reap
children.
Also fixes a certificate issue with the resiliency gate.
Change-Id: I8a795557b0d60338c40b360c947b81a20fd48877
Add the EventRateLimit admission controller, to allow operators to
define rate limits for the k8s API server at the server, namespace,
or user account level.
This also
* cleans up some of the parameters passed into the API server
* replaces the deprecated --admission-control parameter
* applies --repair-malformed-updates consistently, incl examples
* removes unused batch/v2alpha1 runtime config
* https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/
* removes duplicate --service-cluster-ip-range setting
This PS adds EventRateLimits to the bootstrap and anchor API
servers; future work will need to add it to the Keystone
Webhook API server.
Change-Id: I32a2d4add880e50f470e4cb0687e20d16e6e926d
This PS adds the ability to attach a release uuid to pods and rc
objects as desired. This can be used, for example, to force an
artificial manifest change in CICD scenarios, for upgradability
testing purposes.
Change-Id: I8d0ffac306258f940c63799e86e7e26b5c2c5add
This increases isolation of actions against the node API. With the
previous combined CA approach, each node would have a valid key to talk
to each other node. With this separated approach, only the API servers
will have keys with access to the node APIs.
Change-Id: I2705016eb963ca9d2cc2a344047677f4b2cc3025
- Updated apiserver-anchor with a liveness probe.
- Changed apiserver liveness probe to query kubectl.
This allows the pod to restart if it looses access to etcd.
Change-Id: I0ef9cbc941a0533268e4f499a1333e88be3e43a3
- Turn off anonymous-auth.
- Reworked haproxy helm test and updated test images.
- Reworked kubernetes-apiserver readiness and liveness tests.
Change-Id: Ifb39ebed0f9f6e430e97247fceebbd7816f092c7
This adds stability to etcd and enables cleaner waiting by tiller during
deployment of the Kubernetes apiserver and etcd.
* Adds second auxiliary etcd process.
* Enables "sequenced" for remaining ChartGroups.
* Removes unused disks from test VMs.
* Add readiness and liveness probes for kubernetes components
Change-Id: I6f83bb912f76b0ec35503723b417ba45d69e39c5
Phil Sphicas