To avoid cycling the pods in the anchor daemonset too quickly, only
consider a kubernetes-apiserver-anchor pod ready if:
- it created the static manifest kubernetes-apiserver.yaml
- the kubernetes-apiserver pod on the same host is ready
Change-Id: I53dd1c044332946eeb965f07ae828910f00b04c6
This change corrects two rendering issues in the kube-apiserver anchor
script. The details and impact are mentioned below.
1. The kube-apiserver anchor script fails to clean up some files from
the host, because the path is incomplete. For example, the cleanup()
function of the script includes:
rm -f "/host/acconfig.yaml"
instead of
rm -f "/host/etc/kubernetes/apiserver/acconfig.yaml"
2. A recent change to allow fileless command options [0] caused some
extraneous lines to end up in the script. For example, the rendered
script includes:
snapshot_files() {
cp "/tmp/etc/" "${SNAPSHOT_DIR}/etc/kubernetes/apiserver/"
}
compare_copy_files() {
SRC="${SNAPSHOT_DIR}/etc/kubernetes/apiserver/"
DEST="/host/etc/kubernetes/apiserver/"
if [ ! -e "${DEST}" ] || ! cmp -s "${SRC}" "${DEST}"; then
mkdir -p $(dirname "${DEST}")
cp "${SRC}" "${DEST}"
chmod go-rwx "${DEST}"
fi
}
cleanup() {
rm -f "/host/"
}
Since the 'cp' and 'rm' commands don't include '-r', this is actually
non-impacting, other than some log messages.
0: https://review.opendev.org/c/airship/promenade/+/788092
Change-Id: Id0a47727d56268d13ebb4718b8578d94272c2181
This adds "set -u" (in addition to the existing -x) to the anchor
scripts. This should fix an issue seen occasionally in the haproxy
chart which is only explainable by the IDENTIFIER variable failing
to get set correctly.
All variables used in the anchor scripts ought to be defined, and
there's no need to rely on blank strings as defaults.
"set -e" was considered for this, but may have unintended side-effects:
-u should be safe and avoid the issue we've seen.
Change-Id: Idbc2f9f77d4754874999d5d83d322a17076c7392
By design, the anchor pods clean up after their static pods
(and associated secrets/configs) via a hook when they the anchor
pods are stopped, to make sure that cruft is not left lying around
(or running) when an anchor pod is no longer scheduled to a host.
However, it's been observed that on a host under high load, e.g.
if one or two other control plane hosts are down, then the anchor
pods may be stopped in an unplanned manner. This results in
service unavailability for the anchored static manifest pods.
This change makes that cleanup behavior configurable (following the
pattern already implemented in the haproxy chart) but leaves it on by
by default.
Change-Id: Iab14510ef8ea5b9e400e0f744231811117029887
This change accomplishes 2 primary things:
1. It generalizes work to enable the EventRateLimit admission plugin.
2. It restructures the anchor so that during an upgrade an "old" anchor
does not try to coordinate the injection of "new" data from
configmaps/secrets.
It also includes these ancillary changes:
* Clean up apiserver argument specification in the chart.
* De-duplicate and realign apiserver arguments in bootstrapping templates.
It has the side effects of:
* Adding a new field, ".apiserver.arguments" to the Genesis config,
which will be the preferred way to configure bootstrapping apiservers
going forward (in lieu of command_prefix).
Change-Id: I33cfe80ee8e29cd79e479a7985e3c098a2288fda
* added in missing recursive flag to the chmod command used to remove
extraneous permissions from CURATED_DIRS
* added commands to change permissions for manifests and configurations
that are copied to the host
Change-Id: I174db09061c3162db11dd976a55132f5fad7a80d
This update to the usage of `cmp` in the anchor scripts will cause the
scripts to overwrite existing manifests whenever differences are
detected.
This should only occur during genesis, but may occur if a file is
accidentily modified on a node and the anchor then checks the file for
changes.
Change-Id: Ic594fef2c5a5c30a5be3ccc7ab51efcd29c8cdcc
This change includes several interconnected features:
* Migration to Deckhand-based configuration. This is integrated here,
because new configuration data were needed, so it would have been
wasted effort to either implement it in the old format or to update
the old configuration data to Dechkand format.
* Failing faster with stronger validation. Migration to Deckhand
configuration was a good opportunity to add schema validation, which
is a requirement in the near term anyway. Additionally, rendering
all templates up front adds an additional layer of "fail-fast".
* Separation of certificate generation and configuration assembly into
different commands. Combined with Deckhand substitution, this creates
a much clearer distinction between Promenade configuration and
deployable secrets.
* Migration of components to charts. This is a key step that will
enable support for dynamic node management. Additionally, this paves
the way for significant configurability in component deployment.
* Version of kubelet is configurable & controlled via download url.
* Restructuring templates to be more intuitive. Many of the templates
require changes or deletion due to the migration to charts.
* Installation of pre-configured useful tools on hosts, including calicoctl.
* DNS is now provided by coredns, which is highly configurable.
Change-Id: I9f2d8da6346f4308be5083a54764ce6035a2e10c
Phil Sphicas