Since after v3.5.6 etcd-io switched to a
distroless base image. Etcd anchor pods
are now using etcd-utility and etcd is
running a sidecar for health checks.
Change-Id: I198dca1209097de4d60a53a7568f0c4790679599
The extraction of the monolithic hyperkube binary from its container
image to be used as kubelet was last relevant in Kubernetes 1.16. Since
then, the hyperkube image has been deprecated, the structure of the
image has been changed, and it has ultimately been eliminated in
Kubernetes 1.19.
This change cleans up promenade accordingly.
Reverts the following commits:
* 886007b New CLI option to extract hyperkube
* 32a6c15 hyperkube image in promenade init
* 955deed New source for hyperkube binary definition
Change-Id: Ib62ecdf1af13abe8202a4ba4f86c39b9042ed13f
This change allows the HostSystem and Genesis document to specify direct
URLs to files (for example, kubelet and kubectl) that are to be written
to the deployed hosts.
Change-Id: I1828d4a9e654537448631434b26b5becc4d2d717
Some YAML parsers (e.g. libyaml) don't like : without a trailing space.
This adds whitespace to improve parser compatibility and readability.
Change-Id: I62230ab3caef4963b2b63a264396e7057530fd3f
When the genesis and join package source definitions were split, the
.common, .genesis, and .join subkeys were inserted directly under
.properties.packages instead of .properties.packages.properties),
causing anything under packages to erroneously pass.
This change implements the intended validation, allowing packages to be
defined under either under .packages.common, or .packages.genesis and
.packages.join. The expectation is that the genesis node will end up
with the union of what is defined under genesis and common. Required
packages (a runtime and socat) need to be defined in at least one of
those locations. Similarly, join nodes will have the union of join
packages and common packages.
Change-Id: I4a658eef6efbba53ba04b2d8b4ea4711ca0b1ab0
Added ntp server configuration to be used by chrony as the recommended
ntp service for ubuntu 18.04.
Since, chorny is not installed by default in ubuntu 18.04, also added
chrony apt package to be installed on airship nodes.
Change-Id: If1437a79cf89806043f62e2eac49c3b4b5eae2cd
Introduced new name for the field to define package that has files
which will be used as runtime for UCP containers.
Prepared set of yaml files as an example of containerd usage.
Prepared zuul job to use containerd in simple deployment.
Change-Id: Ifc82a505d064c4f13efccfd92ffc336a510220bf
This adds a parameter to the genesis schema
to configure the tiller storage [0] type. For backward
compatibility, by default the parameter is not passed
to tiller, thus relying on the upstream default, which
is 'configmap'.
[0]: https://helm.sh/docs/using_helm/#tiller-s-release-information
Change-Id: I045f8b57f695385b1a502a8f13f61a58d400784e
To be able to run with the nobody user, an init container
is used in the haproxy-anchor pod to change the ownership and
permissions of '/host/etc/promenade/haproxy'. Security conext
was included in 'etc/kubernetes/manifests/haproxy.yaml' and
'promenade/schemas/Genesis.yaml' schema was updated to included
run_as_user property for haproxy pod.
Change-Id: Id248face0be43c417284ceb781997634a9c4dd5e
This change has been tested by the promenade resiliency gate.
This adds configuration for which ports to use for the tiller container
in the bootstrap-armada pod, and changes the defaults to be outside of
`net.ipv4.ip_local_port_range`, since the apiserver container in this pod
dynamically selects ports in that range to connect to etcd, which can
cause conflicts. See [0] for an example.
By default, since we're no longer using the standard tiller ports, this
does mean that we cannot connect to this tiller instance (before it's
replaced by the chart-based instance) via the helm CLI, until it supports
overriding the tiller port to connect to, however this should be
relatively soon [1].
[0]: https://github.com/helm/helm/issues/4886
[1]: https://github.com/helm/helm/pull/5590
Change-Id: Ief11411f079db27489e6974c028f6b7a16bb67bf
- Currently the auxiliary etcd instances remove themselves
after a single non-genesis member joins the cluster. This
leaves the cluster susceptible to non-recoverable disruption
until a 3rd member joins. This change makes the auxiliary control
script wait for a configurable number of non-auxiliary members to
join before removing the auxiliary members.
Change-Id: Ib4968b533e8433e3c40a845d086c7078e807c3e2
Now it's possible to use hyperkube Docker image to extract hyperkube binary.
Use case for this feature is kubelet/kubectl delivery in one binary(hyperkube)
which is built into Docker image. Promenade will extract hyperkube from Docker image,
create symlinks for kubelet/kubectl pointed to hyperkube. To do so promenade container
need to be configured to use Docker on the host where this container will be created.
This is happening only for script generation for genesis node. Later when promenade
will be started as a service pod inside ucp cluster it will generate scripts for joining nodes
by using cached hyperkube from /tmp.
Old way to delivery kubelet from tarball is still supported.
Configuration for the new method.
Need to export environment variables to properly configure Docker in Docker.
Docker socket should be provided as a mounted file inside promenade.
Also need to set temporary permissions for this socket during the build scripts stage.
Example:
DOCKER_SOCK="/var/run/docker.sock"
sudo chmod o+rw $DOCKER_SOCK
export DOCKER_HOST="unix:/${DOCKER_SOCK}"
export PROMENADE_TMP="abs_path_tmp_dir_on_host"
export PROMENADE_TMP_LOCAL="tmp_dir_inside_container"
After genesis scripts generation Docker socket permission should be turned back:
sudo chmod o-rw $DOCKER_SOCK
Change-Id: Ida22ea934fc551fec34df162d8147c8b9e630330
Currently, the package, repository, and key lists are used by up.sh for
genesis and join. This is not desirable when using an in-cluster
mirroring service, as the service address may change after it has been
deployed.
This commit separates the sources for genesis and join to circumvent the
aforementioned pain point. A 'common' entry in the
'promenade/HostSystem/v1' document can be used if a common source for
genesis and join is desired.
Co-authored-by: Rick Bartra <rb560u@att.com>
Change-Id: Ieb2513da0cff587297cfcbf5629d908696349621
Adds an optional external_ip parameter to the prom join script API,
and to the Genesis and KubernetesNode schema.
This is used to populate the host's IP address in its /etc/hosts
file if present, according to normal hosts conventions.
If the value is not passed to prom-join or is absent from a
Genesis or KubernetesNode document, then the hosts file defaults
to the current loopback IP for the hostname (business as usual).
Change-Id: I58dc219923b18aaf9c83453b896ce509664d8766
This change accomplishes 2 primary things:
1. It generalizes work to enable the EventRateLimit admission plugin.
2. It restructures the anchor so that during an upgrade an "old" anchor
does not try to coordinate the injection of "new" data from
configmaps/secrets.
It also includes these ancillary changes:
* Clean up apiserver argument specification in the chart.
* De-duplicate and realign apiserver arguments in bootstrapping templates.
It has the side effects of:
* Adding a new field, ".apiserver.arguments" to the Genesis config,
which will be the preferred way to configure bootstrapping apiservers
going forward (in lieu of command_prefix).
Change-Id: I33cfe80ee8e29cd79e479a7985e3c098a2288fda
This introduces a new document called `EncryptionPolicy` to configure
this behavior. It currently only supports using symmetric encryption
with `GPG`, but that should be available on all Ubuntu systems (which is
what we currently support) and should also be fairly reliable.
Change-Id: I06d4faa119b736773df0d8cbf0e7a23fd98edcdf
Depends-On: https://review.openstack.org/#/c/602175/
- During the genesis or join operation when /etc/hosts and
/etc/resolv.conf are controlled by Promeande, we need to
support including a domain name. This can be configured
by YAML definition or by the join-script API. To support
backward compatability use a default of 'local' when no
domain is specified.
Testing: `./tools/gate.sh resiliency` has passed locally
Change-Id: Ia0d300912d3ec25eb7f1cb9c580eaa40b5b4addb
This provides more robustness in testing and removes a nosec.
Additionally, commit 5a8b1d8 introduced a random failure in the
resiliency gate, due to there being a chance to choose the intentionally
downed node for the join ip.
Change-Id: I77b410b8e51f9d41eca2be4f5f770694140733b4
The image used to test kubectl logs in the validation scripts was hard
coded and is now configurable.
This also makes the power-up-node.sh gate script more robust by making
it wait for the node to be ready.
Change-Id: I531ca8477ac3575dd4249ab5e991881af290fa52
This avoids possible issues when the configuration of the bootstrapping
apiserver differs from the chart's configuration. Issues were
specifically seen when overriding the node port range, but this opens up
additional configuration also.
Change-Id: I2a3fc5847e850c8055c099bac50782debbbabbf4
- Remove apt conf for proxy, rely on environment
- Update schema to correctly support no_proxy
- Update build-image stage to support a proxy
Change-Id: Ie07a72ad35fde57596af88f838c8c1836b1e8510
This removes the reliance on coredns for APIserver discovery, allowing
a simpler configuration that is compatible with corednx 1.0.x
Change-Id: Ia3b7b5627c16ec47af6b0d6d5e8dee2674e9b1ee
* Freeze busybox version for log validation.
* Move static labels to dynamic in remaining places.
* Stop using node-role.kubernetes.io/master= label.
* Update older coredns image usage.
* Add content to plaeholder file to avoid warning.
* Add external DNS check to DNS validation check.
Change-Id: I9d0665a940ab055e6426aeca9c8e2be269e6b13a
This change includes several interconnected features:
* Migration to Deckhand-based configuration. This is integrated here,
because new configuration data were needed, so it would have been
wasted effort to either implement it in the old format or to update
the old configuration data to Dechkand format.
* Failing faster with stronger validation. Migration to Deckhand
configuration was a good opportunity to add schema validation, which
is a requirement in the near term anyway. Additionally, rendering
all templates up front adds an additional layer of "fail-fast".
* Separation of certificate generation and configuration assembly into
different commands. Combined with Deckhand substitution, this creates
a much clearer distinction between Promenade configuration and
deployable secrets.
* Migration of components to charts. This is a key step that will
enable support for dynamic node management. Additionally, this paves
the way for significant configurability in component deployment.
* Version of kubelet is configurable & controlled via download url.
* Restructuring templates to be more intuitive. Many of the templates
require changes or deletion due to the migration to charts.
* Installation of pre-configured useful tools on hosts, including calicoctl.
* DNS is now provided by coredns, which is highly configurable.
Change-Id: I9f2d8da6346f4308be5083a54764ce6035a2e10c
Ruslan Aliev