This PS updates python modules and code to match Airflow 2.6.2:
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfoemed based on
airflow-2.6.2 constraints
- airskiff deploy pipeline was aligned with latest in treasuremap v1.9
Change-Id: If6f57325339995216d2553c7a5ff56e7673b5acc
- armada-airskiff-deploy is voting gate again
- fixed falcon.API deprecation - -> falcon.App
- fixed collections.abc.defaultdict not found error
- fixed tox4 requirements
- implemented requirements-frozen.txt approach to make allike as other
Airship projects
- uplifted docker version in the image building and publishing gate
Change-Id: I337ec07cd6d082acabd9ad65dd9eefb728a43b12
Bumping k8s client to v25.3.0
Cronjob batch v1beta1 no longer available in k8s 1.25
Update tox.ini file to be compatible with v4
Change-Id: Iac79c52c97c9ef1223ae8d502da1572ef8d068fa
Kubernetes v1.22 stopped serving the apiextensions.k8s.io/v1beta1 API
version of CustomResourceDefinition.
This change ensures that the locks.armada.process CRD is created using
the apiextensions.k8s.io/v1 API.
The kubernetes client package is also updated to take advantage of the
dynamic client.
Change-Id: Icd518ab5cbb78e8b15f63d19c51b5f5b9a67e995
Previously, existing dependency symlinks were causing
FileExistsErrors. This simplifies the logic to ensure
they do get deleted before being recreated. It also
moves to erroring if a chart's `charts` directory is
not a directory, since this is something a chart author
ought to fix, since it needs to be a directory in order
to add dependency sub-directories to it.
Change-Id: I7cc1d50f2bf09602866ce2cf13a4d6c8f6e118ce
For now we leave the tiller status enpdpoint, until
Shipyard has had a release to stop depending on it [0].
[0]: https://review.opendev.org/c/airship/shipyard/+/802718
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: If8a02d7118f6840fdbbe088b4086aee9a18ababb
Helm 3 breaking changes (likely non-exhaustive):
- crd-install hook removed and replaced with crds directory in
chart where all CRDs defined in it will be installed before
any rendering of the chart
- test-failure hook annotation value removed, and test-success
deprecated. Use test instead
- `--force` no longer handles recreating resources which
cannot be updated due to e.g. immutability [0]
- `--recreate-pods` removed, use declarative approach instead [1]
[0]: https://github.com/helm/helm/issues/7082
[1]: https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I20ff40ba55197de3d37e5fd647e7d2524a53248f
This removes release rollback/delete functionality. This functionality
was likely not being used and thus was likely not working.
This primary driver for this change is to ease introduction of Helm 3
support. Particularly to avoid having to make API changes related to
the namespacing of helm releases in Helm 3.
This also removes the swagger api documentation as it was not
maintained.
Change-Id: I7edb1c449d43690c87e5bb24726a9fcaf428c00b
This is a pre-requisite for Helm 3 integration, so that these
actions run regardless of whether we are going through the
tiller handler.
Change-Id: I97d7bcc823d11b527fcdaa7967fcab62af1c8161
This reverts commit c75898cd6a.
Airship 2 ended up using the Flux helm-controller instead:
https://github.com/fluxcd/helm-controller
So this is no longer needed. Removing it to get rid of tech
debt to ease introduction of Helm 3 support.
This retains the part of the commit which extracts the
chart download logic to its own handler as this is still useful.
Change-Id: Icb468be2d4916620fd78df250fd038ab58840182
This PS:
1) Looks to improve specific logging in Armada, so that
it's easier to debug deployment related issues
2) Uplifts the k8s Python dependency to 12.0.0
3) Enforces 'watch' timeouts more strictly, as the call to
the Kubernetes Python watch function seemed unreliable.
4) Adds a field selector to the 'watch' stream to look for
the DELETE action to have been completed on the specific
pod/job/cronjob, rather than looking across the whole
namespace or via labels. This will narrow what the watch
is looking at, making the logs less busy.
Change-Id: I1952b0db32fb0b56ffffcddeae0532beb5a27b67
This adds the release name to the beginning of the diff debug
log so that it's easier to correlate which diffs belong to which
components when reading the Armada logs.
Change-Id: If096f049a8057a46951ea70f2223a7a987636f5c
In the occasion of a pod being evicted due to low resource availability,
armada keeps waiting for the Evicted pod to be ready. This commit
removes that behavior since Kubernetes will spun up a new pod.
Story: 2008645
Task: 41906
Signed-off-by: Thiago Brito <thiago.brito@windriver.com>
Change-Id: I7263eebe357b0952375d538555536dc9f7cceff4
Update Helm chart for Armada to use Tiller version 2.16.9.
depends on: https://review.opendev.org/#/c/749497/
Change-Id: I16f7a5e8e571f067154e79a5f2ceb18be7d8db2d
This commit will fix below warning message.
"Post upgrade actions are ignored by Armadaand will not affect deployment.:
Change-Id: I1a8397aa892815bdc96f94a8e49280a4b73a484c
In cases when chart is in a pending state and last deployment age
for chart is less than the chart wait timeout, armada can consume
all the retries without attempting to purge and re-install the
chart.
This PS addresses the above issue by waiting for the chart until
the wait timeout is met and then proceeding ahead with usual
armada process of purging and re-installing the chart if the status
is not DEPLOYED.
Other code changes in this PS is just re-arranging the existing logic to
fix some edge cases and solve pep8 issues.
Change-Id: Id11d0fb06201264015b3064a43e83903f99a30b1
Flake8 version recently updated to include new PEP8 rules. Some of
the codebase is not compliant with the new rules.
Change-Id: Ieb9de200535b6e3fd743e1c0f2e851e81cae79f3
Armada uses a Kubernetes watch to implement its chart wait logic. This
can be a fairly long-lived connection to the Kubernetes API server, and
is vulnerable to disruption (if, for example, the kubernetes apiserver
chart is being upgraded).
This change allows Armada to retry the wait for some specific errors,
including the establishment of a new watch, until the overall chart
timeout is reached.
https://github.com/kubernetes-client/python/issues/972
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))
Change-Id: I3e68a54becadd5b2a2343960a120bdc3de8e8515
The mechanism to disable waits is to explictly set
wait.resources: []
After a recent update [https://review.opendev.org/#/c/697728/], this
configuration results in the default waits (pods and jobs).
This change restores the original functionality.
Change-Id: If5d09f649ca037855c88f928aee6b4dc35ca8b48
Airship 2 is using Argo for workflow management, rather
than the builtin Armada workflow functionality. Hence, this
adds an apply_chart CLI command to apply a single chart at
a time, so that Argo can manage the higher level orchestration.
Airship 2 is also using kubernetes as opposed to Deckhand as the
document store. Hence this adds an ArmadaChart kubernetes CRD,
which can be consumed by the apply_chart CLI command. The chart
`dependencies` feature is intentionally not supported by the CRD,
as there are additional complexities to make that work, and ideally
this feature should be deprecated as charts should be building in
there dependencies before consumption by Armada.
Functional tests are included to excercise these features
against a minikube cluster.
Change-Id: I2bbed83d6d80091322a7e60b918a534188467239
This patch set adds in a missing space in a log message so it can be more
easily parsed, as the previous fields were separated by space.
Change-Id: I96cceb644c8193909a91fb42e13c43db0f83ba8d
Signed-off-by: Tin Lam <tin@irrational.io>
Armada intends to use `propagationPolicy: Foreground` when deleting
resources. However, the empty V1DeleteOptions object in the body of the
delete API call takes priority over the propogation_policy specified as
a query param, resulting in the per-resouce default. For Job and CronJob
resources, the default is Orphan, and for others it is Background.
This change includes the desired propogation_policy in V1DeleteOptions.
Reference: https://kubernetes.io/docs/concepts/workloads/controllers/garbage-collection/#setting-the-cascading-deletion-policy
Change-Id: Iffee12b426ba1e7741eb5bd687ca1b2c11cb071d
Armada has previously named template files relative to the
`templates` dir, whereas the Helm CLI names them relative
to the chart root. This causes `include`s of these templates
to fail.
This change fixes this, for armada/Chart/v2 docs only, since it
is a breaking change, as some charts may have already aligned
with the existing Armada behavior. When updating a release
previously deployed with armada/Chart/v1, the fixed template
names alone will not cause the release to be updated, as the
diff logic accounts for this.
Change-Id: I243073ca4c2e1edbcb0d8f649475f568fc7c818f
Currently the chart `source` schema allows for a proxy server to be
specified, but it is only used for git repos. This patchset allows
the `proxy_server` to also be used for tarball url sources.
Change-Id: I6f90d056fa46f596b1fb248b6c596c58b6513d64
Armada's dry-run option is incomplete, no longer maintained, and offers
little value for the complexity required to maintain it.
This commit is the final in a series of changes to remove the dry-run
feature. Specifically, this change removes the functionality associated
with the dry-run feature.
Story: 2005121
Change-Id: I7dfe5ab27511debe2b8ac01f8e0a696c6126a9f7
Signed-off-by: Drew Walters <andrew.walters@att.com>
This implements Prometheus metric integration, including metric
definition, collection, and exportation.
End user documentation for supported metric data and exportation
interface is included.
Change-Id: Ia0837f28073d6cd8e0220ac84cdd261b32704ae4
The current logic to find valid releases is incorrect resulting in zero
valid releases being found and all currently installed chart releases
with the release_prefix being removed.
The data passed to this method is a list of chart groups with the
following format:
[{'data':
{'chart_group':
[{'data':
{'chart_name': 'ingress',
'release': 'kube-system-ingress'
...
This update adapts the code to the expected format of the data provided.
Change-Id: Ib285cdf21283b8ec73d583fb107ba49514a9dda7
Signed-off-by: Robert Church <robert.church@windriver.com>
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase. This is the effort
to do so for the Armada project.
[0] https://review.opendev.org/#/c/671291/
Change-Id: I4fe916d6e330618ea3a1fccfa4bdfdfabb9ffcb2
Kubernetes requests to create locks in the lock_and_thread decorator
were being done without the bearer token. This creates authentication
errors when communicating with a Kubernetes cluster configured with
an external auth backend. This commit uses the bearer token when
creating the Kubernetes client used to create a lock object for
lock_and_thread
The bearer token is obtained from the tiller object passed to the
function being decorated by lock_and_thread. Currently, all functions
decorated by lock_and_thread passes a tiller object to obtain a token
from. If no tiller object is found in the params, this will be logged
to assist with potential debugging in the future. An exception is not
thrown because a Kubernetes cluster without external auth can be used
without passing any bearer tokens around.
Change-Id: Ic9329c2f605cb508750065ebd0c756869be94199
Signed-off-by: Jerry Sun <jerry.sun@windriver.com>
There is a breaking change in helm 2.14.0 [0]. This is expected to be fixed in helm 2.14.1, reverting until we can update to that.
[0]: https://github.com/helm/helm/issues/5750
This reverts commit 89d98fb827.
Change-Id: Ica6d51b5c67a26c356804fd69d466e88ad5c216b
This is just a basic Helm CLI option, so this moves it to be alongside
the rest of the upgrade options.
Change-Id: I4cbb4f3bfe60240d793a30f7a7d58705024f633c
This introduces v2 docs in order to allow users to opt in to
breaking changes, while still supporting v1 docs for a time
so folks can migrate. At some point v1 doc support will be
removed.
This initial version of v2 docs is experimental. Further
breaking changes will be made before v2 docs are finalized.
A v1-v2 migration guide is included in the documentation.
This also refactors the internal data model to include the full
document structure, such as `metadata` and `schema`, so that
different behavior can be acheived for v1, v2, etc.
Change-Id: Ia0d44ff4276ef4c27f78706ab02c88aa421a307f
Under some wait condition, Armada returns a tuple that contains an extraneous
element that causes a python error. This patch set fixes that so the method
always return a 2-tuple instead sometimes a 3-tuple.
Change-Id: I4c4dfcf03e63f03ad2adc083d39909cf4b47a27f
Signed-off-by: Tin Lam <tin@irrational.io>
If a progress deadline is exceeded while waiting on deployments, Armada
returns three values when two are expected, resulting in a ValueError
exception. This change properly formats the return value.
Change-Id: I49e6c2a022b3bb9bf8d6a01cd2ef261f52eaa426
Helm v2.13.1 has been released [0], and is the next version of Helm
Armada is compatible with. Currently, Armada is not compatible with the
latest version of Helm toolkit due to a divergence caused in Helm v2.13.
This change uplifts Helm to v2.13.1 to restore compatibility with the
latest version of Helm toolkit.
[0] https://github.com/helm/helm/releases/tag/v2.13.1
Change-Id: Ieaf2475562c56530b6ec69c6a43611b4b47b7c83
This creates a new mechanism in Armada to enable functions to only be
run once across multiple instances of Armada working with the same
Kubernetes cluster. This is accomplished by utilizing custom resources
via the Kubernetes API.
This also introduces new config defaults that can be used to configure
the lock timeout, expiration, and update interval.
Some notes on how the lock works:
* Functions to be locked can add the new decorator
* The optional name parameter can be used to create multiple
types of locks which can coexist
* If the lock is unable to be acquired before the timeout a new
exception is raised
* The lock is updated regularly while the decorated function is
still running
* If a lock already exists it will only be overwritten if the
duration since its last update is longer than the expiration time
For now this locking method is being used for components that require
write access to Tiller so that simultaneous write operations are
avoided.
Change-Id: Iee07da9a233ee2e2a54c6bc4881185388b377c05
This adds a new configuration default to specify the Tiller host IP.
This is important to be able to configure in environments where
Armada is unable to find a Tiller pod.
Change-Id: I12fd9fbd16f2b591620e566affcf19f859ed1855
Ruslan Aliev