- armada-airskiff-deploy is voting gate again
- fixed falcon.API deprecation - -> falcon.App
- fixed collections.abc.defaultdict not found error
- fixed tox4 requirements
- implemented requirements-frozen.txt approach to make allike as other
Airship projects
- uplifted docker version in the image building and publishing gate
Change-Id: I337ec07cd6d082acabd9ad65dd9eefb728a43b12
For now we leave the tiller status enpdpoint, until
Shipyard has had a release to stop depending on it [0].
[0]: https://review.opendev.org/c/airship/shipyard/+/802718
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: If8a02d7118f6840fdbbe088b4086aee9a18ababb
Helm 3 breaking changes (likely non-exhaustive):
- crd-install hook removed and replaced with crds directory in
chart where all CRDs defined in it will be installed before
any rendering of the chart
- test-failure hook annotation value removed, and test-success
deprecated. Use test instead
- `--force` no longer handles recreating resources which
cannot be updated due to e.g. immutability [0]
- `--recreate-pods` removed, use declarative approach instead [1]
[0]: https://github.com/helm/helm/issues/7082
[1]: https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I20ff40ba55197de3d37e5fd647e7d2524a53248f
This removes release rollback/delete functionality. This functionality
was likely not being used and thus was likely not working.
This primary driver for this change is to ease introduction of Helm 3
support. Particularly to avoid having to make API changes related to
the namespacing of helm releases in Helm 3.
This also removes the swagger api documentation as it was not
maintained.
Change-Id: I7edb1c449d43690c87e5bb24726a9fcaf428c00b
Moving to falcon 3.0.0+ brings in some changes to the response
object. One of those is the deprecation of the "body" response
field. This PS attempts to get ahead and make the necessary
changes to no longer use deprecated fields.
Change-Id: Iac5d8cd2c658c337dfe7937db8124f3107d77e91
Armada's dry-run option is incomplete, no longer maintained, and offers
little value for the complexity required to maintain it.
This commit is the first in a series of changes to remove the dry-run
feature. Specifically, this change removes the parameter as an option
for the API.
Story: 2005121
Change-Id: If5bd2639fe3e9af3f4cc669cd627b47c1d8fec16
Signed-off-by: Drew Walters <andrew.walters@att.com>
The metrics endpoint can be scraped frequently and produces a large
response body, hence this turns off logging of this data to prevent
the drowning out of other more salient logging.
Change-Id: I399480726ae39963b010bf5b054ce26d73fb28c5
This implements Prometheus metric integration, including metric
definition, collection, and exportation.
End user documentation for supported metric data and exportation
interface is included.
Change-Id: Ia0837f28073d6cd8e0220ac84cdd261b32704ae4
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase. This is the effort
to do so for the Armada project.
[0] https://review.opendev.org/#/c/671291/
Change-Id: I4fe916d6e330618ea3a1fccfa4bdfdfabb9ffcb2
This introduces v2 docs in order to allow users to opt in to
breaking changes, while still supporting v1 docs for a time
so folks can migrate. At some point v1 doc support will be
removed.
This initial version of v2 docs is experimental. Further
breaking changes will be made before v2 docs are finalized.
A v1-v2 migration guide is included in the documentation.
This also refactors the internal data model to include the full
document structure, such as `metadata` and `schema`, so that
different behavior can be acheived for v1, v2, etc.
Change-Id: Ia0d44ff4276ef4c27f78706ab02c88aa421a307f
This change creates a Tiller sidecar in the Armada chart and
configures Armada to use this Tiller by default for its operations.
This allows Armada to communicate with this Tiller without exposing it
to the rest of the cluster.
This also removes `tiller_host` and `tiller_port` as API parameters as
they should now just be configured using the configuration file. When
the Tiller sidecar is enabled, configurations will be overridden to
point to it. Otherwise Armada will rely on the Tiller pod lookup.
While this will later enable the Tiller charts to be removed, they
will not be in this change as there is currently no alternative in
Airship to communicate with the cluster using Helm.
Co-Authored-By: Michael Beaver <michaelbeaver64@gmail.com>
Change-Id: Id881e379be580efd60bae400fa402ce238bfd6ef
This creates a new mechanism in Armada to enable functions to only be
run once across multiple instances of Armada working with the same
Kubernetes cluster. This is accomplished by utilizing custom resources
via the Kubernetes API.
This also introduces new config defaults that can be used to configure
the lock timeout, expiration, and update interval.
Some notes on how the lock works:
* Functions to be locked can add the new decorator
* The optional name parameter can be used to create multiple
types of locks which can coexist
* If the lock is unable to be acquired before the timeout a new
exception is raised
* The lock is updated regularly while the decorated function is
still running
* If a lock already exists it will only be overwritten if the
duration since its last update is longer than the expiration time
For now this locking method is being used for components that require
write access to Tiller so that simultaneous write operations are
avoided.
Change-Id: Iee07da9a233ee2e2a54c6bc4881185388b377c05
Due to kubernetes liveness/readiness probes, health check logging
can make the rest of the logs difficult to use.
Change-Id: If063a291243cf5221b40dac47064cd25372dac95
This is a refactor to any methods in the api and cli to move tiller
operations to a separate handler function to make them uniform
with the rest of the code
Change-Id: Iddd45f9f979307f54ad5bd54c3ee005547dd0909
When running helm tests for a chart release multiple times in a site,
if the previous test pod is not deleted, then the test pod creation
can fail due to a name conflict. Armada/helm support immediate test pod
cleanup, but using this means that upon test failure, the test pod logs will
not be available for debugging purposes. Due to this, the recommended approach
for deleting test pods in Armada has been using `upgrade.pre.delete` actions.
So chart authors can accomplish test pod deletion using this
feature, however, it often takes awhile, usually not until they test upgrading
the chart for chart authors to realize that this is necessary and to get it
implemented.
This patchset automates deletion of test pods directly before running tests by
using the `wait.labels` field in the chart doc when they exist to find all pods
in the release and then using their annotations to determine if they are test
pods and deleting them if so.
A later patchset is planned to implement defaulting of the wait labels when
they are not defined.
Change-Id: I2092f448acb88b5ade3b31b397f9c874c0061668
While authoring [0], it was discovered that Armada has duplicate logic
for deciding if Helm test cleanup should be enabled as well as the tests
themselves. Because of this, changes to test logic (e.g. adding pre-test
actions), requires changing all traces of the repeated logic, which can
lead to inconsistent behavior if not properly addressed. This change
moves all test decision logic to a singular Test handler, implemented by
the `Test` class. This change does NOT change the expected behavior of
testing during upgrades; however, tests initiated from the API and CLI
will not execute when testing a manifest if they are disabled in a
chart, unless using the `--enable-all` flag.
[0] https://review.openstack.org/617834
Change-Id: I1530d7637b0eb6a83f048895053a5db80d033046
We have seen issues with dangling threads in Armada. This is likely due to
a bug [0] in the version of gRPC that we were pinned to.
This patchset:
- moves us to the latest versions of the gRPC python libraries which add
a new `channel.close()` method to cleanup channels.
- implements the python context manager api in the tiller handler
- uses the context manager api to explicitly scope tiller channel creation
and cleanup to each Armada API and CLI call.
This also fixes a couples issues with error handling introduced in [1].
[0]: https://github.com/grpc/grpc/issues/14338
[1]: https://review.openstack.org/#/c/610384
Change-Id: I2577a20fc76c397aa33157dc12a0e1d36f49733e
This adds a `wait.resources` key to chart documents which allows
waiting on a list of k8s type+labels configurations to wait on.
Initially supported types are pods, jobs, deployments, daemonsets, and
statefulsets. The behavior for controller types is similar to that of
`kubectl rollout status`.
If `wait.resources` is omitted, it waits on pods and jobs (if any exist)
as before.
The existing `wait.labels` key still have the same behavior, but if
`wait.resources` is also included, the labels are added to each resource
wait in that array. Thus they serve to specify base labels that apply
to all resources in the release, so as to not have to duplicate them.
This may also be useful later for example to use them as labels to wait
for when deleting a chart.
Controller types additionaly have a `min_ready` field which
represents the minimum amount of pods of the controller which must
be ready in order for the controller to be considered ready. The value
can either be an integer or a percent string e.g. "80%", similar to e.g.
`maxUnavailable` in k8s. Default is "100%".
This also wraps up moving the rest of the wait code into its own module.
Change-Id: If72881af0c74e8f765bbb57ac5ffc8d709cd3c16
Previously the chart `test` key was a boolean. This changes it to an
object which initially supports an `enabled` flag (covering the
previous use case) and adds support for helm's test cleanup option
(underneath an `options` key which mirrors what we have for `upgrade`).
Existing charts will continue to function the same, with cleanup always
turned on, and ability to use the old boolean `test` key for now. When
using the new `test` object however, cleanup defaults to false to match
helm's interface and allow for test pod debugging. Test pods can be
deleted on the next armada apply as well, to allow for debugging in the
meantime, by adding `pre`-`upgrade`-`delete` actions for the test pod.
The `test` commands in the API and CLI now support `cleanup` options as
well.
Change-Id: I92f8822aeaedb0767cb07515d42d8e4f3e088150
- Adding yapf diff to pep8 target
- Adding yapf tox target to do actual format
** The rest of this PS contains formatted code only, no other changes
Change-Id: Idfef60f53565add2d0cf65bb8e5b91072cf0aded
The helm test integration was severely broken, this fixes it by:
* correctly handle tiller test call response
* removes unnecessary call to tiller to get release content
* removes unnecessary call to k8s to check for test pod completion
* moves common logic into a test handler
* adds test coverage for the above
* adds logging for test results streamed from tiller
Change-Id: I09062387a1abc2fc3f6960f987c97248d9e1cb69
The release name was being treated as multiple different values to
mean the same thing, when paired with the 'release_prefix'. This
commit addresses the bug, changing all instances to use the
'release' value instead of 'chart_name' or others.
Note: This is an impacting change, in the sense that it will
cause more reliable behavior in Armada's Apply processing which
could have actual impact while upgrading components installed with
a previous version of Armada. Previuosly undeleted FAILED releases
may now be deleted, and armada test and delete actions may now
run as expected where they didn't run before.
Change-Id: I9893e506274e974cdc8826b1812becf9b89a0ab6
This exposes helm's force and recreate pods flags for upgrade and
rollback. It exposes in the chart manifest an options field underneath
the upgrade field to hold options to pass through to helm, and
initializes it with these two flags. Since rollback is currently a
standalone operation which does not consume manifests, these flags are
directly exposed as api and cli arguments there.
Change-Id: If65c1e97d437d9cf9d5838111fd485c80c76aa1d
This patch set removes E722 pep8 exclusion that allows for "bare"
except: statement.
Change-Id: Icdce885366541b88aabbef35166cf196a588676b
Signed-off-by: Tin Lam <tin@irrational.io>
- fixing wait handling in multiple areas
-- wait for deleted pods before continuing Apply update
-- cleaning up and delineating wait for charts vs chartgroups
-- timeout exceptions to stop execution
- api/cli 'timeout' param now applies to all Charts
- api/cli 'wait' param now applies to all Charts
- update some docs
- several TODOs to be addressed in future PS
Closes #199
Change-Id: I5a697508ce6027e9182f3f1f61757319a3ed3593
This patch set removes unreferenced exception variables indentified by PEP8.
Follow up patch sets will perform:
- Better exception handling and messaging;
- CICD pipeline should correctly fail these PEP8 violation.
Change-Id: I681a2d3c7836eb10a5906d4b035e99aff574b18d
Signed-off-by: Tin Lam <tin@irrational.io>
This ensures Armada's POST apply endpoint returns a 415 as
expected rather than a 500. Unit test added for regression.
Change-Id: I765d5e8992f66a3dba3dc3b883fb4037ba1c6d05
Armada was able to throw exceptions all the way up to invocation. To address:
- remove 'supermutes dotify', which was throwing exceptions
- refactor chartbuilder after removing dotify
- rework some helm wait/timeout logic, exposed during bug squash
- rename some variables to make their function more clear
Note: This has potentially breaking changes to existing charts, in the sense
that documents previously validated (improperly) may now give errors.
Change-Id: I9a6c99aa8ba3d666405e0ccf49a847fd01807b69
Adds the lazy-apps switch to remove some preloading behavior
Removes some undesired header logging.
Change-Id: I5177833e3184e9ab9d6469a2f5bcd8ee673870de
Enhance request logging (and scrub sensitive headers)
Enhance Tiller logging
Update grpcio, unpin from 1.6.0rc1
Plus a couple typo fixes
Plus a couple unused vars
Change-Id: I8afd679f6716c6e1af234a59ac44ba1fdc73cdc8
This adds falcon-based unit tests for
GET '/api/v1.0/test/{release}' endpoint.
This also renames the Test controllers into more specific names
in order to disambiguate the controller names.
Finally, the condition for returning whether a test passed
was changed to be `test_status.result[0].status == 'PASSED'`
as the default cause of 'FAILED' also passes the previous condition:
`test_status.result[0].status` which is wrong.
Change-Id: Ib84bc0b3baf5271cd7417000bbfed56b18840eed
This PS adds validation logic recently implemented in
armada.utils.validate [0] for validating documents and
Armada-generated Manifests to the Test and Tests controller classes.
Also refactors some exception handling for both controller classes to
better bubble up the appropriate exception.
Finally unit tests have been added for the Armada Test controller
to verify above changes work.
[0] https://review.gerrithub.io/#/c/378700/
Change-Id: I01f73c1778bf7c2e38032d5fddabd327c013edbb
BREAKING CHANGE: Armada will no longer support
recursive monolithic documents such that a Manifest
fully defines ChartGroups inline and ChartGroups
fully define Charts inline. Only name-based references
to other documents is supported.
- Author document schemas in standalone
JSON schema files
- Update validation to return all failures available
- Removed unit tests for support of recursive monolithic
documents
Change-Id: Idb91fa552d3d7a3d7d525609d505fe7380443238
Adding tiller-namespace (tn) flag to allow for setting specific tiller namespace
Updated:
- Armada API
- Test API
- Tiller API
- Armada handler
- Tiller handler
- cli/apply
- cli/test
- cli/tiller
Change-Id: I3a18d6ec2ce2f771c9349d6b337537f193f6c73d
This PS allows users to specify the manifest file to use
by the Armada handler by introducing a new flag called
`target_manifest`. This flag was added to the API and
CLI.
A foundation of unit tests for the manifest handler
is included in this PS. Most of the coverage is aimed
at checking the various success and failure cases
surrounding the new target_manifest feature.
Also updates documentation to convey information about
the new flag and clean up some documentation formatting
inconsistencies and typos.
Change-Id: I1d5a3ecc1e99b6479438d0ee5490610178be34fe
- Updating old /validate endpoints to reference the new endpoint at /validatedesign.
- Update docs to reflect /validatedesign and fix error in description.
Change-Id: Ie1fa4e4584c81373fe193b2b2f83cd96ff571437
This allows the user to apply and/or validate a manifest using
either a filepath (as before) or URL.
Addition by sh8121att:
- Create a general document resolver class to handle local paths
and URIs
- Allow multiple filenames and combine them into a single document
set
- Change API to allow for passing document reference URIs to be
resolved server-side rather
- Update validation API to conform to UCP specification
- Dockerfile updates to speed up build
- Fix unit tests
Closes #96
Change-Id: I5a57779f10d1b63ffc161a14afec851a34ae9efe
This PS adds the versions controller to Armada for UCP integration
compliance. It's responsible for returning information about
each of Armada's API versions.
Change-Id: I0ad5f0578ab97a50b30e8322647347aa1ed962dd
This PS sets the foundation for unskipping Armada API unit tests
by making necessary framework changes where necessary as well
as test refactoring so that the tests not only can be unskipped
but also execute successfully.
However, until a feature-rich testing framework is in place, it's
not possible to do end-to-end testing as mocking is currently used
to stub out Tiller API.
Negative RBAC tests will be added in a follow-up.
Included in this PS:
- policy fixture for asserting that expected policies are enforced
- unit tests for tiller/armada API
- test_utils module
Change-Id: I2f454b27b014875bc35fd706f7c0d05364ce562a
- using click framework
- added api client
- allow interactions between code and service endpoints
- documention on the command line
- updated gitignore
Change-Id: Ibe359025f5b35606d876c29fa88e04048f276cc8
Sergiy Markin