Update kubernetes client to v26.1.0
Updating armada to focal base image
Remove xenial and opensuse dockerfiles
Update tox python from py35 to py38
Add apparmor for docker build
Uplift HTK chart version 0.2.52
Bumping up some python dependencies to get in sync with shipyard
Added clear-firewall role for airskiff-deploy playbook
Change-Id: If06a3f60466702d05a21c24a7cb8041bed41507a
Helm 3 breaking changes (likely non-exhaustive):
- crd-install hook removed and replaced with crds directory in
chart where all CRDs defined in it will be installed before
any rendering of the chart
- test-failure hook annotation value removed, and test-success
deprecated. Use test instead
- `--force` no longer handles recreating resources which
cannot be updated due to e.g. immutability [0]
- `--recreate-pods` removed, use declarative approach instead [1]
[0]: https://github.com/helm/helm/issues/7082
[1]: https://helm.sh/docs/howto/charts_tips_and_tricks/#automatically-roll-deployments
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: I20ff40ba55197de3d37e5fd647e7d2524a53248f
This change adds publishing to docs.airshipit.org and updates the theme
to match the other Airship projects on the site. This change also
updates orphaned links and removes the Read the Docs jobs.
The documentation can be found at docs.airshipit.org/armada when this
change merges.
Change-Id: I9641753f6084f911e3286c623d0c2de7b3f6040a
Signed-off-by: Drew Walters <andrew.walters@att.com>
Currently the chart `source` schema allows for a proxy server to be
specified, but it is only used for git repos. This patchset allows
the `proxy_server` to also be used for tarball url sources.
Change-Id: I6f90d056fa46f596b1fb248b6c596c58b6513d64
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase. This is the effort
to do so for the Armada project.
[0] https://review.opendev.org/#/c/671291/
Change-Id: I4fe916d6e330618ea3a1fccfa4bdfdfabb9ffcb2
This introduces v2 docs in order to allow users to opt in to
breaking changes, while still supporting v1 docs for a time
so folks can migrate. At some point v1 doc support will be
removed.
This initial version of v2 docs is experimental. Further
breaking changes will be made before v2 docs are finalized.
A v1-v2 migration guide is included in the documentation.
This also refactors the internal data model to include the full
document structure, such as `metadata` and `schema`, so that
different behavior can be acheived for v1, v2, etc.
Change-Id: Ia0d44ff4276ef4c27f78706ab02c88aa421a307f
In the full airship use case, shipyard sends a deckhand url to
armada which returns all site documents, so this leads to many
many of these unwanted messages which make the logs difficult to
use. Hence this removes these messages.
Change-Id: I297df16d180b9ed6fae6bb8f8a0d96c3533766ae
When running helm tests for a chart release multiple times in a site,
if the previous test pod is not deleted, then the test pod creation
can fail due to a name conflict. Armada/helm support immediate test pod
cleanup, but using this means that upon test failure, the test pod logs will
not be available for debugging purposes. Due to this, the recommended approach
for deleting test pods in Armada has been using `upgrade.pre.delete` actions.
So chart authors can accomplish test pod deletion using this
feature, however, it often takes awhile, usually not until they test upgrading
the chart for chart authors to realize that this is necessary and to get it
implemented.
This patchset automates deletion of test pods directly before running tests by
using the `wait.labels` field in the chart doc when they exist to find all pods
in the release and then using their annotations to determine if they are test
pods and deleting them if so.
A later patchset is planned to implement defaulting of the wait labels when
they are not defined.
Change-Id: I2092f448acb88b5ade3b31b397f9c874c0061668
Armada remediates releases stuck in FAILED status, if not protected,
by purging and re-installing them. This implements the same for other
non-DEPLOYED statuses. For these statuses it guards this with a best
effort determination of whether a previous deployment of the release,
either through armada or the helm CLI, is likely still pending based
on whether it was last deployed within the chart's wait timeout. If
it is deemed likely pending an error is raised, however this
condition will eventually expire on future runs allowing for
eventual remediation.
Reasons why a release may get stuck in statuses other than DEPLOYED
or FAILED include:
1. tiller crashed mid-deployment
2. tiller could not reach kubernetes to update the release state
3. running `helm delete <rel>` (without --purge) (DELETED status)
Change-Id: Ia89cd59f056103dde47980a149c07a2984c4bbb4
While authoring [0], it was discovered that Armada has duplicate logic
for deciding if Helm test cleanup should be enabled as well as the tests
themselves. Because of this, changes to test logic (e.g. adding pre-test
actions), requires changing all traces of the repeated logic, which can
lead to inconsistent behavior if not properly addressed. This change
moves all test decision logic to a singular Test handler, implemented by
the `Test` class. This change does NOT change the expected behavior of
testing during upgrades; however, tests initiated from the API and CLI
will not execute when testing a manifest if they are disabled in a
chart, unless using the `--enable-all` flag.
[0] https://review.openstack.org/617834
Change-Id: I1530d7637b0eb6a83f048895053a5db80d033046
Previously if a chart is not updated, it would simply be skipped over.
Now, the wait/tests are run in this case to ensure the chart success
criteria is/was actually satisfied. It does still skip tests if there
is a last test result recorded as successful already, as an
optimization.
Change-Id: I5dc95fe0f16fe0989761e771c77d2c4fa8f6e7ea
Caching and cleanup of git repository chart sources was previously
implemented. This adds these features for tarball sources as well.
This also implements transitive chart dependency sourcing. Previously
only a single level of dependencies were being downloaded, which
would lead to an error when multiple dependency levels exist.
Change-Id: I988e473a6ea29331e036d26c3ec7269374e0188f
This fixes the following issues with listing releases from tiller,
which could cause Armada to be confused about the state of the
latest release, and do the wrong thing.
- Was not filtering out old releases, so we could find both a
FAILED and DEPLOYED release for the same chart. When this is the
case it likely means the FAILED release is the latest, since
otherwise armada would have purged the release (and all its
history) upon seeing the FAILED release in a previous run.
The issue is that after the purge it would try to upgrade
rather than re-install, since it also sees the old DEPLOYED
release. Also if a release gets manually fixed (DEPLOYED)
outside of armada, armada still sees the old FAILED release,
and will purge the fixed release.
- Was only fetching DEPLOYED and FAILED releases from tiller, so if
the latest release has another status Armada won't see it at all.
This changes to:
- Fetch releases with all statuses.
- Filter out old releases.
- Raise an error if latest release has status other than DEPLOYED
or FAILED, since it's not clear what other action to take in
this scenario.
Change-Id: I84712c1486c19d2bba302bf3420df916265ba70c
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes
Change-Id: I1e2d133a701dc2dade5bfcbdab5c0950cbe7eed5
In some use cases, some site level docs are only included in specific
manifests. This is so sites can call out what they want deployed, however
currently Armada is checking for all documents to exist and leads
to an invalid manifest exception.
This PS removes the '.build_charts_deps()' and 'build_chart_groups()' calls
in 'get_manifest()' so that only chart documents, and chart group documents
are built after finding them within 'build_armada_manfiest()' and
'build_chart_group()'. 'build_armada_manifest()' will now throw the
related 'Could not find chart group... exception' for related chart
and chart group issues. Additional subclass exceptions were added along
with adding traceback to capture the chained exceptions.
Change-Id: Idc8a75b290ac0afb1e177203535b012d589b708f
This PS refines the logic in override.update_manifests when validating
documents to properly deduce the correct exception
that needs to be thrown. Added appropriate logic in armada.py to
handle the exceptions thrwon by override.update_manifests. Also
validate_armada_documents now logs appropriate error/debug messages.
Introduced ArmadaNegativeHandlerTestCase class in test_armada.py,
along with updating/adding unit tests in test_override.py.
Change-Id: I84051ae4901011093f987479861df5f89561bb2c
This patchset adds additional options to the git SSH command
to ensure that no prompt is required to perform the clone.
This is needed for automation purposes. (For background when
first trying to clone something via SSH the client asks the
user whether they want to add the host to the hosts files --
this skips that check).
Change-Id: I752a354c5616f40eb8a0dbcb8606370654cabe14
- Adding yapf diff to pep8 target
- Adding yapf tox target to do actual format
** The rest of this PS contains formatted code only, no other changes
Change-Id: Idfef60f53565add2d0cf65bb8e5b91072cf0aded
The release name was being treated as multiple different values to
mean the same thing, when paired with the 'release_prefix'. This
commit addresses the bug, changing all instances to use the
'release' value instead of 'chart_name' or others.
Note: This is an impacting change, in the sense that it will
cause more reliable behavior in Armada's Apply processing which
could have actual impact while upgrading components installed with
a previous version of Armada. Previuosly undeleted FAILED releases
may now be deleted, and armada test and delete actions may now
run as expected where they didn't run before.
Change-Id: I9893e506274e974cdc8826b1812becf9b89a0ab6
This patchset adds missing Keystone options to registration
of config, including timeout, which is causing the following
error in testing:
packages/oslo_config/cfg.py", line 3130, in _get_opt_info
2018-06-12 20:28:40.318 9 ERROR armada.api raise NoSuchOptError(opt_name, group)
2018-06-12 20:28:40.318 9 ERROR armada.api oslo_config.cfg.NoSuchOptError: no such option timeout in group [keystone_authtoken]
2018-06-12 20:28:40.318 9 ERROR armada.api
This fix is similar to a recent Shipyard patchset [0] which
aimed to solve the same issue but expands on it by following
Nova patterns here: [1].
This also updates the sample config file which now includes
the timeout option.
[0] https://review.openstack.org/#/c/572563/
[1] http://git.openstack.org/cgit/openstack/nova/tree/nova/conf/cinder.py
Change-Id: I2c35bcfe9cacbc3a443f7f0975ad2ab43c7ca7c2
This patch set refactors and fixes the use of the keystoneauth to load
the session from cfg.CONF. This removes the unnecessary wrapping of
utility function into a class, but also allows the code to use other
accepted plugins to form the keystoneauth object.
The need to read environment variables should be handled only in the CLI
and should be accounted for by the CLI framework, not in a server utility
function.
Change-Id: Ib086f103bbb1e27fe8228ccf5f0d40526796e1e5
Signed-off-by: Tin Lam <tin@irrational.io>
- revise wait on namespace+label, only wait on ns+label for
charts we've touched in the current apply loop
- skipping any actions that would change system during dry-run
- skip 'test' and 'wait' during dry-run
- tweaking some logs for insight and readability
Change-Id: I1223f01690832c26ce2faa96e7e64620cf413ac9
This removes references to the old att-comdev git repos, either updating
to the openstack github mirror or deleting as appropriate. Old project
name (UCP) references are left alone for now.
Change-Id: I53b294a3c574fa0bca4037e5e00ad31d92f26977
This patch set removes E722 pep8 exclusion that allows for "bare"
except: statement.
Change-Id: Icdce885366541b88aabbef35166cf196a588676b
Signed-off-by: Tin Lam <tin@irrational.io>
- Validation messaging to match UCP convention
- Adding some missing fields to Chart validation schema
- Minor update: Adding debug logging to each CLI call
- Fixing some typos and exception messages
Change-Id: I7dc1165432c8b3d138cabe6fd5f3a6e1878810ae
Enhance request logging (and scrub sensitive headers)
Enhance Tiller logging
Update grpcio, unpin from 1.6.0rc1
Plus a couple typo fixes
Plus a couple unused vars
Change-Id: I8afd679f6716c6e1af234a59ac44ba1fdc73cdc8
BREAKING CHANGE: Armada will no longer support
recursive monolithic documents such that a Manifest
fully defines ChartGroups inline and ChartGroups
fully define Charts inline. Only name-based references
to other documents is supported.
- Author document schemas in standalone
JSON schema files
- Update validation to return all failures available
- Removed unit tests for support of recursive monolithic
documents
Change-Id: Idb91fa552d3d7a3d7d525609d505fe7380443238
- Add support for SSH key auth using existing config file value
- Add authentication exceptions
- Remove redundant git error handling from Armada handler
Closes #169
Change-Id: Ia0f61e0b74893289bb90560a743a243393d89c56
- adding .editorconfig file
- minor cleanup in various files related to .editorconfig
- typos, whitespace, etc.
- other general housekeeping items on the codebase
Change-Id: I104f8dcb06aafb180da12f7ee4c0ded41fc07b9d
Support proxy for installing armada/Chart/v1, as data.source.proxy_server when data.source.type = 'git'.
Closes #191
Change-Id: I9bfd7dbd63c86f65b24b27bd5eec547f862e2311
This PS unskips the last remaining git test in test_source which
had an unconditional unittests.skip() decorator applied to it.
The source_cleanup test now issues a warning if the repo path
isn't found and the associated test (passing in a bad path)
simply asserts that the warning was called.
This also adds a conditional skip to the git clone unit tests
in the event that the user doesn't have network connectivity.
Change-Id: Ic300c48a4e0c361e37ec15cd3675415f8170a817
This allows the user to apply and/or validate a manifest using
either a filepath (as before) or URL.
Addition by sh8121att:
- Create a general document resolver class to handle local paths
and URIs
- Allow multiple filenames and combine them into a single document
set
- Change API to allow for passing document reference URIs to be
resolved server-side rather
- Update validation API to conform to UCP specification
- Dockerfile updates to speed up build
- Fix unit tests
Closes #96
Change-Id: I5a57779f10d1b63ffc161a14afec851a34ae9efe
- Add --set flag to override manifest values from CLI
- Add --values flag to override manifest values from values file
- Add support to override manifests values with API values option
Closes #146
Change-Id: Iefa14e4d3005aab3ee803ffb65dfe1a867507c0e
This patch set makes Armada pep8 compliant. Note the hapi/** is
autogenerated and therefore should be excluded from linting.
Change-Id: I123eefb543f9bd9cf0bc6bd98ed95646d8d72cc3
-Add functionality to download a tarball from a URL, decompress/extract
the archive, and use as a chart source
-Compartmentalized functionality to later support extracting, but not
downloading, local tarballs
-Refactor specific git utils to general source utils
-Small exception handling bug fix
Sergiy Markin