- armada-airskiff-deploy is voting gate again
- fixed falcon.API deprecation - -> falcon.App
- fixed collections.abc.defaultdict not found error
- fixed tox4 requirements
- implemented requirements-frozen.txt approach to make allike as other
Airship projects
- uplifted docker version in the image building and publishing gate
Change-Id: I337ec07cd6d082acabd9ad65dd9eefb728a43b12
Update kubernetes client to v26.1.0
Updating armada to focal base image
Remove xenial and opensuse dockerfiles
Update tox python from py35 to py38
Add apparmor for docker build
Uplift HTK chart version 0.2.52
Bumping up some python dependencies to get in sync with shipyard
Added clear-firewall role for airskiff-deploy playbook
Change-Id: If06a3f60466702d05a21c24a7cb8041bed41507a
Bumping k8s client to v25.3.0
Cronjob batch v1beta1 no longer available in k8s 1.25
Update tox.ini file to be compatible with v4
Change-Id: Iac79c52c97c9ef1223ae8d502da1572ef8d068fa
Kubernetes v1.22 stopped serving the apiextensions.k8s.io/v1beta1 API
version of CustomResourceDefinition.
This change ensures that the locks.armada.process CRD is created using
the apiextensions.k8s.io/v1 API.
The kubernetes client package is also updated to take advantage of the
dynamic client.
Change-Id: Icd518ab5cbb78e8b15f63d19c51b5f5b9a67e995
For now we leave the tiller status enpdpoint, until
Shipyard has had a release to stop depending on it [0].
[0]: https://review.opendev.org/c/airship/shipyard/+/802718
Signed-off-by: Sean Eagan <seaneagan1@gmail.com>
Change-Id: If8a02d7118f6840fdbbe088b4086aee9a18ababb
- Fix helm CLI download URL, the previous URL is no longer available,
and is superceded by get.helm.sh
- Fix jsonschema dependency spec, 4.x alpha versions were causing the
unit tests to fail for python 3.5
- Remove oslo.db dependency as it was not used and one of its recursive
dependencies (tempita) was failing during setup
Change-Id: I03e8a738bb8aad620903119638210077d211770c
This PS:
1) Looks to improve specific logging in Armada, so that
it's easier to debug deployment related issues
2) Uplifts the k8s Python dependency to 12.0.0
3) Enforces 'watch' timeouts more strictly, as the call to
the Kubernetes Python watch function seemed unreliable.
4) Adds a field selector to the 'watch' stream to look for
the DELETE action to have been completed on the specific
pod/job/cronjob, rather than looking across the whole
namespace or via labels. This will narrow what the watch
is looking at, making the logs less busy.
Change-Id: I1952b0db32fb0b56ffffcddeae0532beb5a27b67
Patch PyYAML (via the pylibyaml library) to automatically enable the
LibYAML parser and emitter, which are faster than the Python versions.
https://pypi.org/project/pylibyaml/
Change-Id: I4a3045e15df909fb4760800ce1fb87e298dd1e67
When pip is upgraded to 20.3, the pip dependency resolver is much more
strict and will no longer install a combination of packages that is mutually
inconsistent[0].
These changes account for the fact that Shipyard imports Armada, Drydock,
Promenade, and Deckhand. Having said that, with pip 20.3, the pip
packages amongst those projects cannot conflict. A follow-up change may
be needed if more conflicts are found.
[0] https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip-dependency-resolver-in-20-2-2020
Change-Id: Icd10a452a3b9f67b3431d75ba00e942fe42d7c22
Now that the Armada gates are running on Focal(20.04), grpcio
and pyflakes need to use a version compatible with python3.8 and Ubuntu
20.04
Change-Id: I1071d211e5a6c4a5fcdad9818f735fcb3c97d962
Armada uses a Kubernetes watch to implement its chart wait logic. This
can be a fairly long-lived connection to the Kubernetes API server, and
is vulnerable to disruption (if, for example, the kubernetes apiserver
chart is being upgraded).
This change allows Armada to retry the wait for some specific errors,
including the establishment of a new watch, until the overall chart
timeout is reached.
https://github.com/kubernetes-client/python/issues/972
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))
Change-Id: I3e68a54becadd5b2a2343960a120bdc3de8e8515
This implements Prometheus metric integration, including metric
definition, collection, and exportation.
End user documentation for supported metric data and exportation
interface is included.
Change-Id: Ia0837f28073d6cd8e0220ac84cdd261b32704ae4
Grpc tools is only needed for `make protoc` to generate the tiller grpc code.
This will result in it being removed from Armada's docker image as well as
Shipyard's once the Armada git reference is uplifted there to include this change.
Change-Id: Iede466db4fd2d157c2f86235dc6b6e35c706d8d8
We have seen issues with dangling threads in Armada. This is likely due to
a bug [0] in the version of gRPC that we were pinned to.
This patchset:
- moves us to the latest versions of the gRPC python libraries which add
a new `channel.close()` method to cleanup channels.
- implements the python context manager api in the tiller handler
- uses the context manager api to explicitly scope tiller channel creation
and cleanup to each Armada API and CLI call.
This also fixes a couples issues with error handling introduced in [1].
[0]: https://github.com/grpc/grpc/issues/14338
[1]: https://review.openstack.org/#/c/610384
Change-Id: I2577a20fc76c397aa33157dc12a0e1d36f49733e
We were seeing false positives when diffing charts to determine
whether an upgrade was necessary. Previously we were serializing the
charts and values and diffing those, but these serializations often
output things in different and non-deterministic order, hence the
false positives. This removes the ordering concerns by puttings things
in maps instead of lists, and comparing those semantically rather than
via serialization. This also improves the diff output to be easier to
read. It also stops caring about diffs in Chart.yaml.
Change-Id: I4c92c2e7c814178c374623ea52d717bdb9f72b11
Armada was able to throw exceptions all the way up to invocation. To address:
- remove 'supermutes dotify', which was throwing exceptions
- refactor chartbuilder after removing dotify
- rework some helm wait/timeout logic, exposed during bug squash
- rename some variables to make their function more clear
Note: This has potentially breaking changes to existing charts, in the sense
that documents previously validated (improperly) may now give errors.
Change-Id: I9a6c99aa8ba3d666405e0ccf49a847fd01807b69
Enhance request logging (and scrub sensitive headers)
Enhance Tiller logging
Update grpcio, unpin from 1.6.0rc1
Plus a couple typo fixes
Plus a couple unused vars
Change-Id: I8afd679f6716c6e1af234a59ac44ba1fdc73cdc8
BREAKING CHANGE: Armada will no longer support
recursive monolithic documents such that a Manifest
fully defines ChartGroups inline and ChartGroups
fully define Charts inline. Only name-based references
to other documents is supported.
- Author document schemas in standalone
JSON schema files
- Update validation to return all failures available
- Removed unit tests for support of recursive monolithic
documents
Change-Id: Idb91fa552d3d7a3d7d525609d505fe7380443238
- using click framework
- added api client
- allow interactions between code and service endpoints
- documention on the command line
- updated gitignore
Change-Id: Ibe359025f5b35606d876c29fa88e04048f276cc8
Version 1.4.0 was released after the `no_proxy` fix, but the fix does
not work in this version. 1.6.0rc1 does respect `no_proxy` as expected.
Change-Id: Ib0e81334765df6d5ffd025b48294fe52724c4506
The previous version did not respect no_proxy settings, causing Armada
to attempt to use the proxy to talk to tiller even when not appropriate,
e.g. on localhost.
Change-Id: Ib4e7bd322c7f5781642800d01f3f984f4e6c4133
- Refactor API structure
- Add API support for existing CLI flags
- Add Keystone token and RBAC authentication
- Add API documentation
- Add API unit tests
- Adds Oslo logging libraries
- Enables logging configuration with a config file
- Enables debug logging with --trace flag
- Supports Docker logs
- Adds logging for tiller
- Add 'oslo.log' package to requirements.txt
- Replace Python logging library with Oslo logging library in server.py
and log.py
- Update iso8601 silencincing to default option