This PS updates python modules and code to match Airflow 2.6.2
- bionic py36 gates were removed
- python code corrected to match new modules versions
- selection of python modules versions was performed based on
airflow-2.6.2 constraints
- a new pegleg validation gate was created based on airskiff deploy
pipeline from treasuremap v1.9
Change-Id: Icc96deb0b745eaab7a19d83b00aad909dcd28c25
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.
Change-Id: Id31d97ced8732d823937fb1f218e7ad8760d735c
This PS delivers focal version of Pegleg image and has the following updates:
- removed release-notes-jobs-python3 gate job because of incompartibility with Sphinx from current requirements
- added focal gate node and switched gates to use it
- added bindep.txt file into project root
- added bindep role into gate jobs
- added ubuntu_focal dockerfile for building focal pegleg image
- switched tox profiles to py38
- uplifted references to shipyard_client, promenade and deckhand projects
- resolved required dependencies conflicts by weakening constraints in Pipfile
- updated tox profile update-requirements for generate requirements.txt and test-requirements.txt
- generated new Pipfile.lock, requirements.txt and test-requirements.txt from Pipfile
- switched tox profiles to use requirements.txt and test-requirements.txt instead of pipenv because of upstream zuul nodes Pypi mirrorring issue
- updated reference to seaworthy site certificates in treasuremap repo
- fixed unit tests issues caused by pytest/mock updates and new openssl version
- fixed focal docker image publishing issue
- added multiprocessing into coverage tests running process
- made unit and coveraget tests more verbosive
Change-Id: I5c4c519dc725cfb8c7b4e14756347c9336028aff
* disable python 35, leave only 36
* switch to bionic nodeset only
* update requirements.txt, Pipfile and lock
Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: Ia3c4df9d1f39f0f2cebf8ba0d89aebc5eec6f674
Remove default pip.conf as some packages are
not availablei from the mirror and cause
the image build jobs to fail.
Change-Id: I125360952190a4dc5cfb98647cd94dcf506f9b71
Update the whitespace linter script to avoid tripping on png files,
using git-grep option:
-I
Don’t match the pattern in binary files.
Change-Id: Ifcd60354086cc606dbd3e71981484201b00c619c
This change makes zuul jobs configurable on project level using variables.
That allows to reuse same jobs in other zuul setup with different base images,
docker registry and proxy settings.
Change-Id: I301ea09f42bc1498d3cba45eede075e0d5a6f8fa
Use apt to install python3-pip, and use pip3 in event system has
both pip2 and pip3 installed. Use apt to install setuptools for
Ansible's consumption.
Change-Id: Icd66c91e1c26627f756373800b3913d8c347e412
Signed-off-by: Alexander Hughes <Alexander.Hughes@pm.me>
Co-Authored-By: Alexey Odinokov <aodinokov@mirantis.com>
This patchset updates pegleg airship clients for shipyard and
deckhand to use the new clients, which support bionic base image.
Change-Id: I266747b84c39984b941afd6454647fe0d5510ca3
This patch brings Pegleg in line with other Airship projects in their
method of updating documentation. This is achieved by:
1. Adding docs-on-readthedocs job
2. Adding readthedocs webhook and project name vars used for Pegleg
3. Removing airship-pegleg-doc-build
4. Removing doc-build playbook
Change-Id: Iaa4139ccb1cd9d7ca20a6b502ef2a152908147d0
Add DISTRO parameter to support multiple distros
Add Dockerfile for opensuse to build leap 15 image.
Change-Id: I7a529476937494e042a4801117489325aa6621c7
Pegleg currently uses `find` to search files in the whitespace-linter
script. A more simplified approach could be taken by using `git grep`
instead. This method levarages .gitignore so a separate list of files
in the script no longer needs to be maintained. This is the method used
by Airship Armada.
Change-Id: I26a2a95f533b9ff62de784d004f25ade552a5b31
This patch set does 2 things:
1) Improves unit test runtime peformance via pytest-xdist [0]
2) Reduces finnicky nature of `is_connected` helpers which
sometimes skip even when there is access to the internet;
logic has been added to make these checks more accurate
to avoid skipping tests
Note that while there are newer alternatives to pytest-xdist they
are only compatible with much newer versions of Python.
[0] https://pypi.org/project/pytest-xdist/
Change-Id: Ib04b48ebabca0551058e5e1065056f4e559fbfe6
The openstack-python36-jobs template already exists; thus there
is no need for airship-pegleg-tox-py36. Since airship-pegleg-tox-py36
installs cfssl as a prerequisite for unit tests, move the install-cfssl.sh
command to tox.ini prior to unit test execution, allowing for the
airship-pegleg-tox-py36 Zuul job and its associated playbook to be
removed.
Change-Id: I66de957a1a57ef246476c1a81954cd0f822cb8be
- Zuul updated ansible to 2.7, no longer uses missing variables.
- Using an if to try and address.
Change-Id: I547097efc9ed2b2ca308cf394fe14dab16abacc4
Consolidating all the linting to a single job rather than having a one-off
just to lint for trailing whitespaces. As most of this projects are python,
this should already be covered by PEP8, but will be scanning for yamls with
trailing whitespaces.
Change-Id: Iee33a69ff234d21c08217faa33a19e11dfef0ad9
Signed-off-by: Tin Lam <tin@irrational.io>
This patch set adds a shell wrapper script to allow
tox to be used to run a subset of unit tests via regexes:
To run all unit tests, execute::
$ tox -epy35
To run unit tests using a regex, execute::
$ tox -epy35 -- <regex>
Change-Id: I2ba1e18226d686cb549a075e020ba02e24204829
With the implementation of revisioned repository to the CLI in
https://review.openstack.org/#/c/577886 there was a change to
the lint command [0], which changed it from being a global lint
to a site-level (targetted lint)... kind of: Only the CLI logic
was modified to support targetted single-site linting. Thus,
the first issue this patch set addresses is implementing the
back-end logic to realize targetted, single-site linting.
The second issue this patch set addresses is re-supporting global
linting (linting all sites within a repository) which means that
this partially reverts [0] which had (kind of) replaced global
linting with per-site linting.
So, this patch set:
1) Implements targetted, single-site linting back-end logic
2) Re-implements global linting for all sites in a repo
3) Adds unit tests for both
4) Adds some helper functions to util.engine.definition to
help with 1) and 2)
[0] https://review.openstack.org/#/c/577886/4/src/bin/pegleg/pegleg/cli.py@191
Change-Id: I5147282556763d93dfaf06912d2c4c876e1bd69f
This takes care of removing -t option when running the script
with automated system (e.g. Jenkins)
Change-Id: I087b32d2635251f967a8f78c4d1f945827573be6
1) Use OCI Image Specs for labels instead of custom 'commit-id=xxxxx'
or legacy "Label Schema"
2) Fix missing git commit id labels on images (.revision)
3) Add human-readable title (.title) of the image, URL (.url), and
a few other properties (annotations) according to the latest Specs
Change-Id: I57318d4662d90b439d4b7766f7c67571e0f69f15
This patch set introduces a developer overview documentation
page to Pegleg to help developers onboard.
Change-Id: Ia453d76f024db39c6bdd97a44bfe1db1c25193f9
Update image reference to quay.io/airshipit vs quay.io/attcomdev.
Set host networking for running pegleg container.
Recent Ubuntu releases are using 127.0.0.53 in /etc/resolv.conf,
and host networking mode is a way to work around local DNS.
Change-Id: Ib3e39e6f83e0bb7a50f92f85b791c8d0b59664d1
This change adds the global zuul pep8 tox job, which runs both
bandit and pep8 using tox. This also removes the two other airship
specific lint-pep8 and bandit zuul jobs since they are both covered
by the default openstack global one.
Also cleaned up the tox.ini by moving the requirements into the
test-requirements.txt file.
Change-Id: Iab37a8090515936732e390b1f7c6d281e014e31c
This patch set updates the default image reference used by
pegleg.sh from a downstream reference to the appropriate
upstream reference.
Change-Id: Id0198c6b69ee145901dae208ab9584e102f39827
l is to let user customize the base image of the component
by passing FROM=myimage during the build process. This would let any
project leveraging Airship ensure that the base image is matching the
security requirements for that project and still use the same Dockerfile.
This will also ease the control of the /etc/apt/source.list
and thereby the result of apt-get update/upgrade procedure.
2. The above goal is achievable by using docker-ce feature such as:
ARG FROM="defaultbaseimage:xx"
FROM ${FROM}
For this reason, the installation of docker.io in the Zuul gating is beeing
replaced by docker-ce.
3. Third Goal is to bring consistency with the other compoenents leveraging
Helm such as the openstack-helm and potentially use bindep the same way
the LOCI images are to ensure
4. The new syntax in the Dockerfile is still commented out until the associated
image builder have been updated to use docker-ce as they have been for the LOCI
images.
Change-Id: I6703589f32487f5668d709f485dae5782b13c002
Remove unused code. Set WORKSPACE default to $PWD and set workdir in the
container, so that users of pegleg.sh do not need to have any knowledge of
the internal "/worksapce" mountpath inside the container, and in general
simplify the execution of container command line tools.
Change-Id: I56dea7d116f88df303aaeca8dc25a7b24cb291a6
This patchset updates docs to doc to align with OpenStack
standard. Follow-up patchset will be needed to publish
documentation to OpenStack [0].
[0] https://docs.openstack.org/doc-contrib-guide/project-guides.html
Change-Id: I90e5f9129207901402e26ed9488ec6e065568fe1
This PS adds the skeleton for a set of zuul checks and gates for
Airship, using the framework from OpenStack-Helm.
Change-Id: I4cc071d0076e265e5c34f15616ccf40c0b3a83a8
The referenced quay.io image was no longer valid. Changed to
use artifactory iamge in atlantafoundry.
Change-Id: I29b384271e5f65e7609ffae52a553bf82f7a0d32
- A single primary repo must be specified which
holds the site_definition.yaml file.
- Zero or more auxiliary repos can be specified which
have additional documents used in the site definition.
- Collected documents are written to a file named after their
source repo. Collection must always be given a output directory
rather than a single file now.
Change-Id: Iceda4da18c4df45d917d88a49144e39e3f1743ed
This PS updates the pegleg script to build and run the docker
image/container with host networking/no networking - enabling it
to be used seamlessly on k8s hosts that do not make use of dockers
iptables rules.
Change-Id: Ia08b805f79ec8f82f3555856f316921f7eddb064
Sergiy Markin