Commit Graph

23 Commits

Author SHA1 Message Date
Ruslan Aliev 85da464cec Add gettext package to docker images
Allows to use envsubst utility within pegleg container.

Signed-off-by: Ruslan Aliev <raliev@mirantis.com>
Change-Id: I7733854253f3d4a6f9367678d93da9d4056e9535
2023-06-21 10:25:22 -05:00
Sergiy Markin 0f2ab241f9 Restored ubuntu_bionic image build
This PS restores image build for ubuntu_bionic and adds appropriate
gates to keep it tested by appropriate functional and integrational
tests.

Change-Id: Id31d97ced8732d823937fb1f218e7ad8760d735c
2023-06-07 21:02:28 +00:00
Sergiy Markin c052d40277 Pegleg focal upgrade
This PS delivers focal version of Pegleg image and has the following updates:
- removed release-notes-jobs-python3 gate job because of incompartibility with Sphinx from current requirements
- added focal gate node and switched gates to use it
- added bindep.txt file into project root
- added bindep role into gate jobs
- added ubuntu_focal dockerfile for building focal pegleg image
- switched tox profiles to py38
- uplifted references to shipyard_client, promenade and deckhand projects
- resolved required dependencies conflicts by weakening constraints in Pipfile
- updated tox profile update-requirements for generate requirements.txt and test-requirements.txt
- generated new Pipfile.lock, requirements.txt and test-requirements.txt from Pipfile
- switched tox profiles to use requirements.txt and test-requirements.txt instead of pipenv because of upstream zuul nodes Pypi mirrorring issue
- updated reference to seaworthy site certificates in treasuremap repo
- fixed unit tests issues caused by pytest/mock updates and new openssl version
- fixed focal docker image publishing issue
- added multiprocessing into coverage tests running process
- made unit and coveraget tests more verbosive

Change-Id: I5c4c519dc725cfb8c7b4e14756347c9336028aff
2023-05-02 15:05:45 +00:00
Maximilian Weiss 9e8216aa5e Fix some outdated Zuul dependency errors
Shpinx incorrectly claims it wants docutils >=0.18 but that is an error
and older versions fail with that, as mentioned here:

https://github.com/sphinx-doc/sphinx/issues/9841

Additionally, the repo URL for OpenSUSE 15.3 python has changed.

Change-Id: I9bee6cf3ad7aaba80a44f2bd2f917b16c776c0d7
2022-01-10 22:18:38 +00:00
anthony.bellino 5acd80abcf Fix opensuse_15 image
Leap 15.3 changes for opensuse image build

Change-Id: I24952bf66f579a5b96ecff7b55fbc39877c93f7c
2021-05-14 13:42:02 +00:00
Phil Sphicas f020bdbc3f Fix ubuntu_xenial build (use pip <21.0)
pip 21.0 requires Python >= 3.6. [0]
The latest official python3 package for xenial is 3.5.1-3 [1]

Until we stop building xenial images, ensure that an older pip version
is used.

0: https://pypi.org/project/pip/21.0/
1: https://packages.ubuntu.com/xenial/python3
Change-Id: I6a51ae5b9e3222ca404c7ccd7dea1209b20ce8fd
2021-02-15 04:33:54 +00:00
Phil Sphicas 3ca39ef54a Include LibYAML in container builds
Updates Dockerfiles to build the LibYAML library, which can provide much
faster YAML parsing and emitting than the native Python library.

https://pyyaml.org/wiki/LibYAML

Change-Id: I4cd48d5d5b5dddc44c88e9e08e405db96359ea6f
2020-09-25 01:01:20 +00:00
Ahmad Mahmoudi def3afff05 Bionic pegleg airship clients
This patchset updates pegleg airship clients for shipyard and
deckhand to use the new clients, which support bionic base image.

Change-Id: I266747b84c39984b941afd6454647fe0d5510ca3
2020-03-13 15:12:52 +00:00
Ian H. Pittwood bbef2431bc Add gpg to bionic packages
The `gpg` package does not come preinstalled on Ubuntu Bionic, but is
required in order to run the `genesis_bundle` command. This change adds
an install command for `gpg` to the Bionic image.

Change-Id: I77fa9151fbc947aabb371581ad4defd2cf37af1c
2020-01-21 08:37:56 -06:00
Ian H Pittwood 33d650c614 Fix OpenSUSE image build
Upgrades Deckhand to revision supporting six 1.12.0
https://review.opendev.org/#/c/677272/

Installs python3 and overrides python3-six version in OpenSUSE image

Reenables OpenSUSE image build gate

Change-Id: Id72dad8e3668d77b06aa8af4278fcdff0cb678eb
2019-08-22 19:46:05 +00:00
Ian H Pittwood 36b8e9fe60 Resolves installation problems in Docker
A recent change to implement Pipenv caused VCS dependencies (Promenade,
Deckhand, and Shipyard) to not be fully installed in Docker images. This
change removes the "editable" tags from the VCS dependencies to ensure a
full install as having "editable" enabled will only install dependencies
in development mode.

Unfortunately, the "editable" tag is required to install the
requirements.txt for a VCS dependency. To get the lower-level
dependencies installed from VCS dependencies, I implemented a few
commands in the Dockerfiles to retrieve the appropriate requirements.txt
and install them before fully installing Pegleg. An upcoming release of
Pipenv will fix the existing problems with VCS dependency resolution at
which point this temporary solution may be removed.

Adds manual installation of VCS dependency requirements.txt in Docker

Removes "editable" tags from VCS dependencies

Moves docker package to deployment packages from dev packages

Adds .env file to track VCS refs used by Docker for requirements.txt

Change-Id: Ifdb1fe960b32280dcb3c5308e56b2d608f848975
2019-08-16 12:24:57 -05:00
Rajeshwari Dharwadkar 70f2db4652 Update base image from leap15.0 to leap15.1
Changes made in opensuse_15 dockerfile to support opensuse leap15.1

Change-Id: Ida9fd504bbfc2f887561a96053196411154f91f4
2019-07-11 17:27:53 -07:00
Hughes, Alexander (ah8742) bf2f3781fb Add Ubuntu Bionic support to Pegleg
From community meeting on 04-June-2019 Ubuntu Xenial is the default
image for Airship projects, but a desire was voiced to also add support
for Ubuntu Bionic at the convenience of the contributors for each
project.

This patch:
1. Adds a new dockerfile specific to ubuntu:18.04 (bionic)
2. Updates gates to be specific about which ubuntu image is being
   checked.
3. Add to .zuul.yaml checks/gates/post jobs for bionic

Change-Id: Ib10641656f48baffec5b03ec48bf864d67209289
2019-06-07 20:05:33 +00:00
Hughes, Alexander (ah8742) 31fd2d8ca3 Update dockerfiles to include ssh client
It was discovered that some base images when used as an override to
the specified default images do not include the openssh package.
This is particularly problematic if a user wishes to use ssh access
when specifying their repositories in the site's site-definition.yaml

Without the openssh package the following error occurs:
ERROR pegleg.engine.util.git:normalize_repo_path [nnn] The
repo_path=ssh://user@domain:port/site-repo is not a valid Git repo

Adding the openssh package does not impact the current base images as
they already include it, but has the added benefit of ensuring that
a non-default base image will still work with Pegleg.

Change-Id: I154c3db5071a373ad16cb0a0c4c6103b6ea8ac4e
2019-06-06 21:10:47 +00:00
Hughes, Alexander (ah8742) 7d440b39e9 Update Pegleg base image to use Ubuntu 16.04
Currently the Pegleg base image is python:3.6, after a full build of
the Pegleg image and pushing it to quay it was discovered that the
final image had more than 600 vulnerabilities in the image scan
report [0].

When inspecting other Airship projects it became evident that only
the Pegleg and Spyglass projects were using python:3.6. The remaining
projects use ubuntu:16.04 as their default base image

Locally scanning with Clair [1] confirmed that the base image plays a
substantial role in the number and severity of vulnerabilities
present in the final Pegleg image. By switching from python:3.6 to
ubuntu:16.04 the number of vulnerabilities reported by Clair was
reduced to 130, none of which were high - from the original 600+ with
~50 high.

This patchset makes the following changes with the aim to reduce the
vulnerability count and severity in the final Pegleg image by:
1. Updating the Dockerfile for Ubuntu builds to use 16.04
2. Updating the Dockerfile to install necessary packages for Pegleg
   to run that are not included with the ubuntu:16.04 base image
3. Renaming the Dockerfile to accurately reflect the Ubuntu
   distribution
4. Updating the docker build jobs in .zuul.yaml to set the
   distribution to ubuntu_xenial
5. Updating the Makefile to set distribution to ubuntu_xenial
6. Updating the pegleg.sh script to use the correct image tag with
   the changes to the distribution in (1-5)
7. Updating the documentation to reflect that the Ubuntu base image
   is 16.04 (Xenial)

[0]: https://quay.io/repository/airshipit/pegleg/manifest/sha256:86d47bf777216eb28c4fc3594e57b0f758fd532b7e88a17ab8e5bd4f42dcd44e?tab=vulnerabilities
[1]: https://github.com/arminc/clair-scanner

Change-Id: I3c5ef761f9ea01b9673f6a2d08c499e8dc409c9d
2019-06-04 16:41:22 +00:00
Rajeshwari Dharwadkar 6ee2aaf845 Support pegleg to run on opensuse leap15 image
Add DISTRO parameter to support multiple distros
Add Dockerfile for opensuse to build leap 15 image.

Change-Id: I7a529476937494e042a4801117489325aa6621c7
2019-05-14 09:41:21 -07:00
Alexander Hughes 50ffabdaf5 Update references from openstack to opendev
The dockerfile and some unit tests were still pointing to review.openstack.org
update those references to review.opendev.org

Change-Id: I161158ac0d66533a1775957864d1bd69dfa9530b
2019-04-24 15:22:53 -05:00
Felipe Monteiro 2a8d2638b3 pki: Port Promenade's PKI catalog into Pegleg
This patch set implements the PKICatalog [0] requirements
as well as PeglegManagedDocument [1] generation requirements
outlined in the spec [2].

Included in this patch set:

* New CLI entry point called "pegleg site secrets generate-pki"
* PeglegManagedDocument generation logic in
  engine.cache.managed_document
* Refactored PKICatalog logic in engine.cache.pki_catalog derived
  from the Promenade PKI implementation [3], responsible for
  generating certificates, CAs, and keypairs
* Refactored PKIGenerator logic in engine.cache.pki_generator
  derived from Promenade Generator implementation [4],
  responsible for reading in pegleg/PKICatalog/v1 documents (as
  well as promenade/PKICatalog/v1 documents for backwards
  compatibility) and generating required secrets and storing
  them into the paths specified under [0]
* Unit tests for all of the above [5]
* Example pki-catalog.yaml document under pegleg/site_yamls
* Validation schema for pki-catalog.yaml (TODO: implement
  validation logic here: [6])
* Updates to CLI documentation and inclusion of PKICatalog
  and PeglegManagedDocument documentation
* Documentation updates with PKI information [7]

TODO (in follow-up patch sets):

* Expand on overview documentation to include new Pegleg
  responsibilities
* Allow the original repository (not the copied one) to
  be the destination where the secrets are written to
* Finish up cert expiry/revocation logic

[0] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#document-generation
[1] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
[2] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html
[3] https://github.com/openstack/airship-promenade/blob/master/promenade/pki.py
[4] https://github.com/openstack/airship-promenade/blob/master/promenade/generator.py
[5] https://review.openstack.org/#/c/611739/
[6] https://review.openstack.org/#/c/608159/
[7] https://review.openstack.org/#/c/611738/

Change-Id: I3010d04cac6d22c656d144f0dafeaa5e19a13068
2019-01-15 13:29:21 -06:00
Zuul 6c6bea992d Merge "Fix: git commit id labels on images" 2018-10-01 12:07:52 +00:00
Felipe Monteiro 893ea9f4bb Standardize Pegleg directory structure
This patch set standardizes the Pegleg directory
structure because of the following reasons:

1) src/bin/pegleg is not necessary and only makes
building (e.g. documentation building) and running
of tox targets unnecessarily difficult.
2) src/bin/pegleg is a Java-like standard that
bears no relevance to Python.

Change-Id: I37d39d3d6186b92f8fbfe234221c9e44da48cf10
2018-09-23 10:33:40 -04:00
Roman Gorshunov de6486c380 Fix: git commit id labels on images
1) Use OCI Image Specs for labels instead of custom 'commit-id=xxxxx'
   or legacy "Label Schema"
2) Fix missing git commit id labels on images (.revision)
3) Add human-readable title (.title) of the image, URL (.url), and
   a few other properties (annotations) according to the latest Specs

Change-Id: I57318d4662d90b439d4b7766f7c67571e0f69f15
2018-09-21 03:31:12 +02:00
Jerome Brette 4727df6b80 Update Dockerfile to allow override of FROM variable
l is to let user customize the base image of the component
by passing FROM=myimage during the build process. This would let any
project leveraging Airship ensure that the base image is matching the
security requirements for that project and still use the same Dockerfile.
This will also ease the control of the /etc/apt/source.list
and thereby the result of apt-get update/upgrade procedure.
2. The above goal is achievable by using docker-ce feature such as:
ARG FROM="defaultbaseimage:xx"
FROM ${FROM}
For this reason, the installation of docker.io in the Zuul gating is beeing
replaced by docker-ce.
3. Third Goal is to bring consistency with the other compoenents leveraging
Helm such as the openstack-helm and potentially use bindep the same way
the LOCI images are to ensure
4. The new syntax in the Dockerfile is still commented out until the associated
image builder have been updated to use docker-ce as they have been for the LOCI
images.

Change-Id: I6703589f32487f5668d709f485dae5782b13c002
2018-07-17 14:37:08 -05:00
Scott Hussey b3ea5de2b8 Update to UCP layout standard
- Create Makefile for image build
- Move Dockerfile into images/pegleg
- Move pegleg module src to src/bin/pegleg

Change-Id: I8fd728888ecfd75fe857da253d6c8cd4fd83f89c
2018-03-05 07:42:00 -06:00