Commit Graph

15 Commits

Author SHA1 Message Date
Alexander Hughes 1c8d92ef6b Standardize Pegleg code with YAPF
This patch addresses inconsistent code style and enforces it with a
gate for future submissions.

Separate work will be done in the future to address several of the
PEP8 ignores for docstrings, and attempt to bring the tests directory
to PEP8 compliance.

This patch:
1. Updates .style.yapf to set the knobs desired for YAPF.
2. Updates tox.ini to allow one of the knobs to work.
3. Removes unused code from several __init__.py files.
4. Updates the YAPF version in test-requirements.txt to latest (this
   is needed for several knobs to work).
5. Stylistic changes to the python codebase in Pegleg.
6. Updates to tox.ini to run YAPF during PEP8 check.

Change-Id: Ieaa0fdef2b601d01c875d64b840986e54df73abf
2019-07-25 17:28:18 +00:00
Lev Morgan b8733ea7ec Make -u required in CLI when required by repo
This patch detects when a repository URL requires username substitution
and raises an exception when no username was specified.

Change-Id: Ia60982ecddd957cff8709118b3eb8a905258dd06
2019-05-15 18:38:38 +00:00
Felipe Monteiro 2a8d2638b3 pki: Port Promenade's PKI catalog into Pegleg
This patch set implements the PKICatalog [0] requirements
as well as PeglegManagedDocument [1] generation requirements
outlined in the spec [2].

Included in this patch set:

* New CLI entry point called "pegleg site secrets generate-pki"
* PeglegManagedDocument generation logic in
  engine.cache.managed_document
* Refactored PKICatalog logic in engine.cache.pki_catalog derived
  from the Promenade PKI implementation [3], responsible for
  generating certificates, CAs, and keypairs
* Refactored PKIGenerator logic in engine.cache.pki_generator
  derived from Promenade Generator implementation [4],
  responsible for reading in pegleg/PKICatalog/v1 documents (as
  well as promenade/PKICatalog/v1 documents for backwards
  compatibility) and generating required secrets and storing
  them into the paths specified under [0]
* Unit tests for all of the above [5]
* Example pki-catalog.yaml document under pegleg/site_yamls
* Validation schema for pki-catalog.yaml (TODO: implement
  validation logic here: [6])
* Updates to CLI documentation and inclusion of PKICatalog
  and PeglegManagedDocument documentation
* Documentation updates with PKI information [7]

TODO (in follow-up patch sets):

* Expand on overview documentation to include new Pegleg
  responsibilities
* Allow the original repository (not the copied one) to
  be the destination where the secrets are written to
* Finish up cert expiry/revocation logic

[0] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#document-generation
[1] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
[2] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html
[3] https://github.com/openstack/airship-promenade/blob/master/promenade/pki.py
[4] https://github.com/openstack/airship-promenade/blob/master/promenade/generator.py
[5] https://review.openstack.org/#/c/611739/
[6] https://review.openstack.org/#/c/608159/
[7] https://review.openstack.org/#/c/611738/

Change-Id: I3010d04cac6d22c656d144f0dafeaa5e19a13068
2019-01-15 13:29:21 -06:00
Tin Lam 1a325a400b Add hacking extension
This patch set adds hacking rule to pegleg and fixes outstanding non-
docstring related violations.

Change-Id: I5bb5e78c211f24cf95669124bfcf9603bea8bf15
Signed-off-by: Tin Lam <tin@irrational.io>
2019-01-01 00:18:03 -06:00
Nishant Kumar cfa57e09a7 FIX: Passing correct fields to process site
This PS passes correct fields which is required for processing
site_definiton.yaml appropriately.

Change-Id: I546caa1a034fe1fd05c3568137982cb7b4ef7bec
2018-10-31 16:58:10 +00:00
Zuul 25dfde96e5 Merge "docs: Add docstring information for pegleg.config" 2018-10-29 16:48:41 +00:00
Felipe Monteiro c498bfee81 fix: Parse revision out of SSH repo url
This patch set adds additional logic to properly handle parsing
out the revision from an SSH repo url. The issue was being
masked by unit tests whose automated logic for calculating
the expected revision mirrored the actual implementation.
Thus, the unit tests have also been refactored to take in
hardcoded expected values to ensure that the assertions are
foolproof around validating expected revisions.

Change-Id: I7aacb4792f6b2dfc08d3a7bb4c3f18bbcfc95b8a
2018-10-25 15:38:28 -04:00
Zuul 62d0faf2ef Merge "docs: Rename downstream-specific terms to generic ones" 2018-10-25 05:24:19 +00:00
Felipe Monteiro 3b419c3bc4 docs: Add docstring information for pegleg.config
Pegleg's config module is used for basically keeping track
of CLI context information within global memory namespace.
None of the functions therein are properly documented which
can lead to confusion. This patch set adds clarifying
documentation for all the functions.

Change-Id: I93545331ffc3a83b593f654ee90fb6af3d067402
2018-10-23 10:46:45 -04:00
Rick Bartra e3d37db45e Allow the repository clone path to be specified in the CLI
As it currently stands, Pegleg clones site repositories into the /tmp
directory. Even if the site repository already exists in the /tmp
directory it is still cloned there which results in wasted disk space.

This commit allows users to pass in a `clone_path` (-p) option to Pegleg
CLI commands that specify where to clone a site repository. If the clone
path matches the path of an existing repository, then a error message is
logged stating so. If the repository already exists in the clone path, the
user can either specify to use that local repo by passing it as the site
repository or they proceed by passing in a different clone path.

This commit also updates the logic that deletes the copy of the repo that
is created in the temporary folder to also delete the parent folder that
contains the copied repo. This scenario happens when using a local
repository as the site repository.

Addionally, this commit adds a cleanup fixture that removes files and
directories created in the temporary folder by the unit tests.

Change-Id: I1b2943493b8f201f337ea60006c009973dd941b3
2018-10-20 15:21:07 -04:00
Felipe Monteiro baf9d9c5db docs: Rename downstream-specific terms to generic ones
This patch set renames gerrit to git because gerrit is
implementation specific throughout the documentation and
inline docstrings. It also removes instances of aic-clcp-.*
as this, again, is downstream-specific.

Change-Id: I7d5c27f959130a8bd05fbc447d22e6609e498696
2018-10-19 01:38:38 -04:00
Felipe Monteiro ed5251e0e4 fix: Enable Pegleg to support manifest repos like AIAB
This patch set enables Pegleg to support repos like Airship in a
Bottle -- those that have site/ type/ global/ folders nested
under deployment_files/. Very particular logic is needed in order
to handle that. CLI unit tests included for validation/regression.

Change-Id: I9f13f59738599f07329ad3e3274eb4590e8638f9
2018-10-18 19:07:31 +01:00
Felipe Monteiro f71c2e38f8 fix: Support cloning URLs that end in .git
This patch set adds support for passing in URLs ending in .git
via the --site-repository (-r) flag, e.g.:

    pegleg site -r https://github.com/openstack/project.git@foo [...]

Unit test added to validate behavior.

Change-Id: If822ed195f07dc25e0590e42d046730632fdaafd
2018-10-01 21:36:24 -05:00
Felipe Monteiro fde70e9218 fix: Allow -r flag to work with remote repository URLs
This patch set fixes an issue where currently -r flag won't
work with remote repository URLs. site_repository.process_repositories
will blow up with an error (for example):

  Error: https://github.com/openstack/airship-treasuremap/site/
  airship-seaworthy/site-definition.yaml not found. Pegleg must be
  run from the root of a configuration repository.

It is apparent that the URL should not be treated as a local path
which is what is happening behind the scenes with this bug.

Also, CLI unit tests are added to validate the intended behavior.
They will be expanded on in future  patch sets.

Change-Id: I618465841f1e455c8f00f046b3c5d22348b99396
2018-09-27 14:27:47 +01:00
Felipe Monteiro 893ea9f4bb Standardize Pegleg directory structure
This patch set standardizes the Pegleg directory
structure because of the following reasons:

1) src/bin/pegleg is not necessary and only makes
building (e.g. documentation building) and running
of tox targets unnecessarily difficult.
2) src/bin/pegleg is a Java-like standard that
bears no relevance to Python.

Change-Id: I37d39d3d6186b92f8fbfe234221c9e44da48cf10
2018-09-23 10:33:40 -04:00