This change adds the site_type parameter in deployment_data
The value will be taken from respective site-definition.yaml
Change-Id: I8e65b39c73c94caf3ed4cc517520b9577160b20d
It is possible for Pegleg to generate a commit on top of a repo if
the repo is dirty (aka, has uncommited/untracked files). This
effectively makes the repo appear "clean", and also changes the head
of the repo. This can potentially interfere with the deployment_data
generation that analyzes the cleanliness of the repo as well as the
commit at the head of the repo.
This patch set updates the deployment_data generation logic, to be
able to detect Pegleg-generated commits at the head of a repo, and
instead go off of the Pegleg-generated commit's parent commit when
generating the data. It also ensures the repo in the data is always
marked dirty if a Pegleg-generated commit is seen, because the
Pegleg-generated commit would not exist unless the repo was dirty.
Change-Id: I863b3f2f661f11c36ba939ee3023f78733021b96
This patch addresses inconsistent code style and enforces it with a
gate for future submissions.
Separate work will be done in the future to address several of the
PEP8 ignores for docstrings, and attempt to bring the tests directory
to PEP8 compliance.
This patch:
1. Updates .style.yapf to set the knobs desired for YAPF.
2. Updates tox.ini to allow one of the knobs to work.
3. Removes unused code from several __init__.py files.
4. Updates the YAPF version in test-requirements.txt to latest (this
is needed for several knobs to work).
5. Stylistic changes to the python codebase in Pegleg.
6. Updates to tox.ini to run YAPF during PEP8 check.
Change-Id: Ieaa0fdef2b601d01c875d64b840986e54df73abf
There were some inconsistencies around whether string parameter passed
into Git exceptions is called repo_path or repo_url, causing the
exception detail to occasionally not get rendered. This change
standardizes on repo_url.
Change-Id: Ibc78b33cc0e1fcd67820954b715f480a5b1f6496
Previously Pegleg would attempt to create directories using the path
specified directly. This path wasn't always an absolute path,
resulting in errors such as:
File "/opt/pegleg/pegleg/engine/util/files.py", line 265, in dump_all
os.makedirs(os.path.dirname(path), exist_ok=True)
File "/usr/local/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''
This bugfix determines an absolute path before attempting to create
any directories.
Change-Id: I84a0e7bc63d6f56a56b9c5c41de1ede99dfbacc7
1. Adds the passphrases generation capability in Pegleg CLI,
so that pegleg can generation random passwords based on a
specification declared in pegleg/PassphrasesCatalog documents
2. Pegleg also wraps the generated passphrase documents in
pegleg managed documents, and encrypts the data.
3. Adds unit test cases for passphrase generation.
4. Updates pegleg CLI document.
Change-Id: I21d7668788cc24a8e0cc9cb0fb11df97600d0090
This removes all PEP8 ignores and places in default settings for flake8.
Change-Id: I3c4df02dea959dfe58f44e7c0e0ac58078a81abc
Signed-off-by: Tin Lam <tin@irrational.io>
This patch set cleans up the current implementation of pegleg. As
all the git exceptions inconsistently inits or override the message.
This also cleans up a handful of incorrect kwarg keywords, and improve
handling of the exception messages.
Change-Id: I438eb032728c71cbf972c2120a76d06106cb1580
Signed-off-by: Tin Lam <tin@irrational.io>
This patch set replaces raising NotADirectoryError after trying
to parse a repository for its root path (normalize_repo_path in
pegleg.engine.util.git) with a better exception
(exceptions.GitInvalidRepoException). It is better because a
folder can still not be a repo, so raising the first exception
isn't apropos.
Next, this patch set changes where the exception is raised --
which is in normalize_repo_path itself, which is more appropriate
as the function is used in many places and so there should be
intrinsic error handling so as to avoid having to wrap it every
time.
Change-Id: I918d8c293f1140eb80c83499dba2c23af232b79e
This patch set adds additional logic to properly handle parsing
out the revision from an SSH repo url. The issue was being
masked by unit tests whose automated logic for calculating
the expected revision mirrored the actual implementation.
Thus, the unit tests have also been refactored to take in
hardcoded expected values to ensure that the assertions are
foolproof around validating expected revisions.
Change-Id: I7aacb4792f6b2dfc08d3a7bb4c3f18bbcfc95b8a
This patch set adds a callback that can be passed to
@click.argument(site_name, callback=process_repositories_callback)
This makes it easy to have Pegleg automatically clone done repositories
for the specified site and check out all specified references for
each repository.
Change-Id: I7726a8aec1b1689bffbdbfe81204ed2a14ac4b87
As it currently stands, Pegleg clones site repositories into the /tmp
directory. Even if the site repository already exists in the /tmp
directory it is still cloned there which results in wasted disk space.
This commit allows users to pass in a `clone_path` (-p) option to Pegleg
CLI commands that specify where to clone a site repository. If the clone
path matches the path of an existing repository, then a error message is
logged stating so. If the repository already exists in the clone path, the
user can either specify to use that local repo by passing it as the site
repository or they proceed by passing in a different clone path.
This commit also updates the logic that deletes the copy of the repo that
is created in the temporary folder to also delete the parent folder that
contains the copied repo. This scenario happens when using a local
repository as the site repository.
Addionally, this commit adds a cleanup fixture that removes files and
directories created in the temporary folder by the unit tests.
Change-Id: I1b2943493b8f201f337ea60006c009973dd941b3
This patch set enables Pegleg to support repos like Airship in a
Bottle -- those that have site/ type/ global/ folders nested
under deployment_files/. Very particular logic is needed in order
to handle that. CLI unit tests included for validation/regression.
Change-Id: I9f13f59738599f07329ad3e3274eb4590e8638f9
If a repository remote ends with `.git` Pegleg is successfully able
to return the repository name. The same is not true in Pegleg if the
repository name does not end in `.git`. This commit allows Pegleg to
support repository names that do and do not end in `.git`.
Change-Id: I45fd15790677e003af7be584221903551022b7d7
This patch set tidies up collect output file names for cloned
repos. As an example, previously the output file could be named
something like tmp5ak7409aic-clcp-security-manifests.yaml -- after
this change it will be aic-clcp-security-manifests.yaml instead
which is much cleaner.
Pegleg "collect" looks at the last piece in a file path to
determine the filename to write to: [0]
This means that changing the clone path here from something like:
/tmp/tmp54d13yairship-pegleg (which will result in collection
output to tmp54d13yairship-pegleg.yaml)
To:
/tmp/tmp54d13y/airship-pegleg (which will result in collection
output to airship-pegleg.yaml)
[0] 2ea1a55a2d/pegleg/engine/site.py (L71)
Change-Id: I2198e299d392e24d376ccfa53bef57fcabf0d41b
This patch set increases test coverage for some missing git
functions and adds some additional edge case/exception handling
logic to those functions.
Change-Id: I54348599ea959691254c678bc836025cf8df8722
This patch set standardizes the Pegleg directory
structure because of the following reasons:
1) src/bin/pegleg is not necessary and only makes
building (e.g. documentation building) and running
of tox targets unnecessarily difficult.
2) src/bin/pegleg is a Java-like standard that
bears no relevance to Python.
Change-Id: I37d39d3d6186b92f8fbfe234221c9e44da48cf10
Sergiy Markin