Debugging pegleg can currently be difficult and the Click CLI does not
easily allow debuggers like pdb or PyCharm to use breakpoints. By moving
all CLI command calls into singular functions, we can easily create an
"if __name__ == '__main__'" entry point to call these functions and
investigate any bugs that may arise.
We also gain the ability to reuse more portions of our code by
refactoring these methods.
Change-Id: Ia9739931273eb6458f82dbb7e702a505ae397ae3
Barbican is being enabled, as such the metadata field should not be
modified by Pegleg. If it says encrypted, then Barbican will
encrypt.
If it says cleartext, Barbican won't. All pegleg needs to do is
decrypt the document prior to bundling it which exists already
without this change.
This reverts commit 2d88f48989.
Change-Id: I8900f910f9816508a8ec5c23932252bb9d1fde09
This patch updates layer for wrapped documents to preserve original
layer. Previously all encrypted documents had site layer.
Update encrypt/decrypt logic when determining global keys.
Update units tests.
Change-Id: I447aeaea08a4514655fcabfc7077b6d4b282e27f
Barbican expects storagePolicy:cleartext when accepting documents
from Pegleg. Once the document is decrypted updated storagePolicy to
cleartext.
During genesis bundle creation, the encrypted documents are
decrypted, then the whole bundle is encrypted. Once the bundle gets
deployed the contained documents should all have the correct storage
policy of cleartext.
As a result, two unit tests are updated to no longer compare the
storagePolicy, as it is expected everything going to Barbican is
cleartext, but in order for Pegleg to know to encrypt the original
document is storagePolicy:encrypted
Change-Id: I5167ce6b3030d143d1ff0b789883529a6557eeca
Pegleg has been using a year old revision of treasuremap for internal
testing using seaworthy. The age of this revision has become a problem
as the certificates in seaworthy expired on August 20th. This change
updates pegleg to the latest version of treasuremap and makes any
necessary path updates for the new version to work with Pegleg tests.
Change-Id: Icea70b8d9bf24c8f85360719945b5899cab6b396
Pytest includes a fixture that can be used to generate temporary
directories. Previously Pegleg had implemented a hombrewed version of a
temporary directory fixture. This change removes the homebrewed version
and replaces it with the tmpdir fixture.
Implement tmpdir fixture in tests
Upgrade all testing packages to use the latest features
Removes unused imports and organizes import lists
Removes mock package requirement and uses unittest.mock, included in
python >3.3
Implements a slightly cleaner method to get proxy info
Change-Id: If66e1cfba858d5fb8948529deb8fb2d32345f630
This patch addresses inconsistent code style and enforces it with a
gate for future submissions.
Separate work will be done in the future to address several of the
PEP8 ignores for docstrings, and attempt to bring the tests directory
to PEP8 compliance.
This patch:
1. Updates .style.yapf to set the knobs desired for YAPF.
2. Updates tox.ini to allow one of the knobs to work.
3. Removes unused code from several __init__.py files.
4. Updates the YAPF version in test-requirements.txt to latest (this
is needed for several knobs to work).
5. Stylistic changes to the python codebase in Pegleg.
6. Updates to tox.ini to run YAPF during PEP8 check.
Change-Id: Ieaa0fdef2b601d01c875d64b840986e54df73abf
This patchset adds support for globally encrypted secrets.
Documents with a "site" layer will be encrypted/decrypted with the
standard PEGLEG_PASSPHRASE and PEGLEG_SALT environment variables.
If any secrets exist for the site with a schema of "global_passphrase"
or "global_salt" their values will be captured and used to decrypt
any secrets that do not belong to "site" layer. If the global keys
do not exist, Pegleg will default to using site keys.
Expected usage:
1. Set site passphrase/salt environment variables
2. Select a global passphrase and salt
3. Use Pegleg's "wrap" command to wrap and encrypt the global keys
4. Encrypt or wrap documents with "global" layer
5. Provide Pegleg path to decrypt
In the case of (4) and (5) Pegleg will determine the correct keys
to use automatically
Change-Id: I5de6d63573619b346fe011628ae21e053e0711f6
Some secrets are being created with undesirable permissions. Upon
inspection it was noticed that in general Pegleg is creating files,
then changing permissions after the fact. This leads to a small
window where the permissions on a file are overly permissive.
This patchset:
1. Sets default umask of 0o027 (640 permissions for files)
2. Explicitly adds the open flag ('r', 'w' etc.) to all open() calls.
3. Replaces sys.stdout.write calls with click.echo() calls to be more
in line with the rest of the project.
4. Re-orders methods that write so that data is always first, and the
path is always second.
5. Updates unit tests.
6. Adds unit tests for testing directory and file permissions.
7. Minor style changes.
Change-Id: I0c154aa311ea371940fd24b0aabf58fffaf1d231
Currently there isn't a uniform or easily expandable way to manage
how Pegleg gets credentials or enforces any complexity on them. This
patchset attempts to address this by:
1. Moving all logic for credentials into config.py
2. Using PeglegSecretManagement as the source of interfacing with
config.py as this code is the entry point for any encryption or
decryption work
3. Remove unnecessary code related to this change
4. Update unit tests
In future patchsets the goal is to use these changes to add in a global
passphrase and salt variable into config.py so that encrypt/decrypt type
commands can be executed one time against a site and intelligently
handle retrieval of global credentials for use with global secrets, site
credentials in the form of environment variables will remain used for
site secrets and will not be overridden by any global operations.
Change-Id: I0b6acd3ef5eab6b1f8931f46544bc53443f5c2c0
Unit tests are warning that yaml.load() without a loader are deprecated.
Switch these calls to yaml.safe_load() to resolve warnings.
Change-Id: Ia8e080fc5317eefe432eee984608df190546530c
This patch handles the case where CA certs or authorities are loaded as
byte strings. It also disables parsing YAML documents with python/object
types directly into (non-dict) Python objects (which is PyYaml's
default behavior), as it creates issues with the PeglegManagedDocument
module.
The patch also fixes a bug where attempting to re-encrypt an already
encrypted file would result in a serialized python object being written
rather than the expected output YAML.
Change-Id: I4b84ee8f9922ae042411e70242ffda4622647e86
This change allows users to specify a directory or file to be decrypted.
Allows directory decryption.
Adds flag to overwrite encrypted file with decrypted data.
Intelligently recognizes paths vs files in CLI input and outputs data
accordingly.
Change-Id: I0d5e77f0eb1adb42165aa9b214aa90a0db0a3131
Decrypt command was previously requiring that specified files have
in their paths the site name. This isn't necessarily always the case
for example we can have global files that need to be decrypted and do
not contain the site name in the filepath, but the site name is
relevant in ensuring based on the site-definition.yaml file that
pegleg uses the correct revision of the global repository.
The end result should be that when decrypting a file, we specify the
site name, pegleg ensures we're on correct revisions of the repos
and if the file exists, decrypt and print to stdout
This patch addresses this by:
1. Updating pegleg.engine.secrets.decrypt to no longer require a
site name.
2. Updating pegleg.cli.decrypt to no longer pass a site name to
pegleg.engine.secrets.decrypt
3. Updating documentation for CLI.
4. Updating unit tests for CLI and secrets.
Change-Id: Ia97518b06a58b069a4d6c0b8d68a37f45e5d31bb
This patch:
1. Allows user to change valid duration of newly generated certs
default=1yr
2. Allows user to check certs that are expiring soon default=60d
Change-Id: Ia5c87a0c52b39b778f425599fa215fb67147c65b
This patch:
1. Sets the salt in config when running genesis bundle
2. Updates the genesis bundle CLI method
3. Adds exception types for credentials
4. Updates unit tests to be compliant with new exceptions
Change-Id: I8869f897e2c25b98c30eaa6be52356aae4ac63b6
1. Add support to pegleg to generate a passphrase from CLI
2. Update unit test to ensure encryption/decryption supports passphrase rotation
3. Update order of import statements to satisfy pep8
4. Add unit test for CLI passphrase generation
5. Resolve merge conflicts via rebase
Change-Id: I5cb9e41b2f0fac2451bd2b74f33c48cda417c22d
1. Adds the passphrases generation capability in Pegleg CLI,
so that pegleg can generation random passwords based on a
specification declared in pegleg/PassphrasesCatalog documents
2. Pegleg also wraps the generated passphrase documents in
pegleg managed documents, and encrypts the data.
3. Adds unit test cases for passphrase generation.
4. Updates pegleg CLI document.
Change-Id: I21d7668788cc24a8e0cc9cb0fb11df97600d0090
Ian H. Pittwood