Update decrypt secrets to return a list of docs

1. Added the method to decrypt a secret file and return its contents
as a list of documents (instead of printing out the file content).
2. Added clarifications for a encrypt and decrypt commands.

Change-Id: I77bce21be214c880c8413f5e6a2d0c2d1993fc8e
This commit is contained in:
Ahmad Mahmoudi 2018-10-31 11:23:59 -05:00
parent 4a352510d2
commit fb8e6f73ac
4 changed files with 154 additions and 30 deletions

View File

@ -397,6 +397,20 @@ Secrets
A sub-group of site command group, which allows you to perform secrets
level operations for secrets documents of a site.
.. note::
For the CLI commands ``encrypt`` and ``decrypt`` in the ``secrets`` command
group, which encrypt or decrypt site secrets, two environment variables,
``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``, are used to capture the
master passphrase, and the salt needed for encryption and decryption of the
site secrets. The contents of ``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``
are not generated by Pegleg, but are created externally, and set by a
deployment engineers or tooling.
A minimum length of 24 for master passphrases will be checked by all CLI
commands, which use the ``PEGLEG_PASSPHRASE``. All other criteria around
master passphrase strength are assumed to be enforced elsewhere.
::
./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options>
@ -406,26 +420,52 @@ Encrypt
^^^^^^^
Encrypt one site's secrets documents, which have the
metadata.storagePolicy set to encrypted, and wrap them in `pegleg managed
documents <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_.
``metadata.storagePolicy`` set to encrypted, and wrap them in
`Pegleg Managed Documents`_
**Note**: The encrypt command is idempotent. If the command is executed more
than once for a given site, it will skip the files, which are already
encrypted and wrapped in a pegleg managed document, and will only encrypt the
documents not encrypted before.
.. note::
The encrypt command is idempotent. If the command is executed more
than once for a given site, it will skip the files, which are already
encrypted and wrapped in a pegleg managed document, and will only encrypt the
documents not encrypted before.
**site_name** (Required).
Name of the site.
Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
repository folder structure. The ``encrypt`` command looks up the
``site-name`` in the site repository, and searches recursively the
``site_name`` folder structure for secrets files (i.e. files with documents,
whose ``encryptionPolicy`` is set to ``encrypted``), and encrypts the
documents in those files.
**-a / --author** (Required)
Identifier for the program or person who is encrypting the secrets documents.
Author is the identifier for the program or the person, who is encrypting
the secrets documents.
Author is intended to document the entity or the individual, who
encrypts the site secrets documents, mostly for tracking purposes, and is
expected to be leveraged in an operator-specific manner.
For instance the ``author`` can be the "userid" of the person running the
command, or the "application-id" of the application executing the command.
**-s / --save-location** (Optional).
Where to output encrypted and wrapped documents. If omitted, the results
will overwrite the original documents.
Where to output the encrypted and wrapped documents.
.. warning::
If the ``save-location`` parameter is not provided, the encrypted result
documents will overwrite the original ``cleartext`` documents for the site.
The reason for this default behavior, is to ensure that site secrets are
only stored on disk or in any version control system as encrypted.
If the user for any reason wants to avoid overwriting the original
cleartext files, the ``save-location`` parameter will provide the option to
override this default behavior, and forces the encrypt command to write
the encrypted documents in a different location than the original
unencrypted files.
Usage:
@ -457,14 +497,16 @@ Example without optional save location:
Decrypt
^^^^^^^
Unwrap an encrypted secrets document from a `pegleg managed
document <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_,
Unwrap an encrypted secrets document from a `Pegleg Managed Documents`_,
decrypt the encrypted secrets, and dump the cleartext secrets file to
``stdout``.
**site_name** (Required).
Name of the site.
Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
repository folder structure. The ``decrypt`` command also validates that the
``site-name`` exists in the file path, before unwrapping and decrypting the
documents in the ``filename``.
**-f / filename** (Required).
@ -598,3 +640,4 @@ P003 - All repos contain expected directories.
.. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/rendering.html
.. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/validation.html
.. _Pegleg Managed Documents: https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument

View File

@ -74,7 +74,7 @@ def decrypt(file_path, site_name):
:param file_path: Path to the file to be unwrapped and decrypted.
:type file_path: string
:param site_name: The name of the site to search for the file.
:type site_name: string providing the site name
:type site_name: string
"""
LOG.info('Started decrypting...')

View File

@ -34,17 +34,29 @@ ENV_SALT = 'PEGLEG_SALT'
class PeglegSecretManagement():
"""An object to handle operations on of a pegleg managed file."""
def __init__(self, file_path):
def __init__(self, file_path=None, docs=None):
"""
Read the source file and the environment data needed to wrap and
process the file documents as pegleg managed document.
Either of the ``file_path`` or ``docs`` must be
provided.
"""
if all([file_path, docs]) or \
not any([file_path, docs]):
raise ValueError(
'Either `file_path` or `docs` must be specified.')
self.__check_environment()
self.file_path = file_path
self.documents = list()
for doc in files.read(file_path):
self.documents.append(PeglegManagedSecret(doc))
if docs:
for doc in docs:
self.documents.append(PeglegManagedSecret(doc))
else:
self.file_path = file_path
for doc in files.read(file_path):
self.documents.append(PeglegManagedSecret(doc))
self.passphrase = os.environ.get(ENV_PASSPHRASE).encode()
self.salt = os.environ.get(ENV_SALT).encode()
@ -119,9 +131,27 @@ class PeglegSecretManagement():
included in a site secrets file, and print the result to the standard
out."""
yaml.safe_dump_all(
self.get_decrypted_secrets(),
sys.stdout,
explicit_start=True,
explicit_end=True,
default_flow_style=False)
def get_decrypted_secrets(self):
"""
Unwrap and decrypt all the pegleg managed documents in a secrets
file, and return the result as a list of documents.
The method is idempotent. If the method is called on not
encrypted files, or documents inside the file, it will return
the original unwrapped and unencrypted documents.
"""
doc_list = []
for doc in self.documents:
# only decrypt an encrypted document
# do not decrypt already decrypted data
if doc.is_encrypted():
doc.set_secret(
decrypt(doc.get_secret(),
@ -129,9 +159,4 @@ class PeglegSecretManagement():
self.salt).decode())
doc.set_decrypted()
doc_list.append(doc.embedded_document)
yaml.safe_dump_all(
doc_list,
sys.stdout,
explicit_start=True,
explicit_end=True,
default_flow_style=False)
return doc_list

View File

@ -22,12 +22,14 @@ import yaml
from pegleg.engine.util import encryption as crypt
from tests.unit import test_utils
from pegleg.engine import secrets
from pegleg.engine.util.pegleg_managed_document import \
PeglegManagedSecretsDocument
from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE
from pegleg.engine.util.pegleg_secret_management import ENV_SALT
from tests.unit.fixtures import temp_path
from pegleg.engine.util import files
TEST_DATA = """
---
@ -57,8 +59,9 @@ def test_encrypt_and_decrypt():
assert data == dec2
@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'aShortPassphrase',
ENV_SALT: 'MySecretSalt'})
@mock.patch.dict(os.environ, {
ENV_PASSPHRASE:'aShortPassphrase',
ENV_SALT: 'MySecretSalt'})
def test_short_passphrase():
with pytest.raises(click.ClickException,
match=r'.*is not at least 24-character long.*'):
@ -72,9 +75,21 @@ def test_PeglegManagedDocument():
assert doc.is_encrypted() is False
@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
ENV_SALT: 'MySecretSalt'})
def test_encrypt_document():
def test_PeglegSecretManagement():
with pytest.raises(ValueError) as err_info:
PeglegSecretManagement(file_path=None, docs=None)
assert 'Either `file_path` or `docs` must be specified.' in str(
err_info.value)
with pytest.raises(ValueError) as err_info:
PeglegSecretManagement(file_path='file_path', docs=['doc1'])
assert 'Either `file_path` or `docs` must be specified.' in str(
err_info.value)
@mock.patch.dict(os.environ, {
ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
ENV_SALT: 'MySecretSalt'})
def test_encrypt_file():
# write the test data to temp file
test_data = yaml.load(TEST_DATA)
dir = tempfile.mkdtemp()
@ -92,3 +107,44 @@ def test_encrypt_document():
doc = doc_mgr.documents[0]
assert doc.is_encrypted()
assert doc.data['encrypted']['by'] == 'test_author'
@mock.patch.dict(os.environ, {
ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
ENV_SALT: 'MySecretSalt'})
def test_encrypt_decrypt_file(temp_path):
# write the test data to temp file
test_data = list(yaml.safe_load_all(TEST_DATA))
file_path = os.path.join(temp_path, 'secrets_file.yaml')
files.write(file_path, test_data)
save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
doc_mgr = PeglegSecretManagement(file_path=file_path)
doc_mgr.encrypt_secrets(save_path, 'test_author')
# read back the encrypted file
doc_mgr = PeglegSecretManagement(save_path)
decrypted_data = doc_mgr.get_decrypted_secrets()
assert test_data[0]['data'] == decrypted_data[0]['data']
assert test_data[0]['schema'] == decrypted_data[0]['schema']
@mock.patch.dict(os.environ, {
ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
ENV_SALT: 'MySecretSalt'})
def test_decrypt_document(temp_path):
# write the test data to temp file
test_data = list(yaml.safe_load_all(TEST_DATA))
save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
doc_mgr = PeglegSecretManagement(docs=test_data)
doc_mgr.encrypt_secrets(save_path, 'test_author')
# read back the encrypted file
with open(save_path) as stream:
encrypted_data = list(yaml.safe_load_all(stream))
# this time pass a list of dicts to peglegSecretManager
doc_mgr = PeglegSecretManagement(docs=encrypted_data)
decrypted_data = doc_mgr.get_decrypted_secrets()
assert test_data[0]['data'] == decrypted_data[0]['data']
assert test_data[0]['schema'] == decrypted_data[0]['schema']
assert test_data[0]['metadata']['name'] == decrypted_data[0][
'metadata']['name']
assert test_data[0]['metadata']['storagePolicy'] == decrypted_data[0][
'metadata']['storagePolicy']