summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorAhmad Mahmoudi <am495p@att.com>2018-10-31 11:23:59 -0500
committerAhmad Mahmoudi <am495p@att.com>2018-11-06 00:22:10 -0600
commitfb8e6f73ac5de16766b2313c285e73d3be6bf372 (patch)
tree941fd989da1aa72d8a3897a973589f9510d2c0bf
parent4a352510d26d9f94d79323ca0c997ea57b1a00a3 (diff)
Update decrypt secrets to return a list of docs
1. Added the method to decrypt a secret file and return its contents as a list of documents (instead of printing out the file content). 2. Added clarifications for a encrypt and decrypt commands. Change-Id: I77bce21be214c880c8413f5e6a2d0c2d1993fc8e
Notes
Notes (review): Code-Review+2: Felipe Monteiro <felipe.monteiro@att.com> Code-Review+2: Bryan Strassner <bryan.strassner@gmail.com> Workflow+1: Bryan Strassner <bryan.strassner@gmail.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Wed, 07 Nov 2018 20:49:49 +0000 Reviewed-on: https://review.openstack.org/614695 Project: openstack/airship-pegleg Branch: refs/heads/master
-rw-r--r--doc/source/cli/cli.rst69
-rw-r--r--pegleg/engine/secrets.py2
-rw-r--r--pegleg/engine/util/pegleg_secret_management.py45
-rw-r--r--tests/unit/engine/test_encryption.py68
4 files changed, 154 insertions, 30 deletions
diff --git a/doc/source/cli/cli.rst b/doc/source/cli/cli.rst
index 192cc95..6bcada7 100644
--- a/doc/source/cli/cli.rst
+++ b/doc/source/cli/cli.rst
@@ -397,6 +397,20 @@ Secrets
397A sub-group of site command group, which allows you to perform secrets 397A sub-group of site command group, which allows you to perform secrets
398level operations for secrets documents of a site. 398level operations for secrets documents of a site.
399 399
400.. note::
401
402 For the CLI commands ``encrypt`` and ``decrypt`` in the ``secrets`` command
403 group, which encrypt or decrypt site secrets, two environment variables,
404 ``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``, are used to capture the
405 master passphrase, and the salt needed for encryption and decryption of the
406 site secrets. The contents of ``PEGLEG_PASSPHRASE``, and ``PEGLEG_SALT``
407 are not generated by Pegleg, but are created externally, and set by a
408 deployment engineers or tooling.
409
410 A minimum length of 24 for master passphrases will be checked by all CLI
411 commands, which use the ``PEGLEG_PASSPHRASE``. All other criteria around
412 master passphrase strength are assumed to be enforced elsewhere.
413
400:: 414::
401 415
402 ./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options> 416 ./pegleg.sh site -r <site_repo> -e <extra_repo> secrets <command> <options>
@@ -406,26 +420,52 @@ Encrypt
406^^^^^^^ 420^^^^^^^
407 421
408Encrypt one site's secrets documents, which have the 422Encrypt one site's secrets documents, which have the
409metadata.storagePolicy set to encrypted, and wrap them in `pegleg managed 423``metadata.storagePolicy`` set to encrypted, and wrap them in
410documents <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_. 424`Pegleg Managed Documents`_
425
426.. note::
411 427
412**Note**: The encrypt command is idempotent. If the command is executed more 428 The encrypt command is idempotent. If the command is executed more
413than once for a given site, it will skip the files, which are already 429 than once for a given site, it will skip the files, which are already
414encrypted and wrapped in a pegleg managed document, and will only encrypt the 430 encrypted and wrapped in a pegleg managed document, and will only encrypt the
415documents not encrypted before. 431 documents not encrypted before.
416 432
417**site_name** (Required). 433**site_name** (Required).
418 434
419Name of the site. 435Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
436repository folder structure. The ``encrypt`` command looks up the
437``site-name`` in the site repository, and searches recursively the
438``site_name`` folder structure for secrets files (i.e. files with documents,
439whose ``encryptionPolicy`` is set to ``encrypted``), and encrypts the
440documents in those files.
420 441
421**-a / --author** (Required) 442**-a / --author** (Required)
422 443
423Identifier for the program or person who is encrypting the secrets documents. 444Author is the identifier for the program or the person, who is encrypting
445the secrets documents.
446Author is intended to document the entity or the individual, who
447encrypts the site secrets documents, mostly for tracking purposes, and is
448expected to be leveraged in an operator-specific manner.
449For instance the ``author`` can be the "userid" of the person running the
450command, or the "application-id" of the application executing the command.
424 451
425**-s / --save-location** (Optional). 452**-s / --save-location** (Optional).
426 453
427Where to output encrypted and wrapped documents. If omitted, the results 454Where to output the encrypted and wrapped documents.
428will overwrite the original documents. 455
456.. warning::
457
458 If the ``save-location`` parameter is not provided, the encrypted result
459 documents will overwrite the original ``cleartext`` documents for the site.
460 The reason for this default behavior, is to ensure that site secrets are
461 only stored on disk or in any version control system as encrypted.
462
463 If the user for any reason wants to avoid overwriting the original
464 cleartext files, the ``save-location`` parameter will provide the option to
465 override this default behavior, and forces the encrypt command to write
466 the encrypted documents in a different location than the original
467 unencrypted files.
468
429 469
430Usage: 470Usage:
431 471
@@ -457,14 +497,16 @@ Example without optional save location:
457Decrypt 497Decrypt
458^^^^^^^ 498^^^^^^^
459 499
460Unwrap an encrypted secrets document from a `pegleg managed 500Unwrap an encrypted secrets document from a `Pegleg Managed Documents`_,
461document <https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument>`_,
462decrypt the encrypted secrets, and dump the cleartext secrets file to 501decrypt the encrypted secrets, and dump the cleartext secrets file to
463``stdout``. 502``stdout``.
464 503
465**site_name** (Required). 504**site_name** (Required).
466 505
467Name of the site. 506Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
507repository folder structure. The ``decrypt`` command also validates that the
508``site-name`` exists in the file path, before unwrapping and decrypting the
509documents in the ``filename``.
468 510
469**-f / filename** (Required). 511**-f / filename** (Required).
470 512
@@ -598,3 +640,4 @@ P003 - All repos contain expected directories.
598 640
599.. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/rendering.html 641.. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/rendering.html
600.. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/validation.html 642.. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/validation.html
643.. _Pegleg Managed Documents: https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument \ No newline at end of file
diff --git a/pegleg/engine/secrets.py b/pegleg/engine/secrets.py
index 61b5e6e..4812ea1 100644
--- a/pegleg/engine/secrets.py
+++ b/pegleg/engine/secrets.py
@@ -74,7 +74,7 @@ def decrypt(file_path, site_name):
74 :param file_path: Path to the file to be unwrapped and decrypted. 74 :param file_path: Path to the file to be unwrapped and decrypted.
75 :type file_path: string 75 :type file_path: string
76 :param site_name: The name of the site to search for the file. 76 :param site_name: The name of the site to search for the file.
77 :type site_name: string providing the site name 77 :type site_name: string
78 """ 78 """
79 79
80 LOG.info('Started decrypting...') 80 LOG.info('Started decrypting...')
diff --git a/pegleg/engine/util/pegleg_secret_management.py b/pegleg/engine/util/pegleg_secret_management.py
index 313bdff..0937e2a 100644
--- a/pegleg/engine/util/pegleg_secret_management.py
+++ b/pegleg/engine/util/pegleg_secret_management.py
@@ -34,17 +34,29 @@ ENV_SALT = 'PEGLEG_SALT'
34class PeglegSecretManagement(): 34class PeglegSecretManagement():
35 """An object to handle operations on of a pegleg managed file.""" 35 """An object to handle operations on of a pegleg managed file."""
36 36
37 def __init__(self, file_path): 37 def __init__(self, file_path=None, docs=None):
38 """ 38 """
39 Read the source file and the environment data needed to wrap and 39 Read the source file and the environment data needed to wrap and
40 process the file documents as pegleg managed document. 40 process the file documents as pegleg managed document.
41 Either of the ``file_path`` or ``docs`` must be
42 provided.
41 """ 43 """
42 44
45 if all([file_path, docs]) or \
46 not any([file_path, docs]):
47 raise ValueError(
48 'Either `file_path` or `docs` must be specified.')
49
43 self.__check_environment() 50 self.__check_environment()
44 self.file_path = file_path 51 self.file_path = file_path
45 self.documents = list() 52 self.documents = list()
46 for doc in files.read(file_path): 53 if docs:
47 self.documents.append(PeglegManagedSecret(doc)) 54 for doc in docs:
55 self.documents.append(PeglegManagedSecret(doc))
56 else:
57 self.file_path = file_path
58 for doc in files.read(file_path):
59 self.documents.append(PeglegManagedSecret(doc))
48 60
49 self.passphrase = os.environ.get(ENV_PASSPHRASE).encode() 61 self.passphrase = os.environ.get(ENV_PASSPHRASE).encode()
50 self.salt = os.environ.get(ENV_SALT).encode() 62 self.salt = os.environ.get(ENV_SALT).encode()
@@ -119,9 +131,27 @@ class PeglegSecretManagement():
119 included in a site secrets file, and print the result to the standard 131 included in a site secrets file, and print the result to the standard
120 out.""" 132 out."""
121 133
134 yaml.safe_dump_all(
135 self.get_decrypted_secrets(),
136 sys.stdout,
137 explicit_start=True,
138 explicit_end=True,
139 default_flow_style=False)
140
141 def get_decrypted_secrets(self):
142 """
143 Unwrap and decrypt all the pegleg managed documents in a secrets
144 file, and return the result as a list of documents.
145
146 The method is idempotent. If the method is called on not
147 encrypted files, or documents inside the file, it will return
148 the original unwrapped and unencrypted documents.
149
150 """
151
122 doc_list = [] 152 doc_list = []
123 for doc in self.documents: 153 for doc in self.documents:
124 # only decrypt an encrypted document 154 # do not decrypt already decrypted data
125 if doc.is_encrypted(): 155 if doc.is_encrypted():
126 doc.set_secret( 156 doc.set_secret(
127 decrypt(doc.get_secret(), 157 decrypt(doc.get_secret(),
@@ -129,9 +159,4 @@ class PeglegSecretManagement():
129 self.salt).decode()) 159 self.salt).decode())
130 doc.set_decrypted() 160 doc.set_decrypted()
131 doc_list.append(doc.embedded_document) 161 doc_list.append(doc.embedded_document)
132 yaml.safe_dump_all( 162 return doc_list
133 doc_list,
134 sys.stdout,
135 explicit_start=True,
136 explicit_end=True,
137 default_flow_style=False)
diff --git a/tests/unit/engine/test_encryption.py b/tests/unit/engine/test_encryption.py
index 5967efc..a5869e8 100644
--- a/tests/unit/engine/test_encryption.py
+++ b/tests/unit/engine/test_encryption.py
@@ -22,12 +22,14 @@ import yaml
22 22
23from pegleg.engine.util import encryption as crypt 23from pegleg.engine.util import encryption as crypt
24from tests.unit import test_utils 24from tests.unit import test_utils
25from pegleg.engine import secrets
26from pegleg.engine.util.pegleg_managed_document import \ 25from pegleg.engine.util.pegleg_managed_document import \
27 PeglegManagedSecretsDocument 26 PeglegManagedSecretsDocument
28from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement 27from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
29from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE 28from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE
30from pegleg.engine.util.pegleg_secret_management import ENV_SALT 29from pegleg.engine.util.pegleg_secret_management import ENV_SALT
30from tests.unit.fixtures import temp_path
31from pegleg.engine.util import files
32
31 33
32TEST_DATA = """ 34TEST_DATA = """
33--- 35---
@@ -57,8 +59,9 @@ def test_encrypt_and_decrypt():
57 assert data == dec2 59 assert data == dec2
58 60
59 61
60@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'aShortPassphrase', 62@mock.patch.dict(os.environ, {
61 ENV_SALT: 'MySecretSalt'}) 63 ENV_PASSPHRASE:'aShortPassphrase',
64 ENV_SALT: 'MySecretSalt'})
62def test_short_passphrase(): 65def test_short_passphrase():
63 with pytest.raises(click.ClickException, 66 with pytest.raises(click.ClickException,
64 match=r'.*is not at least 24-character long.*'): 67 match=r'.*is not at least 24-character long.*'):
@@ -72,9 +75,21 @@ def test_PeglegManagedDocument():
72 assert doc.is_encrypted() is False 75 assert doc.is_encrypted() is False
73 76
74 77
75@mock.patch.dict(os.environ, {ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC', 78def test_PeglegSecretManagement():
76 ENV_SALT: 'MySecretSalt'}) 79 with pytest.raises(ValueError) as err_info:
77def test_encrypt_document(): 80 PeglegSecretManagement(file_path=None, docs=None)
81 assert 'Either `file_path` or `docs` must be specified.' in str(
82 err_info.value)
83 with pytest.raises(ValueError) as err_info:
84 PeglegSecretManagement(file_path='file_path', docs=['doc1'])
85 assert 'Either `file_path` or `docs` must be specified.' in str(
86 err_info.value)
87
88
89@mock.patch.dict(os.environ, {
90 ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
91 ENV_SALT: 'MySecretSalt'})
92def test_encrypt_file():
78 # write the test data to temp file 93 # write the test data to temp file
79 test_data = yaml.load(TEST_DATA) 94 test_data = yaml.load(TEST_DATA)
80 dir = tempfile.mkdtemp() 95 dir = tempfile.mkdtemp()
@@ -92,3 +107,44 @@ def test_encrypt_document():
92 doc = doc_mgr.documents[0] 107 doc = doc_mgr.documents[0]
93 assert doc.is_encrypted() 108 assert doc.is_encrypted()
94 assert doc.data['encrypted']['by'] == 'test_author' 109 assert doc.data['encrypted']['by'] == 'test_author'
110
111
112@mock.patch.dict(os.environ, {
113 ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
114 ENV_SALT: 'MySecretSalt'})
115def test_encrypt_decrypt_file(temp_path):
116 # write the test data to temp file
117 test_data = list(yaml.safe_load_all(TEST_DATA))
118 file_path = os.path.join(temp_path, 'secrets_file.yaml')
119 files.write(file_path, test_data)
120 save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
121 doc_mgr = PeglegSecretManagement(file_path=file_path)
122 doc_mgr.encrypt_secrets(save_path, 'test_author')
123 # read back the encrypted file
124 doc_mgr = PeglegSecretManagement(save_path)
125 decrypted_data = doc_mgr.get_decrypted_secrets()
126 assert test_data[0]['data'] == decrypted_data[0]['data']
127 assert test_data[0]['schema'] == decrypted_data[0]['schema']
128
129
130@mock.patch.dict(os.environ, {
131 ENV_PASSPHRASE:'ytrr89erARAiPE34692iwUMvWqqBvC',
132 ENV_SALT: 'MySecretSalt'})
133def test_decrypt_document(temp_path):
134 # write the test data to temp file
135 test_data = list(yaml.safe_load_all(TEST_DATA))
136 save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
137 doc_mgr = PeglegSecretManagement(docs=test_data)
138 doc_mgr.encrypt_secrets(save_path, 'test_author')
139 # read back the encrypted file
140 with open(save_path) as stream:
141 encrypted_data = list(yaml.safe_load_all(stream))
142 # this time pass a list of dicts to peglegSecretManager
143 doc_mgr = PeglegSecretManagement(docs=encrypted_data)
144 decrypted_data = doc_mgr.get_decrypted_secrets()
145 assert test_data[0]['data'] == decrypted_data[0]['data']
146 assert test_data[0]['schema'] == decrypted_data[0]['schema']
147 assert test_data[0]['metadata']['name'] == decrypted_data[0][
148 'metadata']['name']
149 assert test_data[0]['metadata']['storagePolicy'] == decrypted_data[0][
150 'metadata']['storagePolicy']