summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorpallav <pallavgupta84@gmail.com>2018-09-26 18:50:31 +0530
committerLev Morgan <morgan.lev@gmail.com>2019-01-29 16:24:31 -0600
commitb79d5b7a988c62553bd674d0aa9688d412473426 (patch)
treed5bab1686480853c58c709d81e6f97cd11067396
parent1de8d5b68f4e0ca892a6cc5f89c568a460388900 (diff)
CLI capability to generate and encrypt passphrases
1. Adds the passphrases generation capability in Pegleg CLI, so that pegleg can generation random passwords based on a specification declared in pegleg/PassphrasesCatalog documents 2. Pegleg also wraps the generated passphrase documents in pegleg managed documents, and encrypts the data. 3. Adds unit test cases for passphrase generation. 4. Updates pegleg CLI document. Change-Id: I21d7668788cc24a8e0cc9cb0fb11df97600d0090
Notes
Notes (review): Code-Review+2: Matt McEuen <matt.mceuen@att.com> Code-Review+1: Alexander Hughes <Alexander.Hughes@pm.me> Code-Review+2: Scott Hussey <sthussey@att.com> Workflow+1: Scott Hussey <sthussey@att.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Thu, 31 Jan 2019 16:39:43 +0000 Reviewed-on: https://review.openstack.org/605425 Project: openstack/airship-pegleg Branch: refs/heads/master
-rw-r--r--doc/source/cli/cli.rst89
-rw-r--r--doc/source/exceptions.rst13
-rw-r--r--pegleg/cli.py109
-rw-r--r--pegleg/config.py2
-rw-r--r--pegleg/engine/catalog/pki_generator.py13
-rw-r--r--pegleg/engine/catalog/pki_utility.py3
-rw-r--r--pegleg/engine/catalogs/__init__.py0
-rw-r--r--pegleg/engine/catalogs/base_catalog.py84
-rw-r--r--pegleg/engine/catalogs/passphrase_catalog.py88
-rw-r--r--pegleg/engine/exceptions.py11
-rw-r--r--pegleg/engine/generators/__init__.py0
-rw-r--r--pegleg/engine/generators/base_generator.py79
-rw-r--r--pegleg/engine/generators/passpharase_generator.py90
-rw-r--r--pegleg/engine/secrets.py62
-rw-r--r--pegleg/engine/util/encryption.py2
-rw-r--r--pegleg/engine/util/git.py8
-rw-r--r--pegleg/engine/util/passphrase.py33
-rw-r--r--pegleg/engine/util/pegleg_managed_document.py60
-rw-r--r--pegleg/engine/util/pegleg_secret_management.py60
-rw-r--r--requirements.txt1
-rw-r--r--setup.py3
-rw-r--r--site_yamls/site/passphrase-catalog.yaml212
-rw-r--r--tests/unit/engine/test_generate_passphrases.py178
-rw-r--r--tests/unit/engine/test_secrets.py66
-rw-r--r--tests/unit/test_cli.py31
25 files changed, 1186 insertions, 111 deletions
diff --git a/doc/source/cli/cli.rst b/doc/source/cli/cli.rst
index 02dcfda..16fea95 100644
--- a/doc/source/cli/cli.rst
+++ b/doc/source/cli/cli.rst
@@ -613,6 +613,90 @@ Example:
613 /opt/security-manifests/site/site1/passwords/password1.yaml 613 /opt/security-manifests/site/site1/passwords/password1.yaml
614 614
615 615
616generate
617^^^^^^^^
618A sub-group of secrets command group, which allows you to auto-generate
619secrets documents of a site.
620
621.. note::
622
623 The types of documents that pegleg cli generates are
624 passphrases, certificate authorities, certificates and keys. Passphrases are
625 declared in a new ``pegleg/PassphraseCatalog/v1`` document, while CAs,
626 certificates, and keys are declared in the ``pegleg/PKICatalog/v1``.
627
628 The ``pegleg/PKICatalog/v1`` schema is identical with the existing
629 ``promenade/PKICatalog/v1``, promenade currently uses to generate the site
630 CAs, certificates, and keys.
631
632 The ``pegleg/PassphraseCatalog/v1`` schema is specified in
633 `Pegleg Passphrase Catalog`_
634
635::
636
637./pegleg.sh site -r <site_repo> -e <extra_repo> secrets generate <command> <options>
638
639passphrases
640"""""""""""
641Generates, wraps and encrypts passphrase documents specified in the
642``pegleg/PassphraseCatalog/v1`` document for a site. The site name, and the
643directory to store the generated documents are provided by the
644``site_name``, and the ``save_location`` command line parameters respectively.
645The generated passphrases are stored in:
646
647::
648
649<save_location>/site/<site_name>/passphrases/<passphrase_name.yaml>
650
651The schema for the generated passphrases is defined in
652`Pegleg Managed Documents`_
653
654**site_name** (Required).
655
656Name of the ``site``. The ``site_name`` must match a ``site`` name in the site
657repository folder structure. The ``generate`` command looks up the
658``site-name``, and searches recursively the ``site_name`` folder structure
659in the site repository for ``pegleg/PassphraseCatalog/v1`` documents. Then it
660parses the passphrase catalog documents it found, and generates one passphrase
661document for each passphrase ``document_name`` declared in the site passphrase
662catalog.
663
664**-a / --author** (Required)
665
666
667``Author`` is intended to document the application or the individual, who
668generates the site passphrase documents, mostly for tracking purposes. It
669is expected to be leveraged in an operator-specific manner.
670For instance the ``author`` can be the "userid" of the person running the
671command, or the "application-id" of the application executing the command.
672
673**-s / --save-location** (Required).
674
675Where to output generated passphrase documents. The passphrase documents
676are placed in the following folder structure under ``save_location``:
677
678::
679
680<save_location>/site/<site_name>/secrets/passphrases/<passphrase_name.yaml>
681
682Usage:
683
684::
685
686 ./pegleg.sh site <options> secrets generate passphrases <site_name> -a
687 <author_id> -s <save_location>
688
689Example
690""""""""
691
692::
693
694 ./pegleg.sh site -r /opt/site-manifests \
695 -e global=/opt/manifests \
696 -e secrets=/opt/security-manifests \
697 secrets generate passphrases <site_name> -a <author_id> -s /workspace
698
699
616CLI Repository Overrides 700CLI Repository Overrides
617======================== 701========================
618 702
@@ -719,8 +803,9 @@ Where mandatory encrypted schema type is one of:
719P002 - Deckhand rendering is expected to complete without errors. 803P002 - Deckhand rendering is expected to complete without errors.
720P003 - All repos contain expected directories. 804P003 - All repos contain expected directories.
721 805
722.. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/users/rendering.html 806.. _Deckhand: https://airship-deckhand.readthedocs.io/en/latest/rendering.html
723.. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/overview.html#validation 807.. _Deckhand Validations: https://airship-deckhand.readthedocs.io/en/latest/validation.html
724.. _Pegleg Managed Documents: https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument 808.. _Pegleg Managed Documents: https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
725.. _Shipyard: https://github.com/openstack/airship-shipyard 809.. _Shipyard: https://github.com/openstack/airship-shipyard
726.. _CLI documentation: https://airship-shipyard.readthedocs.io/en/latest/CLI.html#openstack-keystone-authorization-environment-variables 810.. _CLI documentation: https://airship-shipyard.readthedocs.io/en/latest/CLI.html#openstack-keystone-authorization-environment-variables
811.. _Pegleg Passphrase Catalog: https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#document-generation
diff --git a/doc/source/exceptions.rst b/doc/source/exceptions.rst
index ef2f1ad..06585f8 100644
--- a/doc/source/exceptions.rst
+++ b/doc/source/exceptions.rst
@@ -71,3 +71,16 @@ PKI Exceptions
71 :members: 71 :members:
72 :show-inheritance: 72 :show-inheritance:
73 :undoc-members: 73 :undoc-members:
74
75Passphrase Exceptions
76---------------------
77
78.. autoexception:: pegleg.engine.exceptions.PassphraseSchemaNotFoundException
79 :members:
80 :show-inheritance:
81 :undoc-members:
82
83.. autoexception:: pegleg.engine.exceptions.PassphraseCatalogNotFoundException
84 :members:
85 :show-inheritance:
86 :undoc-members:
diff --git a/pegleg/cli.py b/pegleg/cli.py
index 7897802..603e10f 100644
--- a/pegleg/cli.py
+++ b/pegleg/cli.py
@@ -57,17 +57,17 @@ EXTRA_REPOSITORY_OPTION = click.option(
57 'extra_repositories', 57 'extra_repositories',
58 multiple=True, 58 multiple=True,
59 help='Path or URL of additional repositories. These should be named per ' 59 help='Path or URL of additional repositories. These should be named per '
60 'the site-definition file, e.g. -e global=/opt/global -e ' 60 'the site-definition file, e.g. -e global=/opt/global -e '
61 'secrets=/opt/secrets. By default, the revision specified in the ' 61 'secrets=/opt/secrets. By default, the revision specified in the '
62 'site-definition for the site will be leveraged but can be overridden ' 62 'site-definition for the site will be leveraged but can be '
63 'using -e global=/opt/global@revision.') 63 'overridden using -e global=/opt/global@revision.')
64 64
65REPOSITORY_KEY_OPTION = click.option( 65REPOSITORY_KEY_OPTION = click.option(
66 '-k', 66 '-k',
67 '--repo-key', 67 '--repo-key',
68 'repo_key', 68 'repo_key',
69 help='The SSH public key to use when cloning remote authenticated ' 69 help='The SSH public key to use when cloning remote authenticated '
70 'repositories.') 70 'repositories.')
71 71
72REPOSITORY_USERNAME_OPTION = click.option( 72REPOSITORY_USERNAME_OPTION = click.option(
73 '-u', 73 '-u',
@@ -83,13 +83,15 @@ REPOSITORY_CLONE_PATH_OPTION = click.option(
83 '--clone-path', 83 '--clone-path',
84 'clone_path', 84 'clone_path',
85 help='The path where the repo will be cloned. By default the repo will be ' 85 help='The path where the repo will be cloned. By default the repo will be '
86 'cloned to the /tmp path. If this option is included and the repo already ' 86 'cloned to the /tmp path. If this option is '
87 'exists, then the repo will not be cloned again and the user must specify ' 87 'included and the repo already '
88 'a new clone path or pass in the local copy of the repository as the site ' 88 'exists, then the repo will not be cloned again and the '
89 'repository. Suppose the repo name is airship-treasuremap and the clone ' 89 'user must specify a new clone path or pass in the local copy '
90 'path is /tmp/mypath then the following directory is created ' 90 'of the repository as the site repository. Suppose the repo '
91 '/tmp/mypath/airship-treasuremap which will contain the contents of the ' 91 'name is airship-treasuremap and the clone path is '
92 'repo') 92 '/tmp/mypath then the following directory is '
93 'created /tmp/mypath/airship-treasuremap '
94 'which will contain the contents of the repo')
93 95
94ALLOW_MISSING_SUBSTITUTIONS_OPTION = click.option( 96ALLOW_MISSING_SUBSTITUTIONS_OPTION = click.option(
95 '-f', 97 '-f',
@@ -106,7 +108,7 @@ EXCLUDE_LINT_OPTION = click.option(
106 'exclude_lint', 108 'exclude_lint',
107 multiple=True, 109 multiple=True,
108 help='Excludes specified linting checks. Warnings will still be issued. ' 110 help='Excludes specified linting checks. Warnings will still be issued. '
109 '-w takes priority over -x.') 111 '-w takes priority over -x.')
110 112
111WARN_LINT_OPTION = click.option( 113WARN_LINT_OPTION = click.option(
112 '-w', 114 '-w',
@@ -225,7 +227,7 @@ def site(*, site_repository, clone_path, extra_repositories, repo_key,
225 '--save-location', 227 '--save-location',
226 'save_location', 228 'save_location',
227 help='Directory to output the complete site definition. Created ' 229 help='Directory to output the complete site definition. Created '
228 'automatically if it does not already exist.') 230 'automatically if it does not already exist.')
229@click.option( 231@click.option(
230 '--validate', 232 '--validate',
231 'validate', 233 'validate',
@@ -241,7 +243,7 @@ def site(*, site_repository, clone_path, extra_repositories, repo_key,
241 'exclude_lint', 243 'exclude_lint',
242 multiple=True, 244 multiple=True,
243 help='Excludes specified linting checks. Warnings will still be issued. ' 245 help='Excludes specified linting checks. Warnings will still be issued. '
244 '-w takes priority over -x.') 246 '-w takes priority over -x.')
245@click.option( 247@click.option(
246 '-w', 248 '-w',
247 '--warn', 249 '--warn',
@@ -344,8 +346,8 @@ def lint_site(*, fail_on_missing_sub_src, exclude_lint, warn_lint, site_name):
344@click.option( 346@click.option(
345 '--context-marker', 347 '--context-marker',
346 help='Specifies a UUID (8-4-4-4-12 format) that will be used to correlate ' 348 help='Specifies a UUID (8-4-4-4-12 format) that will be used to correlate '
347 'logs, transactions, etc. in downstream activities triggered by this ' 349 'logs, transactions, etc. in downstream activities triggered by this '
348 'interaction ', 350 'interaction ',
349 required=False, 351 required=False,
350 type=click.UUID) 352 type=click.UUID)
351@SITE_REPOSITORY_ARGUMENT 353@SITE_REPOSITORY_ARGUMENT
@@ -375,24 +377,26 @@ def upload(ctx, *, os_project_domain_name,
375 click.echo(ShipyardHelper(ctx).upload_documents()) 377 click.echo(ShipyardHelper(ctx).upload_documents())
376 378
377 379
378@site.group(name='secrets', help='Commands to manage site secrets documents') 380@site.group(
381 name='secrets',
382 help='Commands to manage site secrets documents')
379def secrets(): 383def secrets():
380 pass 384 pass
381 385
382 386
383@secrets.command( 387@secrets.command(
384 'generate-pki', 388 'generate-pki',
385 help=""" 389 help='Generate certificates and keys according to all PKICatalog '
386Generate certificates and keys according to all PKICatalog documents in the 390 'documents in the site. Regenerating certificates can be '
387site. Regenerating certificates can be accomplished by re-running this command. 391 'accomplished by re-running this command.')
388""")
389@click.option( 392@click.option(
390 '-a', 393 '-a',
391 '--author', 394 '--author',
392 'author', 395 'author',
393 help="""Identifying name of the author generating new certificates. Used 396 help='Identifying name of the author generating new certificates. Used'
394for tracking provenance information in the PeglegManagedDocuments. An attempt 397 'for tracking provenance information in the PeglegManagedDocuments. '
395is made to automatically determine this value, but should be provided.""") 398 'An attempt is made to automatically determine this value, '
399 'but should be provided.')
396@click.argument('site_name') 400@click.argument('site_name')
397def generate_pki(site_name, author): 401def generate_pki(site_name, author):
398 """Generate certificates, certificate authorities and keypairs for a given 402 """Generate certificates, certificate authorities and keypairs for a given
@@ -442,27 +446,68 @@ def list_types(*, output_stream):
442 engine.type.list_types(output_stream) 446 engine.type.list_types(output_stream)
443 447
444 448
449@secrets.group(
450 name='generate',
451 help='Command group to generate site secrets documents.')
452def generate():
453 pass
454
455
456@generate.command(
457 'passphrases',
458 help='Command to generate site passphrases')
459@click.argument('site_name')
460@click.option(
461 '-s',
462 '--save-location',
463 'save_location',
464 required=True,
465 help='Directory to store the generated site passphrases in. It will '
466 'be created automatically, if it does not already exist. The '
467 'generated, wrapped, and encrypted passphrases files will be saved '
468 'in: <save_location>/site/<site_name>/secrets/passphrases/ '
469 'directory.')
470@click.option(
471 '-a',
472 '--author',
473 'author',
474 required=True,
475 help='Identifier for the program or person who is generating the secrets '
476 'documents')
477@click.option(
478 '-i',
479 '--interactive',
480 'interactive',
481 is_flag=bool,
482 default=False,
483 help='Generate passphrases interactively, not automatically')
484def generate_passphrases(*, site_name, save_location, author, interactive):
485 engine.repository.process_repositories(site_name)
486 engine.secrets.generate_passphrases(site_name, save_location, author,
487 interactive)
488
489
445@secrets.command( 490@secrets.command(
446 'encrypt', 491 'encrypt',
447 help='Command to encrypt and wrap site secrets ' 492 help='Command to encrypt and wrap site secrets '
448 'documents with metadata.storagePolicy set ' 493 'documents with metadata.storagePolicy set '
449 'to encrypted, in pegleg managed documents.') 494 'to encrypted, in pegleg managed documents.')
450@click.option( 495@click.option(
451 '-s', 496 '-s',
452 '--save-location', 497 '--save-location',
453 'save_location', 498 'save_location',
454 default=None, 499 default=None,
455 help='Directory to output the encrypted site secrets files. Created ' 500 help='Directory to output the encrypted site secrets files. Created '
456 'automatically if it does not already exist. ' 501 'automatically if it does not already exist. '
457 'If save_location is not provided, the output encrypted files will ' 502 'If save_location is not provided, the output encrypted files will '
458 'overwrite the original input files (default behavior)') 503 'overwrite the original input files (default behavior)')
459@click.option( 504@click.option(
460 '-a', 505 '-a',
461 '--author', 506 '--author',
462 'author', 507 'author',
463 required=True, 508 required=True,
464 help='Identifier for the program or person who is encrypting the secrets ' 509 help='Identifier for the program or person who is encrypting the secrets '
465 'documents') 510 'documents')
466@click.argument('site_name') 511@click.argument('site_name')
467def encrypt(*, save_location, author, site_name): 512def encrypt(*, save_location, author, site_name):
468 engine.repository.process_repositories(site_name, overwrite_existing=True) 513 engine.repository.process_repositories(site_name, overwrite_existing=True)
@@ -474,7 +519,7 @@ def encrypt(*, save_location, author, site_name):
474@secrets.command( 519@secrets.command(
475 'decrypt', 520 'decrypt',
476 help='Command to unwrap and decrypt one site ' 521 help='Command to unwrap and decrypt one site '
477 'secrets document and print it to stdout.') 522 'secrets document and print it to stdout.')
478@click.option( 523@click.option(
479 '-f', 524 '-f',
480 '--filename', 525 '--filename',
diff --git a/pegleg/config.py b/pegleg/config.py
index 560f437..be007ca 100644
--- a/pegleg/config.py
+++ b/pegleg/config.py
@@ -26,7 +26,7 @@ except NameError:
26 'clone_path': None, 26 'clone_path': None,
27 'site_path': 'site', 27 'site_path': 'site',
28 'site_rev': None, 28 'site_rev': None,
29 'type_path': 'type', 29 'type_path': 'type'
30 } 30 }
31 31
32 32
diff --git a/pegleg/engine/catalog/pki_generator.py b/pegleg/engine/catalog/pki_generator.py
index 7ef26e6..66a3ec8 100644
--- a/pegleg/engine/catalog/pki_generator.py
+++ b/pegleg/engine/catalog/pki_generator.py
@@ -24,8 +24,7 @@ from pegleg.engine.catalog import pki_utility
24from pegleg.engine.common import managed_document as md 24from pegleg.engine.common import managed_document as md
25from pegleg.engine import exceptions 25from pegleg.engine import exceptions
26from pegleg.engine import util 26from pegleg.engine import util
27from pegleg.engine.util.pegleg_managed_document import \ 27from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
28 PeglegManagedSecretsDocument
29 28
30__all__ = ['PKIGenerator'] 29__all__ = ['PKIGenerator']
31 30
@@ -129,8 +128,8 @@ class PKIGenerator(object):
129 if not docs: 128 if not docs:
130 docs = generator(document_name, *args, **kwargs) 129 docs = generator(document_name, *args, **kwargs)
131 else: 130 else:
132 docs = [PeglegManagedSecretsDocument(doc).pegleg_document 131 docs = PeglegSecretManagement(
133 for doc in docs] 132 docs=docs)
134 133
135 # Adding these to output should be idempotent, so we use a dict. 134 # Adding these to output should be idempotent, so we use a dict.
136 135
@@ -215,6 +214,12 @@ class PKIGenerator(object):
215 LOG.debug('Creating secrets path: %s', dir_name) 214 LOG.debug('Creating secrets path: %s', dir_name)
216 os.makedirs(dir_name) 215 os.makedirs(dir_name)
217 216
217 # Encrypt the document
218 document['data']['managedDocument']['metadata']['storagePolicy']\
219 = 'encrypted'
220 document = PeglegSecretManagement(docs=[
221 document]).get_encrypted_secrets()[0][0]
222
218 with open(output_path, 'a') as f: 223 with open(output_path, 'a') as f:
219 # Don't use safe_dump so we can block format certificate 224 # Don't use safe_dump so we can block format certificate
220 # data. 225 # data.
diff --git a/pegleg/engine/catalog/pki_utility.py b/pegleg/engine/catalog/pki_utility.py
index 780370f..28aedbf 100644
--- a/pegleg/engine/catalog/pki_utility.py
+++ b/pegleg/engine/catalog/pki_utility.py
@@ -298,7 +298,8 @@ class PKIUtility(object):
298 'layeringDefinition': { 298 'layeringDefinition': {
299 'abstract': False, 299 'abstract': False,
300 'layer': 'site', 300 'layer': 'site',
301 } 301 },
302 'storagePolicy': 'cleartext'
302 } 303 }
303 wrapped_data = PKIUtility._block_literal( 304 wrapped_data = PKIUtility._block_literal(
304 data, block_strings=block_strings) 305 data, block_strings=block_strings)
diff --git a/pegleg/engine/catalogs/__init__.py b/pegleg/engine/catalogs/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/pegleg/engine/catalogs/__init__.py
diff --git a/pegleg/engine/catalogs/base_catalog.py b/pegleg/engine/catalogs/base_catalog.py
new file mode 100644
index 0000000..01aaa32
--- /dev/null
+++ b/pegleg/engine/catalogs/base_catalog.py
@@ -0,0 +1,84 @@
1# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15from abc import ABC
16import logging
17import os
18import re
19
20from pegleg import config
21from pegleg.engine.exceptions import PassphraseCatalogNotFoundException
22from pegleg.engine.util import definition
23from pegleg.engine.util import git
24
25LOG = logging.getLogger(__name__)
26
27__all__ = ['BaseCatalog']
28
29
30class BaseCatalog(ABC):
31 """Abstract Base Class for all site catalogs."""
32
33 def __init__(self, kind, sitename, documents=None):
34 """
35 Search for site catalog of the specified ``kind`` among the site
36 documents, and capture the catalog common metadata.
37
38 :param str kind: The catalog kind
39 :param str sitename: Name of the environment
40 :param list documents: Optional list of site documents. If not
41 present, the constructor will use the ``site_name` to lookup the list
42 of site documents.
43 """
44 self._documents = documents or definition.documents_for_site(sitename)
45 self._site_name = sitename
46 self._catalog_path = None
47 self._kind = kind
48 self._catalog_docs = list()
49 for document in self._documents:
50 schema = document.get('schema')
51 if schema == 'pegleg/%s/v1' % kind:
52 self._catalog_docs.append(document)
53 elif schema == 'promenade/%s/v1' % kind:
54 LOG.warning('The schema promenade/%s/v1 is deprecated. Use '
55 'pegleg/%s/v1 instead.', kind, kind)
56 self._catalog_docs.append(document)
57
58 @property
59 def site_name(self):
60 return self._site_name
61
62 @property
63 def catalog_path(self):
64 if self._catalog_path is None:
65 self._set_catalog_path()
66 return self._catalog_path
67
68 def _set_catalog_path(self):
69 repo_name = git.repo_url(config.get_site_repo())
70 catalog_name = self._get_document_name('{}.yaml'.format(self._kind))
71 for file_path in definition.site_files(self.site_name):
72 if file_path.endswith(catalog_name) and repo_name in file_path:
73 self._catalog_path = os.path.join(
74 repo_name, file_path.split(repo_name)[1].lstrip('/'))
75 return
76 # Cound not find the Catalog for this generated passphrase
77 # raise an exception.
78 LOG.error('Catalog path: {} was not found in repo: {}'.format(
79 catalog_name, repo_name))
80 raise PassphraseCatalogNotFoundException()
81
82 def _get_document_name(self, name):
83 s1 = re.sub('(.)([A-Z][a-z]+)', r'\1-\2', name)
84 return re.sub('([a-z0-9])([A-Z])', r'\1-\2', s1).lower()
diff --git a/pegleg/engine/catalogs/passphrase_catalog.py b/pegleg/engine/catalogs/passphrase_catalog.py
new file mode 100644
index 0000000..df6308a
--- /dev/null
+++ b/pegleg/engine/catalogs/passphrase_catalog.py
@@ -0,0 +1,88 @@
1# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15import logging
16
17from pegleg.engine.catalogs.base_catalog import BaseCatalog
18from pegleg.engine.exceptions import PassphraseSchemaNotFoundException
19
20LOG = logging.getLogger(__name__)
21KIND = 'PassphraseCatalog'
22P_DOCUMENT_NAME = 'document_name'
23P_LENGTH = 'length'
24P_DESCRIPTION = 'description'
25P_ENCRYPTED = 'encrypted'
26P_CLEARTEXT = 'cleartext'
27P_DEFAULT_LENGTH = 24
28P_DEFAULT_STORAGE_POLICY = 'encrypted'
29
30__all__ = ['PassphraseCatalog']
31
32
33class PassphraseCatalog(BaseCatalog):
34 """Passphrase Catalog class.
35
36 The object containing methods and attributes to ingest and manage the site
37 passphrase catalog documents.
38
39 """
40
41 def __init__(self, sitename, documents=None):
42 """
43 Parse the site passphrase catalog documents and capture the
44 passphrase catalog data.
45
46 :param str sitename: Name of the environment
47 :param list documents: Environment configuration documents
48 :raises PassphraseSchemaNotFoundException: If it cannot find a
49 ``pegleg/passphraseCatalog/v1`` document.
50 """
51 super(PassphraseCatalog, self).__init__(KIND, sitename, documents)
52 if not self._catalog_docs:
53 raise PassphraseSchemaNotFoundException()
54
55 @property
56 def get_passphrase_names(self):
57 """Return the list of passphrases in the catalog."""
58 return (passphrase[P_DOCUMENT_NAME]
59 for catalog in self._catalog_docs
60 for passphrase in catalog['data']['passphrases'])
61
62 def get_length(self, passphrase_name):
63 """
64 Return the length of the ``passphrase_name``. If the catalog
65 does not specify a length for the ``passphrase_name``, return the
66 default passphrase length, 24.
67 """
68
69 for c_doc in self._catalog_docs:
70 for passphrase in c_doc['data']['passphrases']:
71 if passphrase[P_DOCUMENT_NAME] == passphrase_name:
72 return passphrase.get(P_LENGTH, P_DEFAULT_LENGTH)
73
74 def get_storage_policy(self, passphrase_name):
75 """
76 Return the storage policy of the ``passphrase_name``.
77 If the passphrase catalog does not specify a storage policy for
78 this passphrase, return the default storage policy, "encrypted".
79 """
80
81 for c_doc in self._catalog_docs:
82 for passphrase in c_doc['data']['passphrases']:
83 if passphrase[P_DOCUMENT_NAME] == passphrase_name:
84 if P_ENCRYPTED in passphrase and not passphrase[
85 P_ENCRYPTED]:
86 return P_CLEARTEXT
87 else:
88 return P_DEFAULT_STORAGE_POLICY
diff --git a/pegleg/engine/exceptions.py b/pegleg/engine/exceptions.py
index 2539101..e26c718 100644
--- a/pegleg/engine/exceptions.py
+++ b/pegleg/engine/exceptions.py
@@ -75,3 +75,14 @@ class GitInvalidRepoException(PeglegBaseException):
75class IncompletePKIPairError(PeglegBaseException): 75class IncompletePKIPairError(PeglegBaseException):
76 """Exception for incomplete private/public keypair.""" 76 """Exception for incomplete private/public keypair."""
77 message = ("Incomplete keypair set %(kinds)s for name: %(name)s") 77 message = ("Incomplete keypair set %(kinds)s for name: %(name)s")
78
79
80class PassphraseSchemaNotFoundException(PeglegBaseException):
81 """Failed to find schema for Passphrases rendering."""
82 message = ('Could not find Passphrase schema for rendering Passphrases!')
83
84
85class PassphraseCatalogNotFoundException(PeglegBaseException):
86 """Failed to find Catalog for Passphrases generation."""
87 message = ('Could not find the Passphrase Catalog to generate '
88 'the site Passphrases!')
diff --git a/pegleg/engine/generators/__init__.py b/pegleg/engine/generators/__init__.py
new file mode 100644
index 0000000..e69de29
--- /dev/null
+++ b/pegleg/engine/generators/__init__.py
diff --git a/pegleg/engine/generators/base_generator.py b/pegleg/engine/generators/base_generator.py
new file mode 100644
index 0000000..febc3e1
--- /dev/null
+++ b/pegleg/engine/generators/base_generator.py
@@ -0,0 +1,79 @@
1# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15from abc import ABC
16import logging
17import os
18
19from pegleg.engine import util
20
21__all__ = ['BaseGenerator']
22
23LOG = logging.getLogger(__name__)
24
25
26class BaseGenerator(ABC):
27 """
28 Abstract Base Class, providing the common data and methods for all
29 generator classes
30 """
31
32 def __init__(self, sitename, save_location, author=None):
33 """Constructor for ``BaseGenerator``.
34
35 :param str sitename: Name of the environment.
36 :param str save_location: The destination directory to store the
37 generated documents.
38 :param str author: Identifier for the individual or the application,
39 who requests to generate a document.
40 """
41
42 self._sitename = sitename
43 self._documents = util.definition.documents_for_site(sitename)
44 self._save_location = save_location
45 self._author = author
46
47 @staticmethod
48 def generate_doc(kind, name, storage_policy, secret_data):
49 """
50 Generate a document of the specified ``kind``, with the
51 specified ``storage_policy`` for the ``secret_data``.
52
53 :param str kind: Kind of the secret document.
54 :param str name: Name of the secret document
55 :param str storage_policy: Storage policy for the secret data
56 :param str secret_data: The data to be stored in this document.
57 """
58 return {
59 'schema': 'deckhand/{}/v1'.format(kind),
60 'metadata': {
61 'schema': 'metadata/Document/v1',
62 'name': name,
63 'layeringDefinition': {
64 'abstract': False,
65 'layer': 'site',
66 },
67 'storagePolicy': storage_policy,
68 },
69 'data': secret_data,
70 }
71
72 def get_save_path(self, passphrase_name):
73 """Calculate and return the save path of the ``passphrase_name``."""
74 return os.path.abspath(os.path.join(self._save_location,
75 'site',
76 self._sitename,
77 'secrets',
78 self.kind_path,
79 '{}.yaml'.format(passphrase_name)))
diff --git a/pegleg/engine/generators/passpharase_generator.py b/pegleg/engine/generators/passpharase_generator.py
new file mode 100644
index 0000000..c20f7f8
--- /dev/null
+++ b/pegleg/engine/generators/passpharase_generator.py
@@ -0,0 +1,90 @@
1# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15from getpass import getpass
16import logging
17
18from pegleg.engine.catalogs import passphrase_catalog
19from pegleg.engine.catalogs.passphrase_catalog import PassphraseCatalog
20from pegleg.engine.generators.base_generator import BaseGenerator
21from pegleg.engine.util import files
22from pegleg.engine.util.passphrase import Passphrase
23from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
24
25__all__ = ['PassphraseGenerator']
26
27LOG = logging.getLogger(__name__)
28KIND = 'Passphrase'
29KIND_PATH = 'passphrases'
30
31
32class PassphraseGenerator(BaseGenerator):
33 """
34 Generates passphrases for a given environment, specified in a
35 passphrase catalog.
36 """
37
38 def __init__(self, sitename, save_location, author):
39 """Constructor for ``PassphraseGenerator``.
40
41 :param str sitename: Site name for which passphrases are generated.
42 :param str save_location: The base directory to store the generated
43 passphrase documents.
44 :param str author: Identifying name of the author generating new
45 certificates.
46 """
47
48 super(PassphraseGenerator, self).__init__(
49 sitename, save_location, author)
50 self._catalog = PassphraseCatalog(
51 self._sitename, documents=self._documents)
52 self._pass_util = Passphrase()
53
54 def generate(self, interactive=False):
55 """
56 For each passphrase entry in the passphrase catalog, generate a
57 random passphrase string, based on a passphrase specification in the
58 catalog. Create a pegleg managed document, wrap the generated
59 passphrase document in the pegleg managed document, and encrypt the
60 passphrase. Write the wrapped and encrypted document in a file at
61 <repo_name>/site/<site_name>/secrets/passphrases/passphrase_name.yaml.
62 """
63 for p_name in self._catalog.get_passphrase_names:
64 passphrase = None
65 if interactive:
66 passphrase = getpass(
67 prompt="Input passphrase for {}. Leave blank to "
68 "auto-generate:\n".format(p_name))
69 if not passphrase:
70 passphrase = self._pass_util.get_pass(
71 self._catalog.get_length(p_name))
72 docs = list()
73 storage_policy = self._catalog.get_storage_policy(p_name)
74 docs.append(self.generate_doc(
75 KIND,
76 p_name,
77 storage_policy,
78 passphrase))
79 save_path = self.get_save_path(p_name)
80 if storage_policy == passphrase_catalog.P_ENCRYPTED:
81 PeglegSecretManagement(
82 docs=docs, generated=True, author=self._author,
83 catalog=self._catalog).encrypt_secrets(
84 save_path)
85 else:
86 files.write(save_path, docs)
87
88 @property
89 def kind_path(self):
90 return KIND_PATH
diff --git a/pegleg/engine/secrets.py b/pegleg/engine/secrets.py
index 6d89974..743e86f 100644
--- a/pegleg/engine/secrets.py
+++ b/pegleg/engine/secrets.py
@@ -15,11 +15,12 @@
15import logging 15import logging
16import os 16import os
17 17
18from pegleg.engine.generators.passpharase_generator import PassphraseGenerator
18from pegleg.engine.util import definition 19from pegleg.engine.util import definition
19from pegleg.engine.util import files 20from pegleg.engine.util import files
20from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement 21from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
21 22
22__all__ = ('encrypt', 'decrypt') 23__all__ = ('encrypt', 'decrypt', 'generate_passphrases')
23 24
24LOG = logging.getLogger(__name__) 25LOG = logging.getLogger(__name__)
25 26
@@ -28,22 +29,21 @@ def encrypt(save_location, author, site_name):
28 """ 29 """
29 Encrypt all secrets documents for a site identifies by site_name. 30 Encrypt all secrets documents for a site identifies by site_name.
30 31
31 Parse through all documents related to site_name and encrypt all 32 Parse through all documents related to ``site_name`` and encrypt all
32 site documents which have metadata.storagePolicy: encrypted, and which are 33 site documents, which have metadata.storagePolicy: encrypted, and
33 not already encrypted and wrapped in a PeglegManagedDocument. 34 are not already encrypted and wrapped in a PeglegManagedDocument.
34 Passphrase and salt for the encryption are read from environment 35 ``Passphrase`` and ``salt`` for the encryption are read from environment
35 variables ($PEGLEG_PASSPHRASE and $PEGLEG_SALT respectively). 36 variables``$PEGLEG_PASSPHRASE`` and ``$PEGLEG_SALT`` respectively.
36 By default, the resulting output files will overwrite the original 37 By default, the resulting output files will overwrite the original
37 unencrypted secrets documents. 38 unencrypted secrets documents.
38 :param save_location: if provided, identifies the base directory to store 39
39 the encrypted secrets files. If not provided the encrypted secrets files 40 :param str save_location: if provided, is used as the base directory to
40 will overwrite the original unencrypted files (default behavior). 41 store the encrypted secrets files. If not provided, the encrypted
41 :type save_location: string 42 secrets files will overwrite the original unencrypted files (default
42 :param author: The identifier provided by the application or 43 behavior).
43 the person who requests encrypt the site secrets documents. 44 :param str author: Identifies the individual or application, who
44 :type author: string 45 encrypts the secrets documents.
45 :param site_name: The name of the site to encrypt its secrets files. 46 :param str site_name: The name of the site to encrypt its secrets files.
46 :type site_name: string
47 """ 47 """
48 48
49 files.check_file_save_location(save_location) 49 files.check_file_save_location(save_location)
@@ -51,8 +51,9 @@ def encrypt(save_location, author, site_name):
51 secrets_found = False 51 secrets_found = False
52 for repo_base, file_path in definition.site_files_by_repo(site_name): 52 for repo_base, file_path in definition.site_files_by_repo(site_name):
53 secrets_found = True 53 secrets_found = True
54 PeglegSecretManagement(file_path).encrypt_secrets( 54 PeglegSecretManagement(
55 _get_dest_path(repo_base, file_path, save_location), author) 55 file_path=file_path, author=author).encrypt_secrets(
56 _get_dest_path(repo_base, file_path, save_location))
56 if secrets_found: 57 if secrets_found:
57 LOG.info('Encryption of all secret files was completed.') 58 LOG.info('Encryption of all secret files was completed.')
58 else: 59 else:
@@ -62,11 +63,11 @@ def encrypt(save_location, author, site_name):
62 63
63def decrypt(file_path, site_name): 64def decrypt(file_path, site_name):
64 """ 65 """
65 Decrypt one secrets file and print the decrypted data to standard out. 66 Decrypt one secrets file, and print the decrypted file to standard out.
66 67
67 Search in in secrets file of a site, identified by site_name, for a file 68 Search in secrets file of a site, identified by ``site_name``, for a file
68 named file_name. 69 named ``file_name``.
69 If the file is found and encrypted, unwrap and decrypt it and print the 70 If the file is found and encrypted, unwrap and decrypt it, and print the
70 result to standard out. 71 result to standard out.
71 If the file is found, but it is not encrypted, print the contents of the 72 If the file is found, but it is not encrypted, print the contents of the
72 file to standard out. 73 file to standard out.
@@ -90,7 +91,7 @@ def decrypt(file_path, site_name):
90def _get_dest_path(repo_base, file_path, save_location): 91def _get_dest_path(repo_base, file_path, save_location):
91 """ 92 """
92 Calculate and return the destination base directory path for the 93 Calculate and return the destination base directory path for the
93 encrypted or decrypted secrets files. 94 encrypted secrets files.
94 95
95 :param repo_base: Base repo of the source secrets file. 96 :param repo_base: Base repo of the source secrets file.
96 :type repo_base: string 97 :type repo_base: string
@@ -111,3 +112,20 @@ def _get_dest_path(repo_base, file_path, save_location):
111 return file_path.replace(repo_base, save_location) 112 return file_path.replace(repo_base, save_location)
112 else: 113 else:
113 return file_path 114 return file_path
115
116
117def generate_passphrases(site_name, save_location, author, interactive=False):
118 """
119 Look for the site passphrase catalogs, and for every passphrase entry in
120 the passphrase catalog generate a passphrase document, wrap the
121 passphrase document in a pegleg managed document, and encrypt the
122 passphrase data.
123
124 :param interactive: Whether to generate the results interactively
125 :param str site_name: The site to read from
126 :param str save_location: Location to write files to
127 :param str author:
128 """
129
130 PassphraseGenerator(site_name, save_location, author).generate(
131 interactive=interactive)
diff --git a/pegleg/engine/util/encryption.py b/pegleg/engine/util/encryption.py
index 626a151..c822cbc 100644
--- a/pegleg/engine/util/encryption.py
+++ b/pegleg/engine/util/encryption.py
@@ -25,6 +25,8 @@ KEY_LENGTH = 32
25ITERATIONS = 10000 25ITERATIONS = 10000
26LOG = logging.getLogger(__name__) 26LOG = logging.getLogger(__name__)
27 27
28__all__ = ('encrypt', 'decrypt')
29
28 30
29def encrypt(unencrypted_data, 31def encrypt(unencrypted_data,
30 passphrase, 32 passphrase,
diff --git a/pegleg/engine/util/git.py b/pegleg/engine/util/git.py
index 217ee8f..dfc27b9 100644
--- a/pegleg/engine/util/git.py
+++ b/pegleg/engine/util/git.py
@@ -141,6 +141,14 @@ def _get_current_ref(repo_url):
141 return None 141 return None
142 142
143 143
144def get_remote_url(repo_url):
145 try:
146 repo = Repo(repo_url, search_parent_directories=True)
147 return repo.remotes.origin.url
148 except Exception as e:
149 return None
150
151
144def _try_git_clone(repo_url, 152def _try_git_clone(repo_url,
145 ref=None, 153 ref=None,
146 proxy_server=None, 154 proxy_server=None,
diff --git a/pegleg/engine/util/passphrase.py b/pegleg/engine/util/passphrase.py
new file mode 100644
index 0000000..d81385e
--- /dev/null
+++ b/pegleg/engine/util/passphrase.py
@@ -0,0 +1,33 @@
1# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15
16from random import SystemRandom
17from rstr import Rstr
18import string
19
20__all__ = ['Passphrase']
21
22
23class Passphrase(object):
24
25 def __init__(self):
26 self._pool = string.ascii_letters + string.digits + string.punctuation
27 self._rs = Rstr(SystemRandom())
28
29 def get_pass(self, pass_len=24):
30 """Create and return a random password, of the ``pass_len`` length."""
31 if pass_len < 24:
32 pass_len = 24
33 return self._rs.rstr(self._pool, pass_len)
diff --git a/pegleg/engine/util/pegleg_managed_document.py b/pegleg/engine/util/pegleg_managed_document.py
index 1cfd504..fe05945 100644
--- a/pegleg/engine/util/pegleg_managed_document.py
+++ b/pegleg/engine/util/pegleg_managed_document.py
@@ -15,48 +15,66 @@
15from datetime import datetime 15from datetime import datetime
16import logging 16import logging
17 17
18from pegleg import config
19from pegleg.engine.util import git
20
18PEGLEG_MANAGED_SCHEMA = 'pegleg/PeglegManagedDocument/v1' 21PEGLEG_MANAGED_SCHEMA = 'pegleg/PeglegManagedDocument/v1'
19ENCRYPTED = 'encrypted' 22ENCRYPTED = 'encrypted'
23GENERATED = 'generated'
20STORAGE_POLICY = 'storagePolicy' 24STORAGE_POLICY = 'storagePolicy'
21METADATA = 'metadata' 25METADATA = 'metadata'
22LOG = logging.getLogger(__name__) 26LOG = logging.getLogger(__name__)
23 27
28__all__ = ['PeglegManagedSecretsDocument']
29
24 30
25class PeglegManagedSecretsDocument(object): 31class PeglegManagedSecretsDocument(object):
26 """Object representing one Pegleg managed secret document.""" 32 """Object representing one Pegleg managed secret document."""
27 33
28 def __init__(self, secrets_document): 34 def __init__(self, document, generated=False, catalog=None, author=None):
35
29 """ 36 """
30 Parse and wrap an externally generated document in a 37 Parse and wrap an externally generated document in a
31 pegleg managed document. 38 pegleg managed document.
32 39
33 :param secrets_document: The content of the source document 40 :param document: The content of the source document
34 :type secrets_document: dict 41 :type document: dict
42 :param bool generated: A flag to indicate the documents are
43 auto-generated by pegleg (True), or manually created (False).
44 :param catalog: catalog of the generated secret documents. A catalog
45 must be provided, only if generated is True.
46 :type catalog: A subclass of the ABC
47 pegleg.catalogs.base_catalog.BaseCatalog
35 48
36 """ 49 """
37 50
38 if self.is_pegleg_managed_secret(secrets_document): 51 self._catalog = catalog
39 self._pegleg_document = secrets_document 52 self._author = author
53 self._generated = generated
54 if self.is_pegleg_managed_secret(document):
55 self._pegleg_document = document
40 else: 56 else:
41 self._pegleg_document =\ 57 self._pegleg_document = self.__wrap(
42 self.__wrap(secrets_document) 58 document, generated, catalog, author)
43 self._embedded_document = \ 59 self._embedded_document = \
44 self._pegleg_document['data']['managedDocument'] 60 self._pegleg_document['data']['managedDocument']
45 61
46 @staticmethod 62 @staticmethod
47 def __wrap(secrets_document): 63 def __wrap(secrets_document, generated=False, catalog=None, author=None):
48 """ 64 """
49 Embeds a valid deckhand document in a pegleg managed document. 65 Embeds a valid deckhand document in a pegleg managed document.
50 66
51 :param secrets_document: secrets document to be embedded in a 67 :param secrets_document: secrets document to be embedded in a
52 pegleg managed document. 68 pegleg managed document.
53 :type secrets_document: dict 69 :type secrets_document: dict
70 :param bool generated: A flag to indicate the documents are
71 auto-generated by pegleg (True), or manually created (False).
54 :return: pegleg manged document with the wrapped original secrets 72 :return: pegleg manged document with the wrapped original secrets
55 document. 73 document.
56 :rtype: dict 74 :rtype: dict
57 """ 75 """
58 76
59 return { 77 doc = {
60 'schema': PEGLEG_MANAGED_SCHEMA, 78 'schema': PEGLEG_MANAGED_SCHEMA,
61 'metadata': { 79 'metadata': {
62 'name': secrets_document['metadata']['name'], 80 'name': secrets_document['metadata']['name'],
@@ -78,6 +96,18 @@ class PeglegManagedSecretsDocument(object):
78 } 96 }
79 } 97 }
80 98
99 if generated:
100 doc['data'][GENERATED] = {
101 'at': datetime.utcnow().isoformat(),
102 'by': author,
103 'specifiedBy': {
104 'repo': git.repo_url(config.get_site_repo()),
105 'reference': config.get_site_rev() or 'master',
106 'path': catalog.catalog_path,
107 },
108 }
109 return doc
110
81 @staticmethod 111 @staticmethod
82 def is_pegleg_managed_secret(secrets_document): 112 def is_pegleg_managed_secret(secrets_document):
83 """" 113 """"
@@ -117,18 +147,24 @@ class PeglegManagedSecretsDocument(object):
117 otherwise.""" 147 otherwise."""
118 return ENCRYPTED in self.data 148 return ENCRYPTED in self.data
119 149
150 def is_generated(self):
151 """If the document is already marked auto-generated return True. False
152 otherwise."""
153 return GENERATED in self.data
154
120 def is_storage_policy_encrypted(self): 155 def is_storage_policy_encrypted(self):
121 """If the document's storagePolicy is set to encrypted return True. 156 """If the document's storagePolicy is set to encrypted return True.
122 False otherwise.""" 157 False otherwise."""
123 return STORAGE_POLICY in self._embedded_document[METADATA] \ 158 return STORAGE_POLICY in self._embedded_document[METADATA] \
124 and ENCRYPTED in self._embedded_document[METADATA][STORAGE_POLICY] 159 and ENCRYPTED in self._embedded_document[METADATA][STORAGE_POLICY]
125 160
126 def set_encrypted(self, author): 161 def set_encrypted(self, author=None):
127 """Mark the pegleg managed document as encrypted.""" 162 """Mark the pegleg managed document as encrypted."""
128 self.data[ENCRYPTED] = { 163 self.data[ENCRYPTED] = {
129 'at': datetime.utcnow().isoformat(), 164 'at': datetime.utcnow().isoformat()
130 'by': author,
131 } 165 }
166 if author:
167 self.data[ENCRYPTED]['by'] = author
132 168
133 def set_decrypted(self): 169 def set_decrypted(self):
134 """Mark the pegleg managed document as un-encrypted.""" 170 """Mark the pegleg managed document as un-encrypted."""
diff --git a/pegleg/engine/util/pegleg_secret_management.py b/pegleg/engine/util/pegleg_secret_management.py
index 870ccf4..78e97ed 100644
--- a/pegleg/engine/util/pegleg_secret_management.py
+++ b/pegleg/engine/util/pegleg_secret_management.py
@@ -34,7 +34,8 @@ ENV_SALT = 'PEGLEG_SALT'
34class PeglegSecretManagement(object): 34class PeglegSecretManagement(object):
35 """An object to handle operations on of a pegleg managed file.""" 35 """An object to handle operations on of a pegleg managed file."""
36 36
37 def __init__(self, file_path=None, docs=None): 37 def __init__(self, file_path=None, docs=None, generated=False,
38 catalog=None, author=None):
38 """ 39 """
39 Read the source file and the environment data needed to wrap and 40 Read the source file and the environment data needed to wrap and
40 process the file documents as pegleg managed document. 41 process the file documents as pegleg managed document.
@@ -43,22 +44,40 @@ class PeglegSecretManagement(object):
43 """ 44 """
44 45
45 if all([file_path, docs]) or not any([file_path, docs]): 46 if all([file_path, docs]) or not any([file_path, docs]):
46 raise ValueError('Either `file_path` or `docs` must be specified.') 47 raise ValueError('Either `file_path` or `docs` must be '
48 'specified.')
47 49
50 if generated and not (author and catalog):
51 raise ValueError("If the document is generated, author and "
52 "catalog must be specified.")
48 self.__check_environment() 53 self.__check_environment()
49 self.file_path = file_path 54 self.file_path = file_path
50 self.documents = list() 55 self.documents = list()
56 self._generated = generated
57
51 if docs: 58 if docs:
52 for doc in docs: 59 for doc in docs:
53 self.documents.append(PeglegManagedSecret(doc)) 60 self.documents.append(PeglegManagedSecret(doc,
61 generated=generated,
62 catalog=catalog,
63 author=author))
54 else: 64 else:
55 self.file_path = file_path 65 self.file_path = file_path
56 for doc in files.read(file_path): 66 for doc in files.read(file_path):
57 self.documents.append(PeglegManagedSecret(doc)) 67 self.documents.append(PeglegManagedSecret(doc))
58 68
69 self._author = author
70
59 self.passphrase = os.environ.get(ENV_PASSPHRASE).encode() 71 self.passphrase = os.environ.get(ENV_PASSPHRASE).encode()
60 self.salt = os.environ.get(ENV_SALT).encode() 72 self.salt = os.environ.get(ENV_SALT).encode()
61 73
74 def __iter__(self):
75 """
76 Make the secret management object iterable
77 :return: the wrapped documents
78 """
79 return (doc.pegleg_document for doc in self.documents)
80
62 @staticmethod 81 @staticmethod
63 def __check_environment(): 82 def __check_environment():
64 """ 83 """
@@ -81,7 +100,7 @@ class PeglegSecretManagement(object):
81 'Environment variable {} is not defined or ' 100 'Environment variable {} is not defined or '
82 'is an empty string.'.format(ENV_SALT)) 101 'is an empty string.'.format(ENV_SALT))
83 102
84 def encrypt_secrets(self, save_path, author): 103 def encrypt_secrets(self, save_path):
85 """ 104 """
86 Wrap and encrypt the secrets documents included in the input file, 105 Wrap and encrypt the secrets documents included in the input file,
87 into pegleg manage secrets documents, and write the result in 106 into pegleg manage secrets documents, and write the result in
@@ -97,11 +116,34 @@ class PeglegSecretManagement(object):
97 :type author: string 116 :type author: string
98 """ 117 """
99 118
119 doc_list, encrypted_docs = self.get_encrypted_secrets()
120 if encrypted_docs:
121 files.write(save_path, doc_list)
122 click.echo('Wrote encrypted data to: {}'.format(save_path))
123 else:
124 LOG.debug('All documents in file: {} are either already encrypted '
125 'or have cleartext storage policy. '
126 'Skipping.'.format(self.file_path))
127
128 def get_encrypted_secrets(self):
129 """
130 :return doc_list: The list of documents
131 :rtype doc_list: list
132 :return encrypted_docs: Whether any documents were encrypted
133 :rtype encrypted_docs: bool
134 """
135 if self._generated and not self._author:
136 raise ValueError("An author is needed to encrypt "
137 "generated documents. "
138 "Specify it when PeglegSecretManagement "
139 "is initialized.")
140
100 encrypted_docs = False 141 encrypted_docs = False
101 doc_list = [] 142 doc_list = []
102 for doc in self.documents: 143 for doc in self.documents:
103 # do not re-encrypt already encrypted data 144 # do not re-encrypt already encrypted data
104 if doc.is_encrypted(): 145 if doc.is_encrypted():
146 doc_list.append(doc)
105 continue 147 continue
106 148
107 # only encrypt if storagePolicy is set to encrypted. 149 # only encrypt if storagePolicy is set to encrypted.
@@ -113,16 +155,10 @@ class PeglegSecretManagement(object):
113 155
114 doc.set_secret( 156 doc.set_secret(
115 encrypt(doc.get_secret().encode(), self.passphrase, self.salt)) 157 encrypt(doc.get_secret().encode(), self.passphrase, self.salt))
116 doc.set_encrypted(author) 158 doc.set_encrypted(self._author)
117 encrypted_docs = True 159 encrypted_docs = True
118 doc_list.append(doc.pegleg_document) 160 doc_list.append(doc.pegleg_document)
119 if encrypted_docs: 161 return doc_list, encrypted_docs
120 files.write(save_path, doc_list)
121 LOG.info('Wrote data to: {}.'.format(save_path))
122 else:
123 LOG.debug('All documents in file: {} are either already encrypted '
124 'or have cleartext storage policy. '
125 'Skipping.'.format(self.file_path))
126 162
127 def decrypt_secrets(self): 163 def decrypt_secrets(self):
128 """Decrypt and unwrap pegleg managed encrypted secrets documents 164 """Decrypt and unwrap pegleg managed encrypted secrets documents
diff --git a/requirements.txt b/requirements.txt
index 00af399..477a0fb 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6,5 +6,6 @@ cryptography==2.3.1
6python-dateutil==2.7.3 6python-dateutil==2.7.3
7 7
8# External dependencies 8# External dependencies
9rstr==2.2.6
9git+https://github.com/openstack/airship-deckhand.git@7d697012fcbd868b14670aa9cf895acfad5a7f8d 10git+https://github.com/openstack/airship-deckhand.git@7d697012fcbd868b14670aa9cf895acfad5a7f8d
10git+https://github.com/openstack/airship-shipyard.git@44f7022df6438de541501c2fdd5c46df198b82bf#egg=shipyard_client&subdirectory=src/bin/shipyard_client 11git+https://github.com/openstack/airship-shipyard.git@44f7022df6438de541501c2fdd5c46df198b82bf#egg=shipyard_client&subdirectory=src/bin/shipyard_client
diff --git a/setup.py b/setup.py
index 5406475..d659542 100644
--- a/setup.py
+++ b/setup.py
@@ -31,8 +31,9 @@ setup(
31 'pegleg=pegleg.cli:main', 31 'pegleg=pegleg.cli:main',
32 ]}, 32 ]},
33 include_package_data=True, 33 include_package_data=True,
34 package_dir={'pegleg': 'pegleg'},
34 package_data={ 35 package_data={
35 'schemas': [ 36 'pegleg': [
36 'schemas/*.yaml', 37 'schemas/*.yaml',
37 ], 38 ],
38 }, 39 },
diff --git a/site_yamls/site/passphrase-catalog.yaml b/site_yamls/site/passphrase-catalog.yaml
new file mode 100644
index 0000000..0f19d27
--- /dev/null
+++ b/site_yamls/site/passphrase-catalog.yaml
@@ -0,0 +1,212 @@
1---
2# The purpose of this file is to define the Passpharase certificates for the environment
3#
4schema: pegleg/PassphraseCatalog/v1
5metadata:
6 schema: metadata/Document/v1
7 name: cluster-passphrases
8 layeringDefinition:
9 abstract: false
10 layer: site
11 storagePolicy: cleartext
12data:
13 passphrases:
14 - description: 'short description of the passphrase'
15 document_name: ceph_swift_keystone_password
16 encrypted: true
17 - description: 'short description of the passphrase'
18 document_name: ucp_keystone_admin_password
19 encrypted: true
20 - description: 'short description of the passphrase'
21 document_name: ucp_armada_keystone_password
22 encrypted: true
23 - description: 'short description of the passphrase'
24 document_name: ucp_postgres_admin_password
25 encrypted: true
26 - description: 'short description of the passphrase'
27 document_name: ucp_oslo_db_admin_password
28 encrypted: true
29 - description: 'short description of the passphrase'
30 document_name: ucp_deckhand_postgres_password
31 encrypted: true
32 - description: 'short description of the passphrase'
33 document_name: ucp_deckhand_keystone_password
34 encrypted: true
35 - description: 'short description of the passphrase'
36 document_name: ucp_barbican_keystone_password
37 encrypted: true
38 - description: 'short description of the passphrase'
39 document_name: ucp_barbican_oslo_db_password
40 encrypted: true
41 - description: 'short description of the passphrase'
42 document_name: ucp_oslo_messaging_password
43 encrypted: true
44 - description: 'short description of the passphrase'
45 document_name: ucp_drydock_postgres_password
46 encrypted: true
47 - description: 'short description of the passphrase'
48 document_name: ucp_drydock_keystone_password
49 encrypted: true
50 - description: 'short description of the passphrase'
51 document_name: ucp_maas_postgres_password
52 encrypted: true
53 - description: 'short description of the passphrase'
54 document_name: ucp_keystone_oslo_db_password
55 encrypted: true
56 - description: 'short description of the passphrase'
57 document_name: ucp_promenade_keystone_password
58 encrypted: true
59 - description: 'short description of the passphrase'
60 document_name: ucp_shipyard_keystone_password
61 encrypted: true
62 - description: 'short description of the passphrase'
63 document_name: ucp_shipyard_postgres_password
64 encrypted: true
65 - description: 'short description of the passphrase'
66 document_name: ucp_airflow_postgres_password
67 encrypted: true
68 - description: 'short description of the passphrase'
69 document_name: ucp_rabbitmq_erlang_cookie
70 encrypted: true
71 - description: 'short description of the passphrase'
72 document_name: maas_region_secret
73 encrypted: true
74 - description: 'short description of the passphrase'
75 document_name: osh_barbican_oslo_db_password
76 encrypted: true
77 - description: 'short description of the passphrase'
78 document_name: osh_barbican_oslo_messaging_admin_password
79 encrypted: true
80 - description: 'short description of the passphrase'
81 document_name: osh_barbican_oslo_messaging_password
82 encrypted: true
83 - description: 'short description of the passphrase'
84 document_name: osh_barbican_password
85 encrypted: true
86 - description: 'short description of the passphrase'
87 document_name: osh_barbican_rabbitmq_erlang_cookie
88 encrypted: true
89 - description: 'short description of the passphrase'
90 document_name: osh_cinder_oslo_messaging_admin_password
91 encrypted: true
92 - description: 'short description of the passphrase'
93 document_name: osh_cinder_oslo_messaging_password
94 encrypted: true
95 - description: 'short description of the passphrase'
96 document_name: osh_cinder_password
97 encrypted: true
98 - description: 'short description of the passphrase'
99 document_name: osh_cinder_rabbitmq_erlang_cookie
100 encrypted: true
101 - description: 'short description of the passphrase'
102 document_name: osh_glance_oslo_db_password
103 encrypted: true
104 - description: 'short description of the passphrase'
105 document_name: osh_glance_oslo_messaging_admin_password
106 encrypted: true
107 - description: 'short description of the passphrase'
108 document_name: osh_glance_oslo_messaging_password
109 encrypted: true
110 - description: 'short description of the passphrase'
111 document_name: osh_glance_password
112 encrypted: true
113 - description: 'short description of the passphrase'
114 document_name: osh_glance_rabbitmq_erlang_cookie
115 encrypted: true
116 - description: 'short description of the passphrase'
117 document_name: osh_heat_oslo_db_password
118 encrypted: true
119 - description: 'short description of the passphrase'
120 document_name: osh_heat_oslo_messaging_admin_password
121 encrypted: true
122 - description: 'short description of the passphrase'
123 document_name: osh_heat_oslo_messaging_password
124 encrypted: true
125 - description: 'short description of the passphrase'
126 document_name: osh_heat_password
127 encrypted: true
128 - description: 'short description of the passphrase'
129 document_name: osh_heat_rabbitmq_erlang_cookie
130 encrypted: true
131 - description: 'short description of the passphrase'
132 document_name: osh_heat_stack_user_password
133 encrypted: true
134 - description: 'short description of the passphrase'
135 document_name: osh_heat_trustee_password
136 encrypted: true
137 - description: 'short description of the passphrase'
138 document_name: osh_horizon_oslo_db_password
139 encrypted: true
140 - description: 'short description of the passphrase'
141 document_name: osh_infra_elasticsearch_admin_password
142 encrypted: true
143 - description: 'short description of the passphrase'
144 document_name: osh_infra_grafana_admin_password
145 encrypted: true
146 - description: 'short description of the passphrase'
147 document_name: osh_infra_grafana_oslo_db_password
148 encrypted: true
149 - description: 'short description of the passphrase'
150 document_name: osh_infra_grafana_oslo_db_session_password
151 encrypted: true
152 - description: 'short description of the passphrase'
153 document_name: osh_infra_kibana_admin_password
154 encrypted: true
155 - description: 'short description of the passphrase'
156 document_name: osh_infra_openstack_exporter_password
157 encrypted: true
158 - description: 'short description of the passphrase'
159 document_name: osh_infra_oslo_db_admin_password
160 encrypted: true
161 - description: 'short description of the passphrase'
162 document_name: osh_keystone_admin_password
163 encrypted: true
164 - description: 'short description of the passphrase'
165 document_name: osh_keystone_oslo_db_password
166 encrypted: true
167 - description: 'short description of the passphrase'
168 document_name: osh_keystone_oslo_messaging_admin_password
169 encrypted: true
170 - description: 'short description of the passphrase'
171 document_name: osh_keystone_oslo_messaging_password
172 encrypted: true
173 - description: 'short description of the passphrase'
174 document_name: osh_keystone_rabbitmq_erlang_cookie
175 encrypted: true
176 - description: 'short description of the passphrase'
177 document_name: osh_neutron_oslo_db_password
178 encrypted: true
179 - description: 'short description of the passphrase'
180 document_name: osh_neutron_oslo_messaging_admin_password
181 encrypted: true
182 - description: 'short description of the passphrase'
183 document_name: osh_neutron_oslo_messaging_password
184 encrypted: true
185 - description: 'short description of the passphrase'
186 document_name: osh_neutron_password
187 encrypted: true
188 - description: 'short description of the passphrase'
189 document_name: osh_neutron_rabbitmq_erlang_cookie
190 encrypted: true
191 - description: 'short description of the passphrase'
192 document_name: osh_nova_oslo_db_password
193 encrypted: true
194 - description: 'short description of the passphrase'
195 document_name: osh_nova_oslo_messaging_admin_password
196 encrypted: true
197 - description: 'short description of the passphrase'
198 document_name: osh_nova_oslo_messaging_password
199 encrypted: true
200 - description: 'short description of the passphrase'
201 document_name: osh_nova_password
202 encrypted: true
203 - description: 'short description of the passphrase'
204 document_name: osh_nova_rabbitmq_erlang_cookie
205 encrypted: true
206 - description: 'short description of the passphrase'
207 document_name: osh_oslo_db_admin_password
208 encrypted: true
209 - description: 'short description of the passphrase'
210 document_name: osh_placement_password
211 encrypted: true
212...
diff --git a/tests/unit/engine/test_generate_passphrases.py b/tests/unit/engine/test_generate_passphrases.py
new file mode 100644
index 0000000..d14ed69
--- /dev/null
+++ b/tests/unit/engine/test_generate_passphrases.py
@@ -0,0 +1,178 @@
1# Copyright 2018 AT&T Intellectual Property. All other rights reserved.
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15import os
16import tempfile
17
18import mock
19import string
20import yaml
21
22from pegleg.engine.util.passphrase import Passphrase
23from pegleg.engine.generators.passpharase_generator import PassphraseGenerator
24from pegleg.engine.util import encryption
25from pegleg.engine import util
26import pegleg
27from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE
28from pegleg.engine.util.pegleg_secret_management import ENV_SALT
29
30TEST_PASSPHRASES_CATALOG = yaml.load("""
31---
32schema: pegleg/PassphraseCatalog/v1
33metadata:
34 schema: metadata/Document/v1
35 name: cluster-passphrases
36 layeringDefinition:
37 abstract: false
38 layer: site
39 storagePolicy: cleartext
40data:
41 passphrases:
42 - description: 'short description of the passphrase'
43 document_name: ceph_swift_keystone_password
44 encrypted: true
45 - description: 'short description of the passphrase'
46 document_name: ucp_keystone_admin_password
47 encrypted: true
48 length: 24
49 - description: 'short description of the passphrase'
50 document_name: osh_barbican_oslo_db_password
51 encrypted: true
52 length: 23
53 - description: 'short description of the passphrase'
54 document_name: osh_cinder_password
55 encrypted: true
56 length: 25
57 - description: 'short description of the passphrase'
58 document_name: osh_oslo_db_admin_password
59 encrypted: true
60 length: 0
61 - description: 'short description of the passphrase'
62 document_name: osh_placement_password
63 encrypted: true
64 length: 32
65...
66""")
67
68TEST_REPOSITORIES = {
69 'repositories': {
70 'global': {
71 'revision': '843d1a50106e1f17f3f722e2ef1634ae442fe68f',
72 'url': 'ssh://REPO_USERNAME@gerrit:29418/aic-clcp-manifests.git'
73 },
74 'secrets': {
75 'revision': 'master',
76 'url': ('ssh://REPO_USERNAME@gerrit:29418/aic-clcp-security-'
77 'manifests.git')
78 }
79 }
80}
81
82TEST_SITE_DEFINITION = {
83 'data': {
84 'revision': 'v1.0',
85 'site_type': 'cicd',
86 },
87 'metadata': {
88 'layeringDefinition': {
89 'abstract': 'false',
90 'layer': 'site',
91 },
92 'name': 'test-site',
93 'schema': 'metadata/Document/v1',
94 'storagePolicy': 'cleartext',
95 },
96 'schema': 'pegleg/SiteDefinition/v1',
97}
98
99TEST_SITE_DOCUMENTS = [TEST_SITE_DEFINITION, TEST_PASSPHRASES_CATALOG]
100
101
102def test_passphrase_default_len():
103 p_util = Passphrase()
104 passphrase = p_util.get_pass()
105 assert len(passphrase) == 24
106 alphabet = set(string.punctuation + string.ascii_letters + string.digits)
107 assert any(c in alphabet for c in passphrase)
108
109
110def test_passphrase_short_len():
111 p_util = Passphrase()
112 p = p_util.get_pass(0)
113 assert len(p) == 24
114 p = p_util.get_pass(23)
115 assert len(p) == 24
116 p = p_util.get_pass(-1)
117 assert len(p) == 24
118
119
120def test_passphrase_long_len():
121 p_util = Passphrase()
122 p = p_util.get_pass(25)
123 assert len(p) == 25
124 p = p_util.get_pass(128)
125 assert len(p) == 128
126
127
128@mock.patch.object(
129 util.definition,
130 'documents_for_site',
131 autospec=True,
132 return_value=TEST_SITE_DOCUMENTS)
133@mock.patch.object(
134 pegleg.config,
135 'get_site_repo',
136 autospec=True,
137 return_value='cicd_site_repo')
138@mock.patch.object(
139 util.definition,
140 'site_files',
141 autospec=True,
142 return_value=[
143 'cicd_site_repo/site/cicd/passphrases/passphrase-catalog.yaml', ])
144@mock.patch.dict(os.environ, {
145 ENV_PASSPHRASE: 'ytrr89erARAiPE34692iwUMvWqqBvC',
146 ENV_SALT: 'MySecretSalt'})
147def test_generate_passphrases(*_):
148 dir = tempfile.mkdtemp()
149 os.makedirs(os.path.join(dir, 'cicd_site_repo'), exist_ok=True)
150 PassphraseGenerator('cicd', dir, 'test_author').generate()
151
152 for passphrase in TEST_PASSPHRASES_CATALOG['data']['passphrases']:
153 passphrase_file_name = '{}.yaml'.format(passphrase['document_name'])
154 passphrase_file_path = os.path.join(dir, 'site', 'cicd', 'secrets',
155 'passphrases',
156 passphrase_file_name)
157 assert os.path.isfile(passphrase_file_path)
158 with open(passphrase_file_path) as stream:
159 doc = yaml.load(stream)
160 assert doc['schema'] == 'pegleg/PeglegManagedDocument/v1'
161 assert doc['metadata']['storagePolicy'] == 'cleartext'
162 assert 'encrypted' in doc['data']
163 assert doc['data']['encrypted']['by'] == 'test_author'
164 assert 'generated' in doc['data']
165 assert doc['data']['generated']['by'] == 'test_author'
166 assert 'managedDocument' in doc['data']
167 assert doc['data']['managedDocument']['metadata'][
168 'storagePolicy'] == 'encrypted'
169 decrypted_passphrase = encryption.decrypt(
170 doc['data']['managedDocument']['data'],
171 os.environ['PEGLEG_PASSPHRASE'].encode(),
172 os.environ['PEGLEG_SALT'].encode())
173 if passphrase_file_name == 'osh_placement_password.yaml':
174 assert len(decrypted_passphrase) == 32
175 elif passphrase_file_name == 'osh_cinder_password.yaml':
176 assert len(decrypted_passphrase) == 25
177 else:
178 assert len(decrypted_passphrase) == 24
diff --git a/tests/unit/engine/test_secrets.py b/tests/unit/engine/test_secrets.py
index 4c8dbd0..3940325 100644
--- a/tests/unit/engine/test_secrets.py
+++ b/tests/unit/engine/test_secrets.py
@@ -33,8 +33,10 @@ from pegleg.engine.util.pegleg_secret_management import ENV_PASSPHRASE
33from pegleg.engine.util.pegleg_secret_management import ENV_SALT 33from pegleg.engine.util.pegleg_secret_management import ENV_SALT
34from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement 34from pegleg.engine.util.pegleg_secret_management import PeglegSecretManagement
35from tests.unit import test_utils 35from tests.unit import test_utils
36from tests.unit.fixtures import temp_path, create_tmp_deployment_files, _gen_document 36from tests.unit.fixtures import temp_path, create_tmp_deployment_files, \
37from tests.unit.test_cli import TestSiteSecretsActions, BaseCLIActionTest, TEST_PARAMS 37 _gen_document
38from tests.unit.test_cli import TestSiteSecretsActions, BaseCLIActionTest, \
39 TEST_PARAMS
38 40
39TEST_DATA = """ 41TEST_DATA = """
40--- 42---
@@ -69,10 +71,9 @@ def test_encrypt_and_decrypt():
69 ENV_SALT: 'MySecretSalt' 71 ENV_SALT: 'MySecretSalt'
70}) 72})
71def test_short_passphrase(): 73def test_short_passphrase():
72 with pytest.raises( 74 with pytest.raises(click.ClickException,
73 click.ClickException, 75 match=r'.*is not at least 24-character long.*'):
74 match=r'.*is not at least 24-character long.*'): 76 PeglegSecretManagement(file_path='file_path', author='test_author')
75 PeglegSecretManagement('file_path')
76 77
77 78
78@mock.patch.dict(os.environ, { 79@mock.patch.dict(os.environ, {
@@ -129,6 +130,26 @@ def test_pegleg_secret_management_constructor_with_invalid_arguments():
129 PeglegSecretManagement(file_path='file_path', docs=['doc1']) 130 PeglegSecretManagement(file_path='file_path', docs=['doc1'])
130 assert 'Either `file_path` or `docs` must be specified.' in str( 131 assert 'Either `file_path` or `docs` must be specified.' in str(
131 err_info.value) 132 err_info.value)
133 with pytest.raises(ValueError) as err_info:
134 PeglegSecretManagement(
135 file_path='file_path', generated=True, author='test_author')
136 assert 'If the document is generated, author and catalog must be ' \
137 'specified.' in str(err_info.value)
138 with pytest.raises(ValueError) as err_info:
139 PeglegSecretManagement(
140 docs=['doc'], generated=True)
141 assert 'If the document is generated, author and catalog must be ' \
142 'specified.' in str(err_info.value)
143 with pytest.raises(ValueError) as err_info:
144 PeglegSecretManagement(
145 docs=['doc'], generated=True, author='test_author')
146 assert 'If the document is generated, author and catalog must be ' \
147 'specified.' in str(err_info.value)
148 with pytest.raises(ValueError) as err_info:
149 PeglegSecretManagement(
150 docs=['doc'], generated=True, catalog='catalog')
151 assert 'If the document is generated, author and catalog must be ' \
152 'specified.' in str(err_info.value)
132 153
133 154
134@mock.patch.dict(os.environ, { 155@mock.patch.dict(os.environ, {
@@ -143,14 +164,19 @@ def test_encrypt_decrypt_using_file_path(temp_path):
143 save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml') 164 save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
144 165
145 # encrypt documents and validate that they were encrypted 166 # encrypt documents and validate that they were encrypted
146 doc_mgr = PeglegSecretManagement(file_path=file_path) 167 doc_mgr = PeglegSecretManagement(file_path=file_path, author='test_author')
147 doc_mgr.encrypt_secrets(save_path, 'test_author') 168 doc_mgr.encrypt_secrets(save_path)
148 doc = doc_mgr.documents[0] 169 doc = doc_mgr.documents[0]
149 assert doc.is_encrypted() 170 assert doc.is_encrypted()
150 assert doc.data['encrypted']['by'] == 'test_author' 171 assert doc.data['encrypted']['by'] == 'test_author'
151 172
152 # decrypt documents and validate that they were decrypted 173 # decrypt documents and validate that they were decrypted
153 doc_mgr = PeglegSecretManagement(save_path) 174 doc_mgr = PeglegSecretManagement(
175 file_path=file_path, author='test_author')
176 doc_mgr.encrypt_secrets(save_path)
177 # read back the encrypted file
178 doc_mgr = PeglegSecretManagement(
179 file_path=save_path, author='test_author')
154 decrypted_data = doc_mgr.get_decrypted_secrets() 180 decrypted_data = doc_mgr.get_decrypted_secrets()
155 assert test_data[0]['data'] == decrypted_data[0]['data'] 181 assert test_data[0]['data'] == decrypted_data[0]['data']
156 assert test_data[0]['schema'] == decrypted_data[0]['schema'] 182 assert test_data[0]['schema'] == decrypted_data[0]['schema']
@@ -166,8 +192,9 @@ def test_encrypt_decrypt_using_docs(temp_path):
166 save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml') 192 save_path = os.path.join(temp_path, 'encrypted_secrets_file.yaml')
167 193
168 # encrypt documents and validate that they were encrypted 194 # encrypt documents and validate that they were encrypted
169 doc_mgr = PeglegSecretManagement(docs=test_data) 195 doc_mgr = PeglegSecretManagement(
170 doc_mgr.encrypt_secrets(save_path, 'test_author') 196 docs=test_data, author='test_author')
197 doc_mgr.encrypt_secrets(save_path)
171 doc = doc_mgr.documents[0] 198 doc = doc_mgr.documents[0]
172 assert doc.is_encrypted() 199 assert doc.is_encrypted()
173 assert doc.data['encrypted']['by'] == 'test_author' 200 assert doc.data['encrypted']['by'] == 'test_author'
@@ -177,7 +204,8 @@ def test_encrypt_decrypt_using_docs(temp_path):
177 encrypted_data = list(yaml.safe_load_all(stream)) 204 encrypted_data = list(yaml.safe_load_all(stream))
178 205
179 # decrypt documents and validate that they were decrypted 206 # decrypt documents and validate that they were decrypted
180 doc_mgr = PeglegSecretManagement(docs=encrypted_data) 207 doc_mgr = PeglegSecretManagement(
208 docs=encrypted_data, author='test_author')
181 decrypted_data = doc_mgr.get_decrypted_secrets() 209 decrypted_data = doc_mgr.get_decrypted_secrets()
182 assert test_data[0]['data'] == decrypted_data[0]['data'] 210 assert test_data[0]['data'] == decrypted_data[0]['data']
183 assert test_data[0]['schema'] == decrypted_data[0]['schema'] 211 assert test_data[0]['schema'] == decrypted_data[0]['schema']
@@ -190,6 +218,10 @@ def test_encrypt_decrypt_using_docs(temp_path):
190@pytest.mark.skipif( 218@pytest.mark.skipif(
191 not pki_utility.PKIUtility.cfssl_exists(), 219 not pki_utility.PKIUtility.cfssl_exists(),
192 reason='cfssl must be installed to execute these tests') 220 reason='cfssl must be installed to execute these tests')
221@mock.patch.dict(os.environ, {
222 ENV_PASSPHRASE: 'ytrr89erARAiPE34692iwUMvWqqBvC',
223 ENV_SALT: 'MySecretSalt'
224})
193def test_generate_pki_using_local_repo_path(create_tmp_deployment_files): 225def test_generate_pki_using_local_repo_path(create_tmp_deployment_files):
194 """Validates ``generate-pki`` action using local repo path.""" 226 """Validates ``generate-pki`` action using local repo path."""
195 # Scenario: 227 # Scenario:
@@ -212,6 +244,10 @@ def test_generate_pki_using_local_repo_path(create_tmp_deployment_files):
212@pytest.mark.skipif( 244@pytest.mark.skipif(
213 not pki_utility.PKIUtility.cfssl_exists(), 245 not pki_utility.PKIUtility.cfssl_exists(),
214 reason='cfssl must be installed to execute these tests') 246 reason='cfssl must be installed to execute these tests')
247@mock.patch.dict(os.environ, {
248 ENV_PASSPHRASE: 'ytrr89erARAiPE34692iwUMvWqqBvC',
249 ENV_SALT: 'MySecretSalt'
250})
215def test_check_expiry(create_tmp_deployment_files): 251def test_check_expiry(create_tmp_deployment_files):
216 """ Validates check_expiry """ 252 """ Validates check_expiry """
217 repo_path = str(git.git_handler(TEST_PARAMS["repo_url"], 253 repo_path = str(git.git_handler(TEST_PARAMS["repo_url"],
@@ -228,9 +264,11 @@ def test_check_expiry(create_tmp_deployment_files):
228 continue 264 continue
229 with open(generated_file, 'r') as f: 265 with open(generated_file, 'r') as f:
230 results = yaml.safe_load_all(f) # Validate valid YAML. 266 results = yaml.safe_load_all(f) # Validate valid YAML.
267 results = PeglegSecretManagement(
268 docs=results).get_decrypted_secrets()
231 for result in results: 269 for result in results:
232 if result['data']['managedDocument']['schema'] == \ 270 if result['schema'] == \
233 "deckhand/Certificate/v1": 271 "deckhand/Certificate/v1":
234 cert = result['data']['managedDocument']['data'] 272 cert = result['data']
235 assert not pki_util.check_expiry(cert), \ 273 assert not pki_util.check_expiry(cert), \
236 "%s is expired!" % generated_file.name 274 "%s is expired!" % generated_file.name
diff --git a/tests/unit/test_cli.py b/tests/unit/test_cli.py
index 7d09d1d..ea7dd32 100644
--- a/tests/unit/test_cli.py
+++ b/tests/unit/test_cli.py
@@ -28,7 +28,6 @@ from pegleg.engine.util import git
28from tests.unit import test_utils 28from tests.unit import test_utils
29from tests.unit.fixtures import temp_path 29from tests.unit.fixtures import temp_path
30 30
31
32TEST_PARAMS = { 31TEST_PARAMS = {
33 "site_name": "airship-seaworthy", 32 "site_name": "airship-seaworthy",
34 "site_type": "foundry", 33 "site_type": "foundry",
@@ -67,7 +66,7 @@ class BaseCLIActionTest(object):
67 cls.repo_rev = TEST_PARAMS["repo_rev"] 66 cls.repo_rev = TEST_PARAMS["repo_rev"]
68 cls.repo_name = TEST_PARAMS["repo_name"] 67 cls.repo_name = TEST_PARAMS["repo_name"]
69 cls.treasuremap_path = git.git_handler(TEST_PARAMS["repo_url"], 68 cls.treasuremap_path = git.git_handler(TEST_PARAMS["repo_url"],
70 ref=TEST_PARAMS["repo_rev"]) 69 ref=TEST_PARAMS["repo_rev"])
71 70
72 71
73class TestSiteCLIOptions(BaseCLIActionTest): 72class TestSiteCLIOptions(BaseCLIActionTest):
@@ -377,7 +376,8 @@ class TestSiteCliActions(BaseCLIActionTest):
377 376
378 with mock.patch('pegleg.cli.ShipyardHelper') as mock_obj: 377 with mock.patch('pegleg.cli.ShipyardHelper') as mock_obj:
379 result = self.runner.invoke(cli.site, 378 result = self.runner.invoke(cli.site,
380 ['-r', repo_path, 'upload', self.site_name]) 379 ['-r', repo_path, 'upload',
380 self.site_name])
381 381
382 assert result.exit_code == 0 382 assert result.exit_code == 0
383 mock_obj.assert_called_once() 383 mock_obj.assert_called_once()
@@ -442,6 +442,14 @@ class TestRepoCliActions(BaseCLIActionTest):
442class TestSiteSecretsActions(BaseCLIActionTest): 442class TestSiteSecretsActions(BaseCLIActionTest):
443 """Tests site secrets-related CLI actions.""" 443 """Tests site secrets-related CLI actions."""
444 444
445 @classmethod
446 def setup_class(cls):
447 super(TestSiteSecretsActions, cls).setup_class()
448 cls.runner = CliRunner(env={
449 "PEGLEG_PASSPHRASE": 'ytrr89erARAiPE34692iwUMvWqqBvC',
450 "PEGLEG_SALT": "MySecretSalt"
451 })
452
445 def _validate_generate_pki_action(self, result): 453 def _validate_generate_pki_action(self, result):
446 assert result.exit_code == 0 454 assert result.exit_code == 0
447 455
@@ -455,7 +463,7 @@ class TestSiteSecretsActions(BaseCLIActionTest):
455 for generated_file in generated_files: 463 for generated_file in generated_files:
456 with open(generated_file, 'r') as f: 464 with open(generated_file, 'r') as f:
457 result = yaml.safe_load_all(f) # Validate valid YAML. 465 result = yaml.safe_load_all(f) # Validate valid YAML.
458 assert list(result), "%s file is empty" % filename 466 assert list(result), "%s file is empty" % generated_file
459 467
460 @pytest.mark.skipif( 468 @pytest.mark.skipif(
461 not pki_utility.PKIUtility.cfssl_exists(), 469 not pki_utility.PKIUtility.cfssl_exists(),
@@ -493,9 +501,9 @@ class TestSiteSecretsActions(BaseCLIActionTest):
493 not pki_utility.PKIUtility.cfssl_exists(), 501 not pki_utility.PKIUtility.cfssl_exists(),
494 reason='cfssl must be installed to execute these tests') 502 reason='cfssl must be installed to execute these tests')
495 @mock.patch.dict(os.environ, { 503 @mock.patch.dict(os.environ, {
496 "PEGLEG_PASSPHRASE": "123456789012345678901234567890", 504 "PEGLEG_PASSPHRASE": "123456789012345678901234567890",
497 "PEGLEG_SALT": "123456" 505 "PEGLEG_SALT": "123456"
498 }) 506 })
499 def test_site_secrets_encrypt_local_repo_path(self): 507 def test_site_secrets_encrypt_local_repo_path(self):
500 """Validates ``generate-pki`` action using local repo path.""" 508 """Validates ``generate-pki`` action using local repo path."""
501 # Scenario: 509 # Scenario:
@@ -504,13 +512,15 @@ class TestSiteSecretsActions(BaseCLIActionTest):
504 512
505 repo_path = self.treasuremap_path 513 repo_path = self.treasuremap_path
506 with open(os.path.join(repo_path, "site", "airship-seaworthy", 514 with open(os.path.join(repo_path, "site", "airship-seaworthy",
507 "secrets", "passphrases", "ceph_fsid.yaml"), "r") \ 515 "secrets", "passphrases", "ceph_fsid.yaml"),
516 "r") \
508 as ceph_fsid_fi: 517 as ceph_fsid_fi:
509 ceph_fsid = yaml.load(ceph_fsid_fi) 518 ceph_fsid = yaml.load(ceph_fsid_fi)
510 ceph_fsid["metadata"]["storagePolicy"] = "encrypted" 519 ceph_fsid["metadata"]["storagePolicy"] = "encrypted"
511 520
512 with open(os.path.join(repo_path, "site", "airship-seaworthy", 521 with open(os.path.join(repo_path, "site", "airship-seaworthy",
513 "secrets", "passphrases", "ceph_fsid.yaml"), "w") \ 522 "secrets", "passphrases", "ceph_fsid.yaml"),
523 "w") \
514 as ceph_fsid_fi: 524 as ceph_fsid_fi:
515 yaml.dump(ceph_fsid, ceph_fsid_fi) 525 yaml.dump(ceph_fsid, ceph_fsid_fi)
516 526
@@ -520,7 +530,8 @@ class TestSiteSecretsActions(BaseCLIActionTest):
520 assert result.exit_code == 0 530 assert result.exit_code == 0
521 531
522 with open(os.path.join(repo_path, "site", "airship-seaworthy", 532 with open(os.path.join(repo_path, "site", "airship-seaworthy",
523 "secrets", "passphrases", "ceph_fsid.yaml"), "r") \ 533 "secrets", "passphrases", "ceph_fsid.yaml"),
534 "r") \
524 as ceph_fsid_fi: 535 as ceph_fsid_fi:
525 ceph_fsid = yaml.load(ceph_fsid_fi) 536 ceph_fsid = yaml.load(ceph_fsid_fi)
526 assert "encrypted" in ceph_fsid["data"] 537 assert "encrypted" in ceph_fsid["data"]