Commit Graph

53 Commits

Author SHA1 Message Date
francisy 8ad13a330e Edit cli.rst
Adding in note under environment variables that PROMENADE_ENCRYPTION_KEY environment variable is required

Change-Id: I52c24608b2ecf61cbeea693ccbae1d2dd081ac29
2020-11-16 15:12:12 +00:00
Apurva Gokani 22e6df0870 adding site_type to deployment-data schema
This change adds the site_type parameter in deployment_data
The value will be taken from respective site-definition.yaml

Change-Id: I8e65b39c73c94caf3ed4cc517520b9577160b20d
2020-06-26 12:35:22 -05:00
Andrii Ostapenko 572ce3f6eb Return -o (--output*) option back as deprecated.
Deprecating cli options before removing them is a standard courtesy that
allows project consumers to handle such changes according to their
needs.

Option is removed here: https://review.opendev.org/#/c/701274

Change-Id: Iba4a4574426315a76294269d4287fd3ea18aa99e
2020-01-28 19:53:53 -06:00
Ryan Schroder 4cbc708c1d Standardize save location
Changes to make --save-location standard for all commands

Change-Id: I86a2da01ad1cc1c4d708131b5161182e16b5bb5e
2020-01-15 16:42:20 -06:00
Ian H. Pittwood 1ef5024eb0 Accept multiple path options for decrypt
Change-Id: Ic6c337334f1a21e1f3e303f4ccfb56caeddf4500
2020-01-14 11:42:28 -06:00
Ian H Pittwood ff9c95f423 Add path option to encrypt command
Adds a path option to the encrypt command to specify what directory of
file to encrypt. If path is not specified, all applicable files in the
repositories will be encrypted (this is the current behavior).

Change-Id: Idd5e063a54cf157a8ec761de85cbd67edd05364c
2020-01-07 15:40:05 -06:00
Ryan Schroder add78bb825 [FIX] Docs rendering issues
'--' previously rendered as '-'

Fixes site secrets section of cli.rst being displayed as preformatted
text

Change-Id: Ibac3b3ec07f47cbb049140af6ab2d968e77b756d
2019-12-12 16:04:58 -06:00
Ian H. Pittwood 9163ef08ca Add passphrase catalog override option
Adds an option to specify a passphrase catalog to override catalogs
discovered in the site repository. This allows the generation of a
specified subset of passphrases instead of the entire site's catalog.

Change-Id: I797107234292eea8ca788b7a94ed5e2c90566bf5
2019-12-10 20:40:31 +00:00
Alexander Hughes 1a1c31b5a7 Add generate-pki command as deprecated
In [0] the secrets generate-pki command was moved to secrets
generate certificates. While release notes were added, this change
impacts automation set up for users of Pegleg. This change adds
back the generate-pki command but marks it as deprecated.

[0] https://review.opendev.org/#/c/694810/

Change-Id: I6a3841e5f5313511ec2afd8340bcae5857cd81fa
2019-12-06 20:22:54 +00:00
Ian H. Pittwood 7306c3af9b Move generate-pki under generate sub-command
Change-Id: I83b078d27eb2c07c3be7eb6543de85ce32fb2a6e
2019-11-20 19:16:12 +00:00
Ian H. Pittwood 438db81d27 Update CLI docs for granular verbosity
Updates the CLI documentation for addition of granular verbosity
option.

Related Change: https://review.opendev.org/#/c/684349/

Change-Id: I8ca1e6381e074d7821cd862222dfeac7537a3d1f
2019-09-26 09:43:22 -05:00
Ian H. Pittwood e2dad75a99 Default to non-interactive passphrase generation
This change disables and skips input prompts for generate passphrases.
Using the -i option will now only enable prompts for passphrases that
are set to prompt=True.

Change-Id: Ia932305891259d9d1430e1d184dbf39892d4a5d3
2019-09-16 13:13:04 -05:00
Felipe Monteiro b598e850cc pki: Add documentation for PKI implementation
This adds documentation for the PKI implementation
in Pegleg that was recently ported from Promenade.

Change-Id: I248ec62a0c3523fc5f3950a6a678cf6d998b093a
2019-08-07 14:07:32 +02:00
Alexander Hughes 7018d5941c Support regenerating PKI
This patch adds functionality Pegleg currently lacks: the ability to
regenerate expired certificates.

This patch adds:
1. CLI toggle --regenerate-all to generate_pki.  Default is False,
   which means if no certificates are present, generate what is in
   the pki catalogue. If new certs have been added to the catalogue
   generate just those.  If the --regenerate-all flag is True, then
   Pegleg will ignore any existing certs and regenerate (or generate
   for the first time) all certificates defined in the PKI catalogue.
2. Documentation updates for CLI change.
3. Updates to pki_utility to accomodate the new flag.
4. Updates pki_generator methods to use rendered documents to
   accommodate documents that have to be layered.
5. Updates pki_generator unit tests to include a layering definition
   which is now required to run the commands.

Change-Id: I2d8086770e9226e44598ef40eca790981279f626
2019-08-06 17:15:02 +00:00
Zuul ed478162ca Merge "Remove duplicate flags as options" 2019-06-18 21:06:04 +00:00
Lev Morgan 52b61b8cfd Added cleartext option to passphrase generation
Added a force-cleartext option (false by default) which forces
passphrases to be generated in cleartext rather than encrypted.

Change-Id: I157a40103f67f85a24976b4f59aa46f2d4b92334
2019-06-12 00:42:03 +00:00
Hughes, Alexander (ah8742) 489e641702 Remove duplicate flags as options
Multiple occurences of -p or -f in the same command is vague.
Removing duplicate shorthand flags in favor of the explicit long form
flags instead for several commands.

Change-Id: Ic26360e517ea8f7ad2e0e5354d34a61fd622e0f1
2019-06-10 13:30:41 +00:00
Zuul 0f3620fd08 Merge "Docfix for pegleg generate pki command" 2019-05-31 03:28:09 +00:00
Ian H. Pittwood d6ee04f9a3 Specify collection for upload command
Currently, using the upload command in Pegleg will upload all discovered
collections to Shipyard by repo. Uploading multiple of these repos can
result in 409 errors during uplift scenarios. This change compiles all
documents into a single collection document that can then be uploaded to
Shipyard.

Requires a collection name to be specified that will be used as the
'collection_id' for uploading to Shipyard.

Buffer mode is set by default to 'replace' instead of 'auto'.

Change-Id: I546b03fd82873296fff10aba355a50e4b11352d0
2019-05-28 16:00:21 -05:00
Zuul 849d8dd6f8 Merge "Make -u required in CLI when required by repo" 2019-05-28 19:56:46 +00:00
Ian H. Pittwood 2fa6a1a7bd Allow decryption of directories
This change allows users to specify a directory or file to be decrypted.

Allows directory decryption.

Adds flag to overwrite encrypted file with decrypted data.

Intelligently recognizes paths vs files in CLI input and outputs data 
accordingly.

Change-Id: I0d5e77f0eb1adb42165aa9b214aa90a0db0a3131
2019-05-28 15:08:14 +00:00
Zuul e4496a5530 Merge "Update decrypt command" 2019-05-23 16:20:59 +00:00
Alexander Hughes e391ce9d86 Docfix for pegleg generate pki command
This patchset fixes:
1. -o and -f options are not available, removed both from documentation
2. Added documentation on the --author flag
3. Updated usage example for generate pki with above flag changes

Change-Id: I4ced0e0fb6bf4ea88d7dd4d3a02bb0825422566c
2019-05-22 17:55:17 +00:00
Alexander Hughes 6375158fc1 Pegleg documentation updates
This patch:
1. Moves defaults for flags in line with the flag for improved
   readability.
2. Removes an example that does not apply to that command.

Change-Id: I16575924c6c7cd6389a70cd687a21c2857889d03
2019-05-16 15:37:14 -05:00
Lev Morgan b8733ea7ec Make -u required in CLI when required by repo
This patch detects when a repository URL requires username substitution
and raises an exception when no username was specified.

Change-Id: Ia60982ecddd957cff8709118b3eb8a905258dd06
2019-05-15 18:38:38 +00:00
Zuul 652c3abefd Merge "Fix examples for genesis_bundle" 2019-05-15 16:22:44 +00:00
Ian H. Pittwood 6c7e735d55 Fix examples for genesis_bundle
This change fixes incorrect references to option `--validators` and
changes them to the correct `--include-validators`.

Change-Id: I3cb8b86cc03d6e37c91e14d625fa20ad9488579b
2019-05-15 10:23:17 -05:00
Rajeshwari Dharwadkar 6ee2aaf845 Support pegleg to run on opensuse leap15 image
Add DISTRO parameter to support multiple distros
Add Dockerfile for opensuse to build leap 15 image.

Change-Id: I7a529476937494e042a4801117489325aa6621c7
2019-05-14 09:41:21 -07:00
Alexander Hughes fb5d54fdb9 Update decrypt command
Decrypt command was previously requiring that specified files have
in their paths the site name.  This isn't necessarily always the case
for example we can have global files that need to be decrypted and do
not contain the site name in the filepath, but the site name is
relevant in ensuring based on the site-definition.yaml file that
pegleg uses the correct revision of the global repository.

The end result should be that when decrypting a file, we specify the
site name, pegleg ensures we're on correct revisions of the repos
and if the file exists, decrypt and print to stdout

This patch addresses this by:
1. Updating pegleg.engine.secrets.decrypt to no longer require a
   site name.
2. Updating pegleg.cli.decrypt to no longer pass a site name to
   pegleg.engine.secrets.decrypt
3. Updating documentation for CLI.
4. Updating unit tests for CLI and secrets.

Change-Id: Ia97518b06a58b069a4d6c0b8d68a37f45e5d31bb
2019-05-09 11:30:54 -05:00
Zuul bf2f1d1d23 Merge "Improve files.write, add decrypt output to file" 2019-05-03 21:39:22 +00:00
Lev Morgan 37f922a07e Improve files.write, add decrypt output to file
Add an option, -s, to write decrypted files to a file rather than
stdout. Decryptyed files have their mode set to 600. Also adds a few
improvements to files.write.

Change-Id: Ia1a6de78d401afbea6ee261652f4650071f54b60
2019-05-02 12:30:44 -05:00
Zuul 50dd505136 Merge "Allow buffer mode to be configured for upload" 2019-05-01 01:53:43 +00:00
Alexander Hughes 9f824f878d Add configurable deckhand render validation
Currently deckhand render validation is disabled by default with no
option to override that behavior from the command line.  Resolve this
by:
1. Adding CLI render flag 'validate', default=True
2. Updating CLI documentation
3. Update pegleg.engine.site.render method to include configurable
   validate flag
4. Update pegleg.engine.util.deckhand.deckhand_render method to
   validate=True by default (previously False)
5. Update pegleg.engine.util.deckhand.deckhand_render method to
   perform deckhand's validate all function on rendered documents

NOTE: Validation logic is handled in deckhand, see
https://opendev.org/airship/deckhand/src/branch/master/deckhand/engine/layering.py
https://opendev.org/airship/deckhand/src/branch/master/deckhand/engine/document_validation.py

Change-Id: I042fad4b2bf08c88e3a2eef6a54dede5d45c28f5
2019-04-30 15:41:40 -05:00
Alexander Hughes c788621486 Allow buffer mode to be configured for upload
Shipyard helper's upload documents method supports two default modes
of buffer.  None, and append.  We want to allow the user to dictate
which mode is used to support the other methods Shipyard itself has.

To accomplish this we add a new command line argument, a new variable
and leave existing behavior intact via the new default 'auto' mode

Change-Id: I7a252efa7fe7a766152c42d9398c3290d7e52a13
2019-04-29 12:43:55 -05:00
Alexander Hughes 50ffabdaf5 Update references from openstack to opendev
The dockerfile and some unit tests were still pointing to review.openstack.org
update those references to review.opendev.org

Change-Id: I161158ac0d66533a1775957864d1bd69dfa9530b
2019-04-24 15:22:53 -05:00
Alexander Hughes 363a3cac36 Update validity checks and docs of PKI functions
https://review.openstack.org/#/c/639414/ was merged with outstanding
comments related to PKI cert expiration checks and PKI cert generation

This patch addresses those critiques to:
1. Make documentation clearer and more standard in format
2. Make code clearer by streamlining a validity check

Change-Id: If5352acd33cfd9e3e177aa11ff8f8ba74d5f55af
2019-04-09 12:36:35 -05:00
Alexander Hughes 7c52ab68da PKI Cert generation and check updates
This patch:
1. Allows user to change valid duration of newly generated certs
default=1yr
2. Allows user to check certs that are expiring soon default=60d

Change-Id: Ia5c87a0c52b39b778f425599fa215fb67147c65b
2019-04-08 07:48:32 -05:00
Lev Morgan 215444ef80 Added document wrapping command
Added a new command, site secrets wrap, to wrap bare files (e.g. pem or
crt) in a PeglegManagedDocument and optionally encrypt them.

Change-Id: I12689275c8e5a8854496fd6bbf69ce6e7cd9ad47
2019-03-27 22:20:45 -05:00
Ahmad Mahmoudi c4f25b4d4f CLI: Add command to generate genesis bundle
Added a pegleg cli command to build genesis.sh bundle for
a site deployment.
Pegleg imports promenade engine, and uses promenade to build
and encrypt the genesis.sh deployment bundle.

Change-Id: I1a489459b2c56b7b53018c32aab5e6550c69e1d2
2019-03-07 03:00:30 -06:00
Alexander Hughes 1aa46d77af Update CLI documentation for generate salt and generate passphrase
Documentation previously did not clearly indicate behavior of invalid lengths
Lengths less than 24 default to 24.

Change-Id: I490ac840f4ed215e6bbbc6cc823eb958076c411c
2019-02-22 10:50:05 -06:00
Alexander Hughes 671b77f6a7 Add CLI generation of salt
Salts and Passphrases are both strings used in cryptography.  This patch:
1. Adds CLI generation of salt
2. Adds unit test for CLI generation of salt
3. Updates passphrase.py code to be more generic as it is used to generate
both a passphrase and a salt
4. Update name of passphrase.py to be more generic
5. Update all references to, and tests of passphrase.py
6. Add documentation for CLI generation of salt

Co-Authored-By: chittibabu <cg329x@att.com>

Change-Id: I71858d63a2846290d22be96686ccfea3ba8aa6c0
2019-02-20 16:51:01 +00:00
Alexander Hughes 4b00a4340c Add CLI passphrase generation
1. Add support to pegleg to generate a passphrase from CLI
2. Update unit test to ensure encryption/decryption supports passphrase rotation
3. Update order of import statements to satisfy pep8
4. Add unit test for CLI passphrase generation
5. Resolve merge conflicts via rebase

Change-Id: I5cb9e41b2f0fac2451bd2b74f33c48cda417c22d
2019-02-04 12:32:39 -06:00
pallav b79d5b7a98 CLI capability to generate and encrypt passphrases
1. Adds the passphrases generation capability in Pegleg CLI,
so that pegleg can generation random passwords based on a
specification declared in pegleg/PassphrasesCatalog documents
2. Pegleg also wraps the generated passphrase documents in
pegleg managed documents, and encrypts the data.
3. Adds unit test cases for passphrase generation.
4. Updates pegleg CLI document.

Change-Id: I21d7668788cc24a8e0cc9cb0fb11df97600d0090
2019-01-29 16:24:31 -06:00
Felipe Monteiro 2a8d2638b3 pki: Port Promenade's PKI catalog into Pegleg
This patch set implements the PKICatalog [0] requirements
as well as PeglegManagedDocument [1] generation requirements
outlined in the spec [2].

Included in this patch set:

* New CLI entry point called "pegleg site secrets generate-pki"
* PeglegManagedDocument generation logic in
  engine.cache.managed_document
* Refactored PKICatalog logic in engine.cache.pki_catalog derived
  from the Promenade PKI implementation [3], responsible for
  generating certificates, CAs, and keypairs
* Refactored PKIGenerator logic in engine.cache.pki_generator
  derived from Promenade Generator implementation [4],
  responsible for reading in pegleg/PKICatalog/v1 documents (as
  well as promenade/PKICatalog/v1 documents for backwards
  compatibility) and generating required secrets and storing
  them into the paths specified under [0]
* Unit tests for all of the above [5]
* Example pki-catalog.yaml document under pegleg/site_yamls
* Validation schema for pki-catalog.yaml (TODO: implement
  validation logic here: [6])
* Updates to CLI documentation and inclusion of PKICatalog
  and PeglegManagedDocument documentation
* Documentation updates with PKI information [7]

TODO (in follow-up patch sets):

* Expand on overview documentation to include new Pegleg
  responsibilities
* Allow the original repository (not the copied one) to
  be the destination where the secrets are written to
* Finish up cert expiry/revocation logic

[0] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#document-generation
[1] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html#peglegmanageddocument
[2] https://airship-specs.readthedocs.io/en/latest/specs/approved/pegleg-secrets.html
[3] https://github.com/openstack/airship-promenade/blob/master/promenade/pki.py
[4] https://github.com/openstack/airship-promenade/blob/master/promenade/generator.py
[5] https://review.openstack.org/#/c/611739/
[6] https://review.openstack.org/#/c/608159/
[7] https://review.openstack.org/#/c/611738/

Change-Id: I3010d04cac6d22c656d144f0dafeaa5e19a13068
2019-01-15 13:29:21 -06:00
Evgeny L d3010a213f Update docs to replace `aic-site-clcp-manifests` with `airship-treasuremap`
Change-Id: Ic53d5d768fcecaac07b4703ce818fe73a52d1c56
2018-12-26 21:01:15 +00:00
Nishant Kumar 9113d249ff CLI: Add support for uploading documents to Shipyard
This PS enables Pegleg to upload documents directly to Shipyard
thus ensuring that unencrypted data never gets stored in disk.

The flow for this new CLI command is as follows:

- Collect documents as per the provided site repository
- Decrypt the collected documets(TODO)
- Upload document to Shipyard:
  - one collection per repository will be uploaded to Shipyard

    Eg-
        pegleg site -r /opt/aic-clcp-site-manifests \
        -e global=/opt/aic-clcp-manifests upload <site-name>

        Two collections will be created in shipyard since there are two
        repositories provided. The name of the collections will be the
        name of repositories provided.
  - Commit the documents in shipyard buffer.

Change-Id: I6275252b044ebb82d8bb2009c0bea6ebf7033bce
2018-11-27 19:29:42 +00:00
zhangdebo e3159d223a Fix link addresses
Change-Id: I7086b4cac5e3bcd2ac2a1958b678615e05ccbd1d
2018-11-27 20:09:48 +08:00
Ahmad Mahmoudi fb8e6f73ac Update decrypt secrets to return a list of docs
1. Added the method to decrypt a secret file and return its contents
as a list of documents (instead of printing out the file content).
2. Added clarifications for a encrypt and decrypt commands.

Change-Id: I77bce21be214c880c8413f5e6a2d0c2d1993fc8e
2018-11-06 00:22:10 -06:00
Ahmad Mahmoudi eb0deeb9e5 Pegleg encryption of site secrets
Added secret encryption/decryption to pegleg cli.

Change-Id: I95b993748d99fc4398eee1d1c59e74f382497f74
2018-10-30 16:53:51 +00:00
Zuul 62d0faf2ef Merge "docs: Rename downstream-specific terms to generic ones" 2018-10-25 05:24:19 +00:00