Merge "Spec: Pegleg encryption and decryption"

specs/approved/pegleg-secrets.rst

@@ -43,9 +43,8 @@ The following Airship components will be impacted by this solution:
 
 #. Pegleg: enhanced to generate, rotate, encrypt, and decrypt secrets.
 #. Promenade: PKICatalog will move to Pegleg.
-#. Treasuremap: site manifests augmented to support the updated Secrets schema.
+#. Treasuremap: update site manifests to use new Catalogs.
-#. Airship-in-a-Bottle: site manifests augmented to support the updated
+#. Airship-in-a-Bottle: update site manifests to use new Catalogs.
-   Secrets schema.
 
 Proposed change
 ===============
@@ -79,7 +78,7 @@ example::
       abstract: false
       # Pegleg will initially support generation at site level only
       layer: site
-    storagePolicy: encrypted
+    storagePolicy: cleartext
   data:
     generated:
       at: <timestamp>
@@ -89,6 +88,7 @@ example::
         reference: <git ref-head or similar>
         path: <PKICatalog/PassphraseCatalog details>
     managedDocument:
+      schema: <as appropriate for wrapped document>
       metadata:
         storagePolicy: encrypted
         schema: <as appropriate for wrapped document>
@@ -109,12 +109,13 @@ example::
     layeringDefinition:
       abstract: false
       layer: matching-wrapped-doc
-    storagePolicy: encrypted
+    storagePolicy: cleartext
   data:
     encrypted:
       at: <timestamp>
       by: <author>
     managedDocument:
+      schema: <as appropriate for wrapped document>
       metadata:
         storagePolicy: encrypted
         schema: <as appropriate for wrapped document>
@@ -126,7 +127,7 @@ A PeglegManagedDocument that is both generated via a Catalog, and encrypted
 (as specified by the catalog) will contain both ``generated`` and
 ``encrypted`` stanzas.
 
-Note that this ``encrypted`` has a different purpose than the Deckhand
+Note that this ``encrypted`` key has a different purpose than the Deckhand
 ``storagePolicy: encrypted`` metadata, which indicates an *intent* for Deckhand
 to store a document encrypted at rest in the cluster.  The two can be used
 together to ensure security, however:  if a document is marked as
@@ -134,6 +135,11 @@ together to ensure security, however:  if a document is marked as
 persisted (e.g. to a Git repository) if it is in fact encrypted within
 a PeglegManagedDocument.
 
+Note also that the Deckhand ``storagePolicy`` of the PeglegManagedDocument
+itself is always ``cleartext``, since its data stanza is not encrypted -- it
+only wraps a document that *is* ``storagePolicy: encrypted``.
+This should be implemented as a Pegleg lint rule.
+
 Document Generation
 -------------------
 
@@ -204,7 +210,10 @@ The nonobvious bits of the document described above are:
   replace dashes in the ``document_name`` with underscores.
 * ``length`` is optional, and denotes the length in characters of the
   generated cleartext passphrase data.  If absent, ``length`` defaults
-  to ``24``.
+  to ``24``.  Note that with this length and the selected character set there
+  will be less than 8x10^48 probability of getting a new passphrase that is
+  identical to the previous passphrase.  This is sufficiently random to
+  ensure no duplication of rotated passphrases in practice.
 * ``description`` is optional.
 
 The ``encrypted`` key will be added to the PKICatalog schema, and adds the same
@@ -220,16 +229,27 @@ Committing and pushing the changes will be left to the
 operator or to script-based automation.
 
 For the CLI commands below which encrypt or decrypt secrets, an environment
-variable (e.g. ``$PEGLEG_KEY`` will be use to capture the key/passphrase to use.
+variable (e.g. ``PEGLEG_PASSPHRASE`` will be use to capture the master
-``pegleg site secrets rotate`` will use a second variable
+passphrase to use.  ``pegleg site secrets rotate`` will use a second variable
-(e.g. ``$PEGLEG_PREVIOUS_KEY``) to hold the key/passphrase being rotated
+(e.g. ``PEGLEG_PREVIOUS_PASSPHRASE``) to hold the key/passphrase being rotated
-out.
+out.  The contents of these keys/passphrases are not generated by Pegleg,
+but are created externally and set by a deployment engineer or tooling.
+A configurable minimum length (default 24) for master passphrases will
+be checked by all CLI commands which use the passphrase.  All other criteria
+around passphrase strength are assumed to be enforced elsewhere, as it is an
+external secret that is consumed/used by Pegleg.
 
 ``pegleg site secrets generate passphrases``:  Generate passphrases according to
 all PassphraseCatalog documents in the site.
 Note that regenerating passphrases can be accomplished
 simply by re-running ``pegleg site secrets generate passphrases``.
 
+``pegleg generate passphrase``:  A standalone version of passphrase generation.
+This generates a single passphrase based on the default length, character set,
+and implementation described above, and outputs it to the console.  The
+PassphraseCatalog is not involved in this operation.  This command is suitable
+for generation of a highly-secure Pegleg master passphrase.
+
 ``pegleg site secrets generate pki``:  Generate certificates and keys according
 to all PKICatalog documents in the site.
 Note that regenerating certificates can be accomplished
@@ -258,22 +278,28 @@ original document YAML to standard output.  This is intended to be used when
 an authorized deployment engineer needs to determine a particular cleartext
 secret for a specific operational purpose.
 
-``pegleg site secrets rotate``:  This action re-encrypts encrypted secrets
+``pegleg site secrets rotate passphrases``:  This action re-encrypts
-with a new key/passphrase, and it takes the previously-used key and a new
+encrypted passphrases with a new key/passphrase, and it takes the
-key as input.  It accomplishes its task via two activities:
+previously-used key and a new key as input.  It accomplishes its task via
+two activities:
 
-* For encrypted secrets that were imported from outside of Pegleg
+* For encrypted passphrases that were imported from outside of Pegleg
   (i.e. PeglegManagedDocuments which lack the ``generated`` stanza),
   decrypt them with the old key (in-memory), re-encrypt them with
   the new key, and output the results.
-* Perform a fresh ``pegleg site secrets generate`` process using the new key.
+* Perform a fresh ``pegleg site secrets generate passphrases`` process
-  This will replace all ``generated`` secrets with new secret values
+  using the new key.
+  This will replace all ``generated`` passphrases  with new secret values
   for added security.  There is an assumption here that the only actors
   that need to know generated secrets are the services within the
   Airship-managed cluster, not external services or deployment engineers,
   except perhaps for point-in-time troubleshooting or operational
   exercises.
 
+Similar functionality for rotating certificates (which is expected to have
+a different cadence than passphrase rotation, typically) will be
+added in the future.
+
 Driving deployment of a site directly via Pegleg is follow-on functionality
 which will
 collect site documents, use them to create the ``genesis.sh`` script, and then
@@ -289,7 +315,7 @@ PeglegManagedDocuments will be written (encrypted) to disk.
 To enable special case full site secret decryption, a ``--force-decrypt`` flag
 will be added to ``pegleg collect`` to do this under controlled circumstances,
 and to help bridge the gap with existing CICD pipelines until Pegleg-driven
-site deployment is in place.  It will leverage the ``$PEGLEG_KEY``
+site deployment is in place.  It will leverage the ``PEGLEG_PASSPHRASE``
 variable described above.
 
 Secret Generation
@@ -309,7 +335,33 @@ Python string.ascii_letters, string.digits, and string.punctuation.
 Secret Encryption
 -----------------
 
-Details around encryption will be defined in a follow-on patch set to this spec.
+The Python ``cryptography`` library has been chosen to implement the
+encryption and decryption of secrets within Pegleg.  ``cryptography``
+aims to be the standard cryptographic approach for Python, and takes
+pains to make it difficult to do encryption poorly (via its ``recipes``
+layer), while still allowing access to the algorithmic details when
+truly needed (via its ``hazmat`` layer).  ``cryptography`` is actively
+maintained and is the target encryption library for OpenStack as well.
+
+The ``cryptography.fernet`` module will be used for symmetric encryption.
+It uses AES with a 128-bit key for encryption, and HMAC using SHA256
+for encryption.
+
+Fernet requires as input a URL-safe, base64-encoded 32-byte encryption key,
+which will be derived from the master passphrase passed into Pegleg via
+``PEGLEG_PASSPHRASE`` as described above.
+The example for password-based encryption from the `Fernet documentation`_
+should be followed as a guide. The ``salt`` to be used in key derivation
+will be configurable, and will be set to a fixed value within a built
+Pegleg container via an environment variable passed into the Pegleg
+Dockerfile.  This will allow the salt to be different on an
+operator-by-operator basis.
+
+The ``cryptography.exceptions.InvalidSignature`` exception is thrown by
+``cryptography`` when an attempt is made to decrypt a message with a key that
+is different than the one used to encrypt a message, i.e., when the user has
+supplied an incorrect phassphrase.  It should be handled gracefully by Pegleg,
+resulting in an informative message back to the user.
 
 Security impact
 ===============
@@ -346,6 +398,10 @@ the point-in-time encryption status.  ``storagePolicy`` is still valuable
 in this context to make sure everything that *should* be encrypted *is*,
 prior to performing actions with it (e.g. Git commits).
 
+The ``PyCrypto`` library is a popular solution for encryption in Python;
+however, it is no longer actively maintained.  Following the lead of OpenStack
+and others, we opted instead for the ``cryptography`` library.
+
 This proposed implementation writes the output of generation/encryption events
 back to the same source files from which the original data came.  This is a
 destructive operation; however, it wasn't evident that it is problematic in
@@ -370,3 +426,4 @@ References
 
 .. _Storyboard Story: https://storyboard.openstack.org/#!/story/2003708
 .. _Git branch and revision support: https://review.openstack.org/#/c/577886/
+.. _Fernet documentation: https://cryptography.io/en/latest/fernet/