diff --git a/README.rst b/README.rst index 4d79e9a7..858a2934 100644 --- a/README.rst +++ b/README.rst @@ -2,8 +2,10 @@ Deckhand ======== -Deckhand is a document-based configuration storage service built with -auditability and validation in mind. +Deckhand is a storage service for YAML-based configuration documents, which are +managed through version control and automatically validated. Deckhand provides +users with a variety of different document types that describe complex +configurations using the features listed below. Core Responsibilities ===================== diff --git a/doc/source/buckets.rst b/doc/source/buckets.rst deleted file mode 100644 index efe611a7..00000000 --- a/doc/source/buckets.rst +++ /dev/null @@ -1,32 +0,0 @@ -.. - Copyright 2017 AT&T Intellectual Property. - All Rights Reserved. - - Licensed under the Apache License, Version 2.0 (the "License"); you may - not use this file except in compliance with the License. You may obtain - a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, WITHOUT - WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the - License for the specific language governing permissions and limitations - under the License. - -.. _bucket: - -Buckets -======= - -Collections of documents, called buckets, are managed together. All documents -belong to a bucket and all documents that are part of a bucket must be fully -specified together. - -To create or update a new document in, e.g. bucket ``mop``, one must PUT the -entire set of documents already in ``mop`` along with the new or modified -document. Any documents not included in that PUT will be automatically -deleted in the created revision. - -This feature allows the separation of concerns when delivering different -categories of documents, while making the delivered payload more declarative. diff --git a/doc/source/conf.py b/doc/source/conf.py index 6d6697ca..414f75e5 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -83,7 +83,7 @@ exclude_patterns = [] pygments_style = 'sphinx' # If true, `todo` and `todoList` produce output, else they produce nothing. -todo_include_todos = False +todo_include_todos = True # -- Options for HTML output ---------------------------------------------- diff --git a/doc/source/glossary.rst b/doc/source/glossary.rst index 8ea9a642..021366f9 100644 --- a/doc/source/glossary.rst +++ b/doc/source/glossary.rst @@ -29,7 +29,8 @@ B bucket - Kind of like a Github repository, an ownership class for documents. + A bucket manages collections of documents together, providing write + protections around them. D ~ diff --git a/doc/source/index.rst b/doc/source/index.rst index fe69fbc3..bf691330 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -38,9 +38,9 @@ User's Guide .. toctree:: :maxdepth: 2 + overview documents document_types - buckets validation substitution layering diff --git a/doc/source/overview.rst b/doc/source/overview.rst new file mode 100644 index 00000000..17c4c631 --- /dev/null +++ b/doc/source/overview.rst @@ -0,0 +1,131 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +Overview +======== + +Deckhand is a storage service for YAML-based configuration documents. Deckhand +stores the documents using version control: Each time a collection of documents +is passed to Deckhand, a new revision is created. Thus, documents have a +revision history, allowing complex configurations to be incrementally modified +and validated. For example, if the first revision of documents fail validation, +deployers can make modifications to the documents and submit them to Deckhand +again, until the documents pass validation and are ready to be rendered into +their finalized state. + +Core Responsibilities +===================== + +* *revision history* - improves auditability and enables services to provide + functional validation of a well-defined collection of documents that are + meant to operate together +* *validation* - allows services to implement and register different kinds of + validations and report errors +* *buckets* - allow documents to be owned by different services, providing + write protections around collections of documents +* *layering* - helps reduce duplication in configuration while maintaining + auditability across many sites +* *substitution* - provides separation between secret data and other + configuration data, while also providing a mechanism for documents to + share data among themselves + +Revision History +---------------- + +Like other version control software, Deckhand allows users to track incremental +changes to documents via a revision history, built up through individual +payloads to Deckhand, each forming a separate revision. Each revision, in other +words, contains its own set of immutable documents: Creating a new revision +maintains the existing revision history. + +For more information, see the :ref:`revision-history` section. + +Validation +---------- + +For each created revision, built-in :ref:`document-types` are automatically +validated. Validations are always stored in the database, including detailed +error messages explaining why validation failed, to help deployers rectify +syntactical or semantical issues with configuration documents. Regardless of +validation failure, a new revision is **always** created, except when the +documents are completely malformed. + +Deckhand validation functionality is extensible via ``DataSchema`` documents, +allowing the ``data`` sections of registered document types to be subjected +to user-provided JSON schemas. + +.. note:: + + While Deckhand ingests YAML documents, internally it translates them to + Python objects and can use JSON schemas to valdiate those objects. + +For more information, see the :ref:`validation` section. + +Buckets +------- + +Collections of documents, called buckets, are managed together. All documents +belong to a bucket and all documents that are part of a bucket must be fully +specified together. + +To create or update a new document in, e.g. bucket ``mop``, one must PUT the +entire set of documents already in ``mop`` along with the new or modified +document. Any documents not included in that PUT will be automatically +deleted in the created revision. + +Each bucket provides write protections around a group of documents. That is, +only the bucket that owns a collection of documents can manage those documents. +However, documents can be read across different buckets and used together to +render finalized configuration documents, to be consumed by other services like +Armada, Drydock, Promenade or Shipyard. + +.. todo:: + + Deckhand should offer RBAC (Role-Based Access Control) around buckets. This + will allow deployers to control permissions around who can write or read + documents to or from buckets. + +.. note:: + + The best analogy for a bucket is a folder. Like a folder, which houses files + and offers read and write permissions, a bucket houses documents and offers + read and write permissions around them. + + A bucket is **not** akin to a repository, because a repository has its own + distinct revision history. A bucket, on the other hand, shares its revision + history with every other bucket. + +Layering +-------- + +Layering provides a restricted data inheritance model intended to help reduce +duplication in configuration. A ``LayeringPolicy`` can be created to declare +the order of inheritance via layers for documents. Parent documents can +provide common data to child documents, who can override their parent data +or tweak it in order to achieve more nuanced configuration that builds on top +of common configurations. + +For more information, see the :ref:`layering` section. + +Substitution +------------ + +Substitution is a mechanism for documents to share data among themselves. It +is particularly useful for documents that possess secrets to be stored securely +and on demand provide the secrets to documents that need them. However, +substitution can also apply to any data, not just secrets. + +For more information, see the :ref:`substitution` section.