diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..01950163 --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +# Sphinx documentation +docs/_build/ +docs/build/ \ No newline at end of file diff --git a/README.md b/README.md index 549b50ce..6b8ce1ba 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,9 @@ UCP, or Undercloud Platform, is a broad integration of several components enabling an automated, resilient Kubernetes-based infrastructure for hosting Helm-deployed containerized workloads +Find documentation for Undercloud Platform Integration on +[readthedocs](http://ucpintegration.readthedocs.org). + ## Components ### Shipyard diff --git a/docs/api-conventions.md b/docs/api-conventions.md deleted file mode 100644 index 593938df..00000000 --- a/docs/api-conventions.md +++ /dev/null @@ -1,241 +0,0 @@ -# UCP API conventions -A collection of conventions that components of the UnderCloud Platform (UCP) -utilize for their REST APIs ---- -## Resource path naming - -* Resource paths nodes follow an all lower case naming scheme, and -pluralize the resource names. Nodes that refer to keys, ids or names that are -externally controlled, the external naming will be honored. -* The version of the API resource path will be prefixed before the first -node of the path for that resource using v#.# format. -* By default, the API will be namespaced by /api before the version. For -the purposes of documentation, this will not be specified in each of the -resource paths below. In more complex APIs, it makes sense to allow the /api -node to be more specific to point to a particular service. - -``` -/api/v1.0/sampleresources/ExTeRnAlNAME-1234 - ^ ^ ^ ^ - | | | defer to external naming - | | plural - | lower case - version here -``` ---- -## Status responses -Status responses, and more specifically error responses (HTTP response body -accompanying 4xx and 5xx series responses -where possible) are a customized version of the -[Kubernetes standard for error representation](https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#response-status-kind). -UCP utilizes the details field in a more formalized way to represent multiple -messages related to a status response, as follows: - -``` -{ - "kind": "Status", - "apiVersion": "v1", - "metadata": {}, - "status": "Failure", - "message": "{{UCP Component Name}} {{status phrase}}", - "reason": "{{appropriate reason phrase}}", - "details": { - "errorCount": {{n}}, - "messageList": [ - { "message" : "{{validation failure message}}", "error": true|false}, - ... - ] - }, - "code": {{http status code}} -} -``` - -such that: -* the details field is still optional -* if used, the details follow that format -* the repeating entity inside the messageList can be decorated with as many -other fields as are useful, but at least have a message field and error field. -* the errorCount field is an integer representing the count of messageList -entities that have `error: true` -* when using this document as the body of a HTTP response, `code` is -populated with a valid HTTP -[status code](https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html) - ---- -## Headers -### Required - -
-
X-Auth-Token
-
The auth token to identify the invoking user.
-
- -### Optional - -
-
X-Context-Marker
-
A context id that will be carried on all logs for this client-provided -marker. This marker may only be a 36-character canonical representation of an -UUID (8-4-4-4-12)
-
- -## Validation API -All UCP components that participate in validation of the design supplied to a -site implement a common resource to perform document validations. Document -validations are synchronous and target completion in 30 seconds or less. -Because of the different sources of documents that should be supported, a -flexible input descriptor is used to indicate from where a UCP component will -retrieve the documents to be validated. - -### POST /v1.0/validatedesign -Invokes a UCP component to perform validations against the documents specified -by the input structure. Synchronous. - -#### Input structure -``` -{ - rel : "design", - href: "deckhand+https://{{deckhand_url}}/revisions/{{revision_id}}/rendered-documents", - type: "application/x-yaml" -} -``` -#### Output structure -The output structure reuses the Kubernetes Status kind to represent the result -of validations. The Status kind will be returned for both successful and failed -validation to maintain a consistent of interface. If there are additional -diagnostics that associate to a particular validation, the entry in the error -list may carry fields other than "message". - -Failure message example: -``` -{ - "kind": "Status", - "apiVersion": "v1", - "metadata": {}, - "status": "Invalid", - "message": "{{UCP Component Name}} validations failed", - "reason": "Validation", - "details": { - "errorCount": {{n}}, - "messageList": [ - { "message" : "{{validation failure message}}", "error": true}, - ... - ] - }, - "code": 400 -} -``` - -Success message example: -``` -{ - "kind": "Status", - "apiVersion": "v1", - "metadata": {}, - "status": "Valid", - "message": "{{UCP Component Name}} validations succeeded", - "reason": "Validation", - "details": { - "errorCount": 0, - "messageList": [] - }, - "code": 200 -} -``` - -## Health Check API -Each UCP component shall expose an endpoint that allows other components -to access and validate its health status. The response shall be received -within 30 seconds. - -### GET /v1.0/health -Invokes a UCP component to return its health status - -#### Health Check Output -The current design will be for the UCP component to return an empty response -to show that it is alive and healthy. This means that the UCP component that -is performing the query will receive HTTP response code 204. - -HTTP response code 503 will be returned if the UCP component fails to receive -any response from the component that it is querying. The time out will be set -to 30 seconds. - -### GET /v1.0/health/extended -Invokes a UCP component to return its detailed health status. Authentication -will be required to invoke this API call. - -This feature will be implemented in the future. - -#### Extended Health Check Output -The output structure reuses the Kubernetes Status kind to represent the health -check results. The Status kind will be returned for both successful and failed -health checks to ensure consistencies. The message field will contain summary -information related to the results of the health check. Detailed information -of the health check will be provided as well. - -Failure message example: -``` -{ - "kind": "Status", - "apiVersion": "v1", - "metadata": {}, - "status": "Service Unavailable", - "message": "{{UCP Component Name}} failed to respond", - "reason": "Health Check", - "details": { - "errorCount": {{n}}, - "messageList": [ - { "message" : "{{Detailed Health Check failure information}}", "error": true}, - ... - ] - }, - "code": 503 -} -``` - -Success message example: -``` -{ - "kind": "Status", - "apiVersion": "v1", - "metadata": {}, - "status": "Healthy", - "message": "", - "reason": "Health Check", - "details": { - "errorCount": 0, - "messageList": [] - }, - "code": 200 -} -``` - -## Versions API -Each UCP component shall expose an endpoint that allows other components to -discover its different API versions. - -### GET /versions -Invokes a UCP component to return its list of API versions. - -#### Versions output -Each UCP component shall return a list of its different API versions. The -response body shall be keyed with the name of each API version, with -accompanying information pertaining to the version's `path` and `status`. The -`status` field shall be an enum which accepts the values "stable" and "beta", -where "stable" implies a stable API and "beta" implies an under-development -API. - -Success message example: -``` -{ - "v1.0": { - "path": "/api/v1.0", - "status": "stable" - }, - "v1.1": { - "path": "/api/v1.1", - "status": "beta" - }, - "code": 200 -} -``` diff --git a/docs/requirements.txt b/docs/requirements.txt new file mode 100644 index 00000000..ddc6a294 --- /dev/null +++ b/docs/requirements.txt @@ -0,0 +1,3 @@ +# Documentation +sphinx>=1.6.2 +sphinx_rtd_theme==0.2.4 \ No newline at end of file diff --git a/docs/source/alarming-conditions.rst b/docs/source/alarming-conditions.rst new file mode 100644 index 00000000..5410f8f3 --- /dev/null +++ b/docs/source/alarming-conditions.rst @@ -0,0 +1,21 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _alarming-conditions: + +UCP Alarms and Alert Conditions +=============================== +Under Development diff --git a/docs/source/api-conventions.rst b/docs/source/api-conventions.rst new file mode 100644 index 00000000..4f250de3 --- /dev/null +++ b/docs/source/api-conventions.rst @@ -0,0 +1,291 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _api-conventions: + +API Conventions +=============== + +A collection of conventions that components of the UnderCloud Platform (UCP) +utilize for their REST APIs + +Resource path naming +-------------------- + +- Resource paths nodes follow an all lower case naming scheme, and + pluralize the resource names. Nodes that refer to keys, ids or names that + are externally controlled, the external naming will be honored. +- The version of the API resource path will be prefixed before the first + node of the path for that resource using v#.# format. +- By default and unless otherwise noted, the API will be namespaced by /api + before the version. For the purposes of documentation, this will not be + specified in each of the resource paths below. In more complex APIs, UCP + components may use values other than /api to be more specific to point to a + particular service. + +:: + + /api/v1.0/sampleresources/ExTeRnAlNAME-1234 + ^ ^ ^ ^ + | | | defer to external naming + | | plural + | lower case + version here + +Status responses +---------------- + +Status responses, and more specifically error responses (HTTP response body +accompanying 4xx and 5xx series responses where possible) are a customized +version of the `Kubernetes standard for error representation`_. UCP utilizes +the details field in a more formalized way to represent multiple messages +related to a status response, as follows: + +:: + + { + "kind": "Status", + "apiVersion": "v{{#.#}}", + "metadata": {}, + "status": "{{Success | Failure}}", + "message": "{{message phrase}}", + "reason": "{{reason name}}", + "details": { + "errorCount": {{n}}, + "messageList": [ + { "message" : "{{validation failure message}}", "error": true|false}, + ... + ] + }, + "code": {{http status code}} + } + + +such that: + +* The metadata field is optionally present, as an empty object. Clients should + be ready to receive this field, but services are not required to produce it. +* The message phrase is a terse but descriptive message indicating what has + happened. +* The reason name is the short name indicating the cause of the status. It + should be a camel cased phrase-as-a-word, to mimic the Kubernetes status + usage. +* The details field is optional. +* If used, the details follow the shown format, with a errorCount and + messageList field present. + + - The repeating entity inside the messageList can be decorated with as + many other fields as are useful, but at least have a message field and + error field. + - The errorCount field is an integer representing the count of messageList + entities that have ``error: true`` + +* When using this document as the body of a HTTP response, ``code`` is + populated with a valid `HTTP status code`_ + +Required Headers +---------------- + +X-Auth-Token + The auth token to identify the invoking user. Required unless the resource is + explictly unauthenticated. + +Optional Headers +---------------- + +X-Context-Marker + A context id that will be carried on all logs for this client-provided + marker. This marker may only be a 36-character canonical representation of an + UUID (8-4-4-4-12) + +Validation API +-------------- +All UCP components that participate in validation of the design supplied to a +site implement a common resource to perform document validations. Document +validations are synchronous. +Because of the different sources of documents that should be supported, a +flexible input descriptor is used to indicate from where a UCP component will +retrieve the documents to be validated. + +POST /v1.0/validatedesign +~~~~~~~~~~~~~~~~~~~~~~~~~ +Invokes a UCP component to perform validations against the documents specified +by the input structure. Synchronous. + +Input structure +^^^^^^^^^^^^^^^ + +:: + + { + rel : "design", + href: "deckhand+https://{{deckhand_url}}/revisions/{{revision_id}}/rendered-documents", + type: "application/x-yaml" + } + +Output structure +^^^^^^^^^^^^^^^^ + +The output structure reuses the Kubernetes Status kind to represent the result +of validations. The Status kind will be returned for both successful and failed +validation to maintain a consistent of interface. If there are additional +diagnostics that associate to a particular validation, the entry in the error +list may carry fields other than "message". + +Failure message example:: + + { + "kind": "Status", + "apiVersion": "v1.0", + "metadata": {}, + "status": "Failure", + "message": "{{UCP Component Name}} validations failed", + "reason": "Validation", + "details": { + "errorCount": {{n}}, + "messageList": [ + { "message" : "{{validation failure message}}", "error": true}, + ... + ] + }, + "code": 400 + } + +Success message example:: + + { + "kind": "Status", + "apiVersion": "v1.0", + "metadata": {}, + "status": "Success", + "message": "{{UCP Component Name}} validations succeeded", + "reason": "Validation", + "details": { + "errorCount": 0, + "messageList": [] + }, + "code": 200 + } + +Health Check API +---------------- +Each UCP component shall expose an endpoint that allows other components +to access and validate its health status. Clients of the health check should +wait up to 30 seconds for a health check response from each component. + +GET /v1.0/health +~~~~~~~~~~~~~~~~ +Invokes a UCP component to return its health status. This endpoint is intended +to be unauthenticated, and must not return any information beyond the noted +204 or 503 status response. The component invoked is expected to return a +response in less than 30 seconds. + +Health Check Output +^^^^^^^^^^^^^^^^^^^ +The current design will be for the UCP component to return an empty response +to show that it is alive and healthy. This means that the UCP component that +is performing the query will receive HTTP response code 204. + +HTTP response code 503 with a generic response status or an empty message body +will be returned if the UCP component determines it is in a non-healthy state, +or is unable to reach another component it is dependent upon. + +GET /v1.0/health/extended +~~~~~~~~~~~~~~~~~~~~~~~~~ +UCP components may provide an extended health check. This request invokes a +UCP component to return its detailed health status. Authentication is required +to invoke this API call. + +Extended Health Check Output +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +The output structure reuses the Kubernetes Status kind to represent the health +check results. The Status kind will be returned for both successful and failed +health checks to ensure consistencies. The message field will contain summary +information related to the results of the health check. Detailed information +of the health check will be provided as well. + +Failure message example:: + + { + "kind": "Status", + "apiVersion": "v1.0", + "metadata": {}, + "status": "Failure", + "message": "{{UCP Component Name}} failed to respond", + "reason": "HealthCheck", + "details": { + "errorCount": {{n}}, + "messageList": [ + { "message" : "{{Detailed Health Check failure information}}", "error": true}, + ... + ] + }, + "code": 503 + } + +Success message example:: + + { + "kind": "Status", + "apiVersion": "v1.0", + "metadata": {}, + "status": "Success", + "message": "", + "reason": "HealthCheck", + "details": { + "errorCount": 0, + "messageList": [] + }, + "code": 200 + } + +Versions API +------------ +Each UCP component shall expose an endpoint that allows other components to +discover its different API versions. This endpoint is not prefixed by /api +or a version. + +GET /versions +~~~~~~~~~~~~~ +Invokes a UCP component to return its list of API versions. This endpoint is +intended to be unauthenticated, and must not return any information beyond the +output noted below. + +Versions output +^^^^^^^^^^^^^^^ +Each UCP component shall return a list of its different API versions. The +response body shall be keyed with the name of each API version, with +accompanying information pertaining to the version's `path` and `status`. The +`status` field shall be an enum which accepts the values `stable` and `beta`, +where `stable` implies a stable API and `beta` implies an under-development +API. + +Success message example:: + + { + "v1.0": { + "path": "/api/v1.0", + "status": "stable" + }, + "v1.1": { + "path": "/api/v1.1", + "status": "beta" + }, + "code": 200 + } + +.. _Kubernetes standard for error representation: https://github.com/kubernetes/community/blob/master/contributors/devel/api-conventions.md#response-status-kind +.. _HTTP status code: https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html \ No newline at end of file diff --git a/docs/source/client-software.rst b/docs/source/client-software.rst new file mode 100644 index 00000000..b86d83cf --- /dev/null +++ b/docs/source/client-software.rst @@ -0,0 +1,21 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _client-software: + +UCP Client Software Conventions +=============================== +Under Development diff --git a/docs/source/code-conventions.rst b/docs/source/code-conventions.rst new file mode 100644 index 00000000..14dbf8c5 --- /dev/null +++ b/docs/source/code-conventions.rst @@ -0,0 +1,207 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _code-conventions: + +Code and Project Conventions +============================ + +Conventions and standards that guide the development and arrangement of UCP +component projects. + +Project Structure +----------------- + +Charts +~~~~~~ +Each project that maintains helm charts will keep those charts in a directory +``charts`` located at the root of the project. The charts directory will +contain subdirectories for each of the charts maintained as part of that +project. These subdirectories should be named for the component represented by +that chart. + +e.g.: For project ``foo``, which also maintains the charts for ``bar`` and +``baz``: +- foo/charts/foo contains the chart for ``foo`` +- foo/charts/bar contains the chart for ``bar`` +- foo/charts/baz contains the chart for ``baz`` + +Helm charts utilize the `helm-toolkit`_ supported by the `Openstack-Helm`_ team +and follow the standards documented there. + +Images +~~~~~~ +Each project that creates a `Docker`_ image will keep the dockerfile in a +directory ``images`` located at the root of the project. The images directory +will contain subdirectories for each of the images created as part of that +project. The subdirectory will contain the dockerfile that can be used to +generate the image. + +e.g.: For project ``foo``, which also produces a Docker image for ``bar`` +- foo/images/foo contains the dockerfile for ``foo`` +- foo/images/bar contains the dockerfile for ``bar`` + +Makefile +~~~~~~~~ +Each project must provide a makefile at the root of the project. The makefile +should implement each of the following makefile targets: + +- ``images`` will produce the docker images for the component and each other + component it is responsible for building. +- ``charts`` will helm package all of the charts maintained as part of the + project. +- ``lint`` will perform code linting for the code and chart linting for the + charts maintained as part of the project, as well as any other reasonable + linting activity. +- ``dry-run`` will produce a helm template for the charts maintained as part + of the project. +- ``all`` will run the lint, charts, and images targets. +- ``docs`` should render any documentation that has build steps. +- ``run_{component_name}`` should build the image and do a rudimentary (at + least) test of the image's functionality. +- ``run_images`` performs the inidividual run_{component_name} targets for + projects that produce more than one image. + +Other makefile targets may exist. A ``test`` endpoint is encouraged. For +projects that are Python based, the makefile targets typically reference tox +commands, and those projects will include a tox.ini defining the tox targets. +Note that tox.ini files will reside inside the source directories for modules +within the project, but a top-level tox.ini may exist at the root of the +repository that includes the necessary targets to build documentation. + +Documentation +~~~~~~~~~~~~~ +Also see :ref:`documentation-conventions` + +Documentation source for the component should reside in a 'docs' directory at +the root of the project. + +Linting and Formatting Standards +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Code in the UCP components should follow the prevalent linting and formatting +standards for the language being implemented. In lieu of industry accepted +code formatting standards for a target language, strive for readability and +maintainability. + +=============== ====================================== +Known Standards +------------------------------------------------------- +Language Uses +=============== ====================================== +Python PEP-8 +=============== ====================================== + +UCP components must provide for automated checking of their formatting +standards, such as the lint step noted above in the makefile. Components may +provide automated reformatting. + +Tests Location +~~~~~~~~~~~~~~ +Tests should be in parallel structures to the related code, unless dictated by +target language ecosystem. + +For Python projects, the preferred location for tests is a ``tests`` directory +under the directory for the module. E.g. Tests for module foo: +{root}/src/bin/foo/foo/tests. +An alternataive location is ``tests`` at the root of the project, although this +should only be used if there are not multiple components represented in the +same repository, or if the tests cross the components in the repository. + +Each type of test should be in its own subdirectory of tests, to allow for easy +separation. E.g. tests/unit, tests/functional, tests/integration. + +Source Code Location +~~~~~~~~~~~~~~~~~~~~ +A standard structure for the source code places the source for each module in +a module-named directory under either /src/bin or /src/lib, for executable +modules and shared library modules respectively. Since each module needs its +own setup.py and setup.cfg (python) that lives parallel to the top-level +module (i.e. the package), the directory for the module will contain another +directory named the same. + +For example, Project foo, with module foo_service would have a source structure +that is /src/bin/foo_service/foo_service, wherein the __init__.py for the +package resides. + +Sample Project Structure (Python) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Project ``foo``, supporting multiple executable modules ``foo_service``, +``foo_cli``, and a shared module ``foo_client`` :: + + {root of foo} + |- /docs + |- /etc + | |- /foo + | |- {sample files} + |- /charts + | |- /foo + | |- /bar + |- /images + | |- /foo + | | |- Dockerfile + | |- /bar + | |- Dockerfile + |- /tools + | |- {scripts/utilities supporting build and test} + |- /src + | |- /bin + | | |- /foo_service + | | | |- /foo_service + | | | | |- __init__.py + | | | | |- {source directories and files} + | | | |- /tests + | | | | |- unit + | | | | |- functional + | | | |- setup.py + | | | |- setup.cfg + | | | |- requirements.txt (and related files) + | | | |- tox.ini + | | |- /foo_cli + | | |- /foo_cli + | | | |- __init__.py + | | | |- {source directories and files} + | | |- /tests + | | | |- unit + | | | |- functional + | | |- setup.py + | | |- setup.cfg + | | |- requirements.txt (and related files) + | | |- tox.ini + | |- /lib + | |- /foo_client + | |- /foo_client + | | |- __init__.py + | | |- {source directories and files} + | |- /tests + | | |- unit + | | |- functional + | |- setup.py + | |- setup.cfg + | |- requirements.txt (and related files) + | |- tox.ini + |- Makefile + |- README (suitable for github consumption) + |- tox.ini (primarily for the build of repository-level docs) + +Note that this is a sample structure, and that target languages may preclude +the location of some items (e.g. tests). For those components with language +or ecosystem standards contrary to this structure, ecosystem convention should +prevail. + + +.. _Docker: https://www.docker.com/ +.. _helm-toolkit: https://github.com/openstack/openstack-helm/tree/master/helm-toolkit +.. _Openstack-Helm: https://wiki.openstack.org/wiki/Openstack-helm diff --git a/docs/source/conf.py b/docs/source/conf.py new file mode 100644 index 00000000..334d5742 --- /dev/null +++ b/docs/source/conf.py @@ -0,0 +1,160 @@ +# -*- coding: utf-8 -*- +# +# shipyard documentation build configuration file, created by +# sphinx-quickstart on Sat Sep 16 03:40:50 2017. +# +# This file is execfile()d with the current directory set to its +# containing dir. +# +# Note that not all possible configuration values are present in this +# autogenerated file. +# +# All configuration values have a default; values that are commented out +# serve to show the default. + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) +import sphinx_rtd_theme + + +# -- General configuration ------------------------------------------------ + +# If your documentation needs a minimal Sphinx version, state it here. +# +# needs_sphinx = '1.0' + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.todo', + 'sphinx.ext.viewcode', +] + +# Add any paths that contain templates here, relative to this directory. +# templates_path = [] + +# The suffix(es) of source filenames. +# You can specify multiple suffix as a list of string: +# +# source_suffix = ['.rst', '.md'] +source_suffix = '.rst' + +# The master toctree document. +master_doc = 'index' + +# General information about the project. +project = u'UCP Integration' +copyright = u'2017 AT&T Intellectual Property.' +author = u'Undercloud Authors' + +# The version info for the project you're documenting, acts as replacement for +# |version| and |release|, also used in various other places throughout the +# built documents. +# +# The short X.Y version. +version = u'0.1.0' +# The full version, including alpha/beta/rc tags. +release = u'0.1.0' + +# The language for content autogenerated by Sphinx. Refer to documentation +# for a list of supported languages. +# +# This is also used if you do content translation via gettext catalogs. +# Usually you set "language" from the command line for these cases. +language = None + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This patterns also effect to html_static_path and html_extra_path +exclude_patterns = [] + +# The name of the Pygments (syntax highlighting) style to use. +pygments_style = 'sphinx' + +# If true, `todo` and `todoList` produce output, else they produce nothing. +todo_include_todos = False + + +# -- Options for HTML output ---------------------------------------------- + +# The theme to use for HTML and HTML Help pages. See the documentation for +# a list of builtin themes. +# +html_theme = "sphinx_rtd_theme" +html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] + +# Theme options are theme-specific and customize the look and feel of a theme +# further. For a list of options available for each theme, see the +# documentation. +# +# html_theme_options = {} + +# Add any paths that contain custom static files (such as style sheets) here, +# relative to this directory. They are copied after the builtin static files, +# so a file named "default.css" will overwrite the builtin "default.css". +html_static_path = [] + + +# -- Options for HTMLHelp output ------------------------------------------ + +# Output file base name for HTML help builder. +htmlhelp_basename = 'ucpintdoc' + + +# -- Options for LaTeX output --------------------------------------------- + +latex_elements = { + # The paper size ('letterpaper' or 'a4paper'). + # + # 'papersize': 'letterpaper', + + # The font size ('10pt', '11pt' or '12pt'). + # + # 'pointsize': '10pt', + + # Additional stuff for the LaTeX preamble. + # + # 'preamble': '', + + # Latex figure (float) alignment + # + # 'figure_align': 'htbp', +} + +# Grouping the document tree into LaTeX files. List of tuples +# (source start file, target name, title, +# author, documentclass [howto, manual, or own class]). +latex_documents = [ + (master_doc, 'ucpint.tex', u'UCP Integration Documentation', + u'Undercloud Authors', 'manual'), +] + + +# -- Options for manual page output --------------------------------------- + +# One entry per manual page. List of tuples +# (source start file, name, description, authors, manual section). +man_pages = [ + (master_doc, 'UCPIntegration', u'UCP Integration Documentation', + [author], 1) +] + + +# -- Options for Texinfo output ------------------------------------------- + +# Grouping the document tree into Texinfo files. List of tuples +# (source start file, target name, title, author, +# dir menu entry, description, category) +texinfo_documents = [ + (master_doc, 'UCP Integration', u'UCP Integration Documentation', + author, 'UCP Integration', + 'Undercloud Platform Integration documentation', + 'Miscellaneous'), +] diff --git a/docs/source/conventions.rst b/docs/source/conventions.rst new file mode 100644 index 00000000..146617e5 --- /dev/null +++ b/docs/source/conventions.rst @@ -0,0 +1,50 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _conventions: + +Undercloud Platform Conventions +=============================== +Undercloud Platform components conform to a minimal set of of conventions to +provide for reasonable levels of consistency. + +Language +-------- +While these documents are not an IETF RFC, `RFC 2119`_ provides for useful +language definitions. In this spirit: + +- 'must', 'shall', 'will', and 'required' language indicates inflexible rules. +- 'should' and 'recommended' language is expected to be followed but reasonable + exceptions may exist. +- 'may' and 'can' lanugage is intended to be optional, but will provide a + recommended approach if used. + +Conventions and Standards +------------------------- + +.. toctree:: + :maxdepth: 2 + + alarming-conditions + api-conventions + rbac-conventions + client-software + code-conventions + documentation-conventions + service-logging-conventions + security-conventions + +.. _RFC 2119: https://www.ietf.org/rfc/rfc2119.txt diff --git a/docs/source/documentation-conventions.rst b/docs/source/documentation-conventions.rst new file mode 100644 index 00000000..8ceb3af7 --- /dev/null +++ b/docs/source/documentation-conventions.rst @@ -0,0 +1,85 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _documentation-conventions: + +Documentation +============= +Each UCP component will maintain documentation addressing two audiences: + + #. Consumer documentation + #. Developer documentation + +Consumer Documentation +---------------------- +Consumer documentation is that which is intended to be referenced by users of +the component. This includes information about each of the following: + +- Introduction - the purpose and charter of the software +- Features - capabilies the software has +- Usage - interaction with the software - e.g. API and CLI documentation +- Setup/Installation - how an end user would set up and run the software + including system requirements +- Support - where and how a user engages support or makes change requests for + the software + +Developer Documentation +----------------------- +Developer documentation is used by developers of the software, and addresses +the following topics: + +- Archiecture and Design - features and structure of the software +- Inline, Code, Method - documentaiton specific to the fuctions and procedures + in the code +- Development Environment - explaining how a developer would need to configure + a working environment for the software +- Contribution - how a developer can contribute to the software + +Format +------ +There are multiple means by which consumers and developers will read the +documentation for UCP components. The two common places for UCP components are +`Github`_ in the form of README and code-based documentation, and +`Readthedocs`_ for more complete/formatted documentation. + +Documentation that is expected to be read in Github must exist and may use +either `reStructuredText`_ or `Markdown`_. This generally would be limited to +the README file at the root of the project and/or a documentation directory. +The README should direct users to the published documentation location. + +Documentation intended for Readthedocs will use reStructuredText, and should +provide a `Sphinx`_ build of the documentation. + +Finding Treasuremap +------------------- +`Treasuremap`_ is a project that serves as a starting point for the larger +Containerized Cloud Platform, and provides context for the Undercloud Platform +component projects. + +Undercloud component projects should include the following at the top of the +main/index page of their `Readthedocs`_ documentation: + +.. tip:: + + {{component name}} is part of the containerized Local Control Plane (cLCP). + More details may be found by using the `Treasuremap`_ + +.. _reStructuredText: http://www.sphinx-doc.org/en/stable/rest.html +.. _Markdown: https://daringfireball.net/projects/markdown/syntax +.. _Readthedocs: https://readthedocs.org/ +.. _Github: https://github.com +.. _Sphinx: http://www.sphinx-doc.org/en/stable/index.html +.. _Treasuremap: https://github.com/att-comdev/treasuremap \ No newline at end of file diff --git a/docs/source/docutils.conf b/docs/source/docutils.conf new file mode 100644 index 00000000..b49cd480 --- /dev/null +++ b/docs/source/docutils.conf @@ -0,0 +1,2 @@ +[general] +smart_quotes=no \ No newline at end of file diff --git a/docs/source/index.rst b/docs/source/index.rst new file mode 100644 index 00000000..d85c5540 --- /dev/null +++ b/docs/source/index.rst @@ -0,0 +1,58 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. tip:: + + The Undercloud Platform is part of the containerized Local Control Plane + (cLCP). More details may be found by using the `Treasuremap`_ + +Undercloud Platform Integration +=============================== + +The Undercloud Platform (UCP) is a collection of components that coordinate to +form a means of configuring and deploying and maintaining a `Kubernetes`_ +environment using a declarative set of `yaml`_ documents. + +Approach +-------- +As the UCP revolves around the setup and use of Kubernetes and `Helm`_, +practices take cues from these projects. Since the sibling work of UCP is the +`Openstack Helm`_ project (now an `Openstack`_ projet) cues are also +taken from the Openstack approach. + +Building this Documentation +--------------------------- + +Use of ``sphinx-build -b html docs/source docs/build`` will build a html +version of this documentation that can be viewed using a browser at +docs/build/index.html on the local filesystem. + +Conventions and Standards +------------------------- + +.. toctree:: + :maxdepth: 3 + + conventions + ucp-basic-deployment + + +.. _Helm: https://helm.sh/ +.. _Kubernetes: https://kubernetes.io/ +.. _Openstack: https://www.openstack.org/ +.. _Openstack Helm: https://github.com/openstack/openstack-helm +.. _Treasuremap: https://github.com/att-comdev/treasuremap +.. _yaml: http://yaml.org/ diff --git a/docs/source/rbac-conventions.rst b/docs/source/rbac-conventions.rst new file mode 100644 index 00000000..665aa732 --- /dev/null +++ b/docs/source/rbac-conventions.rst @@ -0,0 +1,21 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _rbac-conventions: + +RBAC Conventions +================ +Under Development diff --git a/docs/source/security-conventions.rst b/docs/source/security-conventions.rst new file mode 100644 index 00000000..00398790 --- /dev/null +++ b/docs/source/security-conventions.rst @@ -0,0 +1,21 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _security-conventions: + +UCP Security Conventions +======================== +Under Development diff --git a/docs/source/service-logging-conventions.rst b/docs/source/service-logging-conventions.rst new file mode 100644 index 00000000..4f0c9f26 --- /dev/null +++ b/docs/source/service-logging-conventions.rst @@ -0,0 +1,65 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _service-logging-conventions: + +Service Logging Conventions +=========================== +UCP services must provide logging, should conform to a standard logging format, +and may utilize shared code to do so. + +Standard Logging Format +----------------------- +The following is the intended format to be used when logging from UCP services. +When logging from those parts that are no services, a close reasonable +approximation is desired. + +:: + + Timestamp Level RequestID ExternalContextID ModuleName(Line) Function - Message + +Where: + +- Timestamp is like ``2006-02-08 22:20:02,165``, or the standard ouptut from + ``%(asctime)s`` +- Level is 'DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL', padded to 8 + characters, left aligned. +- RequestID is the UUID assigned to the request in canonical 8-4-4-4-12 format. +- ExternalContextID is the UUID assigned from the external source (or generated + for the same purpose), in 8-4-4-4-12 format. +- ModuleName is the name of the module or class from which the logging + originates. +- Line is the line number of the logging statement +- Function is the name of the function or method from which the logging + originates +- Message is the text of the message to be logged. + +Example Python Logging Format +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + %(asctime)s %(levelname)-8s %(req_id)s %(external_ctx)s %(user)s %(module)s(%(lineno)d) %(funcName)s - %(message)s' + +See `Python Logging`_ for explanation of format. + +Loggers in Code +--------------- +Components should prefer loggers that are at the module or class level, +allowing for finer grained logging control than a global logger. + + +.. _Python Logging: https://docs.python.org/3/library/logging.html \ No newline at end of file diff --git a/docs/source/ucp-basic-deployment.rst b/docs/source/ucp-basic-deployment.rst new file mode 100644 index 00000000..2a93044c --- /dev/null +++ b/docs/source/ucp-basic-deployment.rst @@ -0,0 +1,175 @@ +.. + Copyright 2017 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _ucp-basic-deployment: + +Deployment Guide +---------------- + +.. note:: + The Undercloud Platform is still under active development and this + guide will evolve along the way + +The current deployment makes use of the `ucp-integration`_ repository to set up +the underlaying Kubernetes infrastructure, Ceph and UCP components. This +approach sets up an 'All-In-One' UCP environment that allows developers to +bring up the UCP components on a single Ubuntu 16.04 Virtual Machine. + +.. note:: + + Note that the minimum recommended size of the VM is 4 vCPUs, 16GB of RAM with + 64GB disk space + + +Pre-Deployment Preparations +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. todo:: + Update these to use env vars as input to the UCP basic process instead of + sed. + +1. Set up `etc/hosts` on a freshly installed Ubuntu 16.04 Virtual Machine such + that the local IP address is referenced by the hostname. + + E.g. `127.0.0.1 host1` + +:: + + HOST_IFACE=$(ip route | grep "^default" | head -1 | awk '{ print $5 }') + LOCAL_IP=$(ip addr | awk "/inet/ && /${HOST_IFACE}/{sub(/\/.*$/,\"\",\$2); print \$2}") + cat << EOF | sudo tee -a /etc/hosts + ${LOCAL_IP} $(hostname) + EOF + +2. Clone the `ucp-integration` repository. +:: + + git clone https://github.com/att-comdev/ucp-integration.git + + +Deployment +~~~~~~~~~~ + +This deploys UCP in a production-like mode using the basic_ucp deployment. +Setup for specific components in a development mode may require some additional +setup prior to these steps. + +1. Switch to root user after performing the steps in the `Pre-Deployment + Preparations`_ section. + +:: + + sudo -i + cd /home/ubuntu/ucp-integration/manifests/basic_ucp/ + +2. Source the set-env.sh script + +:: + + source set-env.sh + +3. Export/override the variables that are unique to the environment. For + instance, we can do the following to update the environment variables for a + VM that is assigned an IP of `30.30.30.4` on interface `ens3` + (network `30.30.30.0/24`): + +:: + + export CEPH_CLUSTER_NET=30.30.30.0/24 + export CEPH_PUBLIC_NET=30.30.30.0/24 + export GENESIS_NODE_IP=30.30.30.4 + export NODE_NET_IFACE=ens3 + export GENESIS_NODE_NAME=$(hostname) + +.. note:: + + These may be overridden directly in the set-env.sh instead of + exporting these as a separate step. + +4. Start the UCP deployment + +:: + + ./deploy_ucp.sh + +Post Deployment +~~~~~~~~~~~~~~~ + +1. The deployment is fully automated and can take a while to complete (it can + take 30 minutes to an hour for a full deployment to complete) + +2. The environment should resemble the following after executing the required + steps: + +:: + + # sudo kubectl get pods -n ucp + NAME READY STATUS RESTARTS AGE + airflow-flower-6cdc6f9cb4-5r62v 1/1 Running 0 3h + airflow-scheduler-6d54445bf8-6ldrd 1/1 Running 0 3h + airflow-web-7bd69d857d-qlptj 1/1 Running 0 3h + airflow-worker-666696d6c5-vffpg 1/1 Running 0 3h + armada-api-84df5b7fc9-4nxp5 1/1 Running 0 4h + barbican-api-85c956c84f-p4q7h 1/1 Running 0 4h + deckhand-5468d59455-2mcqd 1/1 Running 0 4h + drydock-api-f9897cf44-csbc8 1/1 Running 0 4h + drydock-api-f9897cf44-jgv4q 1/1 Running 0 4h + etcd-5bcbbd679c-rb5rf 1/1 Running 0 4h + ingress-api-xvkzx 1/1 Running 0 4h + ingress-error-pages-5d79688f6c-9b8xc 1/1 Running 0 4h + keystone-api-6bc85c98-886mg 1/1 Running 0 4h + maas-rack-5d4b84c4d5-dt87j 1/1 Running 0 4h + maas-region-0 1/1 Running 0 4h + mariadb-0 1/1 Running 0 4h + mariadb-1 1/1 Running 0 4h + mariadb-2 1/1 Running 0 4h + memcached-5bf49657db-kq6qh 1/1 Running 0 4h + postgresql-0 1/1 Running 0 4h + rabbitmq-f68649644-pnw6p 1/1 Running 0 4h + shipyard-6f4c7765d-n2kx6 1/1 Running 0 3h + +To check that all relevant helm charts have been deployed: + +:: + + # sudo helm ls + NAME REVISION UPDATED STATUS CHART NAMESPACE + ucp-armada 1 Fri Dec 1 10:03:44 2017 DEPLOYED armada-0.1.0 ucp + ucp-barbican 1 Fri Dec 1 10:03:47 2017 DEPLOYED barbican-0.1.0 ucp + ucp-calico 1 Fri Dec 1 10:00:05 2017 DEPLOYED calico-0.1.0 kube-system + ucp-calico-etcd 1 Fri Dec 1 09:59:28 2017 DEPLOYED etcd-0.1.0 kube-system + ucp-ceph 1 Fri Dec 1 10:00:58 2017 DEPLOYED ceph-0.1.0 ceph + ucp-coredns 1 Fri Dec 1 10:00:26 2017 DEPLOYED coredns-0.1.0 kube-system + ucp-deckhand 1 Fri Dec 1 10:03:39 2017 DEPLOYED deckhand-0.1.0 ucp + ucp-drydock 1 Fri Dec 1 10:03:37 2017 DEPLOYED drydock-0.1.0 ucp + ucp-etcd-rabbitmq 1 Fri Dec 1 10:02:44 2017 DEPLOYED etcd-0.1.0 ucp + ucp-ingress 1 Fri Dec 1 10:02:45 2017 DEPLOYED ingress-0.1.0 ucp + ucp-keystone 1 Fri Dec 1 10:03:45 2017 DEPLOYED keystone-0.1.0 ucp + ucp-kubernetes-apiserver 1 Fri Dec 1 10:00:32 2017 DEPLOYED apiserver-0.1.0 kube-system + ucp-kubernetes-controller-manager 1 Fri Dec 1 10:00:33 2017 DEPLOYED controller_manager-0.1.0 kube-system + ucp-kubernetes-etcd 1 Fri Dec 1 10:00:31 2017 DEPLOYED etcd-0.1.0 kube-system + ucp-kubernetes-proxy 1 Fri Dec 1 09:58:46 2017 DEPLOYED proxy-0.1.0 kube-system + ucp-kubernetes-scheduler 1 Fri Dec 1 10:00:34 2017 DEPLOYED scheduler-0.1.0 kube-system + ucp-maas 1 Fri Dec 1 10:03:36 2017 DEPLOYED maas-0.1.0 ucp + ucp-maas-postgresql 1 Fri Dec 1 10:02:44 2017 DEPLOYED postgresql-0.1.0 ucp + ucp-rabbitmq 1 Fri Dec 1 10:02:45 2017 DEPLOYED rabbitmq-0.1.0 ucp + ucp-rbac 1 Fri Dec 1 10:00:44 2017 DEPLOYED rbac-0.1.0 kube-system + ucp-shipyard 1 Fri Dec 1 10:38:08 2017 DEPLOYED shipyard-0.1.0 ucp + ucp-ucp-ceph-config 1 Fri Dec 1 10:02:40 2017 DEPLOYED ceph-0.1.0 ucp + ucp-ucp-mariadb 1 Fri Dec 1 10:02:43 2017 DEPLOYED mariadb-0.1.0 ucp + ucp-ucp-memcached 1 Fri Dec 1 10:02:44 2017 DEPLOYED memcached-0.1.0 ucp + +.. _ucp-integration: https://github.com/att-comdev/ucp-integration \ No newline at end of file