Promenade documentation via build_sphinx.

This PS adds tooling and automation to automatically generate
Promenade's documentation into feature-rich HTML pages that can
be hosted.

To run the documentation job, simply execute:

    tox -e docs

Change-Id: I5a925c82544c14c34c1796da5804cf12b45ae575
This commit is contained in:
Felipe Monteiro 2017-10-05 19:51:33 +01:00
parent 4712f60eb4
commit b3de5e990a
10 changed files with 485 additions and 244 deletions

6
.gitignore vendored
View File

@ -12,3 +12,9 @@ __pycache__
/join_image_cache/
/promenade.egg-info
/tmp
.tox/
ChangeLog
# Sphinx documentation
docs/build/
docs/*/_static/

View File

@ -1,163 +0,0 @@
# Promenade Configuration
Promenade is configured using a set Kubernetes-like YAML documents. Many of
these documents can be automatically derived from a few core configuration
documents or generated automatically (e.g. certificates). All of these
documents can be specified in detail allowing for fine-grained control over
cluster deployment.
Generally, these documents have the following form:
```yaml
---
apiVersion: promenade/v1
kind: Kind
metadata:
compliant: metadata
spec:
detailed: data
```
`apiVersion` identifies the document as Promenade configuration. Currently
only `promenade/v1` is supported.
`kind` describe the detailed type of document. Valid kinds are:
- `Certificate` - An x509 certificate.
- `CertificateAuthority` - An x509 certificate authority certificate.
- `CertificateAuthorityKey` - The private key for a certificate authority.
- `CertificateKey` - The private key for a certificate.
- `Cluster` - Cluster configuration containing node host names, IPs & roles.
- `Etcd` - Specific configuration for an etcd cluster.
- `Masters` - Host names and IPs of master nodes.
- `Network` - Configuration details for Kubernetes networking components.
- `Node` - Specific configuration for a single host.
- `PrivateKey` - A private key, e.g. the `controller-manager`'s token signing key.
- `PublicKey` - A public key, e.g. the key for verifying service account tokens.
- `Versions` - Specifies versions of packages and images to be deployed.
`metadata` are used to select specific documents of a given `kind`. For
example, the various services must each select their specific `Certificate`s.
`metadata` are also used by Drydock to select the configuration files that are
needed for a particular node.
`spec` contains specific data for each kind of configuration document.
Additionally, documents for [Armada](https://github.com/att-comdev/armada) are
allowed and will be applied after CNI and DNS are deployed.
## Generating Configuration from Minimal Input
To construct a complete set of cluster configuration, the minimal input are
`Cluster`, `Network` and `Versions` documents. To see complete examples of
these, please see the [example](example/vagrant-input-config.yaml).
The `Cluster` configuration must contain an entry for each host for which
configuration should be generated. Each host must contain an `ip`, and
optionally `roles` and `additional_labels`. Valid `roles` are currently
`genesis` and `master`. `additional_labels` are Kubernetes labels which will
be added to the node.
Here's an example `Cluster` document:
```yaml
apiVersion: promenade/v1
kind: Cluster
metadata:
name: example
target: none
spec:
nodes:
n0:
ip: 192.168.77.10
roles:
- master
- genesis
additional_labels:
- beta.kubernetes.io/arch=amd64
```
The `Network` document must contain:
- `cluster_domain` - The domain for the cluster, e.g. `cluster.local`.
- `cluster_dns` - The IP of the cluster DNS,e .g. `10.96.0.10`.
- `kube_service_ip` - The IP of the `kubernetes` service, e.g. `10.96.0.1`.
- `pod_ip_cidr` - The CIDR from which pod IPs will be assigned, e.g. `10.97.0.0/16`.
- `service_ip_cidr` - The CIDR from which service IPs will be assigned, e.g. `10.96.0.0/16`.
- `etcd_service_ip` - The IP address of the `etcd` service, e.g. `10.96.232.136`.
- `dns_servers` - A list of upstream DNS server IPs.
Optionally, proxy settings can be specified here as well. These should all
generally be set together: `http_proxy`, `https_proxy`, `no_proxy`. `no_proxy`
must include all master IP addresses, and the `kubernetes` service name.
Here's an example `Network` document:
```yaml
apiVersion: promenade/v1
kind: Network
metadata:
cluster: example
name: example
target: all
spec:
cluster_domain: cluster.local
cluster_dns: 10.96.0.10
kube_service_ip: 10.96.0.1
pod_ip_cidr: 10.97.0.0/16
service_ip_cidr: 10.96.0.0/16
etcd_service_ip: 10.96.232.136
dns_servers:
- 8.8.8.8
- 8.8.4.4
http_proxy: http://proxy.example.com:8080
https_proxy: http://proxy.example.com:8080
no_proxy: 192.168.77.10,192.168.77.11,192.168.77.12,127.0.0.1,kubernetes,kubernetes.default.svc.cluster.local
```
The `Versions` document must define the Promenade image to be used and the
Docker package version. Currently, only the versions specified for these two
items are respected.
Here's an example `Versions` document:
```yaml
apiVersion: promenade/v1
kind: Versions
metadata:
cluster: example
name: example
target: all
spec:
images:
promenade: quay.io/attcomdev/promenade:latest
packages:
docker: docker.io=1.12.6-0ubuntu1~16.04.1
```
Given these documents (see the [example](example/vagrant-input-config.yaml)),
Promenade can derive the remaining configuration and generate certificates and
keys using the following command:
```bash
mkdir -p configs
docker run --rm -t \
-v $(pwd):/target \
quay.io/attcomdev/promenade:latest \
promenade -v generate \
-c /target/example/vagrant-input-config.yaml \
-o /target/configs
```
This will generate the following files in the `configs` directory:
- `up.sh` - A script which will bring up a node to create or join a cluster.
- `admin-bundle.yaml` - A collection of generated certificates, private keys
and core configuration.
- `complete-bundle.yaml` - A set of generated documents suitable for upload
into Drydock for future delivery to nodes to be provisioned to join the
cluster.
Additionally, a YAML file for each host described in the `Cluster` document
will be placed here. These files each contain every document needed for that
particular node to create or join the cluster.

View File

@ -1,81 +0,0 @@
# Getting Started
## Development
### Deployment using Vagrant
Deployment using Vagrant uses KVM instead of Virtualbox due to better
performance of disk and networking, which both have significant impact on the
stability of the etcd clusters.
Make sure you have [Vagrant](https://vagrantup.com) installed, then
run `./tools/full-vagrant-setup.sh`, which will do the following:
* Install Vagrant libvirt plugin and its dependencies
* Install NFS dependencies for Vagrant volume sharing
* Install [packer](https://packer.io) and build a KVM image for Ubuntu 16.04
Generate the per-host configuration, certificates and keys to be used:
```bash
mkdir configs
docker run --rm -t -v $(pwd):/target quay.io/attcomdev/promenade:latest promenade -v generate -c /target/example/vagrant-input-config.yaml -o /target/configs
```
Start the VMs:
```bash
vagrant up
```
Start the genesis node:
```bash
vagrant ssh n0 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n0.yaml'
```
Join the master nodes:
```bash
vagrant ssh n1 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n1.yaml'
vagrant ssh n2 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n2.yaml'
```
Join the worker node:
```bash
vagrant ssh n3 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n3.yaml'
```
### Building the image
```bash
docker build -t promenade:local .
```
For development, you may wish to save it and have the `up.sh` script load it:
```bash
docker save -o promenade.tar promenade:local
```
Then on a node:
```bash
PROMENADE_LOAD_IMAGE=/vagrant/promenade.tar bash /vagrant/up.sh /vagrant/path/to/node-config.yaml
```
These commands are combined in a convenience script at `tools/dev-build.sh`.
To build the image from behind a proxy, you can:
```bash
export http_proxy=...
export no_proxy=...
docker build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$http_proxy --build-arg no_proxy=$no_proxy -t promenade:local .
```
## Using Promenade Behind a Proxy
To use Promenade from behind a proxy, use the proxy settings described in the
[configuration docs](configuration.md).

160
docs/source/conf.py Normal file
View File

@ -0,0 +1,160 @@
# -*- coding: utf-8 -*-
#
# promenade documentation build configuration file, created by
# sphinx-quickstart on Sat Sep 16 03:40:50 2017.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.todo',
'sphinx.ext.viewcode'
]
# Add any paths that contain templates here, relative to this directory.
# templates_path = []
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Promenade'
copyright = u'2017, Promenade Authors'
author = u'Promenade Authors'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'0.1.0'
# The full version, including alpha/beta/rc tags.
release = u'0.1.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = []
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
import sphinx_rtd_theme
html_theme = "sphinx_rtd_theme"
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# -- Options for HTMLHelp output ------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'promenadedoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'promenade.tex', u'Promenade Documentation',
u'Promenade Authors', 'manual'),
]
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'Promenade', u'Promenade Documentation',
[author], 1)
]
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'Promenade', u'Promenade Documentation',
author, 'Promenade',
'Tool for bootstrapping a resilient Kubernetes cluster and managing its life-cycle.',
'Miscellaneous'),
]

View File

@ -0,0 +1,165 @@
Promenade Configuration
=======================
Promenade is configured using a set Kubernetes-like YAML documents. Many of
these documents can be automatically derived from a few core configuration
documents or generated automatically (e.g. certificates). All of these
documents can be specified in detail allowing for fine-grained control over
cluster deployment.
Generally, these documents have the following form:
.. code-block:: yaml
---
apiVersion: promenade/v1
kind: Kind
metadata:
compliant: metadata
spec:
detailed: data
``apiVersion`` identifies the document as Promenade configuration. Currently
only ``promenade/v1`` is supported.
``kind`` describe the detailed type of document. Valid kinds are:
- ``Certificate`` - An x509 certificate.
- ``CertificateAuthority`` - An x509 certificate authority certificate.
- ``CertificateAuthorityKey`` - The private key for a certificate authority.
- ``CertificateKey`` - The private key for a certificate.
- ``Cluster`` - Cluster configuration containing node host names, IPs & roles.
- ``Etcd`` - Specific configuration for an etcd cluster.
- ``Masters`` - Host names and IPs of master nodes.
- ``Network`` - Configuration details for Kubernetes networking components.
- ``Node`` - Specific configuration for a single host.
- ``PrivateKey`` - A private key, e.g. the ``controller-manager``'s token signing key.
- ``PublicKey`` - A public key, e.g. the key for verifying service account tokens.
- ``Versions`` - Specifies versions of packages and images to be deployed.
``metadata`` are used to select specific documents of a given ``kind``. For
example, the various services must each select their specific ``Certificate``s.
``metadata`` are also used by Drydock to select the configuration files that are
needed for a particular node.
``spec`` contains specific data for each kind of configuration document.
Additionally, documents for [Armada](https://github.com/att-comdev/armada) are
allowed and will be applied after CNI and DNS are deployed.
Generating Configuration from Minimal Input
-------------------------------------------
To construct a complete set of cluster configuration, the minimal input are
``Cluster``, ``Network`` and ``Versions`` documents. To see complete examples of
these, please see the [example](example/vagrant-input-config.yaml).
The ``Cluster`` configuration must contain an entry for each host for which
configuration should be generated. Each host must contain an ``ip``, and
optionally ``roles`` and ``additional_labels``. Valid ``roles`` are currently
``genesis`` and ``master``. ``additional_labels`` are Kubernetes labels which will
be added to the node.
Here's an example ``Cluster`` document:
.. code-block:: yaml
apiVersion: promenade/v1
kind: Cluster
metadata:
name: example
target: none
spec:
nodes:
n0:
ip: 192.168.77.10
roles:
- master
- genesis
additional_labels:
- beta.kubernetes.io/arch=amd64
The ``Network`` document must contain:
- ````cluster_domain```` - The domain for the cluster, e.g. ``cluster.local``.
- ````cluster_dns```` - The IP of the cluster DNS,e .g. ``10.96.0.10``.
- ``kube_service_ip`` - The IP of the ``kubernetes`` service, e.g. ``10.96.0.1``.
- ``pod_ip_cidr`` - The CIDR from which pod IPs will be assigned, e.g. ``10.97.0.0/16``.
- ``service_ip_cidr`` - The CIDR from which service IPs will be assigned, e.g. ``10.96.0.0/16``.
- ``etcd_service_ip`` - The IP address of the ``etcd`` service, e.g. ``10.96.232.136``.
- ``dns_servers`` - A list of upstream DNS server IPs.
Optionally, proxy settings can be specified here as well. These should all
generally be set together: ``http_proxy``, ``https_proxy``, ``no_proxy``. ``no_proxy``
must include all master IP addresses, and the ``kubernetes`` service name.
Here's an example ``Network`` document:
.. code-block:: yaml
apiVersion: promenade/v1
kind: Network
metadata:
cluster: example
name: example
target: all
spec:
cluster_domain: cluster.local
cluster_dns: 10.96.0.10
kube_service_ip: 10.96.0.1
pod_ip_cidr: 10.97.0.0/16
service_ip_cidr: 10.96.0.0/16
etcd_service_ip: 10.96.232.136
dns_servers:
- 8.8.8.8
- 8.8.4.4
http_proxy: http://proxy.example.com:8080
https_proxy: http://proxy.example.com:8080
no_proxy: 192.168.77.10,192.168.77.11,192.168.77.12,127.0.0.1,kubernetes,kubernetes.default.svc.cluster.local
The ``Versions`` document must define the Promenade image to be used and the
Docker package version. Currently, only the versions specified for these two
items are respected.
Here's an example ``Versions`` document:
.. code-block:: yaml
apiVersion: promenade/v1
kind: Versions
metadata:
cluster: example
name: example
target: all
spec:
images:
promenade: quay.io/attcomdev/promenade:latest
packages:
docker: docker.io=1.12.6-0ubuntu1~16.04.1
Given these documents (see the [example](example/vagrant-input-config.yaml)),
Promenade can derive the remaining configuration and generate certificates and
keys using the following command:
.. code-block:: bash
mkdir -p configs
docker run --rm -t \
-v $(pwd):/target \
quay.io/attcomdev/promenade:latest \
promenade -v generate \
-c /target/example/vagrant-input-config.yaml \
-o /target/configs
This will generate the following files in the ``configs`` directory:
- ``up.sh`` - A script which will bring up a node to create or join a cluster.
- ``admin-bundle.yaml`` - A collection of generated certificates, private keys
and core configuration.
- ``complete-bundle.yaml`` - A set of generated documents suitable for upload
into Drydock for future delivery to nodes to be provisioned to join the
cluster.
Additionally, a YAML file for each host described in the ``Cluster`` document
will be placed here. These files each contain every document needed for that
particular node to create or join the cluster.

View File

@ -0,0 +1,91 @@
Getting Started
===============
Development
-----------
Deployment using Vagrant
^^^^^^^^^^^^^^^^^^^^^^^^
Deployment using Vagrant uses KVM instead of Virtualbox due to better
performance of disk and networking, which both have significant impact on the
stability of the etcd clusters.
Make sure you have [Vagrant](https://vagrantup.com) installed, then
run `./tools/full-vagrant-setup.sh`, which will do the following:
* Install Vagrant libvirt plugin and its dependencies
* Install NFS dependencies for Vagrant volume sharing
* Install [packer](https://packer.io) and build a KVM image for Ubuntu 16.04
Generate the per-host configuration, certificates and keys to be used:
.. code-block:: bash
mkdir configs
docker run --rm -t -v $(pwd):/target quay.io/attcomdev/promenade:latest promenade -v generate -c /target/example/vagrant-input-config.yaml -o /target/configs
Start the VMs:
.. code-block:: bash
vagrant up
Start the genesis node:
.. code-block:: bash
vagrant ssh n0 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n0.yaml'
Join the master nodes:
.. code-block:: bash
vagrant ssh n1 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n1.yaml'
vagrant ssh n2 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n2.yaml'
Join the worker node:
.. code-block:: bash
vagrant ssh n3 -c 'sudo bash /vagrant/configs/up.sh /vagrant/configs/n3.yaml'
Building the image
^^^^^^^^^^^^^^^^^^
.. code-block:: bash
docker build -t promenade:local .
For development, you may wish to save it and have the `up.sh` script load it:
.. code-block:: bash
docker save -o promenade.tar promenade:local
Then on a node:
.. code-block:: bash
PROMENADE_LOAD_IMAGE=/vagrant/promenade.tar bash /vagrant/up.sh /vagrant/path/to/node-config.yaml
These commands are combined in a convenience script at `tools/dev-build.sh`.
To build the image from behind a proxy, you can:
.. code-block:: bash
export http_proxy=...
export no_proxy=...
docker build --build-arg http_proxy=$http_proxy --build-arg https_proxy=$http_proxy --build-arg no_proxy=$no_proxy -t promenade:local .
Using Promenade Behind a Proxy
------------------------------
To use Promenade from behind a proxy, use the proxy settings described in the
[configuration docs](configuration.md).

34
docs/source/index.rst Normal file
View File

@ -0,0 +1,34 @@
..
Copyright 2017 AT&T Intellectual Property.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
=====================================
Welcome to Promenade's documentation!
=====================================
Promenade is a tool for bootstrapping a resilient Kubernetes cluster and
managing its life-cycle.
User's Guide
============
Promenade Configuration Guide
-----------------------------
.. toctree::
:maxdepth: 2
getting-started
configuration

View File

@ -27,3 +27,12 @@ package-data =
[entry_points]
console_scripts =
promenade = promenade.cli:promenade
[build_sphinx]
source-dir = docs/source
build-dir = docs/build
all_files = 1
warning-is-error = 1
[upload_sphinx]
upload-dir = docs/build/html

3
test-requirements.txt Normal file
View File

@ -0,0 +1,3 @@
# Documentation
sphinx>=1.6.2
sphinx_rtd_theme==0.2.4

17
tox.ini Normal file
View File

@ -0,0 +1,17 @@
[tox]
envlist = py35
[testenv]
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
setenv=
PYTHONWARNING=all
commands=
pytest \
{posargs}
[testenv:docs]
whitelist_externals=rm
commands =
rm -rf docs/build
python setup.py build_sphinx {posargs}