Berth documentation via build_sphinx.

This PS adds tooling and automation to automatically generate
Berth's documentation into feature-rich HTML pages that can
be hosted.

To run the documentation job, simply execute:

    tox -e docs

Future docs should be written using the style described at
http://www.sphinx-doc.org/en/stable/markup/code.html and
located in berth/docs/source

This PS also adds basic building blocks required by most any
Python project:

  * test-requirements.txt
  * requirements.txt
  * setup.cfg
  * setup.py
  * added more filters to .gitignore

Change-Id: I792b0244ce9909d70283730a29545304311c6dbc
This commit is contained in:
Felipe Monteiro 2017-10-05 21:52:52 +01:00
parent 09b3e93f0e
commit a821aeff00
12 changed files with 477 additions and 93 deletions

105
.gitignore vendored
View File

@ -1,2 +1,107 @@
# Misc
helm.log
berth-0.1.0.tgz
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
.hypothesis/
.testrepository/*
cover/*
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
target/
# Jupyter Notebook
.ipynb_checkpoints
# pyenv
.python-version
# celery beat schedule file
celerybeat-schedule
# SageMath parsed files
*.sage.py
# dotenv
.env
# virtualenv
.venv
venv/
ENV/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/

View File

@ -1,93 +0,0 @@
Berth is a deliberately minimalist VM runner for Kubernetes.
I'm not 100% sold on the name; before merging we could change it...
## TL;DR Installation Guide
```
# Make sure you have Helm 2.5.x and Kubernetes 1.6.x
#
# edit values.yaml; set class_name and ssh key
#
helm package berth
helm install --name=berth ./berth-0.1.0.tgz # ...
kubectl get pods -o wide
```
You should be able to SSH to your VM at the Kubernetes IP for the
container which you can retrieve with `kubectl get all -o wide`. VNC
access is available on port 5900.
```
ssh -i ./you-ssh-private-key root@ip.of.vm.pod
```
<!-- https://mostsecure.pw/ -->
### Example
[Quick installation / sample](https://asciinema.org/a/4VazbwsokL3zpnGPf27eyFIfe)
### Why this?
The requirements are very narrow right now and the existing
alternatives don't align well at present. This will likely change in
time at which point we can realign the internal implementation.
#### Minimalist requirements
* Run VMs from inside of Kubernetes
* Work with Calico
* Have VM life-cycle match that of pods
* Have VMs benefit from Kubernetes resiliency
* Allow for persistent storage
* Allow for state injection/access from a ConfigMaps
## Requirements:
* Helm 2.5.x
* Kubernetes 1.6.x
This does not need to be installed as part of the OpenStack chart
collection.
## How it works:
At a high level, it works like this:
* Create a SNAT/DNAT enabled linux bridge.
* Assign the bridge a private IP address from a small /30 subnet
(controlled with `VM_IP` and `VM_GW`)
* Plug the VM network interface into the bridge.
* Run a dnsmasq process to allocate the VM the right name-servers, and
DNS search strings extracted from the parent container. Assign the
private IP address to the VM and have it use the bridges IP as its
default gateway.
* Setup SNAT/DNAT on the parent container to do 1:1 mapping of all
ports, all protocols to the VM, except for TCP:5900 to allow for VNC
access (can be controlled with NO_VNC environment variable).
* At this point, VM essentially assumes Pod Assigned IP.
* Feed any meta-data or user-data down into the VM by leveraging these
ConfigMap mounts with the same name and turning them into an ISO
presented to the guest.
The startvm entry-point supports several environment variables:
* `IMG_SOURCE` which is an http or https URL that contains a qcow2
image. It can also be a full path to a local file baked into the
container image, e.g. "/image.qcow"
* `IMG_TARGET` the name to save the image above as in the shared
volume.
It also supports two files, which should be mounted as ConfigMaps if
using Kubernetes at `/userdata` and `/metadata` as YAML files
containing, obviously meta-data and user-data as YAML that will be fed
to the VM as a config-drive iso.
The "pet" version of the image, which is created using qemu-img -b to
base it on the source, is stored in a separate volume dedicated to the
VM itself, and named after the container hostname.
There are a few other parameters you can control as an operator:
* `VM_IP` is the IP address the VM should be allocated by DHCP. The
container will 1:1 NAT except for port 5900 for VNC access (defaults
to 192.168.254.2)
* `VM_GW` is the gateway IP address the VM should use for its default
route (defaults to 192.168.254.1)

105
README.rst Normal file
View File

@ -0,0 +1,105 @@
=====
Berth
=====
Berth is a deliberately minimalist VM runner for Kubernetes.
I'm not 100% sold on the name; before merging we could change it...
TL;DR Installation Guide
========================
.. code-block:: bash
# Make sure you have Helm 2.5.x and Kubernetes 1.6.x
#
# edit values.yaml; set class_name and ssh key
#
helm package berth
helm install --name=berth ./berth-0.1.0.tgz # ...
kubectl get pods -o wide
You should be able to SSH to your VM at the Kubernetes IP for the
container which you can retrieve with `kubectl get all -o wide`. VNC
access is available on port 5900.
.. code-block:: bash
ssh -i ./you-ssh-private-key root@ip.of.vm.pod
Example
-------
`Quick installation / sample <https://asciinema.org/a/4VazbwsokL3zpnGPf27eyFIfe>`_
Why this?
---------
The requirements are very narrow right now and the existing
alternatives don't align well at present. This will likely change in
time at which point we can realign the internal implementation.
Minimalist requirements
-----------------------
* Run VMs from inside of Kubernetes
* Work with Calico
* Have VM life-cycle match that of pods
* Have VMs benefit from Kubernetes resiliency
* Allow for persistent storage
* Allow for state injection/access from a ConfigMaps
Requirements
============
* Helm 2.5.x
* Kubernetes 1.6.x
This does not need to be installed as part of the OpenStack chart
collection.
How it works
============
At a high level, it works like this:
* Create a SNAT/DNAT enabled linux bridge.
* Assign the bridge a private IP address from a small /30 subnet
(controlled with `VM_IP` and `VM_GW`)
* Plug the VM network interface into the bridge.
* Run a dnsmasq process to allocate the VM the right name-servers, and
DNS search strings extracted from the parent container. Assign the
private IP address to the VM and have it use the bridges IP as its
default gateway.
* Setup SNAT/DNAT on the parent container to do 1:1 mapping of all
ports, all protocols to the VM, except for TCP:5900 to allow for VNC
access (can be controlled with NO_VNC environment variable).
* At this point, VM essentially assumes Pod Assigned IP.
* Feed any meta-data or user-data down into the VM by leveraging these
ConfigMap mounts with the same name and turning them into an ISO
presented to the guest.
The startvm entry-point supports several environment variables:
* `IMG_SOURCE` which is an http or https URL that contains a qcow2
image. It can also be a full path to a local file baked into the
container image, e.g. "/image.qcow"
* `IMG_TARGET` the name to save the image above as in the shared
volume.
It also supports two files, which should be mounted as ConfigMaps if
using Kubernetes at `/userdata` and `/metadata` as YAML files
containing, obviously meta-data and user-data as YAML that will be fed
to the VM as a config-drive iso.
The "pet" version of the image, which is created using qemu-img -b to
base it on the source, is stored in a separate volume dedicated to the
VM itself, and named after the container hostname.
There are a few other parameters you can control as an operator:
* `VM_IP` is the IP address the VM should be allocated by DHCP. The
container will 1:1 NAT except for port 5900 for VNC access (defaults
to 192.168.254.2)
* `VM_GW` is the gateway IP address the VM should use for its default
route (defaults to 192.168.254.1)

View File

155
docs/source/conf.py Normal file
View File

@ -0,0 +1,155 @@
# -*- coding: utf-8 -*-
#
# berth documentation build configuration file, created by
# sphinx-quickstart on Sat Sep 16 03:40:50 2017.
#
# This file is execfile()d with the current directory set to its
# containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc']
# Add any paths that contain templates here, relative to this directory.
# templates_path = []
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Berth'
copyright = u'2017, Berth Authors'
author = u'Berth Authors'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = u'0.1.0'
# The full version, including alpha/beta/rc tags.
release = u'0.1.0'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = []
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# If true, `todo` and `todoList` produce output, else they produce nothing.
todo_include_todos = False
# -- Options for HTML output ----------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
import sphinx_rtd_theme
html_theme = "sphinx_rtd_theme"
html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# -- Options for HTMLHelp output ------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'berthdoc'
# -- Options for LaTeX output ---------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'berth.tex', u'Berth Documentation',
u'Berth Authors', 'manual'),
]
# -- Options for manual page output ---------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'Berth', u'Berth Documentation',
[author], 1)
]
# -- Options for Texinfo output -------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'Berth', u'Berth Documentation',
author, 'Berth', 'A deliberately minimalist VM runner for Kubernetes.',
'Miscellaneous'),
]

26
docs/source/index.rst Normal file
View File

@ -0,0 +1,26 @@
..
Copyright 2017 AT&T Intellectual Property.
All Rights Reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
=================================
Welcome to Berth's documentation!
=================================
Berth is a deliberately minimalist VM runner for Kubernetes.
.. toctree::
:maxdepth: 2
readme

1
docs/source/readme.rst Normal file
View File

@ -0,0 +1 @@
.. include:: ../../README.rst

3
requirements.txt Normal file
View File

@ -0,0 +1,3 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.

30
setup.cfg Normal file
View File

@ -0,0 +1,30 @@
[metadata]
name = Berth
summary = A deliberately minimalist VM runner for Kubernetes.
description-file = README.rst
author = Berth Authors
home-page = http://berth.readthedocs.io/en/latest/
classifier =
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.5
[files]
packages =
berth
[build_sphinx]
source-dir = docs/source
build-dir = docs/build
all_files = 1
warning-is-error = 1
[upload_sphinx]
upload-dir = doc/build/html

27
setup.py Normal file
View File

@ -0,0 +1,27 @@
# Copyright 2017 AT&T Intellectual Property. All other rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import setuptools
# In python < 2.7.4, a lazy loading of package `pbr` will break
# setuptools if some other modules registered functions in `atexit`.
# solution from: http://bugs.python.org/issue15881#msg170215
try:
import multiprocessing # noqa
except ImportError:
pass
setuptools.setup(
setup_requires=['pbr>=2.0.0'],
pbr=True)

6
test-requirements.txt Normal file
View File

@ -0,0 +1,6 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
sphinx>=1.6.2
sphinx_rtd_theme==0.2.4

19
tox.ini Normal file
View File

@ -0,0 +1,19 @@
[tox]
envlist = py35
[testenv]
usedevelop = True
whitelist_externals = rm
setenv = VIRTUAL_ENV={envdir}
LANGUAGE=en_US
LC_ALL=en_US.utf-8
deps = -r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
commands =
find . -type f -name "*.pyc" -delete
rm -Rf .testrepository/times.dbm
[testenv:docs]
commands =
rm -rf doc/build
python setup.py build_sphinx {posargs}