airship
/
spyglass
Code Issues Proposed changes

Spyglass Docs Update 

Changed paths for all examples, removed references to Tugboat
and Formation

Change-Id: Ibeb80a1c239169bd4f4ecb124e86671891bef99a
This commit is contained in:
Ryan Schroder 2019-11-21 10:47:55 -06:00
parent 261b064b65
commit bddbcc9acb
4 changed files with 75 additions and 68 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
README.rst
View File

@ -6,22 +6,19 @@ Spyglass is the data extractor tool which can interface with
different input data sources to generate site manifest YAML files.
The data sources will provide all the configuration data needed
for a site deployment. These site manifest YAML files generated
by spyglass will be saved in a Git repository, from where Pegleg
by Spyglass will be saved in a Git repository, from where Pegleg
can access and aggregate them. This aggregated file can then be
fed to shipyard for site deployment / updates.
fed to Shipyard for site deployment/updates.
Spyglass follows plugin model to support multiple input data sources.
Current supported plugins are formation-plugin and Tugboat. Formation
is a rest API based service which will be the source of information
related to hardware, networking, site data. Formation plugin will
interact with Formation API to gather necessary configuration.
Similarly Tugboat accepts engineering spec which is in the form of
spreadsheet and an index file to read spreadsheet as inputs and
generates the site level manifests.
As an optional step it can generate an intermediary yaml which contain
all the information that will be rendered to generate Airship site
manifests. This optional step will help the deployment engineer to
modify any data if required.
Spyglass follows a plugin model to support multiple input data sources.
The currently supported plugin is the XLS plugin (`spyglass-plugin-xls`_).
The XLS plugin accepts engineering spec in the form of a spreadsheet and
an index file to read the spreadsheet as inputs and generates the site
level manifests. As an optional step, it can generate an intermediary
YAML which contains all the information that will be rendered to generate
Airship site manifests. This optional step will help the deployment
engineer modify any data if required.
Getting Started
---------------
@ -29,4 +26,5 @@ For more detailed installation and setup information, please refer to the
`Getting Started`_ guide.
.. _spyglass-plugin-xls: https://opendev.org/airship/spyglass-plugin-xls
.. _`Getting Started`: ./doc/source/getting_started.rst

91
doc/source/cli.rst
View File

@ -28,19 +28,19 @@ The Spyglass CLI is used in conjunction with the script ``tools/spyglass.sh``.
CLI Options
===========
**-v / --verbose** (Optional). False by default.
``-v / --verbose`` (Optional). False by default.
Enable debug logging.
Excel Plugin
************
Commands available under the excel plugin package.
Commands available under the Excel plugin package.
Generate Intermediary
---------------------
Generates an intermediary file from passed excel data.
Generates an intermediary file from passed Excel data.
.. code-block:: bash
@ -52,42 +52,42 @@ Generates an intermediary file from passed excel data.
Options
^^^^^^^
**-d / --intermediary-dir** (Optional).
``-d / --intermediary-dir`` (Optional).
Path where the intermediary file will be created. Must be a writeable
directory.
**-x / --excel-file** (Required for "tugboat" plugin).
``-x / --excel-file`` (Required for XLS plugin).
Path to the engineering excel file. Multiple files can be included, provided
Path to the engineering Excel file. Multiple files can be included, provided
they follow the same specification. Must be readable file(s) in a Microsoft
Excel supported format (.xls, .xslx, etc...).
**-e / --excel-spec** (Required for "tugboat" plugin).
``-e / --excel-spec`` (Required for XLS plugin).
Path to the specification YAML that defines the content of the provided
engineering excel files. Must be a readable file in YAML format.
engineering Excel files. Must be a readable file in YAML format.
**-c / --site-configuration** (Optional).
``-c / --site-configuration`` (Optional).
Path to site specific configuration YAML. Must be a readable file.
**--intermediary-schema** (Optional).
``--intermediary-schema`` (Optional).
Path to the intermediary schema to be used for validation.
**--no-validation** (Optional).
``--no-validation`` (Optional).
Skips validation on generated intermediary data.
**-s / --site-name** (Optional).
``-s / --site-name`` (Optional).
Name of the site for which the intermediary is generated.
Generate Manifests
------------------
Generates manifests from intermediary file created from passed excel data.
Generates manifests from intermediary file created from passed Excel data.
Intermediary data is always generated, but will not be saved unless specified.
.. code-block:: bash
@ -99,48 +99,48 @@ Intermediary data is always generated, but will not be saved unless specified.
Options
^^^^^^^
**-i / --generate-intermediary** (Optional). False by default.
``-i / --generate-intermediary`` (Optional). False by default.
Saves the intermediary file used to make the manifests created by the command.
**-d / --intermediary-dir** (Optional).
``-d / --intermediary-dir`` (Optional).
Path where the intermediary file will be created. Must be a writeable
directory.
**-x / --excel-file** (Required for "tugboat" plugin).
``-x / --excel-file`` (Required for XLS plugin).
Path to the engineering excel file. Multiple files can be included, provided
Path to the engineering Excel file. Multiple files can be included, provided
they follow the same specification. Must be readable file(s) in a Microsoft
Excel supported format (.xls, .xslx, etc...).
**-e / --excel-spec** (Required for "tugboat" plugin).
``-e / --excel-spec`` (Required for XLS plugin).
Path to the specification YAML that defines the content of the provided
engineering excel files. Must be a readable file in YAML format.
engineering Excel files. Must be a readable file in YAML format.
**-c / --site-configuration** (Optional).
``-c / --site-configuration`` (Optional).
Path to site specific configuration YAML. Must be a readable file.
**--intermediary-schema** (Optional).
``--intermediary-schema`` (Optional).
Path to the intermediary schema to be used for validation.
**--no-validation** (Optional).
``--no-validation`` (Optional).
Skips validation on generated intermediary data.
**-s / --site-name** (Optional).
``-s / --site-name`` (Optional).
Name of the site for which the intermediary is generated.
**-t / --template-dir** (Required).
``-t / --template-dir`` (Required).
Path to the Jinja2 template files that will be used to generate manifest files.
Must be a readable directory with Jinja2 files using the .j2 extension.
**-m / --manifest-dir** (Optional).
``-m / --manifest-dir`` (Optional).
Path where generated manifest files should be written. Must be a writeable
directory.
@ -161,7 +161,7 @@ skip intermediary generation if it has already been completed.
Arguments
^^^^^^^^^
**INTERMEDIARY_FILE** (Required).
``INTERMEDIARY_FILE`` (Required).
Path to an existing intermediary YAML file that can be used to generate
manifests.
@ -169,17 +169,17 @@ manifests.
Options
^^^^^^^
**-t / --template-dir** (Required).
``-t / --template-dir`` (Required).
Path to the Jinja2 template files that will be used to generate manifest files.
Must be a readable directory with Jinja2 files using the .j2 extension.
**-m / --manifest-dir** (Optional).
``-m / --manifest-dir`` (Optional).
Path where generated manifest files should be written. Must be a writeable
directory.
**--force** (Optional).
``--force`` (Optional).
Forces manifests to be written, regardless of undefined data.
@ -195,11 +195,11 @@ Validates pegleg documents against their schema.
Options
^^^^^^^
**-d / --document-path**
``-d / --document-path``
Path to the document(s) to validate.
**-p / --schema-path**
``-p / --schema-path``
Path to a schema or directory of schema files used to validate documents in
document path.
@ -220,27 +220,33 @@ Generating intermediary and manifests
.. code-block:: bash
spyglass excel documents -i -x SiteDesignSpec_v0.1.xlsx \
-e excel_spec_upstream.yaml -c site_config.yaml \
-s airship-seaworthy -t <j2 template dir>
spyglass excel documents -i \
-x ../spyglass-plugin-xls/spyglass_plugin_xls/examples/SiteDesignSpec_v0.1.xlsx \
-e ../spyglass-plugin-xls/spyglass_plugin_xls/examples/excel_spec.yaml \
-c spyglass/examples/site_config.yaml \
-s airship-seaworthy -t spyglass/examples/templates/
Generating intermediary without manifests
-----------------------------------------
.. code-block:: bash
spyglass excel intermediary -x SiteDesignSpec_v0.1.xlsx \
-e excel_spec_upstream.yaml -c site_config.yaml \
-s airship-seaworthy
spyglass excel intermediary \
-x ../spyglass-plugin-xls/spyglass_plugin_xls/examples/SiteDesignSpec_v0.1.xlsx \
-e ../spyglass-plugin-xls/spyglass_plugin_xls/examples/excel_spec.yaml \
-c spyglass/examples/site_config.yaml \
-s airship-seaworthy
Generating manifests without intermediary
-----------------------------------------
.. code-block:: bash
spyglass excel documents -x SiteDesignSpec_v0.1.xlsx \
-e excel_spec_upstream.yaml -c site_config.yaml \
-s airship-seaworthy -t <j2 template dir>
spyglass excel documents \
-x ../spyglass-plugin-xls/spyglass_plugin_xls/examples/SiteDesignSpec_v0.1.xlsx \
-e ../spyglass-plugin-xls/spyglass_plugin_xls/examples/excel_spec.yaml \
-c spyglass/examples/site_config.yaml \
-s airship-seaworthy -t spyglass/examples/templates/
Generating manifests using intermediary
***************************************
@ -249,8 +255,9 @@ Generating manifests using intermediary
spyglass mi <intermediary.yaml> -t <j2 template dir>
Where sample 'excel_spec_upstream.yaml', 'SiteDesignSpec_v0.1.xlsx'
'site_config.yaml' and J2 templates can be found under 'spyglass/examples'
Where sample `excel_spec.yaml` and `SiteDesignSpec_v0.1.xlsx` can be found in
spyglass-plugin-xls in the `spyglass_plugin_xls/examples` folder. The Jinja2
templates and `site_config.yaml` can be found in the `spyglass/examples`
folder.
Validate Documents

18
doc/source/developer_quickstart.rst
View File

@ -34,36 +34,36 @@ airship-spyglass directory.
pip3 install pipenv
2. Set up an environment with Pipenv
3. Set up an environment with Pipenv
.. code-block:: console
pipenv install
3. Enter the Pipenv environment.
4. Enter the Pipenv environment.
.. code-block:: console
pipenv shell
4. Install spyglass in the tox environment.
5. Install spyglass in the tox environment.
.. code-block:: console
pip3 install .
5. Run spyglass on the example files to generate an intermediate document.
6. Run spyglass on the example files to generate an intermediate document.
.. code-block:: console
mkdir intermediate
spyglass excel documents -s airship-seaworthy -d intermediate -i \
--excel-spec spyglass/examples/excel_spec.yaml \
--excel-file spyglass/examples/SiteDesignSpec_v0.1.xlsx \
--site-configuration spyglass/examples/site_config.yaml \
--template-dir spyglass/examples/templates/
--excel-spec ../spyglass-plugin-xls/spyglass_plugin_xls/examples/excel_spec.yaml \
--excel-file ../spyglass-plugin-xls/spyglass_plugin_xls/examples/SiteDesignSpec_v0.1.xlsx \
--site-configuration spyglass/examples/site_config.yaml \
--template-dir spyglass/examples/templates/
6. Run spyglass on the intermediate document to generate manifests.
7. Run spyglass on the intermediate document to generate manifests.
.. code-block:: console

8
doc/source/getting_started.rst
View File

@ -25,10 +25,10 @@ Spyglass is a data extraction tool which can interface with
different input data sources to generate site manifest YAML files.
The data sources will provide all the configuration data needed
for a site deployment. These site manifest YAML files generated
by spyglass will be saved in a Git repository, from where Pegleg
by Spyglass will be saved in a Git repository, from where Pegleg
can access and aggregate them. This aggregated file can then be
fed to Shipyard for site deployment / updates.
Reference: https://airshipit.readthedocs.io/projects/specs/en/latest/specs/approved/data_config_generator.html
Reference: `airship-specs`_
Architecture
------------
@ -39,7 +39,7 @@ Architecture
| | | +-------+ |
| | +------>| |Generic| |
+-----------+ | | | |Object | |
|Tugboat(Xl)| I | | | +-------+ |
|Excel | I | | | +-------+ |
|Plugin | N | | | | |
+-----------+ T | | | | |
| E | | | +------+ |
@ -131,3 +131,5 @@ Before using Spyglass you must:
.. code-block:: console
pip3 install pipenv && pipenv install
.. _airship-specs: https://airshipit.readthedocs.io/projects/specs/en/latest/specs/1.x/approved/data_config_generator.html