summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorhemanthnakkina <mail2hemanth.n@gmail.com>2018-11-29 18:18:13 +0530
committerGitHub <noreply@github.com>2018-11-29 18:18:13 +0530
commit92a9e717a46055df159d9fa5c15f866425c82cc6 (patch)
treeb68b6eb5abb6861f7c94eafbbcf731d1bb0168e0
parentecaef2b54975792271fe6705be52b20a047b3407 (diff)
parentc60c0e8b53594e9f6cc493421f68d87ecdf1756b (diff)
Merge pull request #5 from purnendu15/master
Added documentation files
-rw-r--r--README.md9
-rw-r--r--doc/source/getting_started.rst74
-rw-r--r--doc/source/index.rst5
-rw-r--r--doc/source/tugboat.rst97
-rw-r--r--setup.py1
-rw-r--r--spyglass/data_extractor/custom_exceptions.py1
-rw-r--r--spyglass/data_extractor/plugins/tugboat/excel_parser.py3
7 files changed, 177 insertions, 13 deletions
diff --git a/README.md b/README.md
index b0c3c3e..4080d62 100644
--- a/README.md
+++ b/README.md
@@ -23,7 +23,10 @@ all the information that will be rendered to generate Airship site
23manifests. This optional step will help the deployment engineer to 23manifests. This optional step will help the deployment engineer to
24modify any data if required. 24modify any data if required.
25 25
26Basic Usage 26Getting Started
27----------- 27---------------
28For more detailed installation and setup information, please refer to the
29`Getting Started`_ guide.
28 30
29TODO 31
32.. _`Getting Started`: ./doc/source/getting_started.rst
diff --git a/doc/source/getting_started.rst b/doc/source/getting_started.rst
index 1e502f4..892e476 100644
--- a/doc/source/getting_started.rst
+++ b/doc/source/getting_started.rst
@@ -19,7 +19,7 @@ Getting Started
19=============== 19===============
20 20
21What is Spyglass? 21What is Spyglass?
22---------------- 22-----------------
23 23
24Spyglass is a data extraction tool which can interface with 24Spyglass is a data extraction tool which can interface with
25different input data sources to generate site manifest YAML files. 25different input data sources to generate site manifest YAML files.
@@ -28,6 +28,7 @@ for a site deployment. These site manifest YAML files generated
28by spyglass will be saved in a Git repository, from where Pegleg 28by spyglass will be saved in a Git repository, from where Pegleg
29can access and aggregate them. This aggregated file can then be 29can access and aggregate them. This aggregated file can then be
30fed to Shipyard for site deployment / updates. 30fed to Shipyard for site deployment / updates.
31Reference: https://review.openstack.org/#/c/605227
31 32
32Architecture 33Architecture
33------------ 34------------
@@ -62,6 +63,65 @@ Architecture
62 63
63-- 64--
64 65
66Supported Features
67------------------
681. Tugboat Plugin: Supports extracting site data from Excel files and
69 then generate site manifests for sitetype:airship-seaworthy.
70 Find more documentation for Tugboat, see :ref:`tugboatinfo`.
71
722. Remote Data Source Plugin: Supports extracting site data from a REST
73 endpoint.
74
753. YAML Editor for Intermediary YAML: Support runtime editing of missing
76 site parameters
77
78Future Work
79-----------
801) Schema based manifest generation instead of Jinja2 templates. It shall
81be possible to cleanly transition to this schema based generation keeping a unique
82mapping between schema and generated manifests. Currently this is managed by
83considering a mapping of j2 templates with schemas and site type.
84
85List of Generated Site Manifests:
86---------------------------------
87The spyglass uses the plugin data source to generate the following site
88manifests:
89
90- site-definition.yaml
91- profile/
92- profile/region.yaml
93- baremetal/
94- baremetal/nodes.yaml
95- networks/
96- networks/common_addresses.yaml
97- networks/control-plane-addresses.yaml
98- networks/physical/
99- networks/physical/networks.yaml
100- software/
101- software/charts/
102- software/charts/osh/
103- software/charts/osh/openstack-tenant-ceph/
104- software/charts/osh/openstack-tenant-ceph/ceph-client.yaml
105- software/charts/ucp/
106- software/charts/ucp/divingbell/
107- software/charts/ucp/divingbell/divingbell.yaml
108- software/config/
109- software/config/corridor.yaml
110- software/config/common-software-config.yaml
111- deployment/
112- deployment/deployment-strategy.yaml
113- pki/
114- pki/kubelet-node-pkicatalog.yaml
115
116Spyglass maintains corresponding J2 templates for these files
117and then those are processed with site information obtained
118from plugin data source.
119
120In some cases, the site might require additional site
121manifests containing static information independent of the
122plugin data received. In such cases one can just place the
123corresponding J2 templates in the appropriate folder.
124
65Basic Usage 125Basic Usage
66----------- 126-----------
67 127
@@ -72,7 +132,7 @@ Before using Spyglass you must:
72 132
73 .. code-block:: console 133 .. code-block:: console
74 134
75 git clone https://github.com/att-comdev/tugboat/tree/spyglass 135 git clone https://github.com/att-comdev/spyglass
76 136
772. Install the required packages in spyglass: 1372. Install the required packages in spyglass:
78 138
@@ -100,8 +160,7 @@ Options:
100 excel spec 160 excel spec
101 -idir, --intermediary_dir PATH The path where intermediary file needs to be 161 -idir, --intermediary_dir PATH The path where intermediary file needs to be
102 generated 162 generated
103 -e, --edit_intermediary / -nedit, --no_edit_intermediary 163 -e, --edit_intermediary Flag to let user edit intermediary
104 Flag to let user edit intermediary
105 -m, --generate_manifests Generate manifests from the generated 164 -m, --generate_manifests Generate manifests from the generated
106 intermediary file 165 intermediary file
107 -mdir, --manifest_dir PATH The path where manifest files needs to be 166 -mdir, --manifest_dir PATH The path where manifest files needs to be
@@ -115,6 +174,9 @@ Options:
115 20] 174 20]
116 --help Show this message and exit. 175 --help Show this message and exit.
117 176
177--------
178Examples
179--------
118 180
1191. Running Spyglass with Remote Data Source Plugin 1811. Running Spyglass with Remote Data Source Plugin
120 182
@@ -125,8 +187,8 @@ spyglass -mg --type formation -f <URL> -u <user_id> -p <password> -d <site_confi
125spyglass -mg --type tugboat -x <Excel File> -e <Excel Spec> -d <Site Config> -s <Region> --template_dir=<j2 template dir> 187spyglass -mg --type tugboat -x <Excel File> -e <Excel Spec> -d <Site Config> -s <Region> --template_dir=<j2 template dir>
126 188
127for example: 189for example:
128spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<j2 template dir> 190spyglass -mg -t tugboat -x SiteDesignSpec_v1.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<j2 template dir>
191
129Where sample 'excel_spec_upstream.yaml', 'SiteDesignSpec_v0.1.xlsx' 192Where sample 'excel_spec_upstream.yaml', 'SiteDesignSpec_v0.1.xlsx'
130'site_config.yaml' and J2 templates can be found under 'spyglass/examples' 193'site_config.yaml' and J2 templates can be found under 'spyglass/examples'
131folder 194folder
132
diff --git a/doc/source/index.rst b/doc/source/index.rst
index 4c43fa2..b0ec2b9 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -14,9 +14,9 @@
14 License for the specific language governing permissions and limitations 14 License for the specific language governing permissions and limitations
15 under the License. 15 under the License.
16 16
17===================== 17======================
18Spyglass Documentation 18Spyglass Documentation
19===================== 19======================
20 20
21Overview 21Overview
22-------- 22--------
@@ -32,3 +32,4 @@ fed to Shipyard for site deployment / updates.
32 :maxdepth: 2 32 :maxdepth: 2
33 33
34 getting_started 34 getting_started
35 tugboat
diff --git a/doc/source/tugboat.rst b/doc/source/tugboat.rst
new file mode 100644
index 0000000..3236128
--- /dev/null
+++ b/doc/source/tugboat.rst
@@ -0,0 +1,97 @@
1
2 Copyright 2018 AT&T Intellectual Property.
3 All Rights Reserved.
4
5 Licensed under the Apache License, Version 2.0 (the "License"); you may
6 not use this file except in compliance with the License. You may obtain
7 a copy of the License at
8
9 http://www.apache.org/licenses/LICENSE-2.0
10
11 Unless required by applicable law or agreed to in writing, software
12 distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
13 WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
14 License for the specific language governing permissions and limitations
15 under the License.
16
17.. _tugboatinfo:
18
19=======
20Tugboat
21=======
22
23What is Tugboat?
24----------------
25
26Tugboat is a Spyglass plugin to generate airship-seaworthy site manifest files
27from an excel based engineering spec. The plugin is configured with an Excel
28sheet and its corresponding excel specification as inputs. Spyglass uses this
29plugin to construct an intermediary yaml which is processed further using J2
30templates to generate site manifests.
31
32Excel specification
33-------------------
34Excel Spec is like an index to the Excel sheet to look for the data to be
35collected by the tool. Excel Spec Sample specifies all the details that
36need to be filled by the Deployment Engineer.
37
38Below is the definition for each key in the Excel spec
39
40::
41
42
43 ipmi_sheet_name - name of the sheet from where IPMI and host profile information is to be read
44 start_row - row number from where the IPMI and host profile information starts
45 end_row - row number from where the IPMI and host profile information ends
46 hostname_col - column number where the hostnames are to be read from
47 ipmi_address_col - column number from where the ipmi addresses are to be read
48 host_profile_col - column number from where the host profiles are to be read
49 ipmi_gateway_col - column number from where the ipmi gateways are to be read
50 private_ip_sheet - name of the sheet which has the private IP information
51 net_type_col - column number from where the network type is to be read
52 vlan_col - column number from where the network vlan is to be read
53 vlan_start_row - row number from where the vlan information starts
54 vlan_end_row - row number from where the vlan information ends
55 net_start_row - row number from where the network information starts
56 net_end_row - row number from where the network information ends
57 net_col - column number where the IP ranges for network is to be read
58 net_vlan_col - column number where the vlan information is present in the pod wise network section
59 public_ip_sheet - name of the sheet which has the public IP information
60 oam_vlan_col - column number from where the OAM vlan information is to be read from
61 oam_ip_row - row number from where the OAM network information is to be read from
62 oam_ip_col - column number from where the OAM network information is to be read from
63 oob_net_row - row number which has the OOB network subnet ranges
64 oob_net_start_col - column number from where the OOB network ranges start
65 oob_net_end_col - column number from where the OOB network ranges end
66 ingress_ip_row - row number from where the Ingress network information is to be read from
67 dns_ntp_ldap_sheet - name of the sheet which has the DNS, NTP and LDAP information
68 login_domain_row - row number which has the ldap login domain
69 ldap_col - column number which has the all ldap related information
70 global_group - row number which has the ldap group information
71 ldap_search_url_row - row number which has the ldap url
72 ntp_row - row number which has the ntp information
73 ntp_col - column number which has the ntp information
74 dns_row - row number which has the dns information
75 dns_col - column number which has the dns information
76 domain_row - row number which has the domain information
77 domain_col - column number which has the domain information
78 location_sheet - name of the sheet which has the location information
79 column - column number which has all the information
80 corridor_row - row number which has the corridor information
81 site_name_row - row number which has the site name
82 state_name_row - row number which has the state name
83 country_name_row - row number which has the country name
84 clli_name_row - row number which has CLLI information
85
86Example: Tugboat Plugin Usage
87-----------------------------
881. Required Input(Refer to 'spyglass/examples' folder to get these inputs)
89 a) Excel File: SiteDesignSpec_v0.1.xlsx
90 b) Excel Spec: excel_spec_upstream.yaml
91 c) Site Config: site_config.yaml
92 d) Template_dir: '../examples/templates'
93 c) Site name: airship-seaworthy
94
952. Spyglass CLI Command:
96 spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=<relative path to '../examples/templates'
97
diff --git a/setup.py b/setup.py
index e21ec83..2011afc 100644
--- a/setup.py
+++ b/setup.py
@@ -39,6 +39,7 @@ setup(
39 ], 39 ],
40 'data_extractor_plugins': 40 'data_extractor_plugins':
41 ['formation=spyglass.data_extractor.plugins.formation:FormationPlugin', 41 ['formation=spyglass.data_extractor.plugins.formation:FormationPlugin',
42 'tugboat=spyglass.data_extractor.plugins.tugboat.tugboat:TugboatPlugin',
42 ] 43 ]
43 }, 44 },
44 include_package_data=True, 45 include_package_data=True,
diff --git a/spyglass/data_extractor/custom_exceptions.py b/spyglass/data_extractor/custom_exceptions.py
index f43a69f..40f67d6 100644
--- a/spyglass/data_extractor/custom_exceptions.py
+++ b/spyglass/data_extractor/custom_exceptions.py
@@ -35,6 +35,7 @@ class NoSpecMatched(BaseError):
35 self.specs)) 35 self.specs))
36 sys.exit(1) 36 sys.exit(1)
37 37
38
38class MissingAttributeError(BaseError): 39class MissingAttributeError(BaseError):
39 pass 40 pass
40 41
diff --git a/spyglass/data_extractor/plugins/tugboat/excel_parser.py b/spyglass/data_extractor/plugins/tugboat/excel_parser.py
index 83e4f85..f7b78c1 100644
--- a/spyglass/data_extractor/plugins/tugboat/excel_parser.py
+++ b/spyglass/data_extractor/plugins/tugboat/excel_parser.py
@@ -19,8 +19,7 @@ import sys
19import yaml 19import yaml
20from openpyxl import load_workbook 20from openpyxl import load_workbook
21from openpyxl import Workbook 21from openpyxl import Workbook
22from spyglass.data_extractor.custom_exceptions import 22from spyglass.data_extractor.custom_exceptions import NoSpecMatched
23 NoSpecMatched, )
24# from spyglass.data_extractor.custom_exceptions 23# from spyglass.data_extractor.custom_exceptions
25 24
26LOG = logging.getLogger(__name__) 25LOG = logging.getLogger(__name__)