diff --git a/README.md b/README.md index b0c3c3e..4080d62 100644 --- a/README.md +++ b/README.md @@ -23,7 +23,10 @@ all the information that will be rendered to generate Airship site manifests. This optional step will help the deployment engineer to modify any data if required. -Basic Usage ------------ +Getting Started +--------------- +For more detailed installation and setup information, please refer to the +`Getting Started`_ guide. -TODO + +.. _`Getting Started`: ./doc/source/getting_started.rst diff --git a/doc/source/getting_started.rst b/doc/source/getting_started.rst index 1e502f4..892e476 100644 --- a/doc/source/getting_started.rst +++ b/doc/source/getting_started.rst @@ -19,7 +19,7 @@ Getting Started =============== What is Spyglass? ----------------- +----------------- Spyglass is a data extraction tool which can interface with different input data sources to generate site manifest YAML files. @@ -28,6 +28,7 @@ for a site deployment. These site manifest YAML files generated by spyglass will be saved in a Git repository, from where Pegleg can access and aggregate them. This aggregated file can then be fed to Shipyard for site deployment / updates. +Reference: https://review.openstack.org/#/c/605227 Architecture ------------ @@ -62,6 +63,65 @@ Architecture -- +Supported Features +------------------ +1. Tugboat Plugin: Supports extracting site data from Excel files and + then generate site manifests for sitetype:airship-seaworthy. + Find more documentation for Tugboat, see :ref:`tugboatinfo`. + +2. Remote Data Source Plugin: Supports extracting site data from a REST + endpoint. + +3. YAML Editor for Intermediary YAML: Support runtime editing of missing + site parameters + +Future Work +----------- +1) Schema based manifest generation instead of Jinja2 templates. It shall +be possible to cleanly transition to this schema based generation keeping a unique +mapping between schema and generated manifests. Currently this is managed by +considering a mapping of j2 templates with schemas and site type. + +List of Generated Site Manifests: +--------------------------------- +The spyglass uses the plugin data source to generate the following site +manifests: + +- site-definition.yaml +- profile/ +- profile/region.yaml +- baremetal/ +- baremetal/nodes.yaml +- networks/ +- networks/common_addresses.yaml +- networks/control-plane-addresses.yaml +- networks/physical/ +- networks/physical/networks.yaml +- software/ +- software/charts/ +- software/charts/osh/ +- software/charts/osh/openstack-tenant-ceph/ +- software/charts/osh/openstack-tenant-ceph/ceph-client.yaml +- software/charts/ucp/ +- software/charts/ucp/divingbell/ +- software/charts/ucp/divingbell/divingbell.yaml +- software/config/ +- software/config/corridor.yaml +- software/config/common-software-config.yaml +- deployment/ +- deployment/deployment-strategy.yaml +- pki/ +- pki/kubelet-node-pkicatalog.yaml + +Spyglass maintains corresponding J2 templates for these files +and then those are processed with site information obtained +from plugin data source. + +In some cases, the site might require additional site +manifests containing static information independent of the +plugin data received. In such cases one can just place the +corresponding J2 templates in the appropriate folder. + Basic Usage ----------- @@ -72,7 +132,7 @@ Before using Spyglass you must: .. code-block:: console - git clone https://github.com/att-comdev/tugboat/tree/spyglass + git clone https://github.com/att-comdev/spyglass 2. Install the required packages in spyglass: @@ -100,8 +160,7 @@ Options: excel spec -idir, --intermediary_dir PATH The path where intermediary file needs to be generated - -e, --edit_intermediary / -nedit, --no_edit_intermediary - Flag to let user edit intermediary + -e, --edit_intermediary Flag to let user edit intermediary -m, --generate_manifests Generate manifests from the generated intermediary file -mdir, --manifest_dir PATH The path where manifest files needs to be @@ -115,6 +174,9 @@ Options: 20] --help Show this message and exit. +-------- +Examples +-------- 1. Running Spyglass with Remote Data Source Plugin @@ -125,8 +187,8 @@ spyglass -mg --type formation -f -u -p -d -e -d -s --template_dir= for example: -spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir= +spyglass -mg -t tugboat -x SiteDesignSpec_v1.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir= + Where sample 'excel_spec_upstream.yaml', 'SiteDesignSpec_v0.1.xlsx' 'site_config.yaml' and J2 templates can be found under 'spyglass/examples' folder - diff --git a/doc/source/index.rst b/doc/source/index.rst index 4c43fa2..b0ec2b9 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -14,9 +14,9 @@ License for the specific language governing permissions and limitations under the License. -===================== +====================== Spyglass Documentation -===================== +====================== Overview -------- @@ -32,3 +32,4 @@ fed to Shipyard for site deployment / updates. :maxdepth: 2 getting_started + tugboat diff --git a/doc/source/tugboat.rst b/doc/source/tugboat.rst new file mode 100644 index 0000000..3236128 --- /dev/null +++ b/doc/source/tugboat.rst @@ -0,0 +1,97 @@ + + Copyright 2018 AT&T Intellectual Property. + All Rights Reserved. + + Licensed under the Apache License, Version 2.0 (the "License"); you may + not use this file except in compliance with the License. You may obtain + a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + License for the specific language governing permissions and limitations + under the License. + +.. _tugboatinfo: + +======= +Tugboat +======= + +What is Tugboat? +---------------- + +Tugboat is a Spyglass plugin to generate airship-seaworthy site manifest files +from an excel based engineering spec. The plugin is configured with an Excel +sheet and its corresponding excel specification as inputs. Spyglass uses this +plugin to construct an intermediary yaml which is processed further using J2 +templates to generate site manifests. + +Excel specification +------------------- +Excel Spec is like an index to the Excel sheet to look for the data to be +collected by the tool. Excel Spec Sample specifies all the details that +need to be filled by the Deployment Engineer. + +Below is the definition for each key in the Excel spec + +:: + + + ipmi_sheet_name - name of the sheet from where IPMI and host profile information is to be read + start_row - row number from where the IPMI and host profile information starts + end_row - row number from where the IPMI and host profile information ends + hostname_col - column number where the hostnames are to be read from + ipmi_address_col - column number from where the ipmi addresses are to be read + host_profile_col - column number from where the host profiles are to be read + ipmi_gateway_col - column number from where the ipmi gateways are to be read + private_ip_sheet - name of the sheet which has the private IP information + net_type_col - column number from where the network type is to be read + vlan_col - column number from where the network vlan is to be read + vlan_start_row - row number from where the vlan information starts + vlan_end_row - row number from where the vlan information ends + net_start_row - row number from where the network information starts + net_end_row - row number from where the network information ends + net_col - column number where the IP ranges for network is to be read + net_vlan_col - column number where the vlan information is present in the pod wise network section + public_ip_sheet - name of the sheet which has the public IP information + oam_vlan_col - column number from where the OAM vlan information is to be read from + oam_ip_row - row number from where the OAM network information is to be read from + oam_ip_col - column number from where the OAM network information is to be read from + oob_net_row - row number which has the OOB network subnet ranges + oob_net_start_col - column number from where the OOB network ranges start + oob_net_end_col - column number from where the OOB network ranges end + ingress_ip_row - row number from where the Ingress network information is to be read from + dns_ntp_ldap_sheet - name of the sheet which has the DNS, NTP and LDAP information + login_domain_row - row number which has the ldap login domain + ldap_col - column number which has the all ldap related information + global_group - row number which has the ldap group information + ldap_search_url_row - row number which has the ldap url + ntp_row - row number which has the ntp information + ntp_col - column number which has the ntp information + dns_row - row number which has the dns information + dns_col - column number which has the dns information + domain_row - row number which has the domain information + domain_col - column number which has the domain information + location_sheet - name of the sheet which has the location information + column - column number which has all the information + corridor_row - row number which has the corridor information + site_name_row - row number which has the site name + state_name_row - row number which has the state name + country_name_row - row number which has the country name + clli_name_row - row number which has CLLI information + +Example: Tugboat Plugin Usage +----------------------------- +1. Required Input(Refer to 'spyglass/examples' folder to get these inputs) + a) Excel File: SiteDesignSpec_v0.1.xlsx + b) Excel Spec: excel_spec_upstream.yaml + c) Site Config: site_config.yaml + d) Template_dir: '../examples/templates' + c) Site name: airship-seaworthy + +2. Spyglass CLI Command: + spyglass -mg -t tugboat -x SiteDesignSpec_v0.1.xlsx -e excel_spec_upstream.yaml -d site_config.yaml -s airship-seaworthy --template_dir=