Upgrades yapf to newest version, 0.28.0, and runs the formatter to
update all existing code.
Adds entry to .gitignore to ignore pyenv installations.
Change-Id: I11512a8a522cc530165461cc8f52f7ff010dd092
This change makes some mild restructuring and naming changes to the
data-extractor plugin base. Data extraction will now be separated into
two main workflows: loading raw data and source specific parsing into
model objects. Naming of methods was changed to help more accurately
reflect their function in these workflows.
Change-Id: Ia3bc892994ff96cce5c1672fadf35ef2d1c4164b
Increases test coverage of the Spyglass parser engine from 0% to 82%,
bringing overall test coverage to 86.4%. This change moves minimum
coverage to 84%.
Change-Id: I2de496b8d7f4c4252be22c713605fae6fd565b66
Plugins currently use a couple configuration methods to set up field
data for the class. This seems superfluous. This change moves these
configuration steps into the init method so plugin classes can ingest
data as kwargs.
Here is an example of how the change will be implemented in plugins:
https://review.opendev.org/#/c/670171/
Change-Id: Ib26636f1eb4146902ee801af5bcce53d137be2ad
Spyglass has three different files that define exceptions. This change
merges all of those definitions into a single file for simplicity.
Before this change can be merged, spyglass-plugin-xls needs to move the
exceptions needed for its functions.
Related Change: https://review.opendev.org/#/c/667243/
Depends-On: I81c5ca2d9083aece3641bc8b5405dfd44baec810
Change-Id: Ibf34771653247850903e529beb17dbd60ba35fbf
This is a follow-up change to [0] which further implements data objects
to be used in the generation of documents in Jinja2. The following
additions and changes are made:
- Adds helper functions to data objects to filter hosts and networks for
ease of use in Jinja2 templates
- Adds SiteDocumentData factory function to convert intermediary yaml
dictionaries into a SiteDocumentData object with all associated objects
- Updates Jinja2 templates to use data objects
- Cleans up overly complex looping in Jinja2
- Adds tests for new code in models.py
[0] https://review.opendev.org/#/c/662092/
Change-Id: I66ebfeaf5d6ca76b6dee5a2285a74bad8b06b720
This change implements data object models from [0] in data extraction
and parsing. The change results in minor modifications to the outputted
intermediary, which can be seen between these two example intermeidary
files [1].
This fully implements the data objects from models.py in data extraction
and parsing. A follow-up change will implement use of the data objects
in Jinja2. Temporarily, all objects will be converted to dictionaries
for generating documents from templates.
[0] https://review.opendev.org/#/c/658917/
[1] https://www.diffchecker.com/NnjjJrb2
Change-Id: Ifd867787aab541be5dabecf9f6026faa2ec7049e
Adds unit tests for all data objects created in [0]. The changes in [0]
were merged in short succession with [1], causing the test coverage gate
to fail with all the newly introduced lines of code. This change adds
tests for all of the newly added code and increases test coverage
requirement to 60% (currently at 65.93%).
[0] https://review.opendev.org/#/c/658917/
[1] https://review.opendev.org/#/c/663729/
Change-Id: I96931e3e415af80ca5ab9202c2bda0344a9901f0
This change switches Spyglass from storing data in large multi-level
dictionaries to using several data objects. The intent of this change is
to make management of Spyglass data easier for developers to understand.
Implementation of the data objects will be handled in a separate change
to keep code review more manageable.
Change-Id: I101ad1ccbd95822965b8da8b6a644522eb2908e7
This change removes plugins from Spyglass and places them in separate
repositories. Formation, a proprietary plugin, will be removed by this
change and Tugboat will become its own OpenDev maintained repo,
spyglass-plugin-xls. By creating more streamlined plugin management,
end users should be able to more easily create their own plugins for
different data sources.
Related change https://review.opendev.org/#/c/659116/
Depends-On: Ib2f75878b1a29e835cb8e2323aebe9d431c479e7
Change-Id: Ie0eb2e5aefe6bb764e1aa608e53371adaabb9a17
The initial addition of yapf into Spyglass caused a few alignment
issues that were temporarily fixed by disabling yapf. This change adds
a knob to the yapf configuration that causes long function statements
to always break before the first statement. This results in more
consistent, visually pleasing code.
Change-Id: I18f9a7677c61524fed12e71a2ecf1003a6ee0ad9
Spyglass will soon have its plugins separated from its main repo. Many
other OpenStack repositories that utilize plugins have implemented
Python Build Reasonableness to manage their entry points for plugins.
As Spyglass will be managing plugins, it makes sense to stay uniform
with other projects and implement PBR.
Implements PBR using setup.py and setup.cfg files.
Changes README to correct file extension.
Freezes PIP requirements file.
Moves YAPF configuration into setup.cfg file.
Change-Id: I48ac5aeb3e544d3da7ce25bd9b827d8fd9c4e108
Some of the default behaviors of yapf do not match the styling
preferences of Airship and OpenDev, such as line breaks between
dictionary key/value pairs. This change sets the style template for yapf
(PEP-8) and sets a few knobs to help remedy problematic behavior.
Adds .style.yapf configuration file.
Updates styling of existing files using new configuration.
Change-Id: Ifea58ef2d7be93f47dd8e4f42cac6aea514b83bd
The tox configuration needs to be updated in order for it to be helpful
for Zuul. This change makes some minor modifications to the tox config
to improve its usability and uniformity with other Open Stack configs.
Makes bare minimum changes for pep8 tests compliance.
Adds gate scripts for white space linting.
In the future, additional changes will need to be made to add tests to
Spyglass. These changes will need to enable the 'py36' and 'cover' tox
env and set coverage levels. Additional linting should also be
performed to improve Spyglass code readability and organization.
Change-Id: I73f946619786e661d02c69bd8e197453f049e0c7
From Bandit's docs:
It was discovered that some projects used assert to enforce interface
constraints. However, assert is removed with compiling to optimised byte
coden (python -o producing *.pyo files). This caused various protections
to be removed. The use of assert is also considered as general bad
practice in OpenStack codebases.
Change-Id: Ie2b5e9cc3e1afcf9e9bd0f8675947754fa1e4b7d
Ian H Pittwood