Commit Graph

12 Commits

Author SHA1 Message Date
Ian H Pittwood 166483d6ad Fix Spyglass gates
Zuul gates broke for Spyglass due to an update made to either Zuul or
Openstack's Zuul jobs (I wasn't able to pin it down). The update caused
Zuul to attempt to install Spyglass's requirements.txt file instead of
referring to the method given in the tox configuration that utilizes
Pipenv. To bypass this issue, the requirements.txt will be removed and
Spyglass will be fully moved over to using Pipenv.

Removes requirements.txt

Updates Pipfile and Pipfile.lock for new package releases

Updates Dockerfile to install requirements from Pipfile.lock

Change-Id: I347ef6db18a44e1b88c811aa071ba8c60bccbd53
2019-08-14 14:13:47 -05:00
Alexander Hughes 9689dae61f Standardize Spyglass code with YAPF
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase.  This is the effort
to do so for the Spyglass project.

[0] https://review.opendev.org/#/c/671291/

Co-Authored-By: Ian Pittwood <pittwoodian@gmail.com>
Change-Id: I2b0f2491d83675d7742b168ecb9e1dfb9a11e719
2019-07-25 17:01:33 +00:00
Ian H. Pittwood 4909870115 Implement Pipenv dependency management
Pipenv is a tool that brings better package dependency management to
python. It can automatically create and manage virtualenv as well as
managing package dependencies using Pipfile and Pipfile.lock. Adding
this dependency manager into Airship projects will decrease package
version conflicts between projects and help increase security through
hash validation of packages and vulnerability scans.

Changes:
- Imports requirements.txt type files into Pipfile
- Pipenv dependency management in tox
- Switches Safety package for "pipenv check", an implementation of
Safety
- Unpins or loosens pins on all development packages
- Soft pins package dependencies to encourage adoption of bug fixes and
prevent small version mismatches

Pipenv Docs: https://docs.pipenv.org/en/latest/

Helpful Pipenv Guide: https://realpython.com/pipenv-guide/

Change-Id: I2c9cec8acf6b6c6157f807b010def873c349d3ae
2019-07-01 16:42:34 +00:00
Ian Pittwood 1cda5b334e Pin dependency versions
Pins all Spyglass dependencies explicitly for consistency.

Removes unused packages.

Related to https://review.opendev.org/#/c/662069/

Change-Id: I5ca302e7315a8606510c6e83092188291f661760
2019-06-10 10:42:20 -05:00
Hughes, Alexander (ah8742) 6d72d7f9fc Spyglass update requirements.txt
New version of PyYAML available, updating pin

Change-Id: I579515cf7e5e10bc8f3ed0fc3aa504fe85040fc1
2019-06-07 14:45:35 +00:00
Ian H. Pittwood f57073333d Adds Safety dependency vulnerability checks
This change adds dependency vulnerability checks into tox. These checks
are performed by the Safety package which checks requirements files
against a vulnerability database for any issues. If any vulnerabilities
are found, tox will fail its pep8 env.

Safety: https://pyup.io/safety/

Safety Repo: https://github.com/pyupio/safety

Safety Default Vulnerability DB: https://github.com/pyupio/safety-db

Depends on https://review.opendev.org/#/c/658133

Change-Id: Ief52dd664700374c0ebbb7a4ab1f64c9940abcde
2019-05-29 11:46:57 -05:00
Ian Pittwood a002e4203d Separate plugins from Spyglass
This change removes plugins from Spyglass and places them in separate
repositories. Formation, a proprietary plugin, will be removed by this
change and Tugboat will become its own OpenDev maintained repo,
spyglass-plugin-xls. By creating more streamlined plugin management,
end users should be able to more easily create their own plugins for
different data sources.

Related change https://review.opendev.org/#/c/659116/

Depends-On: Ib2f75878b1a29e835cb8e2323aebe9d431c479e7

Change-Id: Ie0eb2e5aefe6bb764e1aa608e53371adaabb9a17
2019-05-29 10:30:14 -05:00
Ian Pittwood 49137d352c Upgrade PyYAML from 3.12 to 5.1
The latest version of PyYAML fixes some security issues found in
previous versions. It also supports python 3.7. No changes between
these versions should break Spyglass functionality. Six is also removed
in this patch as it is no longer used in Spyglass.

https://pyyaml.org/wiki/PyYAML

Change-Id: I76fe7396b3bad3b863fd4e05330bf8f58f21d57a
2019-05-14 11:38:20 -05:00
Ian Pittwood d44dfd7526 Implements PBR for packaging
Spyglass will soon have its plugins separated from its main repo. Many
other OpenStack repositories that utilize plugins have implemented
Python Build Reasonableness to manage their entry points for plugins.
As Spyglass will be managing plugins, it makes sense to stay uniform
with other projects and implement PBR.

Implements PBR using setup.py and setup.cfg files.

Changes README to correct file extension.

Freezes PIP requirements file.

Moves YAPF configuration into setup.cfg file.

Change-Id: I48ac5aeb3e544d3da7ce25bd9b827d8fd9c4e108
2019-05-10 18:49:54 +00:00
Ian Pittwood c46e15385c Refactors CLI to match style of Airship CLIs
The original CLI is based on click, but includes all options stacked on
one main method. This is in contrast to other Airship CLIs such as
Pegleg which utilizes click's nesting features to organize its CLI into
multiple groups and commands that each have their own options.

This change separates the Spyglass CLI into three different commands:
generate intermediary, generate manifests, and generate manifests from
intermediary.

Adds a 'verbose' flag on Spyglass.

Defaults plugin to 'tugboat'.

Adds validation for options that apply specifically to tugboat or
formation.

Related docs change: https://review.opendev.org/#/c/650137/

Change-Id: I92e5f040d5205c3ab36ec1d46ecd57bc97849cef
2019-05-06 16:04:02 -05:00
Alexander Hughes 3bf68e464a PEP-8 code styling and linting
As a newer project, Spyglass is still a work in progress in terms of its
styling. This change is meant to help improve readability and compliance
of Spyglass source code.

To match other Airship projects, this change implements YAPF into tox so
linting is automatically ran. This will hopefully keep formatting grey
areas styled consistently.

PEP-8: https://www.python.org/dev/peps/pep-0008/

OpenStack Style Guidelines:
https://docs.openstack.org/hacking/latest/user/hacking.html

PEP Index: https://www.python.org/dev/peps/

Depends on https://review.openstack.org/#/c/648764.

Change-Id: I45b19cc8a7932fd7823dcb69f64a0a1bf19fc434
2019-04-22 18:46:48 +00:00
Hemanth Nakkina 296705a0a5 Base code for spyglass
- Spyglass skelton with engine, site processor
- Spyglass data extractor with formation plugin
- Docker files and scripts to run spyglass
2018-11-28 15:19:43 +05:30