Commit Graph

21 Commits

Author SHA1 Message Date
Ian H Pittwood 8cc2a62255 Use Pipfile.lock only in tox gates
Pipenv validates that the Pipfile.lock that it uses for installing
packages is not out of date before following through on execution. If it
is not, it will throw an error. This change adds an env that tells
Pipenv to ignore the Pipfile and only use Pipfile.lock so gates do not
fail due to out of date packages.

Change-Id: I2fa541b31e14561c7cefc330e7ef2cc0b0e4abc6
2019-08-14 19:14:51 +00:00
Ian H. Pittwood 61ecee4624 Use py36 job for safety checks
When using pipenv, the targeted version must be used when attempting to
do pipenv check. This change updates the safety dependency check to use
openstack-tox-py36  so the correct python version will be used.

Change-Id: I4b9f71cfaaa74724ecee7b40ea6e9362e836d261
2019-08-14 19:14:25 +00:00
Alexander Hughes 9689dae61f Standardize Spyglass code with YAPF
From recently merged document updates in [0] there is a desire to
standardize the Airship project python codebase.  This is the effort
to do so for the Spyglass project.

[0] https://review.opendev.org/#/c/671291/

Co-Authored-By: Ian Pittwood <pittwoodian@gmail.com>
Change-Id: I2b0f2491d83675d7742b168ecb9e1dfb9a11e719
2019-07-25 17:01:33 +00:00
Zuul 534fe50b9e Merge "Add tests for site processor init" 2019-07-24 13:42:43 +00:00
Ian H. Pittwood 8304add63e Add tests for site processor init
This change adds a couple tests for init in Spyglass' site processor
class, bringing total test coverage up to the goal of 90%.

Change-Id: I81c1dfdf2a9d65589d961451401802621058a9f2
2019-07-23 09:01:47 -05:00
Ian H Pittwood 8349197be8 Move safety checks to new Zuul job
The pyup.io database that is used to check dependencies for
vulnerabilities in Spyglass is currently down. When first implemented,
this check was placed in the tox pep8 job. This change moves the
vulnerability check into its own Zuul job so it can be disabled in
instances such as this where the database is down.

Specifies basepython to python3 for safety and bandit jobs.

Change-Id: I4e13c5341a9e62095587ec1820be79a621380f09
2019-07-23 08:40:51 -05:00
Ian H Pittwood b7c2bc7ccd Add tests for parsing engine
Increases test coverage of the Spyglass parser engine from 0% to 82%,
bringing overall test coverage to 86.4%. This change moves minimum
coverage to 84%.

Change-Id: I2de496b8d7f4c4252be22c713605fae6fd565b66
2019-07-18 14:27:45 +00:00
Ian H. Pittwood ee18e3e94b Upgrade Sphinx package
Sphinx 2.1.0 has a bug [0] that causes whitespace to be excluded in
outputted docs. This change updates Sphinx to peg any version >2.1.0.

Safety dependency vulnerability checks now will also cover the doc
requirements.txt.

[0] https://github.com/sphinx-doc/sphinx/issues/6440

Change-Id: I35f1acf4385821969ffa7c9807cf209a59c1d73e
2019-07-10 10:30:05 -05:00
Ian H. Pittwood 4909870115 Implement Pipenv dependency management
Pipenv is a tool that brings better package dependency management to
python. It can automatically create and manage virtualenv as well as
managing package dependencies using Pipfile and Pipfile.lock. Adding
this dependency manager into Airship projects will decrease package
version conflicts between projects and help increase security through
hash validation of packages and vulnerability scans.

Changes:
- Imports requirements.txt type files into Pipfile
- Pipenv dependency management in tox
- Switches Safety package for "pipenv check", an implementation of
Safety
- Unpins or loosens pins on all development packages
- Soft pins package dependencies to encourage adoption of bug fixes and
prevent small version mismatches

Pipenv Docs: https://docs.pipenv.org/en/latest/

Helpful Pipenv Guide: https://realpython.com/pipenv-guide/

Change-Id: I2c9cec8acf6b6c6157f807b010def873c349d3ae
2019-07-01 16:42:34 +00:00
Ian H. Pittwood 313058b8fb Adds tests for Spyglass data objects
Adds unit tests for all data objects created in [0]. The changes in [0]
were merged in short succession with [1], causing the test coverage gate
to fail with all the newly introduced lines of code. This change adds
tests for all of the newly added code and increases test coverage
requirement to 60% (currently at 65.93%).

[0] https://review.opendev.org/#/c/658917/
[1] https://review.opendev.org/#/c/663729/

Change-Id: I96931e3e415af80ca5ab9202c2bda0344a9901f0
2019-06-18 15:15:28 -05:00
Ian Pittwood 6ee44d4974 Adds unit tests for Spyglass CLI
This change implements unit testing for code in the Spyglass CLI.

Change-Id: I4d57bb4e7ee1a2fed8d10cab5eb10636ec599a17
2019-06-17 16:56:18 +00:00
Ian H. Pittwood 60da55cd18 Schema validation
Adds JSON schema validation to Spyglass.

Change-Id: Ib29bbf9fa02cd6623c75db37a4c8d6f510b52831
2019-06-06 12:56:43 +00:00
Ian H. Pittwood f57073333d Adds Safety dependency vulnerability checks
This change adds dependency vulnerability checks into tox. These checks
are performed by the Safety package which checks requirements files
against a vulnerability database for any issues. If any vulnerabilities
are found, tox will fail its pep8 env.

Safety: https://pyup.io/safety/

Safety Repo: https://github.com/pyupio/safety

Safety Default Vulnerability DB: https://github.com/pyupio/safety-db

Depends on https://review.opendev.org/#/c/658133

Change-Id: Ief52dd664700374c0ebbb7a4ab1f64c9940abcde
2019-05-29 11:46:57 -05:00
Alexander Hughes 2fe2bf5e26 Add readthedocs job for Spyglass
Change-Id: I7973d8e36c2aa9e173522a01076ce73228ba96a9
2019-05-16 14:39:46 +00:00
Ian H. Pittwood b9f2b06607 Enable bandit checks in tox
This change enables bandit in tox checks. Existing bandit errors were
resolved by:

- https://review.opendev.org/#/c/651823/
- https://review.opendev.org/#/c/648764/
- https://review.opendev.org/#/c/651807/

Change-Id: I8dc7ca828907b88af9463f1e9a88a86fbd3b6445
2019-04-29 16:42:25 +00:00
Alexander Hughes 56e37ed5c9 Update Spyglass make images command
Bring files used in make images in line with other airship projects

Change-Id: Idb1858e3bc8fa64cfbeb768d999066cbd832e698
2019-04-24 12:03:32 -05:00
Zuul 7da3e51798 Merge "PEP-8 code styling and linting" 2019-04-24 13:04:51 +00:00
Alexander Hughes 3bf68e464a PEP-8 code styling and linting
As a newer project, Spyglass is still a work in progress in terms of its
styling. This change is meant to help improve readability and compliance
of Spyglass source code.

To match other Airship projects, this change implements YAPF into tox so
linting is automatically ran. This will hopefully keep formatting grey
areas styled consistently.

PEP-8: https://www.python.org/dev/peps/pep-0008/

OpenStack Style Guidelines:
https://docs.openstack.org/hacking/latest/user/hacking.html

PEP Index: https://www.python.org/dev/peps/

Depends on https://review.openstack.org/#/c/648764.

Change-Id: I45b19cc8a7932fd7823dcb69f64a0a1bf19fc434
2019-04-22 18:46:48 +00:00
Tin Lam 0ffa641c0c Remove unnecessary exclusion
Change-Id: Ic9adf554276d0a9ba6c7b3dc43ed14868b3d7a33
Signed-off-by: Tin Lam <tin@irrational.io>
2019-04-19 14:09:27 +00:00
Alexander Hughes 03a058bfcf Various fixes to Spyglass to resolve tox errors
The tox configuration needs to be updated in order for it to be helpful
for Zuul. This change makes some minor modifications to the tox config
to improve its usability and uniformity with other Open Stack configs.

Makes bare minimum changes for pep8 tests compliance.

Adds gate scripts for white space linting.

In the future, additional changes will need to be made to add tests to
Spyglass. These changes will need to enable the 'py36' and 'cover' tox
env and set coverage levels. Additional linting should also be
performed to improve Spyglass code readability and organization.

Change-Id: I73f946619786e661d02c69bd8e197453f049e0c7
2019-04-16 12:22:53 -05:00
Hemanth Nakkina 296705a0a5 Base code for spyglass
- Spyglass skelton with engine, site processor
- Spyglass data extractor with formation plugin
- Docker files and scripts to run spyglass
2018-11-28 15:19:43 +05:30