This patch set removes few pep8/flake8 ignored rules and implemented
the fix in the code to address those rules.
Change-Id: I2e613acd760818a6e18288d284f6224c38c4353a
Signed-off-by: Tin Lam <tin@irrational.io>
This PS adds noauth middleware to bypass keystone authentication
which will occur when Deckhand's server is executed in development
mode. Development mode is enabled by setting development_mode as True
in etc/deckhand/deckhand.conf.sample.
The logic is similar to Drydock's here: [0].
[0] 1c78477e95/drydock_provisioner/util.py (L43)
Co-Authored-By: Luna Das <luna.das@imaginea.com>
Co-Authored-By: Felipe Monteiro <felipe.monteiro@att.com>
Change-Id: I677d3d92768e0aa1a550772700403e0f028b0c59
This patch set adds a new endpoint to the Validations API which allows
for listing all validations for a given revision with details.
The response body for GET /api/v1.0/{revision_id}/validations/detail
looks like:
---
count: 1
next: null
prev: null
results:
- name: promenade-site-validation
url: https://deckhand/api/v1.0/revisions/4/validations/promenade-site-validation/entries/0
status: failure
createdAt: 2017-07-16T02:03Z
expiresAfter: null
expiresAt: null
errors:
- documents:
- schema: promenade/Node/v1
name: node-document-name
- schema: promenade/Masters/v1
name: kubernetes-masters
message: Node has master role, but not included in cluster masters list.
Note that the Validations API in general is currently missing fields
like url (as well as next and prev references) which will be included
in a follow up.
This will enable Shipyard to avoid performing a quadratic number
of API look ups when querying Deckhand's Validations API: [0].
The policy enforced for this endpoint is deckhand:list_validations.
APIImpact
DocImpact
[0] 06b5e82ea8/shipyard_airflow/control/configdocs/deckhand_client.py (L265)
Change-Id: I827e5f47bffb23fa16ee5c8a705058034633baed
This patch set adds a relative symlink from docs/source/releasenotes
to releasenotes/source/ so that releasenotes can be published
to deckhand.readthedocs.io. A new tab link is added on the
documentation page which references Deckhand's release notes.
The command used to generate the symlink was:
ln -rs releasenotes/source/ docs/source/releasenotes
Change-Id: I07058ca47ecc7b1fb211cae53aedc5f796542cff
This PS enforces the design requirement that says that only 1
layering policy can exist in the system at once. Attempting to
create another layering policy with a different name is a 409 error.
The existing layering policy can be updated by passing in a document
with the same `metadata.name` and `schema` as the existing one.
Closes-Bug: https://github.com/att-comdev/deckhand/issues/12
Change-Id: I7cad2d600c931c8701c3faaf2967be782984528b
The Validations API has been introduced to Deckhand, allowing users
to register new validation results in Deckhand, as well as query
the API for validation results for a revision. The validation results
include a list of errors that occurred during document validation.
All functional tests related to the API are now passing.
The following endpoints have been implemented:
* /api/v1.0/revisions/{revision_id}/validations
* /api/v1.0/revisions/{revision_id}/validations/{validation_name}
* /api/v1.0/revisions/{revision_id}/validations/{validation_name}/entries
* /api/v1.0/revisions/{revision_id}/validations/{validation_name}/entries/{entry_id}
Some back-end refactoring was needed to implement this API. In
particular:
- Added a new Validation sqlalchemy DB model
- Introduced DataSchema handling to the engine.document_validation
module so that registered schema validations can be used
- Changed the way the result of the 'deckhand-schema-validation' internal
validation is generated: it is now the amalgamation of all the
internal and registered schema validations executed
- Introduced rawquery generation so that raw SQL queries can be used to
get results from DB
Fixed following bug:
- UniqueConstraint is now used to correctly generate unique constraints
for sqlalchemy models that are supposed to be combinations of columns
Change-Id: I53c79a6544f44ef8beab2600ddc8a3ea91ada903
This PS implements documentation substitution and
the rendered-documents endpoint. Each time the
rendered-documents is queried, the documents for
the reqeust revision_id dynamically undergo
secret substitution.
All functional tests related to secret substitution
have been unskipped.
Deckhand currently does not real testing for
verifying that secret substitution works
for encrypted documents. This will only happen
when integration testing is added to Deckhand to
test its interaction with Keystone and Barbican.
Included in this PS:
- basic implementation for secret substitution
- introduction of jsonpath_ng for searching for and
updating jsonpaths in documents
- rendered-documents endpoint
- unit tests
- all relevant functional tests unskipped
- additional bucket controller tests include RBAC
tests and framework testing RBAC via unit tests
Change-Id: I86f269a5b616b518e5f742a4005891412226fe2a
This PS implements oslo.policy integration in Deckhand.
The policy.py file implements 2 types of functions for
performing policy enforcement in Deckhand: authorize,
which is a decorator that is used directly around
falcon on_HTTP_VERB methods that raises a 403 immediately
if policy enforcement fails; and conditional_authorize,
to be used inside controller code conditionally.
For example, since Deckhand has two types of documents
with respect to security -- encrypted and cleartext
documents -- policy enforcement is conditioned on the
type of the documents' metadata.storagePolicy.
Included in this PS:
- policy framework implementation
- policy in code and policy documentation for all
Deckhand policies
- modification of functional test script to override
default admin-only policies with custom policy file
dynamically created using lax permissions
- bug fix for filtering out deleted documents (and
its predecessors in previous revisions) for
PUT /revisions/{revision_id}/documents
- policy documentation
- basic unit tests for policy enforcement framework
- allow functional tests to be filtered via regex
Due to the size of this PS, functional tests related to
policy enforcement will be done in a follow up.
Change-Id: If418129f9b401091e098c0bd6c7336b8a5cd2359
This PS adds support for filtering revisions and
revision documents documents by any legal filter
(those enumerated in the design document).
Deckhand now supports the following filter arguments:
* schema
* metadata.name
* metadata.label
* metadata.layeringDefinition.abstract
* metadata.layeringDefinition.layer
* status.bucket
Deckhand now supports the following filter arguments for filtering
revisions:
* tag
Deckhand now supports multiple filters, e.g.:
* ?metdata.layeringDefinition.layer=site&metadata.name=foo
Deckhand now supports repeated filters, e.g.:
* ?metadata.label=foo=bar&metadata.label=baz=qux
The following has yet to be implemented will be done in a future
follow-up PS:
- support sorting by specific keywords as well
- support query limit and offset filters
Change-Id: I8558481e075715fe7fab98140094d37782a986d9
This PS fixes an AttributeError raised in buckets controller
when Deckhand tries to parse a malformed YAML which in turn
causes Deckhand to attempt to raise a 400, but accesses an
invalid attribute in the exception object, causing the
AttributeError.
Example error:
AttributeError: 'ScannerError' object has no attribute 'format_message'
Change-Id: I54865e4830d3d34e813b1ecfa3105cf6243f2ca0
Tin Lam