airship
/
deckhand
Code Issues Proposed changes

Functional tests for layering + substitution scenarios 

This PS introduces functional tests for layering and substitution
scenarios working in tandem. Only unit tests currently offer this type
of coverage.

This PS is a follow-up to https://review.gerrithub.io/#/c/395610/
which adds functional tests that test success paths for basic
substitutions including the following edge cases:

  * Substitution/layering works with top layer empty
  * Substitution/layering works with multiple top layers empty
  * Substitution/layering works with intermediate layer empty

These tests are more robust than the ones introduced by
https://review.gerrithub.io/#/c/395388/ while offering
the same exact coverage so the tests in that patch have
been removed.

This PS also fixes a subtle bug related to layering which was
causing the layering + substitution scenarios above to fail.

Change-Id: I9303a93f4d73f7fdca664b861bc5ab06f4162b79
This commit is contained in:
Felipe Monteiro 2018-01-21 12:36:08 -05:00
parent 5108f56d36
commit 5ed2f64e35
5 changed files with 253 additions and 56 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
deckhand/engine/layering.py
View File

@ -346,12 +346,10 @@ class DocumentLayering(object):
# Update the actual document data if concrete.
if not child.is_abstract:
if child.substitutions:
rendered_data.substitutions = child.substitutions
substituted_data = self._apply_substitutions(
rendered_data)
if substituted_data:
rendered_data = substituted_data[0]
child.data = rendered_data.data
substituted_data = self._apply_substitutions(child)
if substituted_data:
rendered_data = substituted_data[0]
child_index = self._layered_documents.index(child)
self._layered_documents[child_index].data = (
rendered_data.data)

230
deckhand/tests/functional/gabbits/document-render-success-single-bucket-with-layering-and-substitution.yaml Normal file
View File

@ -0,0 +1,230 @@
# Tests success paths for basic layering/substitution scenario as well as
# success paths for the following edge cases:
# * Substitution/layering works with top layer empty
# * Substitution/layering works with multiple top layers empty
# * Substitution/layering works with intermediate layer empty
#
# Base case:
# 1. Purges existing data to ensure test isolation.
# 2. Creates basic LayeringPolicy with 2 layers.
# 3. Adds initial documents for layering/substitution base case.
# 4. Verifies fully substituted/layered document data.
# Edge cases:
# 5. Creates LayeringPolicy with 3 layers, where topmost is empty.
# 6. Adds initial documents for layering/substitution base case.
# 7. Verifies fully substituted/layered document data.
# 8. Creates LayeringPolicy with 4 layers, where 2 topmost are empty.
# 9. Adds initial documents for layering/substitution base case.
# 10. Verifies fully substituted/layered document data.
# 11. Creates LayeringPolicy with 5 layers, with empty interspersed layers.
# 12. Adds initial documents for layering/substitution base case.
# 13. Verifies fully substituted/layered document data.
defaults:
request_headers:
content-type: application/x-yaml
response_headers:
content-type: application/x-yaml
tests:
- name: purge
desc: Begin testing from known state.
DELETE: /api/v1.0/revisions
status: 204
response_headers: null
- name: initialize_layering_policy_base_case
desc: |
Initailize the layering policy with 2 layers where the child document
that requires substitution has layer site and the parent document which
is abstract has layer region.
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: |-
---
schema: deckhand/LayeringPolicy/v1
metadata:
schema: metadata/Control/v1
name: layering-policy
data:
layerOrder:
- region
- site
...
- name: initialize_base_case
desc: |
Base case for testing layering alongside substitution using two layers:
region and site, where site layers with region, then receives multiple
substitutions from secret documents.
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: <@resources/layering-and-substitution-example.yaml
- name: verify_base_case
desc: Check for expected substitutions
GET: /api/v1.0/revisions/$RESPONSE['$.[0].status.revision']/rendered-documents
query_parameters:
metadata.name: armada-chart-01
status: 200
response_multidoc_jsonpaths:
$.`len`: 1
$.[0].metadata.name: armada-chart-01
$.[0].data:
chart:
details:
data: here
values:
tls:
certificate: |
CERTIFICATE DATA
key: |
KEY DATA
some_url: http://admin:my-secret-password@service-name:8080/v1
from-parent: parent-val
- name: initialize_layering_policy_with_top_empty_layer
desc: |
Re-initailize the layering policy with 3 layers so that the top is empty
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: |-
---
schema: deckhand/LayeringPolicy/v1
metadata:
schema: metadata/Control/v1
name: layering-policy
data:
layerOrder:
- global # Empty
- region
- site
...
- name: initialize_base_case_with_empty_top_layer
desc: |
Same case as before, except with a top empty layer.
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: <@resources/layering-and-substitution-example.yaml
- name: verify_empty_top_layer
desc: Check for expected substitutions
GET: /api/v1.0/revisions/$RESPONSE['$.[0].status.revision']/rendered-documents
query_parameters:
metadata.name: armada-chart-01
status: 200
response_multidoc_jsonpaths:
$.`len`: 1
$.[0].metadata.name: armada-chart-01
$.[0].data:
chart:
details:
data: here
values:
tls:
certificate: |
CERTIFICATE DATA
key: |
KEY DATA
some_url: http://admin:my-secret-password@service-name:8080/v1
from-parent: parent-val
- name: initialize_layering_policy_with_2_top_empty_layers
desc: |
Re-initailize the layering policy with 4 layers so that the top 2 can be
empty
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: |-
---
schema: deckhand/LayeringPolicy/v1
metadata:
schema: metadata/Control/v1
name: layering-policy
data:
layerOrder:
- meta # Empty
- global # Empty
- region
- site
...
- name: initialize_base_case_with_multiple_empty_top_layers
desc: |
Same case as before, except with multiple empty top layers.
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: <@resources/layering-and-substitution-example.yaml
- name: verify_multiple_empty_top_layers
desc: Check for expected substitutions
GET: /api/v1.0/revisions/$RESPONSE['$.[0].status.revision']/rendered-documents
query_parameters:
metadata.name: armada-chart-01
status: 200
response_multidoc_jsonpaths:
$.`len`: 1
$.[0].metadata.name: armada-chart-01
$.[0].data:
chart:
details:
data: here
values:
tls:
certificate: |
CERTIFICATE DATA
key: |
KEY DATA
some_url: http://admin:my-secret-password@service-name:8080/v1
from-parent: parent-val
- name: initialize_layering_policy_with_interspersed_empty_layers
desc: |
Re-initailize the layering policy with 5 layers with empty interspersed
layers.
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: |-
---
schema: deckhand/LayeringPolicy/v1
metadata:
schema: metadata/Control/v1
name: layering-policy
data:
layerOrder:
- meta # Empty
- global # Empty
- region
- sub-region # Empty
- site
...
- name: initialize_base_case_with_multiple_interspersed_empty_layers
desc: |
Same case as before, except with multiple empty interspersed layers.
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: <@resources/layering-and-substitution-example.yaml
- name: verify_multiple_empty_top_layers
desc: Check for expected substitutions
GET: /api/v1.0/revisions/$RESPONSE['$.[0].status.revision']/rendered-documents
query_parameters:
metadata.name: armada-chart-01
status: 200
response_multidoc_jsonpaths:
$.`len`: 1
$.[0].metadata.name: armada-chart-01
$.[0].data:
chart:
details:
data: here
values:
tls:
certificate: |
CERTIFICATE DATA
key: |
KEY DATA
some_url: http://admin:my-secret-password@service-name:8080/v1
from-parent: parent-val

38
deckhand/tests/functional/gabbits/document-render-success-single-bucket-with-substitution.yaml
View File

@ -3,11 +3,6 @@
# 1. Purges existing data to ensure test isolation
# 2. Adds initial documents from substitution sample of design doc
# 3. Verifies fully substituted document data
# 4. Purges existing data to ensure test isolation
# 5. Adds initial documents from substitution sample of design doc using just
# a single layer in order to verify that substitution is carried out for
# this edge case
# 6. Verifies fully substituted document data (again)
defaults:
request_headers:
@ -48,36 +43,3 @@ tests:
key: |
KEY DATA
some_url: http://admin:my-secret-password@service-name:8080/v1
- name: purge_again
desc: Begin testing from known state.
DELETE: /api/v1.0/revisions
status: 204
response_headers: null
- name: initialize_single_layer
desc: Create initial documents
PUT: /api/v1.0/buckets/mop/documents
status: 200
data: <@resources/design-doc-substitution-sample-single-layer.yaml
- name: verify_substitutions_single_layer
desc: Check for expected substitutions
GET: /api/v1.0/revisions/$RESPONSE['$.[0].status.revision']/rendered-documents
query_parameters:
schema: armada/Chart/v1
status: 200
response_multidoc_jsonpaths:
$.`len`: 1
$.[*].metadata.name: example-chart-01
$.[*].data:
chart:
details:
data: here
values:
tls:
certificate: |
CERTIFICATE DATA
key: |
KEY DATA
some_url: http://admin:my-secret-password@service-name:8080/v1

30
deckhand/tests/functional/gabbits/resources/design-doc-substitution-sample-single-layer.yaml → deckhand/tests/functional/gabbits/resources/layering-and-substitution-example.yaml
View File

@ -1,13 +1,4 @@
---
schema: deckhand/LayeringPolicy/v1
metadata:
schema: metadata/Control/v1
name: layering-policy
data:
layerOrder:
- region
- site
---
schema: deckhand/Certificate/v1
metadata:
name: example-cert
@ -39,10 +30,27 @@ data: my-secret-password
---
schema: armada/Chart/v1
metadata:
name: example-chart-01
schema: metadata/Document/v1
name: armada-chartgroup-01
labels:
key1: value1
layeringDefinition:
abstract: true
layer: region
data:
from-parent: parent-val
---
schema: armada/Chart/v1
metadata:
schema: metadata/Document/v1
name: armada-chart-01
layeringDefinition:
layer: site
parentSelector:
key1: value1
actions:
- method: merge
path: .
substitutions:
- dest:
path: .chart.values.tls.certificate
@ -69,4 +77,4 @@ data:
data: here
values:
some_url: http://admin:INSERT_PASSWORD_HERE@service-name:8080/v1
...
...

1
deckhand/tests/unit/engine/test_document_layering_and_substitution.py
View File

@ -242,7 +242,6 @@ class TestDocumentLayeringWithSubstitution(
"name": "site-cert",
"path": "."
}
}],
}
doc_factory = factories.DocumentFactory(2, [1, 1])