Add 'last-site-action' & 'successful-site-action' Option
Shipyard retrieval of configdocs and renderedconfigdocs to support two new revisions, i.e. 'last-site-action' and 'successful-site-action'. 1) --last-site-action for the documents associated with the last 'successful or failed site action 2) --successful-site-action for the documents associated with the last successful site action This patch set updates the relevant Shipyard API, API client, CLI as well as document for the new options. Change-Id: I60fefa25147ad8e367bc3d1c8d45d4d08a5ee3f1
This commit is contained in:
parent
b7934502d6
commit
c0f8fc4359
|
@ -83,8 +83,8 @@ Document Staging API
|
||||||
--------------------
|
--------------------
|
||||||
Shipyard will serve as the entrypoint for documents (designs, secrets,
|
Shipyard will serve as the entrypoint for documents (designs, secrets,
|
||||||
configurations, etc...) into a site. Documents are posted to Shipyard in
|
configurations, etc...) into a site. Documents are posted to Shipyard in
|
||||||
collections, rather than individually. At any point in time, there will be two
|
collections, rather than individually. At any point in time, there will
|
||||||
represented versions of documents in a site that are accessible via this API:
|
be several versions of documents in a site that are accessible via this API:
|
||||||
|
|
||||||
- The "Committed Documents" version, which represents the last version of
|
- The "Committed Documents" version, which represents the last version of
|
||||||
documents that were successfully commited with a commit_configdocs action.
|
documents that were successfully commited with a commit_configdocs action.
|
||||||
|
@ -93,10 +93,16 @@ represented versions of documents in a site that are accessible via this API:
|
||||||
that only one set of documents maybe posted to the buffer at a time by
|
that only one set of documents maybe posted to the buffer at a time by
|
||||||
default. (This behavior can be overridden by query parameters issued by the
|
default. (This behavior can be overridden by query parameters issued by the
|
||||||
user of Shipyard)
|
user of Shipyard)
|
||||||
|
- The "Last Site Action" version represents the version of documents associated
|
||||||
|
with the last successful or failed site action. Site actions include 'deploy_site'
|
||||||
|
and 'update_site'.
|
||||||
|
- The "Successful Site Action" version represents the version of documents
|
||||||
|
associated with the last successful site action. Site actions include 'deploy_site'
|
||||||
|
and 'update_site'.
|
||||||
|
|
||||||
All versions of documents rely upon Deckhand for storage. Shipyard uses the
|
All versions of documents rely upon Deckhand for storage. Shipyard uses the
|
||||||
tagging features of Deckhand of to find the appropriate Committed Documents
|
tagging features of Deckhand to find the appropriate Committed Documents,
|
||||||
and Shipyard Buffer version.
|
Last Site Action, Successful Site Action and Shipyard Buffer version.
|
||||||
|
|
||||||
/v1.0/configdocs
|
/v1.0/configdocs
|
||||||
~~~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~~~
|
||||||
|
@ -188,7 +194,7 @@ Returns the source documents for a collection of documents
|
||||||
|
|
||||||
Query Parameters
|
Query Parameters
|
||||||
''''''''''''''''
|
''''''''''''''''
|
||||||
version=committed | **buffer**
|
version=committed | last_site_action | successful_site_action | **buffer**
|
||||||
Return the documents for the version specified - buffer by default.
|
Return the documents for the version specified - buffer by default.
|
||||||
|
|
||||||
Responses
|
Responses
|
||||||
|
@ -224,7 +230,7 @@ Returns the full set of configdocs in their rendered form.
|
||||||
|
|
||||||
Query Parameters
|
Query Parameters
|
||||||
''''''''''''''''
|
''''''''''''''''
|
||||||
version=committed|**buffer**
|
version=committed | last_site_action | successful_site_action | **buffer**
|
||||||
Return the documents for the version specified - buffer by default.
|
Return the documents for the version specified - buffer by default.
|
||||||
|
|
||||||
Responses
|
Responses
|
||||||
|
|
|
@ -575,14 +575,16 @@ Sample
|
||||||
get configdocs
|
get configdocs
|
||||||
~~~~~~~~~~~~~~
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Retrieve documents loaded into Shipyard, either committed or from the
|
Retrieve documents loaded into Shipyard. The possible options include last
|
||||||
Shipyard Buffer.
|
committed, last site action, last successful site action and retrieval from
|
||||||
|
the Shipyard Buffer. Site actions include deploy_site and update_site. Note
|
||||||
|
that we can only select one of the options for each CLI call.
|
||||||
|
|
||||||
::
|
::
|
||||||
|
|
||||||
shipyard get configdocs
|
shipyard get configdocs
|
||||||
[<collection>]
|
[<collection>]
|
||||||
[--committed | --buffer]
|
[--committed | --last-site-action | --successful-site-action | --buffer]
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
shipyard get configdocs design
|
shipyard get configdocs design
|
||||||
|
@ -595,6 +597,14 @@ Shipyard Buffer.
|
||||||
\--committed
|
\--committed
|
||||||
Retrieve the documents that have last been committed for this collection
|
Retrieve the documents that have last been committed for this collection
|
||||||
|
|
||||||
|
\--last-site-action
|
||||||
|
Retrieve the documents associated with the last successful or failed site
|
||||||
|
action for this collection
|
||||||
|
|
||||||
|
\--successful-site-action
|
||||||
|
Retrieve the documents associated with the last successful site action
|
||||||
|
for this collection
|
||||||
|
|
||||||
\--buffer
|
\--buffer
|
||||||
Retrive the documents that have been loaded into Shipyard since the
|
Retrive the documents that have been loaded into Shipyard since the
|
||||||
prior commit. If no documents have been loaded into the buffer for this
|
prior commit. If no documents have been loaded into the buffer for this
|
||||||
|
@ -608,7 +618,7 @@ Samples
|
||||||
$ shipyard get configdocs
|
$ shipyard get configdocs
|
||||||
Collection Committed Buffer
|
Collection Committed Buffer
|
||||||
coll1 present unmodified
|
coll1 present unmodified
|
||||||
coll2 not present created
|
coll2 not present created
|
||||||
|
|
||||||
|
|
||||||
::
|
::
|
||||||
|
@ -635,7 +645,7 @@ applying Deckhand layering and substitution.
|
||||||
::
|
::
|
||||||
|
|
||||||
shipyard get renderedconfigdocs
|
shipyard get renderedconfigdocs
|
||||||
[--committed | --buffer]
|
[--committed | --last-site-action | --successful-site-action | --buffer]
|
||||||
|
|
||||||
Example:
|
Example:
|
||||||
shipyard get renderedconfigdocs
|
shipyard get renderedconfigdocs
|
||||||
|
@ -643,6 +653,12 @@ applying Deckhand layering and substitution.
|
||||||
\--committed
|
\--committed
|
||||||
Retrieve the documents that have last been committed.
|
Retrieve the documents that have last been committed.
|
||||||
|
|
||||||
|
\--last-site-action
|
||||||
|
Retrieve the documents associated with the last successful or failed site action.
|
||||||
|
|
||||||
|
\--successful-site-action
|
||||||
|
Retrieve the documents associated with the last successful site action.
|
||||||
|
|
||||||
\--buffer
|
\--buffer
|
||||||
Retrieve the documents that have been loaded into Shipyard since the
|
Retrieve the documents that have been loaded into Shipyard since the
|
||||||
prior commit. (default)
|
prior commit. (default)
|
||||||
|
|
|
@ -26,7 +26,10 @@ from shipyard_airflow.control.configdocs.configdocs_helper import (
|
||||||
from shipyard_airflow.errors import ApiError
|
from shipyard_airflow.errors import ApiError
|
||||||
|
|
||||||
CONF = cfg.CONF
|
CONF = cfg.CONF
|
||||||
VERSION_VALUES = ['buffer', 'committed']
|
VERSION_VALUES = ['buffer',
|
||||||
|
'committed',
|
||||||
|
'last_site_action',
|
||||||
|
'successful_site_action']
|
||||||
|
|
||||||
|
|
||||||
class ConfigDocsStatusResource(BaseResource):
|
class ConfigDocsStatusResource(BaseResource):
|
||||||
|
@ -113,7 +116,8 @@ class ConfigDocsResource(BaseResource):
|
||||||
def get_collection(self, helper, collection_id, version='buffer'):
|
def get_collection(self, helper, collection_id, version='buffer'):
|
||||||
"""
|
"""
|
||||||
Attempts to retrieve the specified collection of documents
|
Attempts to retrieve the specified collection of documents
|
||||||
either from the buffer or committed version, as specified
|
either from the buffer, committed version, last site action
|
||||||
|
or successful site action, as specified
|
||||||
"""
|
"""
|
||||||
return helper.get_collection_docs(version, collection_id)
|
return helper.get_collection_docs(version, collection_id)
|
||||||
|
|
||||||
|
|
|
@ -35,12 +35,16 @@ from shipyard_airflow.errors import ApiError, AppError
|
||||||
CONF = cfg.CONF
|
CONF = cfg.CONF
|
||||||
LOG = logging.getLogger(__name__)
|
LOG = logging.getLogger(__name__)
|
||||||
|
|
||||||
# keys for the revision dict, and consistency of strings for committed
|
# keys for the revision dict, and consistency of strings for committed,
|
||||||
# and buffer.
|
# buffer, site-action-success and site-action-failure.
|
||||||
COMMITTED = 'committed'
|
|
||||||
BUFFER = 'buffer'
|
BUFFER = 'buffer'
|
||||||
|
COMMITTED = 'committed'
|
||||||
|
LAST_SITE_ACTION = 'last_site_action'
|
||||||
LATEST = 'latest'
|
LATEST = 'latest'
|
||||||
REVISION_COUNT = 'count'
|
REVISION_COUNT = 'count'
|
||||||
|
SITE_ACTION_SUCCESS = 'site-action-success'
|
||||||
|
SITE_ACTION_FAILURE = 'site-action-failure'
|
||||||
|
SUCCESSFUL_SITE_ACTION = 'successful_site_action'
|
||||||
|
|
||||||
# string for rollback_commit consistency
|
# string for rollback_commit consistency
|
||||||
ROLLBACK_COMMIT = 'rollback_commit'
|
ROLLBACK_COMMIT = 'rollback_commit'
|
||||||
|
@ -96,7 +100,7 @@ class ConfigdocsHelper(object):
|
||||||
|
|
||||||
def is_buffer_empty(self):
|
def is_buffer_empty(self):
|
||||||
""" Check if the buffer is empty. """
|
""" Check if the buffer is empty. """
|
||||||
return self._get_buffer_revision() is None
|
return self._get_revision(BUFFER) is None
|
||||||
|
|
||||||
def is_collection_in_buffer(self, collection_id):
|
def is_collection_in_buffer(self, collection_id):
|
||||||
"""
|
"""
|
||||||
|
@ -107,12 +111,12 @@ class ConfigdocsHelper(object):
|
||||||
|
|
||||||
# If there is no committed revision, then it's 0.
|
# If there is no committed revision, then it's 0.
|
||||||
# new revision is ok because we just checked for buffer emptiness
|
# new revision is ok because we just checked for buffer emptiness
|
||||||
old_revision_id = self._get_committed_rev_id() or 0
|
old_revision_id = self._get_revision_id(COMMITTED) or 0
|
||||||
|
|
||||||
try:
|
try:
|
||||||
diff = self.deckhand.get_diff(
|
diff = self.deckhand.get_diff(
|
||||||
old_revision_id=old_revision_id,
|
old_revision_id=old_revision_id,
|
||||||
new_revision_id=self._get_buffer_rev_id())
|
new_revision_id=self._get_revision_id(BUFFER))
|
||||||
# the collection is in the buffer if it's not unmodified
|
# the collection is in the buffer if it's not unmodified
|
||||||
return diff.get(collection_id, 'unmodified') != 'unmodified'
|
return diff.get(collection_id, 'unmodified') != 'unmodified'
|
||||||
|
|
||||||
|
@ -142,7 +146,7 @@ class ConfigdocsHelper(object):
|
||||||
# replace the buffer with last commit.
|
# replace the buffer with last commit.
|
||||||
elif buffermode == BufferMode.REPLACE:
|
elif buffermode == BufferMode.REPLACE:
|
||||||
committed_rev_id = None
|
committed_rev_id = None
|
||||||
committed_rev = self._get_committed_revision()
|
committed_rev = self._get_revision(COMMITTED)
|
||||||
if committed_rev:
|
if committed_rev:
|
||||||
committed_rev_id = committed_rev['id']
|
committed_rev_id = committed_rev['id']
|
||||||
if committed_rev_id is None:
|
if committed_rev_id is None:
|
||||||
|
@ -165,8 +169,8 @@ class ConfigdocsHelper(object):
|
||||||
|
|
||||||
# If there is no committed revision, then it's 0.
|
# If there is no committed revision, then it's 0.
|
||||||
# new revision is ok because we just checked for buffer emptiness
|
# new revision is ok because we just checked for buffer emptiness
|
||||||
old_revision_id = self._get_committed_rev_id() or 0
|
old_revision_id = self._get_revision_id(COMMITTED) or 0
|
||||||
new_revision_id = self._get_buffer_rev_id() or old_revision_id
|
new_revision_id = self._get_revision_id(BUFFER) or old_revision_id
|
||||||
|
|
||||||
try:
|
try:
|
||||||
diff = self.deckhand.get_diff(
|
diff = self.deckhand.get_diff(
|
||||||
|
@ -209,12 +213,17 @@ class ConfigdocsHelper(object):
|
||||||
def _get_revision_dict(self):
|
def _get_revision_dict(self):
|
||||||
"""
|
"""
|
||||||
Returns a dictionary with values representing the revisions in
|
Returns a dictionary with values representing the revisions in
|
||||||
Deckhand that Shipyard cares about - committed, buffer,
|
Deckhand that Shipyard cares about - committed, buffer, latest,
|
||||||
and latest, as well as a count of revisions
|
last_site_action and successful_site_action, as well as a count
|
||||||
|
of revisions.
|
||||||
Committed and buffer are revisions associated with the
|
Committed and buffer are revisions associated with the
|
||||||
shipyard tags. If either of those are not present in deckhand,
|
shipyard tags. If either of those are not present in deckhand,
|
||||||
returns None for the value.
|
returns None for the value.
|
||||||
Latest holds the revision information for the newest revision.
|
Latest holds the revision information for the newest revision.
|
||||||
|
Last site action holds the revision information for the most
|
||||||
|
recent site action
|
||||||
|
Successful site action holds the revision information for the
|
||||||
|
most recent successfully executed site action.
|
||||||
"""
|
"""
|
||||||
# return the cached instance version of the revision dict.
|
# return the cached instance version of the revision dict.
|
||||||
if self.revision_dict is not None:
|
if self.revision_dict is not None:
|
||||||
|
@ -222,28 +231,49 @@ class ConfigdocsHelper(object):
|
||||||
# or generate one for the cache
|
# or generate one for the cache
|
||||||
committed_revision = None
|
committed_revision = None
|
||||||
buffer_revision = None
|
buffer_revision = None
|
||||||
|
last_site_action = None
|
||||||
latest_revision = None
|
latest_revision = None
|
||||||
revision_count = 0
|
revision_count = 0
|
||||||
|
successful_site_action = None
|
||||||
try:
|
try:
|
||||||
revisions = self.deckhand.get_revision_list()
|
revisions = self.deckhand.get_revision_list()
|
||||||
revision_count = len(revisions)
|
revision_count = len(revisions)
|
||||||
if revisions:
|
if revisions:
|
||||||
|
# Retrieve latest revision
|
||||||
latest_revision = revisions[-1]
|
latest_revision = revisions[-1]
|
||||||
|
|
||||||
|
# Get required revision
|
||||||
for revision in reversed(revisions):
|
for revision in reversed(revisions):
|
||||||
tags = revision.get('tags', [])
|
tags = revision.get('tags', [])
|
||||||
if COMMITTED in tags or ROLLBACK_COMMIT in tags:
|
|
||||||
|
if (committed_revision is None and (
|
||||||
|
COMMITTED in tags or
|
||||||
|
ROLLBACK_COMMIT in tags)):
|
||||||
committed_revision = revision
|
committed_revision = revision
|
||||||
break
|
|
||||||
else:
|
else:
|
||||||
# there are buffer revisions, only grab it on
|
# there are buffer revisions, only grab it on
|
||||||
# the first pass through
|
# the first pass through
|
||||||
# if the first revision is committed, or if there
|
# if the first revision is committed, or if there
|
||||||
# are no revsisions, buffer revsision stays None
|
# are no revsisions, buffer revsision stays None
|
||||||
if buffer_revision is None:
|
if (committed_revision is None and
|
||||||
|
buffer_revision is None):
|
||||||
buffer_revision = revision
|
buffer_revision = revision
|
||||||
|
|
||||||
|
# Get the revision of the last successful site action
|
||||||
|
if (successful_site_action is None and
|
||||||
|
SITE_ACTION_SUCCESS in tags):
|
||||||
|
successful_site_action = revision
|
||||||
|
|
||||||
|
# Get the revision of the last site action
|
||||||
|
if (last_site_action is None and (
|
||||||
|
SITE_ACTION_SUCCESS in tags or
|
||||||
|
SITE_ACTION_FAILURE in tags)):
|
||||||
|
last_site_action = revision
|
||||||
|
|
||||||
except NoRevisionsExistError:
|
except NoRevisionsExistError:
|
||||||
# the values of None/None/None/0 are fine
|
# the values of None/None/None/0 are fine
|
||||||
pass
|
pass
|
||||||
|
|
||||||
except DeckhandResponseError as drex:
|
except DeckhandResponseError as drex:
|
||||||
raise AppError(
|
raise AppError(
|
||||||
title='Unable to retrieve revisions',
|
title='Unable to retrieve revisions',
|
||||||
|
@ -253,50 +283,32 @@ class ConfigdocsHelper(object):
|
||||||
status=falcon.HTTP_500,
|
status=falcon.HTTP_500,
|
||||||
retry=False)
|
retry=False)
|
||||||
self.revision_dict = {
|
self.revision_dict = {
|
||||||
COMMITTED: committed_revision,
|
|
||||||
BUFFER: buffer_revision,
|
BUFFER: buffer_revision,
|
||||||
|
COMMITTED: committed_revision,
|
||||||
|
LAST_SITE_ACTION: last_site_action,
|
||||||
LATEST: latest_revision,
|
LATEST: latest_revision,
|
||||||
REVISION_COUNT: revision_count
|
REVISION_COUNT: revision_count,
|
||||||
|
SUCCESSFUL_SITE_ACTION: successful_site_action
|
||||||
}
|
}
|
||||||
return self.revision_dict
|
return self.revision_dict
|
||||||
|
|
||||||
def _get_buffer_revision(self):
|
def _get_revision(self, target_revision):
|
||||||
# convenience helper to drill down to Buffer revision
|
# Helper to drill down to the target revision
|
||||||
return self._get_revision_dict().get(BUFFER)
|
return self._get_revision_dict().get(target_revision)
|
||||||
|
|
||||||
def _get_buffer_rev_id(self):
|
def _get_revision_id(self, target_revision):
|
||||||
# convenience helper to drill down to Buffer revision id
|
rev = self._get_revision_dict().get(target_revision)
|
||||||
buf_rev = self._get_revision_dict().get(BUFFER)
|
return rev['id'] if rev else None
|
||||||
return buf_rev['id'] if buf_rev else None
|
|
||||||
|
|
||||||
def _get_latest_revision(self):
|
|
||||||
# convenience helper to drill down to latest revision
|
|
||||||
return self._get_revision_dict().get(LATEST)
|
|
||||||
|
|
||||||
def _get_latest_rev_id(self):
|
|
||||||
# convenience helper to drill down to latest revision id
|
|
||||||
latest_rev = self._get_revision_dict().get(LATEST)
|
|
||||||
return latest_rev['id'] if latest_rev else None
|
|
||||||
|
|
||||||
def _get_committed_revision(self):
|
|
||||||
# convenience helper to drill down to committed revision
|
|
||||||
return self._get_revision_dict().get(COMMITTED)
|
|
||||||
|
|
||||||
def _get_committed_rev_id(self):
|
|
||||||
# convenience helper to drill down to committed revision id
|
|
||||||
committed_rev = self._get_revision_dict().get(COMMITTED)
|
|
||||||
return committed_rev['id'] if committed_rev else None
|
|
||||||
|
|
||||||
def get_collection_docs(self, version, collection_id):
|
def get_collection_docs(self, version, collection_id):
|
||||||
"""
|
"""
|
||||||
Returns the requested collection of docs based on the version
|
Returns the requested collection of docs based on the version
|
||||||
specifier. Since the default is the buffer, only return
|
specifier. The default is set as buffer.
|
||||||
committed if explicitly stated. No need to further check the
|
|
||||||
parameter for validity here.
|
|
||||||
"""
|
"""
|
||||||
LOG.info('Retrieving collection %s from %s', collection_id, version)
|
LOG.info('Retrieving collection %s from %s', collection_id, version)
|
||||||
if version == COMMITTED:
|
if version in [COMMITTED, LAST_SITE_ACTION, SUCCESSFUL_SITE_ACTION]:
|
||||||
return self._get_committed_docs(collection_id)
|
return self._get_target_docs(collection_id, version)
|
||||||
|
|
||||||
return self._get_doc_from_buffer(collection_id)
|
return self._get_doc_from_buffer(collection_id)
|
||||||
|
|
||||||
def _get_doc_from_buffer(self, collection_id):
|
def _get_doc_from_buffer(self, collection_id):
|
||||||
|
@ -311,7 +323,7 @@ class ConfigdocsHelper(object):
|
||||||
if self.is_collection_in_buffer(collection_id):
|
if self.is_collection_in_buffer(collection_id):
|
||||||
# prior check for collection in buffer means the buffer
|
# prior check for collection in buffer means the buffer
|
||||||
# revision exists
|
# revision exists
|
||||||
buffer_id = self._get_buffer_rev_id()
|
buffer_id = self._get_revision_id(BUFFER)
|
||||||
return self.deckhand.get_docs_from_revision(
|
return self.deckhand.get_docs_from_revision(
|
||||||
revision_id=buffer_id, bucket_id=collection_id)
|
revision_id=buffer_id, bucket_id=collection_id)
|
||||||
raise ApiError(
|
raise ApiError(
|
||||||
|
@ -321,51 +333,66 @@ class ConfigdocsHelper(object):
|
||||||
status=falcon.HTTP_404,
|
status=falcon.HTTP_404,
|
||||||
retry=False)
|
retry=False)
|
||||||
|
|
||||||
def _get_committed_docs(self, collection_id):
|
def _get_target_docs(self, collection_id, target_rev):
|
||||||
"""
|
"""
|
||||||
Returns the collection if it exists as committed.
|
Returns the collection if it exists as committed, last_site_action
|
||||||
|
or successful_site_action.
|
||||||
"""
|
"""
|
||||||
committed_id = self._get_committed_rev_id()
|
revision_id = self._get_revision_id(target_rev)
|
||||||
if committed_id:
|
|
||||||
|
if revision_id:
|
||||||
return self.deckhand.get_docs_from_revision(
|
return self.deckhand.get_docs_from_revision(
|
||||||
revision_id=committed_id, bucket_id=collection_id)
|
revision_id=revision_id, bucket_id=collection_id)
|
||||||
# if there is no committed...
|
|
||||||
raise ApiError(
|
raise ApiError(
|
||||||
title='No documents to retrieve',
|
title='No documents to retrieve',
|
||||||
description='There is no committed version of this collection',
|
description=('No collection {} for revision '
|
||||||
|
'{}'.format(collection_id, target_rev)),
|
||||||
status=falcon.HTTP_404,
|
status=falcon.HTTP_404,
|
||||||
retry=False)
|
retry=False)
|
||||||
|
|
||||||
def get_rendered_configdocs(self, version=BUFFER):
|
def get_rendered_configdocs(self, version=BUFFER):
|
||||||
"""
|
"""
|
||||||
Returns the rendered configuration documents for the specified
|
Returns the rendered configuration documents for the specified
|
||||||
revision (by name BUFFER, COMMITTED)
|
revision (by name BUFFER, COMMITTED, LAST_SITE_ACTION,
|
||||||
|
SUCCESSFUL_SITE_ACTION)
|
||||||
"""
|
"""
|
||||||
revision_dict = self._get_revision_dict()
|
revision_dict = self._get_revision_dict()
|
||||||
if version in (BUFFER, COMMITTED):
|
|
||||||
if revision_dict.get(version):
|
# Raise Exceptions if we received unexpected version
|
||||||
revision_id = revision_dict.get(version).get('id')
|
if version not in [BUFFER, COMMITTED, LAST_SITE_ACTION,
|
||||||
try:
|
SUCCESSFUL_SITE_ACTION]:
|
||||||
return self.deckhand.get_rendered_docs_from_revision(
|
raise ApiError(
|
||||||
revision_id=revision_id)
|
title='Invalid version',
|
||||||
except DeckhandError as de:
|
description='{} is not a valid version'.format(version),
|
||||||
raise ApiError(
|
status=falcon.HTTP_400,
|
||||||
title='Deckhand indicated an error while rendering',
|
retry=False)
|
||||||
description=de.response_message,
|
|
||||||
status=falcon.HTTP_500,
|
if revision_dict.get(version):
|
||||||
retry=False)
|
revision_id = revision_dict.get(version).get('id')
|
||||||
else:
|
|
||||||
|
try:
|
||||||
|
return self.deckhand.get_rendered_docs_from_revision(
|
||||||
|
revision_id=revision_id)
|
||||||
|
except DeckhandError as de:
|
||||||
raise ApiError(
|
raise ApiError(
|
||||||
title='This revision does not exist',
|
title='Deckhand indicated an error while rendering',
|
||||||
description='{} version does not exist'.format(version),
|
description=de.response_message,
|
||||||
status=falcon.HTTP_404,
|
status=falcon.HTTP_500,
|
||||||
retry=False)
|
retry=False)
|
||||||
|
|
||||||
|
else:
|
||||||
|
raise ApiError(
|
||||||
|
title='This revision does not exist',
|
||||||
|
description='{} version does not exist'.format(version),
|
||||||
|
status=falcon.HTTP_404,
|
||||||
|
retry=False)
|
||||||
|
|
||||||
def get_validations_for_buffer(self):
|
def get_validations_for_buffer(self):
|
||||||
"""
|
"""
|
||||||
Convenience method to do validations for buffer version.
|
Convenience method to do validations for buffer version.
|
||||||
"""
|
"""
|
||||||
buffer_rev_id = self._get_buffer_rev_id()
|
buffer_rev_id = self._get_revision_id(BUFFER)
|
||||||
if buffer_rev_id:
|
if buffer_rev_id:
|
||||||
return self.get_validations_for_revision(buffer_rev_id)
|
return self.get_validations_for_revision(buffer_rev_id)
|
||||||
raise AppError(
|
raise AppError(
|
||||||
|
@ -682,7 +709,7 @@ class ConfigdocsHelper(object):
|
||||||
"""
|
"""
|
||||||
Convenience method to tag the buffer version.
|
Convenience method to tag the buffer version.
|
||||||
"""
|
"""
|
||||||
buffer_rev_id = self._get_buffer_rev_id()
|
buffer_rev_id = self._get_revision_id(BUFFER)
|
||||||
if buffer_rev_id is None:
|
if buffer_rev_id is None:
|
||||||
raise AppError(
|
raise AppError(
|
||||||
title='Unable to tag buffer as {}'.format(tag),
|
title='Unable to tag buffer as {}'.format(tag),
|
||||||
|
@ -722,7 +749,7 @@ class ConfigdocsHelper(object):
|
||||||
description=drie.response_message)
|
description=drie.response_message)
|
||||||
# reset the revision dict so it regenerates.
|
# reset the revision dict so it regenerates.
|
||||||
self.revision_dict = None
|
self.revision_dict = None
|
||||||
return self._get_buffer_rev_id()
|
return self._get_revision_id(BUFFER)
|
||||||
|
|
||||||
def check_intermediate_commit(self):
|
def check_intermediate_commit(self):
|
||||||
|
|
||||||
|
|
|
@ -25,7 +25,10 @@ from shipyard_airflow.control.configdocs.configdocs_helper import \
|
||||||
from shipyard_airflow.errors import ApiError
|
from shipyard_airflow.errors import ApiError
|
||||||
|
|
||||||
CONF = cfg.CONF
|
CONF = cfg.CONF
|
||||||
VERSION_VALUES = ['buffer', 'committed']
|
VERSION_VALUES = ['buffer',
|
||||||
|
'committed',
|
||||||
|
'last_site_action',
|
||||||
|
'successful_site_action']
|
||||||
|
|
||||||
|
|
||||||
class RenderedConfigDocsResource(BaseResource):
|
class RenderedConfigDocsResource(BaseResource):
|
||||||
|
|
|
@ -320,7 +320,7 @@ def test_get_configdocs_status():
|
||||||
|
|
||||||
def test__get_revision_dict_no_commit():
|
def test__get_revision_dict_no_commit():
|
||||||
"""
|
"""
|
||||||
Tests the processing of revision dict response from dechand
|
Tests the processing of revision dict response from deckhand
|
||||||
with a buffer version, but no committed revision
|
with a buffer version, but no committed revision
|
||||||
"""
|
"""
|
||||||
helper = ConfigdocsHelper(CTX)
|
helper = ConfigdocsHelper(CTX)
|
||||||
|
@ -357,7 +357,7 @@ def test__get_revision_dict_no_commit():
|
||||||
|
|
||||||
def test__get_revision_dict_empty():
|
def test__get_revision_dict_empty():
|
||||||
"""
|
"""
|
||||||
Tests the processing of revision dict response from dechand
|
Tests the processing of revision dict response from deckhand
|
||||||
where the response is an empty list
|
where the response is an empty list
|
||||||
"""
|
"""
|
||||||
helper = ConfigdocsHelper(CTX)
|
helper = ConfigdocsHelper(CTX)
|
||||||
|
@ -375,7 +375,7 @@ def test__get_revision_dict_empty():
|
||||||
|
|
||||||
def test__get_revision_dict_commit_no_buff():
|
def test__get_revision_dict_commit_no_buff():
|
||||||
"""
|
"""
|
||||||
Tests the processing of revision dict response from dechand
|
Tests the processing of revision dict response from deckhand
|
||||||
with a committed and no buffer revision
|
with a committed and no buffer revision
|
||||||
"""
|
"""
|
||||||
helper = ConfigdocsHelper(CTX)
|
helper = ConfigdocsHelper(CTX)
|
||||||
|
@ -412,7 +412,7 @@ def test__get_revision_dict_commit_no_buff():
|
||||||
|
|
||||||
def test__get_revision_dict_commit_and_buff():
|
def test__get_revision_dict_commit_and_buff():
|
||||||
"""
|
"""
|
||||||
Tests the processing of revision dict response from dechand
|
Tests the processing of revision dict response from deckhand
|
||||||
with a committed and a buffer revision
|
with a committed and a buffer revision
|
||||||
"""
|
"""
|
||||||
helper = ConfigdocsHelper(CTX)
|
helper = ConfigdocsHelper(CTX)
|
||||||
|
@ -850,3 +850,37 @@ def test_add_collection():
|
||||||
assert helper.add_collection('mop', 'yaml:yaml') == 5
|
assert helper.add_collection('mop', 'yaml:yaml') == 5
|
||||||
|
|
||||||
mock_method.assert_called_once_with('mop', 'yaml:yaml')
|
mock_method.assert_called_once_with('mop', 'yaml:yaml')
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_revision_dict_last_site_action_and_successful_site_action():
|
||||||
|
"""
|
||||||
|
Tests the processing of revision dict response from deckhand
|
||||||
|
for last_site_action and successful_site_action revision
|
||||||
|
"""
|
||||||
|
helper = ConfigdocsHelper(CTX)
|
||||||
|
helper.deckhand.get_revision_list = lambda: yaml.load("""
|
||||||
|
---
|
||||||
|
- id: 1
|
||||||
|
url: https://deckhand/api/v1.0/revisions/1
|
||||||
|
createdAt: 2018-04-30T21:23Z
|
||||||
|
buckets: [mop]
|
||||||
|
tags: [committed, site-action-success]
|
||||||
|
validationPolicies:
|
||||||
|
site-deploy-validation:
|
||||||
|
status: succeeded
|
||||||
|
- id: 2
|
||||||
|
url: https://deckhand/api/v1.0/revisions/2
|
||||||
|
createdAt: 2018-04-30T23:35Z
|
||||||
|
buckets: [flop, mop]
|
||||||
|
tags: [committed, site-action-failure]
|
||||||
|
validationPolicies:
|
||||||
|
site-deploy-validation:
|
||||||
|
status: succeeded
|
||||||
|
...
|
||||||
|
""")
|
||||||
|
rev_dict = helper._get_revision_dict()
|
||||||
|
successful_site_action = rev_dict.get(
|
||||||
|
configdocs_helper.SUCCESSFUL_SITE_ACTION)
|
||||||
|
last_site_action = rev_dict.get(configdocs_helper.LAST_SITE_ACTION)
|
||||||
|
assert successful_site_action.get('id') == 1
|
||||||
|
assert last_site_action.get('id') == 2
|
||||||
|
|
|
@ -25,7 +25,7 @@ from shipyard_airflow.errors import ApiError
|
||||||
CTX = ShipyardRequestContext()
|
CTX = ShipyardRequestContext()
|
||||||
|
|
||||||
|
|
||||||
def test__validate_version_parameter():
|
def test_validate_version_parameter():
|
||||||
"""
|
"""
|
||||||
test of the version parameter validation
|
test of the version parameter validation
|
||||||
"""
|
"""
|
||||||
|
@ -53,3 +53,35 @@ def test_get_rendered_configdocs():
|
||||||
rcdr.get_rendered_configdocs(helper, version='buffer')
|
rcdr.get_rendered_configdocs(helper, version='buffer')
|
||||||
|
|
||||||
mock_method.assert_called_once_with('buffer')
|
mock_method.assert_called_once_with('buffer')
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_rendered_last_site_action_configdocs():
|
||||||
|
"""
|
||||||
|
Tests the RenderedConfigDocsResource method get_rendered_configdocs
|
||||||
|
for last_site_action tag
|
||||||
|
"""
|
||||||
|
rcdr = RenderedConfigDocsResource()
|
||||||
|
|
||||||
|
with patch.object(
|
||||||
|
ConfigdocsHelper, 'get_rendered_configdocs'
|
||||||
|
) as mock_method:
|
||||||
|
helper = ConfigdocsHelper(CTX)
|
||||||
|
rcdr.get_rendered_configdocs(helper, version='last_site_action')
|
||||||
|
|
||||||
|
mock_method.assert_called_once_with('last_site_action')
|
||||||
|
|
||||||
|
|
||||||
|
def test_get_rendered_successful_site_action_configdocs():
|
||||||
|
"""
|
||||||
|
Tests the RenderedConfigDocsResource method get_rendered_configdocs
|
||||||
|
for successful_site_action tag
|
||||||
|
"""
|
||||||
|
rcdr = RenderedConfigDocsResource()
|
||||||
|
|
||||||
|
with patch.object(
|
||||||
|
ConfigdocsHelper, 'get_rendered_configdocs'
|
||||||
|
) as mock_method:
|
||||||
|
helper = ConfigdocsHelper(CTX)
|
||||||
|
rcdr.get_rendered_configdocs(helper, version='successful_site_action')
|
||||||
|
|
||||||
|
mock_method.assert_called_once_with('successful_site_action')
|
||||||
|
|
|
@ -71,7 +71,8 @@ class ShipyardClient(BaseClient):
|
||||||
Get the collection of documents from deckhand specified by
|
Get the collection of documents from deckhand specified by
|
||||||
collection id
|
collection id
|
||||||
:param collection_id: String, bucket_id in deckhand
|
:param collection_id: String, bucket_id in deckhand
|
||||||
:param version: String, committed|buffer
|
:param version: String, buffer|committed|last_site_action|
|
||||||
|
successful_site_action
|
||||||
:rtype: Response object
|
:rtype: Response object
|
||||||
"""
|
"""
|
||||||
query_params = {"version": version}
|
query_params = {"version": version}
|
||||||
|
@ -90,7 +91,8 @@ class ShipyardClient(BaseClient):
|
||||||
|
|
||||||
def get_rendereddocs(self, version='buffer'):
|
def get_rendereddocs(self, version='buffer'):
|
||||||
"""
|
"""
|
||||||
:param str version: committed|buffer
|
:param str version: committed|buffer|last_site_action|
|
||||||
|
successful_site_action
|
||||||
:returns: full set of configdocs in their rendered form.
|
:returns: full set of configdocs in their rendered form.
|
||||||
:rtype: Response object
|
:rtype: Response object
|
||||||
"""
|
"""
|
||||||
|
|
|
@ -52,10 +52,11 @@ def get_actions(ctx):
|
||||||
|
|
||||||
|
|
||||||
DESC_CONFIGDOCS = """
|
DESC_CONFIGDOCS = """
|
||||||
COMMAND: configdocs \n
|
COMMAND: configdocs
|
||||||
DESCRIPTION: Retrieve documents loaded into Shipyard, either committed or
|
DESCRIPTION: Retrieve documents loaded into Shipyard, either committed or
|
||||||
from the Shipyard Buffer. \n
|
from the Shipyard Buffer.
|
||||||
FORMAT: shipyard get configdocs <collection> [--committed | --buffer] \n
|
FORMAT: shipyard get configdocs <collection>
|
||||||
|
[--committed | --buffer | --last-site-action | --successful-site-action]
|
||||||
EXAMPLE: shipyard get configdocs design
|
EXAMPLE: shipyard get configdocs design
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
@ -76,35 +77,42 @@ SHORT_DESC_CONFIGDOCS = ("Retrieve documents loaded into Shipyard, either "
|
||||||
'--buffer',
|
'--buffer',
|
||||||
'-b',
|
'-b',
|
||||||
flag_value='buffer',
|
flag_value='buffer',
|
||||||
help='Retrive the documents that have been loaded into Shipyard since the '
|
help='Retrieve the documents that have been loaded into Shipyard since '
|
||||||
'prior commit. If no documents have been loaded into the buffer for this '
|
'the prior commit. If no documents have been loaded into the buffer for '
|
||||||
'collection, this will return an empty response (default)')
|
'this collection, this will return an empty response (default)')
|
||||||
|
@click.option(
|
||||||
|
'--last-site-action',
|
||||||
|
'-l',
|
||||||
|
flag_value='last_site_action',
|
||||||
|
help='Holds the revision information for the most recent site action')
|
||||||
|
@click.option(
|
||||||
|
'--successful-site-action',
|
||||||
|
'-s',
|
||||||
|
flag_value='successful_site_action',
|
||||||
|
help='Holds the revision information for the most recent successfully '
|
||||||
|
'executed site action.')
|
||||||
@click.pass_context
|
@click.pass_context
|
||||||
def get_configdocs(ctx, collection, buffer, committed):
|
def get_configdocs(ctx, collection, buffer, committed, last_site_action,
|
||||||
|
successful_site_action):
|
||||||
if collection:
|
if collection:
|
||||||
if buffer and committed:
|
# Get version
|
||||||
ctx.fail(
|
version = get_version(ctx, buffer, committed, last_site_action,
|
||||||
'You must choose whether to retrive the committed OR from the '
|
successful_site_action)
|
||||||
'Shipyard Buffer with --committed or --buffer. ')
|
|
||||||
|
|
||||||
if committed:
|
|
||||||
version = 'committed'
|
|
||||||
|
|
||||||
else:
|
|
||||||
version = 'buffer'
|
|
||||||
|
|
||||||
click.echo(
|
click.echo(
|
||||||
GetConfigdocs(ctx, collection, version).invoke_and_return_resp())
|
GetConfigdocs(ctx, collection, version).invoke_and_return_resp())
|
||||||
|
|
||||||
else:
|
else:
|
||||||
click.echo(GetConfigdocsStatus(ctx).invoke_and_return_resp())
|
click.echo(GetConfigdocsStatus(ctx).invoke_and_return_resp())
|
||||||
|
|
||||||
|
|
||||||
DESC_RENDEREDCONFIGDOCS = """
|
DESC_RENDEREDCONFIGDOCS = """
|
||||||
COMMAND: renderedconfigdocs \n
|
COMMAND: renderedconfigdocs
|
||||||
DESCRIPTION: Retrieve the rendered version of documents loaded into
|
DESCRIPTION: Retrieve the rendered version of documents loaded into
|
||||||
Shipyard. Rendered documents are the "final" version of the documents after
|
Shipyard. Rendered documents are the "final" version of the documents after
|
||||||
applying Deckhand layering and substitution. \n
|
applying Deckhand layering and substitution.
|
||||||
FORMAT: shipyard get renderedconfigdocs [--committed | --buffer] \n
|
FORMAT: shipyard get renderedconfigdocs
|
||||||
|
[--committed | --buffer | --last-site-action | --successful-site-action]
|
||||||
EXAMPLE: shipyard get renderedconfigdocs
|
EXAMPLE: shipyard get renderedconfigdocs
|
||||||
"""
|
"""
|
||||||
|
|
||||||
|
@ -130,19 +138,23 @@ SHORT_DESC_RENDEREDCONFIGDOCS = (
|
||||||
flag_value='buffer',
|
flag_value='buffer',
|
||||||
help='Retrieve the documents that have been loaded into Shipyard since the'
|
help='Retrieve the documents that have been loaded into Shipyard since the'
|
||||||
' prior commit. (default)')
|
' prior commit. (default)')
|
||||||
|
@click.option(
|
||||||
|
'--last-site-action',
|
||||||
|
'-l',
|
||||||
|
flag_value='last_site_action',
|
||||||
|
help='Holds the revision information for the most recent site action')
|
||||||
|
@click.option(
|
||||||
|
'--successful-site-action',
|
||||||
|
'-s',
|
||||||
|
flag_value='successful_site_action',
|
||||||
|
help='Holds the revision information for the most recent successfully '
|
||||||
|
'executed site action.')
|
||||||
@click.pass_context
|
@click.pass_context
|
||||||
def get_renderedconfigdocs(ctx, buffer, committed):
|
def get_renderedconfigdocs(ctx, buffer, committed, last_site_action,
|
||||||
|
successful_site_action):
|
||||||
if buffer and committed:
|
# Get version
|
||||||
ctx.fail(
|
version = get_version(ctx, buffer, committed, last_site_action,
|
||||||
'You must choose whether to retrive the committed documents OR the'
|
successful_site_action)
|
||||||
' docutments in the Shipyard Buffer with --committed or --buffer.')
|
|
||||||
|
|
||||||
if committed:
|
|
||||||
version = 'committed'
|
|
||||||
|
|
||||||
else:
|
|
||||||
version = 'buffer'
|
|
||||||
|
|
||||||
click.echo(GetRenderedConfigdocs(ctx, version).invoke_and_return_resp())
|
click.echo(GetRenderedConfigdocs(ctx, version).invoke_and_return_resp())
|
||||||
|
|
||||||
|
@ -169,3 +181,37 @@ SHORT_DESC_WORKFLOWS = "Lists the workflows from airflow."
|
||||||
def get_workflows(ctx, since):
|
def get_workflows(ctx, since):
|
||||||
|
|
||||||
click.echo(GetWorkflows(ctx, since).invoke_and_return_resp())
|
click.echo(GetWorkflows(ctx, since).invoke_and_return_resp())
|
||||||
|
|
||||||
|
|
||||||
|
def get_version(ctx, buffer, committed, last_site_action,
|
||||||
|
successful_site_action):
|
||||||
|
|
||||||
|
# Check number of optional site parameters
|
||||||
|
# User can only query with 1 of these options
|
||||||
|
optional_site_parameters = []
|
||||||
|
|
||||||
|
if buffer:
|
||||||
|
optional_site_parameters.append('buffer')
|
||||||
|
if committed:
|
||||||
|
optional_site_parameters.append('committed')
|
||||||
|
if last_site_action:
|
||||||
|
optional_site_parameters.append('last_site_action')
|
||||||
|
if successful_site_action:
|
||||||
|
optional_site_parameters.append('successful_site_action')
|
||||||
|
|
||||||
|
if len(optional_site_parameters) > 1:
|
||||||
|
ctx.fail(
|
||||||
|
'You may only choose one of the following options:\n'
|
||||||
|
'--buffer for the documents in the Shipyard buffer\n'
|
||||||
|
'--committed for the last committed revision of the documents\n'
|
||||||
|
'--last-site-action for the documents associated with the last '
|
||||||
|
'successful or failed site action\n'
|
||||||
|
'--successful-site-action for the documents associated with the '
|
||||||
|
'last successful site action\n'
|
||||||
|
'Site actions include deploy_site and update_site.')
|
||||||
|
|
||||||
|
elif len(optional_site_parameters) == 1:
|
||||||
|
return optional_site_parameters[0]
|
||||||
|
|
||||||
|
else:
|
||||||
|
return 'buffer'
|
||||||
|
|
Loading…
Reference in New Issue