895 lines
26 KiB
ReStructuredText
895 lines
26 KiB
ReStructuredText
.. _shipyard_api:
|
||
|
||
Shipyard API
|
||
============
|
||
Logically, the API has three parts to handle the three areas of
|
||
functionality in Shipyard.
|
||
|
||
1. Document Staging
|
||
2. Action Handling
|
||
3. Airflow Monitoring
|
||
|
||
Standards used by the API
|
||
-------------------------
|
||
See `UCP API
|
||
conventions <https://github.com/att-comdev/ucp-integration/blob/master/docs>`__
|
||
|
||
Notes on examples
|
||
-----------------
|
||
Examples assume the following environment variables are set before
|
||
issuing the curl commands shown:
|
||
|
||
::
|
||
|
||
$TOKEN={a valid keystone token}
|
||
$URL={the url and port of the shipyard api}
|
||
|
||
- Examples will use json formatted by the jq command for sake of
|
||
presentation.
|
||
- Actual responses will not formatted.
|
||
- The use of ellipsis indicate a repeated structure in the case of
|
||
lists, or prior/subsequent structure unimportant to the example (or
|
||
considered understood).
|
||
- The content-length response headers have been removed so as to not
|
||
cause confusion with the listed output.
|
||
|
||
Example response for an invalid token:
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
::
|
||
|
||
HTTP/1.1 401 Unauthorized
|
||
content-type: application/json
|
||
x-shipyard-req: a8194b97-8973-4b04-a3b3-2bd319024c5d
|
||
WWW-Authenticate: Keystone uri='http://keystone-api.ucp.svc.cluster.local:80/v3'
|
||
|
||
{
|
||
"apiVersion": "v1.0",
|
||
"status": "Failure",
|
||
"metadata": {},
|
||
"message": "Unauthenticated",
|
||
"code": "401 Unauthorized",
|
||
"details": {
|
||
"errorList": [
|
||
{
|
||
"message": "Credentials are not established"
|
||
}
|
||
],
|
||
"errorCount": 1,
|
||
"errorType": "ApiError"
|
||
},
|
||
"kind": "status",
|
||
"reason": "Credentials are not established"
|
||
}
|
||
|
||
Document Staging API
|
||
--------------------
|
||
Shipyard will serve as the entrypoint for documents (designs, secrets,
|
||
configurations, etc...) into a site. Documents are posted to Shipyard in
|
||
collections, rather than individually. At any point in time, there will be two
|
||
represented versions of documents in a site that are accessible via this API:
|
||
|
||
- The "Committed Documents" version, which represents the last version of
|
||
documents that were successfully commited with a commit_configdocs action.
|
||
- The "Shipyard Buffer" version, which represents the collection of documents
|
||
that have been ingested by this API since the last commited version. Note
|
||
that only one set of documents maybe posted to the buffer at a time by
|
||
default. (This behavior can be overridden by query parameters issued by the
|
||
user of Shipyard)
|
||
|
||
All versions of documents rely upon Deckhand for storage. Shipyard uses the
|
||
tagging features of Deckhand of to find the appropriate Committed Documents
|
||
and Shipyard Buffer version.
|
||
|
||
/v1.0/configdocs/{collection_id}
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
Represents the site configuration documents
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
The documents as noted above (commonly yaml), in a format understood by
|
||
Deckhand
|
||
|
||
POST /v1.0/configdocs/{collection_id}
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Ingests a collection of documents. Synchronous. POSTing an empty body
|
||
indicates that the specified collection should be deleted when the
|
||
Shipyard Buffer is committed. If a POST to the commitconfigdocs is in
|
||
progress, this POST should be rejected with a 409 error.
|
||
|
||
.. note::
|
||
|
||
The expected input type for this request is ‘Content-Type: application/x-yaml’
|
||
|
||
|
||
Query Parameters
|
||
''''''''''''''''
|
||
|
||
- bufferMode=append|replace\|\ **rejectOnContents**
|
||
Indicates how the existing Shipyard Buffer should be handled. By default,
|
||
Shipyard will reject the POST if contents already exist in the Shipyard
|
||
Buffer.
|
||
|
||
- append: Add the collection to the Shipyard Buffer, only if that
|
||
collection doesn’t already exist in the Shipyard Buffer. If the
|
||
collection is already present, the request will be rejected and a 409
|
||
Conflict will be returned.
|
||
- replace: Clear the Shipyard Buffer before adding the specified
|
||
collection.
|
||
|
||
Responses
|
||
'''''''''
|
||
201 Created
|
||
If the documents are successfully ingested, even with validation failures.
|
||
Response message includes:
|
||
|
||
- a list of validation results
|
||
- The response headers will include a Location indicating the GET
|
||
endpoint to retrieve the configDocs
|
||
|
||
409 Conflict
|
||
A condition in the system is blocking this document ingestion
|
||
|
||
- If a commitconfigdocs POST is in progress.
|
||
- If any collections exist in the Shipyard Buffer unless bufferMode=replace
|
||
or bufferMode=append.
|
||
- If bufferMode=append, and the collection being posted is already in the
|
||
Shipyard Buffer
|
||
|
||
GET /v1.0/configdocs/{collection_id}
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Returns the source documents for a collection of documents
|
||
|
||
.. note::
|
||
|
||
The output type for this request is ‘Content-Type: application/x-yaml’
|
||
|
||
Query Parameters
|
||
''''''''''''''''
|
||
version=committed | **buffer**
|
||
Return the documents for the version specified - buffer by default.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
If documents can be retrieved.
|
||
|
||
- If the response is 200 with an empty response body, this indicates
|
||
that the buffer version is attempting to ‘delete’ the collection
|
||
when it is committed. An empty response body will only be possible
|
||
for version=buffer.
|
||
|
||
404 Not Found
|
||
If the collection is not represented
|
||
|
||
- When version=buffer, this indicates that no representations of this
|
||
collection have been POSTed since the last committed version.
|
||
- When version=committed, this indicates that either the collection has
|
||
never existed or has been deleted by a prior commit.
|
||
|
||
/v1.0/renderedconfigdocs
|
||
~~~~~~~~~~~~~~~~~~~~~~~~
|
||
Represents the site configuration documents, as a whole set - does not
|
||
consider collections in any way.
|
||
|
||
GET /v1.0/renderedconfigdocs
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Returns the full set of configdocs in their rendered form.
|
||
|
||
.. note::
|
||
|
||
The output type for this request is 'Content-Type: application/x-yaml'
|
||
|
||
Query Parameters
|
||
''''''''''''''''
|
||
version=committed|**buffer**
|
||
Return the documents for the version specified - buffer by default.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
If documents can be retrieved.
|
||
|
||
|
||
/v1.0/commitconfigdocs
|
||
~~~~~~~~~~~~~~~~~~~~~~
|
||
An RPC style command to trigger a commit of the configuration documents from
|
||
the Shipyard Buffer to the Committed Documents. This resource will support POST
|
||
only.
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
The response will be the list of validations from all downstream systems that
|
||
perform validation during the commit process. The structure will match the
|
||
error response object described in the `UCP API
|
||
conventions <https://github.com/att-comdev/ucp-integration/blob/master/docs>`__
|
||
and will be an aggregation of each UCP component’s responses.
|
||
|
||
POST /v1.0/commitconfigdocs
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Synchronous. Performs the commit of the Shipyard Buffer to the Committed
|
||
Documents. This invokes each of the UCP components to examine the Shipyard
|
||
Buffer version of the configuration documents and aggregate the responses.
|
||
While performing this commit, further POSTing of configdocs, or other commits
|
||
may not be invoked (Shipyard will block those requests with a 409 response). If
|
||
there are any failures to validate, the Shipyard Buffer and Commited Documents
|
||
will remain unchanged. If successful, the Shipyard Buffer will be cleared, and
|
||
the Committed documents will be updated.
|
||
|
||
.. note::
|
||
|
||
If there are unhandled runtime errors during the commitconfigdocs POST, a
|
||
deadlock situation may be possible. Future enhancements may improve this
|
||
handling.
|
||
|
||
Query Parameters
|
||
''''''''''''''''
|
||
force=true | **false**
|
||
By default, false, if there are validation failures the POST will
|
||
fail with a 400 response. With force=true, allows for the commit to
|
||
succeed (with a 200 response) even if there are validation failures
|
||
from downstream components. The aggregate response of validation
|
||
failures will be returned in this case, but the invalid documents
|
||
will still be moved from the Shipyard Buffer to the Committed
|
||
Documents.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
If the validations are successful. Returns an “empty” structure as as
|
||
response indicating no errors. A 200 may also be returned if there
|
||
are validation failures, but the force=true query parameter was
|
||
specified. In this case, the response will contain the list of
|
||
validations.
|
||
400 Bad Request
|
||
If the validations fail. Returns a populated response structure
|
||
containing the aggregation of the failed validations.
|
||
409 Conflict
|
||
If the there is a POST to commitconfigdocs in progress.
|
||
|
||
Example
|
||
'''''''
|
||
|
||
::
|
||
|
||
{
|
||
"apiVersion": "v1",
|
||
"code": "400 Bad Request",
|
||
"details": {
|
||
"errorCount": 2,
|
||
"messageList": [
|
||
{
|
||
"error": true,
|
||
"message": "Error loading effective site: 'NoneType' object is not iterable",
|
||
"name": "Drydock"
|
||
},
|
||
{
|
||
"error": true,
|
||
"message": "Armada unable to validate configdocs",
|
||
"name": "Armada"
|
||
}
|
||
]
|
||
},
|
||
"kind": "Status",
|
||
"message": "Validations failed",
|
||
"metadata": {},
|
||
"reason": "Validation",
|
||
"status": "Invalid"
|
||
}
|
||
|
||
Action API
|
||
----------
|
||
The Shipyard Action API is a resource that allows for creation, control and
|
||
investigation of triggered workflows. These actions encapsulate a command
|
||
interface for the Undercloud Platform. See :ref:`shipyard_action_commands` for
|
||
supported actions
|
||
|
||
/v1.0/actions
|
||
~~~~~~~~~~~~~
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
A list of actions that have been executed through shipyard's action API.
|
||
|
||
::
|
||
|
||
[
|
||
{ Action objects summarized, See below},
|
||
...
|
||
]
|
||
|
||
|
||
GET /v1.0/actions
|
||
^^^^^^^^^^^^^^^^^
|
||
Returns the list of actions in the system that have been posted, and are
|
||
accessible to the current user.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
If the actions can be retrieved.
|
||
|
||
Example
|
||
'''''''
|
||
|
||
::
|
||
|
||
$ curl -X GET $URL/api/v1.0/actions -H "X-Auth-Token:$TOKEN"
|
||
|
||
HTTP/1.1 200 OK
|
||
x-shipyard-req: 0804d13e-08fc-4e60-a819-3b7532cac4ec
|
||
content-type: application/json; charset=UTF-8
|
||
|
||
[
|
||
{
|
||
"dag_status": "failed",
|
||
"parameters": {},
|
||
"steps": [
|
||
{
|
||
"id": "action_xcom",
|
||
"url": "/actions/01BTP9T2WCE1PAJR2DWYXG805V/steps/action_xcom",
|
||
"index": 1,
|
||
"state": "success"
|
||
},
|
||
{
|
||
"id": "dag_concurrency_check",
|
||
"url": "/actions/01BTP9T2WCE1PAJR2DWYXG805V/steps/dag_concurrency_check",
|
||
"index": 2,
|
||
"state": "success"
|
||
},
|
||
{
|
||
"id": "preflight",
|
||
"url": "/actions/01BTP9T2WCE1PAJR2DWYXG805V/steps/preflight",
|
||
"index": 3,
|
||
"state": "failed"
|
||
},
|
||
...
|
||
],
|
||
"action_lifecycle": "Failed",
|
||
"dag_execution_date": "2017-09-23T02:42:12",
|
||
"id": "01BTP9T2WCE1PAJR2DWYXG805V",
|
||
"dag_id": "deploy_site",
|
||
"datetime": "2017-09-23 02:42:06.860597+00:00",
|
||
"user": "shipyard",
|
||
"context_marker": "416dec4b-82f9-4339-8886-3a0c4982aec3",
|
||
"name": "deploy_site"
|
||
},
|
||
...
|
||
]
|
||
|
||
POST /v1.0/actions
|
||
^^^^^^^^^^^^^^^^^^
|
||
Creates an action in the system. This will cause some action to start. The
|
||
input body to this post will represent an action object that has at least these
|
||
fields:
|
||
|
||
name
|
||
The name of the action to invoke, as noted in :ref:`shipyard_action_commands`
|
||
|
||
parameters
|
||
A dictionary of parameters to use for the trigger invocation. The supported
|
||
parameters will vary for the action invoked.
|
||
|
||
::
|
||
|
||
{
|
||
"name" : "action name",
|
||
"parameters" : { varies by action }
|
||
}
|
||
|
||
The POST will synchronously create the action (a shell object that represents
|
||
a DAG invocation), perform any checks to validate the preconditions to run the
|
||
DAG, and trigger the invocation of the DAG. The DAG will run asynchronously in
|
||
airflow.
|
||
|
||
Responses
|
||
'''''''''
|
||
201 Created
|
||
If the action is created successfully, and all preconditions to run the DAG
|
||
are successful. The response body is the action entity created.
|
||
400 Bad Request
|
||
If the action name doesn't exist, or the input entity is otherwise malformed.
|
||
409 Conflict
|
||
For any failed pre-run validations. The response body is the action entity
|
||
created, with the failed validations. The DAG will not begin execution in
|
||
this case.
|
||
|
||
Example
|
||
'''''''
|
||
|
||
::
|
||
|
||
$ curl -D - -d '{"name":"deploy_site"}' -X POST $URL/api/v1.0/actions \
|
||
-H "X-Auth-Token:$TOKEN" -H "content-type:application/json"
|
||
|
||
HTTP/1.1 201 Created
|
||
location: {$URL}/api/v1.0/actions/01BTTMFVDKZFRJM80FGD7J1AKN
|
||
x-shipyard-req: 629f2ea2-c59d-46b9-8641-7367a91a7016
|
||
content-type: application/json; charset=UTF-8
|
||
|
||
{
|
||
"dag_status": "SCHEDULED",
|
||
"parameters": {},
|
||
"dag_execution_date": "2017-09-24T19:05:49",
|
||
"id": "01BTTMFVDKZFRJM80FGD7J1AKN",
|
||
"dag_id": "deploy_site",
|
||
"name": "deploy_site",
|
||
"user": "shipyard",
|
||
"context_marker": "629f2ea2-c59d-46b9-8641-7367a91a7016",
|
||
"timestamp": "2017-09-24 19:05:43.603591"
|
||
}
|
||
|
||
/v1.0/actions/{action_id}
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
Each action will be assigned an unique id that can be used to get
|
||
details for the action, including the execution status.
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
All actions will include fields that indicate the following data:
|
||
|
||
action_lifecycle
|
||
A summarized value indicating the status or lifecycle phase of the action.
|
||
|
||
- Pending - The action is scheduled or preparing for execution.
|
||
- Processing - The action is underway.
|
||
- Complete - The action has completed successfully.
|
||
- Failed - The action has encountered an error, and has failed.
|
||
- Paused - The action has been paused by a user.
|
||
|
||
command audit
|
||
A list of commands that have been issued against the action. Initially,
|
||
the action listed will be “invoke”, but may include “pause”, “unpause”,
|
||
or “stop” if those commands are issued.
|
||
|
||
context_marker
|
||
The user supplied or system assigned context marker associated with the
|
||
action
|
||
|
||
dag_execution_date
|
||
The execution date assigned by the workflow system during action
|
||
creation.
|
||
|
||
dag_status
|
||
Represents the status that airflow provides for an executing DAG.
|
||
|
||
datetime
|
||
The time at which the action was invoked.
|
||
|
||
id
|
||
The identifier for the action, a 26 character ULID assigned during the
|
||
creation of the action.
|
||
|
||
name
|
||
The name of the action, e.g.: deploy_site.
|
||
|
||
parameters
|
||
The parameters configuring the action that were supplied by the user
|
||
during action creation.
|
||
|
||
steps
|
||
The list of steps for the action, including the status for that step.
|
||
|
||
user
|
||
The user who has invoked this action, as acquired from the authorization
|
||
token.
|
||
|
||
validations
|
||
A list of validations that have been done, including any status
|
||
information for those validations. During the lifecycle of the action,
|
||
this list of validations may continue to grow.
|
||
|
||
GET /v1.0/actions/{action_id}
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Returns the action entity for the specified id.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
|
||
Example
|
||
'''''''
|
||
|
||
::
|
||
|
||
$ curl -D - -X GET $URL/api/v1.0/actions/01BTTMFVDKZFRJM80FGD7J1AKN \
|
||
-H "X-Auth-Token:$TOKEN"
|
||
|
||
HTTP/1.1 200 OK
|
||
x-shipyard-req: eb3eacb3-4206-40df-bd91-2a3a6d81cd02
|
||
content-type: application/json; charset=UTF-8
|
||
|
||
{
|
||
"name": "deploy_site",
|
||
"dag_execution_date": "2017-09-24T19:05:49",
|
||
"validations": [],
|
||
"id": "01BTTMFVDKZFRJM80FGD7J1AKN",
|
||
"dag_id": "deploy_site",
|
||
"command_audit": [
|
||
{
|
||
"id": "01BTTMG16R9H3Z4JVQNBMRV1MZ",
|
||
"action_id": "01BTTMFVDKZFRJM80FGD7J1AKN",
|
||
"datetime": "2017-09-24 19:05:49.530223+00:00",
|
||
"user": "shipyard",
|
||
"command": "invoke"
|
||
}
|
||
],
|
||
"user": "shipyard",
|
||
"context_marker": "629f2ea2-c59d-46b9-8641-7367a91a7016",
|
||
"datetime": "2017-09-24 19:05:43.603591+00:00",
|
||
"dag_status": "failed",
|
||
"parameters": {},
|
||
"steps": [
|
||
{
|
||
"id": "action_xcom",
|
||
"url": "/actions/01BTTMFVDKZFRJM80FGD7J1AKN/steps/action_xcom",
|
||
"index": 1,
|
||
"state": "success"
|
||
},
|
||
{
|
||
"id": "dag_concurrency_check",
|
||
"url": "/actions/01BTTMFVDKZFRJM80FGD7J1AKN/steps/dag_concurrency_check",
|
||
"index": 2,
|
||
"state": "success"
|
||
},
|
||
{
|
||
"id": "preflight",
|
||
"url": "/actions/01BTTMFVDKZFRJM80FGD7J1AKN/steps/preflight",
|
||
"index": 3,
|
||
"state": "failed"
|
||
},
|
||
{
|
||
"id": "deckhand_get_design_version",
|
||
"url": "/actions/01BTTMFVDKZFRJM80FGD7J1AKN/steps/deckhand_get_design_version",
|
||
"index": 4,
|
||
"state": null
|
||
},
|
||
...
|
||
],
|
||
"action_lifecycle": "Failed"
|
||
}
|
||
|
||
/v1.0/actions/{action_id}/validationdetails/{validation_id}
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
Allows for drilldown to validation detailed info.
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
The detailed information for a validation
|
||
|
||
::
|
||
|
||
{ TBD }
|
||
|
||
GET /v1.0/actions/{action_id}/validationdetails/{validation_id}
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Returns the validation detail by Id for the supplied action Id.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
|
||
/v1.0/actions/{action_id}/steps/{step_id}
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
Allow for drilldown to step information. The step information includes
|
||
details of the steps excution, successful or not, and enough to
|
||
facilitate troubleshooting in as easy a fashion as possible.
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
A step entity represents detailed information representing a single step
|
||
of execution as part of an action. Not all fields are necessarily
|
||
represented in every returned entity.
|
||
|
||
dag_id
|
||
The name/id of the workflow DAG that contains this step.
|
||
|
||
duration
|
||
The duration (seconds) for the step.
|
||
|
||
end_date
|
||
The timestamp of the completion of the step.
|
||
|
||
execution_date
|
||
The execution date of the workflow that contains this step.
|
||
|
||
index
|
||
The numeric value representing the position of this step in the sequence
|
||
of steps associated with this step.
|
||
|
||
operator
|
||
The name of the processing facility used by the workflow system.
|
||
|
||
queued_dttm
|
||
The timestamp when the step was enqueued by the workflow system.
|
||
|
||
start_date
|
||
The timestamp for the beginning of execution for this step.
|
||
|
||
state
|
||
The execution state of the step.
|
||
|
||
task_id
|
||
The name of the task used by the workflow system (and also representing
|
||
this step name queried in the reqeust.
|
||
|
||
try_number
|
||
A number of retries taken in the case of failure. Some workflow steps
|
||
may be configured to retry before considering the step truly failed.
|
||
|
||
|
||
GET /v1.0/actions/{action_id}/steps/{step_id}
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Returns the details for a step by id for the given action by Id. #####
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
|
||
Example
|
||
'''''''
|
||
|
||
::
|
||
|
||
$ curl -D - \
|
||
-X GET $URL/api/v1.0/actions/01BTTMFVDKZFRJM80FGD7J1AKN/steps/action_xcom \
|
||
-H "X-Auth-Token:$TOKEN"
|
||
|
||
HTTP/1.1 200 OK
|
||
x-shipyard-req: 72daca4d-1f79-4e08-825f-2ad181912a47
|
||
content-type: application/json; charset=UTF-8
|
||
|
||
{
|
||
"end_date": "2017-09-24 19:05:59.446213",
|
||
"duration": 0.165181,
|
||
"queued_dttm": "2017-09-24 19:05:52.993983",
|
||
"operator": "PythonOperator",
|
||
"try_number": 1,
|
||
"task_id": "action_xcom",
|
||
"state": "success",
|
||
"execution_date": "2017-09-24 19:05:49",
|
||
"dag_id": "deploy_site",
|
||
"index": 1,
|
||
"start_date": "2017-09-24 19:05:59.281032"
|
||
}
|
||
|
||
/v1.0/actions/{action_id}/control/{control_verb}
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
Allows for issuing DAG controls against an action.
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
None, there is no associated response entity for this resource
|
||
|
||
POST /v1.0/actions/{action_id}/{control_verb}
|
||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Trigger a control action against an activity.- this includes: pause, unpause
|
||
|
||
Responses
|
||
'''''''''
|
||
202 Accepted
|
||
|
||
Example
|
||
'''''''
|
||
Failure case - command is invalid for the execution state of the action.
|
||
|
||
::
|
||
|
||
$ curl -D - \
|
||
-X POST $URL/api/v1.0/actions/01BTTMFVDKZFRJM80FGD7J1AKN/control/pause \
|
||
-H "X-Auth-Token:$TOKEN"
|
||
|
||
HTTP/1.1 409 Conflict
|
||
content-type: application/json
|
||
x-shipyard-req: 9c9551e0-335c-4297-af93-8440cc6b324f
|
||
|
||
{
|
||
"apiVersion": "v1.0",
|
||
"status": "Failure",
|
||
"metadata": {},
|
||
"message": "Unable to pause action",
|
||
"code": "409 Conflict",
|
||
"details": {
|
||
"errorList": [
|
||
{
|
||
"message": "dag_run state must be running, but is failed"
|
||
}
|
||
],
|
||
"errorCount": 1,
|
||
"errorType": "ApiError"
|
||
},
|
||
"kind": "status",
|
||
"reason": "dag_run state must be running, but is failed"
|
||
}
|
||
|
||
Success case
|
||
|
||
::
|
||
|
||
$ curl -D - \
|
||
-X POST $URL/api/v1.0/actions/01BTTMFVDKZFRJM80FGD7J1AKN/control/pause \
|
||
-H "X-Auth-Token:$TOKEN"
|
||
|
||
HTTP/1.1 202 Accepted
|
||
content-length: 0
|
||
x-shipyard-req: 019fae1c-03b0-4af1-b57d-451ae6ddac77
|
||
content-type: application/json; charset=UTF-8
|
||
|
||
|
||
Airflow Monitoring API
|
||
----------------------
|
||
Airflow has a primary function of scheduling DAGs, as opposed to Shipyard’s
|
||
primary case of triggering DAGs. Shipyard provides functionality to allow for
|
||
an operator to monitor and review these scheduled workflows (DAGs) in addition
|
||
to the ones triggered by Shipyard. This API will allow for accessing Airflow
|
||
DAGs of any type – providing a peek into the totality of what is happening in
|
||
Airflow.
|
||
|
||
/v1.0/workflows
|
||
~~~~~~~~~~~~~~~
|
||
The resource that represents DAGs (workflows) in airflow
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
A list of objects representing the DAGs that have run in airflow.
|
||
|
||
GET /v1.0/workflows
|
||
^^^^^^^^^^^^^^^^^^^
|
||
Queries airflow for DAGs that are running or have run (successfully or
|
||
unsuccessfully) and provides a summary of those things.
|
||
|
||
Query parameters
|
||
''''''''''''''''
|
||
since={iso8601 date (past) or duration}
|
||
optional, a boundary in the past within which to retrieve results. Default is
|
||
30 days in the past.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
|
||
Example
|
||
'''''''
|
||
Notice the workflow_id values, these can be used for drilldown.
|
||
|
||
::
|
||
|
||
curl -D - -X GET $URL/api/v1.0/workflows -H "X-Auth-Token:$TOKEN"
|
||
|
||
HTTP/1.1 200 OK
|
||
content-type: application/json; charset=UTF-8
|
||
x-shipyard-req: 3ab4ccc6-b956-4c7a-9ae6-183c562d8297
|
||
|
||
[
|
||
{
|
||
"execution_date": "2017-10-09 21:18:56",
|
||
"end_date": null,
|
||
"workflow_id": "deploy_site__2017-10-09T21:18:56.000000",
|
||
"start_date": "2017-10-09 21:18:56.685999",
|
||
"external_trigger": true,
|
||
"dag_id": "deploy_site",
|
||
"state": "failed",
|
||
"run_id": "manual__2017-10-09T21:18:56"
|
||
},
|
||
{
|
||
"execution_date": "2017-10-09 21:19:03",
|
||
"end_date": null,
|
||
"workflow_id": "deploy_site__2017-10-09T21:19:03.000000",
|
||
"start_date": "2017-10-09 21:19:03.361522",
|
||
"external_trigger": true,
|
||
"dag_id": "deploy_site",
|
||
"state": "failed",
|
||
"run_id": "manual__2017-10-09T21:19:03"
|
||
}
|
||
...
|
||
]
|
||
|
||
/v1.0/workflows/{workflow_id}
|
||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||
|
||
Entity Structure
|
||
^^^^^^^^^^^^^^^^
|
||
An object representing the information available from airflow regarding
|
||
a DAG’s execution
|
||
|
||
GET /v1.0/workflows/{id}
|
||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||
Further details of a particular workflow’s steps. All steps of all
|
||
sub-dags will be included in the list of steps, as well as section
|
||
indicating the sub-dags for this parent workflow.
|
||
|
||
Responses
|
||
'''''''''
|
||
200 OK
|
||
|
||
Example
|
||
'''''''
|
||
.. note::
|
||
|
||
Sub_dags can be queried to restrict to only that sub-dag’s steps. e.g. using
|
||
this as {workflow_id}:
|
||
deploy_site.preflight.armada_preflight_check__2017-10-09T21:19:03.000000
|
||
|
||
::
|
||
|
||
curl -D - \
|
||
-X GET $URL/api/v1.0/workflows/deploy_site__2017-10-09T21:19:03.000000 \
|
||
-H "X-Auth-Token:$TOKEN"
|
||
|
||
HTTP/1.1 200 OK
|
||
content-type: application/json; charset=UTF-8
|
||
x-shipyard-req: 98d71530-816a-4692-9df2-68f22c057467
|
||
|
||
{
|
||
"execution_date": "2017-10-09 21:19:03",
|
||
"end_date": null,
|
||
"workflow_id": "deploy_site__2017-10-09T21:19:03.000000",
|
||
"start_date": "2017-10-09 21:19:03.361522",
|
||
"external_trigger": true,
|
||
"steps": [
|
||
{
|
||
"end_date": "2017-10-09 21:19:14.916220",
|
||
"task_id": "action_xcom",
|
||
"start_date": "2017-10-09 21:19:14.798053",
|
||
"duration": 0.118167,
|
||
"queued_dttm": "2017-10-09 21:19:08.432582",
|
||
"try_number": 1,
|
||
"state": "success",
|
||
"operator": "PythonOperator",
|
||
"dag_id": "deploy_site",
|
||
"execution_date": "2017-10-09 21:19:03"
|
||
},
|
||
{
|
||
"end_date": "2017-10-09 21:19:25.283785",
|
||
"task_id": "dag_concurrency_check",
|
||
"start_date": "2017-10-09 21:19:25.181492",
|
||
"duration": 0.102293,
|
||
"queued_dttm": "2017-10-09 21:19:19.283132",
|
||
"try_number": 1,
|
||
"state": "success",
|
||
"operator": "ConcurrencyCheckOperator",
|
||
"dag_id": "deploy_site",
|
||
"execution_date": "2017-10-09 21:19:03"
|
||
},
|
||
{
|
||
"end_date": "2017-10-09 21:20:05.394677",
|
||
"task_id": "preflight",
|
||
"start_date": "2017-10-09 21:19:34.994775",
|
||
"duration": 30.399902,
|
||
"queued_dttm": "2017-10-09 21:19:28.449848",
|
||
"try_number": 1,
|
||
"state": "failed",
|
||
"operator": "SubDagOperator",
|
||
"dag_id": "deploy_site",
|
||
"execution_date": "2017-10-09 21:19:03"
|
||
},
|
||
...
|
||
],
|
||
"dag_id": "deploy_site",
|
||
"state": "failed",
|
||
"run_id": "manual__2017-10-09T21:19:03",
|
||
"sub_dags": [
|
||
{
|
||
"execution_date": "2017-10-09 21:19:03",
|
||
"end_date": null,
|
||
"workflow_id": "deploy_site.preflight__2017-10-09T21:19:03.000000",
|
||
"start_date": "2017-10-09 21:19:35.082479",
|
||
"external_trigger": false,
|
||
"dag_id": "deploy_site.preflight",
|
||
"state": "failed",
|
||
"run_id": "backfill_2017-10-09T21:19:03"
|
||
},
|
||
...,
|
||
{
|
||
"execution_date": "2017-10-09 21:19:03",
|
||
"end_date": null,
|
||
"workflow_id": "deploy_site.preflight.armada_preflight_check__2017-10-09T21:19:03.000000",
|
||
"start_date": "2017-10-09 21:19:48.265023",
|
||
"external_trigger": false,
|
||
"dag_id": "deploy_site.preflight.armada_preflight_check",
|
||
"state": "failed",
|
||
"run_id": "backfill_2017-10-09T21:19:03"
|
||
}
|
||
]
|
||
}
|