From 575e7acbb8874263ade536127af63d2e37e13964 Mon Sep 17 00:00:00 2001 From: Scott Hussey Date: Mon, 16 Oct 2017 20:37:31 -0500 Subject: [PATCH] Blueprint for Boot Actions This design is for the framework of deploying actions and assets to nodes being deployed by Drydock. Included is an API for nodes to report back the result status of any running boot actions. Change-Id: I2becd6af33d0202d0da73bab01f21b010fc3ea60 --- docs/source/API.rst | 75 +++++++++++++++++++++++++++ docs/source/bootaction.rst | 103 +++++++++++++++++++++++++++++++++++++ docs/source/index.rst | 9 ++++ docs/{ => source}/task.rst | 4 +- docs/source/topology.rst | 9 ++-- 5 files changed, 195 insertions(+), 5 deletions(-) create mode 100644 docs/source/API.rst create mode 100644 docs/source/bootaction.rst rename docs/{ => source}/task.rst (97%) diff --git a/docs/source/API.rst b/docs/source/API.rst new file mode 100644 index 00000000..b74d4164 --- /dev/null +++ b/docs/source/API.rst @@ -0,0 +1,75 @@ +.. _api: + +=========== +Drydock API +=========== + +The Drydock API is a RESTful interface used for accessing the services provided by Drydock. +All endpoints are located under ``/api//``. + +Secured endpoints require Keystone authentication and proper role assignment for authorization + +v1.0 +==== + +.. _tasks-api: + +tasks API +--------- + +The Tasks API is used for creating and listing asynchronous tasks to be executed by the +Drydock orchestrator. See :ref:`task` for details on creating tasks and field information. + +bootdata +-------- + +The boot data API is used by deploying nodes to load the appropriate boot actions to be +instantiated on the node. It uses alternative authentication and is not accessible with +Keystone. + +GET bootdata/hostname/files +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Returns a gzipped tar file containing all the file-type boot action data assets for +the node ``hostname`` with appropriate permissions set in the tar-file. + +GET bootdata/hostname/units +^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Returns a gzipped tar file containing all the unit-type boot action data assets for +the node ``hostname`` with appropriate permissions set in the tar-file. + +.. _bootaction-api: + +bootaction API +-------------- + +The boot action API is used by deploying nodes to report status and results of running +boot actions. It expects a JSON-formatted body with the top-level entity of an object. +The status of the boot action and any detail status messages for it will be added to the +DeployNode task that prompted the node deployment the boot action is associated with. + +POST bootaction/bootaction-id +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Example:: + + { + "status": "Failure"|"Success", + "details": [ + { + "message": "Boot action status message", + "error": true|false, + ... + }, + ... + ] + } + +POSTs to this endpoint can be made repeatedly omitting the ``status`` field and simply +adding one or more detail status messages. The ``message`` and ``error`` fields are required and +the ``context``, ``context_type`` and ``ts`` fields are reserved. Otherwise the message +object in details can be extended with additional fields as needed. + +Once a POST containing the ``status`` field is made to a bootaction-id, that bootaction-id can no +longer be updated with status changes nor additional detailed status messages. diff --git a/docs/source/bootaction.rst b/docs/source/bootaction.rst new file mode 100644 index 00000000..cec13c28 --- /dev/null +++ b/docs/source/bootaction.rst @@ -0,0 +1,103 @@ +.. _bootaction: + +============ +Boot Actions +============ + +Boot actions can be more accurately described as post-deployment file placement. This file placement +can be leveraged to install actions for servers to take after the permanent OS is installed +and the server is rebooted. Including custom or vendor scripts and a SystemD service to run the +scripts on first boot or on all boots allows almost any action to be configured. + +Boot Action Schema +================== + +Boot actions are configured via YAML documents included in the site topology definition. The schema +for these YAML documents is described below. + +.. code-block:: yaml + + data: + assets: + - path: /save/file/here + location: http://get.data.here/data + type: unit|file|pkg_list + data: | + inline data here + location_pipeline: + - template + data_pipeline + - base64_decode + - template + - base64_encode + permissions: 555 + node_filter: + ... + + +``assets`` is a list of data assets. More details below on how each data asset is rendered. + +``node_filter`` is an optional filter for selecting to which nodes this boot action will apply. +If no node filter is included, all nodes will receive the boot action. Otherwise it will be +only the nodes that match the logic of the filter set. See :ref:`task` for a definition of +the node filter. + +Rendering Data Assets +===================== + +The boot action framework supports assets of several types. ``type`` can be ``unit`` or ``file`` or ``pkg_list``. + + - ``unit`` is a SystemD unit, such as a service, that will be saved to ``path`` and enabled via ``systemctl enable [filename]``. + - ``file`` is simply saved to the filesystem at ``path`` and set with ``permissions``. + - ``pkg_list`` is a list of packages, one per line, that will be installed via apt. + +Data assets of type ``unit`` or ``file`` will be rendered and saved as files on disk and assigned +the ``permissions`` as sepcified. The rendering process can follow a few different paths. + +Referenced vs Inline Data +------------------------- + +The asset contents can be sourced from either the in-document ``data`` field of the asset +mapping or dynamically generated by requesting them from a URL provided in ``location``. +Currently Drydock supports the schemes of ``http``, ``deckhand+http`` and +``promenade+http`` for referenced data. + +Pipelines +--------- + +The boot action framework supports pipelines to allow for some dynamic rendering. There +are separate pipelines for the ``location`` field to build the URL that referenced assets should +be sourced from and the ``data`` field (or the data sourced from resolving the ``location`` field). + +The ``location`` string will be passed through the ``location_pipeline`` before it is queried. This response +or the ``data`` field will then be passed through the ``data_pipeline``. Below are pipeline segments available +for use. + +base64_decode + Decode the data element from base64 + +base64_encode + Encode the data element in base64 + +template + Treat the data element as a Jinja2 template and apply a node context to it. The defined context available + to the template is below. + + - node.network.[network_name].ip - IP address of this node on network [network_name] + - node.network.[network_name].cidr - CIDR of [network_name] + - node.network.[network_name].dns_suffix - DNS suffix of [network_name] + - node.hostname - Hostname of the node + - node.tags - Sequence of tags assigned to this node + - node.labels - Key, value pairs of both explicit and dynamic labels for this node + - action.key - A key that uniquely identifies this boot action on this node. Can be used for signaling boot action result. + - action.report_url - The URL that can be POSTed to for reporting boot action result. + + Also available in the Jinja2 template is the ``urlencode`` filter to encode a string for inclusion + in a URL. + +Reporting Results +================= + +The assets put in place on a server can report the results of applying the boot action using the Drydock :ref:`bootaction-api`. The +report API URL and boot action key are both available via the ``template`` pipeline segment context. It is up to the boot action +assets to implement the call back to the API for reporting whatever data the boot action desires. diff --git a/docs/source/index.rst b/docs/source/index.rst index d668a71f..0e4afb2d 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -36,6 +36,15 @@ Drydock Configuration Guide sampleconf policy-enforcement +API Documentation +----------------- +.. toctree:: + :maxdepth: 1 + + API + task + bootaction + Client Documentation -------------------- diff --git a/docs/task.rst b/docs/source/task.rst similarity index 97% rename from docs/task.rst rename to docs/source/task.rst index a8cdd631..000a3e43 100644 --- a/docs/task.rst +++ b/docs/source/task.rst @@ -1,3 +1,5 @@ +.. _task: + Tasks ===== @@ -9,7 +11,7 @@ the task API for task status and results. Task Document Schema -------------------- -This document can be posted to the Drydock task API to create a new task.:: +This document can be posted to the Drydock :ref:`task-api` to create a new task.:: { "action": "validate_design|verify_site|prepare_site|verify_node|prepare_node|deploy_node|destroy_node", diff --git a/docs/source/topology.rst b/docs/source/topology.rst index 10be7059..ca90fb8a 100644 --- a/docs/source/topology.rst +++ b/docs/source/topology.rst @@ -163,6 +163,7 @@ Ranges cannot overlap. *NOTE: Static routes are not currently implemented beyond specifying a route for ``0.0.0.0/0`` for default route* + ``routes`` defines a list of static routes to be configured on nodes attached to this network. @@ -449,10 +450,10 @@ parts: * The second part is the numeric portion and must be an integer * The third part is a label - * ``m``\|``M``\|``mb``\|``MB``: Megabytes or 10^6 * the numeric - * ``g``\|``G``\|``gb``\|``GB``: Gigabytes or 10^9 * the numeric - * ``t``\|``T``\|``tb``\|``TB``: Terabytes or 10^12 * the numeric - * ``%``: The percentage of total device or volume group space + * m|M|mb|MB: Megabytes or 10^6 * the numeric + * g|G|gb|GB: Gigabytes or 10^9 * the numeric + * t|T|tb|TB: Terabytes or 10^12 * the numeric + * %: The percentage of total device or volume group space Volume Groups and Logical Volumes ---------------------------------