[390773] Support SRIOV fields in definition profiles

- Add hugepages and cpu_sets stanzas to HardwareProfile as the
  size and count of hugepages and the exact CPUs to pin for SRIOV
  are dependent on hardware.
- Add sriov stanza to a node interface to specify vf_count and
  trustedmode. These will be passthrough values as Drydock doesn't
  configure SRIOV.
- Add sriov information to the bootaction context so it can be written
  to disk on a deployed node if needed
- Allow an interface configuration to be skipped when an interface has
  no defined network_link for things like SR-IOV interfaces.
- Add kernel parameter reference support to access hardware profile
  information
- Add unit tests
- Update topology documentation for usage of HardwareProfile
  and kernel parameter references

Change-Id: Iefd326f5c6fad19dbd21300ee249019a3dfd4848

.dockerignore

@@ -1 +1,2 @@
 .tox
+**/build

Makefile

@@ -75,10 +75,19 @@ ifeq ($(PUSH_IMAGE), true)
 	docker push $(IMAGE)
 endif
 
+.PHONY: docs
+docs: clean drydock_docs
+
+.PHONY: drydock_docs
+drydock_docs:
+	tox -e docs
 
 .PHONY: clean
 clean:
 	rm -rf build
+	rm -rf docs/build
+	rm -rf charts/drydock/charts
+	rm -rf charts/drydock/requirements.lock
 
 .PHONY: pep8
 pep8:

docs/source/topology.rst

@@ -227,6 +227,90 @@ reference to the particular physical node. The ``BaremetalNode`` definition will
 reference a ``HostProfile`` and can then extend or override any of the
 configuration values.
 
+
+Hardware Profile
+----------------
+
+The hardware profile is used to convert some abstractions in the HostProfile documents
+into concrete configurations based a particular hardware build. A host profile will
+designate how the bootdisk should be configured, but the hardware profile will
+designate which exact device is used for the bootdisk. This allows a heterogeneous mix
+of hardware in a site without duplicating definitions of how that hardware should
+be configured.
+
+An example HardwareProfile document:
+
+.. code:: yaml
+
+    ---
+    schema: 'drydock/HardwareProfile/v1'
+    metadata:
+      schema: 'metadata/Document/v1'
+      name: AcmeServer
+      storagePolicy: 'cleartext'
+      labels:
+        application: 'drydock'
+    data:
+      vendor: HP
+      generation: '8'
+      hw_version: '3'
+      bios_version: '2.2.3'
+      boot_mode: bios
+      bootstrap_protocol: pxe
+      pxe_interface: 0
+      device_aliases:
+        prim_nic01:
+          address: '0000:00:03.0'
+          dev_type: '82540EM Gigabit Ethernet Controller'
+          bus_type: 'pci'
+        prim_nic02:
+          address: '0000:00:04.0'
+          dev_type: '82540EM Gigabit Ethernet Controller'
+          bus_type: 'pci'
+        primary_boot:
+          address: '2:0.0.0'
+          dev_type: 'VBOX HARDDISK'
+          bus_type: 'scsi'
+      cpu_sets:
+        sriov: '2,4'
+      hugepages:
+        sriov:
+          size: '1G'
+          count: 300
+        dpdk:
+          size: '2M'
+          count: 530000
+
+Device Aliases
+~~~~~~~~~~~~~~
+
+Device aliases are a way of mapping a particular device bus address
+to an alias. In the example above we map the PCI address ``0000:00:03.0``
+to the alias ``prim_nic01``. A host profile or baremetal node definition
+can then provide a configuration using ``prim_nic01`` and Drydock will
+translate that to the correct operating system device name for the NIC device
+at PCI address ``0000.00.03.0``. Currently device aliases are supported
+for network interface slave devices and storage physical devices.
+
+Kernel Parameter References
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Some kernel parameters specified in a host profile rely on particular hardware
+builds, such as ``isolcpus``. To support the greatest flexibility in building
+host profiles, you can specify a few values in a hardware profile that will then
+be sourced when needed by a host profile or baremetal node definition.
+
+* ``cpu_sets``: Each key should have a value of a comma-separated list of CPUs/cores/hyperthreads
+  that would be appropriate for the ``isolcpus`` kernel parameters. A host profile can
+  then select any one of these CPU sets for a host.
+* ``hugepages``: Each key should have a value of a mapping containing two keys: ``size`` and
+  ``count``. Again, a host profile can then select these values when defining kernel parameters
+  for a host. Note the ``size`` field is a string and will be used as-is, so the format must
+  be usable by the kernel.
+
+Host Profiles and Baremetal Nodes
+---------------------------------
+
 Example ``HostProfile`` and ``BaremetalNode`` configuration:
 
 .. code:: yaml
@@ -273,7 +357,7 @@ adopted from *defaults*) and can then again override or append any
 configuration that is specific to that node.
 
 Defining Node Interfaces and Network Addressing
-===============================================
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Node network attachment can be described in a ``HostProfile`` or a
 ``BaremetalNode`` document. Node addressing is allowed only in a
@@ -286,7 +370,7 @@ Once the interface attachments to networks is defined, ``HostProfile`` and
 which network the node should use as the primary route.
 
 Interfaces
-----------
+**********
 
 Interfaces for a node can be described in either a ``HostProfile`` or
 ``BaremetalNode`` definition. This will attach a defined NetworkLink to a host
@@ -333,7 +417,7 @@ that interface for an inherited configuration.
   have trunking enabled or the design validation will fail.
 
 Addressing
-----------
+**********
 
 Addressing for a node can only be defined in a ``BaremetalNode`` definition. The
 ``addressing`` stanza simply defines a static IP address or ``dhcp`` for each
@@ -358,7 +442,7 @@ Example ``addressing`` YAML schema:
 
 
 Defining Node Storage
-=====================
+~~~~~~~~~~~~~~~~~~~~~
 
 Storage can be defined in the ``storage`` stanza of either a HostProfile or
 BaremetalNode document. The storage configuration can describe the creation of
@@ -405,13 +489,13 @@ Example YAML schema of the ``storage`` stanza:
                 mount_options: 'defaults'
 
 Schema
-------
+******
 
 The ``storage`` stanza can contain two top-level keys: ``physical_devices`` and
 ``volume_groups``. The latter is optional.
 
 Physical Devices and Partitions
--------------------------------
+*******************************
 
 A physical device can either be carved up in partitions (including a single
 partition consuming the entire device) or added to a volume group as a physical
@@ -429,7 +513,7 @@ mapping with the following keys
   volume. Incompatible with the ``partitions`` specification.
 
 Partition
-~~~~~~~~~
+^^^^^^^^^
 
 A partition mapping describes a GPT partition on a physical disk. It can be left
 as a raw block device or formatted and mounted as a filesystem.
@@ -451,7 +535,7 @@ as a raw block device or formatted and mounted as a filesystem.
     * ``fs_label``: A filesystem label to assign to the filesystem. Optional.
 
 Size Format
-~~~~~~~~~~~
+^^^^^^^^^^^
 
 The size specification for a partition or logical volume is formed from three
 parts:
@@ -468,7 +552,7 @@ parts:
     * %: The percentage of total device or volume group space
 
 Volume Groups and Logical Volumes
----------------------------------
+*********************************
 
 Logical volumes can be used to create RAID-0 volumes spanning multiple physical
 disks or partitions. Each key in the ``volume_groups`` mapping is a name
@@ -482,7 +566,7 @@ invalid. Each mapping value is another mapping describing the volume group.
   created in the volume group
 
 Logical Volume
-~~~~~~~~~~~~~~
+^^^^^^^^^^^^^^
 
 A logical volume is a RAID-0 volume. Using logical volumes for ``/`` and
 ``/boot`` is supported
@@ -494,3 +578,41 @@ A logical volume is a RAID-0 volume. Using logical volumes for ``/`` and
 * ``filesystem``: A mapping specifying how the logical volume should be
   formatted and mounted. See the *Partition* section above for filesystem
   details.
+
+Platform Configuration
+----------------------
+
+In the ``platform`` stanza you can define the operating system ``image``
+and ``kernel`` to use as well as customize the kernel configuration with
+``kernel_params``.
+
+Image and Kernel Selection
+**************************
+
+The valid ``image`` and ``kernel`` values are dependent on what is supported
+by your node provisioner. In the example of Canonical MaaS using the 16.04 LTS
+image, the values would be ``image: 'xenial'`` and ``kernel: 'ga-16.04'`` for the
+LTS kernel or ``kernel: hwe-16.04`` for the hardware-enablement kernel.
+
+Kernel Parameters
+*****************
+
+The ``kernel_params`` configuration is a mapping. Each key should either be a string
+or boolean value. For boolean ``true`` values, the key will be added to the kernel
+parameter list as a flag. For string values, the key:value pair will be added to the
+kernel parameter list as ``key=value``.
+
+Parameter References
+^^^^^^^^^^^^^^^^^^^^
+
+One special case is supported for values that match a hardware profile reference.
+When the parameter is rendered for a particular node, the value included in the
+kernel parameter list will be sourced from the effective HardwareProfile assigned
+to the node.
+
+* ``hardwareprofile:cpuset.<name>``: Sourced from the hardware profile ``cpu_sets.<name>``
+  value.
+* ``hardwareprofile.hugepages.<name>.size``: Source from the hardware profile
+  ``hugepages.<name>.size`` value.
+* ``hardwareprofile.hugepages.<name>.count``: Source from the hardware profile
+  ``hugepages.<name>.count`` value.

drydock_provisioner/control/middleware.py

@@ -31,7 +31,8 @@ class AuthMiddleware(object):
 
         ctx.set_policy_engine(policy.policy_engine)
 
-        self.logger.debug("Request with headers: %s" % ','.join(req.headers.keys()))
+        self.logger.debug(
+            "Request with headers: %s" % ','.join(req.headers.keys()))
 
         auth_status = req.get_header('X-SERVICE-IDENTITY-STATUS')
         service = True
@@ -101,7 +102,9 @@ class LoggingMiddleware(object):
             'req_id': req.context.request_id,
             'external_ctx': req.context.external_marker,
         }
-        self.logger.info("Request: %s %s %s" % (req.method, req.uri, req.query_string), extra=extra)
+        self.logger.info(
+            "Request: %s %s %s" % (req.method, req.uri, req.query_string),
+            extra=extra)
 
     def process_response(self, req, resp, resource, req_succeeded):
         ctx = req.context
@@ -111,4 +114,6 @@ class LoggingMiddleware(object):
             'external_ctx': ctx.external_marker,
         }
         resp.append_header('X-Drydock-Req', ctx.request_id)
-        self.logger.info("Response: %s %s - %s" % (req.method, req.uri, resp.status), extra=extra)
+        self.logger.info(
+            "Response: %s %s - %s" % (req.method, req.uri, resp.status),
+            extra=extra)

drydock_provisioner/drivers/node/maasdriver/actions/node.py

@@ -968,6 +968,12 @@ class ApplyNodeNetworking(BaseMaasAction):
                         machine.refresh()
 
                         for i in n.interfaces:
+                            if not i.network_link:
+                                self.logger.debug(
+                                    "Interface %s has no network link, skipping configuration."
+                                    % (i.device_name))
+                                continue
+
                             nl = site_design.get_network_link(i.network_link)
 
                             if nl.metalabels is not None:
@@ -1130,7 +1136,8 @@ class ApplyNodeNetworking(BaseMaasAction):
                                             link_iface = machine.interfaces.create_vlan(
                                                 **vlan_options)
                                         except errors.DriverError as ex:
-                                            msg = "Error creating interface: %s" % str(ex)
+                                            msg = "Error creating interface: %s" % str(
+                                                ex)
                                             self.logger.info(msg)
                                             self.task.add_status_msg(
                                                 msg=msg,

drydock_provisioner/error.py

@@ -63,6 +63,37 @@ class UnsupportedDocumentType(DesignError):
     pass
 
 
+class HugepageConfNotFound(DesignError):
+    """
+    **Message:** *Hugepage configuration not found*.
+
+    **Troubleshoot:** *Define the hugepage configuration in the HardwareProfile*.
+    """
+    pass
+
+
+class CpuSetNotFound(DesignError):
+    """
+    **Message:** *CPU set not found*.
+
+    **Troubleshoot:** *Define the CPU set in the HardwareProfile*.
+    """
+    pass
+
+
+class InvalidParameterReference(DesignError):
+    """
+    **Message:** *Invalid configuration specified*.
+
+    **Troubleshoot:** *Only ``cpuset`` and ``hugepages`` can be referenced*.
+
+    **Message:** *Invalid field specified*.
+
+    **Troubleshoot:** *For hugepages, only fields ``size`` and ``count`` are valid*.
+    """
+    pass
+
+
 class StateError(Exception):
     pass
 

drydock_provisioner/ingester/plugins/deckhand.py

@@ -372,6 +372,15 @@ class DeckhandIngester(IngesterPlugin):
             dev_model.address = v.get('address', None)
             model.devices.append(dev_model)
 
+        model.cpu_sets = data.get('cpu_sets', None) or dict()
+
+        model.hugepages_confs = objects.HugepagesConfList()
+
+        for c, d in data.get('hugepages', {}).items():
+            conf = objects.HugepagesConf(
+                name=c, size=d.get('size'), count=d.get('count'))
+            model.hugepages_confs.append(conf)
+
         return model
 
     def process_drydock_hostprofile(self, name, data):
@@ -535,6 +544,12 @@ class DeckhandIngester(IngesterPlugin):
             for n in networks:
                 int_model.networks.append(n)
 
+            if 'sriov' in v:
+                int_model.sriov = True
+                int_model.vf_count = v.get('sriov', {}).get('vf_count', 0)
+                int_model.trustedmode = v.get('sriov', {}).get(
+                    'trustedmode', False)
+
             model.interfaces.append(int_model)
 
         platform = data.get('platform', {})

drydock_provisioner/objects/bootaction.py

@@ -134,31 +134,8 @@ class BootActionAsset(base.DrydockObject):
         :param action_id: a 128-bit ULID boot action id
         :param design_ref: The design ref this bootaction was initiated under
         """
-        node = site_design.get_baremetal_node(nodename)
+        tpl_ctx = self._get_template_context(nodename, site_design, action_id,
-
+                                             design_ref)
-        tpl_ctx = {
-            'node': {
-                'hostname': nodename,
-                'tags': [t for t in node.tags],
-                'labels': {k: v
-                           for (k, v) in node.owner_data.items()},
-                'network': {},
-            },
-            'action': {
-                'key': ulid2.ulid_to_base32(action_id),
-                'report_url': config.config_mgr.conf.bootactions.report_url,
-                'design_ref': design_ref,
-            }
-        }
-
-        for a in node.addressing:
-            if a.address is not None:
-                tpl_ctx['node']['network'][a.network] = dict()
-                tpl_ctx['node']['network'][a.network]['ip'] = a.address
-                network = site_design.get_network(a.network)
-                tpl_ctx['node']['network'][a.network]['cidr'] = network.cidr
-                tpl_ctx['node']['network'][a.network][
-                    'dns_suffix'] = network.dns_domain
 
         if self.location is not None:
             rendered_location = self.execute_pipeline(
@@ -174,6 +151,78 @@ class BootActionAsset(base.DrydockObject):
             value = value.encode('utf-8')
         self.rendered_bytes = value
 
+    def _get_template_context(self, nodename, site_design, action_id,
+                              design_ref):
+        """Create a context to be used for template rendering.
+
+        :param nodename: The name of the node for the bootaction
+        :param site_design: The full site design
+        :param action_id: the ULID assigned to the boot action using this context
+        :param design_ref: The design reference representing ``site_design``
+        """
+
+        return dict(
+            node=self._get_node_context(nodename, site_design),
+            action=self._get_action_context(action_id, design_ref))
+
+    def _get_action_context(self, action_id, design_ref):
+        """Create the action-specific context items for template rendering.
+
+        :param action_id: ULID of this boot action
+        :param design_ref: Design reference representing the site design
+        """
+        return dict(
+            key=ulid2.ulid_to_base32(action_id),
+            report_url=config.config_mgr.conf.bootactions.report_url,
+            design_ref=design_ref)
+
+    def _get_node_context(self, nodename, site_design):
+        """Create the node-specific context items for template rendering.
+
+        :param nodename: name of the node this boot action targets
+        :param site_design: full site design
+        """
+        node = site_design.get_baremetal_node(nodename)
+        return dict(
+            hostname=nodename,
+            tags=[t for t in node.tags],
+            labels={k: v
+                    for (k, v) in node.owner_data.items()},
+            network=self._get_node_network_context(node, site_design),
+            interfaces=self._get_node_interface_context(node))
+
+    def _get_node_network_context(self, node, site_design):
+        """Create a node's network configuration context.
+
+        :param node: node object
+        :param site_design: full site design
+        """
+        network_context = dict()
+        for a in node.addressing:
+            if a.address is not None:
+                network = site_design.get_network(a.network)
+                network_context[a.network] = dict(
+                    ip=a.address,
+                    cidr=network.cidr,
+                    dns_suffix=network.dns_domain)
+                if a.network == node.primary_network:
+                    network_context['default'] = network_context[a.network]
+
+        return network_context
+
+    def _get_node_interface_context(self, node):
+        """Create a node's network interface context.
+
+        :param node: the node object
+        """
+        interface_context = dict()
+        for i in node.interfaces:
+            interface_context[i.device_name] = dict(sriov=i.sriov)
+            if i.sriov:
+                interface_context[i.device_name]['vf_count'] = i.vf_count
+                interface_context[i.device_name]['trustedmode'] = i.trustedmode
+        return interface_context
+
     def resolve_asset_location(self, asset_url):
         """Retrieve the data asset from the url.
 

drydock_provisioner/objects/hostprofile.py

@@ -169,6 +169,14 @@ class HostInterface(base.DrydockObject):
         obj_fields.ObjectField('HardwareDeviceSelectorList', nullable=True),
         'networks':
         obj_fields.ListOfStringsField(nullable=True),
+        'sriov':
+        obj_fields.BooleanField(default=False),
+        # SRIOV virtual functions
+        'vf_count':
+        obj_fields.IntegerField(nullable=True),
+        # SRIOV VF trusted mode
+        'trustedmode':
+        obj_fields.BooleanField(nullable=True),
     }
 
     def __init__(self, **kwargs):

drydock_provisioner/objects/hwprofile.py

@@ -16,6 +16,7 @@
 from oslo_versionedobjects import fields as ovo_fields
 
 import drydock_provisioner.objects as objects
+import drydock_provisioner.error as errors
 import drydock_provisioner.objects.base as base
 import drydock_provisioner.objects.fields as hd_fields
 
@@ -48,6 +49,10 @@ class HardwareProfile(base.DrydockPersistentObject, base.DrydockObject):
         ovo_fields.StringField(nullable=True),
         'devices':
         ovo_fields.ObjectField('HardwareDeviceAliasList', nullable=True),
+        'cpu_sets':
+        ovo_fields.DictOfStringsField(nullable=True),
+        'hugepages_confs':
+        ovo_fields.ObjectField('HugepagesConfList', nullable=True),
     }
 
     def __init__(self, **kwargs):
@@ -62,6 +67,29 @@ class HardwareProfile(base.DrydockPersistentObject, base.DrydockObject):
     def get_name(self):
         return self.name
 
+    def get_hugepage_conf(self, conf_name):
+        """Return the hugepages conf matching ``conf_name``"""
+        if not self.hugepages_confs:
+            raise errors.HugepageConfNotFound(
+                "Hugepage configuration %s not found." % conf_name)
+
+        for c in self.hugepages_confs:
+            if c.name == conf_name:
+                return c
+
+        raise errors.HugepageConfNotFound(
+            "Hugepage configuration %s not found." % conf_name)
+
+    def get_cpu_set(self, set_name):
+        """Return the cpu set matching ``set_name``"""
+        if not self.cpu_sets:
+            raise errors.CpuSetNotFound("CPU set %s not found." % set_name)
+
+        if set_name in self.cpu_sets:
+            return self.cpu_sets[set_name]
+
+        raise errors.CpuSetNotFound("CPU set %s not found." % set_name)
+
     def resolve_alias(self, alias_type, alias):
         for d in self.devices:
             if d.alias == alias and d.bus_type == alias_type:
@@ -82,6 +110,26 @@ class HardwareProfileList(base.DrydockObjectListBase, base.DrydockObject):
     fields = {'objects': ovo_fields.ListOfObjectsField('HardwareProfile')}
 
 
+@base.DrydockObjectRegistry.register
+class HugepagesConf(base.DrydockObject):
+
+    VERSION = '1.0'
+
+    fields = {
+        'name': ovo_fields.StringField(),
+        'size': ovo_fields.StringField(),
+        'count': ovo_fields.NonNegativeIntegerField(),
+    }
+
+
+@base.DrydockObjectRegistry.register
+class HugepagesConfList(base.DrydockObjectListBase, base.DrydockObject):
+
+    VERSION = '1.0'
+
+    fields = {'objects': ovo_fields.ListOfObjectsField('HugepagesConf')}
+
+
 @base.DrydockObjectRegistry.register
 class HardwareDeviceAlias(base.DrydockObject):
 

drydock_provisioner/objects/node.py

@@ -20,6 +20,7 @@ from defusedxml.ElementTree import fromstring
 import logging
 from oslo_versionedobjects import fields as ovo_fields
 
+import drydock_provisioner.error as errors
 import drydock_provisioner.config as config
 import drydock_provisioner.objects as objects
 import drydock_provisioner.objects.hostprofile
@@ -49,9 +50,14 @@ class BaremetalNode(drydock_provisioner.objects.hostprofile.HostProfile):
     # Compile the applied version of this model sourcing referenced
     # data from the passed site design
     def compile_applied_model(self, site_design, state_manager):
+        self.logger.debug("Applying host profile to node %s" % self.name)
         self.apply_host_profile(site_design)
+        self.logger.debug("Applying hardware profile to node %s" % self.name)
         self.apply_hardware_profile(site_design)
         self.source = hd_fields.ModelSource.Compiled
+        self.logger.debug("Resolving kernel parameters on node %s" % self.name)
+        self.resolve_kernel_params(site_design)
+        self.logger.debug("Resolving device aliases on node %s" % self.name)
         self.apply_logicalnames(site_design, state_manager)
         return
 
@@ -87,6 +93,67 @@ class BaremetalNode(drydock_provisioner.objects.hostprofile.HostProfile):
 
         return
 
+    def resolve_kernel_params(self, site_design):
+        """Check if any kernel parameter values are supported references."""
+        if not self.hardware_profile:
+            raise ValueError("Hardware profile not set.")
+
+        hwprof = site_design.get_hardware_profile(self.hardware_profile)
+
+        if not hwprof:
+            raise ValueError("Hardware profile not found.")
+
+        resolved_params = dict()
+        for p, v in self.kernel_params.items():
+            try:
+                rv = self.get_kernel_param_value(v, hwprof)
+                resolved_params[p] = rv
+            except (errors.InvalidParameterReference, errors.CpuSetNotFound,
+                    errors.HugepageConfNotFound) as ex:
+                resolved_params[p] = v
+                msg = ("Error resolving parameter reference on node %s: %s" %
+                       (self.name, str(ex)))
+                self.logger.warning(msg)
+
+        self.kernel_params = resolved_params
+
+    def get_kernel_param_value(self, value, hwprof):
+        """If ``value`` is a reference, resolve it otherwise return ``value``
+
+        Support some referential values to extract data from the HardwareProfile
+
+        hardwareprofile:cpuset.<setname>
+        hardwareprofile:hugepages.<confname>.size
+        hardwareprofile:hugepages.<confname>.count
+
+        If ``value`` matches none of the above forms, just return the value as passed.
+
+        :param value: the value string as specified in the node definition
+        :param hwprof: the assigned HardwareProfile for this node
+        """
+        if value.startswith('hardwareprofile:'):
+            (_, ref) = value.split(':', 1)
+            if ref:
+                (ref_type, ref_val) = ref.split('.', 1)
+                if ref_type == 'cpuset':
+                    return hwprof.get_cpu_set(ref_val)
+                elif ref_type == 'hugepages':
+                    (conf, field) = ref_val.split('.', 1)
+                    hp_conf = hwprof.get_hugepage_conf(conf)
+                    if field in ['size', 'count']:
+                        return getattr(hp_conf, field)
+                    else:
+                        raise errors.InvalidParameterReference(
+                            "Invalid field %s specified." % field)
+                else:
+                    raise errors.InvalidParameterReference(
+                        "Invalid configuration %s specified." % ref_type)
+            else:
+                return value
+
+        else:
+            return value
+
     def get_applied_interface(self, iface_name):
         for i in getattr(self, 'interfaces', []):
             if i.get_name() == iface_name:

drydock_provisioner/orchestrator/orchestrator.py

@@ -247,11 +247,22 @@ class Orchestrator(object):
         Given a fully populated Site model, compute the effective
         design by applying inheritance and references
         """
+        node_failed = []
+
         try:
             nodes = site_design.baremetal_nodes
             for n in nodes or []:
-                n.compile_applied_model(
+                try:
-                    site_design, state_manager=self.state_manager)
+                    n.compile_applied_model(
+                        site_design, state_manager=self.state_manager)
+                except Exception as ex:
+                    node_failed.append(n)
+                    self.logger.error(
+                        "Failed to build applied model for node %s." % n.name)
+            if node_failed:
+                raise errors.DesignError(
+                    "Failed to build applied model for %s" % ",".join(
+                        [x.name for x in node_failed]))
         except AttributeError:
             self.logger.debug(
                 "Model inheritance skipped, no node definitions in site design."

drydock_provisioner/schemas/hardwareProfile.yaml

@@ -34,4 +34,17 @@ data:
     device_aliases:
       type: 'object'
       additionalProperties: true
+    cpu_sets:
+      type: 'object'
+      additionalProperties:
+        type: 'string'
+    hugepages:
+      type: 'object'
+      additionalProperties:
+        type: 'object'
+        propertes:
+          size:
+            type: 'string'
+          count:
+            type: 'number'
   additionalProperties: false

drydock_provisioner/schemas/hostProfile.yaml

@@ -151,5 +151,12 @@ data:
             type: 'array'
             items:
               type: 'string'
+          sriov:
+            type: 'object'
+            properties:
+              vf_count:
+                type: 'number'
+              trustmode:
+                type: 'boolean'
   additionalProperties: false
 ...

drydock_provisioner/statemgmt/state.py

@@ -282,8 +282,8 @@ class DrydockState(object):
         """
         try:
             conn = self.db_engine.connect()
-            query = self.tasks_tbl.insert().values(**(
+            query = self.tasks_tbl.insert().values(
-                task.to_db(include_id=True)))
+                **(task.to_db(include_id=True)))
             conn.execute(query)
             conn.close()
             return True

tests/unit/test_bootaction_asset_render.py

@@ -16,15 +16,12 @@
 import ulid2
 
 from drydock_provisioner.statemgmt.state import DrydockState
-import drydock_provisioner.objects as objects
 
 
-class TestClass(object):
+class TestBootactionRenderAction(object):
     def test_bootaction_render_nodename(self, input_files, deckhand_ingester,
                                         setup):
         """Test the bootaction render routine provides expected output."""
-        objects.register_all()
-
         input_file = input_files.join("deckhand_fullsite.yaml")
 
         design_state = DrydockState()
@@ -43,8 +40,6 @@ class TestClass(object):
     def test_bootaction_render_design_ref(self, input_files, deckhand_ingester,
                                           setup):
         """Test the bootaction render routine provides expected output."""
-        objects.register_all()
-
         input_file = input_files.join("deckhand_fullsite.yaml")
 
         design_state = DrydockState()
@@ -60,3 +55,38 @@ class TestClass(object):
 
         assert 'deckhand_fullsite.yaml' in assets[2].rendered_bytes.decode(
             'utf-8')
+
+    def test_bootaction_network_context(self, input_files,
+                                        deckhand_orchestrator, setup):
+        """Test that a boot action creates proper network context."""
+        input_file = input_files.join("deckhand_fullsite.yaml")
+
+        design_ref = "file://%s" % str(input_file)
+
+        design_status, design_data = deckhand_orchestrator.get_effective_site(
+            design_ref)
+
+        ba = design_data.get_bootaction('helloworld')
+        node = design_data.get_baremetal_node('compute01')
+        net_ctx = ba.asset_list[0]._get_node_network_context(node, design_data)
+
+        assert 'mgmt' in net_ctx
+        assert net_ctx['mgmt'].get('ip', None) == '172.16.1.21'
+
+    def test_bootaction_interface_context(self, input_files,
+                                          deckhand_orchestrator, setup):
+        """Test that a boot action creates proper network context."""
+        input_file = input_files.join("deckhand_fullsite.yaml")
+
+        design_ref = "file://%s" % str(input_file)
+
+        design_status, design_data = deckhand_orchestrator.get_effective_site(
+            design_ref)
+
+        ba = design_data.get_bootaction('helloworld')
+        node = design_data.get_baremetal_node('compute01')
+        iface_ctx = ba.asset_list[0]._get_node_interface_context(node)
+
+        assert 'bond0' in iface_ctx
+        assert iface_ctx['bond0'].get('sriov')
+        assert iface_ctx['bond0'].get('vf_count') == 2

tests/unit/test_param_reference.py

@@ -0,0 +1,50 @@
+# Copyright 2017 AT&T Intellectual Property.  All other rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import logging
+
+from drydock_provisioner.statemgmt.state import DrydockState
+from drydock_provisioner.orchestrator.orchestrator import Orchestrator
+
+LOG = logging.getLogger(__name__)
+
+
+class TestKernelParameterReferences(object):
+    def test_valid_param_reference(self, deckhand_ingester, input_files,
+                                   setup):
+        input_file = input_files.join("deckhand_fullsite.yaml")
+
+        design_state = DrydockState()
+        design_ref = "file://%s" % str(input_file)
+
+        orchestrator = Orchestrator(
+            state_manager=design_state, ingester=deckhand_ingester)
+
+        design_status, design_data = orchestrator.get_effective_site(
+            design_ref)
+
+        assert len(design_data.baremetal_nodes) == 2
+
+        node = design_data.get_baremetal_node("compute01")
+
+        assert node.hardware_profile == 'HPGen9v3'
+
+        isolcpu = node.kernel_params.get('isolcpus', None)
+
+        # '2,4' is defined in the HardwareProfile as cpu_sets.sriov
+        assert isolcpu == '2,4'
+
+        hugepagesz = node.kernel_params.get('hugepagesz', None)
+
+        assert hugepagesz == '1G'

tests/yaml_samples/deckhand_fullsite.yaml

@@ -302,6 +302,9 @@ data:
       networks:
         - mgmt
         - private
+      sriov:
+        vf_count: 2
+        trustedmode: false
   metadata:
     tags:
       - 'test'
@@ -349,6 +352,11 @@ data:
       address: 172.16.2.21
     - network: oob
       address: 172.16.100.21
+  platform:
+    kernel_params:
+      isolcpus: hardwareprofile:cpuset.sriov
+      hugepagesz: hardwareprofile:hugepages.sriov.size
+      hugepages: hardwareprofile:hugepages.sriov.count
   metadata:
     rack: rack2
 ---
@@ -380,6 +388,15 @@ data:
       address: '2:0.0.0'
       dev_type: 'VBOX HARDDISK'
       bus_type: 'scsi'
+  cpu_sets:
+    sriov: '2,4'
+  hugepages:
+    sriov:
+      size: '1G'
+      count: 300
+    dpdk:
+      size: '2M'
+      count: 530000
 ---
 schema: 'drydock/BootAction/v1'
 metadata: