Commit Graph

18 Commits

Author SHA1 Message Date
Sergiy Markin f99abfa433 Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfrmed based on
  airflow-2.6.2 constraints
- postgresql image updated to 14.8

Change-Id: Ibdcc75e600166c20b842508aa5539587cca466f0
2023-08-29 00:50:54 +00:00
Sergiy Markin 375abedb8a Drydock updates
This PS delivers the following updates:
- fixed sample config and policy files generation
- rolled back chart version incremention

Change-Id: I0a7145afd8c81e2bbf36d9437d4eff3c0354667a
2023-04-28 18:39:25 +00:00
Sergiy Markin d00eaf0303 Drydock focal related upgrades
This PS implements the following changes:
- switches freeze approach to requirements-direct.txt and
  requirements-frozen.txt files
- adjusts code tabulation style according to  yapf recommendations
- replaces deprecated usage of responce.body attribute with
  responce.text
- fixes integration tests in controlled by Makefile + tox
- uplifts Helm to v3.9.4

Change-Id: I751db72eb8f670825382f11a36657112faeb169a
2023-04-26 22:32:49 +00:00
Sergiy Markin 415a8b52c5 [focal] Python modules sync with Airship project
- uplifted some python modules
- fixed tox4 requirements
- added focal build node as a default one
- added bindep.txt and bindep role to playbooks and docker image build process
- changes Makefile to reflect GoLang and dependency management changes
- upgraded Helm to v3 for chart build process
- uplifted postgresql version to 14.6
- fixed deprecated falcon.API - replaced with falcon.APP
- fixed upstream docker image publishing process

Change-Id: I307d72bb7680f6f5c71e42ad30666cf786420460
2023-04-08 08:20:22 +00:00
SPEARS, DUSTIN (ds443n) a171f3c7a5 Add postgresql retention cronjob
Adding cronjob to purge the drydock DB based on retention day value. Additionally adding drydock API endpoint for purging the tasks and result_message tables and running vacuum full on drydock DB.

Change-Id: Ibcce61ecdafa637ca3ffec654152060aae26d4b8
2022-05-18 10:55:26 -04:00
Phil Sphicas 611e98de3f Use OOB driver creds for MAAS
During MAAS enlistment (and commissioning), an IPMI account (named
"maas" by default) is created on each node, which MAAS then uses for
power management.

This change allows MAAS to use the same credentials as the ones used by
the OOB driver, by overwriting the power parameters for the discovered
nodes. This includes the power type, so if the node is configured to use
Redfish, then Drydock will update a MAAS node discovered as IPMI to use
Redfish instead.

It also provides an option to instruct MAAS not to recreate IPMI
credentials during commissioning, which is passed through to the MAAS
API. Setting this to true is only supported in MAAS 2.7 or later [0].

The two maasdriver configuration options are introduced in drydock.conf,
along with their default values:

    [maasdriver]
    use_node_oob_params = false
    skip_bmc_config = false

These options do not prevent MAAS from creating the IPMI account during
enlistment - this would require addition MAAS customization.

0: 8842d0bfd3

Change-Id: I24d3bc3b1cc94907d73bc247de3fc06dd4750ab1
2021-07-30 16:39:06 +00:00
Hemanth Nakkina da0c7e831e Add Redfish as OOB driver
This patch implements Refish as new OOB driver for Drydock.
All the existing Drydock Orchestrator actions are implemented.

Change-Id: I31d653fb41189a18c34cfafb0f490ca4f4d661b5
2018-12-08 21:19:30 +05:30
Scott Hussey 6ca7aa4bff Catchup YAPF formatting
Change-Id: Ic54f77b4b0bdd9199fbc10dfdfc43d3af8f0bfd1
2018-09-26 08:57:51 -05:00
Scott Hussey 5344ed1957 Add destroy_node config items to sample
- Geneate a new sample configuration file including
  the items included in the destory_node action

Change-Id: Iaae8c95a7215b28b0fd54675582c86ba1e52a54a
2018-08-30 10:26:32 -05:00
Scott Hussey e2b3e8ee8e Database tunability
Add additional tunables to how SQLalchemy creates connection pools

Also start including static doc assets to they can be published

Change-Id: I268dc265a6b6cf1a200b235a5f99e65e89a95637
2018-08-21 14:09:36 -05:00
Scott Hussey cff7420cff Support links for task status
- Some status changes in a task may have additional information that
  is referenced by a URI link. Support describing these links and
  returning them via API.
- Refactor alembic stuff to better handle table schema updates
- Add unit tests

Change-Id: Iae63a9716f2522578be0244925fc274a4338eac4
2018-07-27 20:16:24 -05:00
Scott Hussey 06d6747b50 Make client HTTP connections resilient
- Make Keystone session use a timeout to prevent hangs
- Support retries
- Make the above configurable

Change-Id: I7123bd2fdcd329eae5b8b40f09168a1d599fa0f7
2018-06-26 14:41:24 -05:00
Scott Hussey 20873ad4f9 Update docs for developer overview
- New diagrams and documents for developer overview
- Update conf.py for docs to work w/ readthedocs.io
- Add policy and config gen to `make docs`
- Update zuul-linter to support checked in images
- Last fix to document publishing

Change-Id: I4faa1b87032ae5b0e786aa0fd998f809124b7987
2018-06-01 14:50:05 -05:00
Aaron Sheffield 1fd8bb9812 Migrating Health Check to UCP Standards
- Added a second health check endpoint /extended to get additional data.
- Conforms health check response with UCP standards.

Change-Id: I9e3ac27ec7e536bb18201f1a4642490725a8062c
2018-03-13 16:37:39 -04:00
Krysta Knight 3d4efe9907 Add Validation API to Drydock
This ps adds the validation endpoint to the Drydock API and includes
the unit tests for post_validation

Change-Id: I09f0602603e46a593dea948d226070d8fb67ff1d
2017-12-04 10:44:33 -06:00
Scott Hussey 455fa1fb79 NoAuth filter for PasteDeploy
Implement a filter usable by PasteDeploy to forge
the headers normally used by keystonemiddleware to signal
a properly authenitcated request. The filter needs to be
added to the pipeline in paste.ini if noauth support is needed
for testing.

- Add PasteDeploy config to disable keystone config loading

Change-Id: Ie33ee86f1ca8209a6d96cf34c41acd7dca848d58
2017-11-27 10:12:19 -05:00
Felipe Monteiro 8afdedab30 Drydock documentation via build_sphinx.
This PS adds tooling and automation to automatically generate
Drydock's documentation into feature-rich HTML pages that can
be hosted.

To run the documentation job, simply execute:

    tox -e docs

A future PS should add warning_is_error to 'build_sphinx' in
setup.py once the import warnings are addressed.

Change-Id: I91a3c585b2c27096e7fde12d180638d1ae4bdb81
2017-10-06 15:05:41 -04:00
Scott Hussey 4ae627be44 Add config generation to tox.ini
Move sample config to etc/drydock
Update docs to generate a config with tox

Update configuration for Keystone

- Add config generation to tox.ini
- Fix default in bootdata config
- Add keystone dependencies
- Add config generator config
- Move sample config to a skeleton etc/drydock tree

Use PasteDeploy for WSGI integration

Using keystonemiddleware outside of a PasteDeploy
pipeline is deprecated. Move Drydock to use PasteDeploy
and integrate with keystonemiddleware

Update Falcon context object

Add keystone identity fields to context object
Clean up context marker field

Fix AuthMiddleware for keystone

Update falcon middleware to harvest headers injected
by keystonemiddleware

Fix context middleware

Update context middleware to enforce
a UUID-formatted external context marker

Lock keystonemiddleware version

Lock keystonemiddleware version to the Newton release

Sample drydock.conf with keystone

This drydock.conf file is known to integrate successfully
with Keystone via keystonemiddleware and the password plugin

Add .dockerignore

Stop adding .tox environment to docker images

Integrate with oslo.policy

Add oslo.policy 1.9.0 to requirements (Newton release)
Add tox job to generate sample policy.yaml
Create DrydockPolicy as facade for RBAC

Inject policy engine into API init

Create a DrydockPolicy instance and inject it into
the Drydock API resources.

Remove per-resource authorization

Update Drydock context and auth middleware

Update Drydock context to use keystone IDs instead of names as required
by oslo.policy
Update AuthMiddleware to capture headers when request provides
a service token

Add RBAC for /designs API

Add RBAC enforcement for GET and POST of
/api/v1.0/designs endpoint

Refactor check_policy

Refactor check_policy into the base class

Enforce RBAC for /designs/id endpoint

Enforce RBAC on /designs/id/parts endpoint

Enforce RBAC on /designs/id/parts/kind

Enforce RBAC on /designs/id/parts/kinds/

Enforce RBAC on /tasks/ endpoints

Create unit tests

- New unit tests for DrydockPolicy
- New unit tests for AuthMiddleware w/ Keystone integration

Address impacting keystonemiddleware bug

Use v4.9.1 to address https://bugs.launchpad.net/keystonemiddleware/+bug/1653646

Add oslo_config fixtures for unit testing

API base class fixes

Fix an import error in API resource base class

More graceful error handling in drydock_client

Create shared function for checking API response status codes

Create client errors for auth

Create specific Exceptions for Unauthorized
and Forbidden responses

Ignore generated sample configs

Lock iso8601 version

oslo.versionedobjects appears to be impcompatible with
iso8601 0.1.12 on Python 3.2+

Update docs for Keystone

Note Keystone as a external depdendency and
add notes on correctly configuring Drydock for
Keystone integration

Add keystoneauth1 to list_opts

Explicitly pull keystoneauth password plugin
options when generating a config template

Update reference config for keystone

Update the reference config template
for Keystone integration

Add keystoneauth1 to requirements

Need to directly include keystoneauth1 so that
oslo_config options can be pulled from it

Update config doc for keystoneauth1

Use the keystoneauth1 generated configuration options
for the configuration  docs

Remove auth options

Force dependence on Keystone as the only authentication
backend

Clean up imports

Fix how falcon modules are imported

Default to empty role list

Move param extraction

Enforce RBAC before starting to parse parameters

Implement DocumentedRuleDefault

Use DocumentedRuleDefault for policy defaults at request
of @tlam. Requires v 1.21.1 of oslo_policy, which is tied
to the Pike openstack release.

Change sample output filenames

Update filenames to follow Openstack convention

Fix tests to use hex formatted IDs

Openstack resource IDs are not hyphenated, so update
unit tests to reflect this

Fix formating and whitespace

Refactor a few small items for code review

Update keystone integration to be more
robust with Newton codebase

Centralize policy_engine reference to
support a decorator-based model

RBAC enforcement decorator

Add units tests for decorator-based
RBAC and the tasks API

Minor refactoring and format changes

Change-Id: I35f90b0c88ec577fda1077814f5eac5c0ffb41e9
2017-08-21 14:35:56 -05:00