Commit Graph

44 Commits

Author SHA1 Message Date
Sergiy Markin f99abfa433 Airflow stable 2.6.2
This PS updates python modules and code to match Airflow 2.6.2:

- bionic py36 gates  were removed
- python code corrected to match new modules versions
- selection of python modules versions was perfrmed based on
  airflow-2.6.2 constraints
- postgresql image updated to 14.8

Change-Id: Ibdcc75e600166c20b842508aa5539587cca466f0
2023-08-29 00:50:54 +00:00
SPEARS, DUSTIN (ds443n) 613dd89fb0 Add ttlSecondsAfterFinished field to cronjob
Adding ttlSecondsAfterFinished option to the chart for db
clean up cronjob
Add history limit options
Add concurrency policy to forbid

Change-Id: I431a9a3692fee36f77c6037031965e58c2c343c0
2023-06-27 16:20:22 -04:00
Sergiy Markin 375abedb8a Drydock updates
This PS delivers the following updates:
- fixed sample config and policy files generation
- rolled back chart version incremention

Change-Id: I0a7145afd8c81e2bbf36d9437d4eff3c0354667a
2023-04-28 18:39:25 +00:00
Sergiy Markin d00eaf0303 Drydock focal related upgrades
This PS implements the following changes:
- switches freeze approach to requirements-direct.txt and
  requirements-frozen.txt files
- adjusts code tabulation style according to  yapf recommendations
- replaces deprecated usage of responce.body attribute with
  responce.text
- fixes integration tests in controlled by Makefile + tox
- uplifts Helm to v3.9.4

Change-Id: I751db72eb8f670825382f11a36657112faeb169a
2023-04-26 22:32:49 +00:00
Sergiy Markin 415a8b52c5 [focal] Python modules sync with Airship project
- uplifted some python modules
- fixed tox4 requirements
- added focal build node as a default one
- added bindep.txt and bindep role to playbooks and docker image build process
- changes Makefile to reflect GoLang and dependency management changes
- upgraded Helm to v3 for chart build process
- uplifted postgresql version to 14.6
- fixed deprecated falcon.API - replaced with falcon.APP
- fixed upstream docker image publishing process

Change-Id: I307d72bb7680f6f5c71e42ad30666cf786420460
2023-04-08 08:20:22 +00:00
SPEARS, DUSTIN (ds443n) a171f3c7a5 Add postgresql retention cronjob
Adding cronjob to purge the drydock DB based on retention day value. Additionally adding drydock API endpoint for purging the tasks and result_message tables and running vacuum full on drydock DB.

Change-Id: Ibcce61ecdafa637ca3ffec654152060aae26d4b8
2022-05-18 10:55:26 -04:00
Sean Eagan c90fa60e2a Helm 3: Fix Job labels
See the dependency below for details.

Depends-On: https://review.opendev.org/c/openstack/openstack-helm-infra/+/811826
Change-Id: Id72e8c97603aa205ea1d071e6b99ad81513bacfc
2021-10-01 13:03:25 -05:00
DeJaeger, Darren (dd118r) 0b002d5b67 Add "labels" to Drydock deployment
Adding said label, that's already defined, to the deployment itself.
This will enable Armada to properly wait for certain percentages
of the deployment replicas to be ready prior to proceeding. Prior to
this change, there wasn't a way to select the Drydock deployment via
labels.

Change-Id: I7c5ed223d54213a1260c27485d0bfd493c09163f
2021-04-30 16:43:05 -04:00
Andrii Ostapenko 3cfe2c25c5
Change helm-toolkit dependency version to ">= 0.1.0"
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0

Change-Id: Ic115755eff68f419116b79102661e9fe1a7b1764
2020-09-24 19:42:45 -05:00
KHIYANI, RAHUL (rk0850) 63f7783da2 Implement helm-toolkit snippet to drydock pods/containers
This updates the drydock chart to include the pod
security context on the pod template.

This also adds the container security context to set
readOnlyRootFilesystem flag to true

Change-Id: Ibeb60d0b88f3519730b5b76996ab137c5af4f4f5
2020-07-02 15:59:14 +00:00
Phil Sphicas aa0cf8c8d0 Fix metadata labels for db-init and db-sync jobs
Corrects a recently introduced rendering error in the chart that
resulted in missing metadata labels for the drydock-db-init and
drydock-db-sync jobs.

https://review.opendev.org/#/c/724768

Change-Id: Ifa01bbc369a33ca3d5482c760a342d873736272e
2020-06-03 03:06:05 +00:00
DODDA, PRATEEK d862a5171f Enabling Apparmor for drydock init and test containers
Change-Id: I7aca8fd1dabc19603a33466175263c4a7f3fa36d
2020-05-26 18:11:39 -05:00
Prateek Dodda 67716a7841 Add Docker default AppArmor profile to drydock
Change-Id: I50be2f08e69123afbef136683134abffc4e44197
2020-02-13 10:42:28 -06:00
Drew Walters 9e51914bf3 charts: Update kubernetes-entrypoint img location
This change updates the location of the kubernetes-entrypoint image to
point to its new home in the airshipit namespace on quay.io [0]. The
stackanetes image is no longer maintained.

[0] https://quay.io/repository/airshipit/kubernetes-entrypoint

Depends-On: 8314c530305a7a14cbf72bf0c2e873e0d01c595c

Change-Id: I08db87c2f97c687bd87162e2f7eaf81abe882c31
Signed-off-by: Drew Walters <andrew.walters@att.com>
2019-10-21 14:53:28 +00:00
Hemanth Nakkina 576f1b0488 Use apps/v1 k8s controllers and add labels
Update apiversion for deployment to apps/v1
Add selector match labels to deployment

This patch is similar to https://review.opendev.org/#/c/638276/
These changes are required to install drydock helm chart on k8s 1.16.0

Change-Id: Ie9b7344fc94058a6212d09a9b96fe1b2b9d07b4e
2019-10-01 03:11:41 +05:30
Evgeny L 2e97bd5b72 Allow to configure service network policy
The patch introduces network policy configuration similar
to openstack-helm services. It allows users to configure
policies depending on the environment.

* Network policies are disabled by default.
* When enabled default policies allow all ingress and
  egress traffic (i.e. policy set to {}), this may be
  changed in future patch-sets.

Change-Id: I2705fcf1d322ed06b124811b4ab91bfdfbdeacf3
2019-09-20 19:52:03 +00:00
Zuul e2bb7cb0f9 Merge "Add release uuid annotation to POD spec" 2019-06-26 21:37:20 +00:00
Kumar, Nishant(nk613n) 143ce06d28 Add release uuid annotation to POD spec
Change-Id: I440af378a520c9fcab0985ea2ad0ba5dca2a0cf2
2019-06-25 14:51:26 +00:00
Dejaeger, Darren (dd118r) 73f6a2188b Add node selector to test pods
This PS looks to add a node selector into the test pod's spec.

Change-Id: I3eb549dbb48dba388896bf489a1a9151dfc56919
2019-06-20 10:37:06 -04:00
Dmitrii Kabanov b2e69ed678 Add possibility to check response code in auth test
This PS allows to check the response code and if it's equal
to 22, the test will be considered as successful.

Change-Id: I3867c551be5785488248e956e6f8a124477232f5
2019-04-12 11:58:55 -07:00
Rahul Khiyani 9b5c1d493e Drydock: Add pod/container security context
This updates the drydock chart to include the pod
security context on the pod template. This changes the pod's
user from root to the nobody user instead

This also adds the container security context to set
allowPrivilegeEscalation to false and readOnlyRootFilesystem to true

Change-Id: I0882622e672e5918da82b58b76697b8974cf0b16
2019-03-29 17:33:37 +00:00
Nishant Kumar ebe95d1875 Use helm toolkit for DB initialization
Depends-On: https://review.openstack.org/#/c/635348/

Change-Id: I55fa7a08b919581552dd512316fc42581762a6a4
2019-03-04 18:02:37 +00:00
Hemanth Nakkina da0c7e831e Add Redfish as OOB driver
This patch implements Refish as new OOB driver for Drydock.
All the existing Drydock Orchestrator actions are implemented.

Change-Id: I31d653fb41189a18c34cfafb0f490ca4f4d661b5
2018-12-08 21:19:30 +05:30
Scott Hussey a2418241ce (fix) Use endpoint for MAAS URL
- Instead of forcing a user to provide the full URL for the MAAS API,
  instead use the endpoints pattern and render the URI via HTK templates.
- Add secret name to chart to support HTK ingress
- Install libyaml to take advantage of faster parsing by pyyaml
- Add exception logging when node compiling fails.
- Add caching of parsed design to gain efficiency
- Add TLS certificate secret for use by the ingress document

Change-Id: I5a2dbc415483c336d38d67edcebdfc5812f7bb0c
2018-11-27 12:04:38 -06:00
Zuul c38a38fa4c Merge "Add release uuid to pods and rc objects (drydock)" 2018-10-04 21:27:18 +00:00
Roman Gorshunov ec17405606 Fix: various documentation and URL fixes
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes

Change-Id: I64fb508a3ff72258d82c795193e918f2945deb29
2018-09-26 08:21:14 +00:00
Scott Hussey 70e5f095d6 (fix) Add nodeSelector to Drydock Pod
Add ingress support

- Add nodeSelectors to the PodSpec for the Drydock API
  pod

- Physical nodes bootstrapping need to access the Drydock
  API to retrieve the bootactions that should be executed during
  deployment. This moves that access from a nodeport to ingress.

Change-Id: I3db41932c567cc85e89ad003389b7a019a10715f
2018-09-17 12:45:24 -05:00
Matt McEuen d8f4a1fbe3 Add release uuid to pods and rc objects (drydock)
This PS adds the ability to attach a release uuid to pods and rc
objects as desired.  This can be used, for example, to force an
artificial manifest change in CICD scenarios, for upgradability
testing purposes.

Change-Id: I784c0ad26c34338c9acda3dbe271f2139ba3f1f9
2018-09-14 18:19:36 -05:00
Matt McEuen 2bfb9c59cd Uplift Drydock to master Helm-Toolkit
Update the Drydock chart to be compatible with the latest Helm-Toolkit,
including taking advantage of some HTK manifest generation functionality
and updating the helm_tk.sh script to pull down master
openstack-helm-infra. Also update the default drydock image to point
to the current airshipit master rather than the old attcomdev in quay.

Change-Id: I9a818ae054361749ce16e9a6213fbeed82581f02
Co-Authored-By: Pete Birley <pete@port.direct>
Signed-off-by: Pete Birley <pete@port.direct>
2018-09-14 11:59:29 -05:00
Matt McEuen b8d4779d58 Update Keystone API ports in Drydock chart
This change modifies the internal Keystone API port in the
Drydock chart from 80 to 5000 and removes the default admin port
to match the Keystone chart provided by OpenStack-Helm.

Change-Id: I46fa68ffad5015a040b0022776459208be511881
2018-08-27 15:49:05 -05:00
Aaron Sheffield 69a6f80031 Add test pods labels.
- Uses helm toolkit to add labels to test pods.

Change-Id: I60009c3c502c76d77f9edbf64d93118079f94253
2018-07-11 08:05:10 -05:00
Scott Hussey 53f6ccbfb7 Clean up gates
- Remove OSH-based gate as it is not relevant at this point
- Add entries for unit testing, linting and security scans
- Some trivial changes to trigger all gates

Change-Id: Icb2731573b45f3824cde8cb44ef64e9d04c017bc
2018-05-25 16:36:45 -05:00
Scott Hussey 97f9fbd12b [Chart] SSH private key support
- Support optionally mounting a SSH private key
  to allow Drydock to interact with remote hosts via
  ssh (e.g. virsh)

Change-Id: Ib83bc53a46497af6d05f4d87595f1000d3178ec0
2018-05-01 21:18:13 +00:00
Scott Hussey 12650c7909 [398387] Resource limits for ks_service job
- Add resource limits to the Pod spec for the
  ks_service job

Change-Id: I107d8108298282431fe767d7e05aad7ae9486eec
2018-03-27 16:45:29 -05:00
Aaron Sheffield 1fd8bb9812 Migrating Health Check to UCP Standards
- Added a second health check endpoint /extended to get additional data.
- Conforms health check response with UCP standards.

Change-Id: I9e3ac27ec7e536bb18201f1a4642490725a8062c
2018-03-13 16:37:39 -04:00
Pete Birley b8e5d932d3 Images: depreciate kolla heat-engine image for LOCI
This PS deprecates the kolla heat-engine image for it's LOCI
replacement.

Change-Id: Ifdab7b23ae15b5f80ee728653dac54c07b5d279f
2018-03-11 19:32:56 -04:00
Scott Hussey 1804203ea1 Improve chart database configurability
- Support configured Postgres admin password
- Use secrets for database job environment setup

Change-Id: Icf7ceb4efb1b1bf976ca36e4fdd21b9b7990bc83
2018-02-01 20:32:21 -05:00
Anthony Lin 253c6f6bb4 RBAC: Update serviceaccount and k8s rbac for drydock
This patch set brings the drydock chart to be inline with OSH* RBAC
approach used in [0] and [1].

[0] https://review.openstack.org/#/c/526464/52
[1] https://review.openstack.org/#/c/529378/

Change-Id: Ia1e5510605e38068e30e966cdd7d030154f5e6f4
2018-01-02 11:10:47 -05:00
Scott Hussey ae87cd1714 Update image and chart mgmt
NOTE: This has become a monolithic commit to get gate
      settings/scripts in place for CI

- Add Makefile with UCP standard entrypoints
- Move Dockerfile into images/drydock per UCP standards
- Add values.yaml entries for uWSGI threads and workers
- Add environment variables to chart Deployment manifest
  for uWSGI thread and workers
- Add threads and workers specification to uWSGI commandline
  in entrypoint
- Test that the Drydock API is responding
- Test that the Drydock API rejects noauth requests
- Fix Makefile utility script to work behind a proxy

Correct task success voting

Some tasks were incorrectly considered partial_success even when
no failure occurred.

- Network configuration erroneously marked messages as errors
- Update result propagation logic to only use the latest retry

The deploy_nodes task ended as incomplete due to a missing
subtask assignment

Also added a node check step to prepare_nodes so that nodes that
are already under provisioner control (MaaS) are not IPMI-rebooted.

Tangential changes:
- added config item to for leadership claim interval
- added some debug logging to bootaction_report task
- fix tasks list API endpoint to generate valid JSON

Improve task concurrency

When tasks are started with a scope of multiple nodes,
split the main task so each node is managed independently
to de-link the progression of nodes.

- Split the prepare_nodes task
- Begin reducing cyclomatic complexity to allow for
  better unit testing
- Improved tox testing to include coverage by default
- Include postgresql integration tests in coverage

Closes #73

Change-Id: I600c2a4db74dd42e809bc3ee499fb945ebdf31f6
2017-12-15 15:33:14 -06:00
portdirect 68a6862857 Images: Remove Kolla-Toolbox image as not required
This ps removes the last references to Kolla-Toolbox which is not
required for keystone management jobs.

Change-Id: Ia02b018889964b0dc1b324fd053c16ed6c995a2f
2017-11-16 12:08:17 -05:00
Scott Hussey 4ddaff690a Update to latest dep_check image
Update the Drydock chart ot use the v.0.2.1 Stackanetes
entrypoint container for dependency checking

Change-Id: Ic136ccc8901fd7bcdd2c6615845765c1338c8ebc
2017-11-15 09:56:04 -06:00
Scott Hussey 111cd7a129 Compat with HTK images structure change
Helm-toolk changed the images structure to include a
tags stanza. Make this chart compatible.

Change-Id: I2111af7e40e6f5490c8ad9d68fbfb97e3e8fea3e
2017-10-27 14:14:47 -04:00
Scott Hussey 1efddeebd3 Remove Promenade generated volume
The post-refactor integration between promenade
and drydock doesn't require (or support) this
volume

Change-Id: Ieca8c4ada7d2469aea17214a163f86ba67a05577
2017-10-27 12:34:56 -04:00
Larry Rensing 50277a63ec Drydock chart
This PS migrates the Drydock chart into this repo.

Update chart with input from previous repo

- Remove default secret names for Keystone jobs
- Use endpoints section for defining ports in service manifest
- Use manifests section for enabling all deployed manifests

Add DB integration

- Introduction of postgresql endpoint for Postgresql
- Addition of db_init and db_sync jobs
- Addition of db-init.sh and db-sync.sh scripts
- Convert conf file to use helm-toolkit templater
- Add database connect string to rendered conf file

Fix copyright notices for AT&T compliance

Change-Id: I1676a41ddbbd05c38f68b2b787924fc973411413
2017-10-27 12:34:38 -04:00