Commit Graph

66 Commits

Author SHA1 Message Date
Sergiy Markin c51e574abd Fix deckhand-api dependences
Also this PS bumps up k8s to 1.29 for gates

Change-Id: Ic7f5dcd681875bc35663a53b2a5b052c20cc3f7a
2024-03-02 00:00:47 +00:00
Sergiy Markin 8d055a0aa9 Deckhand updates
This PS makes the following changes:

- uses deploy-k8s.sh from treasuremap
- makes sure the airskiff-deploy playbook is using 80Gb partition if
  available
- adds available security updates to docker images

Change-Id: I0f330cb15ec32b12703f0bc6620b3f3c797a25bb
2023-08-25 17:57:43 +00:00
Sergiy Markin 03f6932e16 Deckhand updates
This PS delivers the following updates:
- fixed sample config and policy files generation in tox
- rolled back chart version incremention back to 0.2.0

Change-Id: I509030319a724b18bb21f45f7ede7c07ab18e894
2023-04-28 22:18:29 +00:00
Sergiy Markin ac4edb0c64 [focal] Deckhand project updates
- adjusted .gitignore to keep fresh egg-info and omit build artifacts
- fresh egg-info data is needed for promenade that depends on Deckhand
- restored deckhand-functional-uwsgi-py38 gate
- restored deckhand-integration-uwsgi-py38 gate
- made deckhand-airskiff-deployment gate voting ( treasuremap project
  has been updated)
- removed bionic gates
- updated focal dockerfile
- added more binary deps into bindep.txt
- updated deckhand chart values to latest images - focal and wallaby
- fixed python code to compy with CVE's found by fresh version of bandit
- implemented pip freeze approach
- added tox -e freeze profile to manage it
- requirements-frozen.txt is now main file with requirements
- requirements-direct.txt is the file to control deps
- updated setup.cfg to adjust to newer version of setuptools
- fixed airskiff-deploy gate
- fixed docker-image-build playbook to restore Quay repo image publish
- updated other playbooks to include roles from zuul/base-jobs in order
  to setup build hosts properly
- removed workaround with hardcoded dns resolver ip 10.96.0.10 as it
  became obsolette due to recent fix in openstack-helm-infra
- adjusted tools/whitespace-linter.sh script
- tox.ini has been brought to compliance with tox4 requirements
- replaced str() calls with six.text_type() according to D325 Deckhand specific
  commandment from Hacking.rst
- locked python-barbicanclient version with 5.2.0 because of breaking
  changes in the upper versions

Change-Id: I1cd3c97e83569c4db7e958b3400bdd4b7ea5e668
2023-04-20 19:39:43 +00:00
Wahlstedt, Walter (ww229g) 70aa35a396 update to focal and python 3.8
update dockerfile for python deckhand install
add deckhand version to chart 1.0
add chart version 0.2.0
update all packages to latest in requirements.txt
update zuul jobs for focal and python 3.8
remove zuul job functional-uwsgi-py38 in favor of functional-docker-py38
update tox config
typecast to string in re.sub() function
add stestr to test-requirements.txt
add SQLAlchemy jsonpickle sphinx-rtd-theme stestr to requirements.txt
deprecated function: BarbicanException -> BarbicanClientException
fix mock import using unittest
fix import collections to collections.abc
fix for collections modules for older than python 3.10 versions.
deprecated function: json -> to_json
deprecated function:  werkzeug.contrib.profiler ->
    werkzeug.middleware.profiler
deprecated function: falcon.AIP -> falcon.App
deprecation warning: switch from resp.body to resp.text
rename fixtures to dh_fixtures because there is an imported module
    fixtures
switch from stream.read to bounded_stream.read
deprecated function: falcon process_response needed additional parameter
deprecated function: falcon default_exception_handler changed parameter
    order
move from MagicMock object to falcon test generated object to fix
    incompatability with upgraded Falcon module.
Adjust gabbi tests to fix incompatability with upgraded DeepDiff module
update Makefile to execute ubuntu_focal
update HTK (helmtoolkit)
unpin barbican to pass integration tests
Use helm 3 in chart build.
    `helm serve` is removed in helm 3 so this moves
    to using local `file://` dependencies [0] instead.

Change-Id: I180416f480edea1b8968d80c993b3e1fcc95c08d
2023-02-24 10:51:57 -05:00
Sean Eagan 4ba85d2432 Helm 3: Fix Job labels
See the dependency below for details.

Depends-On: https://review.opendev.org/c/openstack/openstack-helm-infra/+/811826
Change-Id: I9f0d9d99b798a41f8d8ac841e3b00e7d8af40ff3
2021-10-01 11:28:22 -05:00
Andrii Ostapenko 1b396f220a Change helm-toolkit dependency version to ">= 0.1.0"
Since we introduced chart version check in gates, requirements are not
satisfied with strict check of 0.1.0

Change-Id: I547a7f0e6106fee2f560b62671e1eceb312e5c4e
2020-09-25 03:35:37 +00:00
Ahmad Mahmoudi 0545625da9 Scaling deckhand uwsgi workers
Updated obsolete uwsgi default configuration parameters for better
performance.
Increased number of worker threads to increase performance.
Uplifted uwsgi to the latest for bug fixes since 2018.

For more info please see:
https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html

Change-Id: Ifedb9c6279e64be86deb6ec375810c5ecf97958a
2020-08-05 22:05:57 +00:00
Zuul 033b278fd0 Merge "Add configmap-hash annotations for deckhand" 2020-07-17 16:10:30 +00:00
DODDA, PRATEEK REDDY 3a3657b6e5 Add configmap-hash annotations for deckhand
Adds configmap-hash annotations to the job-db-init and job-db-sync
for configmap-bin and configmap-etc.

These annotations ensure that if configmaps change, the pods
are redeployed according to their upgrade strategy.

Change-Id: I8ff282d8279c934590d5308e9c26efaf65685e2b
2020-07-06 10:41:28 -05:00
KHIYANI, RAHUL (rk0850) 2f9e0d7601 Implement helm-toolkit snippet to deckhand pods/containers
This updates the deckhand chart to include the pod
security context on the pod template.

This also adds the container security context to set
readOnlyRootFilesystem flag to true

Change-Id: I9bfd889b163e280cf17c4e7b49974a077e889f2f
2020-07-02 09:32:18 -05:00
DODDA, PRATEEK 897f596185 Enabling Apparmor profile to deckhand init containers
Remove OSH Authors copyright

The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: Ib0b21b33d8bf91ea6da4c2421cc81355cf2b23b1
2020-06-26 11:08:56 -05:00
Prateek Dodda 68a3ad1f57 Add Docker default AppArmor profile to deckhand
Depends on https://review.opendev.org/#/c/707475/

Change-Id: I320d02bd987bd8af4448694db2f193f83b010a0f
2020-02-12 22:09:01 +00:00
Hemanth Nakkina abbbf3d6b2 Use apps/v1 k8s controllers and add labels
Update apiversion for deployment to apps/v1
Add selector match labels to deployment

This patch is similar to https://review.opendev.org/#/c/638276/
These changes are required to install deckhand helm chart on k8s 1.16.0

Change-Id: Ifca6020dee953252629f42a1b04f384e959c0916
2019-10-01 03:10:45 +05:30
Evgeny L 2a7c5977cf Allow to configure service network policy
The patch introduces network policy configuration similar
to openstack-helm services. It allows users to configure
policies depending on the environment.

* Network policies are disabled by default.
* When enabled default policies allow all ingress and
  egress traffic (i.e. policy set to {}), this may be
  changed in future patch-sets.

Change-Id: I9ae69e84991f16891830fb7e044a06985eca9d0f
2019-09-20 19:50:43 +00:00
Zuul 134c55805b Merge "Add release uuid annotation to POD spec" 2019-06-26 11:26:52 +00:00
Kumar, Nishant(nk613n) 24b3ad806c Add release uuid annotation to POD spec
Change-Id: I8a0518660eca85e139e0316fa6c3f5be0302a67e
2019-06-25 14:54:38 +00:00
Dejaeger, Darren (dd118r) 45d87614d4 Add node selector to test pod
This PS looks to add a node selector into the test pod's spec.

Change-Id: I4e42627d01f2e0b14c65774895c3b10ad1b47d87
2019-06-20 11:25:16 -04:00
anthony.bellino aeefd9d121 Add pod anti-affinity to Deckhand
This PS adds pod anti-affinity to deckhand pods,
so that the scheduler can constrain pods against labels on other pods
running on the node. The default soft rule is in place so that if the
scheduler can’t satisfy the requirement, the pod will still
be scheduled.

Change-Id: Icab673726d0473662ccf45c4c576fe20912a1260
2019-06-11 01:36:34 +00:00
Zuul 81b2badca5 Merge "Implement Security Context for Deckhand" 2019-04-19 14:14:41 +00:00
pd2839 20c1ae4edf Implement Security Context for Deckhand
Implement container and pod level security context for the following
Deckhand resources:

- Deckhand server deployment

Change-Id: I23cd742cc3b76b4e5de67d3b8bb195ec3899fc0f
2019-04-11 23:47:11 +00:00
Crank, Daniel (dc6350) 661350777b Log client-id in UCP API endpoints
Adds functionality to read context marker and end-user
from request headers and log that information where
available, to aid in tracing transactions that span
multiple Airship components.

Change-Id: I35c9e56f84f29420c4f3c081453cb81aa892fa7d
2019-04-05 09:55:09 +00:00
Zuul 1a9531c3fe Merge "Use helm-toolkit for DB initialization" 2019-03-05 19:21:06 +00:00
Nishant Kumar 97da3c59da Use helm-toolkit for DB initialization
Depends-On: https://review.openstack.org/#/c/635348/
Change-Id: Ie9328755d1cb2f27bfa83c0f5a6568c97befe9bf
2019-03-04 15:12:39 +00:00
anthony.bellino daab07a949 [chart] Enable liveness probe in DH
This is to try to address stuck deckhand-api ponds that never
went to error state in an attempt to self-jolt the pod again.

Change-Id: I70bf57dde5d696bddc68caab2f54826803d82d28
2019-02-28 21:10:46 +00:00
Roman Gorshunov d41e5a44ca Fix: various documentation and URL fixes
1) UCP -> Airship
2) readthedocs.org -> readthedocs.io (there is redirect)
3) http -> https
4) attcomdev -> airshipit (repo on quay.io)
5) att-comdev -> openstack/airship-* (repo on github/openstack git)
6) many URLs have been verified and adjusted to be current
7) no need for 'en/latest/' path in URL of the RTD
8) added more info to some setup.cfg and setup.py files
9) ucp-integration docs are now in airship-in-a-bottle
10) various other minor fixes

Change-Id: I12b2fa8fbec37a483a0ad50382e08f51ed97533a
2018-09-25 11:44:57 +02:00
Zuul eaf866d44e Merge "Adding api for revisions deep diffing" 2018-09-18 20:16:15 +00:00
pallav 9345035522 Adding api for revisions deep diffing
GET /revisions/{{revision_id}}/deepdiff/{{comparison_revision_id}}

 - Added deepdiff api for generating diff between
   two rendered documents.
 - Deep diffing for data and metadata
 - Refactor diff functions
 - Client update
 - Added unit testcases
 - Added funtional testcases
 - Doc update

Change-Id: Ib60fa60a3b33e9125a1595a999272ca595721b38
2018-09-17 17:01:34 +05:30
Zuul 4973cdf97c Merge "Add release uuid to pods and rc objects (deckhand)" 2018-09-16 21:49:50 +00:00
Matt McEuen 9b53244a10 Add release uuid to pods and rc objects (deckhand)
This PS adds the ability to attach a release uuid to pods and rc
objects as desired.  This can be used, for example, to force an
artificial manifest change in CICD scenarios, for upgradability
testing purposes.

Change-Id: I69d7dfebe457423c58dc297ec84d02ca62230020
2018-09-14 15:26:57 -05:00
Tin Lam 7b862e05d8 feat(tls): add tls to ingress for public endpoint
This patch set adds TLS on overridden fqdns for public endpoints for
airship-deckhand. As cacerts are not loaded into the containers, this
only supports certificates that can be externally verified.

Change-Id: I41606129c8d59dfedcb648f5390985a31b690eec
2018-09-13 05:50:14 +00:00
Bryan Strassner a0f00013b5 [Trivial Fix] Change b46enc to b64enc in chart
Change-Id: I58114a678d30e55ce12b7990f683a6b773efb171
2018-08-29 09:47:05 -05:00
Zuul 2316143753 Merge "Update Keystone API ports in Deckhand chart" 2018-08-27 20:34:12 +00:00
Drew Walters d23584869f Update Keystone API ports in Deckhand chart
This change modifies the internal Keystone API port in the Deckhand
chart from 80 to 5000 and removes the default admin port to match
the Keystone chart provided by OpenStack-Helm.

Change-Id: I3861e551ac9ad9fb008e8caf3cfa892ecd4fc657
2018-08-23 22:12:40 +00:00
Pete Birley 36c30d47bf Chart: Use k8s secret to store config
This PS moves the chart to use secrets to store potentially sensitive
config information.

Depends-On: https://review.openstack.org/#/c/593732

Change-Id: I884a68b379beefa3aa73018613ac37c0f3ee089d
Signed-off-by: Pete Birley <pete@port.direct>
2018-08-22 20:52:17 -05:00
Matt McEuen 07186243de Update Deckhand for latest HTK
This updates Deckhand to be compatible with the current
OpenStack-Helm Helm Toolkit.  This includes:
- Using HTK manifest templates
- Refactoring values.yaml structure
- Some other small cleanup

Change-Id: Ib7c2451b46fab20935edb1c768ac56cc6353aa16
2018-08-09 02:53:13 +00:00
Aaron Sheffield 73b9d28def Add test pods labels.
- Uses helm toolkit to add labels to test pods.

Change-Id: Ib931b3573e33896ad3826b05844271bde7b6ebd0
2018-07-11 08:47:48 -05:00
Zuul b8eac3aae0 Merge "chore(image): update image" 2018-06-05 15:15:15 +00:00
melissaml e86fa5300c fix typos in documentation
Change-Id: Ie0db25163554ce6532503eb998a8f3c7f349df11
2018-05-23 20:59:34 +08:00
Tin Lam 2bee79d794 chore(image): update image
This patch set updates the kubernetes-entrypoint image from
v0.3.0 to v0.3.1.

Change-Id: Ic278b8b91e3034173dfad805d1dc5af27e96c43e
Signed-off-by: Tin Lam <tin@irrational.io>
2018-05-19 20:48:08 -05:00
Anthony Lin a004c7a19e Update Deckhand API Pod Labels
As part of ongoing effort to update the "application" and
"component" labels for the UCP components, there is a need
to align with the convention. We will update the label for
the deckhand API pod in this case.

Also updated helm_tk.sh to point to openstack-helm-infra for
reference to helm-toolkit as helm-toolkit has been removed
from the openstack-helm repo [0]

[0] https://review.openstack.org/#/c/558065/

Change-Id: I753c4ce653790250b79986c670224d0962f7676f
2018-05-14 04:54:55 +00:00
Felipe Monteiro 111018ce24 [chart] Remove liveness probe to stop DH pod from being killed
This is to stop the DH pod from being killed in production whenever
DH receives multiple concurrent requests from another service,
causing all its threads to become occupied with servicing those
requests, causing the liveness probe to fail, causing the DH pod
to be killed. This is highly undesirable and as a temporary
workaround we will drop the liveness probe altogether.

This partially reverts I1a1c107706862431e53668a864db622499e63c6f
Additional reading: Id2d4deaaf8bf73d6df4639810e6dee3acf79b05c

Change-Id: Ic81c0c1d6e3cd3ab3b326054b9c882962d240968
2018-05-09 14:48:23 +00:00
Anthony Lin c29ad4406b Change name of Deckhand Container
We will align the name with the rest of the UCP components, i.e.
change it from 'deckhand' to 'deckhand-api'

Change-Id: I4c65ac1e6371ffa80fd8b42cbe979d71b93e99c7
2018-04-19 23:25:20 -04:00
Bryan Strassner 5f1fbbee3c [396582] Add alembic support to Deckhand
Updates Deckhand to use alembic to manage database upgrades.
Moves from creating tables at startup of Deckhand to the
db-sync job.

Change-Id: I6f4cb237fadc46fbee81d1c33096f48a720f589f
2018-04-06 23:30:16 -04:00
Bryan Strassner c962eeb975 [fix] Extend liveness and readiness check times
Under load, Deckhand will fail liveness checks with a 1 second timeout.
This Patchset extends the timout to 10 seconds and spaces the period
between checks to 20 seconds.
Adds labels to keystone user job.

Change-Id: Id2d4deaaf8bf73d6df4639810e6dee3acf79b05c
2018-04-04 15:31:46 -05:00
Felipe Monteiro 5c9efa9d74 Enable multiple threads, disabled muliple workers
This sets multiple threads in Deckhand's chart config (4)
and set workers to just 1.

Deckhand's database is not configured to work with multiprocessing.
Currently there is a data race on acquiring shared SQLAlchemy
engine pooled connection strings when workers > 1. As a
workaround, we use multiple threads but only 1 worker. For more
information, see:

https://github.com/att-comdev/deckhand/issues/20

Change-Id: I60adeffff5461fdda957124232bc5a606baae413
2018-04-02 12:38:20 -04:00
Tin Lam 225638711b Update kubernetes-entrypoint
This patch set updates the kubernetes-entrypoint image to version
3.0.3 inline of the chart used in OpenStack-Helm in [0]. This allows
the chart to use pod dependencies.

[0] https://review.openstack.org/#/c/554268/

Change-Id: I06c874bbe1b39271a94ce1c418c8b1317080dac5
Signed-off-by: Tin Lam <tin@irrational.io>
2018-04-02 10:22:30 -04:00
Anthony Lin f179167e9a [398395] Update Indentation for Resource limits
Update indentation for 'test-deckhand-api'

Change-Id: I0c2ba99c6743f5e7b7cdf26e74479a4ebfe744a1
2018-03-28 13:45:36 +00:00
Anthony Lin a29741e3a7 Deckhand API - Liveness and Readiness Probes
This patch set does the following to enhance health/status checks
on the deckhand-api pod:

1) Add Liveness Probe
2) Update Readiness Probe

Change-Id: I1a1c107706862431e53668a864db622499e63c6f
2018-03-13 15:31:52 +00:00
Pete Birley dc69b7c7b2 Images: depreciate kolla heat-engine image for LOCI
This PS deprecates the kolla heat-engine image for it's LOCI
replacement.

Change-Id: I982fcf0d79db4f564996f76c429b6e43ac6b8fcc
2018-03-09 10:16:54 -05:00