Commit Graph

45 Commits

Author SHA1 Message Date
Sean Eagan 4ba85d2432 Helm 3: Fix Job labels
See the dependency below for details.

Depends-On: https://review.opendev.org/c/openstack/openstack-helm-infra/+/811826
Change-Id: I9f0d9d99b798a41f8d8ac841e3b00e7d8af40ff3
2021-10-01 11:28:22 -05:00
Ahmad Mahmoudi 0545625da9 Scaling deckhand uwsgi workers
Updated obsolete uwsgi default configuration parameters for better
performance.
Increased number of worker threads to increase performance.
Uplifted uwsgi to the latest for bug fixes since 2018.

For more info please see:
https://uwsgi-docs.readthedocs.io/en/latest/ThingsToKnow.html

Change-Id: Ifedb9c6279e64be86deb6ec375810c5ecf97958a
2020-08-05 22:05:57 +00:00
Zuul 033b278fd0 Merge "Add configmap-hash annotations for deckhand" 2020-07-17 16:10:30 +00:00
DODDA, PRATEEK REDDY 3a3657b6e5 Add configmap-hash annotations for deckhand
Adds configmap-hash annotations to the job-db-init and job-db-sync
for configmap-bin and configmap-etc.

These annotations ensure that if configmaps change, the pods
are redeployed according to their upgrade strategy.

Change-Id: I8ff282d8279c934590d5308e9c26efaf65685e2b
2020-07-06 10:41:28 -05:00
KHIYANI, RAHUL (rk0850) 2f9e0d7601 Implement helm-toolkit snippet to deckhand pods/containers
This updates the deckhand chart to include the pod
security context on the pod template.

This also adds the container security context to set
readOnlyRootFilesystem flag to true

Change-Id: I9bfd889b163e280cf17c4e7b49974a077e889f2f
2020-07-02 09:32:18 -05:00
DODDA, PRATEEK 897f596185 Enabling Apparmor profile to deckhand init containers
Remove OSH Authors copyright

The current copyright refers to a non-existent group
"openstack helm authors" with often out-of-date references that
are confusing when adding a new file to the repo.

This change removes all references to this copyright by the
non-existent group and any blank lines underneath.

Change-Id: Ib0b21b33d8bf91ea6da4c2421cc81355cf2b23b1
2020-06-26 11:08:56 -05:00
Prateek Dodda 68a3ad1f57 Add Docker default AppArmor profile to deckhand
Depends on https://review.opendev.org/#/c/707475/

Change-Id: I320d02bd987bd8af4448694db2f193f83b010a0f
2020-02-12 22:09:01 +00:00
Hemanth Nakkina abbbf3d6b2 Use apps/v1 k8s controllers and add labels
Update apiversion for deployment to apps/v1
Add selector match labels to deployment

This patch is similar to https://review.opendev.org/#/c/638276/
These changes are required to install deckhand helm chart on k8s 1.16.0

Change-Id: Ifca6020dee953252629f42a1b04f384e959c0916
2019-10-01 03:10:45 +05:30
Evgeny L 2a7c5977cf Allow to configure service network policy
The patch introduces network policy configuration similar
to openstack-helm services. It allows users to configure
policies depending on the environment.

* Network policies are disabled by default.
* When enabled default policies allow all ingress and
  egress traffic (i.e. policy set to {}), this may be
  changed in future patch-sets.

Change-Id: I9ae69e84991f16891830fb7e044a06985eca9d0f
2019-09-20 19:50:43 +00:00
Zuul 134c55805b Merge "Add release uuid annotation to POD spec" 2019-06-26 11:26:52 +00:00
Kumar, Nishant(nk613n) 24b3ad806c Add release uuid annotation to POD spec
Change-Id: I8a0518660eca85e139e0316fa6c3f5be0302a67e
2019-06-25 14:54:38 +00:00
Dejaeger, Darren (dd118r) 45d87614d4 Add node selector to test pod
This PS looks to add a node selector into the test pod's spec.

Change-Id: I4e42627d01f2e0b14c65774895c3b10ad1b47d87
2019-06-20 11:25:16 -04:00
anthony.bellino aeefd9d121 Add pod anti-affinity to Deckhand
This PS adds pod anti-affinity to deckhand pods,
so that the scheduler can constrain pods against labels on other pods
running on the node. The default soft rule is in place so that if the
scheduler can’t satisfy the requirement, the pod will still
be scheduled.

Change-Id: Icab673726d0473662ccf45c4c576fe20912a1260
2019-06-11 01:36:34 +00:00
pd2839 20c1ae4edf Implement Security Context for Deckhand
Implement container and pod level security context for the following
Deckhand resources:

- Deckhand server deployment

Change-Id: I23cd742cc3b76b4e5de67d3b8bb195ec3899fc0f
2019-04-11 23:47:11 +00:00
Zuul 1a9531c3fe Merge "Use helm-toolkit for DB initialization" 2019-03-05 19:21:06 +00:00
Nishant Kumar 97da3c59da Use helm-toolkit for DB initialization
Depends-On: https://review.openstack.org/#/c/635348/
Change-Id: Ie9328755d1cb2f27bfa83c0f5a6568c97befe9bf
2019-03-04 15:12:39 +00:00
anthony.bellino daab07a949 [chart] Enable liveness probe in DH
This is to try to address stuck deckhand-api ponds that never
went to error state in an attempt to self-jolt the pod again.

Change-Id: I70bf57dde5d696bddc68caab2f54826803d82d28
2019-02-28 21:10:46 +00:00
Zuul 4973cdf97c Merge "Add release uuid to pods and rc objects (deckhand)" 2018-09-16 21:49:50 +00:00
Matt McEuen 9b53244a10 Add release uuid to pods and rc objects (deckhand)
This PS adds the ability to attach a release uuid to pods and rc
objects as desired.  This can be used, for example, to force an
artificial manifest change in CICD scenarios, for upgradability
testing purposes.

Change-Id: I69d7dfebe457423c58dc297ec84d02ca62230020
2018-09-14 15:26:57 -05:00
Tin Lam 7b862e05d8 feat(tls): add tls to ingress for public endpoint
This patch set adds TLS on overridden fqdns for public endpoints for
airship-deckhand. As cacerts are not loaded into the containers, this
only supports certificates that can be externally verified.

Change-Id: I41606129c8d59dfedcb648f5390985a31b690eec
2018-09-13 05:50:14 +00:00
Bryan Strassner a0f00013b5 [Trivial Fix] Change b46enc to b64enc in chart
Change-Id: I58114a678d30e55ce12b7990f683a6b773efb171
2018-08-29 09:47:05 -05:00
Pete Birley 36c30d47bf Chart: Use k8s secret to store config
This PS moves the chart to use secrets to store potentially sensitive
config information.

Depends-On: https://review.openstack.org/#/c/593732

Change-Id: I884a68b379beefa3aa73018613ac37c0f3ee089d
Signed-off-by: Pete Birley <pete@port.direct>
2018-08-22 20:52:17 -05:00
Matt McEuen 07186243de Update Deckhand for latest HTK
This updates Deckhand to be compatible with the current
OpenStack-Helm Helm Toolkit.  This includes:
- Using HTK manifest templates
- Refactoring values.yaml structure
- Some other small cleanup

Change-Id: Ib7c2451b46fab20935edb1c768ac56cc6353aa16
2018-08-09 02:53:13 +00:00
Aaron Sheffield 73b9d28def Add test pods labels.
- Uses helm toolkit to add labels to test pods.

Change-Id: Ib931b3573e33896ad3826b05844271bde7b6ebd0
2018-07-11 08:47:48 -05:00
Anthony Lin a004c7a19e Update Deckhand API Pod Labels
As part of ongoing effort to update the "application" and
"component" labels for the UCP components, there is a need
to align with the convention. We will update the label for
the deckhand API pod in this case.

Also updated helm_tk.sh to point to openstack-helm-infra for
reference to helm-toolkit as helm-toolkit has been removed
from the openstack-helm repo [0]

[0] https://review.openstack.org/#/c/558065/

Change-Id: I753c4ce653790250b79986c670224d0962f7676f
2018-05-14 04:54:55 +00:00
Felipe Monteiro 111018ce24 [chart] Remove liveness probe to stop DH pod from being killed
This is to stop the DH pod from being killed in production whenever
DH receives multiple concurrent requests from another service,
causing all its threads to become occupied with servicing those
requests, causing the liveness probe to fail, causing the DH pod
to be killed. This is highly undesirable and as a temporary
workaround we will drop the liveness probe altogether.

This partially reverts I1a1c107706862431e53668a864db622499e63c6f
Additional reading: Id2d4deaaf8bf73d6df4639810e6dee3acf79b05c

Change-Id: Ic81c0c1d6e3cd3ab3b326054b9c882962d240968
2018-05-09 14:48:23 +00:00
Anthony Lin c29ad4406b Change name of Deckhand Container
We will align the name with the rest of the UCP components, i.e.
change it from 'deckhand' to 'deckhand-api'

Change-Id: I4c65ac1e6371ffa80fd8b42cbe979d71b93e99c7
2018-04-19 23:25:20 -04:00
Bryan Strassner 5f1fbbee3c [396582] Add alembic support to Deckhand
Updates Deckhand to use alembic to manage database upgrades.
Moves from creating tables at startup of Deckhand to the
db-sync job.

Change-Id: I6f4cb237fadc46fbee81d1c33096f48a720f589f
2018-04-06 23:30:16 -04:00
Bryan Strassner c962eeb975 [fix] Extend liveness and readiness check times
Under load, Deckhand will fail liveness checks with a 1 second timeout.
This Patchset extends the timout to 10 seconds and spaces the period
between checks to 20 seconds.
Adds labels to keystone user job.

Change-Id: Id2d4deaaf8bf73d6df4639810e6dee3acf79b05c
2018-04-04 15:31:46 -05:00
Felipe Monteiro 5c9efa9d74 Enable multiple threads, disabled muliple workers
This sets multiple threads in Deckhand's chart config (4)
and set workers to just 1.

Deckhand's database is not configured to work with multiprocessing.
Currently there is a data race on acquiring shared SQLAlchemy
engine pooled connection strings when workers > 1. As a
workaround, we use multiple threads but only 1 worker. For more
information, see:

https://github.com/att-comdev/deckhand/issues/20

Change-Id: I60adeffff5461fdda957124232bc5a606baae413
2018-04-02 12:38:20 -04:00
Anthony Lin f179167e9a [398395] Update Indentation for Resource limits
Update indentation for 'test-deckhand-api'

Change-Id: I0c2ba99c6743f5e7b7cdf26e74479a4ebfe744a1
2018-03-28 13:45:36 +00:00
Anthony Lin a29741e3a7 Deckhand API - Liveness and Readiness Probes
This patch set does the following to enhance health/status checks
on the deckhand-api pod:

1) Add Liveness Probe
2) Update Readiness Probe

Change-Id: I1a1c107706862431e53668a864db622499e63c6f
2018-03-13 15:31:52 +00:00
Anthony Lin 11b08a46f0 Add helm test to Deckhand
Add basic helm test to Deckhand

Change-Id: I9ab7130e3ad6771488ce26054adc72a60d6c7f40
2018-03-08 17:50:00 +00:00
Steve Wilkerson a792d35ad1 Add resource declaration to deckhand job-ks-service chart template
This adds resource declarations to the keystone service job
container for deckhand. It also updates .gitignore to specifically
exclude tarballs and requirements lock files instead of all
contents in charts/deckhand

Change-Id: Ide80834cf636dfc96d207a19f2248fd75aea5ca7
2018-03-03 22:20:42 -05:00
Scott Hussey 9f2a0fb347 Collect profile data on DH requests
- Seeing issues with a lot of Drydock
  requests timing out and it seems to be a
  downstream issue with pulling Deckhand
  docs
- Add jsonpath cacheing as the jsonpath-ng
  parser was consuming 54s of the total 56s
  runtime of a rendered-documents GET call.
  With caching, the call is taking closer to 2s.
- All add a .dockerignore file to make image
  building a little faster

Change-Id: I6ef84ffd946dcf2713b4f7570b985156deb1d697
2018-02-15 13:09:16 -05:00
Anthony Lin d0a42cfc7a Bug Fix - DeckHand/Barbican URI Lookup
We are getting the following error [1] in Armada after [0]
was merged due to missing values in values.yaml

This patch set is meant to correct that

[0] https://review.gerrithub.io/#/c/398810/

[1] Error Messages

2018-02-08 07:02:54.481 1 ERROR armada grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.UNKNOWN, render error in "deckhand/deployment.yaml": template: deckhand/deployment.yaml:36:62: executing "deckhand/deployment.yaml" at <include "helm-toolki...>: error calling include: template: deckhand/charts/helm-toolkit/utils/_hash.tpl:22:4: executing "helm-toolkit.utils.hash" at <include $wtf $contex...>: error calling include: template: deckhand/configmap-etc.yaml:37:20: executing "deckhand/configmap-etc.yaml" at <.Values.conf.deckhan...>: can't evaluate field api_endpoint in type interface {})>

Change-Id: Ie0aad8c2668924589fbad8865c973d86cb8779f7
2018-02-08 07:28:41 +00:00
Alan Meadows 879dfb6129 Add missing barbican api_endpoint to deckhand configuration
Change-Id: Ib951dd99d953c2ca7f25d491411fd4cba0b830ac
2018-02-07 13:38:57 -05:00
Anthony Lin 09dba6a672 Bug Fix - Update Deckhand Ingress Port
The port should be 80 instead as that is the port that is
opened on the Ingress Controller.

Change-Id: Ie0241d5fae5cce0403410e0c9bf3b0ce25a7d84c
2018-02-07 05:01:56 +00:00
Anthony Lin 3bdebba4bb Update Deckhand Chart - Database Configurability
- Support configured Postgres admin password
- Use secrets for database job environment setup
- Remove superuser rights from deckhand user

Change-Id: I9d8eee1af864b0e99ee7c8a01a6bba84cfcb67f9
2018-01-29 05:20:08 +00:00
Mark Burnett 80b787eb08 Make the uWSGI http-timeout configurable
When testing Promenade integration, I found that it was taking more than
the default 60 second timeout to render documents.

Change-Id: Id0ee7ea30b901a60eb95df06a3d25ce4c8689a8a
2018-01-17 15:16:06 -06:00
Anthony Lin a73b118549 Update DeckHand Chart - Multi-Threads/Workers
This p.s. will allow multi-thread/worker parameters to be
configurable in the Deckhand chart so that the values can
be injected into the pod environment. This is a follow up
to the comments made in [0].

Note also that we will need multiple workers in order to
handle concurrent requests from Armada and DryDock to DeckHand
for the rendered document. Multi-threads with single worker
did not work as expected. Test results from our lab environment
suggests that 4 single-threaded workers will be sufficient
for our purpose. Hence we will use that as default override
values for now.

[0] https://review.gerrithub.io/#/c/393679/

Change-Id: I228713ec7b2ec305cbc2c761bc77125ea98e7dfa
2018-01-05 18:40:06 +00:00
Tin Lam 9c91073ac1 RBAC: Update serviceaccount and k8s rbac for deckhand
This patch set brings the deckhand chart to be inline with OSH* RBAC
approach used in [0] and [1].

[0] https://review.openstack.org/#/c/526464/52
[1] https://review.openstack.org/#/c/529378/

Change-Id: I04dc1c8c0b9d675bd87ba3c968ecfb0d55a744c8
2017-12-24 13:39:49 -06:00
Tin Lam 754867a3b2 Change .to_oslo_conf to .to_ini
Currently, openstackhelm-toolkit provides two utility methods:
.to_oslo_conf and .to_ini whose only difference at this time is
the ability to handle multistring in .to_oslo_conf as it is an
openstack concept that is not found in .ini files.

As logging.conf and paste.ini do not support multistring, they do
not need to be render by .to_oslo_conf, and while the code is not
incorrect, using .to_ini would be more appropriate.

Trivial fix

Change-Id: I20ae77842a3406ea64bf40d951adef2e519cbfcb
2017-10-31 14:38:59 -04:00
Anthony Lin d30e93e453 Update DeckHand Chart
There has been recent changes to the Helm Toolkit which broke
the DeckHand Chart

The changes in Helm Toolkit were made to the 'images' definition
in values.yaml to facilitate adding the option to prefix image
name etc

This P.S. updates the DeckHand Chart to align with the recent
changes in Helm Toolkit

Change-Id: I0c9ddfd8b06be7dedcd030d94e381bf4e3f1d210
2017-10-24 00:47:55 +00:00
Anthony Lin 46803b7e60 Move Deckhand Chart
This PS migrates the Deckhand Chart into this repo

Change-Id: I2f16697df2d909e01ce3c35a462474ee857c57e2
2017-10-20 11:20:01 -04:00