Deployment of base UCP control plane
Blocking defect: Ingress requires host port 443 which conflicts with promenade hyperkube config - Provide template for promenade input YAML - Provide template for armada manifest for UCP charts - Provide script to render configs based on template + env vars - Include automation of the genesis + chart deployment Change-Id: I3088520e57f4f19fce5f78608a979e147091ba6e
This commit is contained in:
Scott Hussey
parent
e66a6507ad
commit
71910effc5
|
|
@ -0,0 +1,414 @@
|
|||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: helm-toolkit
|
||||
data:
|
||||
chart_name: helm-toolkit
|
||||
release: helm-toolkit
|
||||
namespace: helm-toolkit
|
||||
timeout: 100
|
||||
values: {}
|
||||
source:
|
||||
type: git
|
||||
location: https://git.openstack.org/openstack/openstack-helm
|
||||
subpath: helm-toolkit
|
||||
reference: master
|
||||
dependencies: []
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ceph
|
||||
data:
|
||||
chart_name: ceph
|
||||
release: ceph
|
||||
namespace: ceph
|
||||
timeout: 3600
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
pre:
|
||||
delete:
|
||||
- name: ceph-bootstrap
|
||||
type: job
|
||||
labels:
|
||||
- application: ceph
|
||||
- component: bootstrap
|
||||
- release_group: armada-ucp
|
||||
- name: ceph-mds-keyring-generator
|
||||
type: job
|
||||
labels:
|
||||
- application: ceph
|
||||
- component: mds-keyring-generator
|
||||
- release_group: armada-ucp
|
||||
- name: ceph-mon-keyring-generator
|
||||
type: job
|
||||
labels:
|
||||
- application: ceph
|
||||
- component: mon-keyring-generator
|
||||
- release_group: armada-ucp
|
||||
- name: ceph-rgw-keyring-generator
|
||||
type: job
|
||||
labels:
|
||||
- application: ceph
|
||||
- component: rgw-keyring-generator
|
||||
- release_group: armada-ucp
|
||||
- name: ceph-storage-keys-generator
|
||||
type: job
|
||||
labels:
|
||||
- application: ceph
|
||||
- component: storage-keys-generator
|
||||
- release_group: armada-ucp
|
||||
- name: ceph-osd-keyring-generator
|
||||
type: job
|
||||
labels:
|
||||
- application: ceph
|
||||
- component: osd-keyring-generator
|
||||
- release_group: armada-ucp
|
||||
values:
|
||||
labels:
|
||||
jobs:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
endpoints:
|
||||
identity:
|
||||
namespace: ucp
|
||||
object_store:
|
||||
namespace: ceph
|
||||
ceph_mon:
|
||||
namespace: ceph
|
||||
ceph:
|
||||
rgw_keystone_auth: true
|
||||
storage:
|
||||
osd_directory: ${CEPH_OSD_DIR}
|
||||
network:
|
||||
public: ${CEPH_PUBLIC_NET}
|
||||
cluster: ${CEPH_CLUSTER_NET}
|
||||
deployment:
|
||||
storage_secrets: true
|
||||
ceph: true
|
||||
rbd_provisioner: true
|
||||
client_secrets: false
|
||||
rgw_keystone_user_and_endpoints: false
|
||||
bootstrap:
|
||||
enabled: true
|
||||
source:
|
||||
type: git
|
||||
location: ${CEPH_CHART_REPO}
|
||||
subpath: ceph
|
||||
reference: ${CEPH_CHART_BRANCH}
|
||||
dependencies:
|
||||
- helm-toolkit
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ucp-ceph-config
|
||||
data:
|
||||
chart_name: ucp-ceph-config
|
||||
release: ucp-ceph-config
|
||||
namespace: ucp
|
||||
timeout: 3600
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
pre:
|
||||
delete:
|
||||
- name: ceph-namespace-client-key-generator
|
||||
type: job
|
||||
labels:
|
||||
- application: ceph
|
||||
- component: namespace-client-key-generator
|
||||
- release_group: armada-ucp
|
||||
values:
|
||||
labels:
|
||||
jobs:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
endpoints:
|
||||
identity:
|
||||
namespace: ucp
|
||||
object_store:
|
||||
namespace: ceph
|
||||
ceph_mon:
|
||||
namespace: ceph
|
||||
ceph:
|
||||
rgw_keystone_auth: true
|
||||
network:
|
||||
public: ${CEPH_PUBLIC_NET}
|
||||
cluster: ${CEPH_CLUSTER_NET}
|
||||
deployment:
|
||||
storage_secrets: false
|
||||
ceph: false
|
||||
rbd_provisioner: false
|
||||
client_secrets: true
|
||||
rgw_keystone_user_and_endpoints: false
|
||||
source:
|
||||
type: git
|
||||
location: ${CEPH_CHART_REPO}
|
||||
subpath: ceph
|
||||
reference: ${CEPH_CHART_BRANCH}
|
||||
dependencies:
|
||||
- helm-toolkit
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ucp-mariadb
|
||||
data:
|
||||
chart_name: ucp-mariadb
|
||||
release: ucp-mariadb
|
||||
namespace: ucp
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
values:
|
||||
labels:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
replicas:
|
||||
server: 1
|
||||
source:
|
||||
type: git
|
||||
location: https://git.openstack.org/openstack/openstack-helm
|
||||
subpath: mariadb
|
||||
dependencies:
|
||||
- helm-toolkit
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ucp-memcached
|
||||
data:
|
||||
chart_name: ucp-memcached
|
||||
release: ucp-memcached
|
||||
namespace: ucp
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
values:
|
||||
labels:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
source:
|
||||
type: git
|
||||
location: https://git.openstack.org/openstack/openstack-helm
|
||||
subpath: memcached
|
||||
dependencies:
|
||||
- helm-toolkit
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ucp-keystone
|
||||
data:
|
||||
chart_name: ucp-keystone
|
||||
release: keystone
|
||||
namespace: ucp
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
pre:
|
||||
delete:
|
||||
- name: keystone-db-sync
|
||||
type: job
|
||||
labels:
|
||||
- job-name: keystone-db-sync
|
||||
- name: keystone-db-init
|
||||
type: job
|
||||
labels:
|
||||
- job-name: keystone-db-init
|
||||
post:
|
||||
delete: []
|
||||
create: []
|
||||
values:
|
||||
conf:
|
||||
keystone:
|
||||
override:
|
||||
paste:
|
||||
override:
|
||||
replicas: 2
|
||||
labels:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
source:
|
||||
type: git
|
||||
location: https://git.openstack.org/openstack/openstack-helm
|
||||
subpath: keystone
|
||||
dependencies:
|
||||
- helm-toolkit
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: maas-postgresql
|
||||
data:
|
||||
chart_name: maas-postgresql
|
||||
release: maas-postgresql
|
||||
namespace: ucp
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
pre:
|
||||
delete: []
|
||||
create: []
|
||||
post:
|
||||
delete: []
|
||||
create: []
|
||||
values:
|
||||
development:
|
||||
enabled: false
|
||||
labels:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
source:
|
||||
type: git
|
||||
location: https://git.openstack.org/openstack/openstack-helm-addons
|
||||
subpath: postgresql
|
||||
reference: master
|
||||
dependencies: []
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: maas
|
||||
data:
|
||||
chart_name: maas
|
||||
release: maas
|
||||
namespace: ucp
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
values:
|
||||
bootdata_url: http://${DRYDOCK_NODE_IP}:${DRYDOCK_NODE_PORT}/api/v1.0/bootdata/
|
||||
labels:
|
||||
rack:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
region:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
network:
|
||||
proxy:
|
||||
node_port:
|
||||
enabled: true
|
||||
port: 31800
|
||||
gui:
|
||||
node_port:
|
||||
enabled: true
|
||||
port: 31900
|
||||
conf:
|
||||
maas:
|
||||
credentials:
|
||||
secret:
|
||||
namespace: ucp
|
||||
url:
|
||||
maas_url: http://${MAAS_NODE_IP}:${MAAS_NODE_PORT}/MAAS
|
||||
proxy:
|
||||
proxy_enabled: '${PROXY_ENABLED}'
|
||||
proxy_server: ${PROXY_ADDRESS}
|
||||
ntp:
|
||||
use_external_only: 'false'
|
||||
ntp_servers: ntp.ubuntu.com
|
||||
dns:
|
||||
require_dnssec: 'no'
|
||||
dns_servers: 8.8.8.8
|
||||
secrets:
|
||||
maas_region:
|
||||
value: 3858a12230ac3c915f300c664f12063f
|
||||
source:
|
||||
type: git
|
||||
location: ${MAAS_CHART_REPO}
|
||||
subpath: maas
|
||||
reference: ${MAAS_CHART_BRANCH}
|
||||
dependencies:
|
||||
- helm-toolkit
|
||||
---
|
||||
schema: armada/Chart/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: drydock
|
||||
data:
|
||||
chart_name: drydock
|
||||
release: drydock
|
||||
namespace: ucp
|
||||
install:
|
||||
no_hooks: false
|
||||
upgrade:
|
||||
no_hooks: false
|
||||
values:
|
||||
images:
|
||||
drydock: ${DRYDOCK_IMAGE}
|
||||
labels:
|
||||
node_selector_key: ucp-control-plane
|
||||
node_selector_value: enabled
|
||||
network:
|
||||
drydock:
|
||||
node_port:
|
||||
enabled: true
|
||||
port: ${DRYDOCK_NODE_PORT}
|
||||
conf:
|
||||
drydock:
|
||||
maasdriver:
|
||||
drydock_provisioner:
|
||||
maas_api_url: http://${MAAS_NODE_IP}:${MAAS_NODE_PORT}/MAAS/api/2.0/
|
||||
source:
|
||||
type: git
|
||||
location: ${DRYDOCK_CHART_REPO}
|
||||
subpath: drydock
|
||||
reference: ${DRYDOCK_CHART_BRANCH}
|
||||
dependencies:
|
||||
- helm-toolkit
|
||||
---
|
||||
schema: armada/Manifest/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ucp-basic
|
||||
data:
|
||||
release_prefix: armada-ucp
|
||||
chart_groups:
|
||||
- ceph
|
||||
- ucp-infra
|
||||
- ucp-services
|
||||
---
|
||||
schema: armada/ChartGroup/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ceph
|
||||
data:
|
||||
description: 'Storage Backend'
|
||||
sequenced: true
|
||||
chart_group:
|
||||
- ceph
|
||||
- ucp-ceph-config
|
||||
---
|
||||
schema: armada/ChartGroup/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ucp-infra
|
||||
data:
|
||||
description: 'UCP Infrastructure'
|
||||
chart_group:
|
||||
- ucp-mariadb
|
||||
- ucp-memcached
|
||||
- maas-postgresql
|
||||
---
|
||||
schema: armada/ChartGroup/v1
|
||||
metadata:
|
||||
schema: metadata/Document/v1
|
||||
name: ucp-services
|
||||
data:
|
||||
description: 'UCP Services'
|
||||
chart_group:
|
||||
- maas
|
||||
- drydock
|
||||
- ucp-keystone
|
||||
...
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
#/bin/bash
|
||||
|
||||
set -x
|
||||
|
||||
# Check that we are root
|
||||
if [[ $(whoami) != "root" ]]
|
||||
then
|
||||
echo "Must be root to run $0"
|
||||
exit -1
|
||||
fi
|
||||
|
||||
|
||||
# Setup environmental variables
|
||||
# with stable defaults
|
||||
|
||||
# Network
|
||||
export CEPH_CLUSTER_NET=${CEPH_CLUSTER_NET:-"NA"}
|
||||
export CEPH_PUBLIC_NET=${CEPH_PUBLIC_NET:-"NA"}
|
||||
export GENESIS_NODE_IP=${GENESIS_NODE_IP:-"NA"}
|
||||
export DRYDOCK_NODE_IP=${DRYDOCK_NODE_IP:-${GENESIS_NODE_IP}}
|
||||
export DRYDOCK_NODE_PORT=${DRYDOCK_NODE_PORT:-31000}
|
||||
export MAAS_NODE_IP=${MAAS_NODE_IP:-${GENESIS_NODE_IP}}
|
||||
export MAAS_NODE_PORT=${MAAS_NODE_PORT:-31900}
|
||||
export MASTER_NODE_IP=${MASTER_NODE_IP:-"NA"}
|
||||
export NODE_NET_IFACE=${NODE_NET_IFACE:-"eth0"}
|
||||
export PROXY_ADDRESS=${PROXY_ADDRESS:-"http://one.proxy.att.com:8080"}
|
||||
export PROXY_ENABLED=${PROXY_ENABLED:-"false"}
|
||||
|
||||
# Storage
|
||||
export CEPH_OSD_DIR=${CEPH_OSD_DIR:-"/var/lib/openstack-helm/ceph/osd"}
|
||||
|
||||
# Hostnames
|
||||
export GENESIS_NODE_NAME=${GENESIS_NODE_NAME:-"node1"}
|
||||
export MASTER_NODE_NAME=${MASTER_NODE_NAME:-"node2"}
|
||||
|
||||
# Charts
|
||||
export CEPH_CHART_REPO=${CEPH_CHART_REPO:-"https://github.com/openstack/openstack-helm"}
|
||||
export CEPH_CHART_BRANCH=${CEPH_CHART_BRANCH:-"master"}
|
||||
export DRYDOCK_CHART_REPO=${DRYDOCK_CHART_REPO:-"https://github.com/att-comdev/aic-helm"}
|
||||
export DRYDOCK_CHART_BRANCH=${DRYDOCK_CHART_BRANCH:-"master"}
|
||||
export MAAS_CHART_REPO=${MAAS_CHART_REPO:-"https://github.com/openstack/openstack-helm-addons"}
|
||||
export MAAS_CHART_BRANCH=${MAAS_CHART_BRANCH:-"master"}
|
||||
|
||||
# Images
|
||||
export DRYDOCK_IMAGE=${DRYDOCK_IMAGE:-"quay.io/attcomdev/drydock:0.2.0-a1"}
|
||||
export ARMADA_IMAGE=${ARMADA_IMAGE:-"quay.io/attcomdev/armada:v0.6.0"}
|
||||
export PROMENADE_IMAGE=${PROMENADE_IMAGE:-"quay.io/attcomdev/promenade:master"}
|
||||
|
||||
# Filenames
|
||||
export ARMADA_CONFIG=${ARMADA_CONFIG:-"armada.yaml"}
|
||||
export PROMENADE_CONFIG=${PROMENADE_CONFIG:-"promenade.yaml"}
|
||||
export UP_SCRIPT_FILE=${UP_SCRIPT_FILE:-"up.sh"}
|
||||
|
||||
# Validate environment
|
||||
if [[ $GENESIS_NODE_IP == "NA" || $MASTER_NODE_IP == "NA" ]]
|
||||
then
|
||||
echo "GENESIS_NODE_IP and MASTER_NODE_IP env vars must be set to correct IP addresses."
|
||||
exit -1
|
||||
fi
|
||||
|
||||
if [[ $CEPH_CLUSTER_NET == "NA" || $CEPH_PUBLIC_NET == "NA" ]]
|
||||
then
|
||||
echo "CEPH_CLUSTER_NET and CEPH_PUBLIC_NET env vars must be set to correct IP subnet CIDRs."
|
||||
exit -1
|
||||
fi
|
||||
|
||||
if [[ $PROXY_ENABLED == 'true' ]]
|
||||
then
|
||||
export http_proxy=$PROXY_ADDRESS
|
||||
export https_proxy=$PROXY_ADDRESS
|
||||
export HTTP_PROXY=$PROXY_ADDRESS
|
||||
export HTTPS_PROXY=$PROXY_ADDRESS
|
||||
fi
|
||||
|
||||
# Install docker
|
||||
apt -qq update
|
||||
apt -y install docker.io jq
|
||||
|
||||
# Required inputs
|
||||
# Promenade input-config.yaml
|
||||
# Armada Manifest for integrated UCP services
|
||||
|
||||
cat promenade.yaml.sub | envsubst > ${PROMENADE_CONFIG}
|
||||
cat armada.yaml.sub | envsubst > ${ARMADA_CONFIG}
|
||||
rm -rf configs
|
||||
mkdir configs
|
||||
|
||||
# Generate Promenade configuration
|
||||
docker run -t -v $(pwd):/target ${PROMENADE_IMAGE} promenade generate -c /target/${PROMENADE_CONFIG} -o /target/configs
|
||||
|
||||
# Do Promenade genesis process
|
||||
cd configs
|
||||
sudo bash ${UP_SCRIPT_FILE} ./${GENESIS_NODE_NAME}.yaml
|
||||
cd ..
|
||||
|
||||
# Setup kubeconfig
|
||||
mkdir ~/.kube
|
||||
cp -r /etc/kubernetes/admin/pki ~/.kube/pki
|
||||
cat /etc/kubernetes/admin/kubeconfig.yaml | sed -e 's/\/etc\/kubernetes\/admin/./' > ~/.kube/config
|
||||
|
||||
# Polling to ensure genesis is complete
|
||||
while [[ -z $(kubectl get pods -n kube-system | grep 'kube-dns' | grep -e '3/3') ]]
|
||||
do
|
||||
sleep 5
|
||||
done
|
||||
|
||||
# Squash Kubernetes RBAC to be compatible w/ OSH
|
||||
kubectl update -f ./rbac-generous-permissions.yaml
|
||||
|
||||
# Do Armada deployment of UCP integrated services
|
||||
docker run -t -v ~/.kube:/armada/.kube -v $(pwd):/target --net=host \
|
||||
${ARMADA_IMAGE} apply /target/${ARMADA_CONFIG} --tiller-host=${GENESIS_NODE_IP} --tiller-port=44134
|
||||
|
||||
# Polling for UCP service deployment
|
||||
|
||||
while [[ -z $(kubectl get pods -n ucp | grep drydock | grep Running) ]]
|
||||
do
|
||||
sleep 5
|
||||
done
|
||||
|
||||
echo 'UCP control plane deployed.'
|
||||
|
|
@ -0,0 +1,295 @@
|
|||
#Copyright 2017 AT&T Intellectual Property. All other rights reserved.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
---
|
||||
# Site/Region wide definitions. Each design part will be a constituent
|
||||
# of the design for exactly one Region
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: Region
|
||||
metadata:
|
||||
name: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
description: Sample site design
|
||||
author: sh8121@att.com
|
||||
spec:
|
||||
# List of query-based definitions for applying tags to deployed nodes
|
||||
tag_definitions:
|
||||
- tag: 'high_memory'
|
||||
# Tag to apply to nodes that qualify for the query
|
||||
definition_type: 'lshw_xpath'
|
||||
# Only support on type for now - 'lshw_xpath' used by MaaS
|
||||
definition: //node[@id="memory"]/'size units="bytes"' > 137438953472
|
||||
# an xpath query that is run against the output of 'lshw -xml' from the node
|
||||
# Image and package repositories needed by Drydock drivers. Needs to be defined
|
||||
repositories:
|
||||
- name: 'ubuntu-main'
|
||||
authorized_keys:
|
||||
- |
|
||||
ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAIEAgqUTJwZEMjZCWOnXQw+FFdvnf/lYrGXm01
|
||||
rf/ZYUanoymkMWIK1/c8a3Ez9/HY3dyfWBcuzlIV4bNCvJcMg4UPuh6NQBJWAlfp7wfW9O
|
||||
8ZyDE3x1FYno5u3OB4rRDcvKe6J0ygPcu4Uec5ASsd58yGnE4zTl1D/J30rNa00si+s= r
|
||||
sa-key-20120124
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: NetworkLink
|
||||
metadata:
|
||||
name: oob
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Describe layer 1 attributes. Primary key is 'name'. These settings will generally be things the switch and server have to agree on
|
||||
labels:
|
||||
- 'noconfig'
|
||||
spec:
|
||||
bonding:
|
||||
# Mode can be 'disabled', '802.3ad', 'balanced-rr', 'active-backup'. Defaults to disabled
|
||||
mode: 'disabled'
|
||||
# Physical link default MTU size. No default
|
||||
mtu: 1500
|
||||
# Physical link speed. Supports 'auto', '100full'. Gigabit+ speeds require auto. No default
|
||||
linkspeed: 'auto'
|
||||
# Settings for using a link for multiple L2 networks
|
||||
trunking:
|
||||
# Trunking mode. Supports 'disabled', '802.1q'. Defaults to disabled
|
||||
mode: disabled
|
||||
# If disabled, what network is this port on. If '802.1q' what is the default network for the port. No default.
|
||||
default_network: oob
|
||||
allowed_networks:
|
||||
- 'oob'
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: NetworkLink
|
||||
metadata:
|
||||
name: pxe-rack1
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Describe layer 1 attributes. Primary key is 'name'. These settings will generally be things the switch and server have to agree on
|
||||
spec:
|
||||
bonding:
|
||||
# Mode can be 'disabled', '802.3ad', 'balanced-rr', 'active-backup'. Defaults to disabled
|
||||
mode: 'disabled'
|
||||
# Physical link default MTU size. No default
|
||||
mtu: 1500
|
||||
# Physical link speed. Supports 'auto', '100full'. Gigabit+ speeds require auto. No default
|
||||
linkspeed: 'auto'
|
||||
# Settings for using a link for multiple L2 networks
|
||||
trunking:
|
||||
# Trunking mode. Supports 'disabled', '802.1q'. Defaults to disabled
|
||||
mode: disabled
|
||||
# If disabled, what network is this port on. If '802.1q' what is the default network for the port. No default.
|
||||
default_network: pxe-rack1
|
||||
allowed_networks:
|
||||
- 'pxe-rack1'
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: Network
|
||||
metadata:
|
||||
name: oob
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Describe layer 2 and 3 attributes. Primary key is 'name'.
|
||||
labels:
|
||||
- 'noconfig'
|
||||
spec:
|
||||
# CIDR representation of network number and netmask
|
||||
cidr: '172.24.10.0/24'
|
||||
# How addresses are allocated on the network. Supports 'static', 'dhcp'. Defaults to 'static'
|
||||
allocation: 'static'
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: Network
|
||||
metadata:
|
||||
name: pxe-rack1
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Describe layer 2 and 3 attributes. Primary key is 'name'.
|
||||
spec:
|
||||
# CIDR representation of network number and netmask
|
||||
cidr: '172.24.1.0/24'
|
||||
# How addresses are allocated on the network. Supports 'static', 'dhcp'. Defaults to 'static'
|
||||
allocation: 'static'
|
||||
routes:
|
||||
# The network being routed to in CIDR notation. Default gateway is 0.0.0.0/0.
|
||||
- subnet: '0.0.0.0/0'
|
||||
# Next hop for traffic using this route
|
||||
gateway: '172.24.1.1'
|
||||
# Selection metric for the host selecting this route. No default
|
||||
metric: 100
|
||||
ranges:
|
||||
# Type of range. Supports 'reserved', 'static' or 'dhcp'. No default
|
||||
- type: 'reserved'
|
||||
# Start of the address range, inclusive. No default
|
||||
start: '172.24.1.1'
|
||||
# End of the address range, inclusive. No default
|
||||
end: '172.24.1.100'
|
||||
- type: 'dhcp'
|
||||
start: '172.24.1.200'
|
||||
end: '172.24.1.250'
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: Network
|
||||
metadata:
|
||||
name: pxe-rack2
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Describe layer 2 and 3 attributes. Primary key is 'name'.
|
||||
spec:
|
||||
# CIDR representation of network number and netmask
|
||||
cidr: '172.24.2.0/24'
|
||||
# How addresses are allocated on the network. Supports 'static', 'dhcp'. Defaults to 'static'
|
||||
allocation: 'static'
|
||||
routes:
|
||||
# The network being routed to in CIDR notation. Default gateway is 0.0.0.0/0.
|
||||
- subnet: '0.0.0.0/0'
|
||||
# Next hop for traffic using this route
|
||||
gateway: '172.24.2.1'
|
||||
# Selection metric for the host selecting this route. No default
|
||||
metric: 100
|
||||
ranges:
|
||||
# Type of range. Supports 'reserved', 'static' or 'dhcp'. No default
|
||||
- type: 'reserved'
|
||||
# Start of the address range, inclusive. No default
|
||||
start: '172.24.2.1'
|
||||
# End of the address range, inclusive. No default
|
||||
end: '172.24.2.100'
|
||||
- type: 'dhcp'
|
||||
start: '172.24.2.200'
|
||||
end: '172.24.2.250'
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: HardwareProfile
|
||||
metadata:
|
||||
name: DellR820v1
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Describe server hardware attributes. Not a specific server, but profile adopted by a server defintion.
|
||||
spec:
|
||||
# Chassis vendor
|
||||
vendor: 'Dell'
|
||||
# Chassis model generation
|
||||
generation: '1'
|
||||
# Chassis model version
|
||||
hw_version: '2'
|
||||
# Certified BIOS version for this chassis
|
||||
bios_version: '2.2.3'
|
||||
# Boot mode. Supports 'bios' or 'uefi'
|
||||
boot_mode: 'bios'
|
||||
# How the node should be initially bootstrapped. Supports 'pxe'
|
||||
bootstrap_protocol: 'pxe'
|
||||
# What network interface to use for PXE booting
|
||||
# for chassis that support selection
|
||||
pxe_interface: '0'
|
||||
# Mapping of hardware alias/role to physical address
|
||||
device_aliases:
|
||||
# the device alias that will be referenced in HostProfile or BaremetalNode design parts
|
||||
- alias: 'pnic01'
|
||||
# The hardware bus the device resides on. Supports 'pci' and 'scsi'. No default
|
||||
bus_type: 'pci'
|
||||
# The type of device as reported by lshw. Can be used to validate hardware manifest. No default
|
||||
dev_type: 'Intel 10Gbps NIC'
|
||||
# Physical address on the bus
|
||||
address: '0000:00:03.0'
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: HostProfile
|
||||
metadata:
|
||||
name: defaults
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Specify a physical server.
|
||||
spec:
|
||||
# The HardwareProfile describing the node hardware. No default.
|
||||
hardware_profile: 'DellR820v1'
|
||||
primary_network: 'pxe-rack1'
|
||||
# OOB access to node
|
||||
oob:
|
||||
# Type of OOB access. Supports 'ipmi'
|
||||
type: 'ipmi'
|
||||
# Which network - as defined in a Network design part - to access the OOB interface on
|
||||
network: 'oob'
|
||||
# Account name for authenticating on the OOB interface
|
||||
account: 'admin'
|
||||
# Credential for authentication on the OOB interface. The OOB driver will interpret this.
|
||||
credential: 'password'
|
||||
# How local node storage is configured
|
||||
storage:
|
||||
physical_devices:
|
||||
sda:
|
||||
labels:
|
||||
bootdrive: true
|
||||
partitions:
|
||||
- name: 'root'
|
||||
size: '10g'
|
||||
bootable: true
|
||||
filesystem:
|
||||
mountpoint: '/'
|
||||
fstype: 'ext4'
|
||||
mount_options: 'defaults'
|
||||
- name: 'boot'
|
||||
size: '1g'
|
||||
filesystem:
|
||||
mountpoint: '/boot'
|
||||
fstype: 'ext4'
|
||||
mount_options: 'defaults'
|
||||
# Physical and logical network interfaces
|
||||
interfaces:
|
||||
# What the interface should be named in the operating system. May not match a hardware device name
|
||||
ens3:
|
||||
# The NetworkLink connected to this interface. Must be the name of a NetworkLink design part
|
||||
device_link: 'pxe-rack1'
|
||||
# Hardware devices that support this interface. For configurating a physical device, this would be a list of one
|
||||
# For bonds, this would be a list of all the physical devices in the bond. These can refer to HardwareProfile device aliases
|
||||
# or explicit device names
|
||||
slaves:
|
||||
- 'ens3'
|
||||
# Network that will be accessed on this interface. These should each be to the name of a Network design part
|
||||
# Multiple networks listed here assume that this interface is attached to a NetworkLink supporting trunking
|
||||
networks:
|
||||
- 'pxe-rack1'
|
||||
platform:
|
||||
# Which image to deploy on the node, must be available in the provisioner. Defaults to 'ubuntu/xenial'
|
||||
image: 'ubuntu/xenial'
|
||||
# Which kernel to enable. Defaults to generic, can also be hwe (hardware enablement)
|
||||
kernel: 'generic'
|
||||
# K/V list of kernel parameters to configure on boot. No default. Use value of true for params that are just flags
|
||||
metadata:
|
||||
# Explicit tags to propagate to Kubernetes. Simple strings of any value
|
||||
rack: rack1
|
||||
---
|
||||
apiVersion: 'drydock/v1'
|
||||
kind: BaremetalNode
|
||||
metadata:
|
||||
name: node2
|
||||
region: atl_foundry
|
||||
date: 17-FEB-2017
|
||||
author: sh8121@att.com
|
||||
description: Specify a physical server.
|
||||
spec:
|
||||
host_profile: defaults
|
||||
addressing:
|
||||
# The name of a defined Network design part also listed in the 'networks' section of a interface definition
|
||||
- network: 'pxe-rack1'
|
||||
# Address should be an explicit IP address assignment or 'dhcp'
|
||||
address: '172.24.1.101'
|
||||
- network: 'oob'
|
||||
address: '172.24.10.101'
|
||||
metadata:
|
||||
tags:
|
||||
- 'masters'
|
||||
...
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
---
|
||||
apiVersion: promenade/v1
|
||||
kind: Cluster
|
||||
metadata:
|
||||
name: example
|
||||
target: none
|
||||
spec:
|
||||
nodes:
|
||||
${GENESIS_NODE_NAME}:
|
||||
ip: ${GENESIS_NODE_IP}
|
||||
roles:
|
||||
- master
|
||||
- genesis
|
||||
additional_labels:
|
||||
- beta.kubernetes.io/arch=amd64
|
||||
- ucp-control-plane=enabled
|
||||
- ceph-mon=enabled
|
||||
- ceph-osd=enabled
|
||||
- ceph-mds=enabled
|
||||
${MASTER_NODE_NAME}:
|
||||
ip: ${MASTER_NODE_IP}
|
||||
roles:
|
||||
- master
|
||||
additional_labels:
|
||||
- beta.kubernetes.io/arch=amd64
|
||||
- ucp-control-plane=enabled
|
||||
- ceph-mon=enabled
|
||||
- ceph-osd=enabled
|
||||
- ceph-mds=enabled
|
||||
---
|
||||
apiVersion: promenade/v1
|
||||
kind: Network
|
||||
metadata:
|
||||
cluster: example
|
||||
name: example
|
||||
target: all
|
||||
spec:
|
||||
cluster_domain: cluster.local
|
||||
cluster_dns: 10.96.0.10
|
||||
kube_service_ip: 10.96.0.1
|
||||
pod_ip_cidr: 10.97.0.0/16
|
||||
service_ip_cidr: 10.96.0.0/16
|
||||
calico_etcd_service_ip: 10.96.232.136
|
||||
calico_interface: ${NODE_NET_IFACE}
|
||||
dns_servers:
|
||||
- 8.8.8.8
|
||||
- 8.8.4.4
|
||||
---
|
||||
apiVersion: promenade/v1
|
||||
kind: Versions
|
||||
metadata:
|
||||
cluster: example
|
||||
name: example
|
||||
target: all
|
||||
spec:
|
||||
images:
|
||||
armada: ${ARMADA_IMAGE}
|
||||
calico:
|
||||
cni: quay.io/calico/cni:v1.9.1
|
||||
etcd: quay.io/coreos/etcd:v3.2.1
|
||||
node: quay.io/calico/node:v1.3.0
|
||||
policy-controller: quay.io/calico/kube-policy-controller:v0.6.0
|
||||
kubernetes:
|
||||
apiserver: gcr.io/google_containers/hyperkube-amd64:v1.6.7
|
||||
controller-manager: quay.io/attcomdev/kube-controller-manager:v1.6.7
|
||||
dns:
|
||||
dnsmasq: gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.2
|
||||
kubedns: gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.2
|
||||
sidecar: gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.2
|
||||
etcd: quay.io/coreos/etcd:v3.2.1
|
||||
kubectl: gcr.io/google_containers/hyperkube-amd64:v1.6.7
|
||||
proxy: gcr.io/google_containers/hyperkube-amd64:v1.6.7
|
||||
scheduler: gcr.io/google_containers/hyperkube-amd64:v1.6.7
|
||||
promenade: ${PROMENADE_IMAGE}
|
||||
tiller: gcr.io/kubernetes-helm/tiller:v2.5.0
|
||||
packages:
|
||||
docker: docker.io=1.12.6-0ubuntu1~16.04.1
|
||||
dnsmasq: dnsmasq=2.75-1ubuntu0.16.04.2
|
||||
socat: socat=1.7.3.1-1
|
||||
additional_packages:
|
||||
- ceph-common=10.2.7-0ubuntu0.16.04.1
|
||||
...
|
||||
|
|
@ -0,0 +1,16 @@
|
|||
---
|
||||
apiVersion: rbac.authorization.k8s.io/v1alpha1
|
||||
kind: ClusterRoleBinding
|
||||
metadata:
|
||||
name: generous-permissions
|
||||
roleRef:
|
||||
apiGroup: rbac.authorization.k8s.io
|
||||
kind: ClusterRole
|
||||
name: cluster-admin
|
||||
subjects:
|
||||
- kind: Group
|
||||
name: system:masters
|
||||
- kind: Group
|
||||
name: system:authenticated
|
||||
- kind: Group
|
||||
name: system:unauthenticated
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
# Artifacts to deploy a basic UCP control plane
|
||||
|
||||
The scripts and artifacts in this directory can be used to deploy
|
||||
a basic UCP control plane on a single node.
|
||||
|
||||
1. Generate Promenade configuration and certificates
|
||||
2. Run Promenade genesis process to bootstrap Kubernetes
|
||||
3. Deploy Ceph using Armada
|
||||
4. Deploy UCP backend services (MariaDB, Postgres) using Armada
|
||||
5. Deploy Drydock and MaaS using Armada
|
||||
|
||||
## Setup environment for your environment
|
||||
|
||||
This deployment process utilizes several environment variables to
|
||||
customize the deployment to your environment. The set-env.sh file has
|
||||
an example environment known to work.
|
||||
|
||||
* CEPH\_CLUSTER\_NET
|
||||
|
||||
The CIDR of the network(s) that Ceph will utilize for storage replication and
|
||||
other intra-cluster communication. Can be a comma-separated list of CIDRs.
|
||||
|
||||
* CEPH\_PUBLIC\_NET
|
||||
|
||||
The CIDR of the network(s) that Ceph will utilize for accepting requests
|
||||
for storage provisioning. Can be a comma-separated list of CIDRs.
|
||||
|
||||
* CEPH\_OSD\_DIR
|
||||
|
||||
The directory Ceph will use for OSD storage
|
||||
|
||||
* GENESIS\_NODE\_IP
|
||||
|
||||
The IP address of the genesis node or VM.
|
||||
|
||||
* MASTER\_NODE\_IP
|
||||
|
||||
The IP address of the second node to be added to the cluster. Scripting does not yet
|
||||
support deployment of this node, but it is *REQUIRED* to be included in the bootstrap
|
||||
configuration
|
||||
|
||||
* DRYDOCK\_NODE\_IP
|
||||
|
||||
The IP address of the node that will host the Drydock container. Defaults to the genesis
|
||||
node which is normally correct.
|
||||
|
||||
* MAAS\_NODE\_IP
|
||||
|
||||
The IP address of the node that will hsot the MaaS container. Defaults to the genesis
|
||||
node which is normally correct.
|
||||
|
||||
* NODE\_NET\_IFACE
|
||||
|
||||
The NIC interface on each node that Calico should use to access the underlay network. Defaults
|
||||
to 'eth0'
|
||||
|
||||
* PROXY\_ADDRESS
|
||||
|
||||
If a HTTP/HTTPS proxy is needed for public access, specify the address here in URL format.
|
||||
|
||||
* PROXY\_ENABLED
|
||||
|
||||
Whether to enable proxy use. Should be 'true' or 'false', defaults to 'false'.
|
||||
|
||||
* GENESIS\_NODE\_NAME
|
||||
|
||||
The hostname of the genesis node. REQUIRED to be accurate. Defaults to 'node1'
|
||||
|
||||
* MASTER\_NODE\_NAME
|
||||
|
||||
The hostname of the master (or second) node. REQUIRED to be accurate. Defaults to 'node2'
|
||||
|
||||
* \*\_CHART\_REPO
|
||||
|
||||
The Git repository used for pulling charts. \* can be any of 'CEPH', 'DRYDOCK' or 'MAAS'
|
||||
|
||||
* \*\_CHART\_BRANCH
|
||||
|
||||
The Git branch used for pulling charts. \* can be any of 'CEPH', 'DRYDOCK' or 'MAAS'
|
||||
|
||||
* \*\_IMAGE
|
||||
|
||||
The Docker image file used for deployments and running commands. \* can be any of 'DRYDOCK',
|
||||
'ARMADA', 'PROMENADE'.
|
||||
|
||||
## Run the deployment
|
||||
|
||||
Once all of the above environmental variables are correct, run `deploy_ucp.sh` as root.
|
||||
|
|
@ -0,0 +1,15 @@
|
|||
# Known working integrations
|
||||
export CEPH_CLUSTER_NET=172.24.1.0/24
|
||||
export CEPH_PUBLIC_NET=172.24.1.0/24
|
||||
export GENESIS_NODE_IP=172.24.1.100
|
||||
export MASTER_NODE_IP=172.24.1.101
|
||||
export NODE_NET_IFACE=ens3
|
||||
export DRYDOCK_CHART_REPO=https://github.com/sh8121att/helm_charts
|
||||
export DRYDOCK_CHART_BRANCH=master
|
||||
export MAAS_CHART_REPO=https://github.com/sh8121att/helm_charts
|
||||
export MAAS_CHART_BRANCH=master
|
||||
export CEPH_CHART_BRANCH=84901ac56db1647e40fe0015983653e48ff90f5b
|
||||
export DRYDOCK_IMAGE=docker.io/sthussey/drydock:bonding-rc3
|
||||
export ARMADA_IMAGE=quay.io/attcomdev/armada:master
|
||||
export PROMENADE_IMAGE=quay.io/attcomdev/promenade:v0.2.0
|
||||
export CEPH_OSD_DIR=/var/lib/openstack-helm/ceph/osd
|
||||
Loading…
Reference in New Issue