summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorChris Wedgwood <cw@f00f.org>2017-08-15 21:56:12 +0000
committerChris Wedgwood <cw@f00f.org>2017-10-30 20:37:43 +0000
commit92a678bb5ca1bf1f13eb24aa82bca664e1204b4f (patch)
treed9affd297a1f708db71dfbf5f3ef66b1af90840b
parenta821aeff005797b77992244d912fad7ab4ed19a7 (diff)
Use overrides (values) to describe VMs
Previously the VM configuration was embedded in the charts templates; this change moves things into values as we iterate over things created resources as required. Change-Id: I75e60bd31ee5debda90a703775215e92480d572c
-rw-r--r--Dockerfile11
-rw-r--r--Makefile15
-rw-r--r--README.md105
-rw-r--r--TODO23
-rwxr-xr-xberth/templates/bin/_startvm.txt111
-rw-r--r--berth/templates/deployment.yaml132
-rw-r--r--berth/templates/iter-configmap.yaml33
-rw-r--r--berth/templates/iter-deployment.yaml98
-rw-r--r--berth/templates/iter-pvc.yaml23
-rw-r--r--berth/values.yaml14
-rw-r--r--examples/bogus-vm.yaml10
-rw-r--r--examples/cirros-test.yaml22
-rw-r--r--examples/demo-ub14-apache.yaml33
-rw-r--r--examples/ub16-smp-test.yaml36
-rwxr-xr-xtools/gate/setup.sh13
-rwxr-xr-xtools/gate/test.sh41
16 files changed, 507 insertions, 213 deletions
diff --git a/Dockerfile b/Dockerfile
new file mode 100644
index 0000000..35b6874
--- /dev/null
+++ b/Dockerfile
@@ -0,0 +1,11 @@
1FROM ubuntu:16.04
2
3RUN apt-get update && apt-get install -y qemu-kvm dnsmasq bridge-utils mkisofs curl jq wget iptables
4RUN apt-get clean
5RUN rm -f /var/lib/apt/lists/* || true
6
7ENTRYPOINT ["/bin/sleep", "infinity"]
8
9VOLUME "/image"
10EXPOSE 22
11CMD []
diff --git a/Makefile b/Makefile
index 982c7e8..aced210 100644
--- a/Makefile
+++ b/Makefile
@@ -9,22 +9,23 @@ test: install
9 9
10install: build 10install: build
11 @echo 11 @echo
12 -helm delete --purge berth >>helm.log 2>&1 12 -helm delete --purge berth
13 @echo 13 @echo
14 @[ -f override.yaml ] || touch override.yaml 14 helm install --name=berth --debug ./berth
15 helm install ./berth-0.1.0.tgz --values=override.yaml --name=berth >>helm.log 2>&1 15 helm upgrade --debug berth ./berth \
16 @sleep 5.0 # give k8s a chance to see the IP 16 --values examples/cirros-test.yaml \
17 --values examples/demo-ub14-apache.yaml \
18 --values examples/ub16-smp-test.yaml
19 @sleep 5 # give k8s a chance to see the IP
17 @echo 20 @echo
18 kubectl get pods -o wide 21 kubectl get pods -o wide
19 22
20build: 23build:
21 @echo 24 @echo
22 helm lint berth 25 helm lint berth
23 @echo
24 helm package berth
25 26
26clean: 27clean:
27 rm -f berth-0.1.0.tgz helm.log 28 rm -f *~ */*~ */*/*~ berth-0.1.0.tgz
28 29
29.PHONY: 30.PHONY:
30 all default build clean 31 all default build clean
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..7534aa8
--- /dev/null
+++ b/README.md
@@ -0,0 +1,105 @@
1Berth is a deliberately minimalist VM runner for Kubernetes.
2
3## TL;DR Installation Guide
4
5Install the bare Berth chart:
6```
7# Have (recent) Helm and Kubernetes (2.5.x and 1.6.x or later)
8#
9helm install --name=berth ./berth
10kubectl get pods -o wide
11```
12
13This should happen error free and be fairly quick. At this stage
14install one of the examples:
15```
16# helm upgrade berth ./berth --values=examples/cirros-test.yaml
17```
18
19You should be able to SSH to your VM at the Kubernetes IP for the
20container which you can retrieve with `kubectl get all -o wide`. VNC
21access is available on port 5900.
22
23Additional examples may be added with additional `--values` arguments.
24For example:
25```
26# helm upgrade berth ./berth --values=examples/cirros-test.yaml --values=examples/demo-ub14-apache.yaml
27```
28
29You can use a similar method to purge all VMs:
30```
31# helm upgrade berth ./berth --set 'vmlist={}'
32
33```
34
35
36<!-- https://mostsecure.pw/ -->
37
38### Example
39
40[Quick installation / sample](https://asciinema.org/a/4VazbwsokL3zpnGPf27eyFIfe)
41
42### Why this?
43
44The requirements are very narrow right now and the existing
45alternatives don't align well at present. This will likely change in
46time at which point we can realign the internal implementation.
47
48#### Minimalist requirements
49* Run VMs from inside of Kubernetes
50* Work with Calico
51* Have VM life-cycle match that of pods
52* Have VMs benefit from Kubernetes resiliency
53* Allow for persistent storage
54* Allow for state injection/access from a ConfigMaps
55
56## Requirements:
57* Helm 2.5.x
58* Kubernetes 1.6.x
59
60This does not need to be installed as part of the OpenStack chart
61collection.
62
63## How it works:
64
65At a high level, it works like this:
66* Create a SNAT/DNAT enabled linux bridge.
67* Assign the bridge a private IP address from a small /30 subnet
68 (controlled with `VM_IP` and `VM_GW`)
69* Plug the VM network interface into the bridge.
70* Run a dnsmasq process to allocate the VM the right name-servers, and
71 DNS search strings extracted from the parent container. Assign the
72 private IP address to the VM and have it use the bridge's IP as its
73 default gateway.
74* Setup SNAT/DNAT on the parent container to do 1:1 mapping of all
75 ports, all protocols to the VM, except for TCP:5900 to allow for VNC
76 access (can be controlled with NO_VNC environment variable).
77* At this point, VM essentially assumes Pod Assigned IP.
78* Feed any meta-data or user-data down into the VM by leveraging these
79 ConfigMap mounts with the same name and turning them into an ISO
80 presented to the guest.
81
82The startvm entry-point supports several environment variables:
83
84* `IMG_SOURCE` which is an http or https URL that contains a qcow2
85 image. It can also be a full path to a local file baked into the
86 container image, e.g. "/image.qcow"
87* `IMG_TARGET` the name to save the image above as in the shared
88 volume.
89
90It also supports two files, which should be mounted as ConfigMaps if
91using Kubernetes at `/userdata` and `/metadata` as YAML files
92containing, obviously meta-data and user-data as YAML that will be fed
93to the VM as a config-drive iso.
94
95The "pet" version of the image, which is created using qemu-img -b to
96base it on the source, is stored in a separate volume dedicated to the
97VM itself, and named after the container hostname.
98
99There are a few other parameters you can control as an operator:
100
101* `VM_IP` is the IP address the VM should be allocated by DHCP. The
102 container will 1:1 NAT except for port 5900 for VNC access (defaults
103 to 192.168.254.2)
104* `VM_GW` is the gateway IP address the VM should use for its default
105 route (defaults to 192.168.254.1)
diff --git a/TODO b/TODO
deleted file mode 100644
index cd21407..0000000
--- a/TODO
+++ /dev/null
@@ -1,23 +0,0 @@
1[Put these in Jira]
2
3*Require* ssh key, throw an error if missing.
4
5Chart definition vs values.yaml:
6
7 Move image source/target details from chart to values
8 (IMG_SOURCE/IMG_TARGET)
9
10 Move the VM definitions entirely into the values.yaml file; have
11 helm deployment process iterate over that creating multiple pods,
12 one for each VM.
13
14 Ideally this will allow incremental updates of pods without having
15 to know about the chart internals.
16
17 XXX Get examples of where we've done this before from Alan XXX
18
19Add Dockerfile; update the container image we use to Ubuntu 16.04 with
20a suitable KVM/Qemu.
21
22Consider moving to stateful sets for cleaner PVC associations. See
23OSH MariaDB chart for an example of this.
diff --git a/berth/templates/bin/_startvm.txt b/berth/templates/bin/_startvm.txt
index 52f1efe..0de881f 100755
--- a/berth/templates/bin/_startvm.txt
+++ b/berth/templates/bin/_startvm.txt
@@ -1,16 +1,17 @@
1#!/bin/bash 1#!/bin/bash
2 2
3set -x 3set -ex
4 4
5echo "VER-0.1.0-1.2" 5# FIXME; right now this doens't work, need to work out why
6#set -o pipefail
6 7
7# Returns the integer representation of an IP arg, passed in ascii 8# Returns the integer representation of an IP arg, passed in ascii
8# dotted-decimal notation (x.x.x.x) 9# dotted-decimal notation (x.x.x.x)
9atoi() { 10atoi() {
10 IP=$1; IPNUM=0 11 IP=$1; IPNUM=0
11 for (( i=0 ; i<4 ; ++i )); do 12 for (( i=0 ; i<4 ; ++i )); do
12 ((IPNUM+=${IP%%.*}*$((256**$((3-${i})))))) 13 ((IPNUM+=${IP%%.*}*$((256**$((3-${i}))))))
13 IP=${IP#*.} 14 IP=${IP#*.}
14 done 15 done
15 echo $IPNUM 16 echo $IPNUM
16} 17}
@@ -27,23 +28,23 @@ itoa() {
27generate_cloud_drive() { 28generate_cloud_drive() {
28 metadata=/metadata 29 metadata=/metadata
29 if [ ! -f $metadata ]; then 30 if [ ! -f $metadata ]; then
30 metadata="" 31 metadata=""
31 fi 32 fi
32 33
33 userdata=/userdata 34 userdata=/userdata
34 if [ ! -f $userdata ]; then 35 if [ ! -f $userdata ]; then
35 userdata="" 36 userdata=""
36 fi 37 fi
37 38
38 if [ "$metadata" == "" -a "$userdata" == "" ]; then 39 if [ "$metadata" == "" -a "$userdata" == "" ]; then
39 return 40 return
40 fi 41 fi
41 42
42 TMPDIR=`mktemp -d -t aicvm.XXXXXX` 43 TMPDIR=`mktemp -d -t aicvm.XXXXXX`
43 44
44 if [ $? -ne 0 ]; then 45 if [ $? -ne 0 ]; then
45 echo "Fail to create temporaily directory" 46 echo "Fail to create temporaily directory"
46 exit 1 47 exit 1
47 fi 48 fi
48 49
49 # create form of config drive 50 # create form of config drive
@@ -54,17 +55,17 @@ generate_cloud_drive() {
54 cd $OLD_PWD 55 cd $OLD_PWD
55 56
56 if [ -f $metadata ]; then 57 if [ -f $metadata ]; then
57 cp $metadata ${TMPDIR}/openstack/2012-08-10/meta_data.json 58 cp $metadata ${TMPDIR}/openstack/2012-08-10/meta_data.json
58 fi 59 fi
59 if [ -f $userdata ]; then 60 if [ -f $userdata ]; then
60 cp $userdata ${TMPDIR}/openstack/2012-08-10/user_data 61 cp $userdata ${TMPDIR}/openstack/2012-08-10/user_data
61 fi 62 fi
62 63
63 iso="cloud-drive.iso" 64 iso="cloud-drive.iso"
64 mkisofs -R -V config-2 -o $iso ${TMPDIR} 65 mkisofs -R -V config-2 -o $iso ${TMPDIR}
65 if [ $? -ne 0 ]; then 66 if [ $? -ne 0 ]; then
66 echo Fail to create cloud-drive ISO image for cloud-init 67 echo Fail to create cloud-drive ISO image for cloud-init
67 exit 1 68 exit 1
68 fi 69 fi
69 echo $iso 70 echo $iso
70} 71}
@@ -110,38 +111,38 @@ if [ -e /dev/vm/root ]; then
110else 111else
111 112
112 if [ -e "${IMG_TARGET}" ]; then 113 if [ -e "${IMG_TARGET}" ]; then
113 BASE=${IMG_TARGET} 114 BASE=${IMG_TARGET}
114 else 115 else
115 116
116 if [ ! -d "/image" ]; then 117 if [ ! -d "/image" ]; then
117 echo "/image directory does not exist, failed to mount volume?" 118 echo "/image directory does not exist, failed to mount volume?"
118 exit 2 119 exit 2
119 fi 120 fi
120 121
121 if [ ! -e "/image/${IMG_TARGET}" ]; then 122 if [ ! -e "/image/${IMG_TARGET}" ]; then
122 echo "Fetching missing image target" 123 echo "Fetching missing image target"
123 curl ${IMG_SOURCE} > /image/${IMG_TARGET} 124 curl ${IMG_SOURCE} > /image/${IMG_TARGET}
124 fi 125 fi
125 126
126 BASE=/image/${IMG_TARGET} 127 BASE=/image/${IMG_TARGET}
127 fi 128 fi
128 129
129 if [ ! -d "/image" ]; then 130 if [ ! -d "/image" ]; then
130 echo "/image directory does not exist, failed to mount volume /image?" 131 echo "/image directory does not exist, failed to mount volume /image?"
131 exit 2 132 exit 2
132 fi 133 fi
133 134
134 if [ -z "${HOSTNAME}" ]; then 135 if [ -z "${HOSTNAME}" ]; then
135 echo "Could not find HOSTNAME var. Did you specify a HOSTNAME environment variable?" 136 echo "Could not find HOSTNAME var. Did you specify a HOSTNAME environment variable?"
136 fi 137 fi
137 138
138 KVM_IMAGE=/image/${HOSTNAME}.qcow2 139 KVM_IMAGE=/image/${HOSTNAME}.qcow2
139 140
140 if [ -e "${KVM_IMAGE}" ]; then 141 if [ -e "${KVM_IMAGE}" ]; then
141 echo "Image ${KVM_IMAGE} already exists. Not recreating" 142 echo "Image ${KVM_IMAGE} already exists. Not recreating"
142 else 143 else
143 qemu-img create -f qcow2 -b ${BASE} \ 144 qemu-img create -f qcow2 -b ${BASE} \
144 $KVM_IMAGE > /dev/null 145 $KVM_IMAGE > /dev/null
145 if [[ $? -ne 0 ]]; then 146 if [[ $? -ne 0 ]]; then
146 echo "Failed to create qcow2 image" 147 echo "Failed to create qcow2 image"
147 exit 3 148 exit 3
@@ -173,14 +174,14 @@ cidr2mask() {
173 local partial_octet=$(($1%8)) 174 local partial_octet=$(($1%8))
174 175
175 for ((i=0;i<4;i+=1)); do 176 for ((i=0;i<4;i+=1)); do
176 if [ $i -lt $full_octets ]; then 177 if [ $i -lt $full_octets ]; then
177 mask+=255 178 mask+=255
178 elif [ $i -eq $full_octets ]; then 179 elif [ $i -eq $full_octets ]; then
179 mask+=$((256 - 2**(8-$partial_octet))) 180 mask+=$((256 - 2**(8-$partial_octet)))
180 else 181 else
181 mask+=0 182 mask+=0
182 fi 183 fi
183 test $i -lt 3 && mask+=. 184 test $i -lt 3 && mask+=.
184 done 185 done
185 186
186 echo $mask 187 echo $mask
@@ -196,6 +197,24 @@ setup_bridge_networking() {
196 NAMESERVER=( `grep nameserver /etc/resolv.conf | grep -v "#" | cut -f2 -d ' '` ) 197 NAMESERVER=( `grep nameserver /etc/resolv.conf | grep -v "#" | cut -f2 -d ' '` )
197 NAMESERVERS=`echo ${NAMESERVER[*]} | sed "s/ /,/"` 198 NAMESERVERS=`echo ${NAMESERVER[*]} | sed "s/ /,/"`
198 SEARCH=( `grep -E ^search /etc/resolv.conf | grep -v "#" | cut -f2- -d ' ' | tr ' ' ','` ) 199 SEARCH=( `grep -E ^search /etc/resolv.conf | grep -v "#" | cut -f2- -d ' ' | tr ' ' ','` )
200 # MAC=$(ip addr show $IFACE | grep ether | sed -e 's/^[[:space:]]*//g' -e 's/[[:space:]]*\$//g' | cut -f2 -d ' ')
201 # HOST_IP=$(ip addr show dev $IFACE | grep "inet $IP" | awk '{print $2}' | cut -f1 -d/)
202 # HOST_CIDR=$(ip addr show dev $IFACE | grep "inet $IP" | awk '{print $2}' | cut -f2 -d/)
203 # HOST_NETMASK=$(cidr2mask $HOST_CIDR)
204 # HOST_GATEWAY=$(ip route get 8.8.8.8 | grep via | cut -f3 -d ' ')
205 # NAMESERVER=$(grep nameserver /etc/resolv.conf | grep -v "#" | cut -f2 -d ' ') )
206 # NAMESERVERS=$(echo ${NAMESERVER[*]} | sed "s/ /,/")
207 # SEARCH=$(grep -E ^search /etc/resolv.conf | grep -v "#" | cut -f2- -d ' ' | tr ' ' ',')
208
209 # fail if any of the above aren't suitable # here
210 [ -n "$MAC" ]
211 [ -n "$HOST_IP" ]
212 [ -n "$HOST_CIDR" ]
213 [ -n "$HOST_NETMASK" ]
214 [ -n "$HOST_GATEWAY" ]
215 [ -n "$NAMESERVER" ]
216 [ -n "$NAMESERVERS" ]
217 [ -n "$SEARCH" ]
199 218
200 # we must enable forwarding inside the container 219 # we must enable forwarding inside the container
201 echo 1 > /proc/sys/net/ipv4/ip_forward 220 echo 1 > /proc/sys/net/ipv4/ip_forward
@@ -205,11 +224,11 @@ setup_bridge_networking() {
205 # specify NO_VNC as an environment variable to disable this 224 # specify NO_VNC as an environment variable to disable this
206 # functionality 225 # functionality
207 if [ -z $NO_VNC ]; then 226 if [ -z $NO_VNC ]; then
208 iptables -t nat -A PREROUTING -p tcp \! --dport 5900 -d $HOST_IP -j DNAT --to-destination $VM_IP 227 iptables -t nat -A PREROUTING -p tcp \! --dport 5900 -d $HOST_IP -j DNAT --to-destination $VM_IP
209 iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP 228 iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP
210 else 229 else
211 iptables -t nat -A PREROUTING -d $HOST_IP -j DNAT --to-destination $VM_IP 230 iptables -t nat -A PREROUTING -d $HOST_IP -j DNAT --to-destination $VM_IP
212 iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP 231 iptables -t nat -A POSTROUTING -s $VM_IP -j SNAT --to-source $HOST_IP
213 fi 232 fi
214 233
215 # generate VM specifics 234 # generate VM specifics
@@ -258,4 +277,14 @@ setup_bridge_networking
258HOST_IP=`ip addr show dev $IFACE | grep "inet $IP" | awk '{print $2}' | cut -f1 -d/` 277HOST_IP=`ip addr show dev $IFACE | grep "inet $IP" | awk '{print $2}' | cut -f1 -d/`
259VNC="-vnc $HOST_IP:0" 278VNC="-vnc $HOST_IP:0"
260 279
261exec $LAUNCHER qemu-system-x86_64 -enable-kvm $VNC `eval echo $KVM_BLK_OPTS` `eval echo $KVM_NET_OPTS` -usbdevice tablet -nographic $KVM_ARGS 280exec $LAUNCHER qemu-system-x86_64 \
281 -smp "$IMG_VCPU" \
282 -m "$IMG_RAM_MB" \
283 -machine q35 \
284 -cpu host,+x2apic \
285 -vga vmware \
286 -enable-kvm \
287 $VNC \
288 `eval echo $KVM_BLK_OPTS` \
289 `eval echo $KVM_NET_OPTS` \
290 -usbdevice tablet -nographic $KVM_ARGS
diff --git a/berth/templates/deployment.yaml b/berth/templates/deployment.yaml
deleted file mode 100644
index 52ed2aa..0000000
--- a/berth/templates/deployment.yaml
+++ /dev/null
@@ -1,132 +0,0 @@
1# FIXME(cw); refactor into multiple per-function files
2
3# FIXME(cw) consider using OSH helm-toolkit.utils.template
4{{- define "template" -}}
5{{- $name := index . 0 -}}
6{{- $context := index . 1 -}}
7{{- $last := base $context.Template.Name }}
8{{- $wtf := $context.Template.Name | replace $last $name -}}
9{{ include $wtf $context }}
10{{- end -}}
11
12---
13apiVersion: v1
14kind: ConfigMap
15metadata:
16 name: cloudinit
17data:
18 metadata: |
19 { "uuid": "example-01-vm.mydomain.com" }
20 userdata: |
21 #cloud-config
22 fqdn: example-01-vm.mydomain.com
23 users:
24 - name: root
25 ssh-authorized-keys:
26 - {{ .Values.auth.ssh_key }}
27 ssh_pwauth: True
28 runcmd:
29 - [ apt-get, update ]
30 - [ apt-get, install, -y, --force-yes, apache2 ]
31---
32apiVersion: v1
33kind: ConfigMap
34metadata:
35 name: configmap-startvm
36data:
37 startvm: |
38 #!/bin/bash
39 #
40 # start of startvm
41{{ tuple "bin/_startvm.txt" . | include "template" | indent 4 }}
42 # end of startvm
43---
44kind: PersistentVolumeClaim
45apiVersion: v1
46metadata:
47 name: example-01-vm
48spec:
49 accessModes: [ "ReadWriteOnce" ]
50 resources:
51 requests:
52 storage: {{ .Values.volume.size }}
53{{ if not .Values.volume.class_name }}
54 storageClassName: {{ .Values.volume.class_name }}
55{{ end }}
56---
57apiVersion: extensions/v1beta1
58kind: Deployment
59metadata:
60 name: berth
61spec:
62 replicas: 1
63 template:
64 metadata:
65 labels:
66 app: berth
67 annotations:
68 pod.beta.kubernetes.io/hostname: example-01-vm
69 spec:
70 nodeSelector:
71 {{ .Values.labels.node_selector_key }}: {{ .Values.labels.node_selector_value }}
72 hostNetwork: false
73 hostPID: false
74 securityContext:
75 runAsUser: 0
76 containers:
77 - name: example-01-vm
78 imagePullPolicy: IfNotPresent
79 image: {{ .Values.images.entrypoint }}
80 env:
81 - name: IMG_SOURCE
82 value: http://stupidest.org/vm/ubuntu-14.04-amd64.img
83 - name: IMG_TARGET
84 value: ubuntu-14.04-amd64.img
85 securityContext:
86 privileged: true
87 command:
88 - /usr/local/bin/startvm
89 ports:
90 - containerPort: {{ .Values.network.port }}
91 - containerPort: {{ .Values.network.vnc }}
92 readinessProbe:
93 tcpSocket:
94 port: {{ .Values.network.vnc }}
95 volumeMounts:
96 - name: volume-startvm
97 mountPath: /usr/local/bin/startvm
98 subPath: startvm
99 - name: image
100 mountPath: /image
101 - name: dev
102 mountPath: /dev
103 - name: sys
104 mountPath: /sys
105 - name: cloudinit
106 mountPath: /userdata
107 subPath: userdata
108 - name: cloudinit
109 mountPath: /metadata
110 subPath: metadata
111 volumes:
112 - name: volume-startvm
113 configMap:
114 name: configmap-startvm
115 defaultMode: 0755
116 - name: image
117 persistentVolumeClaim:
118 claimName: example-01-vm
119 - name: dev
120 hostPath:
121 path: /dev
122 - name: sys
123 hostPath:
124 path: /sys
125 - name: cloudinit
126 configMap:
127 name: cloudinit
128 items:
129 - key: userdata
130 path: userdata
131 - key: metadata
132 path: metadata
diff --git a/berth/templates/iter-configmap.yaml b/berth/templates/iter-configmap.yaml
new file mode 100644
index 0000000..2b2fcff
--- /dev/null
+++ b/berth/templates/iter-configmap.yaml
@@ -0,0 +1,33 @@
1
2{{- define "template" -}}
3{{- $name := index . 0 -}}
4{{- $context := index . 1 -}}
5{{- $last := base $context.Template.Name }}
6{{- $wtf := $context.Template.Name | replace $last $name -}}
7{{ include $wtf $context }}
8{{- end -}}
9
10---
11
12apiVersion: v1
13kind: ConfigMap
14metadata:
15 name: configmap-generic-startvm
16data:
17 startvm: |
18{{ tuple "bin/_startvm.txt" . | include "template" | indent 4 }}
19
20---
21
22{{- range $name, $vm := .Values.vmlist }}
23{{- if $vm.enabled }}
24apiVersion: v1
25kind: ConfigMap
26metadata:
27 name: configmap-cloudconfig-{{ $name }}
28data:
29 metadata: {{ toYaml $vm.cloudconfig.metadata | indent 4 }}
30 userdata: {{ toYaml $vm.cloudconfig.userdata | indent 4 }}
31---
32{{- end }}
33{{- end }}
diff --git a/berth/templates/iter-deployment.yaml b/berth/templates/iter-deployment.yaml
new file mode 100644
index 0000000..e615a98
--- /dev/null
+++ b/berth/templates/iter-deployment.yaml
@@ -0,0 +1,98 @@
1
2{{- $envAll := . }}
3
4{{ range $name, $vm := .Values.vmlist }}
5
6# id: {{- $name }}
7{{- if $vm.enabled }}
8# vm enabled
9apiVersion: extensions/v1beta1
10kind: Deployment
11metadata:
12 name: berth-{{ $name }}
13spec:
14 replicas: 1
15 template:
16 metadata:
17 labels:
18 app: berth
19 annotations:
20 pod.beta.kubernetes.io/hostname: {{ $name }}
21 spec:
22 nodeSelector:
23 {{ $envAll.Values.labels.node_selector_key }}: {{ $envAll.Values.labels.node_selector_value }}
24 hostNetwork: false
25 hostPID: false
26 securityContext:
27 runAsUser: 0
28 containers:
29 - name: {{ $name }}
30 imagePullPolicy: IfNotPresent
31 image: {{ $envAll.Values.images.vmrunner }}
32 env:
33 - name: IMG_SOURCE
34 value: {{ $vm.vmconfig.rootfs.sourceurl }}
35 - name: IMG_TARGET
36 value: {{ $vm.vmconfig.rootfs.localtarget }}
37 - name: IMG_VCPU
38 value: "{{ $vm.vmconfig.cpu.vcpu }}"
39 - name: IMG_RAM_MB
40 value: "{{ $vm.vmconfig.cpu.ram_mb }}"
41 securityContext:
42 privileged: true
43 command:
44 - /usr/local/bin/startvm
45{{- if $vm.netconfig.ports }}
46 ports:
47{{- range $for, $port := $vm.netconfig.ports }}
48 - containerPort: {{ $port }}
49{{- end }}
50{{- end }}
51{{- if $vm.netconfig.readinessTcpProbe }}
52 readinessProbe:
53 tcpSocket:
54 port: {{ $vm.netconfig.readinessTcpProbe }}
55{{- end }}
56 volumeMounts:
57 - name: volume-startvm
58 mountPath: /usr/local/bin/startvm
59 subPath: startvm
60 - name: image
61 mountPath: /image
62 - name: dev
63 mountPath: /dev
64 - name: sys
65 mountPath: /sys
66 - name: volume-cloudinit
67 mountPath: /userdata
68 subPath: userdata
69 - name: volume-cloudinit
70 mountPath: /metadata
71 subPath: metadata
72 volumes:
73 - name: volume-startvm
74 configMap:
75 name: configmap-generic-startvm
76 defaultMode: 0755
77 - name: image
78 persistentVolumeClaim:
79 claimName: {{ $name }}
80 - name: dev
81 hostPath:
82 path: /dev
83 - name: sys
84 hostPath:
85 path: /sys
86 - name: volume-cloudinit
87 configMap:
88 name: configmap-cloudconfig-{{ $name }}
89 items:
90 - key: userdata
91 path: userdata
92 - key: metadata
93 path: metadata
94{{- else }}
95# {{ $name }} not enabled!
96{{- end }}
97---
98{{- end }}
diff --git a/berth/templates/iter-pvc.yaml b/berth/templates/iter-pvc.yaml
new file mode 100644
index 0000000..f5bdb79
--- /dev/null
+++ b/berth/templates/iter-pvc.yaml
@@ -0,0 +1,23 @@
1
2{{- range $name, $vm := .Values.vmlist }}
3# id: {{- $name }}
4{{- if $vm.enabled }}
5# vm enabled
6# {{ $vm.vmconfig.rootfs.pvc_size }} {{ $vm.vmconfig.rootfs.pvc_class }}
7kind: PersistentVolumeClaim
8apiVersion: v1
9metadata:
10 name: {{ $name }}
11spec:
12 accessModes: [ "ReadWriteOnce" ]
13 resources:
14 requests:
15 storage: {{ $vm.vmconfig.rootfs.pvc_size }}
16{{- if $vm.vmconfig.rootfs.pvc_class }}
17 storageClassName: {{ $vm.vmconfig.rootfs.pvc_class}}
18{{- end }}
19{{- else }}
20# vm disabled - skipping
21{{- end }}
22---
23{{- end }}
diff --git a/berth/values.yaml b/berth/values.yaml
index 58b1402..b1d6b2b 100644
--- a/berth/values.yaml
+++ b/berth/values.yaml
@@ -1,18 +1,12 @@
1 1
2auth:
3 ssh_key:
4
5images: 2images:
6 entrypoint: quay.io/attcomdev/kvm-manager:latest 3 vmrunner: quay.io/attcomdev/kvm-manager:latest
7 4
8labels: 5labels:
9 node_selector_key: openstack-control-plane 6 node_selector_key: openstack-control-plane
10 node_selector_value: enabled 7 node_selector_value: enabled
11 8
12volume: 9# by default in the chart's values.yaml this is empty; use and
13 class_name: 10# override file and helm --values=... for your specific values
14 size: 25Gi 11vmlist: { }
15 12
16network:
17 port: 22
18 vnc: 5900
diff --git a/examples/bogus-vm.yaml b/examples/bogus-vm.yaml
new file mode 100644
index 0000000..03a4d62
--- /dev/null
+++ b/examples/bogus-vm.yaml
@@ -0,0 +1,10 @@
1# this vm isn't fully defined, not enabling it is enough to have the
2# templates skip over the details and not barf
3vmlist:
4 - bogus-vm:
5 enabled: false
6 extra: "thang"
7 vmconfig: "this is wrong"
8 cloudconfig:
9 metadata: "{ "
10 userdata: false
diff --git a/examples/cirros-test.yaml b/examples/cirros-test.yaml
new file mode 100644
index 0000000..e2d8657
--- /dev/null
+++ b/examples/cirros-test.yaml
@@ -0,0 +1,22 @@
1vmlist:
2 cirros-test:
3 enabled: true
4 vmconfig:
5 cpu:
6 vcpu: 1
7 ram_mb: 256
8 rootfs:
9 sourceurl: http://stupidest.org/vm/cirros-0.3.5-x86_64-disk.img
10 localtarget: cirros-vm.qcow2
11 pvc_size: 128Mi
12 netconfig:
13 ports:
14 ssh: 22
15 vnc: 5900
16 cloudconfig:
17 metadata: |
18 { "uuid": "093772fe-d6a3-4eea-84bc-5966661a0c3e" }
19 userdata: |
20 #cloud-config
21 fqdn: cirros.example.com
22 bogus_not_used: "this won't hurt anthing but shouldn't be here"
diff --git a/examples/demo-ub14-apache.yaml b/examples/demo-ub14-apache.yaml
new file mode 100644
index 0000000..0d4a780
--- /dev/null
+++ b/examples/demo-ub14-apache.yaml
@@ -0,0 +1,33 @@
1vmlist:
2 demo-ub14-apache:
3 enabled: true
4 vmconfig:
5 cpu:
6 vcpu: 1
7 ram_mb: 1024
8 rootfs:
9 sourceurl: http://stupidest.org/vm/ubuntu-14.04-amd64.img
10 localtarget: ubuntu-14.04-amd64.qcow2
11 pvc_size: 5Gi
12 pvc_class: nfs
13 netconfig:
14 ports:
15 ssh: 22
16 vnc: 5900
17 readinessTcpProbe: 22
18 cloudconfig:
19 metadata: |
20 { "uuid": "apache-demo.example.com" }
21 userdata: |
22 #cloud-config
23 fqdn: apache-demo.example.com
24 users:
25 - name: root
26 ssh-authorized-keys:
27 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII+k1AYpUX7Y6+pVzkw3JPbRPNpoh7m1rZBP4Qa37Wz2 user@host"
28 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPE/zGmNx3W52ztb/2vvTcgUN7RGbq172QXGcXKAagU1 user@host"
29 - "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVeMUpC6Ba0UwyLeCCWexr/sc4kYwKW6mTSTpcQHDm+F5UqoW3pnbGPn3WSJB2AbCMP9oG0qQrLP2zHmE7fyog= otheruser@otherhost"
30 ssh_pwauth: True
31 runcmd:
32 - [ apt-get, update ]
33 - [ apt-get, install, -y, --force-yes, apache2 ]
diff --git a/examples/ub16-smp-test.yaml b/examples/ub16-smp-test.yaml
new file mode 100644
index 0000000..5551071
--- /dev/null
+++ b/examples/ub16-smp-test.yaml
@@ -0,0 +1,36 @@
1vmlist:
2 ub16-smp-test:
3 enabled: true
4 vmconfig:
5 cpu:
6 vcpu: 2
7 ram_mb: 2048
8 rootfs:
9 sourceurl: http://stupidest.org/vm/cw-ub16-test-3a.qcow2
10 localtarget: cw-ub16-test.qcow2
11 pvc_size: 7Gi
12 netconfig:
13 ports:
14 ssh: 22
15 vnc: 5900
16 cloudconfig:
17 metadata: |
18 {
19 "uuid": "093772fe-d6a3-4eea-84bc-5966661a0c3e",
20 "name": "my-name",
21 "instance-id": "my-instance-id",
22 "availability-zone": "my-az",
23 "hostname": "my-hostname",
24 "local-hostname": "my-local-hostname",
25 "launch-index": "123"
26 }
27 userdata: |
28 #cloud-config
29 fqdn: ub16-smp-test.example.com
30 users:
31 - name: root
32 ssh-authorized-keys:
33 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII+k1AYpUX7Y6+pVzkw3JPbRPNpoh7m1rZBP4Qa37Wz2 user@host"
34 - "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPE/zGmNx3W52ztb/2vvTcgUN7RGbq172QXGcXKAagU1 user@host"
35 - "ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKVeMUpC6Ba0UwyLeCCWexr/sc4kYwKW6mTSTpcQHDm+F5UqoW3pnbGPn3WSJB2AbCMP9oG0qQrLP2zHmE7fyog= otheruser@otherhost"
36 ssh_pwauth: True
diff --git a/tools/gate/setup.sh b/tools/gate/setup.sh
new file mode 100755
index 0000000..313066e
--- /dev/null
+++ b/tools/gate/setup.sh
@@ -0,0 +1,13 @@
1#!/bin/sh
2
3set -ex
4
5. /etc/os-release
6type=${ID_LIKE:=ID}
7
8if [ "$type" == "debian" ] ; then
9 apt-get update
10 apt-get install netcat-openbsd jq
11else
12 yum install netcat jq
13fi
diff --git a/tools/gate/test.sh b/tools/gate/test.sh
new file mode 100755
index 0000000..62f582d
--- /dev/null
+++ b/tools/gate/test.sh
@@ -0,0 +1,41 @@
1#!/bin/bash
2
3set -ex
4
5NS=berth
6
7helm install --name=berth --debug ./berth --values=examples/cirros-test.yaml --namespace="${NS}"
8
9# wait until we get a PODIP
10while : ; do
11 PODIP=$(kubectl -n "${NS}" get pods -o wide -o json | jq -r '.items[].status.podIP')
12 if [ -n "$PODIP" -a "null" != "$PODIP" ] ; then
13 break
14 fi
15 echo "waiting for PODIP"
16 # XXX
17 kubectl get pods --all-namespaces
18 sleep 2
19done
20
21kubectl -n "${NS}" get pods
22
23# wait for pod to come up say something on ssh
24timeout=60
25t=0
26while : ; do
27 if echo "bye" | nc "${PODIP}" 22 | grep --quiet ^SSH ; then
28 echo "VM up"
29 break
30 fi
31 if [ $t -gt $timeout ] ; then
32 exit 2
33 fi
34 t=$(($t + 5))
35 sleep 2
36done
37
38# verify we can cleanup
39helm upgrade berth ./berth --values=berth/values.yaml
40
41helm delete --purge berth