79 lines
2.8 KiB
ReStructuredText
79 lines
2.8 KiB
ReStructuredText
Genesis Troubleshooting
|
|
=======================
|
|
|
|
genesis.sh
|
|
----------
|
|
|
|
Kubernetes services failures
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
Before the Armada manifests are applied, the genesis.sh script will bring basic
|
|
kubernetes services online by starting docker containers for these services.
|
|
|
|
One of the first services to be brought up is the kubernetes API. If it fails to
|
|
come up, you may see a repeated error as follows from the genesis.sh script:
|
|
|
|
.. code-block:: console
|
|
|
|
.The connection to the server apiserver.kubernetes.promenade:6443 was
|
|
refused - did you specify the right host or port?
|
|
|
|
Check that the hostname in your Genesis.yaml matches the hostname of the
|
|
machine you are trying to install onto. If they do not match, change one to
|
|
match the other. If you change Genesis.yaml, then re-generate the Promenade
|
|
payloads.
|
|
|
|
If the hostnames match, check the container logs under /var/log/pods to see the
|
|
reason for the provisioning failure. (``kubectl logs`` function will not be
|
|
available if the API container is not running).
|
|
|
|
Armada failures
|
|
^^^^^^^^^^^^^^^
|
|
|
|
When executing genesis.sh, you may encounter failures from Armada in the
|
|
provisioning of other containers. For example:
|
|
|
|
.. code-block:: console
|
|
|
|
CRITICAL armada [-] Unhandled error: armada.exceptions.tiller_exceptions.ReleaseException: Failed to Install release: barbican
|
|
|
|
Use ``kubectl logs`` on the failed pod to determine the reason for the failure.
|
|
E.g.:
|
|
|
|
.. code-block:: console
|
|
|
|
sudo kubectl logs barbican-api-5b8bccdf8f-x7sld --namespace=ucp
|
|
|
|
Other errors may point to configuration errors. For example:
|
|
|
|
.. code-block:: console
|
|
|
|
CRITICAL armada [-] Unhandled error: armada.exceptions.source_exceptions.GitLocationException: master is not a valid git repository.
|
|
|
|
In this case, the git branch name was inadvertently substituted for the git URL
|
|
in one of the chart definitions in ``bootstrap-armada.yaml``.
|
|
|
|
Post-run failures
|
|
^^^^^^^^^^^^^^^^^
|
|
|
|
At its conclusion, the genesis script will output the list of containers
|
|
provisioned and their status, as reported by kubernetes. It is possible that
|
|
some containers may not be in a Running state. E.g.:
|
|
|
|
.. code-block:: console
|
|
|
|
ucp promenade-api-6696769cd-qwpzf 0/1 ImagePullBackOff 0 10h
|
|
|
|
For general failures, ``kubectl logs`` may be used as in the previous section.
|
|
In this case, it was necessary to run ``kubectl describe`` on the pod to get the
|
|
details of the image pull failure. E.g.:
|
|
|
|
.. code-block:: console
|
|
|
|
kubectl describe pod promenade-api-7dc54d47c-qw27m --namespace=ucp
|
|
|
|
In this particular incident report, the problem was a missing certificate on the
|
|
bare metal node which caused the image download to fail. Installing the
|
|
certificate, restarting the docker service, and then waiting for the container
|
|
to retry resolved this particular issue.
|