Readme for airship-utils
Along with the readme change the following small changes were made: - make it possible to install arbitrary packages with install_packages.sh script - add rsh and nis to list of default blacklisted packages to satisfy initial requirements. Change-Id: I84d81422e11f5de2d99fa30b1974513f62313386
This commit is contained in:
Andrey Volkov
Andrey Volkov
parent
b1b4575eff
commit
c6a9abf0a9
190
README.md
190
README.md
|
|
@ -1,65 +1,185 @@
|
|||
# docker-aptly
|
||||
# Airship-utils
|
||||
|
||||
## Features
|
||||
Airship-utils is a collection of tools that can accompany other airship-* projects.
|
||||
Currently, airship-utils contains the following components:
|
||||
|
||||
- Packages are downloaded during the docker image build
|
||||
- GPG keys for signature may be generated during the docker image build or existing ones are used
|
||||
- Nginx blacklist support at runtime
|
||||
- miniMirror
|
||||
|
||||
## Quickstart
|
||||
## miniMirror
|
||||
|
||||
The main difference with the upstream repo is packages saved inside a docker image.
|
||||
During the image building /opt/update_mirror_ubuntu.sh is called to create mirrors, update them,
|
||||
merge all in one snapshot and publish it. By default, a new GPG key is generated for making a signature for repo.
|
||||
miniMirror is a combination of package mirroring tool (Aptly) and a web server (Nginx)
|
||||
packed into a container and a helm chart for k8s deployment. See [1] for additional info.
|
||||
|
||||
There are two modes: filtered build that fetches only packages specified in assets/packages and
|
||||
unfiltered build that fetches all packages. The filtered build is used by default.
|
||||
### Features
|
||||
|
||||
To fetch all packages the following command can be used:
|
||||
- Packages are downloaded during the docker image build.
|
||||
A list of packages can be specified with particular versions or
|
||||
without them to get the current snapshots. Currently miniMirror
|
||||
focuses on Ubuntu repositories only.
|
||||
- GPG key for signature can be generated during the docker image build
|
||||
or existing one can be used. To make a signature for packages Aptly
|
||||
requires a private key, it is not possible to use a signature from
|
||||
original mirror.
|
||||
- Packages blacklist support at runtime. An additional Nginx
|
||||
configuration can be provided to block specific package
|
||||
installation. By default packages contains the following regexp in name are blocked:
|
||||
- telnet
|
||||
- ftp
|
||||
- \brsh\b
|
||||
- \bnis\b
|
||||
|
||||
### How to build miniMirror image?
|
||||
|
||||
#### General desription
|
||||
|
||||
As Debian packages are downloaded and saved inside a docker image, it
|
||||
may take some time to build the image. The process of building the image
|
||||
includes the following steps:
|
||||
|
||||
- Prepare GPG environment (see assets/startup.sh for details).
|
||||
- Put into right places or generate GPG key depending on the build
|
||||
configuration. See "Step by step guide" step 2 for details.
|
||||
- Update GPG keyring.
|
||||
- Create packages infrastructure (see assets/update_mirror_ubuntu.sh).
|
||||
- Create Aptly mirrors.
|
||||
- Fetch packages from upstream repositories according to the mirror
|
||||
configurations.
|
||||
- Merge repositories. For example, by default xenial, xenial-updates,
|
||||
and xenial-security are used. Packages from each repository are
|
||||
merged into one with latest wins strategy.
|
||||
- Publish repository to directory Nginx will serve static files
|
||||
from.
|
||||
|
||||
#### Configuration
|
||||
|
||||
The following build args are available:
|
||||
|
||||
Repository configuration:
|
||||
|
||||
- UPSTREAM_URL - a URL packages are downloaded from (mirror URL).
|
||||
- UPSTREAM_KEY_URL - a URL for public GPG key if it is not on a default location.
|
||||
- UBUNTU_RELEASE - a release name for a Ubuntu distribution.
|
||||
- COMPONENTS - a list of repository components separated by space.
|
||||
For example, values can be main, universe, restricted, multiverse [2].
|
||||
- REPOS - a list of repository types separated by space.
|
||||
For example xenial, xenial-updates, xenial-security, xenial-backports.
|
||||
|
||||
Packages configuration:
|
||||
- MODE - a string determining if all packages should be downloaded or
|
||||
specific only. Possible values: packages or all.
|
||||
- PACKAGE_FILE - a file name where a list of packages is saved. If
|
||||
MODE=packages the file must be available in assets/packages
|
||||
directory.
|
||||
|
||||
GPG key configuration:
|
||||
By default GPG key for making package signature is generated during
|
||||
the build. If you have a GPG key already you can put private and
|
||||
public keys in assets/gpg dir. Keys must have special names: aptly.sec
|
||||
and aptly.pub. You may configure GPG key params via the following arguments:
|
||||
- FULL_NAME - a full name for a GPG key.
|
||||
- EMAIL_ADDRESS - an email for a GPG key.
|
||||
- GPG_PASSWORD - a passphrase for a GPG key. This can be used both for
|
||||
GPG key generation and GPG key usage.
|
||||
|
||||
Nginx configuration:
|
||||
- HOSTNAME - server_name configuration for Nginx.
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/urpylka/docker-aptly.git
|
||||
docker build docker-aptly --build-arg MODE=all
|
||||
git clone https://git.openstack.org/openstack/airship-utils
|
||||
docker build airship-utils \
|
||||
--UBUNTU_RELEASE=bionic \
|
||||
--build-arg FULL_NAME="John Smith" \
|
||||
--build-arg EMAIL_ADDRESS="john.smith@example.com" \
|
||||
--build-arg GPG_PASSWORD="PickAPassword" \
|
||||
--build-arg HOSTNAME=_
|
||||
```
|
||||
|
||||
By default GPG key for making package signature are generated during the build.
|
||||
You may configure GPG key params via build arguments: FULL_NAME, EMAIL_ADDRESS, and GPG_PASSWORD, like:
|
||||
### Step by step guide
|
||||
|
||||
This is an example of how miniMirror can be used.
|
||||
|
||||
1) Prepare a list of packages needed for a miniMirror image.
|
||||
|
||||
```bash
|
||||
docker build docker-aptly \
|
||||
--build-arg FULL_NAME="First Last" \
|
||||
--build-arg EMAIL_ADDRESS="youremail@example.com" \
|
||||
--build-arg GPG_PASSWORD="PickAPassword"
|
||||
cd airship-utils
|
||||
cat << 'EOF' > assets/packages/my_packages
|
||||
mysql-client-5.7 (= 5.7.24-0ubuntu0.16.04.1)
|
||||
mysql-client-core-5.7
|
||||
postgresql-client-9.5 (= 9.5.14-0ubuntu0.16.04)
|
||||
postgresql-client-common
|
||||
EOF
|
||||
```
|
||||
|
||||
If you have a GPG key already you can put private and public key in assets/gpg dir.
|
||||
Keys must have special names: aptly.sec and aptly.pub
|
||||
For example:
|
||||
2) Prepare a GPG key for making package signature.
|
||||
|
||||
GPG public and private keys should be named as assets/gpg/aptly.pub and assets/gpg/aptly.key.
|
||||
|
||||
```bash
|
||||
cp <my private key> docker-aptly/assets/gpg/aptly.sec
|
||||
cp <my public key> docker-aptly/assets/gpg/aptly.pub
|
||||
|
||||
docker build docker-aptly \
|
||||
--build-arg GPG_PASSWORD="GPG passphrase for my private key"
|
||||
mkdir -p /opt/aptly
|
||||
export FULL_NAME='John Smith'
|
||||
export EMAIL_ADDRESS='john.smith@example.com'
|
||||
export GPG_PASSWORD='my_passphrase'
|
||||
bash assets/gpg_batch.sh
|
||||
gpg -v --batch --gen-key /opt/gpg_batch
|
||||
mv /opt/aptly/* assets/gpg/
|
||||
rm /opt/gpg_batch
|
||||
```
|
||||
|
||||
To use the Nginx blacklist feature a volume with Nginx config has to be mounted at runtime.
|
||||
By default, the following keywords are blocked: telnet, ftp.
|
||||
If no volume is mounted then no blacklist will be used.
|
||||
3) Build docker image.
|
||||
|
||||
```bash
|
||||
docker build . -t mini-mirror \
|
||||
--build-arg PACKAGE_FILE=my_packages \
|
||||
--build-arg GPG_PASSWORD="$GPG_PASSWORD"
|
||||
```
|
||||
|
||||
4) Test miniMirror container.
|
||||
|
||||
Start miniMirror container.
|
||||
|
||||
```bash
|
||||
docker run -d \
|
||||
--publish 8080:80 \
|
||||
--volume $(pwd)/assets/nginx:/opt/nginx \
|
||||
--name mini-mirror \
|
||||
mini-mirror
|
||||
```
|
||||
|
||||
Run another container and install packages there.
|
||||
|
||||
```bash
|
||||
docker run --network host \
|
||||
--env PACKAGES='mysql-client-5.7 postgresql-client-9.5' \
|
||||
--name target \
|
||||
--volume $(pwd)/tools:/opt \
|
||||
ubuntu:16.04 /opt/install_packages.sh
|
||||
```
|
||||
|
||||
### How to blacklist miniMirror packages
|
||||
|
||||
To use the Nginx blacklist feature a volume with Nginx config has to
|
||||
be mounted at runtime. If no volume is mounted then no blacklist will
|
||||
be used.
|
||||
|
||||
```bash
|
||||
docker run \
|
||||
--name aptly \
|
||||
--name mini-mirror \
|
||||
--detach \
|
||||
--publish 8080:80 \
|
||||
--volume $(pwd)/assets/nginx:/opt/nginx \
|
||||
aptly:test
|
||||
mini-mirror
|
||||
```
|
||||
___
|
||||
|
||||
For additional docs see https://github.com/amadev/docker-aptly
|
||||
## References
|
||||
|
||||
* [1] https://review.openstack.org/#/c/611376/
|
||||
* [2] https://help.ubuntu.com/community/Repositories
|
||||
|
||||
## Copyright
|
||||
|
||||
* Copyright 2018 AT&T Intellectual Property
|
||||
* Copyright 2018 Artem B. Smirnov
|
||||
* Copyright 2016 Bryan J. Hong
|
||||
* Licensed under the Apache License, Version 2.0
|
||||
|
|
|
|||
|
|
@ -1,3 +1,3 @@
|
|||
location ~ (telnet|ftp) {
|
||||
location ~ (telnet|ftp|\brsh\b|\bnis\b) {
|
||||
return 404;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -9,5 +9,8 @@ apt install -y curl
|
|||
curl -s localhost:8889/aptly_repo_signing.key | apt-key add -
|
||||
echo 'deb http://localhost:8889 xenial main' > /etc/apt/sources.list
|
||||
apt-get update
|
||||
apt-cache policy accountsservice
|
||||
apt-get install -y accountsservice
|
||||
PACKAGES=${PACKAGES:-accountsservice}
|
||||
for package in $PACKAGES; do
|
||||
apt-cache policy "$package"
|
||||
apt-get install -y "$package"
|
||||
done
|
||||
|
|
|
|||
Loading…
Reference in New Issue