SNAS MySQL Docker Install Steps

MySQL container includes MariaDB, MySQL consumer and the REST API. This container providers everything needed to get started with collecting data from the SNAS collector Kafka integration.

Container Includes:

MariaDB 10.2 - MySQL server (listening port TCP 3306)
SNAS MySQL Consumer - Latest Consumer that puts all data into MySQL
DB_REST - Latest REST interface for MySQL database

Recommended Current Linux Distributions:

Ubuntu 16.04/Xenial
CentOS 7/RHEL 7

Installation Steps

1) Install docker

Docker host should be Linux x86_64. Follow the Docker Instructions to install docker.

2) Download the docker image

docker pull openbmp/mysql

3) Create MySQL volumes

MySQL/MariaDB uses a shared container (host) volume so that if you upgrade, restart or change the container it doesn’t lose the database contents. The database will be initialized if the volume is empty. If the volume is not empty, the database will be left unchanged.

When starting the container you will need to map a host file system to /data/mysql for the container. You do this using the -v <host path>:/data/mysql. The below examples default to the host path of /var/openbmp/mysql

On host create MySQL shared directory:

mkdir -p /var/openbmp/mysql
chmod 777 /var/openbmp/mysql

NOTE

The mode of 777 can be changed to chown but you’ll have to get that ID by looking at the file owner after starting the container.

Applying Latest Database Schema

To reinit the database and apply the latest schema use docker run with the -e REINIT_DB=1 option.

4) Add persistent configs [OPTIONAL]

On host create persistent config location

mkdir -p /var/openbmp/config
chmod 777 /var/openbmp/config

config/hosts

You can add custom host entries so that the collector will reverse lookup IP addresses using a persistent hosts file.

Run docker with -v /var/openbmp/config:/config to make use of the persistent config files.

You can also add other hosts into a container’s /etc/hosts file by using one or more –add-host flags.

5) Run docker container

Memory for MySQL

MySQL requires a lot of memory in order to run well. Currently there is not a consistent way to check on the container memory limit. The `-e MEM=size_in_GB should be specified in gigabytes (e.g. 16 for 16GB of RAM). If you fail to supply this variable, the default will use /proc/meminfo . In other words, the default is to assume no memory limit.

Environment Variables

Below table lists the environment variables that can be used with docker -e <name=value>

NAME	Value	Details
KAFKA_FQDN	hostname	REQUIRED. Fully qualified hostname that can be resolved inside docker container (e.g. `localhost`).
MEM	RAM in GB	The size of RAM allowed for container in gigabytes. (e.g. `-e MEM=15`)
GROUP_ID	string	The Kafka consumer group ID, default is ‘openbmp-mysql-consumer’
MYSQL_ROOT_PASSWORD	password	MySQL root user password. The default is OpenBMP. The root password can be changed using standard MySQL instructions. If you do change the password, you will need to run the container with this env set.
MYSQL_OPENBMP_PASSWORD	password	MySQL openbmp user password. The default is openbmp. You can change the default openbmp user password using standard mysql instructions. If you change the openbmp user password you MUST use this env.

IMPORTANT:

• You MUST define the KAFKA_FQDN as a ‘hostname’ that can be resolved inside the docker container.

• KAFKA_FQDN is used by Kafka to advertise the leader (advertised.host.name) which handles all read and write requests for a partition. If it can not be resolved, there will be no messages published or consumed (without a clear error message in the logs).

• If you are planning to have your own clients outside the container that need access to Kafka running inside the docker container, then the ‘hostname’ must be resolvable inside the container as well as on the hosts where the container and the clients are running.

Run normally

docker run -d --name=openbmp_mysql -e KAFKA_FQDN=localhost -e MEM=15 \
     -v /var/openbmp/mysql:/data/mysql -v /var/openbmp/config:/config \
     -p 3306:3306 -p 8001:8001 \
     openbmp/mysql

NOTE

Allow at least a few minutes for mysql to init the database on first start.

Monitoring/Troubleshooting

You can navigate all the log files from within the container. Connect to container using:

docker exec -it openbmp_mysql bash

Or, you can use standard docker exec commands on host to monitor the log files. To monitor logs, use:

docker exec openbmp_mysql tail -f /var/log/*.log

Or, you can monitor the docker container by getting the console logs. This is useful if the container exits due to invalid start or for another reason. To see console logs for collector, use:

docker logs openbmp_mysql

Once the container is running you can connect to MySQL database using any ODBC/JDBC/MySQL client.

You can also connect to the database REST interface on port 8001. For example:

http://localhost:8001/db_rest/v1/routers

System Start/Restart Config (Ubuntu 16.04/Xenial)

By default, the containers will not start automatically on system boot/startup. You can use the below example to instruct the container to start automatically.

You can read more at Docker Admin Guide on how to start containers automatically.

IMPORTANT

The --name=openbmp_mysql parameter given to the docker run command is used with the -a openbmp_mysql parameter below to start the container by name instead of container ID. You can use whatever name you want, but make sure to use the same name used in docker run.

cat <<END > /etc/init/mysql-openbmp.conf
description "OpenBMP MySQL container"
author "tim@openbmp.org"
start on filesystem and started docker
stop on runlevel [!2345]
respawn
script
  /usr/bin/docker start -a openbmp_mysql
end script
END

SNAS MySQL Install