SNAS MySQL Docker Install Steps
MySQL container includes MariaDB, MySQL consumer and the REST API. This container providers everything needed to get started with collecting data from the SNAS collector Kafka integration.
- MariaDB 10.2 - MySQL server (listening port TCP 3306)
- SNAS MySQL Consumer - Latest Consumer that puts all data into MySQL
- DB_REST - Latest REST interface for MySQL database
Recommended Current Linux Distributions:
- Ubuntu 16.04/Xenial
- CentOS 7/RHEL 7
1) Install docker
Docker host should be Linux x86_64. Follow the Docker Instructions to install docker.
2) Download the docker image
docker pull openbmp/mysql
3) Create MySQL volumes
MySQL/MariaDB uses a shared container (host) volume so that if you upgrade, restart or change the container it doesn’t lose the database contents. The database will be initialized if the volume is empty. If the volume is not empty, the database will be left unchanged.
When starting the container you will need to map a host file system to /data/mysql for the container. You do this using the
-v <host path>:/data/mysql. The below examples default to the host path of
On host create MySQL shared directory:
mkdir -p /var/openbmp/mysql chmod 777 /var/openbmp/mysql
The mode of 777 can be changed to chown
but you’ll have to get that ID by looking at the file owner after starting the container.
Applying Latest Database Schema
To reinit the database and apply the latest schema use docker run with the
-e REINIT_DB=1 option.
4) Add persistent configs [OPTIONAL]
On host create persistent config location
mkdir -p /var/openbmp/config chmod 777 /var/openbmp/config
You can add custom host entries so that the collector will reverse lookup IP addresses using a persistent hosts file.
Run docker with
-v /var/openbmp/config:/config to make use of the persistent config files.
You can also add other hosts into a container’s /etc/hosts file by using one or more –add-host flags.
5) Run docker container
Memory for MySQL
MySQL requires a lot of memory in order to run well. Currently there is not a consistent way to check on the container memory limit. The `
-e MEM=size_in_GB should be specified in gigabytes (e.g. 16 for 16GB of RAM). If you fail to supply this variable, the default will use /proc/meminfo . In other words, the default is to assume no memory limit.
Below table lists the environment variables that can be used with
docker -e <name=value>
|KAFKA_FQDN||hostname||REQUIRED. Fully qualified hostname that can be resolved inside docker container (e.g.
|MEM||RAM in GB||The size of RAM allowed for container in gigabytes. (e.g.
|GROUP_ID||string||The Kafka consumer group ID, default is ‘openbmp-mysql-consumer’|
|MYSQL_ROOT_PASSWORD||password||MySQL root user password. The default is OpenBMP. The root password can be changed using standard MySQL instructions. If you do change the password, you will need to run the container with this env set.|
|MYSQL_OPENBMP_PASSWORD||password||MySQL openbmp user password. The default is openbmp. You can change the default openbmp user password using standard mysql instructions. If you change the openbmp user password you MUST use this env.|
• You MUST define the KAFKA_FQDN as a ‘hostname’ that can be resolved inside the docker container.
• We recommend to set it to ‘localhost’ (or ‘127.0.0.1’) if you are not planning to have your own clients (consumers or producers) outside this container.
• KAFKA_FQDN is used by Kafka to advertise the leader (advertised.host.name) which handles all read and write requests for a partition. If it can not be resolved, there will be no messages published or consumed (without a clear error message in the logs).
• If you are planning to have your own clients outside the container that need access to Kafka running inside the docker container, then the ‘hostname’ must be resolvable inside the container as well as on the hosts where the container and the clients are running.
docker run -d --name=openbmp_mysql -e KAFKA_FQDN=localhost -e MEM=15 \ -v /var/openbmp/mysql:/data/mysql -v /var/openbmp/config:/config \ -p 3306:3306 -p 8001:8001 \ openbmp/mysql
Allow at least a few minutes for mysql to init the database on first start.
You can navigate all the log files from within the container. Connect to container using:
docker exec -it openbmp_mysql bash
Or, you can use standard docker exec commands on host to monitor the log files. To monitor logs, use:
docker exec openbmp_mysql tail -f /var/log/*.log
Or, you can monitor the docker container by getting the console logs. This is useful if the container exits due to invalid start or for another reason. To see console logs for collector, use:
docker logs openbmp_mysql
Once the container is running you can connect to MySQL database using any ODBC/JDBC/MySQL client.
You can also connect to the database REST interface on port 8001. For example:
System Start/Restart Config (Ubuntu 16.04/Xenial)
By default, the containers will not start automatically on system boot/startup. You can use the below example to instruct the container to start automatically.
You can read more at Docker Admin Guide on how to start containers automatically.
--name=openbmp_mysqlparameter given to the
docker runcommand is used with the
-a openbmp_mysqlparameter below to start the container by name instead of container ID. You can use whatever name you want, but make sure to use the same name used in docker run.
cat <<END > /etc/init/mysql-openbmp.conf description "OpenBMP MySQL container" author "firstname.lastname@example.org" start on filesystem and started docker stop on runlevel [!2345] respawn script /usr/bin/docker start -a openbmp_mysql end script END