Running Storm in local mode using docker
First, we need to install docker, i am using ubuntu 20.04. https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-20-04
sudo snap install docker
Download the storm docker image:
sudo docker pull storm
Apache Zookeeper is a must for running a Storm cluster. Start it first. Since the Zookeeper “fails fast” it’s better to always restart it.
sudo docker run -d --restart always --name some-zookeeper zookeeper
The Nimbus daemon has to be connected with the Zookeeper. It’s also a “fail fast” system.
sudo docker run -d --restart always --name some-nimbus --link some-zookeeper:zookeeper storm storm nimbus
Persisting data
sudo docker run -it -v /logs -v /data storm storm nimbus
Finally start a single Supervisor node. It will talk to the Nimbus and Zookeeper.
sudo docker run -d --restart always --name supervisor --link some-zookeeper:zookeeper --link some-nimbus:nimbus storm storm supervisor
Now you can submit a topology to our cluster.
sudo docker run --link some-nimbus:nimbus --rm -it -v $(pwd):/path/ storm storm jar /path/target/YOURJAR-SNAPSHOT.jar com.b2wdigital.MAIN_CLASSL "/path/config/CONFIG_FILE.yml"
Optionally, you can start the Storm UI.
sudo docker run -d -p 8080:8080 --restart always --name ui --link some-nimbus:nimbus storm storm ui
List all dockers containers
docker ps
To access a container type
sudo docker exec -it CONTAINER_ID /bin/bash
The supervisor controls the topologies, to see the logs you need to acess the supervisor.
sudo docker exec -it d0a9495c68a2 /bin/bash
To list all topologies
storm list
To kill a topology
storm kill topology_name