Why do you need Docker?
- You want to set up an application stack with different technologies (database server, client/web servers, monitoring, etc.)
- You need to ensure all those components are compatible with the OS, as well as with each other
- When you upgrade a component to a newer version, you need to check compatibility between components again!
- Developers can also have different workspaces
- When a new teammate joins the project, setup can become tedious (many instructions/commands, check OS and versions of components, etc.)
- With docker: you can run each component in a separate container with its own dependencies & own libraries
- You only need to build the docker configuration once and developers can get started with a simple docker command, just need docker installed on their systems
- Ensures that components will run the same way in different workspaces every time
<br/>
What does Docker provide:
1. Image
- Package or template
- Used to create one or more containers
- Specifies dependencies/software
2. Containers
- Running instances of images
- Own environments and set of processes
- Runs the same way everywhere
<br/>
Create express app
1. Create new folder and move to that folder
2. Create express app
npx express-generator
3. Install dependencies
npm install
4. Run the express app
npm start
<br/>
Testing Docker
docker run docker/whalesay cowsay boo
1. Docker will first check if you have the image already, but if you don’t have it, Docker will automatically download the image from dockerhub
- Dockerhub is a public repository of images
2. When the image is found, Docker will instantiate a container and execute a command inside the container (in this case, it will print a whale saying ‘boo’)
3. We can also create our own images
<br/>
Prepare Dockerfile for express app
# base image (all dockerfiles should have this)
FROM node:12.2.0
# set working directory
WORKDIR /app
### install and cache app dependencies
# copy package.json & package-lock.json to ./ inside the Docker image
COPY package*.json ./
# install dependencies of the app
RUN npm install
# copy the source code to the Docker image
COPY . .
# expose the port where the app will listen to
EXPOSE 3000
# start the server
CMD ["npm", "start"]
1. FROM
- Defines what the base OS/image should be for the container
- Can use any existing image (from Dockerhub or something you created)
2. WORKDIR
- Sets the working directory in the image for the instructions that follow it
3. COPY
- Copies files from the local system onto the Docker image
- In the second COPY statement, the first . means all files in the same directory as the Dockerfile (in this case, all files of your express app)
- the second . indicates the current directory in your image (in this case, /app)
4. RUN
- Instructs Docker to run a particular command in the image
5. EXPOSE
- Tells Docker what port the container will listen to
6. CMD
- Defines the command that will be executed within the container when it starts
<br/>
Build Docker image
docker build . -t bootcamp_express
1. The . is the path to the Dockerfile
2. Layered architecture while building
- Has steps
- Docker will reuse existing steps for the succeeding builds (amazing!)
<br/>
List images
docker images
1. You should see bootcamp_express
2. Remember: images are just templates and not the actual running instances
<br/>
Run a container using the bootcamp_express image
1. To create a running container,
docker run bootcamp_express
- You will see output from the terminal. Docker executed the
npm start
for you - With this, you were able to create a container with everything your app needs. You don’t need those dependencies installed on your system anymore to make the app run.
2. Check running containers using
docker ps
3. List all containers using
docker ps -a
4. Docker gives container names automatically
5. You can specify a name using:
docker run --name express_app bootcamp_express
6. Notice: it did not rename the first container, instead it created a new one
7. Stop the first container using
docker stop <container_name>
8. Delete the first container using
docker stop <container_name>
9. Now that our express app is running, try to access localhost:3000 (does not work)
EXPOSE 3000
tells Docker that the app is listening to port 3000 INSIDE the Docker network, and NOT your host- We need to tell Docker that we want to map a host port to the Docker container’s port 3000
- Stop and delete the old container, then map the host port to the container port
docker stop express_app docker rm express_app docker run --name express_app -p 3000:3000 bootcamp_express
10. Now, if we access http://localhost:3000 from our browser, we can access the app
<br/>
Start a MySQL container
1. You can also run a container by pulling an existing image from Dockerhub
2. Run a MySQL container by using
docker run --name mysql -p 3306:3306 -e MYSQL_ROOT_PASSWORD=password -e MYSQL_DATABASE=mydb mysql:5.7
3. We need to set MYSQL_ROOT_PASSWORD environment variable for this (MySQL’s rules: https://hub.docker.com/_/mysql/)
4. We can set the env vars using -e
5. We also set the port mapping so we can access MySQL from the host
6. We will also be using version 5.7 of MySQL
7. Check running containers (docker ps
)
8. Check MySQL Workbench if server is running
<br/>
Set up db-migrate
1. Now, we need to set up a tool to execute MySQL queries automatically from the express app
2. We will use db-migrate, so we don’t need to manually set up the db every time we instantiate a container (ex. CREATE TABLEs)
3. Install db-migrate using
npm install db-migrate
4. Check if db-migrate is installed
node ./node_modules/db-migrate/bin/db-migrate
5. Create a database
directory in your express app and create database.json file
6. Add the following configuration details in database.json
{
"dev": {
"driver": "mysql",
"database": "mydb",
"host": "localhost",
"port": "3306",
"user": "root",
"password": "password",
"multipleStatements": true
}
}
7. Install the MySQL driver used by db-migrate
npm install db-migrate-mysql
8. Create a db using the configuration we created above
node ./node_modules/db-migrate/bin/db-migrate db:create mydb --config database/database.json -e dev
- The --config tells db-migrate what configuration file to use
- The -e tells db-migrate which settings to use defined in the configuration
9. Check MySQL Workbench if your db was created
10. Create a table using migration scripts
Create a migrations folder inside database
Generate migration templates using
node ./node_modules/db-migrate/bin/db-migrate create init_tables --config ./database/database.json -m ./database/migrations --sql-file
- The -m tells db-migrate where to put the migration templates
- The --sql-file option generates a .sql file for us
- To make .sql file generation the default, add "sql-file": true in database.json
- Go to *-init-tables-up.sql and add a simple CREATE TABLE statement
CREATE TABLE `users` ( `id` INT NOT NULL, `name` VARCHAR(16) NOT NULL, `password` VARCHAR(16) NOT NULL );
11. Run the migration using
node ./node_modules/db-migrate/bin/db-migrate up -e dev --config ./database/database.json -m ./database/migrations
12. Check MySQL Workbench if your new table was created
<br/>
Putting the express app and MySQL servers together
1. Now, we know how to start our express app and a MySQL server using Docker but what we really want to happen is running both services together with 1 command
2. Create a docker-compose.yml file with the express app
- In this example, each service maps to 1 container
- The volumes field mounts sample-app to the app directory inside the image
- Whenever this container instantiates, it will contain the latest code from your local machine
version: "3"
services:
express_app:
build: ./sample-app
ports:
- 3000:3000
volumes:
- ./sample-app/:/app
3. Run the container using
docker-compose up
- It automatically builds the express app using the Dockerfile found in sample-app
- It also sets the port mapping
4. Verify if you can access localhost:3000 from your browser (it should work)
5. To stop docker-compose, run
docker-compose down
6. Add the MySQL service to docker-compose.yml
db_server:
image: mysql:5.7
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=mydb
7. Add environment variables to your express app
- We have already put an environment variable in the MySQL service. We can also put env vars in our express app
- Now we are going to add environment variables for your database config (database.json)
- Add these in your express app service
express_app: build: ./sample-app ports: - 3000:3000 volumes: - ./sample-app/:/app environment: - DB_NAME=mydb - DB_HOST=db_server - DB_PORT=3306 - DB_USER=root - DB_PASSWORD=password
- Use the variables in database.json
- db-migrate will automatically replace those variables with the values you set under environment
{ "dev": { "driver": "mysql", "database": { "ENV": "DB_NAME" }, "host": { "ENV": "DB_HOST" }, "port": { "ENV": "DB_PORT" }, "user": { "ENV": "DB_USER" }, "password": { "ENV": "DB_PASSWORD" }, "multipleStatements": true } }
8. Run both services together using docker-compose up
9. Congratulations! You started your express app and MySQL service with 1 command
10. If you check MySQL workbench, though, your db and table don’t exist
<br/>
Start up with set up
1. We need to run the db-migrate db:create and up commands when our express app begins
2. Stop and remove the containers using docker-compose down
3. Add a depends_on field in the express_app service
- This is to ensure the MySQL server starts before the express app
express_app: build: ./sample-app ports: - 3000:3000 volumes: - ./sample-app/:/app environment: - DB_NAME=mydb - DB_HOST=db_server - DB_PORT=3306 - DB_USER=root - DB_PASSWORD=password depends_on: - db_server
4. Create a start.sh
file in sample-app and put the following
- This also uses the environment variable (DB_NAME) you created earlier
#!/bin/bash
# run create/update migration scripts
echo "create-if-not-exist database"
node ./node_modules/db-migrate/bin/db-migrate db:create ${DB_NAME} -e dev --config ./database/database.json
echo "finished create-if-not-exist database"
echo "update database"
node ./node_modules/db-migrate/bin/db-migrate up -e dev --config ./database/database.json -m ./database/migrations
echo "finished update database"
# run node app
echo "running node app"
npm start
5. Add a command field to the express_app service
express_app:
build: ./sample-app
ports:
- 3000:3000
volumes:
- ./sample-app/:/app
environment:
- DB_NAME=mydb
- DB_HOST=db_server
- DB_PORT=3306
- DB_USER=root
- DB_PASSWORD=password
depends_on:
- db_server
command: "bash start.sh db_server:3306"
- This will run the db-migrate commands that we need
- Putting this will override the CMD we put in the Dockerfile, so we need to start the server here using npm start
6. Try running docker-compose up now
7. We will get an error. This is because we tried to send a request to the MySQL server before it finished setting up
- We need to wait for the MySQL server to finish before running db-migrate
- Stop docker-compose
8. Put these before the db-migrate commands in your start.sh to add a wait mechanism
# wait for db Docker to start up
: ${SLEEP_LENGTH:=2}
wait_for() {
echo Waiting for $1 to listen on $2...
while ! nc -z $1 $2; do echo sleeping; sleep $SLEEP_LENGTH; done
}
for var in "$@"
do
host=${var%:*}
port=${var#*:}
wait_for $host $port
done
9. We also need to download a dependency (netcat) to use the wait mechanism. Add this in your Dockerfile in sample-app
# base image (all dockerfiles should have this)
FROM node:12.2.0
# set working directory
WORKDIR /app
### install and cache app dependencies
# copy package.json & package-lock.json to ./ inside the Docker image
COPY package*.json ./
# install dependencies of the app
RUN npm install
# install dependencies for start.sh
RUN apt-get update && apt-get install -y netcat
# copy the source code to the Docker image
COPY . .
# expose the port where the app will listen to
EXPOSE 3000
# start the server
CMD ["npm", "start"]
- Make sure this change is reflected. Delete the old image so a new image will be built
- To delete an image, use docker rmi <image_name>
10. Finally, change the command in the docker-compose.yml to include the parameters in the start.sh (the hostname and port number)
express_app: build: ./sample-app ports: - 3000:3000 volumes: - ./sample-app/:/app environment: - DB_NAME=mydb - DB_HOST=db_server - DB_PORT=3306 - DB_USER=root - DB_PASSWORD=password depends_on: - db_server command: "bash start.sh db_server:3306"
11. Now, try running docker-compose up
12. Verify on MySQL Workbench if your database and table were created. 13. Congratulations! Now with just 1 command, you were able to start a MySQL server, start an express server, and also create a database with 1 table.