Dockerize Selenium Tests and Run As Part of GitHub Actions Pipeline

Automation Approach

Test automation is one of the most important parts in the modern software development process and it is required to be placed in order to implement modern DevOps best practices. With an automation in your development life cycle, developing your application continuously becomes more structural, free of huge manual testing concerns and matching some level of quality standard everytime.

Today, I am going to tell you how to integrate your selenium test in your application pipeline and run them as part of your CI/CD process. Therefore, I will create one demo react application, one selenium framework with demo test, dockerize the app and tests, prepare demo Github Actions pipeline, docker hub for keeping images and many more interesting things. And of course it will be kind of MVP and many more things can be added/removed. So if you are ready, let’s get started to build this.

Note: Tools are chosen randomly. You can choose any other tools for any operation, i.e. cypress for automation, CircleCI for CI/CD. I choose selenium because generally people tend to run selenium in a virtual host machine and not try to dockerize it so I just wanted to show how you can apply this with selenium.

Installation:

You need to install these tools in order to build the project.

Required tools:

• Java 8+
• Maven
• Docker desktop
• Nodejs
• Github account
• Intellij Idea
• VS Code

First MVP — Create React App

I will create a react app first and then run selenium tests against this application. I will use the create react app command to create a react application and it creates a ready application in a couple of seconds.

Open terminal and navigate where you want to create the app folder.

Run:

npx create-react-app my-app

this will create a react application in my-app folder. You can change directory to this folder and run react app

create-react-app output

cd my-app
npm start

npm start output

Open the browser and navigate to localhost:3000

localhost:3000

That’s it. You have one application running now. But as you notice, it is running in local machine so, before moving to selenium test, let’s quickly dockerize the application and run it in a docker container.

Dockerize React app

First of all, it would be good to keep the npm start command in a separate file so you can just quickly call it with the sh command. So let’s create a bin folder in the root and create a run.sh file in this bin folder. And put react app start command there

#!bin/bash
npm start

Secondly you need to create a dockerfile. Let’s keep it in the .docker/frontend folder to keep the project more organized. Create .docker/frontend in the root of your project and create a Dockerfile.dev file in it.

FROM node:16-alpine

WORKDIR /app

COPY . /app/
RUN npm install

ENTRYPOINT [ "sh", "bin/run.sh" ]

and since you will use it for only development and test purposes, name it Dockerfile.dev.

You can test your setup with these command:

docker build -f .docker/frontend/dockerfile.dev -t frontend .
docker run -p 3000:3000 frontend

Open the localhost:3000 and verify everything is correct.

Now, you are good to go with the selenium test.

Selenium Project setup
I won’t tell you every detail of the selenium tests since it is not our topic today but just share the general structure of the test framework. You can check the test repo from this link. https://github.com/selcuktemizsoy/demo-ui-selenium-cucumber

Here is the general project structure

Selenium project structure

I will pass my test config as .env file so that in the future I can use different values and run my test in a different environment with another file.

Here is the how .env file looks like:

.env file

Note: 172.17.0.1 is the docker host ip address. I will run all the things on docker so I am configuring my project to run with docker. But for futher test development, it would be better to run test in local machine without dealing with docker build so in order to achieve this you can create one moe .env file and name it as .env.local and use it for local test development purpose.

.env.local

host=react app host address.

grid=selenium grid hub.

browser=browser type (I will run test in selenium grid so chose remote webdriver)

I have adjusted my driver class to take the browser and grid address here.

Driver

It is also supporting chrome and firefox. But for now I will continue with the remote firefox driver. (Note: I have faced some errors for remote chrome so I continued with firefox. I couldn’t find time to investigate the problem but if I could find a solution, I will push it to the test repo.

My test written in cucumber:

Feature: My first feature
 Scenario: my first scenario
  When I go to homepage
  Then homepage should open

And here is the step implementation.

Step definition

As you see, it is simply going to host and check if the text is correct or not.

To test the initial setup, you can choose the local config to run your test. It doesn’t require to have grid up and running so you can just run them:

env $(cat .env.local) mvn test

Note: Here I am giving my config to test with the env command. And then running the test with the mvn command.

Dockerize Selenium Test suite

Okay, you have a demo app running in docker container and you also have one selenium test running against react app. Since you are using a separate repo, and you may want to run your test in any environment, it is better to dockerize your test and keep it in a separate registry. Of course there are lots of other options but dockerizing the test and run them as part of the pipeline makes your job easier and you can deploy your tests to wherever you want if you have docker version of your test. So let’s create .docker folder inside of the selenium repo and create a dockerfile:

FROM maven:3.6-jdk-11
WORKDIR /tests
COPY ../pom.xml .
RUN mvn install
COPY .. .
ENTRYPOINT [ "tail", "-f", "/dev/null" ]

Lastly, create a bin folder again and put a test script inside of it. run.sh:

#!/bin/bash
env $(cat .env) mvn test

I gave an unnecessary command for entry point since in pipeline, you will trigger your test as a separate step, as a result you need to start your container first and then give this run.sh.

Now the automation framework is almost done. You didn’t test it because you haven’t completed selenium grid installation so that it will give an error when you try to start your test container. So just postpone it for now and you will get back there later.

Github actions for your tests

Now, since you have a docker container version of your tests, you need to push this docker image to somewhere else to reuse them. In a real development environment you will use your organization docker container registry and it will be private. Since this is not a real project I will use the public registry in docker hub. To be able to push test images to docker hub, you need to have some workflow for your repository. As I said before, this project will use github actions, now it is time to configure the github workflow to build the test container and push it to the docker hub registry.

Let’s create a .github/workflow folder and put github-actions.yaml in this folder.

This is the file that tells github to run some workflow after you push some changes to your test suite.

Here is the github actions file:

name: Selenium test CI pipeline
on:
  push:
    branches: [ master ]
jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout the code
        uses: actions/checkout@v3
      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_HUB_USERNAME }}
          password: ${{ secrets.DOCKER_HUB_PASSWORD }}
      - name: Setup Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Build images
        uses: docker/build-push-action@v3
        with:
          context: .
          file: .docker/Dockerfile
          push: true
          tags: selcuktemizsoy/selenium-demo:latest

It starts with name to name it workflow. And you can state when this workflow runs. In this case, it is only running when somebody pushes to master branch.

In the jobs section, it is defining which job will be running. First, you need to checkout the code and then you have to make docker login to be able to push your images to docker hub. And then setting up docker buildx to build and push docker images. Finally, it is time to build and push docker images to docker hub.

You can find more information related to github action here. https://docs.github.com/en/actions

Now, you are good to go but as you notice you have to put your secrets to your github repo so that github actions can take username and password there and use it. So you have to go to the repository page(create one if you haven’t created one before) Settings>Secrets>Actions>New Repository Secret and your secrets from there.

Adding Secrets

Secrets page

Now you are ready to push your test repo and see how your github actions work. After pushing you can go to repository page and navigate the actions tab and see something like that:

Github action page workflow in queue

After it finished:

While it is building, you can click any job to see how it is working.

Now, your test image is ready. it is deployed on your docker hub and now you can fetch anywhere you want. So let’s get back to the frontend project and complete the setup with grid and other required tools.

Docker-Compose for test environment

Now, you need to configure your test environment and run tests. You can use docker-compose to have a test environment and run this anywhere you want. As you remember, I told you that using selenium grid, docker compose setup will also include selenium grid.

In the root folder, create docker-compose.ci.yml file and put the required image in it.

version: '3.9'

services: 
  frontend:
    build:
      context: .
      dockerfile: .docker/frontend/Dockerfile.dev
    ports:
      - '3000:3000'

  selenium-hub:
    image: selenium/hub:4.6.0-20221104
    ports: 
      - '4442:4442'
      - '4443:4443'
      - '4444:4444'

  firefox:
    image: selenium/node-firefox:4.6.0-20221104
    depends_on:
      - selenium-hub
    volumes:
      - /dev/shm:/dev/shm  
    shm_size: 2gb      
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_SESSION_TIMEOUT=25
      - SE_NODE_MAX_INSTANCE=1
      - SE_NODE_MAX_SESSIONS=1

  selenium-host:
    image: selcuktemizsoy/selenium-demo:latest

As you see, it is quite easy. Selenium hub and node images coming from selenium, you don’t need to do anything. Selenium host is your test suite image and it includes your tests. You have already pushed it to docker hub and it is coming from there. The only thing that you need to do is add your app image. But since you have already created this dockerfile, it is enough to pass the dockerfile path to compose and that’s it.

Now you can test your setup with those commands:

docker-compose -f docker-compose.ci.yml build
docker-compose -f docker-compose.ci.yml up -d

after everything ready (app and selenium grid)

docker-compose -f docker-compose.ci.yml exec -T selenium-host sh bin/run.sh

after test finishes, you can stop env:

docker-compose -f docker-compose.ci.yml down

Before moving github action for the frontend app, it would be good to add some script to check if your selenium hub is ready or not. In bin folder, create one wait.sh file and put this command inside of it.

#!bin/bash
until $(curl --output /dev/null --silent --head --fail http://localhost:4444 && curl --output /dev/null --silent --head --fail http://localhost:3000 ); do
echo "waiting for selenium hub and react app being started"
sleep 1
done

it will wait until the app and selenium hub is ready.

Makefile to consolidate commands

One last thing before moving to github actions, it would be good to add a makefile to keep the app, docker and test command.

To achieve this, create a file in the root and name it as Makefile. Inside of it, prepare your commands. I prepared mines as:

# building docker images 
build-ci-env:
 docker-compose -f docker-compose.ci.yml build
# starting docker-compose  
start-ci-env:
 docker-compose -f docker-compose.ci.yml up -d
# wait for app and hub 
wait-ci-env:
 sh bin/wait.sh
# start selenium test inside of the container 
start-ci-test:
 docker-compose -f docker-compose.ci.yml exec -T selenium-host sh bin/run.sh
# stop environment  
stop-ci-env:
 docker-compose -f docker-compose.ci.yml down

Now, you can easily use commands to start, stop, build etc.

Github actions for react app

It is time to create github actions and let’s create .github/workflows folder and create file github-actions.yml

In this time, you need to build your environment in CI and run tests. And also you can add more steps to build your application code and deploy it in any environment. But since it is not our topic today, it will include steps only related to tests.

And here is the general workflow:

name: react app CI
on:
  push:
    branches: [master]

jobs:
  build-and-test-frontend:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout the code
        uses: actions/checkout@v3
      - name: Setup Docker Buildx
        uses: docker/setup-buildx-action@v2
      - name: Setup CI environment
        run: make build-ci-env
      - name: Start CI environment
        run: make start-ci-env
      - name: Wait for selenium hub to be ready
        run: make wait-ci-env
      - name: Start tests
        run: make start-ci-test     
      - name: Deploy
        run: echo "you can add deployment steps from now on. Remaining steps will be running after the previous steps have passed."

As you see, it starts with checking out the code again. Secondly, setting up docker buildx to build docker images. After adding buildx step, actually, it is same with your local computer. First, build-env and then start this environment. Next wait until setup is ready. Finally, you can just run the test. I put an example deploy step here but now it is just empty.

Test the pipeline

Okay, the project setup is finished. Let’s push our app to the github repo to see if everything is working or not.

Test Workflow

And we have all the green light. Congrats 🎉 You are done!.

Now, let’s take closer to test step:

Test step from workflow

Now, you can see your test output like in your machine. So until now everything is green but your aim was to measure application quality and block the development if something is wrong. So let’s evaluate this by changing some source code in the frontend app. Open the src/app.js file and change the text inside of the p tag. I just removed some part of the text and am expecting to fail in the pipeline.

Let’s push it and see

You see, the deployment step didn’t work and your assertion failed as expected. So, all the setups are working perfectly.

Summary

In conclusion, I explained how to prepare a pipeline for your selenium test and block the development if your tests failed. I have shown you how to dockerize your test suite, using docker compose for running the test environment, using selenium grid with docker compose and run your test against your application as part of your pipeline. It is a demo setup and many more things can be added/removed. i.e you can choose having one repo, another automation/test tool, triggering another pipeline from your app pipeline etc. Today I chose this kind of setup and I hope it will give you an idea about having a fully automation flow in your software development life cycle. I hope you like it and stay tuned for upcoming articles.

Projects:

Test suite:

https://github.com/selcuktemizsoy/demo-ui-selenium-cucumber

React app:

https://github.com/selcuktemizsoy/demo-react-app

You can follow me on Linkedin