Introduction
CI/CD (Continuous Integration and Continuous Deployments) have become a requirement towards seamless deployments and automation of software releases. Currently, every single app whether small developer applications, going up till big organization deployment releases have adopted CICD and automating deployment workflows and releases.
One recent game changer in automating deployment world is the emerging of GitHub actions, a CICD tool from GitHub that aims to simplify and manage both small and large deployment workflows and CICD processes easily and seamlessly. We know Jenkins is king in CICD, but when we come to GitHub Actions, should we say simplified Jenkins tied to your GitHub repository?
If you've used GitHub actions before, you'll notice that the compute engine that is provided to you by GitHub is not managed by you. It's compute given to you by GitHub actions, maybe a container dedicated to your build and once completed the container is wiped 🤷♂️. But Imagine you were running a build process for an app that has tight security and just can't be cloned on just any container or compute that is publicly accessible.
In this article, we will create a self hosted runner for our GitHub actions workflow runs and the runner will be based on the Amazon Linux 2023 Operating System. In later sections of the article, we will try out a simple build process using the runner.
Create the EC2 Instance (CLI)
I'm a big fan of the terminal and code, in this section we will create an EC2 instance using the CLI. Feel free to use the AWS management console to create the instance. But if you are following along, below is the command you can use to create the EC2 Instance on AWS using the CLI.
aws ec2 run-instances \
--image-id \
resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 \
--instance-type t2.micro \
--region us-west-2 \
--key-name aws-key-us-east-1 \
--security-group-ids sg-004a7650
Ensure you edit the command by replacing t2.micro with your instance type of choice and us-west-2 with the region you're familiar with deployments, aws-key-us-east-1 with the name of your existing key pair on AWS to SSH to the instance. You can leave that out if you will connect to the machine using the management console. Lastly, sg-004a7650 with a security group ID existing on AWS.
The if the command is successful, you should have a JSON response such as this in your terminal/command prompt. Note take note of the architecture type x86_64 it will be useful to us later in the article.
Head over to the console and grab the public IP of the instance and use it to SSH or connect to the instance using you preferred choice, because we will be executing some commands in the machine. In this article, the choice of connecting to the instance is using SSH.
Download Self Hosted Runner
In this article, the runner that will be configured will be dedicated to a specific repository. So whenever there is a change in out git SCM the build will be triggered or can be manually triggered and the build process should be executed on the Amazon Linux 2023 instance.
Head over to your git repository, click on the actions tab select runners. If you don't have a repo to use, you can fork mine and follow along.
https://github.com/realexcel2021/automation-task
Click on New Runner, and click on New Self Hosted Runners. For the OS, select Linux and the architecture should be x64. If you recall, x86_64 is our architecture which is valid.
Next up, we start the installation process. Create a folder in the Linux machine where we can put all the files we will download to set up the actions runner and switch to the folder. The command below creates the folder named actions-runner .
mkdir actions-runner && cd actions-runner
Download the latest version of the actions runner using the command below
curl -o actions-runner-linux-x64-2.313.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.313.0/actions-runner-linux-x64-2.313.0.tar.gz
To validate if the hash to ensure actions runner will be installed correctly on the instance, first install shasum a package that computes SHA messages using the command below.
sudo yum install perl-Digest-SHA -y
Use the next command to validate the actions-runner check sum
echo "56910d6628b41f99d9a1c5fe9df54981ad5d8c9e42fc14899dcc177e222e71c4 actions-runner-linux-x64-2.313.0.tar.gz" | shasum -a 256 -c
You should get an OK response
Extract the installer using the command below. The extract should provide some bash scripts and other files.
tar xzf ./actions-runner-linux-x64-2.313.0.tar.gz
Configure Self Hosted Runner
In this step, we will configure our Amazon Linux 2023 to be the runner for our GitHub actions runs on the selected repository.
Before you configure the runner, using the ./config.sh command, we will need to install some packages first because by default, some packages needed for GitHub actions runner app are not installed on Amazon Linux 2023. So to avoid jumping into errors, let's install the needed packages.
First, install lld one of Dotnet core dependencies. First update update the yum database using the command below
sudo dnf makecache --refresh
The output should look something like this
Next, install lld using the command below
sudo dnf -y install lld
Next, you have to install libicu using the command below
sudo yum install libicu -y
Head Back to the Add self hosted runner page on your GitHub repo and copy the config command specific to your repository in the configure section and paste in the linux machine.
./config.sh --url https://github.com/<your username>/<repo name> --token AVIFMFHEM
Note: If the config file returns a 404 error, please refresh your GitHub page to get a new token
You will be presented with a runner registration on the command line to register the runner and give it a name. In this example, I will name the runner docker-runner. For the runner group, select default. I also configured the runner to have label of docker-runner and the folder for all the builds and works will be in the _work folder which will be automatically created.
Now if I head back to the GitHub repository in the Actions tab > Runners > Self Hosted runners, you should find the docker-runner.
Lastly, execute the runner
./run.sh
If the runner successfully executed, you should get a response saying connected to GitHub and waiting to pick up job.
Please Note that you do not need to open any inbound rules to allow connectivity to GitHub actions.
Note that the ./run.sh executes a bash script that waits on your terminal or run on the background based on sessions for the job executions to pick up jobs. But you can run the GitHub actions job as a service on the Linux machine, so whenever the machine is rebooted or started, it automatically starts the GitHub actions app.
To run the GitHub actions app as a service and configure it to run when the machine starts up, execute the commands below:
Note: Run the following commands In the folder where you downloaded GitHub actions and configured it, in our case theactions-runnerfolder.
./svc.sh install $USER
You can start the service, stop the service and also see the service's status.
sudo ./svc.sh start # to start the service
To check the service status
sudo ./svc.sh status
You should see an output similar to this:
To stop the service (don't do this if you're following the demo)
sudo ./svc.sh stop
Execute a Job in the Self Hosted Runner
To execute a GitHub actions job in the self hosted runner, it's pretty easy. Just simply refer to the name or the tag of the runner you configured in the configuration section inside your actions yml file.
For example, In this demo, we configured the actions runner name to be docker-runner so in the runs-on section of any job I wish to execute on the self hosted runner, I'll replace with docker-runner instead of the usual ubuntu latest.
You can replace with whatever name you gave your runner. If you're following along, and you forked this project and you want to try out the pipeline on the Amazon Linux machine, you will need to replace some things and also have docker installed on the Linux machine. Use the commands below to install docker:
sudo dnf install docker -y
Add the user to the docker group to run docker commands without sudo.
sudo usermod -aG docker $USER
Note: You might need to stop and start the GitHub actions app again so GitHub actions can execute docker commands without sudo
In .github/workflows/build-image.yml file, replace the environment variable DOCKER_IMAGE_NAME with a valid image name you have in your docker hub.
Next, head to your GitHub page, in the repository and set some secrets. DOCKER_USERNAME --> Your docker hub username DOCKER_PASSWORD --> Your docker hub password or temporary token.
Save and push. Head back to your actions tab and manually trigger the job because the job has a manual workflow_dispatch configured to it. It should prompt you for a run id, enter a unique id , just anything then click Run workflow.
It should trigger the workflow on the Amazon Linux 2023 machine.
In my Amazon Linux machine, I have the build image with tags.
Conclusion
Congratulations on configuring a GitHub actions self hosted runner with Amazon Linux 2023. So whenever you need to have full control of your CICD pipeline, or you need more memory for your CICD process, feel free to always use self hosted runners. But note that you manage everything.
If you had any issues configuring this, or you have feedback of any kind please do reach out on Twitter (X) @sheriffexcel