Hey everybody today I'll show you a very rudimentary continuous integration and continuous delivery pipeline using Gitlab's CI process. Think of this a boilerplate or launch platform for future coding projects.

You may already know about Gitlab and how it functions as a git repository, however what you may not know is that it can also function as a software integration/deployment automation tool similar to Travis CI, Jenkins, and TeamCity.
Gitlab-CI is the name of automation portion of Gitlab and is freely available to anyone who has an account on Gitlab.


Overview

Diagram of the process

Let me explain the process in the diagram above.

Step 1. User commits changes to the project repo on Gitlab.
Step 2. Gitlab-CI is initated with the presences of the gitlab-ci.yml file.
Step 3. Gitlab-CI deploys user's project on AWS EC2 instance. AWS will use a customized AMI that we will create for the EC2 instance.


Prerequisites

  1. AWS account (I'm using a free tier account).
  2. Gitlab project repo.
  3. Booze for a successful deploy.  :D

Time to get started!

Create and Clone the Repo

First thing I'll do is go to Gitlab and create a repo for my project. To do this click on the green button that says "New project".

In the next page give your project a name. Mine is called aws-deploy.

Now clone the empty repo.

git clone https://gitlab.com/REPO.git

Next create an ssh-key, the private key will need to be baked into the AMI.

ssh-keygen -t rsa -C "youre@email.com" -b 4096

This should have created a private and public (.pub) key.

Hop into Gitlab's settings so that we can give it the public key.

Paste your public key (the file that ends in .pub) in the box and save it.

Cool now we should be able to clone the repo using ssh instead of https.

Before we leave gitlab's configuration page, let's setup the variables for the AWS secret and access keys. Go to your project's repo settings and the click on "CI/CD". Expand the "Variables" section and create an entry for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (they have to be spelled like provided for Gitlab to use them). Then fill out the values that AWS has provided you.


Create a custom AMI

Log into the AWS console and create an EC2 instance. After your instance is up and running, transfer your private key to the instance's /tmp directory with the scp command.

scp -i my-ec2-pkey.pem gitlab-pkey-file ec2-user@123.123.123.123:/tmp

SSH into the new instance and then move the private key into the ec2-user's .ssh directory.

mv /tmp/PRIVATE_KEY ~/.ssh

Go back to the console and right click on the instance and select "Image" > "Create Image".

This should open another window. Give the image a name and description. Then click on "Create Image".

Once your AMI is created copy the AMI ID and save it in a text editor for use later in this guide.


Deployment Files

user_data.txt

Time to create the files in our repo. First I'll create a file called user_data.txt. This will be used by AWS to kick start the EC2 instance during it's provisioning process. This is the same process as specifying the user data settings within the AWS Console for standard EC2 instances.

user_data.txt:

#!/bin/bash
yum update -y && yum upgrade -y
yum install -y vim net-tools curl wget git
eval "$(ssh-agent -s)"
ssh-add /root/.ssh/gitlab1
git clone git@gitlab.com:pafable/aws-deploy.git /appl
date > /appl/creation_date.txt

As you can see above it will update the instance and install some packages most importantly - git. Then it will add the private key for gitlab so when you run git to clone a repo, you will not be prompted. Lastly it will create a file called creation_date.txt and put the date the instance was created in there.

deploy.sh

The next step is to create the deploy.sh script. When this is executed, gitlab will execute the aws command to create an EC2 instance. Make sure to replace --image-id with your AMI ID and --key-name with your ssh key. You can also rename the hostname (HSTNM) to whatever you want.

Obviously because this is only for my testing I'm using a t2.micro instance type. If you're using this for production you will want to scale up.  Perhaps an m5.large for production use cases.

deploy.sh:

#!/bin/bash
HSTNM='GITLAB-CI-TEST-01'
aws ec2 run-instances \
    --image-id YOUR_AMI_ID \
    --count 1 --instance-type t2.micro  \
    --region us-east-1 \
    --security-group-ids 'sg-0a8520b66d05305b4' \
    --key-name 'YOUR_SSH_KEY' \
    --user-data file://user_data.txt \
    --tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=$HSTNM}]"

gitlab-ci.yml

Finally create the gitlab-ci.yml file. Once gitlab sees this in your project any new commits will instantly be ran in the CI/CD pipeline process. When gitlab kicks off the process it will  build, test, and deploy from a container.

gitlab-ci.yml:

image: python:latest

aws_test:
  stage: test
  before_script: 
    - echo "Running tests"
  script:
    - echo "bye"

aws_deploy:
  stage: deploy
  before_script:
    - curl "https://s3.amazonaws.com/aws-cli/awscli-bundle.zip" -o "awscli-bundle.zip"
    - unzip awscli-bundle.zip
    - ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
  script:
    - chmod +x deploy.sh
    - ./deploy.sh
    - echo "Deployment is complete"

For now you don't have to worry about the image that the containers will use. I specified the python image because this will serve as a basis for a future project/blog post using python. You can choose not to specify it for now if you choose not to.

As you can see I'm using two stages test and deploy. The test stage, named aws_test, will test the code and see if it passes, if it passes it will proceed to the deploy stage. The deploy stage which is named aws_deploy will download and install aws cli before executing the deploy.sh script.

Once you have all of the files created, save them all and commit the changes to your repo.

If you jump back to Gitlab you should see the Gitlab-CI job starting up.

Now if you go to your AWS console and look at your available EC2 instance, you should see the EC2 instance that was spun up. Also if you navigate to /appl you will see the creation date text file along with the files from the repo!