Build a CICD pipeline using Gitlab, Terraform and Aws.

In this article we will explore on the reusable CICD pipeline for any resoure that can be provisioned on AWS cloud. In this Article we will build the pipeline for aws S3 storage service and enabling and disabling its lifecycle configuration.

Source code repo :

Note :

please change bucket name in variable .the code will work for you.

Jaffarterraform786/Gitlab-CicD-s3 (github.com)

Figure : Gitlab CICD For S3 Resources.

Procedure:

1. Structure of S3 module:

S3 is a global service. A bucket is a region-specific service.

S3 lifecycle use case:

Let's say we have a newspaper website; they are hosting it on S3. when the news is flash news or fresh news, they need to access the data immediately. if news become old, they need to move data from one storage class to other storage class. this use case we will be implementing through S3 lifecycle rules and storage classes with CICD pipeline.

S3 Storage classes:

Figure s3 storage classes.

In our pipeline script when we enable s3 lifecycle rules the data will move from S3 Standard to Glacier .

we will creating the following resources in the pipeline.

1. S3 Bucket.
2. Enable logs
3. Enable server-side Encryption.
4. Enable Lifecycle Rules.
5. Enable ACL
6. Enable Versioning

CICD Configurations:

GitLab offers a location for online code storage and capabilities for issue tracking and CI/CD.

What is CICD :

“A continuous integration and continuous deployment (CI/CD) pipeline is a series of steps that must be performed in order to deliver a new version of software. CI/CD pipelines are a practice focused on improving software delivery throughout the software development life cycle via automation.”

Creating a Fresh project:

Goto project and create a new project

create a new repository and upload the files.

Source code is available in below repository.

Jaffarterraform786/Gitlab-CicD-s3 (github.com)

figure files in Gitlab repo

Gitlab Runner:

1. GitLab Runner is open-source and written in Go.

2. GitLab Runner is an application that works with GitLab CI/CD to run jobs in a pipeline.

3. GitLab Runner can also run inside a Docker container or be deployed into a Kubernetes cluster.

Shared runners:

Shared runners are available to every project in a GitLab instance.

Figure Enabling shared runner.

Specific Runners :

Use specific runners when you want to use runners for specific projects. For example, when you have:

• Jobs with specific requirements, like a deploy job that requires credentials.
• Projects with a lot of CI activity that can benefit from being separate from other runners.

Figure Creating shared runner.

Configuring Environment variables:

Goto Settings ……………..>> Variables …….

Figure Configuring Env variables.

CICD pipeline stages :

1. Checkov :
2. Init
3. validate
4. plan
5. Apply
6. Destroy.

Checkov stage :

Checkov is a static code analysis tool for scanning infrastructure as code (IaC) files for misconfigurations that may lead to security or compliance problems.

Checkov helps to detect violations and naming rules errors for the infrastructure if it is not defined as per the company naming standards .

Figure :checov script

we are allowing failure for this stage so that even the stage fails the next stages will execute.

Init & validate & plan stage :

In init stage it downloads the plugins and in valiadte stage it checks for synatax errors and in plan stage it shoiws the reaources to be provisioned.

we are saving the plan in a file by name planfile.

Figure init & plan & valiadte stage.

In the show stage we will be viewing the resources plan file .and in apply stage we are adding a manual interfer for provisioning the resources.

Figure show & plan & apply stage ..

the final CICD pipeline to be run now..

go to

CICD in deployments …..> pipelines

Figure final pipeline.