October 6, 2019

AWS CI Build Pipeline

Simple Pipeline with Deployment from S3 Bucket artefact

Entrypoint is the aws tutorials page https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials.html

I start with the 2 stage ci pipeline https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-simple-s3.html This includes the following steps:

  • Create a S3 bucket for the pipeline artefacts
  • Create an EC2 linux instance and install CodeDeploy agent
    • create instance rule (S3 list read policy)
    • launch instance (AMI)
    • connect to instance with ssh ssh -i ~/.ssh/aws.pem ec2-user@ec2-52-29-201-4.eu-central-1.compute.amazonaws.com
  • Create an application in CodeDeploy
    • Create an application
    • Create an deployment group
      • Create a service role (CodeDeployServiceRole)
  • Create pipeline in CodePipeline

Deployment from Github repo

  • Create a github repo with with demo project from aws tutorial
    • Add a buildspec.yml file to define the build process for CodeBuild build stage:
version: 0.2

phases:
  install:
    runtime-versions:
      docker: 18

artifacts:
  files:
    - 'scripts/*'
    - 'appspec.yml'
    - 'index.html'
    - 'LICENSE'
    - 'README.md'
  • Create a CodeBuild project
  • Change the CodePipeline to a 3 stage pipeline
    • Source -- Source provider is GitHub
    • Build -- Packaging is done with the CodeBuilder
    • Deploy -- Uses the input artefact from CodeBuilder output

Build and Test from Bitbucket Repo

CodePipeline does not support Bitbucket as a source stage. On the other hand, CodeBuild supports Bitbucket and can even react on events like pull-requests or pushes. This source uses a workaround to make CodePipeline work with Bitbucket repos: https://lgallardo.com/2018/09/07/codepipeline-bitbucket

I follow the same approach to implement a ci pipeline from a Bitbucket repo:

  1. Have a CodeBuild project that acts on a push in the repo and just copies over the master branch into a versioned zip file in S3. The configuration of this packaging of the master branch into the zip file can be configured with a buildspec.yml file, I called pre-buildspec.yml:
version: 0.2

phases:
  install:
    runtime-versions:
      docker: 18

artifacts:
  files:
    - '**/*'
  name: bitbucket-master-branch.zip
  1. Within CodePipeline I defined the zip file in S3 as the source for the pipeline. The change detection is handled by CloudWatch. There was no further configuration necessary. With every push to the repo, the pre-build and zip generation is triggered, which then triggers the start of the ci pipeline.
  2. In the build stage the CodeBuild project uses the 'aws/codebuild/standard:2.0' managed image with 'PrivilegedMode: true' and builds the project using the buildspec.yml file:
version: 0.2

phases:
  install:
    runtime-versions:
      docker: 18

  build:
    commands:
      - docker-compose -f ci.yml run django pytest