How to Build an AWS CI/CD Pipeline with CodePipeline and CodeBuild

A deployment process that relies on someone running commands manually is not a deployment process. It is a ritual with undocumented dependencies, inconsistent outcomes, and a single point of failure in the form of the person who knows the steps.

AWS CodePipeline and CodeBuild give you the building blocks for a proper automated pipeline. This guide covers how to assemble them into something production-ready.

What Each Service Does

AWS CodePipeline is the orchestration layer. It defines the stages of your pipeline (source, build, test, deploy), connects them together, and manages the flow of artefacts between them. It is event-driven: a change in your source repository triggers the pipeline automatically.

AWS CodeBuild is the build and test execution environment. It spins up a managed container, runs your build commands as defined in a buildspec.yml file, and produces output artefacts. You pay per build minute, with no idle infrastructure.

AWS CodeDeploy handles the deployment stage: pushing built artefacts to EC2, ECS, Lambda, or on-premise targets. For Kubernetes deployments, CodePipeline typically integrates with kubectl commands run via CodeBuild or with Amazon EKS native tooling.

Together they cover the full CI/CD loop without requiring you to manage Jenkins infrastructure or operate a third-party CI platform.

Prerequisites

Before you create the pipeline, you need:

  • A source repository: CodeCommit, GitHub, GitHub Enterprise, Bitbucket, or GitLab (via CodeConnections)
  • An S3 bucket for pipeline artefacts (CodePipeline creates this automatically if you use the console wizard, but creating it explicitly gives you more control)
  • An IAM role for CodePipeline with permissions to access your source, trigger CodeBuild, and deploy to your target
  • An IAM role for CodeBuild with permissions to pull from ECR, write to S3, and access any other resources it needs during build

Keep IAM roles tightly scoped. CodeBuild environments run your build commands with the permissions of their IAM role. An over-permissioned CodeBuild role is a significant blast radius if someone commits a malicious dependency.

Step 1: Create the buildspec.yml

The buildspec.yml file lives in your repository root and tells CodeBuild what to do. A minimal example for a Node.js application:

```yaml

version: 0.2

phases:

install:

runtime-versions:

nodejs: 20

commands:

- npm ci

pre_build:

commands:

- echo Running tests

- npm test

build:

commands:

- echo Building application

- npm run build

post_build:

commands:

- echo Build complete

artifacts:

files:

- '*/'

base-directory: dist

cache:

paths:

- node_modules/*/

```

Key points:

  • Use npm ci rather than npm install in CI environments. It installs exactly what is in package-lock.json rather than resolving afresh.
  • Run tests in pre_build so a test failure stops the pipeline before the build stage runs.
  • Use the cache block to cache node_modules between builds. This reduces build times substantially for large dependency trees.
  • The artifacts block defines what gets passed to the next pipeline stage. Only include what the deployment stage needs.

For Docker builds:

```yaml

version: 0.2

phases:

pre_build:

commands:

- aws ecr get-login-password --region $AWSDEFAULTREGION | docker login --username AWS --password-stdin $ECR_REGISTRY

- IMAGETAG=$(echo $CODEBUILDRESOLVEDSOURCEVERSION | cut -c 1-7)

build:

commands:

- docker build -t $ECRREGISTRY/$IMAGEREPONAME:$IMAGETAG .

- docker tag $ECRREGISTRY/$IMAGEREPONAME:$IMAGETAG $ECRREGISTRY/$IMAGEREPO_NAME:latest

post_build:

commands:

- docker push $ECRREGISTRY/$IMAGEREPONAME:$IMAGETAG

- docker push $ECRREGISTRY/$IMAGEREPO_NAME:latest

- printf '[{"name":"%s","imageUri":"%s"}]' $CONTAINERNAME $ECRREGISTRY/$IMAGEREPONAME:$IMAGE_TAG > imagedefinitions.json

artifacts:

files: imagedefinitions.json

```

The imagedefinitions.json artefact is what CodeDeploy uses to know which container image to deploy to ECS.

Step 2: Create the CodeBuild Project

In the AWS Console under CodeBuild:

  1. Create a new build project
  2. Source: leave empty (CodePipeline passes source automatically)
  3. Environment: Managed image, Amazon Linux, Standard runtime, the latest image version. Enable privileged mode if you are building Docker images.
  4. Service role: create a new role or attach an existing one. Ensure it has ecr:GetAuthorizationToken, ecr:BatchCheckLayerAvailability, ecr:PutImage, and write access to your artefact S3 bucket.
  5. Buildspec: use the buildspec.yml file in the repository
  6. Artefacts: CodePipeline manages this, so set to no artefacts at the project level

Enable CloudWatch Logs for the build project. Build logs in CloudWatch are how you diagnose failures. Without them you are reading truncated console output.

Step 3: Create the CodePipeline

In the AWS Console under CodePipeline:

Stage 1: Source

Select your source provider and repository. For GitHub, you will need to create a CodeConnections connection (previously called CodeStar Connections) to authorise AWS access to your GitHub organisation.

Set the branch to trigger from (typically main for production pipelines). Enable "Start the pipeline on source code change" to trigger automatically on push.

Stage 2: Build

Select AWS CodeBuild and choose the project you created. Input artefacts: the source output from Stage 1. Output artefacts: name this something like BuildOutput.

Stage 3: Deploy

The deployment stage depends on your target:

For ECS (using rolling deployment):

  • Provider: Amazon ECS
  • Cluster and service name
  • Image definitions file: imagedefinitions.json (from your build artefacts)

For ECS (using blue/green via CodeDeploy):

  • Provider: AWS CodeDeploy
  • Requires an AppSpec file and CodeDeploy deployment group configured for ECS

For Lambda:

  • Provider: AWS CloudFormation (deploying a SAM or CDK template) or AWS Lambda (for direct function updates)

For EC2:

  • Provider: AWS CodeDeploy with an EC2 deployment group
  • Requires the CodeDeploy agent running on target instances

Step 4: Add Manual Approval for Production

A pipeline that deploys straight to production on every commit is usually too aggressive. Add a manual approval stage between staging and production:

  1. Add a new stage after your staging deploy stage
  2. Action provider: Manual approval
  3. Configure SNS notifications to alert your team when approval is pending
  4. Optionally add a review URL pointing to your staging environment

Approvals create an explicit checkpoint. They also produce an audit trail in CodePipeline showing who approved which deployment and when.

Step 5: Environment-Specific Configuration

Applications need different configuration in different environments. Do not bake environment-specific values into your artefacts.

Patterns for environment configuration in AWS pipelines:

AWS Systems Manager Parameter Store: Store environment-specific config as SSM parameters. Reference them in your buildspec or application startup using the SSM SDK or aws ssm get-parameter CLI calls. Encrypted parameters use KMS.

AWS Secrets Manager: For secrets (database passwords, API keys, certificates). Rotate automatically. Reference via SDK or environment variable injection.

CodeBuild environment variables: Suitable for non-sensitive pipeline configuration. Set in the CodeBuild project or override per pipeline stage. Not appropriate for secrets.

Do not pass secrets as plaintext environment variables. They appear in build logs.

Monitoring the Pipeline

A pipeline that nobody watches is not CI/CD. It is wishful thinking.

Set up CloudWatch Events rules to notify your team on pipeline failures. A simple setup: CloudWatch Event rule targeting an SNS topic, SNS topic with email or Slack (via Lambda) subscription, triggered on codepipeline-pipeline-stage-execution-state-change where state is FAILED.

If you use Datadog, the AWS CodePipeline integration surfaces pipeline state, stage durations, and failure events in your Datadog dashboards alongside the deployment's downstream effects on application performance. This gives you a causal view: deployment at 14:32 → error rate increase at 14:35 → rollback at 14:40.

Common Mistakes

Not caching dependencies. A 5-minute build that could be 90 seconds wastes compute costs and slows feedback loops.

Over-broad IAM on the CodeBuild role. The build environment has network access and AWS credentials. Scope permissions to exactly what the build needs.

No artefact versioning strategy. If every build overwrites the same S3 key or ECR latest tag, you lose the ability to roll back to a specific previous build. Use commit SHA or build number in artefact names and image tags.

Deploying to production from a branch other than main. Source stage branch configuration gets overlooked. Verify which branch triggers which pipeline.

No rollback plan. CodeDeploy supports automatic rollback on deployment failure. Configure it. A deployment that rolls back automatically on high error rates is far less damaging than one that sits broken until someone notices.

Where Critical Cloud Comes In

Building the pipeline is one part of the problem. Ensuring it deploys reliably, that failures are caught before they reach production, and that post-deployment observability closes the feedback loop is the other part. Critical Cloud manages AWS environments for tech-led businesses with Datadog observability integrated across the deployment pipeline and the application layer. See how Critical Support works.