Feature branch CI/CD strategy

[flochaz]

Context

Current examples only show how to develop, integrate and deploy following a trunk base strategy with immutable infrastructure (main branch built once and promoted automatically to each stages).

Goal

Offer an alternative approach based on gitflow / feature branching strategy.

[flochaz]

Option to look into : https://github.com/MHacker9404/cdk-pipeline-multi-branch

[jpaas]

Would also like this feature. In our current home-built CICD, we can deploy or undeploy any branch into staging.

Bonus points (we do this): All our staging deployments share the same infrastructure (NATs, databases, ALB, etc.) i.e. anything that is a significant cost.

[npvisual]

@flochaz : not sure if this would be helpful, so please disregard if not.

We've been using the CDK and self-mutating pipelines with a dev / qa / prod branch scheme for the infrastructure and feature / dev / qa / prod for the artifacts being deployed.

• repo A --> CodeBuild --> ECR image (backend)
• repo B --> CodeBuild --> S3 bucket (frontend)
• repo C --> self-mutating pipelines --> infrastructure

The feature branch for the infrastructure piece would be very interesting for developers to test changes before pushing to dev. For us, because the development / staging / prod environments were similar in nature (by design), having a feature branch wasn't too much of a deal breaker for the infrastructure piece since we could test in the development environments ; but it would have been nice for sure before rolling out large infrastructure changes to avoid disruptions to all development environments.

It was however very important for the artifacts. So developer environments, for example, can be pointed to any artifact (ECR and S3 bucket) using a "descriptor" file, but they mostly were automatically updated every time a new ECR image was pushed (latest) because the dev branch was updated.

The downside to this is that we have to maintain several descriptor files in the infrastructure git repo (repo C) -- different for the dev / qa and prod branches. And each AWS account had to be bootstrapped with the proper branch (once and done).

Until now we had separated dev, staging and prod environments in 3 different AWS accounts and each tenant was isolated in their own slice of a VPC.

Now, we're trying to migrate to one AWS account per tenant per environment (why we're looking into the Bootstrap Kit) ; so we're looking at things a little differently. It would be ideal if the self-mutating pipeline that dictates the infrastructure for each account could be descriptor-less.

Coming back to the feature branch strategy, I think it would be interesting to have an organizational unit structured like this :

• org
  • tenants
    • tenant-name
      • dev --> any feature branch (but dev branch by default and automatically updated using latest)
      • staging --> qa branch (automatically updated using latest)
      • prod --> main branch (using specific artifact build)
  • cicd
    • common ECR repo (feature / dev / staging / prod)
    • S3 artifacts (feature / dev / staging / prod)

You can extend or "shrink" this to match your SDLC processes. But having a development environment that can :

• match a feature or dev branch for the infrastructure constructs
• match any branch for the artifacts
  would probably be the most flexible solution so developers can test any artifact. Keeping the artifact identifiers outside of the code repo (via SSM ?) would ease our process.

Note : the structure depicted above didn't address "developer" environments, which could be included in an SDLC OU, as shown in the AWS Bootstrap Kit example, with 1 AWS account per developer

• org
  • tenants
    • ...
  • sdlc
    • lucy
      • feature1234 --> feature branch (automatically updated using latest of feature build, for either ECR or S3, or both)
      • dev --> dev branch (automatically updated using latest)
      • staging --> qa branch (automatically updated using latest)
    • joe
      • dev --> dev branch (automatically updated using latest)
      • ...

This also makes cost attribution a lot easier.