I'm seeking wisdom on CI/CD pipeline design.

I primarily work in infrastructure and am currently tasked with designing and building a CI/CD pipeline for the first time. While I've used Git personally, I have limited professional experience with it.(T_T)

I'm building the pipeline on AWS using CodePipeline with the following basic structure:

GitLab Private (branch merge) → CodeBuild → Approval → CodeDeploy

My specific question:

Should I create separate pipelines for test and production environments, or is a single pipeline more appropriate? I'm considering these two patterns:

Pattern 1: Separate pipelines for test and production

GitLab Private (merge to develop branch) → CodeBuild → Approval → CodeDeploy (test)
GitLab Private (merge to main branch) → CodeBuild → Approval → CodeDeploy (production)

Pattern 2: Single pipeline for both environments

GitLab Private (merge to develop branch) → CodeBuild → Approval → CodeDeploy (test) → Approval → CodeDeploy (production)

I've temporarily implemented Pattern 1 using CloudFormation, so making changes would be straightforward.

Additional requirements from the application team:

1. They want to maintain separate configuration files for test and production environments (separate from the build artifacts)
2. They'd prefer to use the artifacts built for the test environment in production if possible

My current thoughts:

• For configuration files, I believe we can store them in separate paths and control them by environment using variables like ${DEPLOYMENT_GROUP_NAME}
• With Pattern 1, the build artifacts should be identical since they use the same source code, but I can't guarantee 100% identical behavior in the CodeDeploy container

I think both patterns could work, but I'm wondering what's considered best practice or more common in the industry.

---

Additional context:

• We're using GitLab Flow for branch design but expecting minimal usage (1-2 people, with SI work only a few times per year), so we're keeping it simple with just feature, develop, and main branches
• This is somewhat of a pilot implementation as most team members (including myself) have limited professional Git experience

Regarding different pipelines for environments, there is unfortunately no right or wrong answer, only trade offs based on requirements. Instead, I want to bring your question back to the fundamentals…

DevOps is a collaborative model about Flow, Feedback and Experimentation. Flow is about optimizing the path from ideation to production. Feedback is about sensing capabilities to detect failures at all stages of the product. Experimentation is about continuous learning and improvement. Your question is primarily about Flow, but the other aspects play into this as well.

Regarding GitLab Flow, which is similar to GitHub Flow, ReleaseFlow, or other “trunk-based” branching strategies, there is no develop branch. The goal is to have small feature branches that merge into main, and your workflow can automatically start to move those changes from CI, to dev, test, and prod.

In a “trunk-based” model, mature teams use Feature Flags in the code to control the release of functionality to end users. Often, Feature Flags are stored in an external system and can be toggled on a schedule, programmatically, or manually. Some Feature Flag tools support rolling out features to different user groups or other characteristics. In such a model, the window from commit to production is extremely short, and the rollout and rollback of functionality is controlled by flipping a bit. Teams that adopt this model can deploy to production hundreds of times per week. You might not be there, but that’s the goal.

Every build is considered a “release candidate”, but not all “release candidates” are deployed to production. This is where Feedback is critical. A successful workflow builds automated gates that perform tests and checks to certify the candidate and halt the workflow if those gates fail. Presently, you’re using Approvals as a gate likely with manual verification. As your test automation improves those gates might change. Figure out what other controls and gates are part of your Flow and automate them.

Experimentation has many shapes. Mature teams use blue/green or canary deployments into production where features are slowly rolled out or tested in production before going live. Other forms of Experimentation are concepts like Chaos testing. At the heart of Experimentation is data - metrics collected as part of Feedback. Teams pivot quickly based on experiments. Experimentation drives continuous learning and improvement. Nothing is one and done.

Which leads me back to your original question. Start with something that works to empower Flow. In my view, DevOps is a collaborative mindset that empowers all members of the team. For this reason, I like a single pipeline that is straight-forward to use and can be run by any member of the team. When you start to divide the pipeline into many parts, you introduce division in the team with things like separation of duties, specialized knowledge, required permissions, etc. These are things that kill Flow.

Do what makes sense to start to deliver value quickly. Measure and improve. Involve the entire team to empower it.