I’m working on setting up CI/CD pipelines for a batch project in GitHub using GitHub Actions. The project’s directory structure looks like this:
batch_project/
├── alpha/
│ ├── program_1.txt
│ ├── program_2.txt
│ └── program_3.txt
├── beta/
│ ├── program_1.txt
│ ├── program_2.txt
│ └── program_3.txt
├── gamma/
│ ├── program_1.txt
│ ├── program_2.txt
│ └── program_3.txt
└── .github/
└── workflows/
└── ci-cd.yml
Workflow Objective: Branching strategy: feature/* branches should deploy to the dev environment. release/* branches should deploy to the uat environment. main branch should deploy to prod.
Triggering:
Whenever there’s a code push in any of the programs in the alpha, beta, or gamma modules, the workflow should trigger builds and deployments specific to those changes.
Problem: Everything works well except when I raise a pull request from a feature/* or release/* branch to the main branch. The workflow step meant to list the modified files returns empty, even though I’ve made changes in the pull request.
Here’s the part of my workflow that isn’t working as expected:
MODIFIED_FILES=$(git diff --name-only origin/$BASE_BRANCH..${{ github.sha }})
Expected Behavior:
I expect MODIFIED_FILES to contain the list of modified files in the branch when I raise the pull request (from feature/* or release/* to main). However, it’s returning an empty result.
What I've Tried:
Ensuring the repository is fetched with full history:
git fetch --all
Using both the two-dot .. and three-dot ... syntax with git diff, but neither has worked when raising a pull request.
Question:
How can I modify my GitHub Actions workflow so that the git diff command correctly identifies and lists the modified files in a pull request between branches?
Here is the workflow code
name: mock CI/CD
on:
workflow_dispatch:
push:
branches:
- 'feature/*'
- 'release/*'
- 'main'
paths:
- alpha/**
- beta/**
- gamma/**
pull_request:
branches:
- 'release/*'
- main
jobs:
build:
runs-on: ubuntu-latest
outputs:
alpha_exists: ${{ steps.set_programs.outputs.alpha_exists }}
beta_exists: ${{ steps.set_programs.outputs.beta_exists }}
gamma_exists: ${{ steps.set_programs.outputs.gamma_exists }}
steps:
- name: Checkout code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Set environment based on the branch or PR
id: set_env
run: |
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
BRANCH="${{ github.event.pull_request.base.ref }}"
else
BRANCH="${{ github.ref_name }}"
fi
if [[ "$BRANCH" == feature/* || "$BRANCH" == bugfix/* ]]; then
ENVIRONMENT="dev"
elif [[ "$BRANCH" == release/* ]]; then
ENVIRONMENT="uat"
elif [[ "$BRANCH" == main || "$BRANCH" == hotfix/* ]]; then
ENVIRONMENT="prod"
else
echo "Unknown branch, exiting"
exit 1
fi
echo "environment is set to $ENVIRONMENT"
echo "ENVIRONMENT=$ENVIRONMENT" >> $GITHUB_ENV
echo "BRANCH=$BRANCH" >> $GITHUB_ENV
# GitHub summary for this step
echo "### Environment and Branch Info" >> $GITHUB_STEP_SUMMARY
echo "- **Branch**: $BRANCH" >> $GITHUB_STEP_SUMMARY
echo "- **Environment**: $ENVIRONMENT" >> $GITHUB_STEP_SUMMARY
- name: Extract branch name
shell: bash
run: echo "branch=${GITHUB_HEAD_REF:-${GITHUB_REF#refs/heads/}}" >> $GITHUB_OUTPUT
id: extract_branch
- name: List all modified files in the branch
id: changed_files
run: |
git fetch --all
# Determine base branch based on the event type
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
BASE_BRANCH="${{ github.event.pull_request.base.ref }}"
else
BASE_BRANCH="main"
fi
# Verify the BASE_BRANCH exists after fetching
if git show-ref --verify --quiet refs/remotes/origin/$BASE_BRANCH; then
echo "Base branch exists remotely."
else
echo "Base branch does not exist remotely. Exiting."
exit 1
fi
# Get the current branch dynamically based on the event
if [[ "${{ github.event_name }}" == "pull_request" ]]; then
CURRENT_BRANCH="${{ github.event.pull_request.head.ref }}"
else
CURRENT_BRANCH="${{ github.ref_name }}"
fi
# Fetch the current branch if not already available
git fetch origin $CURRENT_BRANCH
# Display git rev-parse for base branch and current commit
BASE_BRANCH_HASH=$(git rev-parse origin/$BASE_BRANCH)
CURRENT_SHA="${{ github.sha }}"
echo "Base branch ($BASE_BRANCH) commit hash: $BASE_BRANCH_HASH"
echo "Current commit hash (github.sha): $CURRENT_SHA"
# Check branch and commits step summary
echo "### Branch & Commit Hashes" >> $GITHUB_STEP_SUMMARY
echo "- **Base branch**: $BASE_BRANCH" >> $GITHUB_STEP_SUMMARY
echo "- **Current branch**: ${{ steps.extract_branch.outputs.branch }}" >> $GITHUB_STEP_SUMMARY
echo "- **Base branch commit ($BASE_BRANCH)**: $BASE_BRANCH_HASH" >> $GITHUB_STEP_SUMMARY
echo "- **Current branch commit (github.sha)**: $CURRENT_SHA" >> $GITHUB_STEP_SUMMARY
# Use two-dot (..) to compare the commits directly between the base branch and the current commit
MODIFIED_FILES=$(git diff --name-only origin/$BASE_BRANCH..$CURRENT_SHA)
# MODIFIED_FILES=$(git diff --name-only origin/${{ steps.extract_branch.outputs.branch }}..origin/$BASE_BRANCH)
echo "Modified files: $MODIFIED_FILES"
echo "$MODIFIED_FILES" | tr '\n' ',' > files_list.txt
echo "MODIFIED_FILES=$(cat files_list.txt)" >> $GITHUB_ENV
# Add modified files to the step summary
echo "### Modified Files" >> $GITHUB_STEP_SUMMARY
if [ -z "$MODIFIED_FILES" ]; then
echo "- No modified files" >> $GITHUB_STEP_SUMMARY
else
echo "- $MODIFIED_FILES" >> $GITHUB_STEP_SUMMARY
fi
- name: Set program files and zip them
id: set_programs
run: |
MODIFIED_FILES="${{ env.MODIFIED_FILES }}"
ALPHA_FILES=()
BETA_FILES=()
GAMMA_FILES=()
IFS=',' read -ra FILE_ARRAY <<< "$MODIFIED_FILES"
for file in "${FILE_ARRAY[@]}"; do
if [[ "$file" == alpha/* ]]; then
ALPHA_FILES+=("$file")
elif [[ "$file" == beta/* ]]; then
BETA_FILES+=("$file")
elif [[ "$file" == gamma/* ]]; then
GAMMA_FILES+=("$file")
fi
done
function zip_and_upload {
PROGRAM_NAME=$1
FILES=("${!2}")
ZIP_NAME="${PROGRAM_NAME}_modified_files.zip"
if [ ${#FILES[@]} -gt 0 ]; then
zip -r "$ZIP_NAME" "${FILES[@]}"
echo "Zipped $PROGRAM_NAME files into $ZIP_NAME"
mv "$ZIP_NAME" "$PROGRAM_NAME.zip"
echo "::set-output name=${PROGRAM_NAME}_exists::true"
else
echo "No modified files for $PROGRAM_NAME."
echo "::set-output name=${PROGRAM_NAME}_exists::false"
fi
}
zip_and_upload "alpha" ALPHA_FILES[@]
zip_and_upload "beta" BETA_FILES[@]
zip_and_upload "gamma" GAMMA_FILES[@]
- name: Upload Alpha files
if: steps.set_programs.outputs.alpha_exists == 'true'
uses: actions/upload-artifact@v4
with:
name: alpha_modified_files
path: alpha.zip
- name: Upload Beta files
if: steps.set_programs.outputs.beta_exists == 'true'
uses: actions/upload-artifact@v4
with:
name: beta_modified_files
path: beta.zip
- name: Upload Gamma files
if: steps.set_programs.outputs.gamma_exists == 'true'
uses: actions/upload-artifact@v4
with:
name: gamma_modified_files
path: gamma.zip
deploy:
needs: build
runs-on: ubuntu-latest
if: github.event_name == 'push' && ( startsWith(github.ref, 'refs/heads/release/') || startsWith(github.ref, 'refs/heads/feature/') || startsWith(github.ref, 'refs/heads/main/') )
steps:
- name: Download Alpha files
if: needs.build.outputs.alpha_exists == 'true'
uses: actions/download-artifact@v4
with:
name: alpha_modified_files
path: downloaded_files/alpha
- name: Download Beta files
if: needs.build.outputs.beta_exists == 'true'
uses: actions/download-artifact@v4
with:
name: beta_modified_files
path: downloaded_files/beta
- name: Download Gamma files
if: needs.build.outputs.gamma_exists == 'true'
uses: actions/download-artifact@v4
with:
name: gamma_modified_files
path: downloaded_files/gamma
- name: Extract and list downloaded files
run: |
echo "Listing downloaded files..."
for program in alpha beta gamma; do
if [ -d "downloaded_files/$program" ]; then
echo "Contents of $program artifact:"
ls "downloaded_files/$program"
else
echo "Artifact for $program does not exist."
fi
done
- name: Deployment into ${{ env.ENVIRONMENT }} environment
run: |
echo "Deploying to the ${{ env.ENVIRONMENT }} environment..."
Based on the suggestions I have displayed the commit SHA , Base branch and current branch details in step summary to investigate the current branch vs base branch difference
feature branch workflow github step summary
Environment and Branch Info
Branch: feature/feat-1
Environment: dev
Branch & Commit Hashes
Base branch: main
Current branch: feature/feat-1
Base branch commit (main): 20d76b92a509ca96db91b82b3e9c2c4029ff7374
Current branch commit (github.sha): 75f0ae5c36546b5d4d59920ff821cfa7ef59289a
Modified Files
alpha/program_1.txt
PR from feature branch to main branch workflow github step summary
Environment and Branch Info
Branch: main
Environment: prod
Branch & Commit Hashes
Base branch: main
Current branch: feature/feat-1
Base branch commit (main): 20d76b92a509ca96db91b82b3e9c2c4029ff7374
Current branch commit (github.sha): fa3e7838d7c818cf418f47a20d5f8636b5ee8ecb
Modified Files
alpha/program_1.txt
PR merge feature branch to main branch workflow github step summary
Environment and Branch Info
Branch: main
Environment: prod
Branch & Commit Hashes
Base branch: main
Current branch: main
Base branch commit (main): bcf15d0f7bce60e77d71f501b9c105ea48c20ff3
Current branch commit (github.sha): bcf15d0f7bce60e77d71f501b9c105ea48c20ff3
Modified Files
No modified files
Problem is sha is getting same, expectation is get the PR commit sha vs main branch sha.
Problem using git diff when I have multiple features development and merge back to develop branch.
Base on docs:
The
pull_requestwebhook event payload is empty for merged pull requests and pull requests that come from forked repositories.
The value of
GITHUB_REFvaries for a closed pull request depending on whether the pull request has been merged or not.
If a pull request was closed but not merged, it will berefs/pull/PULL_REQUEST_NUMBER/merge.
If a pull request was closed as a result of being merged, it will be the fully qualified ref of the branch it was merged into, for example/refs/heads/main.
Meaning, when the PR is in merge state, it's all main, and you can see that also on your output:
Base branch: main
Current branch: main
Base branch commit (main): bcf15d0f7bce60e77d71f501b9c105ea48c20ff3
Current branch commit (github.sha): bcf15d0f7bce60e77d71f501b9c105ea48c20ff3
There's no base branch info anymore, and sha you'll get is of main itself.
What you can do, and I can't test, but I believe will work, is:
git diff --name-only c05f017^ c05f017
For the second issue, if I understand correctly, the merges are happening very close to each other, and one take the changes of the other as well as its own.
Probably very delicate timing.
The solution I see is rebasing and pushing the branch before build-release, something like (Remember that I can't test it, and it's verbose for explanation purposes, it can be shrunk to one step):
jobs:
rebase:
runs-on: ubuntu-latest
steps:
- name: Checkout the target branch
uses: actions/checkout@v4 # Change to 4 in general
with:
ref: ${{ github.event.pull_request.base.ref }}
fetch-depth: 0
clean: true
- name: Checkout the current branch
run: |
git fetch origin ${{ github.event.pull_request.head.ref }}
git checkout ${{ github.event.pull_request.head.ref }}
- name: Rebase the current branch onto the target branch
run: |
git rebase ${{ github.event.pull_request.base.ref }}
- name: Push the rebased branch
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
git push origin ${{ github.event.pull_request.head.ref }} --force
build:
needs: rebase
...
# your workflow