I would like to allow two jobs to run in parallel and both use the same environment. I have attached a sample github actions .yml file with comments to better explain what I need. I am new to the community so thank you for the help in advance!
I made a simplified code with the areas I need help with below.
name: Need two jobs to run in parallel and use the same `venv` from the first job
on: push
jobs:
install-dependency:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11.9'
- name: Install Dependencies
run: |
python -m venv venv
# Save the VENV somehow!
source venv/bin/activate
pip install -r requirements.txt
run-pytest:
runs-on: ubuntu-latest
needs: [install-dependency]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
- name: Run Pytest
run: |
source ${{ insert-venv-path }}/bin/activate
pip install pytest
pytest --maxfail=1 --disable-warnings -v
run-other-pytest:
runs-on: ubuntu-latest
needs: [install-dependency]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
- name: Run Pytest
run: |
source ${{ insert-venv-path }}/bin/activate
pip install pytest
pytest --maxfail=1 --disable-warnings -v
I know github makes it so that every job runs on a different environment but I still thought this was possible. Please correct me if I am wrong.
All jobs in a workflow run on a different and clean runner. This is to prevent polluting your environment, file system, etc by changes from other/previous jobs.
The only way to share file system from install-dependency job is for it to upload a zip file of venv directory using
- uses: actions/upload-artifact@v4
with:
name: venv
path: ./venv
and then download it in each of the other jobs and activate:
- uses: actions/download-artifact@v4
with:
name: venv
path: ./venv
- name: Activate
run: source venv/bin/activate
Here is the complete workflow:
name: venv
on:
workflow_dispatch:
jobs:
install-dependency:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.11.9'
- name: Install Dependencies
run: |
python -m venv venv
source venv/bin/activate
python -m pip install requests
- uses: actions/upload-artifact@v4
with:
name: venv
path: ./venv
run-pytest:
runs-on: ubuntu-latest
needs: [install-dependency]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11.9'
- uses: actions/download-artifact@v4
with:
name: venv
path: ./venv
- name: Activate
run: source venv/bin/activate
run-other-pytest:
runs-on: ubuntu-latest
needs: [install-dependency]
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: '3.11.9'
- uses: actions/download-artifact@v4
with:
name: venv
path: ./venv
- name: Activate
run: source venv/bin/activate