Testing in CI vs testing in CD
I try to understand how testing fits into continuous integration and continuous delivery practices.
This article about CI states that automated tools are used to assert the new code’s correctness before integration.
Before new code is merged it must pass the CI test assertion suite which will prevent any new regressions.
But how can we build and test a new version of software without integrating all changes?
On the other hand, this page says that CI is a process that occurs prior to a build in which code is tested. And there is also a picture showing tests after version control. So, it looks like changes are first integrated in the version control, then software is built and tested.
Are tests in CI performed before or after integration of changes? And what happens if these tests fail? Will changes be rolled back then?
Another thing I find unclear is that testing is a part of both CI and CD phases. How does testing in CI differ from testing in CD?
CI is essentially a process to build each software component. Testing in CI = unit tests + any tests related to specific component being built in isolation from other components. In example, in your CI step you can run compiled jar against mock db and run some tests on that.
CI tests are performed after integration of changes but only on a single component being built. Usual flow - you do a Git commit, that triggers CI, tests would cover the code including that Git commit.
If CI tests fail, you should get notified and the build should not be promoted to CD step. In other words, the CICD pipeline aborts. Changes won't be rollbacked though - instead developers would need to decide what to do.
Testing in CD = integration testing, where software stack is deployed to some non-prod environment and being tested there. Such tests can be automated or manual. Note, that here you can implement auto-rollback to previous version of the stack.
I.e. see sample here of how we do automated CD testing - https://worklifenotes.com/2021/02/03/automated-tests-ci-cd-workflow-with-reliza-hub/
- Running xinc on OpenBSD's Apache Server
- How to change or specify a DVC experiment name?
- Best practice for SPA rollback on Azure Static Webapp with Azure Devops Deployment?
- Azure Pipeline: Unable to locate executable file: 'gulp'. during task Gulp@1
- How to only run a step if the workflow is running in the main repository in Github Actions?
- How to add release number from pipeline into K6-InfluxDB-Grafana stack so that in grafana we can filter results based on Release Number as well
- How to find Package.json version, and use as variable in Github Action
- How can I install and use compilers for embedded C on an external server?
- Run notification when status changes from previous run of the workflow on the same branch
- Using multiple runners in one gitlab-ci pipeline action
- Trigger Azure DevOps CI Build when pushing a branch with a certain name
- Change port for TeamCity web server
- Matching nvmrc node version to Docker node base image
- Run ASP.NET Core Backend in Azure DevOps CI Environment with Docker
- Gitlab CI/CD variables are not being passed down to Firebase when deploying
- Cannot understand how to download secure files in GitLab
- Use a different cargo profile for CI
- npm ci can only install packages with an existing package-lock.json or npm-shrinkwrap.json with lockfileVersion >= 1
- Azure YAML Pipeline schedule to run on the last Sunday of the month at 4 AM
- How to use markdown for description from a file in gitlab CI using release API
- Artisan migrate with ci/cd on azure container app
- AWS CodePipeline not extracting artifact before deployment
- Does implementing CI/CD require prerequisite steps?
- Docker in production
- In Azure DevOps, How to cancel the current build to directly start the new one?
- Github actions share workspace/artifacts between jobs?
- Hudson Build-Time Trend Plugin?
- Gitlab CI Terraform use plan and apply in diferent stage
- Github Action boolean input with default value
- How can I skip old vulnerabilities and break only with new ones using Snyk scan on Azure DevSecOps?