mirror of
https://gitlab.com/gitlab-org/gitlab-foss.git
synced 2025-07-23 00:47:51 +00:00
107 lines
5.0 KiB
Markdown
107 lines
5.0 KiB
Markdown
# GitLab QA - End-to-end tests for GitLab
|
|
|
|
This directory contains [end-to-end tests](../doc/development/testing_guide/end_to_end/_index.md)
|
|
for GitLab. It includes the test framework and the tests themselves.
|
|
|
|
The tests can be found in `qa/specs/features` (not to be confused with the unit
|
|
tests for the test framework, which are in `spec/`).
|
|
|
|
Tests use [GitLab QA project](https://gitlab.com/gitlab-org/gitlab-qa) for environment orchestration in CI jobs.
|
|
|
|
## What is it?
|
|
|
|
GitLab QA is an end-to-end test framework designed for test GitLab.
|
|
|
|
These are black-box and entirely click-driven end-to-end tests you can run against any existing instance of GitLab.
|
|
|
|
## How does it work?
|
|
|
|
1. When we release a new version of GitLab, we build a Docker images for it.
|
|
1. Along with GitLab Docker Images we also build and publish GitLab QA images.
|
|
1. QA image can be used to execute e2e tests against running instance of GitLab.
|
|
|
|
## Validating GitLab views / partials / selectors in merge requests
|
|
|
|
We recently added a new CI job that is going to be triggered for every push
|
|
event in CE and EE projects. The job is called `qa:selectors` and it will
|
|
verify coupling between page objects implemented as a part of GitLab QA
|
|
and corresponding views / partials / selectors in CE / EE.
|
|
|
|
Whenever `qa:selectors` job fails in your merge request, you are supposed to
|
|
fix [page objects](../doc/development/testing_guide/end_to_end/page_objects.md). You should also trigger end-to-end tests
|
|
using `package-and-qa` manual action, to test if everything works fine.
|
|
|
|
## How can I use it?
|
|
|
|
You can use GitLab QA to exercise tests on any live instance! If you don't
|
|
have an instance available you can follow the instructions below to use
|
|
the [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit).
|
|
This is the recommended option if you would like to contribute to the tests.
|
|
|
|
Note that tests are using `Chrome` web browser by default so it should be installed and present in `PATH`.
|
|
|
|
## CI
|
|
|
|
Tests are executed in merge request pipelines as part of the development lifecycle. CI pipeline setup and different types of environments are described in [test pipelines](../doc/development/testing_guide/end_to_end/test_pipelines.md) documentation page.
|
|
|
|
### Including tests in other projects
|
|
|
|
Pipeline template for `test-on-omnibus` E2E tests is designed in a way so it can be included as a child pipeline in other projects.
|
|
|
|
Minimal configuration example would look like this:
|
|
|
|
```yaml
|
|
qa-test:
|
|
stage: test
|
|
variables:
|
|
RELEASE: EE
|
|
trigger:
|
|
strategy: depend
|
|
forward:
|
|
yaml_variables: true
|
|
pipeline_variables: true
|
|
include:
|
|
- project: gitlab-org/gitlab
|
|
ref: master
|
|
file: .gitlab/ci/test-on-omnibus/main.gitlab-ci.yml
|
|
```
|
|
|
|
To set GitLab version used for testing, following environment variables can be used:
|
|
|
|
- `RELEASE`: `omnibus` release, can be string value `EE` or `CE` for nightly release of enterprise or community edition of GitLab or can be fully qualified Docker image name
|
|
- `QA_IMAGE`: Docker image of qa code. By default inferred from `RELEASE` but can be explicitly overridden with this variable
|
|
|
|
#### Test specific environment variables
|
|
|
|
Special GitLab configurations require various specific environment variables to be present for tests to work. These can be provisioned automatically using `terraform` setup in [engineering-productivity-infrastructure](https://gitlab.com/gitlab-org/quality/engineering-productivity-infrastructure/-/tree/main/qa-resources/modules/e2e-ci) project.
|
|
|
|
### Logging
|
|
|
|
By default tests on CI use `info` log level. `debug` level is still available in case of failure debugging. Logs are stored in jobs artifacts.
|
|
|
|
### Writing tests
|
|
|
|
- [Writing tests from scratch tutorial](../doc/development/testing_guide/end_to_end/beginners_guide.md)
|
|
- [Best practices](../doc/development/testing_guide/best_practices.md)
|
|
- [Using page objects](../doc/development/testing_guide/end_to_end/page_objects.md)
|
|
- [Guidelines](../doc/development/testing_guide/_index.md)
|
|
- [Tests with special setup for local environments](../doc/development/testing_guide/end_to_end/running_tests_that_require_special_setup.md)
|
|
|
|
### Running tests
|
|
|
|
- [Against your GDK environment](../doc/development/testing_guide/end_to_end/running_tests/_index.md#against-your-gdk-environment)
|
|
- [Against GitLab in Docker](../doc/development/testing_guide/end_to_end/running_tests/_index.md#against-gitlab-in-docker)
|
|
- [Specific types of tests](../doc/development/testing_guide/end_to_end/running_tests/_index.md#specific-types-of-tests)
|
|
- [Test configuration](../doc/development/testing_guide/end_to_end/running_tests/_index.md#test-configuration)
|
|
|
|
### Building a Docker image to test
|
|
|
|
Once you have made changes to the CE/EE repositories, you may want to build a
|
|
Docker image to test locally instead of waiting for the `gitlab-ce-qa` or
|
|
`gitlab-ee-qa` nightly builds. To do that, you can run **from the top `gitlab`
|
|
directory** (one level up from this directory):
|
|
|
|
```sh
|
|
docker build -t gitlab/gitlab-ce-qa:nightly --file ./qa/Dockerfile ./
|
|
```
|