Testing¶
Note
Unit tests and integration tests are automatically executed with every PR to https://github.com/CSCfi/beacon-python
Unit Testing¶
In order to run the unit tests, security checks with bandit, Sphinx documentation check for links consistency and HTML output and flake8 (coding style guide) tox. To run the unit tests in parallel use:
$ tox -p auto
To run environments seprately use:
$ # list environments
$ tox -l
$ # run flake8
$ tox -e flake8
$ # run bandit
$ tox -e bandit
$ # run docs
$ tox -e docs
Integration Testing¶
In order to run the integration tests, makes use of Source to Image:
$ git clone https://github.com/CSCfi/beacon-python
$ cd beacon-python
$ # please make sure to commit any changes before building otherwise s2i will not integrate them in build
$ s2i build . centos/python-36-centos7 cscfi/beacon-python
$ cd deploy/test
$ docker-compose up -d
$ docker-compose exec beacon beacon_init data/ALL.chrMT.phase3_callmom-v0_4.20130502.genotypes.vcf.gz data/example_metadata.json
$ docker-compose exec beacon beacon_init data/ALL.chrMT.phase3_callmom-v0_4.20130502.genotypes.vcf.gz /exdata/example_metadata_registered.json
$ docker-compose exec beacon beacon_init data/ALL.chrMT.phase3_callmom-v0_4.20130502.genotypes.vcf.gz /exdata/example_metadata_controlled.json
$ docker-compose exec beacon beacon_init data/ALL.chrMT.phase3_callmom-v0_4.20130502.genotypes.vcf.gz /exdata/example_metadata_controlled1.json
$ python test/run_tests.py
The integration tests will build a docker image of the beacon-python
and make use of
docker compose to deploy the Web Server and an associated
PostgreSQL Database as well as a mock OAuth2 authentication server.
Next step is to load mock data, as illustrated by the
docker-compose exec
commands above, and last step is to run the integration tests.
The integ_test.py
contains the actual tests while the run_tests
discovers the tests and runs them.
Load Testing¶
Load Testing scenarios
- Scenario 1
GET /info
- Scenario 2
GET /query
andPOST /query
(all, miss, hit) - Scenario 3
GET /query
andPOST /query
(all, miss, hit and (REGISTERED)) - Scenario 4
GET /query
andPOST /query
that has multiple parameters for a more complex query (e.g. startMin, startMax and others) also test with (all, miss, hit) and (REGISTERED)
Example testing with locust.io
from locust import HttpLocust, TaskSet, task
class APIBehavior(TaskSet):
"""Test Tasks for Beacon API."""
@task
def get_info(self):
"""Test the info endpoint.
The only endpoint that has some sort of caching.
"""
self.client.get("/")
@task
def get_query(self):
"""Test GET query endpoint."""
self.client.get("/query?assemblyId=GRCh38&referenceName=MT&start=9&referenceBases=T&alternateBases=C&includeDatasetResponses=HIT")
class APITest(HttpLocust):
"""Test Beacon API."""
task_set = APIBehavior
min_wait = 5000
max_wait = 9000