OpenStack

OpenStack logo

Gerrit

Hack OpenStack

Tests

Tools

See also

nose usage and pbr (github).

testr: isolate bug

https://rbtcollins.wordpress.com/2015/12/02/diagnosing-flaky-tests/

Load a subunit file downloaded from OpenStack gates:

(py27) $ wget http://logs.openstack.org/45/275645/1/check/gate-glance-python27/e3fffca/testrepository.subunit.gz
(py27) $ gunzip testrepository.subunit.gz
(py27) $ testr load testrepository.subunit

testr

Run tests:

  • run –no-parallel: run all tests in a single process
  • testr run –until-failure: run forever, until a test fails

Analyze latest run:

  • testr last –subunit|subunit2ls: list tests of the previous run
  • testr run –analyze-isolation: try to isolate the failing test, find the minimum tests to reproduce the fail

subunit tools:

  • subunit-filter
  • subunit-ls

Bisection: diagnosing flaky tests.

notes

Run unit tests:

. .tox/py27/bin/activate
testr run

Shell commands to run unit tests:

set -e && \
        TEMP_REZ=`mktemp -t` && \
        python setup.py testr --slowest --testr-args='--subunit  ' \
                | tee $$TEMP_REZ | subunit2pyunit || true ; \
        cat $$TEMP_REZ | subunit-filter -s --no-passthrough | subunit-stats ; \
        rm -f $$TEMP_REZ ;
  • --slowest shows the statistics at the end of the test run. Nothing fancy.
  • --testr-args='--subunit tells testr to output a subunit2 format for its unit tests. subunit2 format is a BINARY format, which you shouldn’t output to the screen.
  • subunit2pyunit will convert that to a nicer output
  • tee $$TEMP_REZ .. cat $$TEMP_REZ | subunit-filter -s --no-passthrough | subunit-stats shows the nice statistics about the test run (eg: how many tests in total, how many skips, how many failed, how many success)

testr: list skipped tests

testr last --subunit|subunit-filter -s|subunit-ls >A
testr last --subunit|subunit-filter -s --no-skip|subunit-ls >B
diff -u A B

tox/testr: “db type could not be determined” error

testr uses a database to store test results. If the database is created by Python 2, Python 3 cannot read it and then you get the error “db type could not be determined”.

Workaround: remove .testrepository directory and rerun tox again.

tox/testr: “gdbm is missing”

If you run tox -e py34 and then tox -e py27, the second commands may fail because Python 2.7 does not have the gdbm module.

On Ubuntu, type:

sudo apt-get install -y python-gdbm

testr: “local variable ‘run_subunit_content’ referenced before assignment” error

See Error message opaque when .testrepository files are unreadable.

Re-run a single failing test

testtools

Re-run a single test with testtools:

$ tox -e py33
...
FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok
...
$ . .tox/py33/bin/activate
$ python -m testtools.run tests.test_swiftclient.TestPutObject.test_unicode_ok
 Tests running...
 ======================================================================
 FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok
 ----------------------------------------------------------------------
 ...
 Ran 1 test in 0.002s

tox

Re-run a single test with tox+testr:

$ tox -e py33
...
FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok
...
$ tox -e py33 -- --isolated tests.test_swiftclient.TestPutObject.test_unicode_ok
...
FAIL: tests.test_swiftclient.TestPutObject.test_unicode_ok
...

Note

Enter the virtualenv and type testr run tests.test_swiftclient.TestPutObject.test_unicode_ok should work, but it doesn’t in the Python 3.3 virtual environment of python-sphinxclient?!

nose

Re-run a single test with nose:

$ nosetests
...
======================================================================
FAIL: tests.test_command_helpers.TestStatHelpers.test_stat_account_human
----------------------------------------------------------------------
...

$ nosetests tests.test_command_helpers:TestStatHelpers.test_stat_account_human

RabbitMQ

Type:

dnf install -y rabbitmq-server
vim /etc/rabbitmq/rabbitmq.config
# in "{rabbit," uncomment:
#    {loopback_users, []}
# (no trailing comma ",")
sudo systemctl restart rabbitmq-server
sudo systemctl status rabbitmq-server
sudo rabbitmqctl change_password guest password

DevStack

To install Fedora:

OpenStack openstack_citest

For MySQL you can use the following commands:

mysql -u root
mysql> CREATE USER 'openstack_citest'@'localhost' IDENTIFIED BY
       'openstack_citest';
mysql> GRANT ALL PRIVILEGES ON * . * TO 'openstack_citest'@'localhost';
mysql> FLUSH PRIVILEGES;

http://docs.openstack.org/developer/oslo.db/contributing.html#how-to-run-unit-tests

Ceilometer

Install dependencies:

sudo yum install mariadb-devel mongodb-server rabbitmq-server

Start MongoDB server:

sudo systemctl start mongod
sudo systemctl start rabbitmq-server

Copy Ceilometer config:

tox -e genconfig
sudo mkdir /etc/ceilometer
sudo cp -R etc/ceilometer/ /etc/ceilometer/

Configure Ceilometer database:

[database]
connection = mongodb://127.0.0.1:27017/ceilometer

Create the DB:

ceilometer-dbsync

Run collector in debug:

ceilometer-collector -d

Send a sample:

ceilometer-send-sample --sample-name name --sample-resource resource

Show meters in MongoDB:

$ mongo
> use ceilometer
> db.meter.find()
(...)
^D

Note: If you get the error “mongo: symbol lookup error: mongo: undefined symbol: _ZN2v86LockerC1EPNS_7IsolateE” when running the “mongo” command, see the bug mongo client lookup error. The bug occurs if you installed the package “v8” from the Chromium repository.