py.test implements all aspects of configuration, collection, running and reporting by calling well specified hooks. Virtually any Python module can be registered as a plugin. It can implement any number of hook functions (usually two or three) which all have a pytest_ prefix, making hook functions easy to distinguish and find. There are three basic location types:
local conftest.py plugins contain directory-specific hook implementations. Session and test running activities will invoke all hooks defined in conftest.py files closer to the root of the filesystem. Example: Assume the following layout and content of files:
a/conftest.py:
def pytest_runtest_setup(item):
# called for running each test in 'a' directory
print ("setting up", item)
a/test_in_subdir.py:
def test_sub():
pass
test_flat.py:
def test_flat():
pass
Here is how you might run it:
py.test test_flat.py # will not show "setting up"
py.test a/test_sub.py # will show "setting up"
Note
If you have conftest.py files which do not reside in a python package directory (i.e. one containing an __init__.py) then “import conftest” can be ambiguous because there might be other conftest.py files as well on your PYTHONPATH or sys.path. It is thus good practise for projects to either put conftest.py under a package scope or to never import anything from a conftest.py file.
Installing a plugin happens through any usual Python installation tool, for example:
pip install pytest-NAME
pip uninstall pytest-NAME
If a plugin is installed, py.test automatically finds and integrates it, there is no need to activate it. Here is a initial list of known plugins:
You may discover more plugins through a pytest- pypi.python.org search.
If you want to write a plugin, there are many real-life examples you can copy from:
All of these plugins implement the documented well specified hooks to extend and add functionality.
If you want to make your plugin externally available, you may define a so-called entry point for your distribution so that py.test finds your plugin module. Entry points are a feature that is provided by setuptools or Distribute. py.test looks up the pytest11 entrypoint to discover its plugins and you can thus make your plugin available by definig it in your setuptools/distribute-based setup-invocation:
# sample ./setup.py file
from setuptools import setup
setup(
name="myproject",
packages = ['myproject']
# the following makes a plugin available to py.test
entry_points = {
'pytest11': [
'name_of_plugin = myproject.pluginmodule',
]
},
)
If a package is installed this way, py.test will load myproject.pluginmodule as a plugin which can define well specified hooks.
py.test loads plugin modules at tool startup in the following way:
You can require plugins in a test module or a conftest file like this:
pytest_plugins = "name1", "name2",
When the test module or conftest plugin is loaded the specified plugins will be loaded as well. You can also use dotted path like this:
pytest_plugins = "myapp.testsupport.myplugin"
which will import the specified module as a py.test plugin.
If a plugin wants to collaborate with code from another plugin it can obtain a reference through the plugin manager like this:
plugin = config.pluginmanager.getplugin("name_of_plugin")
If you want to look at the names of existing plugins, use the --traceconfig option.
If you want to find out which plugins are active in your environment you can type:
py.test --traceconfig
and will get an extended test header which shows activated plugins and their names. It will also print local plugins aka conftest.py files when they are loaded.
You can prevent plugins from loading or unregister them:
py.test -p no:NAME
This means that any subsequent try to activate/load the named plugin will it already existing. See Finding out which plugins are active for how to obtain the name of a plugin.
You can find the source code for the following plugins in the pytest repository.
_pytest.assertion | support for presenting detailed information in failing assertions. |
_pytest.capture | |
_pytest.config | |
_pytest.doctest | |
_pytest.genscript | generate a single-file self-contained version of py.test |
_pytest.helpconfig | |
_pytest.junitxml | report test results in JUnit-XML format, for use with Hudson and build integration servers. |
_pytest.mark | generic mechanism for marking and selecting python functions. |
_pytest.monkeypatch | monkeypatching and mocking functionality. |
_pytest.nose | |
_pytest.pastebin | submit failure or test session information to a pastebin service. |
_pytest.pdb | |
_pytest.pytester | |
_pytest.python | |
_pytest.recwarn | recording warnings during test function execution. |
_pytest.resultlog | log machine-parseable test session result information in a plain |
_pytest.runner | basic collect and runtest protocol implementations |
_pytest.main | |
_pytest.skipping | |
_pytest.terminal | |
_pytest.tmpdir | |
_pytest.unittest |
py.test calls hook functions to implement initialization, running, test execution and reporting. When py.test loads a plugin it validates that each hook function conforms to its respective hook specification. Each hook function name and its argument names need to match a hook specification. However, a hook function may accept fewer parameters by simply not specifying them. If you mistype argument names or the hook name itself you get an error showing the available arguments.
(deprecated) modify command line arguments before option parsing.
return initialized config object, parsing the specified args.
return dict of name->object to be made globally available in the py.test/pytest namespace. This hook is called before command line options are parsed.
register argparse-style options and ini-style config values.
This function must be implemented in a plugin and is called once at the beginning of a test run.
Parameters: | parser – To add command line options, call parser.addoption(...). To add ini-file values call parser.addini(...). |
---|
Options can later be accessed through the config object, respectively:
The config object is passed around on many internal objects via the .config attribute or can be retrieved as the pytestconfig fixture or accessed via (deprecated) pytest.config.
called for performing the main command line action. The default implementation will invoke the configure hooks and runtest_mainloop.
All all runtest related hooks receive a pytest.Item object.
implements the runtest_setup/call/teardown protocol for the given test item, including capturing exceptions and calling reporting hooks.
Parameters: |
|
---|---|
Return boolean: | True if no further hook implementations should be invoked. |
called after pytest_runtest_call.
Parameters: | nexitem – the scheduled-to-be-next test item (None if no further test item is scheduled). This argument can be used to perform exact teardowns, i.e. calling just enough finalizers so that nextitem only needs to call setup-functions. |
---|
return a _pytest.runner.TestReport object for the given pytest.Item and _pytest.runner.CallInfo.
For deeper understanding you may look at the default implementation of these hooks in _pytest.runner and maybe also in _pytest.pdb which interacts with _pytest.capture and its input/output capturing in order to immediately drop into interactive debugging when a test failure occurs.
The _pytest.terminal reported specifically uses the reporting hook to print information about a test run.
py.test calls the following hooks for collecting files and directories:
return True to prevent considering this path for collection. This hook is consulted for all files and directories prior to calling more specific hooks.
called before traversing a directory for collection files.
return collection Node or None for the given path. Any new node needs to have the specified parent as a parent.
For influencing the collection of objects in Python modules you can use the following hook:
return custom item/collector for a python object in a module, or None.
After collection is complete, you can modify the order of items, delete or otherwise amend the test items:
Session related reporting hooks:
And here is the central hook for reporting about test execution:
There are few hooks which can be used for special reporting or interaction with exceptions:
Result/Exception info a function invocation.
context of invocation: one of “setup”, “call”, “teardown”, “memocollect”
None or ExceptionInfo object.
Basic test report object (also used for setup and teardown calls if they fail).
normalized collection node id
a (filesystempath, lineno, domaininfo) tuple indicating the actual location of a test item - it might be different from the collected one e.g. if a method is inherited from a different module.
a name -> value dictionary containing all keywords and markers associated with a test invocation.
test outcome, always one of “passed”, “failed”, “skipped”.
None or a failure representation.
one of ‘setup’, ‘call’, ‘teardown’ to indicate runtest phase.
list of (secname, data) extra information which needs to marshallable
time it took to run just the test