Python Module¶
Here we document how to use and extend the python bindings for a project created with this cookiecutter. See `cpptools.readthedocs.io/en/latest/python.html https://cpptools.readthedocs.io/en/latest/python.html>`_ for the python documentation of a sample project created with this cookiecutter.
Folder Structure¶
We use pybind11 to create the python bindings.
The python subfolder contains all the code related
to the python bindings.
The module/{{cookiecutter.python_package_name}}
subfolder contains all the *.py
files of the module.
The src folder contains the *.cpp
files used to export the C++ functionality to python via pybind11.
The test
folder contains all python tests.
{{cookiecutter.github_project_name}}
├── ...
├── python
│ ├── module
│ │ └── {{cookiecutter.python_package_name}}
│ │ ├── __init__.py
│ │ └── ...
│ ├── src
│ │ ├── CMakeLists.txt
│ │ ├── main.cpp
│ │ ├── def_build_config.cpp
│ │ ├── ...
│ └── test
│ ├── test_build_configuration.py
│ └── ...
└── ...
Build System¶
To build the python package use the python-module
target.
make python-module
This will build the *.cpp
files in the src
folder and copy the folder module/{{cookiecutter.python_package_name}}
folder to build location of the python module, namely ${CMAKE_BINARY_DIR}/python/module/
where ${CMAKE_BINARY_DIR}
is the build directory.
Usage¶
After a successfully building and installing the python module can be imported like the following:
import {{cookiecutter.python_package_name}}
config = {{cookiecutter.python_package_name}}.BuildConfiguration
print(config.VERSION_MAJOR)
Run Python Tests¶
To run the python test suite use the python-test target:
make python-test
Adding New Python Functionality¶
We use pybind11 to export functionality from C++ to Python.
pybind11 can create modules from C++ without the use of any *.py
files.
Nevertheless we prefer to have a regular Python package with a proper __init__.py
. From the __init__.py
we import all the C++ / pybind11 exported functionality from the build submodule named _{{cookiecutter.python_package_name}}
.
This allows us to add new functionality in different ways:
new functionality from c++ via pybind11
new puren python functionality
Add New Python Functionality from C++¶
To export functionality from C++ to python via pybind11 it is
good practice to split functionality in multiple def_*.cpp
files.
This allow for readable code, and parallel builds.
To add news functionality we create a new file, for example def_new_stuff.cpp
.
#include "pybind11/pybind11.h"
#include "pybind11/numpy.h"
#include <iostream>
#include <numeric>
#define FORCE_IMPORT_ARRAY
#include "xtensor-python/pyarray.hpp"
#include "xtensor-python/pytensor.hpp"
// our headers
#include "{{cookiecutter.cpp_root_folder_name}}/{{ cookiecutter.package_name}}.hpp"
namespace py = pybind11;
namespace {{cookiecutter.cpp_namespace}} {
void def_new_stuff(py::module & m)
{
py::def('new_stuff',[](xt::pytensor<1,double> values){
return values * 42.0;
});
}
}
Next we need to declare and call the def_new_stuff
from main.cpp
.
To declare the function modify the following block in main.cpp
namespace {{cookiecutter.cpp_namespace}} {
// ....
// ....
// ....
// implementation in def_myclass.cpp
void def_class(py::module & m);
// implementation in def_myclass.cpp
void def_build_config(py::module & m);
// implementation in def.cpp
void def_build_config(py::module & m);
// implementation in def.cpp
void def_build_config(py::module & m);
// implementation in def_new_stuff.cpp
void def_new_stuff(py::module & m); // <- our new functionality
}
After declaring the function def_new_stuff
, we can call def_new_stuff
. We modify the PYBIND11_MODULE
in
code:main.cpp:
// Python Module and Docstrings
PYBIND11_MODULE(_{{cookiecutter.python_package_name}} , module)
{
xt::import_numpy();
module.doc() = R"pbdoc(
_{{cookiecutter.python_package_name}} python bindings
.. currentmodule:: _{{cookiecutter.python_package_name}}
.. autosummary::
:toctree: _generate
BuildConfiguration
MyClass
new_stuff
)pbdoc";
{{cookiecutter.cpp_namespace}}::def_build_config(module);
{{cookiecutter.cpp_namespace}}::def_class(module);
{{cookiecutter.cpp_namespace}}::def_new_stuff(module); // <- our new functionality
// make version string
std::stringstream ss;
ss<<{{cookiecutter.cpp_macro_prefix}}_VERSION_MAJOR<<"."
<<{{cookiecutter.cpp_macro_prefix}}_VERSION_MINOR<<"."
<<{{cookiecutter.cpp_macro_prefix}}_VERSION_PATCH;
module.attr("__version__") = ss.str();
}
We need to add this file to the CMakeLists.txt
file at
{cookiecutter.github_project_name}}/python/src/CMakeLists.txt
The file needs to be passed as an argument to the pybind11_add_module
function.
# add the python library
pybind11_add_module(${PY_MOD_LIB_NAME}
main.cpp
def_build_config.cpp
def_myclass.cpp
def_new_stuff.cpp # <- our new functionality
)
Now we are ready to build the freshly added functionality.
make python-test
After a successful build we can use the new functionality from python.
import numpy as np
import {{cookiecutter.python_package_name}}
{{cookiecutter.python_package_name}}.new_stuff(numpy.arange(5), dtype='float64')
Add New Pure Python Functionality¶
To add new pure Python functionality,
just add the desired function / classes to
a new *.py
file and put this file to the
module/{{cookiecutter.python_package_name}}
subfolder.
After adding the new file, cmake needs to be rerun since we copy the content module/{{cookiecutter.python_package_name}}
during the build process.
Adding New Python Tests¶
We use pytest as python test framework.
To add new tests, just add new test_*.py
files
to the test subfolder.
To run the actual test use the python-test
target
make python-test