使用pybind11从c++调用Python函数

dzjeubhm  于 2023-01-18  发布在  Python
关注(0)|答案(3)|浏览(337)

我尝试使用Pybind11从包含main()函数的C代码中调用python函数。但我发现可用的参考资料很少。大多数现有文档都在讨论相反的方向,即从Python调用C
有没有完整的例子说明如何做到这一点?我找到的唯一参考是:https://github.com/pybind/pybind11/issues/30
但它的信息很少。

qkf9rpyu

qkf9rpyu1#

你问题的答案其实有两部分:一个是关于从C调用Python函数,另一个是关于嵌入解释器。
在pybind11中调用一个函数只不过是把这个函数放到一个pybind11::object变量中,你可以在这个变量上调用operator()来尝试调用这个对象(它不一定是一个函数,只要是一个可调用的函数就行了:例如,它也可以是一个带有__call__方法的对象)。例如,要从C
代码调用math.sqrt(2),您可以用途:

auto math = py::module::import("math");
auto resultobj = math.attr("sqrt")(2);
double result = resultobj.cast<double>();

或者你可以把它浓缩成

double result = py::module::import("math").attr("sqrt")(2).cast<double>();

问题的第二部分涉及到如何从C可执行文件中实现这一点。当构建一个可执行文件时(即当你的C代码包含main()时),你必须先将Python解释器嵌入到你的二进制文件中,然后才能用Python做任何事情(比如调用Python函数)。
嵌入式支持是当前pybind11 master分支(将成为2.2版本)中增加的一个新特性,下面是一个基本的例子,它启动一个嵌入式Python解释器并调用一个Python函数(math.sqrt):

#include <pybind11/embed.h>
#include <iostream>

namespace py = pybind11;

int main() {
    py::scoped_interpreter python;

    auto math = py::module::import("math");
    double root_two = math.attr("sqrt")(2.0).cast<double>();

    std::cout << "The square root of 2 is: " << root_two << "\n";
}

输出:

The square root of 2 is: 1.41421

调用函数和嵌入的更多示例和文档分别位于http://pybind11.readthedocs.io/en/master/advanced/pycpp/object.htmlhttp://pybind11.readthedocs.io/en/master/advanced/embedding.html

enyaitl3

enyaitl32#

jasons的回答非常中肯,但是我想添加一个稍微复杂(并且简洁)的例子,用numpy输入调用python方法,我想展示两点:
1.我们可以使用py::reinterpret_borrow<py::function>py::object转换为py::function
1.我们可以输入std::vector,它会自动转换为numpy.array
请注意,用户有责任确保PyModule.attr实际上是一个python函数,还要注意,类型转换适用于各种c++类型(详细信息请参见here)。
在这个例子中,我想使用scipy.optimize.minimize方法,它的起点是从c++接口提供的。

#include <iostream>
#include <vector>
#include <pybind11/pybind11.h>
#include <pybind11/embed.h>  // python interpreter
#include <pybind11/stl.h>  // type conversion

namespace py = pybind11;

int main() {
  std::cout << "Starting pybind" << std::endl;
  py::scoped_interpreter guard{}; // start interpreter, dies when out of scope

  py::function min_rosen =
      py::reinterpret_borrow<py::function>(   // cast from 'object' to 'function - use `borrow` (copy) or `steal` (move)
          py::module::import("py_src.exec_numpy").attr("min_rosen")  // import method "min_rosen" from python "module"
      );

  py::object result = min_rosen(std::vector<double>{1,2,3,4,5});  // automatic conversion from `std::vector` to `numpy.array`, imported in `pybind11/stl.h`
  bool success = result.attr("success").cast<bool>();
  int num_iters = result.attr("nit").cast<int>();
  double obj_value = result.attr("fun").cast<double>();
}

使用Python脚本py_src/exec_numpy.py

import numpy as np
from scipy.optimize import minimize, rosen, rosen_der

def min_rosen(x0):
    res = minimize(rosen, x0)
    return res

希望这对某人有帮助!

brqmpdu1

brqmpdu13#

1.项目结构

  • CMakeLists.txt
  • calc.py
  • main.cpp
  1. main.cpp
#include <pybind11/embed.h>
#include <iostream>
namespace py = pybind11;
using namespace py::literals;

int main() {
    py::scoped_interpreter guard{};

    // append source dir to sys.path, and python interpreter would find your custom python file
    py::module_ sys = py::module_::import("sys");
    py::list path = sys.attr("path");
    path.attr("append")("..");

    // import custom python class and call it
    py::module_ tokenize = py::module_::import("calc");
    py::type customTokenizerClass = tokenize.attr("CustomTokenizer");
    py::object customTokenizer = customTokenizerClass("/Users/Caleb/Desktop/codes/ptms/bert-base");
    py::object res = customTokenizer.attr("custom_tokenize")("good luck");

    // show the result
    py::list input_ids = res.attr("input_ids");
    py::list token_type_ids = res.attr("token_type_ids");
    py::list attention_mask = res.attr("attention_mask");
    py::list offsets = res.attr("offset_mapping");
    std::string message = "input ids is {},\noffsets is {}"_s.format(input_ids, offsets);
    std::cout << message << std::endl;
}
  1. calc.py
from transformers import BertTokenizerFast

class CustomTokenizer(object):
    def __init__(self, vocab_dir):
        self._tokenizer = BertTokenizerFast.from_pretrained(vocab_dir)

    def custom_tokenize(self, text):
        return self._tokenizer(text, return_offsets_mapping=True)

def build_tokenizer(vocab_dir: str) -> BertTokenizerFast:
    tokenizer = BertTokenizerFast.from_pretrained(vocab_dir)
    return tokenizer

def tokenize_text(tokenizer: BertTokenizerFast, text: str) -> dict:
    res = tokenizer(text, return_offsets_mapping=True)
    return dict(res)
  1. CMakeLists.txt
cmake_minimum_required(VERSION 3.4)
project(example)
set(CMAKE_CXX_STANDARD 11)

# set pybind11 dir
set(pybind11_DIR /Users/Caleb/Softwares/pybind11)
find_package(pybind11 REQUIRED)

# set custom python interpreter(under macos)
link_libraries(/Users/Caleb/miniforge3/envs/py38/lib/libpython3.8.dylib)

add_executable(example main.cpp)
target_link_libraries(example PRIVATE pybind11::embed)

相关问题