TensorRT C++ 批量推理笔记

x33g5p2x  于2022-03-28 转载在 其他  
字(0.7k)|赞(0)|评价(0)|浏览(525)

batch为1时,如下:

void* buffers[2];
buffers[inputIndex] = inputbuffer;
buffers[outputIndex] = outputBuffer;

但是batch_size大于1时,inputbuffer和outputBuffer如何设置,以及如何构建引擎?

批量推理准备数据:

void parseYolov5(cv::Mat& img,ICudaEngine* engine,IExecutionContext* context,std::vector<Yolo::Detection>& batch_res)
{
    // 准备数据 ---------------------------
    static float data[BATCH_SIZE * 3 * INPUT_H * INPUT_W];  //输入
    static float prob[BATCH_SIZE * OUTPUT_SIZE];            //输出

    assert(engine->getNbBindings() == 2);
    void* buffers[2];
    // In order to bind the buffers, we need to know the names of the input and output tensors.
    // Note that indices are guaranteed to be less than IEngine::getNbBindings()
    const int inputIndex = engine->getBindingIndex(INPUT_BLOB_NAME);
    const int outputIndex = engine->getBindin

相关文章