Paddle 关于C#调用dll方式,CPU端部署,同时进行两个或多个模型推理报错

j13ufse2  于 2022-11-03  发布在  C#
关注(0)|答案(3)|浏览(381)

版本,环境信息:
PaddlePaddle2.0.0rc;PaddleClas-dygraph;CPU-mkl预测库;i7,win10

在AISTUDIO云端学习得到的模型。在CPU上进行部署,使用C#调用dll的方式
C++代码如下,为了能够运行两个模型预测,定义了PaddleClasClass类。
同时载入ResNet18和ResNet50模型进行预测,会直接报错,报错后重启程序,仅仅执行一个模型的预测也会报错,需要等一会才能正常进行一个模型的预测

代码运行应该没什么问题,我还尝试了,使用GPU预测库,使用GPU同时进行两个模型的预测使正常的。


# include "opencv2/core.hpp"

# include "opencv2/imgcodecs.hpp"

# include "opencv2/imgproc.hpp"

# include <chrono>

# include <iomanip>

# include <iostream>

# include <opencv2/core/utils/filesystem.hpp>

# include <ostream>

# include <vector>

# include <cstring>

# include <fstream>

# include <numeric>

# include <include/cls.h>

# include <include/cls_config.h>

# include <Windows.h>

using namespace std;
using namespace cv;
using namespace PaddleClas;

class PaddleClasClass
{
public:

    explicit PaddleClasClass() {};

    Classifier Classifier_Public;
    void _LoadModel(char* configPath)
    {
        ClsConfig config(configPath);

        Classifier_Public = Classifier(config.cls_model_path, config.cls_params_path,
            config.use_gpu, config.gpu_id, config.gpu_mem,
            config.cpu_math_library_num_threads, config.use_mkldnn,
            config.resize_short_size, config.crop_size);

        std::cout << "1" << std::endl;
    }

    /// <summary>
    /// 预测图像
    /// </summary>
    /// <param name="height">图像的高</param>
    /// <param name="width">图像的宽</param>
    /// <param name="channels">图像的通道数</param>
    /// <param name="frame_src">图像所转换成的数组</param>
    /// <param name="class_id">图像分类</param>
    /// <param name="score">图像分类得分</param>
    void _PredictImage(int height, int width, int channels, BYTE* frame_src, int& class_id, float& score, float* all_score)
    {
        //根据输入判断图片格式
        int format;
        switch (channels)
        {
        case 1:
            format = CV_8UC1;
            break;
        case 2:
            format = CV_8UC2;
            break;
        case 3:
            format = CV_8UC3;//bgr
            break;
        default:
            format = CV_8UC4;
            break;
        }

        cv::Mat srcimg(height, width, format, frame_src);
        cv::cvtColor(srcimg, srcimg, cv::COLOR_BGR2RGB);//转化成rgb
        std::cout << "2" << std::endl;

        auto start = std::chrono::system_clock::now();
        Classifier_Public.Run(srcimg);
        auto end = std::chrono::system_clock::now();
        auto duration =
            std::chrono::duration_cast<std::chrono::microseconds>(end - start);
        std::cout << "CostClas "
            << double(duration.count()) *
            std::chrono::microseconds::period::num /
            std::chrono::microseconds::period::den
            << " s" << std::endl;
        std::cout << "3" << std::endl;

        class_id = Classifier_Public.class_id;
        std::cout << "4" << std::endl;
        score = Classifier_Public.score;
        std::cout << "5" << std::endl;

        for (int i = 0; i < Classifier_Public.all_score.size(); i++)
        {
            all_score[i] = Classifier_Public.all_score[i];
        }

        //all_score = Classifier_Public.all_score;

    }
private:
};

PaddleClasClass PaddleClasClass1;
PaddleClasClass PaddleClasClass2;

extern "C" __declspec(dllexport) void  _LoadModel2(char* config);
extern "C" __declspec(dllexport) void  _LoadModel1(char* config);
extern "C" __declspec(dllexport) void _PredictImage2(int height, int width, int channels, BYTE * frame_src, int& class_id, float& score, float* all_score);
extern "C" __declspec(dllexport) void _PredictImage1(int height, int width, int channels, BYTE * frame_src, int &class_id, float &score, float* all_score);

void _LoadModel1(char* configPath)
{
    PaddleClasClass1._LoadModel(configPath);
}

void _PredictImage1(int height, int width, int channels, BYTE* frame_src, int& class_id, float& score, float* all_score)
{
    PaddleClasClass1._PredictImage(height, width, channels,frame_src, class_id, score,  all_score);
}

void _LoadModel2(char* configPath)
{
    PaddleClasClass2._LoadModel(configPath);
}

void _PredictImage2(int height, int width, int channels, BYTE* frame_src, int& class_id, float& score, float* all_score)
{
    PaddleClasClass2._PredictImage(height, width, channels, frame_src, class_id, score, all_score);
}

为使您的问题得到快速解决,在建立Issue前,请您先通过如下方式搜索是否有相似问题:【搜索issue关键字】【使用labels筛选】【官方文档】

如果您没有查询到相似问题,为快速解决您的提问,建立issue时请提供如下细节信息:

  • 标题:简洁、精准描述您的问题,例如“最新预测库的API文档在哪儿 ”
  • 版本、环境信息:

   1)PaddlePaddle版本:请提供您的PaddlePaddle版本号(如1.1)或CommitID
   2)CPU:预测若用CPU,请提供CPU型号,MKL/OpenBlas/MKLDNN/等数学库使用情况
   3)GPU:预测若用GPU,请提供GPU型号、CUDA和CUDNN版本号
   4)系统环境:请您描述系统类型、版本(如Mac OS 10.14),Python版本
注:您可以通过执行 summary_env.py 获取以上信息。
-预测信息
   1)C++预测:请您提供预测库安装包的版本信息,及其中的version.txt文件
   2)CMake包含路径的完整命令
   3)API信息(如调用请提供)
   4)预测库来源:官网下载/特殊环境(如BCLOUD编译)

  • 复现信息:如为报错,请给出复现环境、复现步骤
  • 问题描述:请详细描述您的问题,同步贴出报错信息、日志/代码关键片段

Thank you for contributing to PaddlePaddle.
Before submitting the issue, you could search issue in the github in case that th
If there is no solution,please make sure that this is an inference issue including the following details :

System information

-PaddlePaddle version (eg.1.1)or CommitID
-CPU: including CPUMKL/OpenBlas/MKLDNN version
-GPU: including CUDA/CUDNN version
-OS Platform (eg.Mac OS 10.14)
-Python version
-Cmake orders
-C++version.txt
-API information
Note: You can get most of the information by running summary_env.py .

To Reproduce

Steps to reproduce the behavior

Describe your current behavior
Code to reproduce the issue
Other info / logs

f2uvfpb9

f2uvfpb91#

您好,我们已经收到了您的问题,会安排技术人员尽快解答您的问题,请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时,您也可以通过查看 官网API文档常见问题历史IssueAI社区 来寻求解答。祝您生活愉快~

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the APIFAQGithub Issue and AI community to get the answer.Have a nice day!

lpwwtiir

lpwwtiir2#

  1. CPU下,可以尝试关闭mkldnn,跑2个模型
  2. 可以获取下最新提供的预测库,修复了windows下,mkldnn预测的一些问题
8dtrkrch

8dtrkrch3#

  1. CPU下,可以尝试关闭mkldnn,跑2个模型
  2. 可以获取下最新提供的预测库,修复了windows下,mkldnn预测的一些问题
    1、我试了关闭mkldnn,可以同时预测两个模型了,但是推理时间相应的变长了,原来大概是20ms,现在是60ms,这是我不想看到的
    2、最新的预测库?我是在 https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/advanced_guide/inference_deployment/inference/windows_cpp_inference.html
    下载的,是2.0.0rc0对应的版本,这个是最新的么?

我还尝试了1.8.5的预测库,也是无法同时运行两个模型

相关问题