I'm currently working with a VisionFive2, after reading in lots of places and forums I've seen this firmware is the best one to make inferences in RISC-V. I've installed it and it works with the examples. What I need now is to be able to use my own models. I've trained a model and I've got the param and bin files. I did this using ultalytics, and as a result to the conversion I got a folder that included the following code:

import numpy as np
import ncnn
import torch

def test_inference():
    torch.manual_seed(0)
    in0 = torch.rand(1, 3, 640, 640, dtype=torch.float)
    out = []

    with ncnn.Net() as net:
        net.load_param("C:/Users/vbarreiro/Desktop/traffic_lights_ncnn_model/model.ncnn.param")
        net.load_model("C:/Users/vbarreiro/Desktop/traffic_lights_ncnn_model/model.ncnn.bin")

        with net.create_extractor() as ex:
            ex.input("in0", ncnn.Mat(in0.squeeze(0).numpy()).clone())

            _, out0 = ex.extract("out0")
            out.append(torch.from_numpy(np.array(out0)).unsqueeze(0))

    if len(out) == 1:
        return out[0]
    else:
        return tuple(out)

if __name__ == "__main__":
    print(test_inference())

That gave me the following result:

tensor([[[5.1116e+00, 1.0162e+01, 2.2278e+01,  ..., 5.5348e+02,
          5.8452e+02, 5.9801e+02],
         [5.8423e+00, 4.2124e+00, 3.7082e+00,  ..., 5.9824e+02,
          5.8354e+02, 5.8141e+02],
         [9.3326e+00, 1.5370e+01, 9.2346e+00,  ..., 1.6684e+02,
          1.1096e+02, 8.3524e+01],
         ...,
         [6.0677e-07, 4.2708e-08, 1.7761e-06,  ..., 1.6703e-07,
          6.4154e-07, 1.5601e-06],
         [8.0432e-07, 3.1833e-07, 1.0405e-04,  ..., 2.6704e-06,
          4.4843e-06, 4.1172e-06],
         [1.6021e-08, 2.1035e-09, 1.3104e-07,  ..., 3.1003e-08,
          7.8096e-08, 1.5352e-07]]])

I guess that means it works, but I don't really understand because I'm kinda new to this things. After seeing this I tried to make the inference on images. To do this I used Copilot, and it returned the following code:

def inference_on_image(image_path):
    torch.manual_seed(0)
    image = Image.open(image_path)
    image = np.array(image).astype(np.float32)
    image = torch.from_numpy(image).permute(2, 0, 1).unsqueeze(0)
    out = []
    with ncnn.Net() as net:
        net.load_param("C:/Users/vbarreiro/Desktop/traffic_lights_ncnn_model/model.ncnn.param")
        net.load_model("C:/Users/vbarreiro/Desktop/traffic_lights_ncnn_model/model.ncnn.bin")
        with net.create_extractor() as ex:
            ex.input("in0", ncnn.Mat(image.squeeze(0).numpy()).clone())
            _, out0 = ex.extract("out0")
            out.append(torch.from_numpy(np.array(out0)).unsqueeze(0))
    if len(out) == 1:
        return out[0]
    else:
        return tuple(out)

This returned:

tensor([[[ 1.2000e+01,  1.3567e+01, -1.1973e+01,  ...,  6.3999e+02,
           6.7200e+02,  6.7197e+02],
         [ 4.0000e+00,  3.9998e+00, -4.0000e+00,  ...,  6.2669e+02,
           6.3733e+02,  7.1768e+02],
         [ 8.0000e+01,  9.2865e+01,  1.5995e+02,  ...,  2.8798e+02,
           2.8800e+02,  3.5205e+02],
         ...,
         [ 4.1560e-39,  4.1560e-39,  4.1560e-39,  ...,  4.1560e-39,
           4.1560e-39,  4.1560e-39],
         [ 4.1560e-39,  4.1560e-39,  4.1560e-39,  ...,  4.1560e-39,
           4.1560e-39,  6.6484e-37],
         [ 4.1560e-39,  4.1560e-39,  4.1560e-39,  ...,  4.1560e-39,
           4.1560e-39,  4.1560e-39]]])

How can I convert this to show bounding boxes and probabilities as ultralytics does?