flutter 从视频帧解压缩回调复制图像缓冲区输出

yzuktlbb  于 2022-12-19  发布在  Flutter
关注(0)|答案(1)|浏览(122)

我是Swift新手,需要使用VideoToolBox API进行H264解码。我的应用程序使用Flutter SDK编写,正在使用MethodChannel()类调用Swift函数。要解码帧,请使用函数VTDecompressionSessionDecodeFrame()。解压缩会话完成后,回调接收解码的帧和状态。首先我需要复制状态来观察我是否提供了正确的NAL/有效载荷。m facing正在将结果复制到回调函数,然后复制到调用VTDecompressionSessionDecodeFrame()的函数。我需要将状态/图像返回到调用函数,以便可以为MethodChannel()创建响应
我在这个问题上的第一次尝试是分配一个类变量,它最终包含提供给回调的imageBuffer变量的数据。但是由于编译器错误,这没有起作用。我尝试分配内存并将imageBuffer的每个元素复制到类变量。
在Flutter中,有ValueChangeNotifier()类,并认为可以在Swift中使用类似的类,但这似乎不是正确的实现。
下面的代码来自AppDelegate.swift文件。我提供了最少的代码来强调这个问题。

@UIApplicationMain
@objc class AppDelegate: FlutterAppDelegate {
  override func application(
    _ application: UIApplication,
    didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?
  ) -> Bool {
      let controller : FlutterViewController = window?.rootViewController as! FlutterViewController
      let decodeChannel = FlutterMethodChannel(name: "adhoc.flutter.dev/decode",
                                       binaryMessenger: controller.binaryMessenger)
      
      decodeChannel.setMethodCallHandler( { [weak self]
          (call: FlutterMethodCall, result: @escaping FlutterResult) -> Void in
          
          guard call.method == "decodeFrame" else {
              result(FlutterMethodNotImplemented)
              return
          }
          self?.decodeFrame(call: call, result: result)
      })
      
    GeneratedPluginRegistrant.register(with: self)
    return super.application(application, didFinishLaunchingWithOptions: launchOptions)
  }
    
    private func decodeFrame(call: FlutterMethodCall, result: FlutterResult) {\
        var status: OSStatus = 0
        // Receive encoded frame from MethodChannel
        // convert to UInt8 array
     
        var callback = VTDecompressionOutputCallbackRecord()
        callback.decompressionOutputCallback = { (outputRefCon, frameRefCon, status, infoFlags, imageBuffer, timeStamp, presentationDuration) in
          print("done")
            // How may I copy status/imageBuffer to caller of VTDecompressionSessionDecodeFrame?
            }
            
       }
        // configure decoder, set sampleBuffer, etc. Note not all steps included, just setting the 
        // callback for the session and the decode. 
        if(status == noErr)
        {
            status = VTDecompressionSessionCreate(allocator: kCFAllocatorDefault, formatDescription: formatDesc!, decoderSpecification: nil, imageBufferAttributes: nil, outputCallback: &callback, decompressionSessionOut: &session);
        } else{
            rtn = ["image": [], "status": status];
            return result(rtn)
        }

        if(status == noErr) {
            status = VTDecompressSessionDecodeFrame(session!, sampleBuffer: sampleBuffer!, flags:               
                [._EnableTemporalProcessing], frameRefCon: frameRef, infoFlagsOut: nil)
        }

        // Return [UInt8] image and status code
    }
}
hmae6n7t

hmae6n7t1#

通过对函数使用以下替代解决:

func VTDecompressionSessionDecodeFrame(
    _ session: VTDecompressionSession,
    sampleBuffer: CMSampleBuffer,
    flags decodeFlags: VTDecodeFrameFlags,
    infoFlagsOut: UnsafeMutablePointer<VTDecodeInfoFlags>?,
    outputHandler: @escaping VTDecompressionOutputHandler
) -> OSStatus

还有一种方法可以将self传递给回调函数,但还没有真正研究过。

相关问题