如何使用不同分辨率的视频导出拼贴视频?我试图实现像下面显示的第一张图片,我使用AVCustomEdit演示,并已做到这一点,我创建了AVMutableVideoComposition
,将所有视频trackID传递到customVideoCompositorClass
,并获取所有视频CVPixelBuffer
,然后在MTLTexture
中转换,然后渲染所有纹理,但问题是我的视频输出大小是方形的(destinationTexture)和视频大小是纵向或横向的,这就是为什么每个视频都被挤压,我怎么能旋转每个视频的缩放位置和遮罩形状?如何使用cifilters?我应该将每个CVPixelBuffer
转换为ciImage,然后将ciImage转换回CVPixelBuffer
吗?
override func renderPixelBuffer(backgroundTexture: MTLTexture,
firstPixelBuffer: CVPixelBuffer,
secondPixelBuffer: CVPixelBuffer,
thirdPixelBuffer: CVPixelBuffer,
fourthPixelBuffer: CVPixelBuffer,
destinationPixelBuffer: CVPixelBuffer) {
// Create a MTLTexture from the CVPixelBuffer.
guard let firstTexture = buildTextureForPixelBuffer(firstPixelBuffer) else { return }
guard let secondTexture = buildTextureForPixelBuffer(secondPixelBuffer) else { return }
guard let thirdTexture = buildTextureForPixelBuffer(thirdPixelBuffer) else { return }
guard let fourthTexture = buildTextureForPixelBuffer(fourthPixelBuffer) else { return }
guard let destinationTexture = buildTextureForPixelBuffer(destinationPixelBuffer) else { return }
/*
We must maintain a reference to the pixel buffer until the Metal rendering is complete. This is because the
'buildTextureForPixelBuffer' function above uses CVMetalTextureCacheCreateTextureFromImage to create a
Metal texture (CVMetalTexture) from the IOSurface that backs the CVPixelBuffer, but
CVMetalTextureCacheCreateTextureFromImage doesn't increment the use count of the IOSurface; only the
CVPixelBuffer, and the CVMTLTexture own this IOSurface. Therefore we must maintain a reference to either
the pixel buffer or Metal texture until the Metal rendering is done. The MTLCommandBuffer completion
handler below is then used to release these references.
*/
pixelBuffers = RenderPixelBuffers(firstBuffer: firstPixelBuffer,
secondBuffer: secondPixelBuffer,
thirdBuffer: thirdPixelBuffer,
fourthBuffer: fourthPixelBuffer,
destinationBuffer: destinationPixelBuffer)
// Create a new command buffer for each renderpass to the current drawable.
let commandBuffer = commandQueue.makeCommandBuffer()!
commandBuffer.label = "MyCommand"
/*
Obtain a drawable texture for this render pass and set up the renderpass
descriptor for the command encoder to render into.
*/
let renderPassDescriptor = setupRenderPassDescriptorForTexture(destinationTexture)
// Create a render command encoder so we can render into something.
let renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
renderEncoder.label = "MyRenderEncoder"
guard let renderPipelineState = renderPipelineState else { return }
modelConstants.modelViewMatrix = matrix_identity_float4x4
// Render background texture.
renderTexture(renderEncoder, texture: backgroundTexture, pipelineState: renderPipelineState)
var translationMatrix = matrix_float4x4(translation: simd_float3(-0.5, 0.5, 0))
// var rotationMatrix = matrix_float4x4(rotationZ: radians(fromDegrees: -90))
var scaleMatrix = matrix_float4x4(scaling: 0.25)
var modelMatrix = translationMatrix * scaleMatrix
modelConstants.modelViewMatrix = modelMatrix
// Render first texture.
renderTexture(renderEncoder, texture: firstTexture, pipelineState: renderPipelineState)
// translationMatrix = matrix_float4x4(translation: simd_float3(0.5, -0.5, 0))
// rotationMatrix = matrix_float4x4(rotationZ: radians(fromDegrees: -45))
// scaleMatrix = matrix_float4x4(scaling: 0.5)
// modelMatrix = translationMatrix * scaleMatrix * rotationMatrix
// modelConstants.modelViewMatrix = modelMatrix
// // Render second texture.
// renderTexture(renderEncoder, texture: secondTexture, pipelineState: renderPipelineState)
//
// // Render third texture.
// renderTexture(renderEncoder, texture: thirdTexture, pipelineState: renderPipelineState)
//
// // Render fourth texture.
// renderTexture(renderEncoder, texture: fourthTexture, pipelineState: renderPipelineState)
// We're done encoding commands.
renderEncoder.endEncoding()
// Use the command buffer completion block to release the reference to the pixel buffers.
commandBuffer.addCompletedHandler({ _ in
self.pixelBuffers = nil // Release the reference to the pixel buffers.
})
// Finalize rendering here & push the command buffer to the GPU.
commandBuffer.commit()
}
2条答案
按热度按时间rvpgvaaj1#
我推荐使用一个叫做MetalPetal的库。这是一个基于Metal的图像处理框架。你必须将CVPixelBuffer转换为MetalImage,即MTIImage。然后你可以在图像中做任何事情,比如有预制的过滤器,你可以应用它,或者你可以使用甚至CIFilter或你的自定义过滤器,你可以变换,旋转,裁剪每一帧,以便拼贴帧是准确的。然后你必须再次将MTIimage转换为cvpixelbuffer。在这里你也可以CIImage,但它会很慢,我猜。你得到的盒子图像可能是渲染大小。请查看渲染大小。
ia2d9nvy2#
我推荐使用Harbeth库,这是一个基于Metal的图像处理框架。您可以直接添加过滤器到CVPixelBuffer,并最终导出视频。