我已经实现了类似于Apple's example的解决方案,用iPhone激光雷达摄像头捕获深度数据。主要代码片段如下:
1.设置深度格式
let device = AVCaptureDevice.default(.builtInLiDARDepthCamera, for: .video, position: .back)!
let vidFormatsWithDepth = device.formats.filter { format in
format.formatDescription.dimensions.width == 1920 &&
format.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_420YpCbCr8BiPlanarFullRange &&
!format.isVideoBinned &&
!format.supportedDepthDataFormats.isEmpty &&
format.supportedDepthDataFormats.contains { $0.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_DepthFloat16 }
}
if let format = vidFormatsWithDepth.first {
let depthFormats = format.supportedDepthDataFormats.filter { $0.formatDescription.mediaSubType.rawValue == kCVPixelFormatType_DepthFloat16 }
try! device.lockForConfiguration()
device.activeFormat = format
device.activeDepthDataFormat = depthFormats.last
device.unlockForConfiguration()
}
1.照片输出
func setUpPhotoOutput() {
photoOutput = AVCapturePhotoOutput()
photoOutput.maxPhotoQualityPrioritization = .quality
self.captureSession.addOutput(photoOutput)
photoOutput.isDepthDataDeliveryEnabled = photoOutput.isDepthDataDeliverySupported
}
1.拍摄照片
var format: [String: Any] = [:]
if photoOutput.availablePhotoPixelFormatTypes.contains(kCVPixelFormatType_420YpCbCr8BiPlanarFullRange) {
format[kCVPixelBufferPixelFormatTypeKey as String] = kCVPixelFormatType_420YpCbCr8BiPlanarFullRange
}
let settings = AVCapturePhotoSettings(format: format)
settings.isDepthDataDeliveryEnabled = photoOutput.isDepthDataDeliveryEnabled
settings.isDepthDataFiltered = photoOutput.isDepthDataDeliveryEnabled
settings.embedsDepthDataInPhoto = photoOutput.isDepthDataDeliveryEnabled
photoOutput.capturePhoto(with: settings , delegate: self)
1.处理捕获的照片数据
func createPhotoFile(
photo: AVCapturePhoto
) {
let customizer = PhotoDataCustomizer()
var mainImageData = photo.fileDataRepresentation(with: customizer)!
// note mainImageData should have embeded depth data, but...
let imageSource = CGImageSourceCreateWithData(mainImageData as CFData, nil)!
let depthDataDict = CGImageSourceCopyAuxiliaryDataInfoAtIndex(
imageSource,
0,
kCGImageAuxiliaryDataTypeDepth
)
let disparityDataDict = CGImageSourceCopyAuxiliaryDataInfoAtIndex(
imageSource,
0,
kCGImageAuxiliaryDataTypeDisparity
)
print("depthDataDict", depthDataDict ?? "nil")
print("disparityDataDict", disparityDataDict ?? "nil")
// ... both depthDataDict and disparityDataDict come out as nil
}
class PhotoDataCustomizer: NSObject, AVCapturePhotoFileDataRepresentationCustomizer {
func replacementDepthData(for photo: AVCapturePhoto) -> AVDepthData? {
let depthData = photo.depthData?.converting(toDepthDataType: kCVPixelFormatType_DepthFloat16)
return depthData
}
}
AVCapturePhoto的photo.depthData是存在的(不是nil),我希望它是嵌入的,如果settings.embedsDepthDataInPhoto = true
,但两个深度数据变体(kCGImageAuxiliaryDataTypeDepth,kCGImageAuxiliaryDataTypeDisparity)从CGImageSource中出来为nil。
如何正确读取照片文件中的深度数据...还是首先正确地写入深度数据?
1条答案
按热度按时间wnrlj8wa1#
我能够通过手动添加它来保存深度数据,绕过
var mainImageData = photo.fileDataRepresentation(with: customizer)!
,如下所示: