xcode Swift Vision API人员细分未屏蔽人员

wi3ka0sx  于 2022-12-24  发布在  Swift
关注(0)|答案(1)|浏览(198)

我一直在尝试学习苹果的Vision API来分割照片中的人物。我遇到的问题是背景蒙版图像完全取代了我的“Selfie”图像。Selfie图像清晰而不模糊,所以我认为图像质量不是问题所在。这是我在这段视频中看到的相同实现:https://developer.apple.com/videos/play/wwdc2021/10040/
有人能检查一下我的实现吗?如果我错过了什么,可以告诉我吗?我希望这个“空间”图像能取代我自拍的背景。
导入UIKit导入Vision导入核心图像导入核心图像.CIFilterBuiltins

class ImageSegmentationVC: UIViewController {
    
    let backgroundImageView: UIImageView = {
        let imageView = UIImageView(image: UIImage(named: "Selfie"))
        imageView.translatesAutoresizingMaskIntoConstraints = false
        imageView.contentMode = .scaleAspectFit
        return imageView
    }()
    
    // The Vision PersonSegmentation requests 
    private var segmentationRequest = VNGeneratePersonSegmentationRequest()
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        view.addSubview(backgroundImageView)
        backgroundImageView.topAnchor.constraint(equalTo: view.topAnchor).isActive = true
        backgroundImageView.leadingAnchor.constraint(equalTo: view.leadingAnchor).isActive = true
        backgroundImageView.trailingAnchor.constraint(equalTo: view.trailingAnchor).isActive = true
        backgroundImageView.bottomAnchor.constraint(equalTo: view.bottomAnchor).isActive = true
        
        guard let backgroundImage = backgroundImageView.image else { return }
        intializeRequests(for: backgroundImage)
        
    }
    
    private func intializeRequests(for image: UIImage) {
        // Update segmentation properties
        segmentationRequest.qualityLevel = .accurate
        segmentationRequest.outputPixelFormat = kCVPixelFormatType_OneComponent8
        
        if let image = image.cgImage {
            generatePhoto(backgroundImage: image)
        }
    }
    
    func generatePhoto(backgroundImage: CGImage) {
        ///2 Create Request Handler
        let requestHandler = VNImageRequestHandler(cgImage: backgroundImage, options: [:])
        
        try? requestHandler.perform([segmentationRequest])
        
        guard let maskPixelBuffer = segmentationRequest.results?.first?.pixelBuffer else {
            return
        }
        
        let maskImage = CGImage.create(pixelBuffer: maskPixelBuffer)
        applyingMask(buffer: maskPixelBuffer)
    }
    
    func applyingMask(buffer: CVPixelBuffer) {
    let input = CIImage(cgImage: backgroundImageView.image!.cgImage!)
    let mask = CIImage(cvPixelBuffer: buffer)
    let background = CIImage(image: UIImage(named: "starfield")!)!
    
    let maskScaleX = input.extent.width / mask.extent.width
    let maskScaleY = input.extent.height / mask.extent.height
    let maskScaled = mask.transformed(by: __CGAffineTransformMake(maskScaleX, 0, 0, maskScaleY, 0, 0))
    
    let backgroundScaleX = input.extent.width / background.extent.width
    let backgroundScaleY = input.extent.height / background.extent.height
    let backgroundScaled = background.transformed(by: __CGAffineTransformMake(backgroundScaleX, 0, 0, backgroundScaleY, 0, 0))
    
    let blendFilter = CIFilter.blendWithRedMask()
    blendFilter.inputImage = input
    blendFilter.backgroundImage = backgroundScaled
    blendFilter.maskImage = maskScaled
    
    let blendedImage = blendFilter.outputImage
    
    backgroundImageView.image = UIImage(ciImage: blendedImage!)
}

这是它当前的样子

euoag5mw

euoag5mw1#

我上面的解决方案是正确的。它不起作用的原因是因为我在模拟器上运行。但在实际设备上测试时,它起作用了。不知道为什么模拟器不起作用。目前运行的是英特尔2018 MacBook Pro。

相关问题