Currently iOS does not support markers or targets in a similar fashion to Vuforia (at the moment of this writing iOS 11.3 has a similar feature but it’s not released). So what is the best way to drop an animation on a target? Assuming you’re dealing with a rectangular object we can use the Vision framework’s VNDetectRectanglesRequest
// viewController let pixelBuffer = self.sceneView.session.currentFrame?.capturedImage let ciImage = CIImage(cvImageBuffer: pixelBuffer!) let handler = VNImageRequestHandler(ciImage: ciImage) let rectService = RectangleDetectionService(sceneView: self.sceneView) let rectangleRequest = VNDetectRectanglesRequest(completionHandler: rectService.handleRectangles) do { try handler.perform([rectangleRequest]) } catch { log.error(error) }
Above we’ve transformed the pixel buffer being received from ARKit (sceneView
) into a ciImage
that we then pass to the Vision Framework’s request handler. Then we create a new instance of our custom
// RectangleDetectionService (rectService) func handleRectangles(request: VNRequest, error: Error?) { guard let observations = request.results as? [VNRectangleObservation] else { return } // get the highest confidence rectangle observation let highConfidenceObservation = observations.max { a, b in a.confidence < b.confidence } guard let highestConfidenceObservation = highConfidenceObservation else { return } // Calculates the position of the 4 corners let points = [highConfidenceObservation?.topLeft, highConfidenceObservation?.topRight, highConfidenceObservation?.bottomRight, highConfidenceObservation?.bottomLeft] // Calculates center point let center = CGPoint(x: (highConfidenceObservation?.boundingBox.midX)!, y: (highConfidenceObservation?.boundingBox.midY)!) // create a hit test simulating a user touch at the point within the sceneview let hitTestResults = self.sceneView.hitTest(center, types: [.existingPlaneUsingExtent, .featurePoint]) guard let result = hitTestResults.first else { print("Rectangle detection error") return } // now that we've calculated a center point and created a hit test in the arView anchor = ARAnchor(transform: result.worldTransform) self.sceneView.session.add(anchor: anchor) }
You should now have an ARAnchor
at the center point of the detected rectangle