Currently iOS does not support markers or targets in a similar fashion to Vuforia (at the moment of this writing iOS 11.3 has a similar feature but it’s not released). So what is the best way to drop an animation on a target? Assuming you’re dealing with a rectangular object we can use the Vision framework’s VNDetectRectanglesRequest
// viewController
let pixelBuffer = self.sceneView.session.currentFrame?.capturedImage
let ciImage = CIImage(cvImageBuffer: pixelBuffer!)
let handler = VNImageRequestHandler(ciImage: ciImage)
let rectService = RectangleDetectionService(sceneView: self.sceneView)
let rectangleRequest = VNDetectRectanglesRequest(completionHandler: rectService.handleRectangles)
do {
try handler.perform([rectangleRequest])
} catch {
log.error(error)
}
Above we’ve transformed the pixel buffer being received from ARKit (sceneView) into a ciImage that we then pass to the Vision Framework’s request handler. Then we create a new instance of our custom
// RectangleDetectionService (rectService)
func handleRectangles(request: VNRequest, error: Error?) {
guard let observations = request.results as? [VNRectangleObservation] else {
return
}
// get the highest confidence rectangle observation
let highConfidenceObservation = observations.max { a, b in a.confidence < b.confidence }
guard let highestConfidenceObservation = highConfidenceObservation else {
return
}
// Calculates the position of the 4 corners
let points = [highConfidenceObservation?.topLeft,
highConfidenceObservation?.topRight,
highConfidenceObservation?.bottomRight,
highConfidenceObservation?.bottomLeft]
// Calculates center point
let center = CGPoint(x: (highConfidenceObservation?.boundingBox.midX)!,
y: (highConfidenceObservation?.boundingBox.midY)!)
// create a hit test simulating a user touch at the point within the sceneview
let hitTestResults = self.sceneView.hitTest(center, types: [.existingPlaneUsingExtent, .featurePoint])
guard let result = hitTestResults.first else {
print("Rectangle detection error")
return
}
// now that we've calculated a center point and created a hit test in the arView
anchor = ARAnchor(transform: result.worldTransform)
self.sceneView.session.add(anchor: anchor)
}
You should now have an ARAnchor at the center point of the detected rectangle



