iPhone back camera cannot focus correctly

I've been making an iOS camera app and trying to solve this problem for two days (but cannot solve this).

What I'm working on now is change the focus and exposure automatically depending on the user's tapped location. Sometimes it works fine (maybe about 20% in total), but mostly it fails. Especially when I try to focus on a far object (like 5+metre) or when there are two objects and try to switch the focus of one object to another. The image below is an example.

The yellow square locates where the user tapped and even though I tapped the black cup in the first picture, the camera still focuses on the red cup.

    override func touchesBegan(_ touches: Set<UITouch>, with event: UIEvent?) {
        let touchPoint = touches.first! as UITouch
        let focusPoint = touchPoint.location(in: lfView)
        print("focusPoint \(focusPoint)")
    
        showPointOfInterestViewAtPoint(point: focusPoint)
        setFocus(focusMode: .autoFocus, exposureMode: .autoExpose, atPoint: focusPoint, shouldMonitorSujectAreaChange: true)
    }

    func setFocus(focusMode: AVCaptureDevice.FocusMode, exposureMode: AVCaptureDevice.ExposureMode, atPoint devicePoint: CGPoint, shouldMonitorSujectAreaChange: Bool) {
    
        guard let captureDevice = captureDevice else { return }
    
        do {
            try captureDevice.lockForConfiguration()
        } catch let error as NSError { return }
    
        if captureDevice.isFocusPointOfInterestSupported, captureDevice.isFocusModeSupported(focusMode) {
            captureDevice.focusPointOfInterest = devicePoint
            captureDevice.focusMode = focusMode
            print("devicePoint: \(devicePoint)")
        
        }
    
        // other codes in here...
    
        captureDevice.isSubjectAreaChangeMonitoringEnabled = shouldMonitorSujectAreaChange
        captureDevice.unlockForConfiguration()
    }

I called the setFocus function in touchesBegan function and both focusPoint & devicePoint comments show the same coordinate, like (297.5, 88.0).

When I tapped the black cup in the picture, I can see the iPhone camera is zooming in and out a little bit, like same as when I use the default iPhone camera app and try to focus on an object. So I guess my camera app is trying to focus on the black cup but it fails.

Since this is not an error, I'm not sure which code to change. Is there any clue what is going on here and what causes this problem?

ADD THIS PART LATER

I also read this document and it says

This property’s CGPoint value uses a coordinate system where {0,0} is the top-left of the picture area and {1,1} is the bottom-right.

As I wrote before, the value of devicePoint gives me more than 1, like 297.5, 88.0. Does this cause the problem?

Thanks to @Artem I was able to solve the problem. All I needed to do was convert the absolute coordinate to the value used in focusPointOfInterest (min (0,0) to max (1,1)).

Thank you, Artem!!