Swift - UIPanGestureRecognizer selecting all layers when only want the top layer selected

I have function which creates a drag line to connect 2 buttons to each other. This works fine but if some buttons overlap each other, it will select both if I drag over where they overlap. I only want to connect the top button.

I think the issue is with the sender.location selecting layers on top and below. Is there a way to tell the sender.location to only select the top view? Thanks for any input and direction

func addPanReconiser(view: UIView){

    let pan = UIPanGestureRecognizer(target: self, action: #selector(DesignViewController.panGestureCalled(_:)))
    view.addGestureRecognizer(pan)
}

@objc func panGestureCalled(_ sender: UIPanGestureRecognizer) {

    let currentPanPoint = sender.location(in: self.view)

    switch sender.state {
    case .began:

        panGestureStartPoint = currentPanPoint
        self.view.layer.addSublayer(lineShape)

    case .changed:
        let linePath = UIBezierPath()
        linePath.move(to: panGestureStartPoint)
        linePath.addLine(to: currentPanPoint)

        lineShape.path = linePath.cgPath
        lineShape.path = CGPath.barbell(from: panGestureStartPoint, to: currentPanPoint, barThickness: 2.0, bellRadius: 6.0)

        for button in buttonArray {
            let point = sender.location(in: button)

            if button.layer.contains(point) {
                button.layer.borderWidth = 4
                button.layer.borderColor = UIColor.blue.cgColor
            } else {
                button.layer.borderWidth = 0
                button.layer.borderColor = UIColor.clear.cgColor
            }
        }

    case .ended:

        for button in buttonArray {
            let point = sender.location(in: button)

            if button.layer.contains(point){

                //DO my Action here
                lineShape.path = nil
                lineShape.removeFromSuperlayer()

            }
        }
    default: break
    }
  }
}

Note: some of the lines of codes are from custom extensions. I kept them in as they were self explanatory.

Thanks for the help

There is a way to walk around. It seems like you simply want your gesture end up at one button above all the others, thus by adding a var outside the loop and each time a button picked, comparing with the var of its level at z.

case .ended:
        var pickedButton: UIButton?
        for button in buttonArray {
            let point = sender.location(in: button)

            if button.layer.contains(point){
                if pickedButton == nil {
                    pickedButton = button
                } else {
                    if let parent = button.superView, parent.subviews.firstIndex(of: button) > parent.subviews.firstIndex(of: pickedButton!) {
                        pickedButton = button
                    }
                }
            }
        }
        //DO my Action with pickedButton here
        lineShape.path = nil
        lineShape.removeFromSuperlayer()