swiftmacosnstextviewmousedownnslayoutmanager

NSLayoutManager returns incorrect glyphIndex for mousePosition in NSTextView

I have an NSTextView subclass as the right item in an NSSplitViewController, the left panel is an NSOutlineView. To process mouse clicks while the command key is pressed in the text view, I added the following to find the glyph that is under the mouse: 

override func mouseDown(with event: NSEvent) {
    guard
        let lm = self.layoutManager,
        let tc = self.textContainer
    else { return }

    let localMousePosition = convert(event.locationInWindow, to: nil)
    var partial = CGFloat(1.0)
    let glyphIndex = lm.glyphIndex(for: localMousePosition, in: tc, fractionOfDistanceThroughGlyph: &partial)

    print(glyphIndex)
}

However, this results in an index that is about 10 or so too high, thus selecting the wrong glyph. My text view has only monospaced characters, so the offset is not caused by additional glyphs.

Interestingly, if I collapse the left panel (in code or in the storyboard), I get the correct index. But the offset in the x direction is more than the width of the left panel.

What am I missing here, is there an error in the code above?

Solution

  • Based on the comment by @Willeke above, I made the following change to my code:

    localMousePosition = convert(event.locationInWindow, from: nil)

if enclosingScrollView?.frame.width == window?.frame.width {
    // left panel is collapsed, so use convert: to:
    localMousePosition = convert(event.locationInWindow, to: nil)
}

    Seems to work.

    The problem was that I was using localMousePosition = convert(event.locationInWindow, to: nil) in mouseMoved as well (localMousePosition is a property of the view). Changing that to localMousePosition = convert(event.locationInWindow, from: nil) made it all work. Thanks again to @Willeke for pointing this out.