javaperformancereal-timelow-latencyproject-panama

Is it possible to copy from native memory to a ByteBuffer using the new FFM API without producing temp objects?


We have an open-source project using sun.misc.Unsafe that we are migrating to the new Foreign Function & Memory (FFM) API. For our particular use-case (low-latency trading systems) it is important not to create any garbage (temp discarded instances) when receiving a message as these systems handle hundreds of thousands of messages per second.

We encountered an issue when copying from native memory straight into a direct ByteBuffer using the FFM API.

Using sun.misc.Unsafe, copying from memory to a ByteBuffer can be done without creating temporary objects and garbage collector overhead. However, with the FFM API, achieving the same does not seem currently possible without generating garbage via a call to MemorySegment.ofBuffer.

Is there a way to currently do that with FFM API that we are unaware of?

Below the current code we have with sun.misc.Unsafe, which does not produce any garbage, and the only way we are aware of doing it with the FFM API, which produces garbage.

// With sun.misc.Unsafe:
@Override
public void getByteBuffer(long address, ByteBuffer dst, int len) {
    if (!dst.isDirect()) {
        throw new RuntimeException("getByteBuffer can only take a direct byte buffer!");
    }
    try {
        long dstAddress = (long) addressField.get(dst); // get the memory address of this ByteBuffer
        dstAddress += dst.position(); // adjust the address for the ByteBuffer current position

        unsafe.copyMemory(address, dstAddress, len); // copy without temp objects

        dst.position(dst.position() + len); // adjust the ByteBuffer position to reflect the copy operation
    } catch(Exception e) {
        throw new RuntimeException(e);
    }
}
// With FFM API:
@Override
public void getByteBuffer(long address, ByteBuffer dst, int len) {
    if (!dst.isDirect()) {
        throw new RuntimeException("getByteBuffer can only take a direct byte buffer!");
    }

    long offset = address - this.address; // offset in our 'segment'

    // Wrap the destination ByteBuffer in a temporary segment
    // This segment's data starts at dst.position()
    MemorySegment dstSegment = MemorySegment.ofBuffer(dst); // <====== temp object created here

    // Copy from 'segment' at 'offset' → dstSegment at offset 0 → length = 'len'
    dstSegment.copyFrom(this.segment, offset, 0, len);
    
    dst.position(dst.position() + len);
}

Note that with the sun.misc.Unsafe version we’re not really using the ByteBuffer itself but its underlying native memory address. Perhaps I’m overlooking something, but it seems that the FFM API should offer a way to copy data directly between native memory regions (the classic C memcpy)


Solution

  • I think this should satisfy your requirements. The culprit is to create a constant MemorySegment that acts as the entirety of the memory, then you can operate on that segment as if you are working with raw memory:

    // With FFM API:
    static final MemorySegment EVERYTHING_SEGMENT = MemorySegment.NULL.reinterpret(Long.MAX_VALUE);
    
    @Override
    public void getByteBuffer(long address, ByteBuffer dst, int len) {
        if (!dst.isDirect()) {
            throw new RuntimeException("getByteBuffer can only take a direct byte buffer!");
        }
    
        long dstAddress = (long) addressField.get(dst) + dst.position();
        EVERYTHING_SEGMENT.copyFrom(this.segment, offset, dstAddress, len);
        dst.position(dst.position() + len);
    }
    

    Furthermore, if it is possible, I would suggest moving away from the ByteBuffer API, too.