I am attempting to test the Foreign Function and Memory features of Java 21. Here is my code :
public static void main(String[] args) {
// 1. Find foreign function on the C library path
Linker linker = Linker.nativeLinker();
SymbolLookup stdlib = linker.defaultLookup();
MethodHandle radixsort = linker.downcallHandle(stdlib.find("radixsort").orElseThrow(), FunctionDescriptor.ofVoid(ValueLayout.ADDRESS, ValueLayout.JAVA_INT, ValueLayout.ADDRESS, ValueLayout.JAVA_CHAR));
// 2. Allocate on-heap memory to store four strings
String[] javaStrings = {"mouse", "cat", "dog", "car"};
// 3. Use try-with-resources to manage the lifetime of off-heap memory
try (Arena offHeap = Arena.ofConfined()) {
// 4. Allocate a region of off-heap memory to store four pointers
MemorySegment pointers = offHeap.allocateArray(ValueLayout.ADDRESS, javaStrings.length);
// 5. Copy the strings from on-heap to off-heap
for (int i = 0; i < javaStrings.length; i++) {
MemorySegment cString = offHeap.allocateUtf8String(javaStrings[i]);
pointers.setAtIndex(ValueLayout.ADDRESS, i, cString);
assert cString.getUtf8String(0).length() > 0;
}
// 6. Sort the off-heap data by calling the foreign function
radixsort.invoke(pointers, javaStrings.length, MemorySegment.NULL, '\0');
// 7. Copy the (reordered) strings from off-heap to on-heap
for (int i = 0; i < javaStrings.length; i++) {
MemorySegment cString = pointers.getAtIndex(ValueLayout.ADDRESS, i);
javaStrings[i] = cString.getUtf8String(0);
}
} // 8. All off-heap memory is deallocated here
catch (Throwable e) {
throw new RuntimeException(e);
}
assert Arrays.equals(javaStrings, new String[]{"car", "c∞at", "dog", "mouse"}); // true
}
but I am encountering this error :
Exception in thread "main" java.lang.RuntimeException: java.lang.IndexOutOfBoundsException: Out of bound access on segment MemorySegment{ heapBase: Optional.empty address:105553153094096 limit: 0 }; new offset = 0; new length = 1
At the line :
javaStrings\[i\] = cString.getUtf8String(0);
It seems that MemorySegment cString = pointers.getAtIndex(ValueLayout.ADDRESS, i);
returns a MemorySegment with a byteSize = 0
When I use cString.getUtf8String(0);
on the original cString, I do not encounter an error.
It seems that pointers.setAtIndex
and pointers.getAtIndex
do not preserve the MemorySegment byteSize. ?
It seems that pointers.setAtIndex and pointers.getAtIndex do not preserve the MemorySegment byteSize. ?
Yes, that is correct. The address is stored into off-heap memory as a plain memory address. The size is dropped. So, when reading an address back from off-heap memory, the created memory segment has size 0.
This is explained in the javadoc of MemorySegment as well:
When interacting with foreign functions, it is common for those functions to allocate a region of memory and return a pointer to that region. Modeling the region of memory with a memory segment is challenging because the Java runtime has no insight into the size of the region. Only the address of the start of the region, stored in the pointer, is available.
...
The MemorySegment API uses zero-length memory segments to represent:
- pointers returned from a foreign function;
- pointers passed by a foreign function to an upcall stub; and
- pointers read from a memory segment (more on that below).
Your use case is the latter of the three.
You have to use MemorySegment::reinterpret
to resize the segment explicitly. Since for a C string the actual size is not known (it's indicated by a null terminator), you can resize the segment to Long.MAX_VALUE
to make it accessible:
// 7. Copy the (reordered) strings from off-heap to on-heap
for (int i = 0; i < javaStrings.length; i++) {
MemorySegment cString = pointers.getAtIndex(ValueLayout.ADDRESS, i);
cString = cString.reinterpret(Long.MAX_VALUE); // <----
javaStrings[i] = cString.getUtf8String(0);
}
Alternatively, you can create an AddressLayout
with a target layout that has the needed size, to avoid the separate call to reinterpret
:
AddressLayout UNBOUNDED_POINTER = ValueLayout.ADDRESS
.withTargetLayout(MemoryLayout.sequenceLayout(Long.MAX_VALUE, ValueLayout.JAVA_BYTE));
And then use that to do the dereference:
// 7. Copy the (reordered) strings from off-heap to on-heap
for (int i = 0; i < javaStrings.length; i++) {
MemorySegment cString = pointers.getAtIndex(UNBOUNDED_POINTER, i);
// no reinterpret needed
javaStrings[i] = cString.getUtf8String(0);
}