I have an Android app that reads images to external OES texture, binds a frame buffer with a texture attachment and renders to it to convert it to a normal OpenGL texture.
The buffer is then unbound and rendered again to render to the screen. This process uses GLSurfaceView and works well.
I then want to pass the regular OpenGL texture to a native library for post processing (specifically it would pass to TFLite GPU delegate which supports running on OpenGL textures)
In order for the texture to be accessible in the native library, when I create the library I pass the current OpenGL context as a share context to the function that creates the context in the native library (it's an standalone library, which can run on many inputs and it runs its own threading mechanism so it would run on a different thread than the GLSurfaceView thread)
If I add the code that creates initialize the native library, then the rendered texture is all black, and when trying to unbind to draw on the screen the call to GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER) return GL_FRAMEBUFFER_UNDEFINED
The context seems to still be OK (calling eglGetCurrentContext() returns the same value as before the intiailization of the native library)
My thinking is that somehow when sharing the context it messes up the buffers of the original context, even though they're created after the native library is created
Here is the relevant codes
MainActivity.kt
override fun onCreate(savedInstanceState: Bundle?) {
super.onCreate(savedInstanceState)
supportActionBar?.hide()
surfaceView = GLSurfaceView(this)
surfaceView.setPreserveEGLContextOnPause(true);
surfaceView.setEGLContextClientVersion(3);
surfaceView.setEGLConfigChooser(8, 8, 8, 8, 16, 0);
surfaceView.setRenderer(this);
surfaceView.setRenderMode(GLSurfaceView.RENDERMODE_CONTINUOUSLY);
getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN, WindowManager.LayoutParams.FLAG_FULLSCREEN);
setContentView(surfaceView);
}
override fun onSurfaceCreated(p0: GL10?, p1: EGLConfig?) {
val glContext: Long = eglGetCurrentContext().nativeHandle
// Initialize the library and share the context, before generating any buffers
// If I comment this out, everything works well
library = NativeLib.initialize(glContext)
val textures = IntArray(1)
GLES20.glGenTextures(1, textures, 0)
cameraTextureId = textures[0]
val textureTarget = GLES11Ext.GL_TEXTURE_EXTERNAL_OES
GLES20.glBindTexture(textureTarget, cameraTextureId)
GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE)
GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE)
GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR)
GLES20.glTexParameteri(textureTarget, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR)
surfaceTexture = SurfaceTexture(cameraTextureId)
...
// Initialize shaders and attributes
// create glTexture
...
GLES20.glDisable(GLES20.GL_DEPTH_TEST)
GLES20.glDisable(GLES20.GL_CULL_FACE)
val values = IntArray(1)
GLES20.glGenFramebuffers(1, values, 0)
frameBuffer = values[0]
}
override fun onDrawFrame(p0: GL10?) {
bindFramebuffer(glTexture)
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
GLES20.glActiveTexture(GLES20.GL_TEXTURE0);
surfaceTexture?.updateTexImage()
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR);
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE);
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE);
....
// setup coordinates and attribute pointers
...
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
unbindFrameBuffer()
// Now draw to screen
GLES20.glDisable(GLES20.GL_DEPTH_TEST);
GLES20.glDepthMask(false);
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4);
ShaderUtil.checkGLError(TAG, "glDrawArrays");
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, 0);
}
private fun _bindFrameBuffer(frameBuffer_: Int, texture: Int?, width: Int, height: Int)
{
GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, frameBuffer_)
texture?.apply {
GLES20.glFramebufferTexture2D(
GLES20.GL_FRAMEBUFFER,
GLES20.GL_COLOR_ATTACHMENT0,
GLES20.GL_TEXTURE_2D,
this,
0
)
}
val status = GLES20.glCheckFramebufferStatus(GLES20.GL_FRAMEBUFFER)
if (status != GLES20.GL_FRAMEBUFFER_COMPLETE) {
throw java.lang.RuntimeException("Framebuffer not complete, status=$status")
}
GLES20.glViewport(0, 0, width, height)
}
fun bindFramebuffer(texture: Int) {
_bindFrameBuffer(frameBuffer, texture, targetSurfaceWidth, targetSurfaceHeight)
}
fun unbindFrameBuffer() {
_bindFrameBuffer(0, null, screenSurfaceWidth, screenSurfaceHeight)
}
Native library
create_egl_context(EGLContext share_context)
{
m_egl_display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
eglInitialize(m_egl_display, nullptr, nullptr);
const EGLint config_attr[] = {
// clang-format off
EGL_RENDERABLE_TYPE, EGL_OPENGL_ES3_BIT_KHR,
// Allow rendering to pixel buffers or directly to windows.
EGL_SURFACE_TYPE,
EGL_PBUFFER_BIT | EGL_WINDOW_BIT,
EGL_RED_SIZE, 8,
EGL_GREEN_SIZE, 8,
EGL_BLUE_SIZE, 8,
EGL_ALPHA_SIZE, 8, // if you need the alpha channel
EGL_DEPTH_SIZE, 16, // if you need the depth buffer
EGL_NONE
// clang-format on
};
EGLint w, h, format;
EGLint numConfigs;
EGLConfig config = nullptr;
/* Here, the application chooses the configuration it desires.
* find the best match if possible, otherwise use the very first one
*/
eglChooseConfig(m_egl_display, config_attr, &config,1, &numConfigs);
assert(numConfigs);
if (config == nullptr) {
std::cerr << "Failed getting config" << std::endl;
return EGL_NO_CONTEXT;
}
const EGLint context_attr[] = {
// clang-format off
EGL_CONTEXT_CLIENT_VERSION, 3,
EGL_NONE
// clang-format on
};
auto egl_context = eglCreateContext(m_egl_display, config, share_context, context_attr);
eglMakeCurrent(m_egl_display, EGL_NO_SURFACE, EGL_NO_SURFACE, egl_context);
return egl_context;
}
Each part independently works well (I have unit tests for the library which loads an image using OpenCV to an OpenGL texture and then runs the pipeline), but when combining them together they fail
The issue was that creating the NativeLib on the same GLThread caused the current thread's context to be changed to the new shared context.
2 possible solutions that I can see
Run the native lib creation in a separate thread
...
thread(start = true) {
library = NativeLib.initialize(glContext)
}.join()
...
Or save the current thread's display and read/draw surfaces before creating the library and then setting it back after
val context = EGL14.eglGetCurrentContext()
val display = EGL14.eglGetCurrentDisplay()
val readSurface = EGL14.eglGetCurrentSurface(EGL14.EGL_READ)
val drawSurface = EGL14.eglGetCurrentSurface(EGL14.EGL_DRAW)
library = NativeLib.initialize(glContext)
// Restore current
EGL14.eglMakeCurrent(display, drawSurface, readSurface, context)
Both methods tested and work. I chose to go with the second 1 which seems cleaner in my opinion