androidreact-nativesurfaceviewvulkan

SurfaceView inside of React Native app makes everything blank


I am creating a React Native Android application, and I want to embed a control which is rendered using Vulkan. I implemented this control by creating a subclass of SurfaceView called VkSurfaceView and implementing the Vulkan code in Rust with JNI calls.

I turned on the debug setting that shows the boundaries of all of the views so that it's clear that the views are still there.

Here's the Vulkan control (currently, it just renders a triangle) inside of a simple Android layout (no React Native):

enter image description here

Here's my React Native layout without the Vulkan control (the square represents where I am trying to put the Vulkan control):

enter image description here

But when I put the Vulkan control, then everything else goes dark, even though the layout inspector and the debug overlay indicate that the other views are still there:

enter image description here

Why might this be happening? As shown above, the Vulkan control works inside of pure-Android layouts, so I think that it must be interacting unfavorably with React Native in some way. However, I have found examples like this and this that are able to use SurfaceView/TextureView/VideoView in React Native without it causing problems.

Edit: Even if I comment out all of the code in VkSurfaceView, the black screen issue still occurs (but the triangle disappears, obviously), so I don't think this is happening because of an issue with my Vulkan code.

Edit 2: It works if I derive from TextureView instead of SurfaceView. But why? I suspect that it has something to do with the differences explained here, but I still don't understand why it's important whether the view is composited using OpenGL or not.

Here's my code:

React Native index.js:

import React from 'react';
import {
  AppRegistry,
  StyleSheet,
  Text,
  View
} from 'react-native';
import VkSurfaceView from './surface-view';

const HelloWorld = () => {
  return (
    <View style={[styles.container, {
      // Try setting `flexDirection` to `"row"`.
      flexDirection: "column"
    }]}>
      <Text style={styles.hello}>Hello, World</Text>
      <View style={{ width: 100, height: 100, alignSelf: 'center', backgroundColor: 'pink' }}>
        <VkSurfaceView style={{ width: 100, height: 100 }}/>
      </View>
    </View>
  );
};
var styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    backgroundColor: '#ffffff'
  },
  hello: {
    fontSize: 20,
    textAlign: 'center',
    margin: 10,
    color: 'green'
  }
});

AppRegistry.registerComponent(
  'MyReactNativeApp',
  () => HelloWorld
);

React Native surface-view.js:

import { requireNativeComponent } from 'react-native';

module.exports = requireNativeComponent('SPVkSurfaceView');

Java SurfaceReactPackage.java:

public class SurfaceReactPackage implements ReactPackage {
    @NonNull
    @Override
    public List<NativeModule> createNativeModules(@NonNull ReactApplicationContext reactContext) {
        return Collections.emptyList();
    }

    @NonNull
    @Override
    public List<ViewManager> createViewManagers(
            @NonNull ReactApplicationContext reactContext) {
        return Collections.singletonList(
                new ReactSurfaceManager(reactContext)
        );
    }
}

Java ReactSurfaceManager.java:

public class ReactSurfaceManager extends SimpleViewManager<VkSurfaceView> {
    public static final String REACT_CLASS = "SPVkSurfaceView";
    ReactApplicationContext mCallerContext;

    public ReactSurfaceManager(ReactApplicationContext reactContext) {
        mCallerContext = reactContext;
    }

    @NonNull
    @Override
    public VkSurfaceView createViewInstance(ThemedReactContext context) {
        return new VkSurfaceView(context);
    }

    @Override
    public String getName() {
        return REACT_CLASS;
    }
}

Java MainActivity.java

public class MainActivity extends AppCompatActivity implements DefaultHardwareBackBtnHandler {
    private ReactRootView mReactRootView;
    private ReactInstanceManager mReactInstanceManager;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        // initialize native code library before using VkSurfaceView
        MyNativeLibary.init();

        // initialize React Native
        mReactRootView = new ReactRootView(this);
        List<ReactPackage> packages = new PackageList(getApplication()).getPackages();
        packages.add(new SurfaceReactPackage());

        mReactInstanceManager = ReactInstanceManager.builder()
                .setApplication(getApplication())
                .setCurrentActivity(this)
                .setBundleAssetName("index.android.bundle")
                .setJSMainModulePath("index")
                .addPackages(packages)
                .setUseDeveloperSupport(BuildConfig.DEBUG)
                .setInitialLifecycleState(LifecycleState.RESUMED)
                .build();
        // The string here (e.g. "MyReactNativeApp") has to match
        // the string in AppRegistry.registerComponent() in index.js
        mReactRootView.startReactApplication(mReactInstanceManager, "MyReactNativeApp", null);

        setContentView(mReactRootView);
    }

    @Override
    protected void onPause() {
        super.onPause();

        if (mReactInstanceManager != null) {
            mReactInstanceManager.onHostPause(this);
        }
    }

    @Override
    protected void onResume() {
        super.onResume();

        if (mReactInstanceManager != null) {
            mReactInstanceManager.onHostResume(this, this);
        }
    }

    @Override
    protected void onDestroy() {
        super.onDestroy();

        if (mReactInstanceManager != null) {
            mReactInstanceManager.onHostDestroy(this);
        }
        if (mReactRootView != null) {
            mReactRootView.unmountReactApplication();
        }
    }

    @Override
    public void invokeDefaultOnBackPressed() {
        this.onBackPressed();
    }

    @Override
    public void onBackPressed() {
        if (mReactInstanceManager != null) {
            mReactInstanceManager.onBackPressed();
        } else {
            super.onBackPressed();
        }
    }
}

Solution

  • One solution was to use TextureView instead of SurfaceView. However, this didn't seem to work too well, as I would get timeout issues related to frame buffering every few seconds. The solution that I actually ended up using was to wrap my SurfaceView inside of a FrameLayout.