I have a React Native project. this app should captures the user screen and activities in the foreground. React Native cannot do this thing so I used Native Modules to write java. I made two java file which are ScreenCaptureModule.java and ScreenCaptureService.java. in this app I reach to: when the user opens the app he will see a Start Screen Capture button. when the user press on this button it will show a Dialog box like this:
AlertDialog.Builder builder = new AlertDialog.Builder(currentActivity);
builder.setTitle("Welcome, user!")
.setMessage("You are about to use my app, which captures your screen every one second. Are you willing to proceed?")
.setPositiveButton("Yes", new DialogInterface.OnClickListener() {
@Override
public void onClick(DialogInterface dialog, int which) {
Intent serviceIntent = new Intent(currentActivity, ScreenCaptureService.class);
currentActivity.startService(serviceIntent);
currentActivity.finish();
}
})
.setNegativeButton("No", new DialogInterface.OnClickListener() {
@Override
public void onClick(DialogInterface dialog, int which) {
// User declined, do nothing
}
})
.show();
if the user press on Yes the app will close and it will show a Notification like this:
private void showNotification() {
NotificationManager notificationManager = (NotificationManager) getSystemService(Context.NOTIFICATION_SERVICE);
if (notificationManager == null) {
return;
}
String channelId = "screen_capture_channel";
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
NotificationChannel channel = new NotificationChannel(channelId, "Screen Capture", NotificationManager.IMPORTANCE_DEFAULT);
notificationManager.createNotificationChannel(channel);
}
NotificationCompat.Builder builder = new NotificationCompat.Builder(this, channelId)
.setContentTitle("Screen Capture")
.setContentText("Your screen is currently being captured. You can find the captures in the Pictures Directory.")
.setSmallIcon(R.drawable.ic_notification)
.setPriority(NotificationCompat.PRIORITY_DEFAULT)
.setAutoCancel(true);
notificationManager.notify(1, builder.build());
}
after the Notification shown it should capture the screen every one second and save it in the Pictures Directory. it shows the Notification but it doesn't save it in Pictures Directory. this is the problem. and now
I'll give you the code of ScreenCapture.java , ScreenCaptureService.java and React Native to see How I call the methods. Again: the problem is when the app starting in the foreground it doesn't capture or save the image in the Pictures directory. it only shows a notification.
public class ScreenCaptureModule extends ReactContextBaseJavaModule {
private static final int REQUEST_MEDIA_PROJECTION = 1;
private final MediaProjectionManager mediaProjectionManager;
private ImageReader imageReader;
private int screenshotCounter = 1;
private final ReactApplicationContext reactContext;
public ScreenCaptureModule(ReactApplicationContext reactContext) {
super(reactContext);
this.reactContext = reactContext;
mediaProjectionManager = (MediaProjectionManager) reactContext.getSystemService(Context.MEDIA_PROJECTION_SERVICE);
reactContext.addActivityEventListener(activityEventListener);
}
@NonNull
@Override
public String getName() {
return "ScreenCaptureModule";
}
@ReactMethod
public void startScreenCapture() {
Activity currentActivity = getCurrentActivity();
if (currentActivity != null) {
// Here is the show of dialog box when click on start capture button
}
}
private final ActivityEventListener activityEventListener = new BaseActivityEventListener() {
@Override
public void onActivityResult(Activity activity, int requestCode, int resultCode, Intent data) {
if (requestCode == REQUEST_MEDIA_PROJECTION && resultCode == Activity.RESULT_OK) {
MediaProjection mediaProjection = mediaProjectionManager.getMediaProjection(resultCode, data);
DisplayMetrics metrics = new DisplayMetrics();
WindowManager windowManager = (WindowManager) getReactApplicationContext().getSystemService(Context.WINDOW_SERVICE);
windowManager.getDefaultDisplay().getMetrics(metrics);
@SuppressLint("WrongConstant") ImageReader imageReader = ImageReader.newInstance(metrics.widthPixels, metrics.heightPixels, PixelFormat.RGBA_8888, 1);
VirtualDisplay virtualDisplay = mediaProjection.createVirtualDisplay(
"ScreenCapture",
metrics.widthPixels,
metrics.heightPixels,
metrics.densityDpi,
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
imageReader.getSurface(),
null,
null
);
imageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Image image = null;
FileOutputStream fos = null;
Bitmap bitmap = null;
try {
image = reader.acquireLatestImage();
if (image != null) {
Image.Plane[] planes = image.getPlanes();
ByteBuffer buffer = planes[0].getBuffer();
int pixelStride = planes[0].getPixelStride();
int rowStride = planes[0].getRowStride();
int rowPadding = rowStride - pixelStride * metrics.widthPixels;
bitmap = Bitmap.createBitmap(metrics.widthPixels + rowPadding / pixelStride, metrics.heightPixels, Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(buffer);
File screenshotsDir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES);
File screenshotFile = new File(screenshotsDir, "screenshot" + screenshotCounter + ".png");
screenshotCounter++;
fos = new FileOutputStream(screenshotFile);
bitmap.compress(Bitmap.CompressFormat.PNG, 100, fos);
fos.flush(); // Add this line to flush the data to the file
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (bitmap != null) {
bitmap.recycle();
}
if (image != null) {
image.close();
}
if (virtualDisplay != null) {
virtualDisplay.release();
}
if (mediaProjection != null) {
mediaProjection.stop();
}
}
}
}, null);
}
}
};
@SuppressLint("WrongConstant")
private VirtualDisplay createVirtualDisplay(MediaProjection mediaProjection) {
DisplayMetrics metrics = new DisplayMetrics();
WindowManager windowManager = (WindowManager) reactContext.getSystemService(Context.WINDOW_SERVICE);
windowManager.getDefaultDisplay().getMetrics(metrics);
imageReader = ImageReader.newInstance(
metrics.widthPixels,
metrics.heightPixels,
PixelFormat.RGBA_8888,
1
);
return mediaProjection.createVirtualDisplay(
"ScreenCapture",
metrics.widthPixels,
metrics.heightPixels,
metrics.densityDpi,
DisplayManager.VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR,
imageReader.getSurface(),
null,
null
);
}
}
public class ScreenCaptureService extends Service {
private static final int SCREEN_CAPTURE_INTERVAL = 1000; // 1 second
private Handler handler;
private Runnable captureRunnable;
private ImageReader imageReader;
private int screenshotCounter = 1;
private VirtualDisplay virtualDisplay;
@Override
public void onCreate() {
super.onCreate();
handler = new Handler();
captureRunnable = new Runnable() {
@Override
public void run() {
captureScreen();
handler.postDelayed(this, SCREEN_CAPTURE_INTERVAL);
}
};
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
handler.post(captureRunnable);
showNotification();
return START_STICKY;
}
@Override
public void onDestroy() {
super.onDestroy();
handler.removeCallbacks(captureRunnable);
}
@Nullable
@Override
public IBinder onBind(Intent intent) {
return null;
}
private void captureScreen() {
imageReader.setOnImageAvailableListener(new ImageReader.OnImageAvailableListener() {
@Override
public void onImageAvailable(ImageReader reader) {
Image image = null;
FileOutputStream fos = null;
Bitmap bitmap = null;
try {
image = reader.acquireLatestImage();
if (image != null) {
Image.Plane[] planes = image.getPlanes();
ByteBuffer buffer = planes[0].getBuffer();
int pixelStride = planes[0].getPixelStride();
int rowStride = planes[0].getRowStride();
int rowPadding = rowStride - pixelStride * image.getWidth();
bitmap = Bitmap.createBitmap(image.getWidth() + rowPadding / pixelStride, image.getHeight(), Bitmap.Config.ARGB_8888);
bitmap.copyPixelsFromBuffer(buffer);
File screenshotsDir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_PICTURES);
File screenshotFile = new File(screenshotsDir, "screenshot" + screenshotCounter + ".png");
screenshotCounter++;
fos = new FileOutputStream(screenshotFile);
bitmap.compress(Bitmap.CompressFormat.PNG, 100, fos);
fos.flush();
}
} catch (IOException e) {
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (bitmap != null) {
bitmap.recycle();
}
if (image != null) {
image.close();
}
if (virtualDisplay != null) {
virtualDisplay.release();
}
}
}
}, null);
}
private void showNotification() {
Here is the Notification
}
}
const {ScreenCaptureModule} = NativeModules
const App = () => {
const startScreenCapture = () => {
ScreenCaptureModule.startScrerenCapture();
};
return (
<View>
<Button onPress={startScreenCapture} title="Start Screen Capture" />
</View>
)
}
export default App
This is the full code of my project. I think the issue is that while the notification function works as expected, no captures are stored in the designated directory happened because I have something wrong in the captureScreen function in ScreenCaptureService.java.
Note: I granted the WRITE_EXTERNAL_PERMISSION. and I'll give you my Manifest.xml:
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.MEDIA_CONTENT_CONTROL" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />
<application
android:name=".MainApplication"
android:label="@string/app_name"
android:icon="@mipmap/ic_launcher"
android:roundIcon="@mipmap/ic_launcher_round"
android:allowBackup="false"
android:theme="@style/AppTheme">
<activity
android:name=".MainActivity"
android:label="@string/app_name"
android:configChanges="keyboard|keyboardHidden|orientation|screenLayout|screenSize|smallestScreenSize|uiMode"
android:launchMode="singleTask"
android:windowSoftInputMode="adjustResize"
android:exported="true">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<service android:name="com.test.ScreenCaptureService" android:exported="false" />
</application>
</manifest>
I have examined your code, and it seems that MediaProjection instance is not being passed properly to your ScreenCaptureService, and therefore, screen capturing does not start.
In your ScreenCaptureModule.java, you are creating VirtualDisplay which initiates capturing and is being released in the same scope. Thus, your capturing process ends immediately after you chose to start screen recording.
A better approach would be not to start and then stop MediaProjection immediately but to keep it running as long as you need to capture the screenshots:
In your ScreenCaptureService.java, add a static method to start your service and pass MediaProjection data as an extra in the Intent:
public class ScreenCaptureService extends Service {
public static final String EXTRA_RESULT_CODE = "resultCode";
public static final String EXTRA_RESULT_DATA = "resultData";
public static void start(Context context, int resultCode, Intent resultData) {
Intent i = new Intent(context, ScreenCaptureService.class);
i.putExtra(EXTRA_RESULT_CODE, resultCode);
i.putExtra(EXTRA_RESULT_DATA, resultData);
context.startService(i);
}
...
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
int resultCode = intent.getIntExtra(EXTRA_RESULT_CODE, 0);
Intent resultData = intent.getParcelableExtra(EXTRA_RESULT_DATA);
// Initialize MediaProjection using data received from ScreenCaptureModule
MediaProjection mediaProjection = mediaProjectionManager.getMediaProjection(resultCode, resultData);
// Initialize your VirtualDisplay and ImageReader here
// Don't forget to stop MediaProjection when the Service is destroyed
}
...
}
In your ScreenCaptureModule.java, replace currentActivity.startService(serviceIntent); with ScreenCaptureService.start(currentActivity, resultCode, data);:
// ... Your existing dialog code ... .
setPositiveButton("Yes", new DialogInterface.OnClickListener() {
@Override
public void onClick(DialogInterface dialog, int which) {
ScreenCaptureService.start(currentActivity, resultCode, data)
currentActivity.finish();
}
})
// ...
In your module, do not release MediaProjection, VirtualDisplay, and Image within onImageAvailable. The capturing should be stopped once your service is stopped:
@Override
public void onDestroy() {
super.onDestroy();
handler.removeCallbacks(captureRunnable);
// Stop MediaProjection right here:
mediaProjection.stop();
// Release VirtualDisplay if it's not already released:
if (virtualDisplay != null) {
virtualDisplay.release();
}
}
Following these steps should keep the MediaProjection alive and capturing during the lifetime of your ScreenCaptureService. With the right code in the ScreenCaptureService's onStartCommand method, you should be able to start capturing screen shots immediately after Service is instantiated, storing them in the Pictures directory.