I'm getting the xml dump from different apps running the uiautomator dump command within a service of an app i built, in a rooted phone. Everything works fine when Chrome or Android System WebView are set as WebView implementation with a version different to 84. When version is 84, only in WebView apps, I can't get the views, the xml only shows that there's a WebView but doesn't show it's child elements.
I tried the same with Appium (just for testing purposes, in case of me missing something, because I need to get the xml within my app without being plugged to a pc), and the behavior is the same (as far as I know, Appium also uses uiautomator to dump the views).
I have tried in Android 8 with Chrome 81.0.4044.138, and in Android 9 with Android System WebView 77.0.3865.92, and works fine. When I update the Android 9 phone Android System WebView to most recent version (84.0.4147.125), cant't get elements from WebView apps, neither with Chrome, just get the WebView element.
What I want to know is if the last Android WebView versions (both Chrome and Android System, version 84) have something new that I'm missing, that don't let uiautomator to dump the xml file correctly from WebView-based apps. Maybe if I have to do something else, or if it's a bug. Thank you!
I found a faster and non-root way of getting the view hierarchy that also works fine in most Chrome versions (in case of webview-based apps), even in Chrome 80+, and it was ussing AccesibilityService.
So, i first created a service which extends AccessibilityService:
public class BasicAccessibilityService extends AccessibilityService {
private static BasicAccessibilityService instance;
public static BasicAccessibilityService getInstance(){
return instance;
}
@Override
protected void onServiceConnected() {
instance = this;
super.onServiceConnected();
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
return super.onStartCommand(intent, flags, startId);
}
@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
}
@Override
public void onInterrupt() {
}
@Override
public void onDestroy() {
super.onDestroy();
}
@Override
public void onCreate() {
super.onCreate();
}
@Override
public AccessibilityNodeInfo getRootInActiveWindow() {
try {
return super.getRootInActiveWindow();
} catch (Throwable ignored) {
return null;
}
}
}
In Manifest:
<service
android:name="BasicAccessibilityService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService"
/>
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="@xml/service_config" />
</service>
In service_config.xml:
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
xmlns:android="http://schemas.android.com/apk/res/android"
android:description="@string/accessibility_service_description"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFlags="flagReportViewIds|flagDefault|flagRequestFilterKeyEvents|flagRetrieveInteractiveWindows|flagEnableAccessibilityVolume"
android:canRetrieveWindowContent="true"
android:canRequestTouchExplorationMode="true"
android:accessibilityFeedbackType="feedbackGeneric"
android:canPerformGestures="true"
android:canRequestFilterKeyEvents="true"
android:notificationTimeout="0"
/>
Now the Accessibility Service is set up, and can be called whenever we want to get the view hierarchy. Remember that for it to work the app must have the accessibility permission granted. When the permission is granted the onServiceConnected method is automatically called, so we can call our service from it instance and get the views like:
BasicAccessibilityService.getInstance().getRootInActiveWindow()
This will return an AccessibilityNodeInfo which contains the current views information.
If you need to get the view hierarchy as an XML, you can use the AccessibilityNodeInfoDumper class from the uiautomator source code, which can be found here:
Also you will have to add to your code the AccessibilityNodeInfoHelper found in here:
When i implemented the above code to parse the AccessibilityNodeInfo to XML, i got almost the same problem i had before when getting the views with uiautomator dump. I found that the problem was in the isVisibleToUser method within dumpNodeRec in the AccessibilityNodeInfoDumper class.
For some reason, with some apps the isVisibleToUser method returns false with views that are visible inside webview-based application, and this behaviour changes depending on Chrome or System Web View version. So a possible solution is to not use that method and obtain all views, even those who are "invisible", or another option is to make your own method to determine if some view is visible or not.
For me was enought commenting the line that use the isVisibleToUser method. Hope this helps anyone who needs to do something similar.