androidfirebasetestingfirebase-remote-configfirebase-ab-testing

Firebase A/B Testing live data not displayed for Android app


The A/B tests for the Android app of my project do not display any live data on the main A/B testing page. Live data is displayed for the iOS version.

I’ve tested the experiments on a couple of Android devices and I’m receiving the correct values from the FirebaseRemoteConfig so the experiments are running correctly.

Here's a screenshot of what I mean

The iOS test is on the left and the Android one is on the right. Although the iOS test was only made 50 minutes ago, it shows data whereas the Android test was created over 20 hours ago and shows no live data.

I understand that "0 Total Users" can be shown for 24 hours or more after the experiment is started so I'm not concerned about that. But since the iOS test shows the live data chart, I find it strange that the Android one doesn't.

The tests are working in devices but I'll add some code as well. Here's how I'm fetching the remote config values.

    final FirebaseRemoteConfig remoteConfig = FirebaseRemoteConfig.getInstance();
    remoteConfig.setConfigSettings(new FirebaseRemoteConfigSettings.Builder()
            .build());

    HashMap<String, Object> defaults = new HashMap<>();
    defaults.put(PARAMETER_NAME, DEFAULT_VALUE);
    remoteConfig.setDefaults(defaults);
    long cacheExpirationSeconds = BuildConfig.DEBUG ? 0
            : TimeUnit.HOURS.toSeconds(12);
    remoteConfig.fetch(cacheExpirationSeconds).addOnSuccessListener(new OnSuccessListener<Void>() {
        @Override
        public void onSuccess(Void aVoid) {
            remoteConfig.activateFetched();
        }
    });

Any ideas why this is would be helpful, thanks.


Solution

  • Seems this is related to an issue with the latest version of Firebase config SDK. From the release notes:

    The latest release of the Firebase Android SDK for Remote Config (v16.1.2) causes A/B Testing to not work as expected. Remote Config SDK v16.1.2 does not collect user behavior for A/B Testing experiments which causes reports to show that an experiment has zero users.

    If you've configured both A/B Testing and Remote Config in your project, use v16.1.0 of the Remote Config Android SDK.

    I changed to v16.1.0 and it's showing the live data now.

    Also, the notes mention the following:

    Note that downgrading Remote Config to v16.1.0 requires that the following Android SDKs (if they're used in your project with Remote Config) to be at the following versions:

    com.google.firebase:firebase-ads:17.1.1

    com.google.firebase:firebase-analytics:16.0.5

    com.google.firebase:firebase-dynamic-links:16.1.3

    com.google.firebase:firebase-invites:16.0.5

    com.google.firebase:firebase-core:16.0.5

    I'm only using Firebase core but changing the version to 16.0.5 created conflicts with other Google libraries. I then used the latest version, 16.0.6, and the remote config and live data still work fine. Just for anyone else having some issues with that.