Unable to get provider com.google.firebase.perf.provider.FirebasePerfProvider on pre-lollipop devices

On all versions before v21 (i.e. 16-19), my app crashes on start because of the following error:

E/AndroidRuntime: FATAL EXCEPTION: main
java.lang.RuntimeException: Unable to get provider com.google.firebase.perf.provider.FirebasePerfProvider: java.lang.ClassNotFoundException: com.google.firebase.perf.provider.FirebasePerfProvider
    at android.app.ActivityThread.installProvider(ActivityThread.java:4563)
    at android.app.ActivityThread.installContentProviders(ActivityThread.java:4190)
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4132)
    at android.app.ActivityThread.access$1300(ActivityThread.java:130)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1255)
    at android.os.Handler.dispatchMessage(Handler.java:99)
    at android.os.Looper.loop(Looper.java:137)
    at android.app.ActivityThread.main(ActivityThread.java:4745)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553)
    at dalvik.system.NativeStart.main(Native Method)
 Caused by: java.lang.ClassNotFoundException: com.google.firebase.perf.provider.FirebasePerfProvider
    at dalvik.system.BaseDexClassLoader.findClass(BaseDexClassLoader.java:61)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:501)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:461)
    at android.app.ActivityThread.installProvider(ActivityThread.java:4548)
    at android.app.ActivityThread.installContentProviders(ActivityThread.java:4190) 
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4132) 
    at android.app.ActivityThread.access$1300(ActivityThread.java:130) 
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1255) 
    at android.os.Handler.dispatchMessage(Handler.java:99) 
    at android.os.Looper.loop(Looper.java:137) 
    at android.app.ActivityThread.main(ActivityThread.java:4745) 
    at java.lang.reflect.Method.invokeNative(Native Method) 
    at java.lang.reflect.Method.invoke(Method.java:511) 
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:786) 
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:553) 
    at dalvik.system.NativeStart.main(Native Method) 

On Android 5 and up, my app works without any issues.

I am targeting SDK version 27 and my minimum version is 14. Yes, I have also enabled Multidex support. I have seen some answers about that, but they aren't relevant to my issue as I have this enabled for years already. Currently I am also using Firebase services 11.8.0, the latest.

Fixed it!

Although I had multidex imported and enabled in Gradle, I had to change my Application class to extend MultidexApplication instead of Application.

Hope it helps others too!