I was thinking of implementing a manual mediation solution to test ad networks and command apps to alternate between them by checking a flag.
Is it a bad idea to do this using a web hosted file which contains the flag? The app will check the flag's value at start up and prioritize the networks by it.
If the solution is sufficient, could it actually be effective to switch the priorities manually by analyzing the reports of each network in the recent days and see which performs better?
You could build and maintain all that infrastructure yourself. That's what I originally did 4 years ago. Or you can delegate the mediation responsibilities to a platform like Admob. That's what I do now.
Admob will let you manually choose the order of the networks or you can have it auto order them based on recent past performance (which I what I do now).
IMHO you should save your energy for making your app better, not on peripheral items like this.