Is there a way to sample from an image dataset in such a way, as to maintain the highest amount of variance in it, keeping intact all possible outliers and most amount of information ?
Yes, I think it is possible up to some extent. One of the ways would be to compute a histogram of the image's pixel distribution and define a parameter VarianceThreshold. Use the discussion here, to determine how you can calculate variance in your case. Now it should be easily to determine the samples (where the variance is above threshold) and their indices in the image array.