Should you use the AtomicReference for the Singleton Pattern?

I ran across the AtomicReference class and was wondering if this could be a good way of creating a Singleton that is mutable and could be replaced for testing.

I know the double-lock checking has issues so I didn't want to go that route. Corrected as of JDK 1.5, https://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html.
Also I would prefer to lazily instantiate the singleton instead of initializing it. For example, during testing I wouldn't want to use the default class SimpleExample but in certain cases it is either expensive to create or problematic in certain environments. I would prefer to be able to replace it prior to the request.

public class SingletonExample {

  private static AtomicReference<SingletonExample> sInstance = new AtomicReference<>();

  private SingletonExample() {
  }

  public static SingletonExample getInstance() {
      return sInstance.updateAndGet(u -> u != null ? u : new SingletonExample());
  }

  @VisibleForTesting
  static void setInstance(SingletonExample singletonExample) {
      sInstance.set(singletonExample);
  }
}

Here are my questions:

1. Is there a big performance hit?
2. Is there an issue that I am not aware of using it as a Singleton?
3. What are reasonable alternatives?

1. Is there a big performance hit?

As with anything: it depends.

"With low to moderate contention, atomics offer better scalability; with high contention, locks offer better contention avoidance."

From "Java Concurrency In Practice" by Goetz section 15.3.2

2. Is there an issue that I am not aware of using it as a Singleton?

Because you're not using locking, the parameter lamda looks like it could be run by multiple threads at the same time. So you could have multipe SingletonExample objects created and set, but each thread would see each update:

public static SingletonExample getInstance() {
  return sInstance.updateAndGet(u -> u != null ? u : new SingletonExample());
}

3. What are reasonable alternatives?

I would stick with simple locking, since:

1. It works as you intended. So you only create SingletonExample once.
2. Allows you to group state variables together, which you can't do with volatile or AtomicReferences.