Conditionally passing ownership for members

Assume the following sketch:

struct C {
    (either T or T&) m_t;
    C(T& t):
        (instantiate m_t as T& m_t(t))
    {}
    C(T&& t):
        (instantiate m_t as T(t))
    {}
};

such that a C either has or has not ownership of t, depending on how the C was constructed. Is that possible, possibly without resorting to shared_ptr and thus having to move everything to heap? I'm trying around with std::variant, but without success so far.

Something along these lines perhaps:

struct C {
  std::optional<T> t_holder;
  T& m_t;
  C(T& t) : m_t(t) {}
  C(T&& t) : t_holder(std::move(t)), m_t(*t_holder) {}
};

Would also need a copy and/or move constructor (the implicit one won't do the right thing); and the copy/move assignment operators would have to be deleted as I don't think they could be implemented reasonably. One way to salvage assignment would be to replace T& with std::reference_wrapper<T>; it behaves almost like a reference, but can be rebound.