How to pass unique_ptr by reference polymorphically?

I have a pure virtual class Base and some derived classes ChildA (A/B/C etc):

class Base {
    ...
}

class ChildA : public Base {
    ...
}

I need specific control of the ownership of these child classes, so I spawn them via factory functions and std::unique_ptr.

At some point during their creation/setup, I need to modify them (not make copies or change ownership) in a manner that is common to all of the derived classes, so I want to use a function that accepts their base type and takes by reference:

void modifyInstance(std::unique_ptr<Base>& baseInst);

But when I try to use this function like so:

bool childAFactory(std::unique_ptr<ChildA>& buffer)
{
    buffer = std::make_unique<ChildA>();
    modifyInstance(buffer);
} 

I get this error:

error: non-const lvalue reference to type 'unique_ptr<Base>' cannot bind to a value of unrelated type 'unique_ptr<ChildA>'

Is it not possible to take advantage of polymorphism in this particular fashion? The only similar threads I can find are where one is trying to pass by value in which case you obviously have to use std::move() and give up ownership.

I would have the function take a regular Base pointer and use unique_ptr::get() but I also need to be able to reset the pointer within the function conditionally (so the pointer and its contents need to be able to be modified, hence passing a unique_ptr by reference).

I'd do this:

bool modifyInstance(Base* baseInst);

bool childAFactory(std::unique_ptr<ChildA>& buffer)
{
    buffer = std::make_unique<ChildA>();
    if (!modifyInstance(buffer.get()))
        buffer.reset();
}

That is, let the factory remain responsible for lifetimes, and just return an error result from modifyInstance() when the object needs to be destroyed.