D3D12_HEAP_TYPE_READBACK and DXGI_ERROR_DEVICE_HUNG

I'm getting an DXGI_ERROR_DEVICE_HUNG crash. I can get this to go away by taking out one of the three following

• DirectML work.
• ID3D12GraphicsCommandList::CopyResource whose destination is the readback buffer, source is downstream of the output of the DirectML work.
• copying from the mapped readback buffer into a vector.

Edit: it turns out that my synchronization was actually good, I've even stopped frame buffering the render work, and I was already buffering the readback, so I'm as sure as I can be that the reads to the readback and writes from it are not happening at the same time.

I've stopped persistent mapping the readbacks, and now call map for each read.

All to no avail. No debug messages. No useful dread DRED info.

Turns out that taking out the following calls also caused it to stop crashing:

• ID3D12GraphicsCommandList::CopyResourcewhose source was a HEAP_TYPE_UPLOADresource.
• taking out a function that was actually accidently reading from persistently mapped memory of this HEAP_TYPE_UPLOAD resource

The docs say this is really bad:

Applications should avoid CPU reads from pointers to resources on UPLOAD heaps, even accidently. CPU reads will work, but are prohibitively slow on many common GPU architectures