HRESULT 0x8876086c D3DERR_INVALIDCALL with DirectX9 pretty much following the documentation example

So i wanna draw an overlay over another window, but im getting no real runtime error the visual Studio debugging tools tell me that the result of

HRESULT res = object->CreateDeviceEx(D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, hWND, D3DCREATE_HARDWARE_VERTEXPROCESSING, &params, NULL, &device);

is 0x8876086c. So here are the snippets of my code that are important and lead to this error(D3DERR_INVALIDCALL), which leads to the device being a nullpointer, which means i can't do anything with it.

I couldn't really figure out what led to this as i pretty much followed the documentation

int Paint::init(HWND hWND) {
if (FAILED(Direct3DCreate9Ex(D3D_SDK_VERSION, &object))) {
    exit(1);
}
ZeroMemory(&params, sizeof(params));

params.BackBufferWidth = width;
params.BackBufferHeight = height;
params.Windowed = true;
params.hDeviceWindow = hWND;
params.MultiSampleQuality = D3DMULTISAMPLE_NONE;
params.BackBufferFormat = D3DFMT_A8R8G8B8;
params.EnableAutoDepthStencil = TRUE;
params.AutoDepthStencilFormat = D3DFMT_D16;


HRESULT res = object->CreateDeviceEx(D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, hWND, D3DCREATE_HARDWARE_VERTEXPROCESSING, &params, NULL, &device);

and in the header file:

class Paint {
private:
IDirect3D9Ex* object = NULL;
IDirect3DDevice9Ex* device = NULL;
DWORD behaviorFlags = D3DCREATE_HARDWARE_VERTEXPROCESSING;

D3DPRESENT_PARAMETERS params;
ID3DXFont* font = 0;
HWND TargetHWND;
int width, height;

int init(HWND(hWND));
}

Solution

D3DPRESENT_PARAMETERS params = {};

// Use Win32 BOOL "TRUE" instead of C++ "true"
params.Windowed = TRUE;
params.hDeviceWindow = m_window;

// params.BackBufferWidth, BackBufferHeight are ignored for Windowed = TRUE

// For Windowed = TRUE, use params.BackBufferFormat = D3DFMT_UNKNOWN, which is zero.

// For params.BackBufferCount zero is assumed to be 1, but best practice
// would be to set it
params.BackBufferCount = 1;

// You used D3DMULTISAMPLE_NONE for the MultiSampleQuality instead of MultiSampleType. 
// It's all zero anyhow.

params.EnableAutoDepthStencil = TRUE;
params.AutoDepthStencilFormat = D3DFMT_D16;

// --->>> This is the actual bug: there is no valid SwapEffect that has a value of zero <<<---
params.SwapEffect = D3DSWAPEFFECT_DISCARD;

You are making the assumption that the Direct3D9 device supports D3DCREATE_HARDWARE_VERTEXPROCESSING, but you haven't validated it actually supports it. That said, D3DCREATE_SOFTWARE_VERTEXPROCESSING has known performance issues on Windows 10 so you should probably just require HW anyhow.

You should not be using legacy Direct3D9 or Direct3D9Ex for new projects. It's mostly emulated on newer versions of Windows, has lots of strange behaviors, and is almost 20 years old at this point. There's no support for the Direct3D 9 debug device on Windows 8.x or Windows 10. You should consider Direct3D 11 as a much better starting place for developers new to DirectX.