I am creating a small game engine in Direct X 11 which is going great but i noticed that 3D objects as well as 2D Objects seem very rough around the edges when rendering. I previously came from Direct X 9 and this issue did not exist there. I was wondering how can I properly enable anti aliasing or some kind of multi-sampling so that my textures don't come out with jagged edges.
swapChainDesc.SampleDesc.Count = 2;
swapChainDesc.SampleDesc.Quality = 0;
depthBufferDesc.SampleDesc.Count = 2;
depthBufferDesc.SampleDesc.Quality = 0;
These are the count and quality combinations but no matter what setting is here there is no visual difference or Direct3D will fail to create the device. My GPU is not overwriting anything since the settings are designed to use whatever the application decides.
In addition to enabling the MSAA sample count for the swap chain's render target, your depth/stencil buffer, and using the correct values for related views (D3D11_TEX2DMS_RTV, D3D11_DSV_DIMENSION_TEXTURE2DMS), you need to actually set D3D11_RASTERIZER_DESC.MultisampleEnable to TRUE in your rasterizer states:
CD3D11_RASTERIZER_DESC rastDesc(D3D11_FILL_SOLID, D3D11_CULL_NONE, FALSE,
D3D11_DEFAULT_DEPTH_BIAS, D3D11_DEFAULT_DEPTH_BIAS_CLAMP,
D3D11_DEFAULT_SLOPE_SCALED_DEPTH_BIAS, TRUE, FALSE,
TRUE /*this is MSAA enable*/, FALSE);
See the Simple rendering lesson for DirectX Tool Kit for a more detailed example.
With Direct3D Feature Level 10.1 or greater hardware will support 4x MSAA. Direct3D Feature Level 11.0 or greater hardware will support 4x and 8x MSAA. Everything should support 2x MSAA.