buffervulkanshader-storage-buffer

Vulkan Buffer WorkGroupID not returning actual value when large number of elements


Creating a buffer with pow(2, 24) and a local_size_x = 64 for the layout input qualifier will return WorkGroupID = 262143 which is all fine due to pow(2,24) / 64 - 1, it is zero indexed.

However if we increase the global dimension / no elements / size of the problem to pow(2, 25) lets say WorkGroupID will return values without a reason, they do not match the math.

Here are some limits that the device got that I think matter:

maxStorageBufferRange:          uint32_t = 4294967295
        maxComputeSharedMemorySize:     uint32_t = 32768
        maxComputeWorkGroupCount:       uint32_t[3] = 00000202898A8EC4
            maxComputeWorkGroupCount[0]:    uint32_t = 65535
            maxComputeWorkGroupCount[1]:    uint32_t = 65535
            maxComputeWorkGroupCount[2]:    uint32_t = 65535
        maxComputeWorkGroupInvocations: uint32_t = 1024
        maxComputeWorkGroupSize:        uint32_t[3] = 00000202898A8ED4
            maxComputeWorkGroupSize[0]:     uint32_t = 1024
            maxComputeWorkGroupSize[1]:     uint32_t = 1024
            maxComputeWorkGroupSize[2]:     uint32_t = 1024

I do not go overboard with allocating more elements that the device supports. So after 2 days + 16 hrs I still did not figure out whats going on...

WorkGroupSize, WorkGroupID, LocalInvocationID and GlobalInvocationID presents the same problem when I reach a n no. of elements. It is no wonder that GlobalInvocationID presents the same problem due to how it is calculated...

#version 450
// Size of the Local Work-group is defined trough input layout qualifier
layout(local_size_x = 64, local_size_y = 1, local_size_z = 1) in;


layout(set = 0, binding = 0) buffer deviceBuffer
{
   uint x[];
};


void main() {
    uint i = gl_GlobalInvocationID.x;
    //uint i = gl_WorkGroupSize.x * gl_WorkGroupID.x * gl_LocalInvocationID.x;


    //x[i] += x[i];

// Total No. of Work Items (threads) in Global Dimension
//x[i] = gl_NumWorkGroups.x;

// Size of Work Dimension specified in Input Layout Qualifier
//x[i] = gl_WorkGroupSize.x;

// Is given by Global Dimension / Work Group Size
x[i] = gl_WorkGroupID.x;

//x[i] = gl_LocalInvocationID.x;

}

Solution

  • maxComputeWorkGroupCount[0]: uint32_t = 65535
    maxComputeWorkGroupCount[1]: uint32_t = 65535
    maxComputeWorkGroupCount[2]: uint32_t = 65535
    

    vkCmdDispatch have the size in x = pow(2, 25), y = 1, z = 1

    Based on the info you provided groupCountX = 225 = 33554432, but the limit is maxComputeWorkGroupCount[0] = 65535 = 216-1.

    The Vulkan specification Valid Usage for vkCmdDispatch says:

    • groupCountX must be less than or equal to VkPhysicalDeviceLimits::maxComputeWorkGroupCount[0]

    Violating Valid Usage is undefined behavior. "Undefined behavior" means anything from "everything seemingly working fine" to "your PC colapses into a black hole and destroys this solar system". For all intents and purposes violating Valid Usage is a logical error of the application code.