cgcccompiler-optimizationstrict-aliasing

GCC optimizing with -Os incorrectly assumes pointer to be NULL


Recently we switched to a more recent GCC and it optimized away a whole function and replaced it with "null pointer access" trap code instead when optimizing for size. Looking at godbolt, the issue appeared with GCC 11.1 when optimizing with -Os. Optimizing with -O2 and -O3 works fine, even with -fstrict-aliasing.

Simplified, the code looks like this (godbolt link):

#include <inttypes.h>
#include <stddef.h>
#include <string.h>

typedef struct bounds_s {
  uint8_t *start;
  uint8_t *end;
} bounds_s_t;

static void reserve_space(bounds_s_t *bounds, size_t len, uint8_t **element)
{
  if (bounds->start + len > bounds->end) {
    return;
  }
  
  *element = bounds->start;
  bounds->start += len;
}

void bug(uint8_t *buffer, size_t size)
{
  bounds_s_t bounds;
  uint32_t *initialize_this;
  
  initialize_this = NULL;
  bounds.start = buffer;
  bounds.end = buffer + size;

  reserve_space(&bounds, sizeof(*initialize_this), (uint8_t **)&initialize_this);

  uint32_t value = 1234;
  memcpy(initialize_this, &value, sizeof(*initialize_this));
}

And leads to the following assembly:

bug:
        xor     eax, eax
        mov     DWORD PTR ds:0, eax
        ud2

What optimization makes GCC think that the initialize_this variable is NULL? The only thing that comes to my mind is breaking strict aliasing rules. But can typecasting double pointers from uint32_t ** to uint8_t ** really be the issue here and lead to such heavy consequences?


Solution

  • *element = bounds->start; violates strict aliasing:

    Additionally, there is no control in the code to ensure the value of bounds->start is an address properly aligned for a uint32_t.

    These problems can be fixed:

    The code also does not show the origins of buffer. If it is dynamically allocated space, then, once initialize_this is properly set as described above, *initialize_this may be used as an ordinary uint32_t. In particular, there is no need to use memcpy to copy a value into it; it may be set with *initialize_this = 1234;. However, if buffer is created in some other way, such as a declared array of uint8_t, then aliasing problems may remain.