Recently we switched to a more recent GCC and it optimized away a whole function and replaced it with "null pointer access" trap code instead when optimizing for size. Looking at godbolt, the issue appeared with GCC 11.1 when optimizing with -Os
. Optimizing with -O2
and -O3
works fine, even with -fstrict-aliasing
.
Simplified, the code looks like this (godbolt link):
#include <inttypes.h>
#include <stddef.h>
#include <string.h>
typedef struct bounds_s {
uint8_t *start;
uint8_t *end;
} bounds_s_t;
static void reserve_space(bounds_s_t *bounds, size_t len, uint8_t **element)
{
if (bounds->start + len > bounds->end) {
return;
}
*element = bounds->start;
bounds->start += len;
}
void bug(uint8_t *buffer, size_t size)
{
bounds_s_t bounds;
uint32_t *initialize_this;
initialize_this = NULL;
bounds.start = buffer;
bounds.end = buffer + size;
reserve_space(&bounds, sizeof(*initialize_this), (uint8_t **)&initialize_this);
uint32_t value = 1234;
memcpy(initialize_this, &value, sizeof(*initialize_this));
}
And leads to the following assembly:
bug:
xor eax, eax
mov DWORD PTR ds:0, eax
ud2
What optimization makes GCC think that the initialize_this
variable is NULL? The only thing that comes to my mind is breaking strict aliasing rules. But can typecasting double pointers from uint32_t **
to uint8_t **
really be the issue here and lead to such heavy consequences?
*element = bounds->start;
violates strict aliasing:
element
has type uint8_t **
, *element
has type uint8_t *
, so this is storing into *element
with type uint8_t *
.element
has the address of initialize_this
, which has declared type, and hence effective type, uint32_t *
.uint32_t *
with a type of uint8_t *
does not conform to any of the aliasing rules in C 2018 6.5 7.-fstrict-aliasing
is enabled by -Os
.uint32_t *
after initialize_this = NULL;
, that initialize_this
is unchanged from a null pointer.Additionally, there is no control in the code to ensure the value of bounds->start
is an address properly aligned for a uint32_t
.
These problems can be fixed:
reserve_space
should also pass an alignment requirement, which can be computed by the calling using _Alignof (uint32_t *)
(and, with the expected forthcoming C standard, _Alignof (typeof (initialize_this))
). reserve_space
should add padding bytes as necessary to make the starting address a multiple of this requirement.uint8_t **
, reserve_space
should return a void *
pointing to the reserved space. The calling routine may then assign this to initialize_this
, and the implicit conversion of assignment will automatically convert it to the correct type.The code also does not show the origins of buffer
. If it is dynamically allocated space, then, once initialize_this
is properly set as described above, *initialize_this
may be used as an ordinary uint32_t
. In particular, there is no need to use memcpy
to copy a value into it; it may be set with *initialize_this = 1234;
. However, if buffer
is created in some other way, such as a declared array of uint8_t
, then aliasing problems may remain.