The expression ptr + a < ptr is equivalent to a < 0, and an optimizing compiler is likely to make that replacement, thereby removing a range check that might have been necessary for security. If `a` is known to be non-negative, the compiler can even replace ptr + a < ptr with false.

The reason is that pointer arithmetic overflow in C/C++ is undefined behavior. The optimizing compiler can assume that the program has no undefined behavior, which means that adding a positive number to `ptr` cannot produce a pointer less than `ptr`.

To check whether an index a is less than the length of an array, simply compare these two numbers as unsigned integers: a < ARRAY_LENGTH. If the length of the array is defined as the difference between two pointers ptr and p_end, write a < p_end - ptr. If a is signed, cast it to unsigned in order to guard against negative a. For example, write (size_t)a < p_end - ptr.

An invalid check for pointer overflow is most often seen as part of checking whether a number a is too large by checking first if adding the number to ptr goes past the end of an allocation and then checking if adding it to ptr creates a pointer so large that it overflows and wraps around.

In both of these checks, the operations are performed in the wrong order. First, an expression that may lead to undefined behavior is evaluated (ptr + a), and then the result is checked for being in range. But once undefined behavior has happened in the pointer addition, it cannot be recovered from: it's too late to perform the range check after a possible pointer overflow.

While it's not the subject of this query, the expression ptr + a < ptr_end is also an invalid range check. It's undefined behavor in C/C++ to create a pointer that points more than one past the end of an allocation.

The next example shows how to portably check whether a number is outside the range of an allocation between ptr and ptr_end.

  • Pointer Overflow Checking [Embedded in Academia]
  • GCC and pointer overflows [LWN]
    •