The topic of detecting NaN in HLSL shaders comes across quite frequently. NaN(s) can create a mess, specially when using previous frames’ data as they propagate across the screen.

Renderdoc has a great overlay for displaying NaN(s) but other tools may not have that and sometimes it is best to catch these at runtime using your own debug visualization.

HLSL has the isnan() intrinsic to check for NaN but these can get optimized away unless you force IEEE strictness with -Gis.

The following includes a few code snippets with explanation on how to check for NaN using your own hand-rolled code.

bool IsNaN(float x)
{
    return !(x < 0.f || x > 0.f || x == 0.f);
}

This one is self-explanatory and one of the first things that pop up when you google NaN check. I’ve tested it and it works, but it requires some level of trust. Perhaps we can do better..

HLSL float(s) use IEEE-754 32-bit single precision representation, albeit with some of the rules relaxed.

There is a single sign bit followed by a 8 bit exponent and a 23 bit mantissa or significand.

In this representation, a NaN is identified by a very specific bit pattern – an exponent which consists of all 1’s and a mantissa whose value is non-zero.

An exponent of all 1’s is the bit pattern

0111 1111 1000 0000 0000 0000 0000 0000

.. or 0x7f800000 in hex.

Let’s rewrite the NaN check to look for this.

bool IsNaN(float x)
{
    // 1. Extract the bits in the float by interpreting it as an unsigned int
    // 2. Bitwise AND to isolate the exponent and right shift to erase the mantissa bits
    uint exponent = (asuint(x) & 0x7f800000) >> 23;

    // 1. Extract bits in float as earlier
    // 2. Bitwise AND to extract the 23 bits of the mantissa
    uint mantissa = (asuint(x)) & 0x7fffff;

    // Check that exponent is all 1 and mantissa is non-zero
    return exponent == 0xff && mantissa != 0;
}

The above code can be condensed down to the following. I’ll leave you to figure out that this does the same thing.

bool IsNaN(float x)
{
    return (asuint(x) & 0x7fffffff) > 0x7f800000;
}

Here is an image of a Renderdoc-style NaN overlay but running in realtime. The red pixels are NaNs.

Pretty neat, huh?