Denormalized numbers C#

Question

I recently came across denormalized definition and I understand that there are some numbers that cannot be represented in a normalized form because they are too small to fit into its corresponding type. According with IEEE

So what I was trying to do is catch when a denormalized number is being passed as a parameter to avoid calculations with this numbers. If I am understanding correct I just need to look for numbers within the Range of denormalized

private bool IsDenormalizedNumber(float number)
{
    return Math.Pow(2, -149) <= number && number<= ((2-Math.Pow(2,-23))*Math.Pow(2, -127)) ||
           Math.Pow(-2, -149) <= number && number<= -((2 - Math.Pow(2, -23)) * Math.Pow(2, -127));
}

Is my interpretation correct?

Answer 1

I think a better approach would be to inspect the bits. Normalized or denormalized is a characteristic of the binary representation, not of the value itself. Therefore, you will be able to detect it more reliably this way and you can do so without and potentially dangerous floating point comparisons.

I put together some runnable code for you, so that you can see it work. I adapted this code from a similar question regarding doubles. Detecting the denormal is much simpler than fully excising the exponent and significand, so I was able to simplify the code greatly.

As for why it works... The exponent is stored in offset notation. The 8 bits of the exponent can take the values 1 to 254 (0 and 255 are reserved for special cases), they are then offset adjusted by -127 yielding the normalized range of -126 (1-127) to 127 (254-127). The exponent is set to 0 in the denormal case. I think this is only required because .NET does not store the leading bit on the significand. According to IEEE 754, it can be stored either way. It appears that C# has opted for dropping it in favor of a sign bit, though I don't have any concrete details to back that observation.

In any case, the actual code is quite simple. All that is required is to excise the 8 bits storing the exponent and test for 0. There is a special case around 0, which is handled below.

NOTE: Per the comment discussion, this code relies on platform specific implementation details (x86_64 in this test case). As @ChiuneSugihara pointed out, the CLI does not ensure this behavior and it may differ on other platforms, such as ARM.

using System;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            Console.WriteLine("-120, denormal? " + IsDenormal((float)Math.Pow(2, -120)));
            Console.WriteLine("-126, denormal? " + IsDenormal((float)Math.Pow(2, -126)));
            Console.WriteLine("-127, denormal? " + IsDenormal((float)Math.Pow(2, -127)));
            Console.WriteLine("-149, denormal? " + IsDenormal((float)Math.Pow(2, -149)));
            Console.ReadKey();
        }

        public static bool IsDenormal(float f)
        {
            // when 0, the exponent will also be 0 and will break
            // the rest of this algorithm, so we should check for
            // this first
            if (f == 0f)
            {
                return false;
            }
            // Get the bits
            byte[] buffer = BitConverter.GetBytes(f);
            int bits = BitConverter.ToInt32(buffer, 0);
            // extract the exponent, 8 bits in the upper registers,
            // above the 23 bit significand
            int exponent = (bits >> 23) & 0xff;
            // check and see if anything is there!
            return exponent == 0;
        }
    }
}

The output is:

-120, denormal? False
-126, denormal? False
-127, denormal? True
-149, denormal? True

Sources:
extracting mantissa and exponent from double in c#
https://en.wikipedia.org/wiki/IEEE_floating_point
https://en.wikipedia.org/wiki/Denormal_number
http://csharpindepth.com/Articles/General/FloatingPoint.aspx

Code adapted from:
extracting mantissa and exponent from double in c#

Answer 2

From my understanding denormalized numbers are there for help with underflows in some cases (see answer to Denormalized Numbers - IEEE 754 Floating Point).

So to get a denormalized number you would need to explicitly create it or else cause an underflow. In the first case it seems unlikely that a literal denormalized number would be specified in code, and even if someone tried it I am not sure that .NET would allow it. In the second case as long as you are in a checked context you should get an OverflowException thrown for any overflow or underflow in an arithmetic computation so that would guard against the possibility of getting a denormalized number. In an unchecked context I am not sure if an underflow will take you to a denormalized number, but you can try it and see if you are wanting to run calculations in unchecked.

Long story short you can not worry about it if you are running in checked and try an underflow and see in unchecked if you want to run in that context.

EDIT

I wanted to update my answer since a comment didn't feel substantial enough. First off I struck out the comment I made about the checked context since that only applies to non-floating point calculations (like int) and not to float or double. That was my mistake on that one.

The issue with denormalized numbers is that they are not consistent in the CLI. Notice how I am using "CLI" and not "C#" because we need to go lower level than just C# to understand the issue. From The Common Language Infrastructure Annotated Standard Partition I Section 12.1.3 the second note (page 125 of the book) it states:

This standard does not specify the behavior of arithmetic operations on denormalized floating point numbers, nor does it specify when or whether such representations should be created. This is in keeping with IEC 60559:1989. In addition, this standard does not specify how to access the exact bit pattern of NaNs that are created, nor the behavior when converting a NaN between 32-bit and 64-bit representation. All of this behavior is deliberately left implementation specific.

So at the CLI level the handling of denormalized numbers is deliberately left to be implementation specific. Furthermore, if you look at the documentation for float.Epsilon (found here), which is the smallest positive number representable by a float you will get a denormalized number on most machines that matches what is listed in the documentation (which is approximately 1.4e-45). This is what @Kevin Burdett was most likely seeing in his answer. That being said if you scroll down farther on the page you will see the following quote under "Platform Notes"

On ARM systems, the value of the Epsilon constant is too small to be detected, so it equates to zero. You can define an alternative epsilon value that equals 1.175494351E-38 instead.

So there are portability issues that can come into play when you are dealing with manually handling denormalized numbers even just for the .NET CLR (which is an implementation of the CLI). In fact this ARM specific value is kind of interesting since it appears to be a normalized number (I used the function from @Kevin Burdett with IsDenormal(1.175494351E-38f) and it returned false). In the CLI proper the concerns are more severe since there is no standardization on their handling by design according to the annotation on the CLI standard. So this leaves questions about what would happen with the same code on Mono or Xamarin for instance which is a difference implementation of the CLI than the .NET CLR.

In the end I am right back to my previous advice. Just don't worry about denormalized numbers, they are there to silently help you and it is hard to imagine why you would need to specifically single them out. Also as @HansPassant mentioned you most likely won't even encounter anyway. It is just hard to imagine how you would be going under the smallest, positive normalized number in double which is absurdly small.