MEL How-To #39

How do I deal with those weird float values that I get instead of zero (e.g. -8.881784329e-017)?

These results represent the almost-but-not-quite-zero values that Maya often returns. Unless you're modelling atoms you can safely ignore values this small.

To counter this behavior, I typically run calculations through a precision rounder, such as the following:

proc float precision( float $value, int $dec )
// $value == value to round off
// $dec == number of decimal places
{
  float $bump = 0.5;
  if ($value < 0.0)
    $bump = -0.5;

  $value = trunc( $value * pow( 10, $dec ) + $bump ) / pow( 10, $dec );

  return $value;
}

Duncan Brinsmead (Chief Scientist, Alias) noted that my function does not work properly when supplied with a negative number of decimal places (for rounding to 10s, 100s, etc). Mr. Brinsmead generously provided his own function which handles this properly:

// Procedure Name: roundoff
//
// Author: Duncan Brinsmead
//
// Description:
// simple function to round float values to a particular decimal
//
// Usage:  roundoff <float value> <number of decimal places>
//
// examples
//
//  roundoff -1.119 2
//  Result: -1.12
//
//  roundoff 256.812 0
//  Result: 257
//
//  roundoff 128.1 -1
//  Result: 130

global proc float roundoff( float $f, int $n )
{
  // we divide if n < 0 to avoid numeric
  // precision problems
  if( $n > 0 )
  {
    float $roundScale = pow(10,$n);
    if( $f > 0 )
      return( ((float)(int)($f * $roundScale + 0.5)) /$roundScale );
    else
      return( ((float)(int)($f * $roundScale - 0.5)) /$roundScale );
  }
  else
  {
    float $roundScale = pow(10,-$n);
    if( $f > 0 )
      return( ((float)(int)($f/$roundScale + 0.5)) *$roundScale );
    else
      return( ((float)(int)($f/$roundScale - 0.5)) *$roundScale );

  }
}

Reproduced with permission from the author.

Converting from scientific to fixed point

David Biggs contributed the following "formatFloat" MEL procedure which can return a fixed point representation from the scientific format used for these infinitesimal floating point values.

proc string formatFloat(float $f,int $precision)
{
  int $wasNegative = 0;
  if($f < 0)
  {
   $f *= -1;
   $wasNegative = 1;
  }
  string $result = "";

  int $b = 0;
  int $n = 12;
  while($n > -$precision)
  {
    float $temp = pow(10,$n);
    int $g = (int)($f/$temp);
    if($n == -1)
    {
      if($b == 0) {
        $b = 1;
        $result += "0";
      }
      $result += ".";
    }

    if($g > 0)
    {
      $b = 1;
      $f = $f -($g * $temp);
      $result += $g;
    }
    else if($b == 1)
    {
      $result += "0";
    }
    $n--;

  }
  if($wasNegative) $result = "-" + $result;
  return $result;
}

Example:

formatFloat(-8.881784329e-017,25);
// Result: -0.000000000000000088817843 //

It can be susceptible to floating point precision loss (like all things that use floating point), so be wary of its results with values more significant than these infinitesimal values.

formatFloat( -0.08881784329, 25 );
// Result: -0.088817843290000001793391 //

formatFloat( 888.1784329, 25 );
// Result: 888.178432899999961579272769 //

Acknowledgements

23 Sep 2004