math/big: printing is non-linear and very slow for large Floats

[robpike]

The Text method of big.Float produces a textual representation of the floating point number. I have a program that can create very large numbers and they are slow to print. Moreover, the time to print them does not scale sensibly with the size of the number:

% time ivy -e '1e10000' > /dev/null

real 0m0.005s
user 0m0.002s
sys 0m0.003s
% time ivy -e '1e100000' > /dev/null

real 0m0.032s
user 0m0.027s
sys 0m0.004s
% time ivy -e '1e1000000' > /dev/null

real 0m2.729s
user 0m2.702s
sys 0m0.017s
% time ivy -e '1e10000000' > /dev/null

real 7m11.246s
user 7m7.678s
sys 0m1.834s
%

I have investigated to some extent and the time is definitely being spent inside Text.

[griesemer]

Analysis: For large exponents, the decimal conversion creates the exact binary value in "full precision", which is then converted into a decimal string at decimal.go:75 by calling m.decimalString(), and then rounded to the desired number of decimal digits. For large positive exponents, most of the time is spent in this function (natconv.go:240) and eventually in nat.string (natconv.go:253).

Here's a small driver function to call the involved routines and expose the behaviour:

func TestFloatTextLarge(t *testing.T) {
    f := new(Float).SetFloat64(0.5)
    for e := 10; e < 1e9; e *=10 {
        x := new(Float).SetPrec(256).SetMantExp(f, e)
        println(e, x.Text('g', 10))
    }
}

When the number of desired digits is small and exponents large, we should avoid this conversion path.

[griesemer]

From e-mail thread:

here is an idea about how to do the floating point prints more
efficiently. it works but doesn't have enough precision. what it does
is break out the exponent, scales it to a power of 10 that will
display as "e+(exp)", and then multiplies the reside onto the
mantissa. The mantissa is now in range so prints quickly.

this code works but is missing many important properties, such as
handling negative exponents, adjustable precision, and so on, and the
math with the exponent needs to be done with high precision, but the
idea is sound i think.

-rob

// Assumes positive exponent.
var mant big.Float
exp := f.MantExp(&mant)
fexp := newF().SetInt64(int64(exp))
fexp.Mul(fexp, floatLog2)
fexp.Quo(fexp, newF().SetFloat64(2.3025850929940456840179914546843642076011014886287729760333279009675726096773667591101485283253788949))
// log 2 to many digits.
f64exp, _ := fexp.Float64()
// We now have a floating-point base 10 exponent.
// The integer part is what we will show.
i64exp := int64(f64exp)
// Now compute 10**(fractional part).
scale := math.Pow(10, f64exp - float64(i64exp))
// Rescale mantissa.
mant.Mul(&mant, newF().SetFloat64(scale))
return fmt.Sprintf("%ge+%d\n", &mant, i64exp)

[robpike]

A more thorough implementation is now in robpike.io/ivy/value/bigfloat.go in the dev branch. It is fairly well tested and is now used by ivy for any value with huge exponents.

[bradfitz]

What's the status here? Is something happening or should we punt this to Go 1.7?

[griesemer]

Not addressed yet. I think it's a localized fix, but subtle to get right.
Still hoping to address it but perhaps it won't make it.

• gri

On Wed, Dec 9, 2015 at 7:43 AM, Brad Fitzpatrick notifications@github.com
wrote:

What's the status here? Is something happening or should we punt this to
Go 1.7?

—
Reply to this email directly or view it on GitHub
#11068 (comment).

[griesemer]

Started.