Skip to content

Conversation

drinkcat
Copy link
Collaborator

@drinkcat drinkcat commented Jun 4, 2025

See #8031.

This fixes GNUTest sort-float, and actually improves a bit on GNU coreutils behaviour as we now use arbitrary precision to sort floats.

This comes with significant performance cost, so I tried to recover some of the performance by optimizing parsing, but couldn't get back to original runtime. We're still much faster than GNU sort.

./sort-main is current upstream/main, ./sort-baseline is after just the first commit, and target/release/sort is after optimizations. sort is GNU coreutils:

cargo build -r -p uu_sort && LANG= LC_ALL= hyperfine -L sort sort,./sort-main,./sort-baseline,target/release/sort "{sort} -g tmp/floatdata"
Benchmark 1: sort -g tmp/floatdata
  Time (mean ± σ):     320.1 ms ±  11.2 ms    [User: 1007.3 ms, System: 10.5 ms]
  Range (min … max):   306.5 ms … 339.8 ms    10 runs
 
Benchmark 2: ./sort-main -g tmp/floatdata
  Time (mean ± σ):      76.4 ms ±   8.9 ms    [User: 348.1 ms, System: 22.1 ms]
  Range (min … max):    59.6 ms …  92.1 ms    45 runs
 
Benchmark 3: ./sort-baseline -g tmp/floatdata
  Time (mean ± σ):     164.4 ms ±  12.8 ms    [User: 559.4 ms, System: 27.9 ms]
  Range (min … max):   143.5 ms … 194.2 ms    18 runs
 
Benchmark 4: target/release/sort -g tmp/floatdata
  Time (mean ± σ):     142.1 ms ±  15.3 ms    [User: 575.7 ms, System: 30.7 ms]
  Range (min … max):   108.6 ms … 165.3 ms    18 runs
 
Summary
  ./sort-main -g tmp/floatdata ran
    1.86 ± 0.29 times faster than target/release/sort -g tmp/floatdata
    2.15 ± 0.30 times faster than ./sort-baseline -g tmp/floatdata
    4.19 ± 0.51 times faster than sort -g tmp/floatdata

test_sort: Add one more test checking arbitrary precision handling

test_sort: Add more sort use cases

test_g_float comes from GNU test, the other one is manually crafted.

docs/src/extensions: Sort uses arbitrary precision decimal numbers

uucore: num_parser: Optimize bigdecimal create when exponent is 0

Makes creating float number without an exponent part quite a bit
faster. Saves about 9% speed in sort -g.

uucore: num_parser: Optimize parse_digits_count

parse_digits_count is a significant hotspot in parsing code.
In particular, any add/mul operation on BigUint is fairly slow,
so it's better to accumulate digits in a u64, then add them
to the resulting BigUint.

Saves about 15-20% performance in sort -g.

uucore: num_parser: Improve scale conversion to i64

It turns out repeatedly calling i64::MAX.into() and i64::MIN.into()
is actually very expensive. Just do the conversion first, and if
it fails, we know why.

Sadly there is still a conversion happening under the hood in
-exponent + scale, but that'd need to be fixed in Bigint.

Improves sort -g performance by ~5%.

sort: Make use of ExtendedBigDecimal in -g sorting

This provides better precision than f64, which we need.

Fixed #8031.

drinkcat added 7 commits June 3, 2025 19:54
This provides better precision than f64, which we need.

Fixed uutils#8031.
It turns out repeatedly calling i64::MAX.into() and i64::MIN.into()
is actually very expensive. Just do the conversion first, and if
it fails, we know why.

Sadly there is still a conversion happening under the hood in
`-exponent + scale`, but that'd need to be fixed in Bigint.

Improves sort -g performance by ~5%.
parse_digits_count is a significant hotspot in parsing code.
In particular, any add/mul operation on BigUint is fairly slow,
so it's better to accumulate digits in a u64, then add them
to the resulting BigUint.

Saves about 15-20% performance in `sort -g`.
Makes creating float number without an exponent part quite a bit
faster. Saves about 9% speed in sort -g.
test_g_float comes from GNU test, the other one is manually crafted.
Copy link

github-actions bot commented Jun 4, 2025

GNU testsuite comparison:

Skipping an intermittent issue tests/misc/stdbuf (passes in this run but fails in the 'main' branch)

@sylvestre sylvestre merged commit 9e21259 into uutils:main Jun 6, 2025
74 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

GNUTest sort-float fails
2 participants