Skip to content

cksum implementation up to 17x slower than GNU for some large files #8573

@BAMF0

Description

@BAMF0

I benchmarked the uutils version against the GNU version of cksum using hyperfine and noticed that there was a significant difference in performance for some large files.

  • OS release: Ubuntu Questing Quokka (development branch) 25.10
  • GNU coreutils version: 9.5-1ubuntu2+0.0.0~ubuntu20
  • uutils coreutils version: 0.1.0

Edit 1: corrected filename on hyperfine benchmark reproduction for truncated_4G_file.

Reproduce

Generate two large files:

  • oneline_4G.txt - a single line file consisting of a sequence of A's (4GB in size)
dd if=/dev/zero bs=1M count=4096 status=progress | tr '\0' 'A' |
head -c 4294967295 > oneline_4G.txt && echo >> oneline_4G.txt
  • truncated_4G_file - a truncated file (4GB in size)
truncate -s 4G truncated_4G_file

Run hyperfine on oneline_4G.txt:

hyperfine 'cksum oneline_4G.txt' 'coreutils cksum oneline_4G.txt' -N --warmup 3

Output

Benchmark 1: cksum oneline_4G.txt
  Time (mean ± σ):     427.0 ms ±  13.8 ms    [User: 119.7 ms, System: 307.1 ms]
  Range (min … max):   411.5 ms … 452.2 ms    10 runs
 
Benchmark 2: coreutils cksum oneline_4G.txt
  Time (mean ± σ):      7.578 s ±  0.025 s    [User: 7.195 s, System: 0.382 s]
  Range (min … max):    7.538 s …  7.633 s    10 runs
 
Summary
  cksum oneline_4G.txt ran
   17.75 ± 0.58 times faster than coreutils cksum oneline_4G.txt

Run hyperfine on truncated_4G_file:

hyperfine 'cksum truncated_4G_file' 'coreutils cksum truncated_4G_file' -N --warmup 3

Output

Benchmark 1: cksum truncated_4G_file
  Time (mean ± σ):     438.6 ms ±  20.4 ms    [User: 119.6 ms, System: 318.8 ms]
  Range (min … max):   422.8 ms … 491.1 ms    10 runs
 
Benchmark 2: coreutils cksum truncated_4G_file
  Time (mean ± σ):      7.605 s ±  0.023 s    [User: 7.209 s, System: 0.395 s]
  Range (min … max):    7.572 s …  7.651 s    10 runs
 
Summary
  cksum truncated_4G_file ran
   17.34 ± 0.81 times faster than coreutils cksum truncated_4G_file

Reference Machine

  • OS: Ubuntu Questing Quokka (development branch) 25.10
  • Hardware Model: Lenovo Legion Slim 5 16IRH8
  • Processor: 13th Gen Intel® Core™ i7-13620H × 16
  • Memory: DDR5 16.0 GiB @ 5600 MT/s

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions