float16 support in web runtimes

[josephrocca]

Is your feature request related to a problem? Please describe.
It's seemingly quite common for models to use float16, and the web runtimes don't support that data type. It's possible to convert float16 values to float32 values, and this may work fine for most models, but this still results in .onnx files that are twice as big as they need to be.

I'm not sure how the tensor datatypes map to actual wasm/js datatypes, but even if float16 were in some sense "emulated", that would be useful in reducing the .onnx file sizes, even if it has no impact on performance. It would also make the web runtime compatible with more models so that it works "out of the box", instead of requiring (what was for me, as a noob) a tedious process of converting float16 to float32.

I haven't tested it, but I think TF.js can do a sort of "rehydration" of float16 values to float32 after downloading (at least, it seems like it going by this pull request that I came across - though it's aimed at post-training quantization). If that is indeed the case, then perhaps their implementation/choices would be a useful reference.

Here are the currently-supported data types for the wasm runtime:

System information

• ONNX Runtime version (you are using): https://cdn.jsdelivr.net/npm/onnxruntime-web@1.9.0/dist/ort.js

Thanks!

[hariharans29]

CC: @hanbitmyths / @fs-eire

[josephrocca]

Potentially relevant: gpuweb/gpuweb#658 (comment) (Looks like early releases of WebGPU may ship with f16 support)

[lsb]

So far the easiest alternative to float16 support is converting float32 to bfloat16 and back again, to throw away half of the precision, and hope that Accept-Encoding: gzip picks up the slack, or dealing with quantization operators. Having half-precision floats would be useful, thank you for ONNX

[snnn]

This issue has been automatically closed as 'not planned' because it has been marked as 'stale' for more than 30 days without activity. If you believe this is still an issue, please feel free to reopen it.