Running container rootless does not work anymore

[scriptator]

System Info

Any environment, as long as you the TGI container with user != 0.

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

Symptom

I cannot run the current TGI image with user different than root

docker run --platform linux/amd64 -it --user 1000 ghcr.io/huggingface/text-generation-inference:3.1.1

fails immediately with the following error message

unable to refresh ld cache, not a big deal in most cases
/usr/local/bin/text-generation-launcher: error while loading shared libraries: libpython3.11.so.1.0: cannot open shared object file: No such file or directory

Error analysis

• The python library which cannot be found is installed in /root/.local/share/uv/python/cpython-3.11.11-linux-x86_64-gnu/lib/
• The directory /root/ is not readable for non-root users.
• I suspect that the problem has been introduced with Moving to uv instead of poetry. #2919 because release v3.0.1 still works.

Expected behavior

I expect to be able to run TGI with non-root users, e.g. on Kubernetes. This was possible previously.

[andresC98]

Hi, we currently have the same issue:

#3043 (comment)

[sebastianliebscher]

Same issue too with v3.2.3. Setting different env vars did not fix the issue. Building a custom docker image shouldn't be the solution as proposed in the linked issue . Any ideas?

[sebastianliebscher]

Also related: v3.1.1 introduced #2999