py_image/py3_image entrypoint conflicts with a custom Python container toolchain

[mmikitka]

I am using Python's official python:3.7.3-slim image from Dockerhub. Here is my pull command:

[container_pull(
    name = name,
    registry = "index.docker.io",
    repository = "library/python",
    digest = "sha256:64d8fdeff90572068f5fb5e7fa0d67e28fa4629f54682a67156c6d6fbf3125b9",
) for name in [
    "py3_image_base",
    "py3_debug_image_base",
]]

As a result, I have a custom Python toolchain for the container execution target:

# Modified from @rules_docker//toolchains:BUILD
py_runtime(
    name = "container_py2_runtime",
    interpreter_path = "/usr/bin/python",
    python_version = "PY2",
)

# Path in the python:slim-3.7 image
py_runtime(
    name = "container_py3_runtime",
    interpreter_path = "/usr/local/bin/python3",
    python_version = "PY3",
)

py_runtime_pair(
    name = "container_py_runtime_pair",
    py2_runtime = ":container_py2_runtime",
    py3_runtime = ":container_py3_runtime",
)

toolchain(
    name = "container_py_toolchain",
    exec_compatible_with = [
        "@io_bazel_rules_docker//platforms:run_in_container",
    ],
    toolchain = ":container_py_runtime_pair",
    toolchain_type = "@bazel_tools//tools/python:toolchain_type",
)

I then register the toolchain and execution platforms as such (copied and altered from rules_docker/python3/image.bzl):

register_toolchains(
    "//toolchain:container_py_toolchain",
)

register_execution_platforms(
    "@local_config_platform//:host",
    "@io_bazel_rules_docker//platforms:local_container_platform",
)

My py3_image rule looks like this

DEPS = [
    requirement("certifi"),
    requirement("chardet"),
    requirement("idna"),
    requirement("prometheus_client"),
    requirement("requests"),
    requirement("urllib3"),
]

py3_image(
    name = "image",
    srcs = ["probe.py"],
    layers = DEPS + [
        ":probe",
    ],
    main = "probe.py",
)

I then build the image:

bazel run --toolchain_resolution_debug //insights/monitor/probe/pipeline:image

and I get this

/usr/bin/docker: Error response from daemon: OCI runtime create failed: container_linux.go:345: starting container process caused "exec: \"/usr/bin/python\": stat /usr/bin/python: no such file or directory": unknown.

This seems to be due to the entrypoint attribute on the app_layer conflicting with the toolchain. Since PYTHON_BINARY is extracted from the toolchain and injected into image.binary as the default Python path, it seems as though the ["/usr/bin/python"] entrypoint can be removed altogether.

A PR with this change is forthcoming.

[nlopezgi]

Thanks for opening this issue. it does make sense to remove the entry point now that py toolchains takes care of it

[mmikitka]

Alternative solution: expose the entrypoint attribute to the py_image and py3_image rules. PR forthcoming with this proposal.

[nlopezgi]

Alternative solution: expose the entrypoint attribute to the py_image and py3_image rules. PR forthcoming with this proposal.

Lets try to avoid that option, full explanation why here: #309 (comment)

[purkhusid]

I think I'm also hitting this problem. I've got a custom toolchain set up by following this: https://thethoughtfulkoala.com/posts/2020/05/16/bazel-hermetic-python.html

And then I use python:buster-slim as a base image and get Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"/usr/bin/python\": stat /usr/bin/python: no such file or directory": unknown.

[purkhusid]

And if I use the workaround suggested here: #498 (comment)
I get:

raceback (most recent call last):
  File "/app/docker.binary", line 386, in <module>
    Main()
  File "/app/docker.binary", line 376, in Main
    os.execv(args[0], args)
FileNotFoundError: [Errno 2] No such file or directory: '/app/docker.binary.runfiles/python_interpreter/python_bin'

[danny-skydio]

@purkhusid you might want to try to add two symlinks. I was able to get things to run with:

py3_image(
    name = "image",
    srcs = [":main"],
    base = "@python3_debian//image",  # python is installed at /usr/local/bin/python
    main = "main.py",
)

container_image(
    name = "container",
    base = ":image",
    cmd = [],
    symlinks = {
        # Local python toolchain wants /usr/local/python3.8
        "/usr/bin/python3.8": "/usr/local/bin/python",
        # For py3_image's entrypoint
        "/usr/bin/python": "/usr/local/bin/python",
    },
)

Admittedly, this is pretty awkward and probably isn't very portable.

[jessecureton]

Has there been any more thought into making the generated app_layer use an entrypoint based on the in-container toolchain?

This would simplify a decent bit of user-facing code without workarounds like crafting an intermediary image with symlinks.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_docker!

[github-actions]

This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"

[cnsgsz]

The symlink approach does not work for apache/beam_python3.8_sdk as the image already has /usr/bin/python3 (and py3_image's xx.binary is hard-coded to use that).

Reordering the registration order seems to resolve the issue for me:

register_execution_platforms(
    "@io_bazel_rules_docker//platforms:local_container_platform",
    "@local_config_platform//:host",
)

#1858 also provides a solution.