Skip to content

gRPC: GCSFuse hangs while repeatedly writing over a file  #2784

@raj-prince

Description

@raj-prince

Describe the issue
Using gRPC as the client protocol, GCSFuse hangs while performing write benchmark via fio. Fio tries to repeatedly write over 31MiB file again and again.

System & Version (please complete the following information):

  • OS: all
  • Platform all

GCSFuse retry logs:

time="19/11/2024 10:21:13.336232" severity=INFO message="Retrying for the error: rpc error: code = Internal desc = We encountered an internal error. Please try again. Internal details: AKTLlfWsJ27WDixaDHRc4nJoW/3oJ1LLBWLN7rOdM+pNkPz/0wmNWvTRrYlF66IG1JigwKBkcc7uXtNog21dIDZcgWz3W6F1+PKMVuxRVkzFSdzLaGAI563pkrCwEg2Uq7mhcsSoDVBb0yCfgxGLqZSdvqXh8a97ZET4lpIKO5XW1Vlfks0yZrpM0gDO10nHZQhGszh0MX47sK8j+SX72krdm29cHzVYYAeDYzy+0elm95Ex9mjsFpGezxj185Os6MOlUm5oy68iR7NMg91g/fyPnETbb2OJIBD/EXH2pWusvC0xb3Os3oSl8z8w9vJ0Vu/Kr2ZzdhNYRNcv4pX0pKdQZUAODQ+mtu5eS36TvgAIHfkX+pFbe1vwShUckLL9ztaaatLYtBeE2fubiQyhU005x0GznI23IlNRCy5jwFWPG4hYVXhZsRe6aOD73YGUDNrj9Ghc0Fbf9Bd6T+rkLxmHfvpXN5AddHySAdc8gC0Tjjq1oK3FpC6hDtR6/gmRAOtpBuxOApTbYQz3cTc1XXj8KUfNPkWfOqL27NyKDObnDI0RdbI4sn8Haiv3DkJiAkNTPdp1a59dVEwVtMMuSRSiKqsWEC+wTJUaDQalgAjBJ+G9XpQSXilvS0177DKoEtfTTIwBicPJZzPudKpVRVbNqj7TqwLcHQWOkoxpjopRCwWRsMko6iJNTEiouc5A3WrN1pijyVkDokjASYDzoQMBiGCKeNYom9RXrvu4k6eucA/UVoHKUL76pVLc9rXAP1EhEN9YWHJIiSFmqaawr2Ihd//QPNf/WhNKiEXUK+W1/rrD10c7YbolFbbjLYP2K2r4kIouRBg+Lq9TlA+iAOTJC6FpDFW/5SRrbaE1KPoHTrdveuCuhxoOj6a+VIR6lxlMRafKft8+YZCnpHI3+RqMzPNwrSCCsxrAtqRnYxUAkzhBL5pqfmBjNAKnPLU4Mc1zUXnsEIfOaX2HaFtb3zxgAma8eiJ9jg=="
time="19/11/2024 10:21:13.339231" severity=INFO message="Retrying for the error: rpc error: code = Internal desc = We encountered an internal error. Please try again. Internal details: AKTLlfX1m0YzxQ/eIsVy+txZtjR5ajyLn4MNOYZs/cpcLngYkQ1r+bqxB4wO4iPR21DM1YQChfa4hyIATAXkF+8yRE+rPGzwmI7kOVRXw/7X10uRMcsdE+NlK8M6KYgDZpNXv19KUa8esMl+H9Z0bZK9iCcFJgTTErsBXf0YeinLpmXkpXdIgKtnLAM4OCUfVAXwXD2xyu3m6uxIkAcMf0GiBj1Wcb5s38isjXidIpWoO/kRin0JN7qo3vLciHIIH5G0VHCNvYgAwMmEedzabLEbsKw8PHH00OfAXWAKmOAjlfvaJmNJ2gwiPZ+OwQbmlE1qYN98D6zJTwlvRdJC/t/4/rpLRF7yWcigkS2Z47tnD7Dy+uqq9OfxeUjLK2bBYw79lhHYehtSzlDJ3tdY+Qpz6VyBqR2XPl1f1zi0FBlsPpuosfSDge0vPqotmiXBkNj2+wFl50VmtPF50b4dYeOUpnyAd0+O1p44HoQptqzQa1tapCl+J0WviJrFMrp5OD7v7RD9jh2HnsdOWbfrmVzGq6QsJItZndvTJLRBa0IXR/7qhEVExV1/2b4D656o2rzlnjWlepaQoQPhDIOGnJQgh8s6QR8+wom0miDIMQJB0MThZcjtCPs0ByzRd+Jjw7tnYf/DBVXVvGg5/lQwgMj9xZGXLotKRvl3bCUqQAc3D+TQvtVUQlO/LH6ix1nBeu/TTV2Prag6t7ywANL4j3dZFZukbJjznCmk4nBAxF5ArOq+WAkkxo3suHwaKTv2mRVU35WcSNlZJHpUKdTqVq95po6Mlx55KbBjeavm6T7any8+yQQo0KRtAhBbqyK7529IJqiz271A4itshN042zTtHNhowLslAXcgN8H3nYdhnLhkA45R3GSRCOJklt+ON0XhmOljMUuBF0ezPeHTz7FnNCM+bmRse6H6UwOrth7yIK0S/T/9zj4ppIFVw8QvrT7q/IV3MCtl+X+em1+WzE8wgiAL1HX15A=="

SLO:
We strive to respond to all bug reports within 24 business hours provided the information mentioned above is included.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions