feat(codecs): Support GELF chunking for encoding

[aramperes]

Summary

Implements the GELF Chunking specification when the gelf codec is used with the socket sink in udp mode. This increases the possible message size from 8KiB to 1MiB to Graylog UDP server, and up to 8MiB when used to bridge to another Vector instance over UDP.

Implementation notes

• The new Chunking trait can be used to implement any chunking method. It requires the contract chunk(Bytes) -> Vec<Bytes>
• Passing a Chunking implementation (Chunker) to the UDP sink is optional, as to skip a Vec heap allocation when chunking is not needed, with the goal of maintaining performance for all other codecs that do not use chunking.
• Chunking is only ever called for the socket sink in UDP mode, but this could later be supported by any other sink that could benefit from it.

Vector configuration

Client side Vector (sink):

sources:
  in:
    type: http_server
    address: 0.0.0.0:8000
    codec: json

transforms:
  remap:
    type: remap
    inputs:
      - in
    source: .host = "vector"

sinks:
  sink_socket:
    type: socket
    inputs:
      - remap
    mode: udp
    address: 127.0.0.1:7008
    encoding:
      codec: gelf
      gelf:
        max_chunk_size: 65507  # localhost limit

Server side Vector (source):

sources:
  in_udp:
    type: socket
    mode: udp
    address: 127.0.0.1:7008
    decoding:
      codec: gelf
    framing:
      method: chunked_gelf

sinks:
  sink_console:
    type: console
    inputs:
      - in_udp
    encoding:
      codec: json

How did you test this PR?

Tested using the configs above, Vector-to-Vector. Confirmed that data sent through the "client" Vector (gelf sink) is replicated on the "server" side Vector (gelf source).

My main motivation for implementing this feature is to support Vector-to-Vector event bridging over UDP, for events larger than 64KiB.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the no-changelog label to this PR.

References

• Closes: GELF codec needs to support chunking #13292
• Related: feat(codecs): Implement chunked GELF decoding #20859
• Specification: https://go2docs.graylog.org/current/getting_in_log_data/gelf.html#chunking
• Inspired from: https://github.com/Graylog2/go-gelf/blob/v2/gelf/udpwriter.go

[aramperes]

@pront (Sorry for the nudge) is there any chance this could get reviewed before the merge freeze for 0.50? Thanks 😊

[pront]

Hi @aramperes, this PR looks good to me. Left a few comments, but nothing major.

[pront]

A few checks are failing. We can merge once those are fixed.