Add support for brotli decoding (#5783)

* Add support for Brotli decoding

When the brotli or brotlicffi packages are installed,
urllib3.util.make_headers() inserts ',br' in the Accept-Encoding header
and decodes br from the answers.

* Create the default Accept-Encoding header once

* Preserve the previous delimiter behavior

* Update prose in quickstart.rst

Co-authored-by: Seth Michael Larson <[email protected]>

Author: dilyanpalauzov

HISTORY.md

@@ -5,6 +5,8 @@ dev
 ---
 
 -   \[Short description of non-trivial change.\]
+- Requests Brotli compression, if either the `brotli` or `brotlicffi` package
+  is installed.
 
 **Dependencies**
 

docs/community/faq.rst

@@ -11,6 +11,9 @@ Encoded Data?
 Requests automatically decompresses gzip-encoded responses, and does
 its best to decode response content to unicode when possible.
 
+When either the `brotli <https://pypi.org/project/Brotli/>`_ or `brotlicffi <https://pypi.org/project/brotlicffi/>`_
+package is installed, requests also decodes Brotli-encoded responses.
+
 You can get direct access to the raw response (and even the socket),
 if needed as well.
 

docs/user/quickstart.rst

@@ -128,6 +128,9 @@ You can also access the response body as bytes, for non-text requests::
 
 The ``gzip`` and ``deflate`` transfer-encodings are automatically decoded for you.
 
+The ``br``  transfer-encoding is automatically decoded for you if a Brotli library
+like `brotli <https://pypi.org/project/brotli>`_ or `brotlicffi <https://pypi.org/project/brotli>`_ is installed.
+
 For example, to create an image from binary data returned by a request, you can
 use the following code::
 

requests/utils.py

@@ -20,6 +20,7 @@
 import warnings
 import zipfile
 from collections import OrderedDict
+from urllib3.util import make_headers
 
 from .__version__ import __version__
 from . import certs
@@ -41,6 +42,11 @@
 
 DEFAULT_PORTS = {'http': 80, 'https': 443}
 
+# Ensure that ', ' is used to preserve previous delimiter behavior.
+DEFAULT_ACCEPT_ENCODING = ", ".join(
+    re.split(r",\s*", make_headers(accept_encoding=True)["accept-encoding"])
+)
+
 
 if sys.platform == 'win32':
     # provide a proxy_bypass version on Windows without DNS lookups
@@ -835,7 +841,7 @@ def default_headers():
     """
     return CaseInsensitiveDict({
         'User-Agent': default_user_agent(),
-        'Accept-Encoding': ', '.join(('gzip', 'deflate')),
+        'Accept-Encoding': DEFAULT_ACCEPT_ENCODING,
         'Accept': '*/*',
         'Connection': 'keep-alive',
     })