Skip to content

Commit ffbaed8

Browse files
authored
update docstrings (#8142)
1 parent bec822c commit ffbaed8

File tree

1 file changed

+9
-5
lines changed

1 file changed

+9
-5
lines changed

haystack/components/fetchers/link_content.py

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -52,12 +52,16 @@ def _binary_content_handler(response: Response) -> ByteStream:
5252
@component
5353
class LinkContentFetcher:
5454
"""
55-
LinkContentFetcher is a component for fetching and extracting content from URLs.
55+
Fetches and extracts content from URLs.
5656
57-
It supports handling various content types, retries on failures, and automatic user-agent rotation for failed web
58-
requests.
57+
It supports various content types, retries on failures, and automatic user-agent rotation for failed web
58+
requests. Use it as the data-fetching step in your pipelines.
59+
60+
You may need to convert LinkContentFetcher's output into a list of documents. Use HTMLToDocument
61+
converter to do this.
62+
63+
### Usage example
5964
60-
Usage example:
6165
```python
6266
from haystack.components.fetchers.link_content import LinkContentFetcher
6367
@@ -84,7 +88,7 @@ def __init__(
8488
For multiple URLs, it logs errors and returns the content it successfully fetched.
8589
:param user_agents: [User agents](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/User-Agent)
8690
for fetching content. If `None`, a default user agent is used.
87-
:param retry_attempts: Specifies how many times you want it to retry to fetch the URL's content.
91+
:param retry_attempts: The number of times to retry to fetch the URL's content.
8892
:param timeout: Timeout in seconds for the request.
8993
"""
9094
self.raise_on_failure = raise_on_failure

0 commit comments

Comments
 (0)