Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add markdown() method to Response class #340

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions curl_cffi/requests/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,33 @@ async def aclose(self):
if self.astream_task:
await self.astream_task


def markdown(self) -> str:
"""
Extract markdown text from the response content.

Returns:
str: The markdown text extracted from the response content.

Raises:
ImportError: If the required dependencies for markdown extraction are not installed.
"""
try:
from bs4 import BeautifulSoup
except ImportError:
raise ImportError("BeautifulSoup is required for markdown extraction. Install it using 'pip install beautifulsoup4'.")

try:
import html2text
except ImportError:
raise ImportError("html2text is required for markdown extraction. Install it using 'pip install html2text'.")

soup = BeautifulSoup(self.content, 'html.parser')
text_maker = html2text.HTML2Text()
text_maker.ignore_links = True
return text_maker.handle(str(soup))


# It prints the status code of the response instead of
# the object's memory location.
def __repr__(self) -> str:
Expand Down
Loading