Hey DEV Community! 👋
Today, I want to share a super useful Python script I built that uses AI to automatically fetch and summarize any web article. It’s perfect for saving time and automating your daily reading list!
🛠️ What This Script Does:
- Takes any article URL.
- Extracts the main text content cleanly.
- Uses an AI model to generate a bullet-point summary.
💻 The Code
python
import requests
from bs4 import BeautifulSoup
from transformers import pipeline
def summarize_article(url):
print("⏳ Fetching article content...")
# 1. Fetch the webpage
response = requests.get(url)
if response.status_code != 200:
return "Failed to retrieve the article."
# 2. Parse HTML and extract text
soup = BeautifulSoup(response.text, 'html.parser')
paragraphs = soup.find_all('p')
article_text = ' '.join([p.get_text() for p in paragraphs])
# 3. Initialize the AI summarization pipeline
print("🤖 Generating summary using AI...")
summarizer = pipeline("summarization", model="facebook/bart-large-cnn")
# Limit text length for the model
input_text = article_text[:2000]
summary = summarizer(input_text, max_length=130, min_length=30, do_sample=False)
return summary[0]['summary_text']
# Example Usage
if __name__ == "__main__":
article_url = "https://dev.to/t/python"
result = summarize_article(article_url)
print("\n📝 AI Summary:")
print(result)
---
💼 **Need a custom Python automation tool or web scraper?** You can hire me directly on Fiverr: [Hire Me on Fiverr](https://www.fiverr.com/kabunji_uxui)
Top comments (1)
Spot on! As developers shifting towards AI automation, ensuring that LLM evaluations focus heavily on correctness and safety is no longer optional. A minor evaluation failure can break an entire production script. Great breakdown!