How to Build an Autonomous Blog Writer with CrewAI and Python

Those who have just started writing, and even experienced writers, face this problem on a day-to-day basis.

They sit in front of a laptop screen for a couple of hours before rage-quitting over

• what to write,
• how to write it,
• which research papers are the best,
• and how to format the post.

The main problem is that writing requires patience, research, and good structure before it can be considered a solid piece of information.

To overcome this, we are going to learn how to automate the task and reduce the effort and hassle of writing.

We will also make the process more interesting, even for experienced writers.

Disclaimer: Think of AI as an assistant, not a replacement for your effort.

Please don't use these tools to spam blogs created purely for monetary gain.

What is CrewAI

CrewAI is an open-source framework and an enterprise platform used to build multi-agent AI systems that orchestrate and collaborate on a complex task, where each agent has a defined role for handling different parts of a job.

You can consider CrewAI as an organisation, agents as the employees of that organisation, and tools as the skill set of those agents.

Main Components of CrewAI

• Crew: A crew is a team of agents specialised in a single task. It can comprise a single agent or multiple agents, depending on the use case. Just as an organisation can have multiple departments, CrewAI can have multiple crews.
• Agents: Agents are the employees responsible for completing a specialised task, such as research, drafting, editing, etc.
• Tools: Tools are the helping hands of agents and carry out certain tasks, such as web search, API calls, etc.

Prerequisites

• AI API keys (paid) or Ollama local setup (free)
• A code editor (VS Code, Cursor, or similar)
• Concepts of generative AI and prompt engineering
• CrewAI basics
• Python basics

LLM Setup

CrewAI requires the configuration of an LLM, so gather the API keys and required details from your LLM provider and write them down in a .env file later, in the format below.

AZURE_OPENAI_API_KEY=your-azure-key
AZURE_OPENAI_ENDPOINT=https://your-resource-name.openai.azure.com/
AZURE_OPENAI_API_VERSION=2024-10-21
AZURE_OPENAI_DEPLOYMENT=your-deployment-name

For my development, I am going to use Ollama, a free, open-source tool for running LLMs locally.

You can use it too, and refer to my blog, How I Built My Own Voice Agent Prototype Using Ollama, ElevenLabs & Whisper?, for setting up Ollama on your local system.

OLLAMA_MODEL=llama3.1:8b
OLLAMA_BASE_URL=http://localhost:11434
OTEL_SDK_DISABLED=true

Python Setup

Once your LLM credentials are written down securely, we can proceed with setting up Python on our system.

Python is a programming language used for the development of many kinds of coding projects, and AI/ML is one of them.

You can download it by visiting the official website.

After downloading and setting up Python on your system, create an empty folder and open it inside your code editor.

Make sure you have installed the Python extension in the editor for better and faster development and debugging.

With that, the Python setup is complete.

To set up CrewAI, we need to install the dependencies of CrewAI in our project using the uv package manager.

Open the terminal and install the uv package manager using the command below.

#Windows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

#Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

#MacOS
curl -LsSf https://astral.sh/uv/install.sh | sh

Once the package manager has installed, restart your terminal and verify it using the below command.

if the command fails, there might be an issue with the installation process

uv --version

After installing the package manager, use the following command to set up the boilerplate of your project.

uv init

After initialising, you need to install a few dependencies, so use the following command to download CrewAI and other libraries.

uv add crewai ddgs requests beautifulsoup4 python-dotenv markdown xhtml2pdf python-docx pyyaml

CrewAI Setup

Once Python and all its libraries are in place, we can start building our project using CrewAI.

Use the following command to set up your blog writing flow.

crewai create

After running the command, you will be prompted with a form where you need to provide basic information about the project you want to build with CrewAI.

I will cover each approach in a different blog.

For our use case, we are going to create a flow named Blogwriting.

What would you like to create?

    crew         A team of AI agents working together
  → flow         A deterministic workflow with full control over agents and crews
    tool         A custom tool for the CrewAI Tool Repository
    skill        An agent skill with instructions and optional assets
    template     A remote project template from the CrewAI gallery


Name of your flow › BlogWriting

Once your CrewAI foundation is created, you will see a similar folder structure in your code editor.

AutonomousBlogWriting/
 .venv/                        
 .gitignore
 .python-version               
  main.py                       
  pyproject.toml                
  README.md
  uv.lock
  blogwriting/                 
      .env                      
      .gitignore
       pyproject.toml            
       README.md
       src/
         blogwriting/
             __init__.py
             main.py           
             crew.py 
             crews/content_crew        
                 config/
                     agents.yaml   
                     tasks.yaml 
                 content_crew.py  
             tools/
                 __init__.py
                 custom_tool.py  

Our first task is to set up the environment variables, so go to AutonomousBlogWriting/blogwriting/.env and paste your LLM credentials in the format I showed you above.

Code

Now that every element is in place, we are going to start combining all those things to build our project.

If you get stuck during this process, I recommend you fork my Git repo to get a better understanding of the code

GitHub — Autonomous Writer Git Repository

Config

To build an organisation, we first need employees who are specialists in their respective domains, so go to AutonomousBlogWriting/blogwriting/src/blogwriting/crews/content_crew/config and start updating the YAML files.

agents.yaml — In this file, you need to define all the agents (employees) you want to work with in your organisation.

Each agent has three main components:

• role — The title of your agent
• goal — The responsibilities of your agent
• backstory — short summary of how your agent will perform its task

We are going to create four agents: researcher, writer, editor, and formatter.

You can always refer to my Git repo for reference, as I have defined all four agents in my YAML file.

tasks.yaml — In this file, we are going to define all the respective tasks your agents will perform to generate the expected output.

An agent can have a single task or multiple tasks, depending on the use case.

We are going to create four tasks, one for each agent's goal.

Similar to agents, tasks also have three main components:

• description — A detailed structure of the steps the agent has to take to generate the desired results
• expected_output — The format in which the agent will generate the result. It can be normal text, a markdown file, code, etc.
• agent — The name of the agent you want this task to be assigned to.

Tool

Agents require tools to fetch details from the internet, make API calls, fetch data from databases, and much more.

We need to create a web search tool that lets us find the most relevant information about the topic the user is searching for.

For this, we are going to create a custom tool inside AutonomousBlogWriting/blogwriting/src/blogwriting/tools and name it web_search_tool.py.

It helps us fetch a dictionary of URLs related to the topic and then read the details from those URLs to create a new blog as the output.

from typing import Type

import requests
from bs4 import BeautifulSoup
from crewai.tools import BaseTool
from ddgs import DDGS
from pydantic import BaseModel, Field


class WebSearchInput(BaseModel):
    query: str = Field(..., description="The search query, e.g. 'latest trends in RAG'")
    max_results: int = Field(5, description="Number of results to return (1-10)")


class WebSearchTool(BaseTool):
    name: str = "web_search"
    description: str = (
        "Search the web for up-to-date information. Returns title, URL and snippet "
        "for each result. Use it to research topics and collect sources."
    )
    args_schema: Type[BaseModel] = WebSearchInput

    def _run(self, query: str, max_results: int = 5) -> str:
        max_results = max(1, min(int(max_results), 10))
        try:
            results = DDGS().text(query, max_results=max_results)
        except Exception as e:
            return f"Search failed: {e}. Try rephrasing the query or retry shortly."
        if not results:
            return "No results found. Try a broader or differently worded query."
        return "\n\n".join(
            f"[{i}] {r.get('title', '').strip()}\n"
            f"URL: {r.get('href', '')}\n"
            f"Snippet: {r.get('body', '').strip()}"
            for i, r in enumerate(results, 1)
        )


class PageReaderInput(BaseModel):
    url: str = Field(..., description="Full URL of the page to read (include https://)")
    max_chars: int = Field(4000, description="Maximum characters of page text to return")


class PageReaderTool(BaseTool):
    name: str = "read_webpage"
    description: str = (
        "Fetch a web page and return its main text. Use on a promising URL from "
        "web_search when the snippet isn't enough."
    )
    args_schema: Type[BaseModel] = PageReaderInput

    def _run(self, url: str, max_chars: int = 4000) -> str:
        try:
            resp = requests.get(
                url, timeout=15,
                headers={"User-Agent": "Mozilla/5.0 (compatible; BlogFlowBot/1.0)"},
            )
            resp.raise_for_status()
        except Exception as e:
            return f"Could not fetch {url}: {e}"
        soup = BeautifulSoup(resp.text, "html.parser")
        for tag in soup(["script", "style", "nav", "footer", "header", "aside", "form"]):
            tag.decompose()
        text = " ".join(soup.get_text(separator=" ").split())
        return text[:max_chars] + ("... [truncated]" if len(text) > max_chars else "")

LLM and Crew

Everything around the project is set up.

Now we have to build our crew and flow, then merge them into main.py.

Start your Ollama server and create a file named config.py inside AutonomousBlogWriting/blogwriting/src/blogwriting.

import os

from crewai import LLM
from dotenv import load_dotenv

load_dotenv()


def get_llm(temperature: float = 0.4) -> LLM:
    return LLM(
        model=f"ollama/{os.getenv('OLLAMA_MODEL', 'llama3.1:8b')}",
        base_url=os.getenv("OLLAMA_BASE_URL", "http://localhost:11434"),
        temperature=temperature,
    )

We are going to call our LLM from this file and set it in our content_crew.py file, where we are going to define our agents and tasks in CrewAI and call them based on the agent role and task description.

Below is our content_crew.py file. You need to update it in AutonomousBlogWriting/blogwriting/src/blogwriting/crews/content_crew/content_crew.py.

from pathlib import Path

import yaml
from crewai import Agent, Crew, Process, Task

from config import get_llm

CONFIG_DIR = Path(__file__).parent / "config"
AGENTS = yaml.safe_load((CONFIG_DIR / "agents.yaml").read_text(encoding="utf-8"))
TASKS = yaml.safe_load((CONFIG_DIR / "tasks.yaml").read_text(encoding="utf-8"))


def run_step(task_key: str, inputs: dict, tools=None, temperature=0.4, max_iter=None) -> str:
    task_cfg = dict(TASKS[task_key])
    agent_key = task_cfg.pop("agent")          # "agent: researcher" -> look up in agents.yaml

    extra = {"max_iter": max_iter} if max_iter else {}
    agent = Agent(
        config=AGENTS[agent_key],
        tools=tools or [],
        llm=get_llm(temperature),
        allow_delegation=False,
        verbose=True,
        **extra,
    )
    task = Task(config=task_cfg, agent=agent)
    crew = Crew(agents=[agent], tasks=[task], process=Process.sequential, verbose=True)
    return str(crew.kickoff(inputs=inputs).raw).strip()   # fills {topic}, {draft}, etc.

Exporters

exporters.py is a file that we are going to use to store the LLM response in our desired format, such as Markdown, DOCX, PDF, etc.

Create a new file named exporters.py in AutonomousBlogWriting/blogwriting/src/blogwriting.

For now, this file supports a limited number of formats that the LLM response can be converted to.

If you need more, you can update this file according to your needs.

import re
from pathlib import Path
from typing import Callable, Dict, Tuple

import markdown as md_lib

Meta = Dict[str, str]

CSS = """
@page { size: A4; margin: 2cm; }
body { font-family: Helvetica, Arial, sans-serif; font-size: 11pt; line-height: 1.5; color: #222; }
h1 { font-size: 22pt; margin-bottom: 4pt; }
h2 { font-size: 15pt; margin-top: 18pt; }
h3 { font-size: 12pt; }
a { color: #1a5fb4; }
code, pre { font-family: Courier; font-size: 9.5pt; background-color: #f2f2f2; }
blockquote { margin-left: 12pt; color: #555; }
table { border-collapse: collapse; }
td, th { border: 1px solid #bbb; padding: 4pt; }
"""


def split_front_matter(text: str) -> Tuple[Meta, str]:
    m = re.match(r"^---\s*\n(.*?)\n---\s*\n(.*)$", text.strip(), re.S)
    if not m:
        return {}, text.strip()
    meta: Meta = {}
    for line in m.group(1).splitlines():
        if ":" in line:
            k, v = line.split(":", 1)
            meta[k.strip().lower()] = v.strip().strip("'\"")
    return meta, m.group(2).strip()


def slugify(text: str) -> str:
    return re.sub(r"[^a-z0-9]+", "-", text.lower()).strip("-")[:60] or "blog-post"


def _html_doc(meta: Meta, body: str) -> str:
    html_body = md_lib.markdown(body, extensions=["extra", "sane_lists", "tables", "fenced_code"])
    title = meta.get("title", "Blog Post")
    return (
        f"<html><head><meta charset='utf-8'><title>{title}</title>"
        f"<style>{CSS}</style></head><body>{html_body}</body></html>"
    )


def _to_md(meta: Meta, body: str, path: Path, raw: str) -> None:
    path.write_text(raw.strip() + "\n", encoding="utf-8")


def _to_html(meta: Meta, body: str, path: Path, raw: str) -> None:
    path.write_text(_html_doc(meta, body), encoding="utf-8")


def _to_pdf(meta: Meta, body: str, path: Path, raw: str) -> None:
    from xhtml2pdf import pisa

    with open(path, "wb") as f:
        status = pisa.CreatePDF(src=_html_doc(meta, body), dest=f, encoding="utf-8")
    if status.err:
        raise RuntimeError("PDF generation failed")


def _to_txt(meta: Meta, body: str, path: Path, raw: str) -> None:
    text = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", r"\1 (\2)", body)
    text = re.sub(r"^#{1,6}\s*", "", text, flags=re.M)
    text = re.sub(r"(\*\*|__|`)", "", text)
    path.write_text(text.strip() + "\n", encoding="utf-8")


_INLINE = re.compile(r"(\*\*.+?\*\*|`.+?`|\*.+?\*)")


def _add_runs(par, text: str) -> None:
    text = re.sub(r"\[([^\]]+)\]\(([^)]+)\)", r"\1 (\2)", text)
    for part in _INLINE.split(text):
        if not part:
            continue
        if part.startswith("**") and part.endswith("**") and len(part) > 4:
            par.add_run(part[2:-2]).bold = True
        elif part.startswith("`") and part.endswith("`") and len(part) > 2:
            par.add_run(part[1:-1]).font.name = "Courier New"
        elif part.startswith("*") and part.endswith("*") and len(part) > 2:
            par.add_run(part[1:-1]).italic = True
        else:
            par.add_run(part)


def _to_docx(meta: Meta, body: str, path: Path, raw: str) -> None:
    from docx import Document

    doc = Document()
    buf: list = []

    def flush():
        if buf:
            _add_runs(doc.add_paragraph(), " ".join(buf))
            buf.clear()

    in_code = False
    for line in body.splitlines():
        if line.strip().startswith("```"):
            flush()
            in_code = not in_code
            continue
        if in_code:
            par = doc.add_paragraph()
            par.add_run(line).font.name = "Courier New"
            continue
        s = line.strip()
        if not s:
            flush()
        elif (h := re.match(r"^(#{1,6})\s+(.*)", s)):
            flush()
            doc.add_heading(h.group(2), level=min(len(h.group(1)) - 1, 4))
        elif re.match(r"^[-*+]\s+", s):
            flush()
            _add_runs(doc.add_paragraph(style="List Bullet"), re.sub(r"^[-*+]\s+", "", s))
        elif re.match(r"^\d+[.)]\s+", s):
            flush()
            _add_runs(doc.add_paragraph(style="List Number"), re.sub(r"^\d+[.)]\s+", "", s))
        elif s.startswith(">"):
            flush()
            _add_runs(doc.add_paragraph(style="Intense Quote"), s.lstrip("> "))
        elif re.match(r"^(-{3,}|\*{3,})$", s):
            flush()
        else:
            buf.append(s)
    flush()
    doc.save(path)


EXPORTERS: Dict[str, Tuple[str, Callable]] = {
    "markdown": (".md", _to_md),
    "md": (".md", _to_md),
    "html": (".html", _to_html),
    "pdf": (".pdf", _to_pdf),
    "docx": (".docx", _to_docx),
    "txt": (".txt", _to_txt),
}

SUPPORTED_FORMATS = sorted(set(EXPORTERS) - {"md"})


def export(markdown_text: str, fmt: str, out_dir: str, fallback_title: str = "blog post") -> Path:
    fmt = fmt.lower().strip()
    if fmt not in EXPORTERS:
        raise ValueError(f"Unsupported format '{fmt}'. Choose from: {', '.join(SUPPORTED_FORMATS)}")
    ext, writer = EXPORTERS[fmt]
    meta, body = split_front_matter(markdown_text)
    out = Path(out_dir)
    out.mkdir(parents=True, exist_ok=True)
    path = out / f"{slugify(meta.get('title', fallback_title))}{ext}"
    writer(meta, body, path, markdown_text)
    return path

Flow and Main

This is the easiest and last part of our project.

To bring your code live, you just have to create a sequential flow, because there is no separate routing, and every time a query is run, we need to go through a defined number of steps, calling our agents one by one without any loops or conditions.

In AutonomousBlogWriting/blogwriting/src/blogwriting, create a new file named flow.py and paste the code below for the sequential flow of your crew

import re
from pathlib import Path

from crewai.flow.flow import Flow, listen, start
from pydantic import BaseModel

from crews.content_crew.content_crew import run_step
from exporters import export
from tools.web_search_tool import PageReaderTool, WebSearchTool


class BlogState(BaseModel):
    topic: str = ""
    audience: str = "general readers"
    tone: str = "conversational and informative"
    word_count: int = 900
    output_format: str = "markdown"      # markdown | html | pdf | docx | txt
    output_dir: str = "output"
    research_notes: str = ""
    draft: str = ""
    edited_post: str = ""
    final_markdown: str = ""
    output_path: str = ""


def _clean_markdown(text: str) -> str:
    m = re.match(r"^```(?:markdown|md)?\s*\n(.*)\n```\s*$", text.strip(), re.S)
    return (m.group(1) if m else text).strip()


class BlogFlow(Flow[BlogState]):
    def _save_step(self, name: str, text: str) -> None:
        d = Path(self.state.output_dir) / "intermediate"
        d.mkdir(parents=True, exist_ok=True)
        (d / f"{name}.md").write_text(text, encoding="utf-8")

    @start()
    def research(self):
        s = self.state
        s.research_notes = run_step(
            "research_task", s.model_dump(),
            tools=[WebSearchTool(), PageReaderTool()], temperature=0.2, max_iter=10,
        )
        self._save_step("1_research", s.research_notes)

    @listen(research)
    def drafting(self):
        s = self.state
        s.draft = run_step("writing_task", s.model_dump(), temperature=0.7)
        self._save_step("2_draft", s.draft)

    @listen(drafting)
    def editing(self):
        s = self.state
        s.edited_post = run_step("editing_task", s.model_dump(), temperature=0.3)
        self._save_step("3_edited", s.edited_post)

    @listen(editing)
    def formatting(self):
        s = self.state
        raw = run_step("formatting_task", s.model_dump(), temperature=0.1)
        s.final_markdown = _clean_markdown(raw)
        path = export(s.final_markdown, s.output_format, s.output_dir, fallback_title=s.topic)
        s.output_path = str(path)
        print(f"Final output written to: {path}")
        return s.output_path

In the above file, we have created some variables that contain predefined values, and topic is the only variable that is not optional.

To change these variables at runtime, we are going to add arguments in our main file, such as word count, tone, etc.

Below is the main.py file, which lets you run your crew.

import argparse
import sys
from pathlib import Path

sys.path.insert(0, str(Path(__file__).parent / "src"))

from flow import BlogFlow  # noqa: E402
from exporters import SUPPORTED_FORMATS  # noqa: E402



def main() -> None:
    p = argparse.ArgumentParser(description="Autonomous blog-writing Flow (CrewAI + Ollama)")
    p.add_argument("--topic", required=True, help="Blog topic")
    p.add_argument("--audience", default="general readers")
    p.add_argument("--tone", default="conversational and informative")
    p.add_argument("--words", type=int, default=900, help="Target word count")
    p.add_argument("--format", default="markdown", choices=SUPPORTED_FORMATS + ["md"],
                   help="Output file format")
    p.add_argument("--output-dir", default="output")
    p.add_argument("--plot", action="store_true", help="Save a flow diagram (flow.html) and exit")
    args = p.parse_args()

    flow = BlogFlow()
    if args.plot:
        flow.plot("blog_flow")
        return

    result = flow.kickoff(inputs={
        "topic": args.topic,
        "audience": args.audience,
        "tone": args.tone,
        "word_count": args.words,
        "output_format": args.format,
        "output_dir": args.output_dir,
    })
    print(f"\nDone -> {result}")


if __name__ == "__main__":
    main()

All our files are in place. All we have to do is open our terminal and run main.py to generate amazing ideas.

python main.py --topic "C++ vs Rust: which is faster?" --format pdf

With this, we have come to the end of our blog. If you are stuck somewhere, have any opinions, or want me to cover a specific topic, the comments section is open.

Feel free to drop your thoughts.

Thank you for reading!

---

For more such content, visit my Medium blog.