<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Kacper Włodarczyk</title>
    <description>The latest articles on DEV Community by Kacper Włodarczyk (@deenuu1).</description>
    <link>https://dev.to/deenuu1</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F997289%2F50074490-7c28-44da-9a80-f389f20d3691.jpeg</url>
      <title>DEV Community: Kacper Włodarczyk</title>
      <link>https://dev.to/deenuu1</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/deenuu1"/>
    <language>en</language>
    <item>
      <title>Pydantic Deep Agents 0.3.3: ACP, Thinking, Lifecycle Hooks, and Opinionated Defaults</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Thu, 02 Apr 2026 15:29:56 +0000</pubDate>
      <link>https://dev.to/deenuu1/pydantic-deep-agents-033-acp-thinking-lifecycle-hooks-and-opinionated-defaults-32g4</link>
      <guid>https://dev.to/deenuu1/pydantic-deep-agents-033-acp-thinking-lifecycle-hooks-and-opinionated-defaults-32g4</guid>
      <description>&lt;p&gt;We just released &lt;a href="https://github.com/vstorm-co/pydantic-deepagents" rel="noopener noreferrer"&gt;pydantic-deep 0.3.3&lt;/a&gt; — and this is the biggest release since we open-sourced the project. ACP support, deep subagents by default, thinking, Anthropic caching, lifecycle hooks, skills as slash commands, and a provider setup wizard.&lt;/p&gt;

&lt;p&gt;Let's walk through the changes.&lt;/p&gt;

&lt;h2&gt;
  
  
  ACP: Your Agent in Any Editor
&lt;/h2&gt;

&lt;p&gt;ACP (Agent Client Protocol) is a standardized protocol that lets AI agents run inside editors. Think of it like LSP, but for AI agents instead of language servers.&lt;/p&gt;

&lt;p&gt;Our new &lt;code&gt;apps/acp/&lt;/code&gt; adapter exposes any pydantic-deep agent as an ACP-compatible server. The same agent you run in your terminal now runs in Zed with zero code changes.&lt;/p&gt;

&lt;p&gt;What you get:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Streaming text deltas&lt;/strong&gt; — real-time output, not waiting for completion&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tool call visibility&lt;/strong&gt; — see tool names, arguments, and results (not a black box)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Model switching&lt;/strong&gt; — change from Claude to GPT mid-session&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Session management&lt;/strong&gt; — conversation persistence across editor restarts&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Auto-detect provider&lt;/strong&gt; — reads your API keys, no manual config&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The adapter wraps the same &lt;code&gt;create_deep_agent()&lt;/code&gt; you already use. No new API to learn.&lt;/p&gt;

&lt;p&gt;Because ACP is a protocol (not a Zed-specific plugin), it'll work with any editor that adopts it. We expect more editors to support ACP in the coming months.&lt;/p&gt;

&lt;h2&gt;
  
  
  Subagents Are Now Deep Agents by Default
&lt;/h2&gt;

&lt;p&gt;This is the change that affects the most users.&lt;/p&gt;

&lt;p&gt;Previously, subagents were plain pydantic-ai Agents — lightweight, but limited. They couldn't read files, search the web, or remember things between runs.&lt;/p&gt;

&lt;p&gt;Now, every subagent (built-in and custom) is created via &lt;code&gt;create_deep_agent()&lt;/code&gt; with:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Filesystem access (read, write, edit, grep, glob)&lt;/li&gt;
&lt;li&gt;Web search and web fetch&lt;/li&gt;
&lt;li&gt;Persistent memory&lt;/li&gt;
&lt;li&gt;Large output eviction (auto-save to files when output exceeds 20K tokens)&lt;/li&gt;
&lt;li&gt;Orphaned tool call patching&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If your custom subagent doesn't specify &lt;code&gt;agent&lt;/code&gt; or &lt;code&gt;agent_factory&lt;/code&gt;, it automatically gets the full deep agent factory. You don't have to change anything — your subagents just got more capable.&lt;/p&gt;

&lt;p&gt;We also replaced &lt;code&gt;include_general_purpose_subagent&lt;/code&gt; with &lt;code&gt;include_builtin_subagents&lt;/code&gt;, which adds a "research" deep agent for codebase exploration and web research.&lt;/p&gt;

&lt;h2&gt;
  
  
  Thinking Enabled by Default
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;thinking="high"&lt;/code&gt; is now the default. This enables model reasoning via pydantic-ai's &lt;code&gt;Thinking&lt;/code&gt; capability.&lt;/p&gt;

&lt;p&gt;We support 7 levels:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;create_deep_agent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;anthropic:claude-opus-4-6&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;thinking&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;high&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  &lt;span class="c1"&gt;# default
&lt;/span&gt;    &lt;span class="c1"&gt;# Options: True, False, "minimal", "low", "medium", "high", "xhigh"
&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For models that don't support thinking (like GPT-4.1), the parameter is silently ignored.&lt;/p&gt;

&lt;h2&gt;
  
  
  Anthropic Prompt Caching — On by Default
&lt;/h2&gt;

&lt;p&gt;Three new defaults enabled automatically:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;anthropic_cache_instructions&lt;/code&gt; — cache system prompt&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;anthropic_cache_tool_definitions&lt;/code&gt; — cache tool schemas&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;anthropic_cache_messages&lt;/code&gt; — cache conversation history&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;This reduces token costs and latency for Anthropic models significantly. For non-Anthropic models, these settings are silently ignored.&lt;/p&gt;

&lt;h2&gt;
  
  
  5 New Lifecycle Hooks
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;HookEvent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Enum&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;BEFORE_RUN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;before_run&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;AFTER_RUN&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;after_run&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;RUN_ERROR&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;run_error&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;BEFORE_MODEL_REQUEST&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;before_model_request&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;AFTER_MODEL_REQUEST&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;after_model_request&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These map directly to pydantic-ai's lifecycle hooks. Use cases:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Session tracking&lt;/strong&gt; — log when an agent run starts and ends&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;LLM call logging&lt;/strong&gt; — capture every model request for debugging&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Error alerts&lt;/strong&gt; — get notified when a run fails&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Cost monitoring&lt;/strong&gt; — track token usage per request&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Skills as Slash Commands
&lt;/h2&gt;

&lt;p&gt;Skills now work as slash commands in the CLI. Type &lt;code&gt;/code-review&lt;/code&gt; and the skill activates directly from the picker.&lt;/p&gt;

&lt;p&gt;Discovery follows a 3-tier hierarchy:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Built-in&lt;/strong&gt; (&lt;code&gt;apps/cli/skills/&lt;/code&gt;) — ships with pydantic-deep&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;User&lt;/strong&gt; (&lt;code&gt;~/.pydantic-deep/skills/&lt;/code&gt;) — your personal skills&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Project&lt;/strong&gt; (&lt;code&gt;.pydantic-deep/skills/&lt;/code&gt;) — project-specific skills&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Later sources override earlier ones by name, so you can customize built-in skills per project.&lt;/p&gt;

&lt;h2&gt;
  
  
  Other Notable Changes
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;compact_conversation&lt;/code&gt; tool&lt;/strong&gt; — the agent can manually trigger context compression with an optional focus topic&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Provider setup wizard&lt;/strong&gt; — first-run auto-detects missing API keys and guides through provider selection&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;/provider&lt;/code&gt; slash command&lt;/strong&gt; — switch AI provider and model mid-session&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;/config&lt;/code&gt; slash command&lt;/strong&gt; — view and change settings interactively&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;approve_tools&lt;/code&gt; config&lt;/strong&gt; — choose which tools need user approval (default: &lt;code&gt;["execute"]&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Enhanced BASE_PROMPT&lt;/strong&gt; — Claude Code-inspired sections for code quality, careful execution, and formatting&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Context files simplified&lt;/strong&gt; to &lt;code&gt;AGENTS.md&lt;/code&gt; and &lt;code&gt;SOUL.md&lt;/code&gt; only&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Opinionated Defaults
&lt;/h2&gt;

&lt;p&gt;The philosophy behind 0.3.3: &lt;strong&gt;make the powerful thing the default thing.&lt;/strong&gt;&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Setting&lt;/th&gt;
&lt;th&gt;Before&lt;/th&gt;
&lt;th&gt;After&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Memory&lt;/td&gt;
&lt;td&gt;off&lt;/td&gt;
&lt;td&gt;on&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Thinking&lt;/td&gt;
&lt;td&gt;off&lt;/td&gt;
&lt;td&gt;&lt;code&gt;"high"&lt;/code&gt;&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Prompt caching&lt;/td&gt;
&lt;td&gt;off&lt;/td&gt;
&lt;td&gt;on (Anthropic)&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Subagent type&lt;/td&gt;
&lt;td&gt;plain Agent&lt;/td&gt;
&lt;td&gt;deep agent&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Max nesting depth&lt;/td&gt;
&lt;td&gt;0&lt;/td&gt;
&lt;td&gt;1&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Eviction limit&lt;/td&gt;
&lt;td&gt;none&lt;/td&gt;
&lt;td&gt;20K tokens&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Patch tool calls&lt;/td&gt;
&lt;td&gt;off&lt;/td&gt;
&lt;td&gt;on&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;You get a capable, production-ready agent with &lt;code&gt;create_deep_agent("anthropic:claude-opus-4-6")&lt;/code&gt;. No configuration needed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Get Started
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;pydantic-deep-agents
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or try the CLI:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;pydantic-deep-agents[cli]
pydantic-deep
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Full changelog: &lt;a href="https://github.com/vstorm-co/pydantic-deepagents/blob/main/CHANGELOG.md" rel="noopener noreferrer"&gt;CHANGELOG.md&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Repo: &lt;a href="https://github.com/vstorm-co/pydantic-deepagents" rel="noopener noreferrer"&gt;github.com/vstorm-co/pydantic-deepagents&lt;/a&gt;&lt;/p&gt;




&lt;p&gt;&lt;em&gt;We build open-source AI agent tooling at &lt;a href="https://vstorm.co" rel="noopener noreferrer"&gt;Vstorm&lt;/a&gt;. pydantic-deep is our framework — modular, type-safe, production-tested across 30+ deployments.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>agents</category>
      <category>ai</category>
      <category>opensource</category>
      <category>python</category>
    </item>
    <item>
      <title>Pydantic AI Capabilities, Hooks &amp; Agent Specs - What Changed and How Our Libraries Migrated</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Mon, 30 Mar 2026 14:38:10 +0000</pubDate>
      <link>https://dev.to/deenuu1/pydantic-ai-capabilities-hooks-agent-specs-what-changed-and-how-our-libraries-migrated-4i18</link>
      <guid>https://dev.to/deenuu1/pydantic-ai-capabilities-hooks-agent-specs-what-changed-and-how-our-libraries-migrated-4i18</guid>
      <description>&lt;p&gt;Pydantic AI just shipped the biggest API change since launch. Capabilities, hooks, and agent specs landed in v1.71+, and they fundamentally change how you extend agents.&lt;/p&gt;

&lt;p&gt;We maintain 5 open-source libraries built on top of Pydantic AI: pydantic-ai-shields (formerly pydantic-ai-middleware), pydantic-ai-subagents, pydantic-ai-summarization, pydantic-ai-backend, and the full-stack AI agent template. All five have been migrated. This article covers what changed, why it matters, and real before/after code from our repos.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Are Capabilities?
&lt;/h2&gt;

&lt;p&gt;Capabilities are reusable, composable units of agent behavior. Instead of threading multiple configuration arguments separately -- tools here, instructions there, model settings somewhere else -- a capability bundles everything into a single &lt;code&gt;capabilities&lt;/code&gt; parameter on the &lt;code&gt;Agent&lt;/code&gt; constructor.&lt;/p&gt;

&lt;p&gt;Each capability can provide:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Tools&lt;/strong&gt; (via &lt;code&gt;get_toolset()&lt;/code&gt;)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Instructions&lt;/strong&gt; (static strings or dynamic callables)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Model settings&lt;/strong&gt; (per-step configuration)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Lifecycle hooks&lt;/strong&gt; (before/after/wrap patterns for runs, model requests, tool calls)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tool preparation&lt;/strong&gt; (filter or modify tool definitions per step)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The base class is &lt;code&gt;AbstractCapability&lt;/code&gt;. You subclass it, override the methods you need, and pass instances to the agent:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai.capabilities&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;AbstractCapability&lt;/span&gt;

&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;openai:gpt-4.1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;capabilities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;MyCapability&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="nc"&gt;AnotherCapability&lt;/span&gt;&lt;span class="p"&gt;()],&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Multiple capabilities compose automatically. Before-hooks fire in order (cap1 then cap2), after-hooks fire reversed (cap2 then cap1), and wrap-hooks nest as middleware layers. This is not something we had to build -- the framework handles it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Are Hooks?
&lt;/h2&gt;

&lt;p&gt;Hooks are the lifecycle interception points within capabilities. Pydantic AI provides hooks at four levels:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Run hooks&lt;/strong&gt; -- &lt;code&gt;before_run&lt;/code&gt;, &lt;code&gt;wrap_run&lt;/code&gt;, &lt;code&gt;after_run&lt;/code&gt;, &lt;code&gt;on_run_error&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Node hooks&lt;/strong&gt; -- &lt;code&gt;before_node_run&lt;/code&gt;, &lt;code&gt;wrap_node_run&lt;/code&gt;, &lt;code&gt;after_node_run&lt;/code&gt;, &lt;code&gt;on_node_run_error&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Model request hooks&lt;/strong&gt; -- &lt;code&gt;before_model_request&lt;/code&gt;, &lt;code&gt;wrap_model_request&lt;/code&gt;, &lt;code&gt;after_model_request&lt;/code&gt;, &lt;code&gt;on_model_request_error&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Tool hooks&lt;/strong&gt; -- split into validation and execution phases, each with before/wrap/after/error variants&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Plus &lt;code&gt;prepare_tools&lt;/code&gt; for filtering tool visibility per step.&lt;/p&gt;

&lt;p&gt;That's roughly 20 hook points across 4 lifecycle levels. Error hooks use a neat pattern: &lt;strong&gt;raise to propagate, return to recover&lt;/strong&gt;. If your error handler raises the original exception, it propagates unchanged. Raise a different exception to transform the error. Return a result to suppress it entirely.&lt;/p&gt;

&lt;p&gt;For simple use cases, the &lt;code&gt;Hooks&lt;/code&gt; capability gives you decorator-based registration without subclassing:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai.capabilities&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Hooks&lt;/span&gt;

&lt;span class="n"&gt;hooks&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Hooks&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

&lt;span class="nd"&gt;@hooks.on.before_model_request&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;log_request&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;request_context&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Sending &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;request_context&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;messages&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt; messages&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;request_context&lt;/span&gt;

&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;openai:gpt-4.1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;capabilities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;hooks&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What Are Agent Specs?
&lt;/h2&gt;

&lt;p&gt;Agent specs separate agent configuration from code entirely. You define your agent in YAML or JSON:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="na"&gt;model&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;anthropic:claude-opus-4-6&lt;/span&gt;
&lt;span class="na"&gt;instructions&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;You are a helpful assistant.&lt;/span&gt;
&lt;span class="na"&gt;capabilities&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;WebSearch&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;Thinking&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;effort&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;high&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then load it:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;from_file&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;agent.yaml&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Capabilities that implement &lt;code&gt;get_serialization_name()&lt;/code&gt; and &lt;code&gt;from_spec()&lt;/code&gt; are automatically available. This means your custom capabilities can be YAML-driven too.&lt;/p&gt;

&lt;h2&gt;
  
  
  How Our Libraries Migrated
&lt;/h2&gt;

&lt;h3&gt;
  
  
  pydantic-ai-middleware to pydantic-ai-shields
&lt;/h3&gt;

&lt;p&gt;This was the most dramatic change. Our middleware library had grown to include &lt;code&gt;MiddlewareAgent&lt;/code&gt;, &lt;code&gt;MiddlewareChain&lt;/code&gt;, &lt;code&gt;ParallelMiddleware&lt;/code&gt;, &lt;code&gt;ConditionalMiddleware&lt;/code&gt;, &lt;code&gt;PipelineSpec&lt;/code&gt;, config loaders, a compiler -- a whole parallel abstraction layer.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;We deleted all of it.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The v0.3.0 release renamed the package to &lt;code&gt;pydantic-ai-shields&lt;/code&gt; and rebuilt everything as capabilities.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Before (middleware era):&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai_middleware&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;MiddlewareAgent&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;CostTrackingMiddleware&lt;/span&gt;

&lt;span class="n"&gt;middleware_agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;MiddlewareAgent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;agent&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;middlewares&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;CostTrackingMiddleware&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;budget_limit_usd&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;5.0&lt;/span&gt;&lt;span class="p"&gt;)]&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;middleware_agent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Hello&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;After (capabilities era):&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai_shields&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;CostTracking&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;PromptInjection&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;PiiDetector&lt;/span&gt;

&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;openai:gpt-4.1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;capabilities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;
        &lt;span class="nc"&gt;CostTracking&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;budget_usd&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mf"&gt;5.0&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="nc"&gt;PromptInjection&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;sensitivity&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;high&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="nc"&gt;PiiDetector&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;detect&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;email&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;ssn&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;credit_card&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]),&lt;/span&gt;
    &lt;span class="p"&gt;],&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;agent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Hello&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No wrapper agent. No middleware chain. The shields are just capabilities that hook into &lt;code&gt;before_run&lt;/code&gt;, &lt;code&gt;after_run&lt;/code&gt;, &lt;code&gt;prepare_tools&lt;/code&gt;, and &lt;code&gt;before_tool_execute&lt;/code&gt; as needed.&lt;/p&gt;

&lt;p&gt;The new package ships 10 capabilities: &lt;code&gt;CostTracking&lt;/code&gt;, &lt;code&gt;ToolGuard&lt;/code&gt;, &lt;code&gt;InputGuard&lt;/code&gt;, &lt;code&gt;OutputGuard&lt;/code&gt;, &lt;code&gt;AsyncGuardrail&lt;/code&gt; for infrastructure, plus &lt;code&gt;PromptInjection&lt;/code&gt;, &lt;code&gt;PiiDetector&lt;/code&gt;, &lt;code&gt;SecretRedaction&lt;/code&gt;, &lt;code&gt;BlockedKeywords&lt;/code&gt;, and &lt;code&gt;NoRefusals&lt;/code&gt; as zero-dependency content shields.&lt;/p&gt;

&lt;h3&gt;
  
  
  pydantic-ai-subagents
&lt;/h3&gt;

&lt;p&gt;The subagents library now exposes &lt;code&gt;SubAgentCapability&lt;/code&gt; that bundles the subagent toolset and dynamic instructions:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;subagents_pydantic_ai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;SubAgentCapability&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;SubAgentConfig&lt;/span&gt;

&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;openai:gpt-4.1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;capabilities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;SubAgentCapability&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;subagents&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;
            &lt;span class="nc"&gt;SubAgentConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;researcher&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Researches topics&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="n"&gt;instructions&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;You are a research assistant.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="p"&gt;)],&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The capability provides tools via &lt;code&gt;get_toolset()&lt;/code&gt; and injects instructions via &lt;code&gt;get_instructions()&lt;/code&gt;. It also supports agent spec serialization.&lt;/p&gt;

&lt;h3&gt;
  
  
  pydantic-ai-summarization
&lt;/h3&gt;

&lt;p&gt;Four capabilities replace the old middleware-based context management:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;SummarizationCapability&lt;/code&gt;&lt;/strong&gt; -- triggers LLM summarization when thresholds are reached&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;SlidingWindowCapability&lt;/code&gt;&lt;/strong&gt; -- zero-cost alternative that discards oldest messages&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;LimitWarnerCapability&lt;/code&gt;&lt;/strong&gt; -- injects warnings when limits approach&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;code&gt;ContextManagerCapability&lt;/code&gt;&lt;/strong&gt; -- full package: token tracking, auto-compression, tool output truncation
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai_summarization.capability&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;ContextManagerCapability&lt;/span&gt;

&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;openai:gpt-4.1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;capabilities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;ContextManagerCapability&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;max_tokens&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;100_000&lt;/span&gt;&lt;span class="p"&gt;)],&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  pydantic-ai-backend
&lt;/h3&gt;

&lt;p&gt;Our filesystem toolkit became &lt;code&gt;ConsoleCapability&lt;/code&gt; -- bundling tools, instructions, and permission enforcement:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai_backends&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;ConsoleCapability&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai_backends.permissions&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;READONLY_RULESET&lt;/span&gt;

&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;openai:gpt-4.1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;capabilities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;ConsoleCapability&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;permissions&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;READONLY_RULESET&lt;/span&gt;&lt;span class="p"&gt;)],&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Key Takeaways
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;1. Delete your abstraction layers.&lt;/strong&gt; We removed thousands of lines of middleware code. The framework does it better now.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;2. Composition is free.&lt;/strong&gt; Multiple capabilities stack without you writing any merge logic.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;3. Agent specs change deployment.&lt;/strong&gt; Define agent behavior in YAML, deploy by changing a config file.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;4. The migration is mechanical.&lt;/strong&gt; For each middleware hook, there's a direct capability equivalent.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;5. Think in capabilities, not agents.&lt;/strong&gt; The old pattern: build a specialized agent. The new pattern: build a capability, attach it to any agent.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://ai.pydantic.dev/capabilities/" rel="noopener noreferrer"&gt;Pydantic AI capabilities docs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://ai.pydantic.dev/hooks/" rel="noopener noreferrer"&gt;Pydantic AI hooks docs&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/vstorm-co/pydantic-ai-middleware" rel="noopener noreferrer"&gt;pydantic-ai-shields&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/vstorm-co/pydantic-ai-subagents" rel="noopener noreferrer"&gt;pydantic-ai-subagents&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/vstorm-co/pydantic-ai-summarization" rel="noopener noreferrer"&gt;pydantic-ai-summarization&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/vstorm-co/pydantic-ai-backend" rel="noopener noreferrer"&gt;pydantic-ai-backend&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/vstorm-co/full-stack-fastapi-nextjs-llm-template" rel="noopener noreferrer"&gt;full-stack AI agent template&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
    </item>
    <item>
      <title>Full RAG Pipeline: 4 Vector Stores, Hybrid Search, and Reranking in One Template</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Wed, 25 Mar 2026 01:25:12 +0000</pubDate>
      <link>https://dev.to/deenuu1/full-rag-pipeline-4-vector-stores-hybrid-search-and-reranking-in-one-template-1ef0</link>
      <guid>https://dev.to/deenuu1/full-rag-pipeline-4-vector-stores-hybrid-search-and-reranking-in-one-template-1ef0</guid>
      <description>&lt;h1&gt;
  
  
  We Added Full RAG to Our Open-Source AI Template: 4 Vector Stores, Hybrid Search, and Reranking
&lt;/h1&gt;

&lt;p&gt;&lt;em&gt;One template, every RAG decision already made — from vector store to reranking strategy.&lt;/em&gt;&lt;/p&gt;




&lt;p&gt;You know the drill. You want to add RAG to your AI app. So you start: pick a vector database, write an embedding pipeline, figure out chunking, wire up retrieval, add it to your agent as a tool, build a frontend to manage documents...&lt;/p&gt;

&lt;p&gt;Three weeks later you have a working prototype. Then someone asks "can we try Qdrant instead of Milvus?" and you realize your vector store is hardcoded in 14 places.&lt;/p&gt;

&lt;p&gt;We just shipped v0.2.2 of our open-source full-stack AI template, and RAG was the biggest addition. Not a toy demo — a production pipeline with 4 vector stores, 4 embedding providers, hybrid search, reranking, document versioning, and a management dashboard. All configurable. All swappable.&lt;/p&gt;

&lt;p&gt;Here's what we built and why.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm Kacper, AI Engineer at &lt;a href="https://vstorm.co" rel="noopener noreferrer"&gt;Vstorm&lt;/a&gt; — an Applied Agentic AI Engineering Consultancy. We've shipped 30+ production AI agent implementations and open-source our tooling at &lt;a href="https://github.com/vstorm-co" rel="noopener noreferrer"&gt;github.com/vstorm-co&lt;/a&gt;. Connect with me on &lt;a href="https://www.linkedin.com/in/kacper-wlodarczyk/" rel="noopener noreferrer"&gt;LinkedIn&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  The Architecture: 5 Steps, Every One Configurable
&lt;/h2&gt;

&lt;p&gt;Every RAG system does the same thing: &lt;strong&gt;parse → chunk → embed → store → search&lt;/strong&gt;. The difference is how many decisions you have to make at each step.&lt;/p&gt;

&lt;p&gt;In our template, each step is a pluggable abstraction:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Document Upload
  │
  ├── Parse: PyMuPDF (default) | LlamaParse (130+ formats) | python-docx
  │
  ├── Chunk: recursive (default) | markdown | fixed
  │     └── chunk_size=512, overlap=50 (configurable via env vars)
  │
  ├── Embed: OpenAI | Voyage | Gemini (multimodal) | SentenceTransformers (local)
  │     └── dimensions auto-derived from model name
  │
  ├── Store: Milvus | Qdrant | ChromaDB | pgvector
  │
  └── Search: vector | hybrid (BM25 + vector + RRF) | + reranking (Cohere | CrossEncoder)
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You pick your stack during project generation. The template wires everything up. No glue code.&lt;/p&gt;

&lt;h2&gt;
  
  
  4 Vector Stores, 1 Interface
&lt;/h2&gt;

&lt;p&gt;The biggest design decision was making vector stores swappable. We implemented &lt;code&gt;BaseVectorStore&lt;/code&gt; with four backends:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;BaseVectorStore&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ABC&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;insert_document&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Document&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;SearchResult&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;delete_document&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;document_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;get_collection_info&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;CollectionInfo&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Milvus&lt;/strong&gt; — production-grade, runs as 3 Docker services (etcd + MinIO + Milvus). Best for large-scale deployments. Cosine similarity with IVF_FLAT indexing.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Qdrant&lt;/strong&gt; — single Docker service, great balance of performance and simplicity. Our default recommendation for most teams.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;ChromaDB&lt;/strong&gt; — embedded mode, zero Docker required. Perfect for prototyping and local development. Just &lt;code&gt;pip install chromadb&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;pgvector&lt;/strong&gt; — uses your existing PostgreSQL. No new infrastructure. HNSW indexing. If you already have Postgres, this is the lowest-friction option.&lt;/p&gt;

&lt;p&gt;Switching between them? One environment variable:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# In your .env:&lt;/span&gt;
&lt;span class="nv"&gt;VECTOR_STORE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;qdrant    &lt;span class="c"&gt;# or: milvus, chromadb, pgvector&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The template handles connection strings, Docker services, schema creation, and index configuration automatically.&lt;/p&gt;

&lt;h2&gt;
  
  
  Hybrid Search: Why Vector-Only Isn't Enough
&lt;/h2&gt;

&lt;p&gt;Pure vector search works well for semantic queries ("documents about building safety"). It fails on exact matches ("find contract #2024-0847") because embeddings don't preserve exact strings.&lt;/p&gt;

&lt;p&gt;Our hybrid search combines both:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;retrieve&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="c1"&gt;# Step 1: Vector search (semantic)
&lt;/span&gt;    &lt;span class="n"&gt;raw_results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;limit&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;fetch_multiplier&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# Step 2: BM25 keyword search
&lt;/span&gt;    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_hybrid_enabled&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;bm25_results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_bm25_search&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;limit&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="n"&gt;fetch_multiplier&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;bm25_results&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="n"&gt;raw_results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_rrf_fuse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;raw_results&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;bm25_results&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# Step 3: Rerank (optional)
&lt;/span&gt;    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;should_rerank&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;rerank_service&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;results&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;rerank_service&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;rerank&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;raw_results&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;top_k&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;limit&lt;/span&gt; &lt;span class="o"&gt;*&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="n"&gt;limit&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The fusion uses &lt;strong&gt;Reciprocal Rank Fusion (RRF)&lt;/strong&gt; — a simple but effective algorithm that combines rankings from multiple sources:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="nd"&gt;@staticmethod&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_rrf_fuse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;vector_results&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;bm25_results&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;k&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;60&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;scores&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{}&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;rank&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;vector_results&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="n"&gt;scores&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;scores&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mf"&gt;1.0&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;rank&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;rank&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="nf"&gt;enumerate&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;bm25_results&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;key&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;r&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;[:&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
        &lt;span class="n"&gt;scores&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;scores&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;key&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mf"&gt;1.0&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;k&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;rank&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;sorted_by_score&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;scores&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Enable it with one env var: &lt;code&gt;RAG_HYBRID_SEARCH=true&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Reranking: The Quality Multiplier
&lt;/h2&gt;

&lt;p&gt;Initial retrieval casts a wide net. Reranking narrows it down. We support two options:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cohere Reranker&lt;/strong&gt; (API) — the fastest way to improve retrieval quality. Send your results + query, get them re-scored by a model trained specifically for relevance ranking:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;rerank&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;documents&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;rerank-v3.5&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;top_n&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;top_k&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;CrossEncoder&lt;/strong&gt; (local) — runs a SentenceTransformers cross-encoder model locally. No API calls, no data leaves your infrastructure:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;pairs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[[&lt;/span&gt;&lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;results&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;scores&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;predict&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;pairs&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  &lt;span class="c1"&gt;# Runs locally on CPU/GPU
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The pipeline is: retrieve 3× more results than needed → rerank → return top-k. This consistently improves precision without touching your embeddings or vector store.&lt;/p&gt;

&lt;h2&gt;
  
  
  Document Versioning: SHA256 Dedup
&lt;/h2&gt;

&lt;p&gt;Re-ingesting a document shouldn't create duplicates. Our pipeline uses content hashing:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;ingest_file&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;filepath&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;replace&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;document&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;processor&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;process_file&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;filepath&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# Check for existing version by source path or content hash
&lt;/span&gt;    &lt;span class="n"&gt;existing_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_find_existing_by_source&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;source_path&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="n"&gt;existing_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;existing_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_find_existing_by_hash&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;metadata&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content_hash&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="c1"&gt;# Replace old chunks with new ones
&lt;/span&gt;    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;existing_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;delete_document&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;existing_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;insert_document&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;collection_name&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;document&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Google Drive sync? Same logic — changed files get re-embedded, unchanged files skip.&lt;/p&gt;

&lt;h2&gt;
  
  
  4 Embedding Providers
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Provider&lt;/th&gt;
&lt;th&gt;Model&lt;/th&gt;
&lt;th&gt;Dimensions&lt;/th&gt;
&lt;th&gt;API Key?&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;OpenAI&lt;/td&gt;
&lt;td&gt;text-embedding-3-small&lt;/td&gt;
&lt;td&gt;1536&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Voyage&lt;/td&gt;
&lt;td&gt;voyage-3&lt;/td&gt;
&lt;td&gt;1024&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Gemini&lt;/td&gt;
&lt;td&gt;gemini-embedding-exp-03-07&lt;/td&gt;
&lt;td&gt;3072&lt;/td&gt;
&lt;td&gt;Yes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;SentenceTransformers&lt;/td&gt;
&lt;td&gt;all-MiniLM-L6-v2&lt;/td&gt;
&lt;td&gt;384&lt;/td&gt;
&lt;td&gt;No (local)&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Dimensions are auto-derived from the model name — no manual configuration:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;EMBEDDING_DIMENSIONS&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text-embedding-3-small&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1536&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;voyage-3&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;1024&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;gemini-embedding-exp-03-07&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;3072&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;all-MiniLM-L6-v2&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;384&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Gemini is the interesting one — it supports &lt;strong&gt;multimodal embeddings&lt;/strong&gt;. Text and images in the same vector space. We use it for image description extraction from PDFs.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Agent Integration
&lt;/h2&gt;

&lt;p&gt;RAG becomes an agent tool — &lt;code&gt;search_knowledge_base&lt;/code&gt; — available to all 5 AI frameworks (Pydantic AI, LangChain, LangGraph, CrewAI, DeepAgents):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;search_knowledge_base&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;collection&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;documents&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;collections&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  &lt;span class="c1"&gt;# Multi-collection search
&lt;/span&gt;    &lt;span class="n"&gt;top_k&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;int&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Search with automatic reranking &amp;amp; hybrid search if enabled.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Results include source attribution: filename, page number, chunk number, and similarity score. The agent's system prompt instructs it to cite sources with &lt;code&gt;[1]&lt;/code&gt;, &lt;code&gt;[2]&lt;/code&gt; references.&lt;/p&gt;

&lt;h2&gt;
  
  
  Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;RAG is a pipeline of 5 decisions&lt;/strong&gt; (parse, chunk, embed, store, search) — our template makes each one configurable without code changes&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Vector-only search misses exact matches&lt;/strong&gt; — hybrid (BM25 + vector + RRF) catches both semantic and keyword queries&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Reranking is the cheapest quality improvement&lt;/strong&gt; — 3× over-retrieve + rerank consistently beats tuning embeddings&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Document versioning prevents duplicate chunks&lt;/strong&gt; — SHA256 content hash + source path tracking&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;One env var switches everything&lt;/strong&gt; — &lt;code&gt;VECTOR_STORE=pgvector&lt;/code&gt;, &lt;code&gt;RAG_HYBRID_SEARCH=true&lt;/code&gt;, &lt;code&gt;EMBEDDING_MODEL=voyage-3&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  Try it yourself
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://github.com/vstorm-co/full-stack-ai-agent-template" rel="noopener noreferrer"&gt;full-stack-ai-agent-template&lt;/a&gt; — generates production-ready FastAPI + Next.js AI apps with full RAG pipeline&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;fastapi-fullstack
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Related:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://oss.vstorm.co/tools/ai-agent-configurator/" rel="noopener noreferrer"&gt;AI Agent Configurator&lt;/a&gt; — configure 75+ options visually, download as ZIP&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://oss.vstorm.co/guides/" rel="noopener noreferrer"&gt;Step-by-step guides&lt;/a&gt; — 50 tutorials across 5 frameworks&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;More from Vstorm's open-source ecosystem:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://oss.vstorm.co" rel="noopener noreferrer"&gt;All our open-source projects&lt;/a&gt; — 13 packages for the Pydantic AI ecosystem&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/vstorm-co/awesome-pydantic-ai" rel="noopener noreferrer"&gt;awesome-pydantic-ai&lt;/a&gt; — curated list of Pydantic AI resources and tools&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://vstorm.co" rel="noopener noreferrer"&gt;vstorm.co&lt;/a&gt; — our consultancy (30+ AI agent implementations)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If this was useful, follow me on &lt;a href="https://www.linkedin.com/in/kacper-wlodarczyk/" rel="noopener noreferrer"&gt;LinkedIn&lt;/a&gt; for daily AI agent insights.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>ai</category>
      <category>programming</category>
      <category>beginners</category>
    </item>
    <item>
      <title>It's been a while since my last post, but I'm back with new content for you.</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Tue, 17 Mar 2026 12:38:26 +0000</pubDate>
      <link>https://dev.to/deenuu1/its-been-a-while-since-my-last-post-but-im-back-with-new-content-for-you-24bo</link>
      <guid>https://dev.to/deenuu1/its-been-a-while-since-my-last-post-but-im-back-with-new-content-for-you-24bo</guid>
      <description>&lt;div class="ltag__link"&gt;
  &lt;a href="/deenuu1" class="ltag__link__link"&gt;
    &lt;div class="ltag__link__pic"&gt;
      &lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F997289%2F50074490-7c28-44da-9a80-f389f20d3691.jpeg" alt="deenuu1"&gt;
    &lt;/div&gt;
  &lt;/a&gt;
  &lt;a href="https://dev.to/deenuu1/from-0-to-production-ai-agent-in-30-minutes-full-stack-template-with-5-ai-frameworks-3b4o" class="ltag__link__link"&gt;
    &lt;div class="ltag__link__content"&gt;
      &lt;h2&gt;From 0 to Production AI Agent in 30 Minutes — Full-Stack Template with 5 AI Frameworks&lt;/h2&gt;
      &lt;h3&gt;Kacper Włodarczyk ・ Mar 17&lt;/h3&gt;
      &lt;div class="ltag__link__taglist"&gt;
        &lt;span class="ltag__link__tag"&gt;#ai&lt;/span&gt;
        &lt;span class="ltag__link__tag"&gt;#programming&lt;/span&gt;
        &lt;span class="ltag__link__tag"&gt;#python&lt;/span&gt;
        &lt;span class="ltag__link__tag"&gt;#softwareengineering&lt;/span&gt;
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/a&gt;
&lt;/div&gt;


</description>
      <category>ai</category>
      <category>programming</category>
      <category>python</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>From 0 to Production AI Agent in 30 Minutes — Full-Stack Template with 5 AI Frameworks</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Tue, 17 Mar 2026 12:37:41 +0000</pubDate>
      <link>https://dev.to/deenuu1/from-0-to-production-ai-agent-in-30-minutes-full-stack-template-with-5-ai-frameworks-3b4o</link>
      <guid>https://dev.to/deenuu1/from-0-to-production-ai-agent-in-30-minutes-full-stack-template-with-5-ai-frameworks-3b4o</guid>
      <description>&lt;p&gt;Every AI project starts the same way.&lt;/p&gt;

&lt;p&gt;You need a FastAPI backend. Then authentication — JWT tokens, refresh logic, user management. Then a database — PostgreSQL, migrations, async connections. Then WebSocket streaming for real-time AI responses. Then a frontend — Next.js, state management, chat UI. Then Docker. Then CI/CD.&lt;/p&gt;

&lt;p&gt;Three days of boilerplate before you write a single line of AI code.&lt;/p&gt;

&lt;p&gt;I've set up this stack from scratch more times than I'd like to admit. After the third project where I copy-pasted the same auth middleware, the same WebSocket handler, the same Docker Compose config — I decided to build a generator that does all of it in one command.&lt;/p&gt;

&lt;p&gt;The result: [[full-stack-ai-agent-template]] — an open-source full-stack template with 5 AI frameworks, 75+ configuration options, and a web configurator that generates your entire project in minutes.&lt;/p&gt;

&lt;p&gt;614 stars on GitHub. Used by teams at NVIDIA, Pfizer, TikTok, and others. And you can go from zero to a running production AI agent in about 30 minutes.&lt;/p&gt;

&lt;p&gt;Let me walk you through exactly how.&lt;/p&gt;




&lt;p&gt;&lt;em&gt;I'm Kacper, AI Engineer at &lt;a href="https://vstorm.co" rel="noopener noreferrer"&gt;Vstorm&lt;/a&gt; — an Applied Agentic AI Engineering Consultancy. We've shipped 30+ production AI agent implementations and open-source our tooling at &lt;a href="https://github.com/vstorm-co" rel="noopener noreferrer"&gt;github.com/vstorm-co&lt;/a&gt;. Connect with me on &lt;a href="https://www.linkedin.com/in/kacper-wlodarczyk/" rel="noopener noreferrer"&gt;LinkedIn&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;




&lt;h2&gt;
  
  
  Step 1: Open the Web Configurator
&lt;/h2&gt;

&lt;p&gt;Go to &lt;a href="https://oss.vstorm.co/full-stack-ai-agent-template/configurator/" rel="noopener noreferrer"&gt;oss.vstorm.co/full-stack-ai-agent-template/configurator/&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;No CLI installation needed. No pip. Just a browser.&lt;/p&gt;

&lt;p&gt;The configurator gives you a visual interface to pick every option for your project. Database, auth, AI framework, background tasks, observability, frontend — all of it. You see the full config before you generate anything.&lt;/p&gt;

&lt;p&gt;Alternatively, if you prefer the terminal:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;fastapi-fullstack
fastapi-fullstack
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This launches the interactive wizard that walks you through the same options.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 2: Pick a Preset (or Go Custom)
&lt;/h2&gt;

&lt;p&gt;The template ships with three presets that cover the most common use cases:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Preset&lt;/th&gt;
&lt;th&gt;What you get&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--minimal&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Bare FastAPI app — no database, no auth, no extras&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--preset ai-agent&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;PostgreSQL + JWT auth + AI agent + WebSocket streaming + conversation persistence + Redis&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;--preset production&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Full production setup — Redis, caching, rate limiting, Sentry, Prometheus, Kubernetes&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;For this walkthrough, I'll use the &lt;strong&gt;AI Agent&lt;/strong&gt; preset with Pydantic AI — the most common starting point for AI applications:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;fastapi-fullstack create my_ai_app &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--preset&lt;/span&gt; ai-agent &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--ai-framework&lt;/span&gt; pydantic_ai &lt;span class="se"&gt;\&lt;/span&gt;
  &lt;span class="nt"&gt;--frontend&lt;/span&gt; nextjs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That single command generates a full-stack project with:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;FastAPI backend with async PostgreSQL&lt;/li&gt;
&lt;li&gt;JWT authentication with user management&lt;/li&gt;
&lt;li&gt;Pydantic AI agent with WebSocket streaming&lt;/li&gt;
&lt;li&gt;Conversation persistence (chat history saved to DB)&lt;/li&gt;
&lt;li&gt;Redis for caching and sessions&lt;/li&gt;
&lt;li&gt;Next.js 15 frontend with React 19 and Tailwind CSS v4&lt;/li&gt;
&lt;li&gt;Docker Compose for the full stack&lt;/li&gt;
&lt;li&gt;GitHub Actions CI/CD&lt;/li&gt;
&lt;li&gt;Logfire observability&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Step 3: Look at What You Got
&lt;/h2&gt;

&lt;p&gt;The generated project follows a clean layered architecture — Repository + Service pattern, inspired by real production codebases:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;my_ai_app/
├── backend/
│   ├── app/
│   │   ├── main.py              # FastAPI app with lifespan
│   │   ├── api/routes/v1/       # Versioned API endpoints
│   │   ├── core/                # Config, security, middleware
│   │   ├── db/models/           # SQLAlchemy models
│   │   ├── schemas/             # Pydantic schemas
│   │   ├── repositories/        # Data access layer
│   │   ├── services/            # Business logic
│   │   ├── agents/              # AI agents (this is where your code goes)
│   │   └── commands/            # Django-style CLI commands
│   ├── cli/                     # Project CLI
│   ├── tests/                   # pytest test suite
│   └── alembic/                 # Database migrations
├── frontend/
│   ├── src/
│   │   ├── app/                 # Next.js App Router
│   │   ├── components/          # React components (chat UI included)
│   │   ├── hooks/               # useChat, useWebSocket
│   │   └── stores/              # Zustand state management
├── docker-compose.yml
├── Makefile
├── CLAUDE.md                    # AI coding assistant context
└── AGENTS.md                    # Multi-agent project guide
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Notice the &lt;code&gt;CLAUDE.md&lt;/code&gt; and &lt;code&gt;AGENTS.md&lt;/code&gt; files — the generated project is optimized for AI coding assistants like Claude Code, Cursor, and Copilot. It follows progressive disclosure best practices so your AI assistant understands the project structure immediately.&lt;/p&gt;

&lt;h2&gt;
  
  
  Step 4: Start Everything with Docker
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;cd &lt;/span&gt;my_ai_app
make docker-up        &lt;span class="c"&gt;# Backend + PostgreSQL + Redis&lt;/span&gt;
make docker-frontend  &lt;span class="c"&gt;# Next.js frontend&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's it. Two commands. The entire stack is running:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;API&lt;/strong&gt;: &lt;a href="http://localhost:8000" rel="noopener noreferrer"&gt;http://localhost:8000&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;API Docs&lt;/strong&gt;: &lt;a href="http://localhost:8000/docs" rel="noopener noreferrer"&gt;http://localhost:8000/docs&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Frontend&lt;/strong&gt;: &lt;a href="http://localhost:3000" rel="noopener noreferrer"&gt;http://localhost:3000&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Admin Panel&lt;/strong&gt;: &lt;a href="http://localhost:8000/admin" rel="noopener noreferrer"&gt;http://localhost:8000/admin&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you prefer running without Docker, the template generates a Makefile with shortcuts:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;make &lt;span class="nb"&gt;install&lt;/span&gt;       &lt;span class="c"&gt;# Install Python + Node dependencies&lt;/span&gt;
make docker-db     &lt;span class="c"&gt;# Start just PostgreSQL&lt;/span&gt;
make db-migrate    &lt;span class="c"&gt;# Create initial migration&lt;/span&gt;
make db-upgrade    &lt;span class="c"&gt;# Apply migrations&lt;/span&gt;
make create-admin  &lt;span class="c"&gt;# Create admin user&lt;/span&gt;
make run           &lt;span class="c"&gt;# Start backend&lt;/span&gt;
&lt;span class="nb"&gt;cd &lt;/span&gt;frontend &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; bun dev  &lt;span class="c"&gt;# Start frontend&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 5: Your AI Agent Is Already Working
&lt;/h2&gt;

&lt;p&gt;Open &lt;code&gt;http://localhost:3000&lt;/code&gt;, log in, and start chatting. The AI agent is already wired up — WebSocket streaming, conversation history, tool calls — all functional out of the box.&lt;/p&gt;

&lt;p&gt;Here's what the generated agent looks like:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# app/agents/assistant.py
&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pydantic_ai&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;RunContext&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;dataclasses&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;dataclass&lt;/span&gt;

&lt;span class="nd"&gt;@dataclass&lt;/span&gt;
&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Deps&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;
    &lt;span class="n"&gt;db&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;AsyncSession&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;

&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;Deps&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;openai:gpt-4o-mini&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;system_prompt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;You are a helpful assistant.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="nd"&gt;@agent.tool&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;search_database&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;RunContext&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;Deps&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;query&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;list&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;dict&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Search the database for relevant information.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="c1"&gt;# Access user context and database via ctx.deps
&lt;/span&gt;    &lt;span class="bp"&gt;...&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Type-safe. Dependency injection built in. Tool calling with full context access. This isn't a toy example — it's the same pattern we use in production at [[Vstorm]].&lt;/p&gt;

&lt;p&gt;The WebSocket endpoint handles streaming automatically:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="nd"&gt;@router.websocket&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;/ws&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;agent_ws&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;websocket&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;WebSocket&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;websocket&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;accept&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;

    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;agent&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;stream&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;user_input&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;websocket&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_json&lt;/span&gt;&lt;span class="p"&gt;({&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;text_delta&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;content&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;event&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;content&lt;/span&gt;
        &lt;span class="p"&gt;})&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Step 6: Customize the AI Layer
&lt;/h2&gt;

&lt;p&gt;Here's the key insight: &lt;strong&gt;everything except the AI agent is production-ready infrastructure that you don't need to touch&lt;/strong&gt;. Auth works. Database works. Streaming works. Frontend works.&lt;/p&gt;

&lt;p&gt;You modify one directory: &lt;code&gt;app/agents/&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Want to change from OpenAI to Anthropic? Update the model string:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;agent&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Agent&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;Deps&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;](&lt;/span&gt;
    &lt;span class="n"&gt;model&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;anthropic:claude-sonnet-4-5&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;system_prompt&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;You are a helpful assistant.&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Want to add a tool? Add a function:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="nd"&gt;@agent.tool&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;get_weather&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;ctx&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;RunContext&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="n"&gt;Deps&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="n"&gt;city&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;Get current weather for a city.&lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;
    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;httpx&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;AsyncClient&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;resp&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="k"&gt;await&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://api.weather.com/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;city&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;resp&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;json&lt;/span&gt;&lt;span class="p"&gt;()[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;summary&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Want to switch to LangChain or CrewAI entirely? Regenerate the project with a different &lt;code&gt;--ai-framework&lt;/code&gt; flag. The rest of the stack stays the same.&lt;/p&gt;

&lt;h2&gt;
  
  
  5 AI Frameworks, One Template
&lt;/h2&gt;

&lt;p&gt;The template supports five AI frameworks, all with the same backend infrastructure:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Framework&lt;/th&gt;
&lt;th&gt;Best for&lt;/th&gt;
&lt;th&gt;Observability&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Pydantic AI&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Type-safe agents, dependency injection&lt;/td&gt;
&lt;td&gt;Logfire&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;LangChain&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Chains, existing LangChain tooling&lt;/td&gt;
&lt;td&gt;LangSmith&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;LangGraph&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Complex multi-step workflows, ReAct agents&lt;/td&gt;
&lt;td&gt;LangSmith&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;CrewAI&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Multi-agent crews, role-based agents&lt;/td&gt;
&lt;td&gt;LangSmith&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;DeepAgents&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Claude Code-style agentic coding, HITL&lt;/td&gt;
&lt;td&gt;LangSmith&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;You pick the framework when generating the project. The WebSocket streaming, conversation persistence, auth, and frontend all work the same way regardless of which framework you choose.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Generate with LangGraph&lt;/span&gt;
fastapi-fullstack create my_app &lt;span class="nt"&gt;--preset&lt;/span&gt; ai-agent &lt;span class="nt"&gt;--ai-framework&lt;/span&gt; langgraph &lt;span class="nt"&gt;--frontend&lt;/span&gt; nextjs

&lt;span class="c"&gt;# Generate with CrewAI&lt;/span&gt;
fastapi-fullstack create my_app &lt;span class="nt"&gt;--preset&lt;/span&gt; ai-agent &lt;span class="nt"&gt;--ai-framework&lt;/span&gt; crewai &lt;span class="nt"&gt;--frontend&lt;/span&gt; nextjs
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  75+ Configuration Options
&lt;/h2&gt;

&lt;p&gt;Beyond AI frameworks, the template covers the full spectrum of production needs:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Databases:&lt;/strong&gt; PostgreSQL (async), MongoDB (async), SQLite&lt;br&gt;
&lt;strong&gt;ORMs:&lt;/strong&gt; SQLAlchemy, SQLModel&lt;br&gt;
&lt;strong&gt;Auth:&lt;/strong&gt; JWT + refresh tokens, API keys, Google OAuth&lt;br&gt;
&lt;strong&gt;Background tasks:&lt;/strong&gt; Celery, Taskiq, ARQ&lt;br&gt;
&lt;strong&gt;Observability:&lt;/strong&gt; Logfire, LangSmith, Sentry, Prometheus&lt;br&gt;
&lt;strong&gt;Infrastructure:&lt;/strong&gt; Docker, Kubernetes, GitHub Actions, GitLab CI, Traefik, Nginx&lt;br&gt;
&lt;strong&gt;Frontend:&lt;/strong&gt; Next.js 15 with React 19, TypeScript, Tailwind CSS v4, dark mode, i18n&lt;br&gt;
&lt;strong&gt;Extras:&lt;/strong&gt; Redis caching, rate limiting, SQLAdmin panel, webhooks, S3 file storage, RAG with Milvus&lt;/p&gt;

&lt;p&gt;Every option is a boolean flag. No Jinja template hacking. No post-generation cleanup. The generator produces clean code that only includes what you selected.&lt;/p&gt;
&lt;h2&gt;
  
  
  Key Takeaways
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;The web configurator at &lt;a href="https://oss.vstorm.co/full-stack-ai-agent-template/configurator/" rel="noopener noreferrer"&gt;oss.vstorm.co&lt;/a&gt; lets you visually configure and download a full-stack AI project&lt;/strong&gt; — no CLI needed.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Three presets (minimal, ai-agent, production) cover 90% of use cases&lt;/strong&gt; — customize from there.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;5 AI frameworks share the same infrastructure&lt;/strong&gt; — switch frameworks without rewriting your backend.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;The generated code is production-grade, not a prototype&lt;/strong&gt; — layered architecture, async everywhere, type-safe.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;You modify &lt;code&gt;app/agents/&lt;/code&gt; and nothing else&lt;/strong&gt; — auth, streaming, persistence, frontend are done.&lt;/li&gt;
&lt;/ul&gt;


&lt;h2&gt;
  
  
  Try it yourself
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://github.com/vstorm-co/full-stack-ai-agent-template" rel="noopener noreferrer"&gt;full-stack-ai-agent-template&lt;/a&gt; — Production-ready full-stack AI agent template with 5 frameworks and 75+ options.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;fastapi-fullstack
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Or use the &lt;a href="https://oss.vstorm.co/full-stack-ai-agent-template/configurator/" rel="noopener noreferrer"&gt;Web Configurator&lt;/a&gt; — no installation needed.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;More from Vstorm's open-source ecosystem:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;a href="https://oss.vstorm.co" rel="noopener noreferrer"&gt;All our open-source projects&lt;/a&gt; — 13 packages for the Pydantic AI ecosystem&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://github.com/vstorm-co/awesome-pydantic-ai" rel="noopener noreferrer"&gt;awesome-pydantic-ai&lt;/a&gt; — curated list of Pydantic AI resources and tools&lt;/li&gt;
&lt;li&gt;
&lt;a href="https://vstorm.co" rel="noopener noreferrer"&gt;vstorm.co&lt;/a&gt; — our consultancy (30+ AI agent implementations)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If this was useful, follow me on &lt;a href="https://www.linkedin.com/in/kacper-wlodarczyk/" rel="noopener noreferrer"&gt;LinkedIn&lt;/a&gt; for daily AI agent insights.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>programming</category>
      <category>python</category>
      <category>softwareengineering</category>
    </item>
    <item>
      <title>Pydantic-DeepAgents: A Lightweight, Production-Ready Framework for Building Autonomous AI Agents</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Mon, 22 Dec 2025 01:19:27 +0000</pubDate>
      <link>https://dev.to/deenuu1/pydantic-deepagents-a-lightweight-production-ready-framework-for-building-autonomous-ai-agents-2l3i</link>
      <guid>https://dev.to/deenuu1/pydantic-deepagents-a-lightweight-production-ready-framework-for-building-autonomous-ai-agents-2l3i</guid>
      <description>&lt;p&gt;&lt;em&gt;Inspired by LangChain deepagents — but simpler, type-safe, and with Docker sandboxing built-in&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;In 2025, autonomous AI agents are no longer just research prototypes — they’re powering real-world automation, code generation tools, data pipelines, and intelligent assistants. However, many popular agent frameworks come with heavy dependencies, complex graphs, and a steep learning curve that makes production deployment challenging.&lt;/p&gt;

&lt;p&gt;That’s why we at &lt;strong&gt;Vstorm&lt;/strong&gt; built &lt;strong&gt;Pydantic-DeepAgents&lt;/strong&gt; — a minimal yet powerful open-source framework that extends &lt;strong&gt;Pydantic-AI&lt;/strong&gt; with everything you need to create reliable, production-grade agents.&lt;/p&gt;

&lt;p&gt;GitHub repository: &lt;a href="https://github.com/vstorm-co/pydantic-deepagents" rel="noopener noreferrer"&gt;https://github.com/vstorm-co/pydantic-deepagents&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fz36u3scp2j9q9veuk8o6.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fz36u3scp2j9q9veuk8o6.png" alt=" " width="800" height="409"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  What makes Pydantic-DeepAgents different?
&lt;/h3&gt;

&lt;p&gt;We were heavily inspired by LangChain’s excellent &lt;a href="https://github.com/langchain-ai/deepagents" rel="noopener noreferrer"&gt;deepagents&lt;/a&gt; project — a clean implementation of “deep agent” patterns including planning loops, tool calling, subagent delegation, and human-in-the-loop workflows.&lt;/p&gt;

&lt;p&gt;Instead of reinventing the wheel, we asked: &lt;em&gt;What if we built the same powerful patterns, but fully in the Pydantic-AI ecosystem?&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The result is a framework that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Keeps dependencies lightweight (no LangGraph, no massive ecosystem)&lt;/li&gt;
&lt;li&gt;Leverages Pydantic’s native type-safety and validation for structured outputs&lt;/li&gt;
&lt;li&gt;Adds production-focused features missing from many alternatives&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  Core Features
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Planning &amp;amp; Reasoning&lt;/strong&gt; — TodoToolset for autonomous task breakdown and self-correction&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Filesystem Access&lt;/strong&gt; — Full read/write operations with FilesystemToolset&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Subagent Delegation&lt;/strong&gt; — Break complex tasks into specialized subagents (SubAgentToolset)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Extensible Skills System&lt;/strong&gt; — Define new agent capabilities with simple Markdown prompts (perfect for rapid iteration)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Multiple Backends&lt;/strong&gt; — In-memory, persistent filesystem, secure &lt;strong&gt;DockerSandbox&lt;/strong&gt; (isolated code execution), and CompositeBackend&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;File Uploads&lt;/strong&gt; — Seamless processing of uploaded files via &lt;code&gt;run_with_files()&lt;/code&gt; or &lt;code&gt;deps.upload_file()&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Context Management&lt;/strong&gt; — Automatic summarization for long-running conversations&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Human-in-the-Loop&lt;/strong&gt; — Built-in confirmation workflows for critical actions&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Streaming Support&lt;/strong&gt; — Token-by-token responses for responsive UIs&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Structured Outputs&lt;/strong&gt; — Type-safe Pydantic models via &lt;code&gt;output_type&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  See It in Action
&lt;/h3&gt;

&lt;p&gt;We’ve included a complete full-stack demo application (FastAPI backend + streaming web UI) that demonstrates:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Live agent reasoning traces&lt;/li&gt;
&lt;li&gt;File uploads and processing&lt;/li&gt;
&lt;li&gt;Human approval steps&lt;/li&gt;
&lt;li&gt;Streaming responses&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Demo app: &lt;a href="https://github.com/vstorm-co/pydantic-deepagents/tree/main/examples/full_app" rel="noopener noreferrer"&gt;https://github.com/vstorm-co/pydantic-deepagents/tree/main/examples/full_app&lt;/a&gt;&lt;br&gt;&lt;br&gt;
Quick video walkthrough: &lt;a href="https://drive.google.com/file/d/1hqgXkbAgUrsKOWpfWdF48cqaxRht-8od/view?usp=sharing" rel="noopener noreferrer"&gt;https://drive.google.com/file/d/1hqgXkbAgUrsKOWpfWdF48cqaxRht-8od/view?usp=sharing&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  When to choose Pydantic-DeepAgents?
&lt;/h3&gt;

&lt;p&gt;Choose it when you want:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A clean, maintainable agent architecture without framework bloat&lt;/li&gt;
&lt;li&gt;Strong guarantees around data validation and structured responses&lt;/li&gt;
&lt;li&gt;Secure execution (Docker sandbox out of the box)&lt;/li&gt;
&lt;li&gt;Fast prototyping with Markdown-defined skills&lt;/li&gt;
&lt;li&gt;Easy deployment in production environments&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;It’s particularly great if you’re already using Pydantic-AI, prefer minimalism, or need agents that interact safely with files and external tools.&lt;/p&gt;

&lt;h3&gt;
  
  
  Get Started Today
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;pydantic-deep
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Check out the repository, star it if you find it useful, and feel free to open issues or PRs — we’d love contributions!&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/vstorm-co/pydantic-deepagents" rel="noopener noreferrer"&gt;https://github.com/vstorm-co/pydantic-deepagents&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;We’re excited to see what you build with it.&lt;/p&gt;

&lt;p&gt;— Team at Vstorm (&lt;a href="https://vstorm.co" rel="noopener noreferrer"&gt;https://vstorm.co&lt;/a&gt;)&lt;/p&gt;

</description>
      <category>programming</category>
      <category>python</category>
      <category>langchain</category>
      <category>agents</category>
    </item>
    <item>
      <title>[Python] How to get full url from shortened url</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Thu, 16 Nov 2023 00:59:32 +0000</pubDate>
      <link>https://dev.to/deenuu1/python-how-to-get-full-url-from-shortened-url-24no</link>
      <guid>https://dev.to/deenuu1/python-how-to-get-full-url-from-shortened-url-24no</guid>
      <description>&lt;p&gt;Sometimes when you scrape a website, you may have encountered the fact that the website returns shortened URLs to sources from other websites.&lt;/p&gt;

&lt;p&gt;As in this case, for example, &lt;code&gt;https://upflix.pl/r/Qb64Ar&lt;/code&gt; this link consists of a domain and some random characters. The way a shortened link works is that it redirects you to another page. Therefore, the &lt;code&gt;status_code&lt;/code&gt; that our query returns is &lt;code&gt;302&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Sometimes it happens that we need a full URL to get that can do this with a few lines of Python code and the requests library.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;requests
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We will use the &lt;code&gt;head&lt;/code&gt; method to perform this function&lt;br&gt;
This method is similar to &lt;code&gt;get&lt;/code&gt; with the difference that it does not return any content, only headers.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;head&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;short_url&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;After executing the query, we can check the headers that were returned.&lt;br&gt;
There is information here such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;date&lt;/li&gt;
&lt;li&gt;type of website content&lt;/li&gt;
&lt;li&gt;character encoding&lt;/li&gt;
&lt;li&gt;FULL LINK
and many other information you can see below.
&lt;/li&gt;
&lt;/ul&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s"&gt;'Date'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'Thu, 16 Nov 2023 00:43:13 GMT'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'Content-Type'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'text/html; charset=UTF-8'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'Connection'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'keep-alive'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'location'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'https://www.imdb.com/title/tt14060708/'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'vary'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'Origin'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'x-powered-by'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'PHP/7.3.33'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'x-frame-options'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'SAMEORIGIN'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'CF-Cache-Status'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'DYNAMIC'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'Report-To'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'{"endpoints":[{"url":"https:&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s"&gt;/a.nel.cloudflare.com&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s"&gt;/report&lt;/span&gt;&lt;span class="se"&gt;\\&lt;/span&gt;&lt;span class="s"&gt;/v3?s=bgvCMcMQg1ZkjanlgqzemKUHHthalhb%2FAT72Q58O8a22eFmkeb%2FyeeIMfKkGFwt8WmkMB6dv28F1G2CdH134Kilk%2BcdQNweIZ3O%2FN9KlQf1A2VF%2Bm3yYT89rvjU%3D"}],"group":"cf-nel","max_age":604800}'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'NEL'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'{"success_fraction":0,"report_to":"cf-nel","max_age":604800}'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'Strict-Transport-Security'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'max-age=15552000; includeSubDomains; preload'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'X-Content-Type-Options'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'nosniff'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'Server'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'cloudflare'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'CF-RAY'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'826bb2ce597ebfda-WAW'&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;'alt-svc'&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;'h3=":443"; ma=86400'&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  Full code
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="nn"&gt;requests&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="nn"&gt;typing&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Optional&lt;/span&gt; 

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;get_full_url&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;short_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;Optional&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;head&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;short_url&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;status_code&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;302&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;headers&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;headers&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;headers&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"location"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt; 

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



</description>
      <category>python</category>
      <category>tutorial</category>
      <category>beginners</category>
      <category>learning</category>
    </item>
    <item>
      <title>Best Youtube channels about Python</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Fri, 03 Nov 2023 14:41:21 +0000</pubDate>
      <link>https://dev.to/deenuu1/best-youtube-channels-about-python-585m</link>
      <guid>https://dev.to/deenuu1/best-youtube-channels-about-python-585m</guid>
      <description>&lt;p&gt;Hi, in today's article, I would like to show you the list of Youtube channels about Python programming which I enjoy watching. If you know other channels not presented here, please share them in the comments.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The list is presented in random order!&lt;/strong&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  1. Pixegami
&lt;/h1&gt;

&lt;p&gt;&lt;a href="https://www.youtube.com/@pixegami/featured"&gt;https://www.youtube.com/@pixegami/featured&lt;/a&gt;&lt;br&gt;
On this channel, you can find a complete Python course from scratch, tutorials on AWS, Docker, FastAPI, and much more. The materials are of a high standard and everything is well presented. There aren't many movies there, but you'll find something for yourself.&lt;br&gt;
I recommend this movie - &lt;a href="https://www.youtube.com/watch?v=Hiabp1GY8fA"&gt;https://www.youtube.com/watch?v=Hiabp1GY8fA&lt;/a&gt; thanks to it, I easily created a simple website for a client, bought my first domain, and got to know the Vercel website&lt;/p&gt;

&lt;h2&gt;
  
  
  2. ArjanCodes
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.youtube.com/@ArjanCodes/videos"&gt;https://www.youtube.com/@ArjanCodes/videos&lt;/a&gt;&lt;br&gt;
This channel has almost 300 videos and you can find many Python topics here.&lt;br&gt;
I especially recommend older videos on software architecture and the "Code Roasts" series.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Coding Crashcourses
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.youtube.com/@codingcrashcourses8533/featured"&gt;https://www.youtube.com/@codingcrashcourses8533/featured&lt;/a&gt;&lt;br&gt;
This is a channel created for fans of all language models. Here you will find a lot of guides about the LangChain library and something about FastAPI&lt;/p&gt;

&lt;h2&gt;
  
  
  4. NeetCode &amp;amp; NeetCodeIO
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.youtube.com/@NeetCode/videos"&gt;https://www.youtube.com/@NeetCode/videos&lt;/a&gt;&lt;br&gt;
&lt;a href="https://www.youtube.com/@NeetCodeIO"&gt;https://www.youtube.com/@NeetCodeIO&lt;/a&gt;&lt;br&gt;
Currently one of my favorites, you will find a lot of videos about algorithms and data structures here. If you have a problem with a task on Leetcode, check if it has not already been discussed on this channel.&lt;br&gt;
I also recommend visiting his website - &lt;a href="https://www.youtube.com/redirect?event=channel_header&amp;amp;redir_token=QUFFLUhqbXZKblNJMlBKcl9JdjAyQWh5Nzg3YmpybUpzUXxBQ3Jtc0tuTGc2R3IwZVJFVEE0U2pwZWFLVXpQdlpqQ2NaSEZjY3ctZG1hVWZQbE1kQThObmFucHBrMWNtMWl2R3ROcV8wQlF6TkU3cno4bHBNeVZ5YmhWSjZJZzB4WW4xWWdXZ0xkOURZTmM3clJKWjJVckdhSQ&amp;amp;q=https%3A%2F%2Fneetcode.io"&gt;neetcode.io&lt;/a&gt;&lt;a href="https://www.youtube.com/@NeetCodeIO/about"&gt;&lt;/a&gt; where you will find a roadmap for learning algorithms through tasks on leetcode.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Indently
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.youtube.com/@Indently/videos"&gt;https://www.youtube.com/@Indently/videos&lt;/a&gt;&lt;br&gt;
This is my new discovery, the channel has 763 and you can find videos about many programming languages ​​here. Currently, I am starting a series about a not-yet-very popular framework - Flute&lt;/p&gt;

&lt;h2&gt;
  
  
  6. thenewboston
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.youtube.com/@thenewboston/featured"&gt;https://www.youtube.com/@thenewboston/featured&lt;/a&gt;&lt;br&gt;
From this channel, I highly recommend the "Python and React" and "Pro Django" series. If you want to deepen your knowledge of Django, this is a great place&lt;/p&gt;

&lt;h2&gt;
  
  
  7. Django road
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://www.youtube.com/@djangoroad/videos"&gt;https://www.youtube.com/@djangoroad/videos&lt;/a&gt;&lt;br&gt;
Short and often interesting videos on various topics from the Django framework&lt;/p&gt;

</description>
      <category>python</category>
      <category>tutorial</category>
      <category>beginners</category>
      <category>programming</category>
    </item>
    <item>
      <title>[Python] A Comprehensive Guide to Scraping Instagram Data. How to bypass Instagram login while scraping - Meta Spy</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Sat, 28 Oct 2023 21:00:44 +0000</pubDate>
      <link>https://dev.to/deenuu1/a-comprehensive-guide-to-scraping-instagram-data-how-to-bypass-instagram-login-while-scraping-facebook-spy-meta-spy-3p4a</link>
      <guid>https://dev.to/deenuu1/a-comprehensive-guide-to-scraping-instagram-data-how-to-bypass-instagram-login-while-scraping-facebook-spy-meta-spy-3p4a</guid>
      <description>&lt;p&gt;&lt;strong&gt;Meta Spy: &lt;a href="https://github.com/DEENUU1/meta-spy" rel="noopener noreferrer"&gt;https://github.com/DEENUU1/meta-spy&lt;/a&gt;&lt;/strong&gt; &lt;br&gt;
&lt;strong&gt;Full code is available here: &lt;a href="https://pastebin.com/QMmDUZtj" rel="noopener noreferrer"&gt;https://pastebin.com/QMmDUZtj&lt;/a&gt;&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;Demo: &lt;a href="https://github.com/DEENUU1/meta-spy/blob/main/assets/instagram/imagescraper.gif?raw=true" rel="noopener noreferrer"&gt;https://github.com/DEENUU1/meta-spy/blob/main/assets/instagram/imagescraper.gif?raw=true&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Info
&lt;/h2&gt;

&lt;p&gt;This article is based on my project which I am still developing - Meta Spy (Facebook Spy before) this week I started to add commands for scraping data from Instagram, my idea is to expand this app for all Meta applications and also add Flet framework as a GUI because typing this commands is making me bored.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to bypass login ?
&lt;/h2&gt;

&lt;p&gt;Bypassing Instagram's login process might sound like a daunting task, but it's surprisingly straightforward. We'll extract the sessionid key from a browser where we're already logged in and integrate it into the Selenium driver. Here's a step-by-step guide:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Launch Instagram in your browser and press F12 to open the Developer Tools.&lt;/li&gt;
&lt;li&gt;In the Developer Tools sidebar, select "Data."&lt;/li&gt;
&lt;li&gt;Locate and select the "Cookies" option, then choose cookies for instagram.com.&lt;/li&gt;
&lt;li&gt;Copy the sessionid value.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fswh65jqnbkw0mx0w2udh.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fswh65jqnbkw0mx0w2udh.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  It's time to write some code
&lt;/h2&gt;

&lt;p&gt;Now that we've covered the initial steps, it's time to dive into the code.&lt;/p&gt;

&lt;h3&gt;
  
  
  Setting Up Chrome Driver Options
&lt;/h3&gt;

&lt;p&gt;To begin, we'll create a class with a static method that simplifies the configuration of the Chrome driver. This class will serve as the foundation for our scraper.&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;


&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;typing&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;List&lt;/span&gt;  
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;time&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;sleep&lt;/span&gt;  
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;selenium.webdriver.common.by&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;By&lt;/span&gt;  
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;selenium&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;webdriver&lt;/span&gt;  
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;selenium.webdriver.support.ui&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;WebDriverWait&lt;/span&gt;  
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;selenium.webdriver.chrome.options&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Options&lt;/span&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Scraper&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  

    &lt;span class="nd"&gt;@staticmethod&lt;/span&gt;  
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_chrome_driver_configuration&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;Options&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Options&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--disable-notifications&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--disable-extensions&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--disable-popup-blocking&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--disable-default-apps&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--disable-infobars&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--disable-web-security&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;  
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--disable-features=IsolateOrigins,site-per-process&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;  
        &lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;  
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--enable-features=NetworkService,NetworkServiceInProcess&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;  
        &lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_argument&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;--profile-directory=Default&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;chrome_options&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_experimental_option&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;excludeSwitches&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;enable-logging&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;  
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;chrome_options&lt;/span&gt;



&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;
&lt;h3&gt;
  
  
  Implementing the Base Scraper Class
&lt;/h3&gt;

&lt;p&gt;While this tutorial might appear to introduce more classes than necessary, it aligns with our modular approach to project development. This approach allows us to showcase the complete implementation of specific functionalities.&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;BaseInstagramScraper&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;Scraper&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;  
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;base_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
        &lt;span class="nf"&gt;super&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_user_id&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_base_url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;base_url&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;format&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_user_id&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_driver&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;webdriver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Chrome&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;options&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_chrome_driver_configuration&lt;/span&gt;&lt;span class="p"&gt;())&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_driver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_base_url&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_wait&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;WebDriverWait&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_driver&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;



&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;
&lt;h3&gt;
  
  
  Scoll
&lt;/h3&gt;

&lt;p&gt;Retrieving the full content from Instagram profiles requires scrolling, but it's not as simple as a one-time scroll-and-scrape process. When scrolling through a profile, data appears and disappears dynamically. As only a few rows of images are visible at a time, scrolling to the end and scraping the data is not feasible. To address this, we've created a function that provides a callback mechanism for dynamic content retrieval.&lt;/p&gt;

&lt;p&gt;Our standard function scrolls the page down and captures all the visible content. However, in this case, dynamic data retrieval is necessary.&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;scroll_page_callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;driver&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;callback&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
    &lt;span class="sh"&gt;"""&lt;/span&gt;&lt;span class="s"&gt;  
    Scrolls the page to load more data from a website    &lt;/span&gt;&lt;span class="sh"&gt;"""&lt;/span&gt;    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
        &lt;span class="n"&gt;last_height&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;driver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute_script&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;return document.body.scrollHeight&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;consecutive_scrolls&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;  

        &lt;span class="k"&gt;while&lt;/span&gt; &lt;span class="n"&gt;consecutive_scrolls&lt;/span&gt; &lt;span class="o"&gt;&amp;lt;&lt;/span&gt; &lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
            &lt;span class="n"&gt;driver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute_script&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;window.scrollTo(0, document.body.scrollHeight);&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  

            &lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
            &lt;span class="n"&gt;new_height&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;driver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;execute_script&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;return document.body.scrollHeight&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  

            &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;new_height&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="n"&gt;last_height&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
                &lt;span class="n"&gt;consecutive_scrolls&lt;/span&gt; &lt;span class="o"&gt;+=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;  
            &lt;span class="k"&gt;else&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
                &lt;span class="n"&gt;consecutive_scrolls&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;  

            &lt;span class="n"&gt;last_height&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;new_height&lt;/span&gt;  

            &lt;span class="nf"&gt;callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;driver&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  

    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;Exception&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
        &lt;span class="n"&gt;logs&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log_error&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Error occurred while scrolling: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;



&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;
&lt;h3&gt;
  
  
  Scraping data
&lt;/h3&gt;

&lt;p&gt;Now, let's put all the pieces together and explore the main class responsible for scraping Instagram data.&lt;/p&gt;
&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;ProfileScraper&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;BaseInstagramScraper&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;  
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
        &lt;span class="nf"&gt;super&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;base_url&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;https://www.instagram.com/&lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="s"&gt;/&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_driver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;add_cookie&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;  
            &lt;span class="p"&gt;{&lt;/span&gt;  
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;name&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sessionid&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;value&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;your_sessionid_goes_HERE&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;domain&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;.instagram.com&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  
            &lt;span class="p"&gt;}&lt;/span&gt;  
        &lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;_refresh_driver&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;  

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_refresh_driver&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="bp"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
        &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_driver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;refresh&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;



&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;The ProfileScraper class inherits from the BaseInstagramScraper, which already includes Chrome driver configurations and more. We add the sessionid cookie to the driver, ensuring that the "value" field contains your sessionid. Next, we call the method:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;

self._refresh_driver


&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This method refreshes the driver and correctly loads any newly added cookies.&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;extract_images&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;List&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nb"&gt;str&lt;/span&gt;&lt;span class="p"&gt;]:&lt;/span&gt;  
    &lt;span class="n"&gt;extracted_image_urls&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[]&lt;/span&gt;  
    &lt;span class="k"&gt;try&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  

        &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;extract_callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;driver&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;  
            &lt;span class="n"&gt;img_elements&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_driver&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;find_elements&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;  
                &lt;span class="n"&gt;By&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;CLASS_NAME&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;x5yr21d.xu96u03.x10l6tqk.x13vifvy.x87ps6o.xh8yej3&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;  
            &lt;span class="p"&gt;)&lt;/span&gt;  
            &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;img_element&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;img_elements&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
                &lt;span class="n"&gt;src_attribute&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;img_element&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get_attribute&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;src&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
                &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;src_attribute&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="n"&gt;src_attribute&lt;/span&gt; &lt;span class="ow"&gt;not&lt;/span&gt; &lt;span class="ow"&gt;in&lt;/span&gt; &lt;span class="n"&gt;extracted_image_urls&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
                    &lt;span class="c1"&gt;#print(f"Extracted image URL: {src_attribute}")  
&lt;/span&gt;                    &lt;span class="n"&gt;extracted_image_urls&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;append&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;src_attribute&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
        &lt;span class="nf"&gt;scroll_page_callback&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;_driver&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;extract_callback&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  

    &lt;span class="k"&gt;except&lt;/span&gt; &lt;span class="nb"&gt;Exception&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
        &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;An  error occurred while extracting images: &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;e&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  

    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;extracted_image_urls&lt;/span&gt;



&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;The core of this class lies in the &lt;code&gt;extract_images&lt;/code&gt; method, which returns a list of all scraped image URLs. Inside this method, we find the &lt;code&gt;extract_callback&lt;/code&gt; function. It identifies image elements, prints them to the console, and adds them to the &lt;code&gt;extracted_image_url&lt;/code&gt; list, checking for duplicates.&lt;/p&gt;

&lt;p&gt;Finally, we call the &lt;code&gt;scroll_page_callback&lt;/code&gt; function with the Chrome driver and the data extraction function as arguments, ensuring that our scraper works seamlessly.&lt;/p&gt;

&lt;p&gt;With this comprehensive guide, you're well-equipped to dive into Instagram data scraping with Meta Spy. As we continue developing this project, expect more features and functionalities that expand its capabilities across all Meta applications. And don't forget, our plans to integrate Flet as a GUI promise to make the experience even more user-friendly. Happy scraping!&lt;/p&gt;

&lt;h3&gt;
  
  
  Running code
&lt;/h3&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;


&lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;__name__&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;__main__&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;  
    &lt;span class="n"&gt;scraper&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;ProfileScraper&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;sawardega_wataha&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;  
    &lt;span class="n"&gt;data&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;scraper&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;extract_images&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;  
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;len&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;  
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;



&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;Inside ProfileScraper class add a user_id from instagram account.&lt;/p&gt;

&lt;h2&gt;
  
  
  Results
&lt;/h2&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;

&lt;span class="o"&gt;&amp;gt;&lt;/span&gt; python .&lt;span class="se"&gt;\m&lt;/span&gt;ain.py
33 &lt;span class="c"&gt;# This is a number of scraped urls &lt;/span&gt;
&lt;span class="c"&gt;# This is a full url to the scraped image &lt;/span&gt;
https://scontent-waw1-1.cdninstagram.com/v/t51.2885-15/387688415_1338700880368645_3875950289382108239_n.jpg?stp&lt;span class="o"&gt;=&lt;/span&gt;dst-jpg_e35&amp;amp;efg&lt;span class="o"&gt;=&lt;/span&gt;eyJ2ZW5jb2RlX3RhZyI6ImltYWdlX3VybGdlbi4xNDQweDE4MDAuc2RyIn0&amp;amp;_nc_ht&lt;span class="o"&gt;=&lt;/span&gt;scontent-waw1-1.cdnin
stagram.com&amp;amp;_nc_cat&lt;span class="o"&gt;=&lt;/span&gt;101&amp;amp;_nc_ohc&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nt"&gt;-w6WTMiiWj4AX-_Qfkt&lt;/span&gt;&amp;amp;edm&lt;span class="o"&gt;=&lt;/span&gt;ACWDqb8BAAAA&amp;amp;ccb&lt;span class="o"&gt;=&lt;/span&gt;7-5&amp;amp;ig_cache_key&lt;span class="o"&gt;=&lt;/span&gt;MzIxMTM2ODUyNjYzMDkzMTEzMA%3D%3D.2-ccb7-5&amp;amp;oh&lt;span class="o"&gt;=&lt;/span&gt;00_AfDoHMVh0dS6msk5yKaW9d81HCeCSgBUJzW82sKRHYRvwQ&amp;amp;oe&lt;span class="o"&gt;=&lt;/span&gt;65433911&amp;amp;_nc_sid&lt;span class="o"&gt;=&lt;/span&gt;ee9879



&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>python</category>
      <category>selenium</category>
      <category>tutorial</category>
      <category>opensource</category>
    </item>
    <item>
      <title>[Python] Project ideas for every level of advancement</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Fri, 27 Oct 2023 20:38:12 +0000</pubDate>
      <link>https://dev.to/deenuu1/python-project-ideas-for-every-level-of-advancement-3g5f</link>
      <guid>https://dev.to/deenuu1/python-project-ideas-for-every-level-of-advancement-3g5f</guid>
      <description>&lt;p&gt;This list contains 6 project ideas that I made  while learning programming. I can't say whether these projects will be easy for you or what tools you should use, I'm just sharing my experiences and hope they inspire you.&lt;br&gt;
&lt;strong&gt;They are listed in a random order.&lt;/strong&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Space Catalog
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack: Python, Tkinter&lt;/strong&gt;&lt;br&gt;
&lt;a href="https://github.com/DEENUU1/Space-catalog" rel="noopener noreferrer"&gt;Source code&lt;/a&gt;&lt;br&gt;
&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fxt442aj37o3rxntru9rt.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fxt442aj37o3rxntru9rt.gif" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This was my first "bigger" project that I created over a year ago. It's a simple GUI application that allows users to access information about planets and search  for "Image of the Day" from the NASA API.&lt;br&gt;
I used Tkinter to build the GUI. However, if I were to create something similar today, I would use Flet, which is a relatively new framework for building GUI applications on Windows, Linux, macOS, Android, and iOS. &lt;br&gt;
Of course, you don't have to create an application about space; it could be about car brands, plants, or anything else.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Weather Music
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack: Python, Django, HTML, Bootstrap, CSS, Docker, PostgreSQL&lt;/strong&gt;&lt;br&gt;
&lt;a href="https://github.com/DEENUU1/DJANGO-WEATHER-MUSIC" rel="noopener noreferrer"&gt;Source code&lt;/a&gt;&lt;br&gt;
&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fcs7f0ag8omfpqylwilhs.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fcs7f0ag8omfpqylwilhs.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I believe this was my first Django project. This program retrieves the user's location and, with the help of an external API, provides the current weather for that location.&lt;br&gt;
However, it seemed too simple to me, so I expanded the program to include access to Spotify playlists. The algorithm, though simple, selects appropriate playlists for the user based on the weather conditions.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. OLX Analytics
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack: Python, Flask, HTML, CSS, Bootstrap, Docker, SQLite, APScheduler&lt;/strong&gt;&lt;br&gt;
&lt;a href="https://github.com/DEENUU1/OLX-Analytics" rel="noopener noreferrer"&gt;Source code&lt;/a&gt;&lt;br&gt;
&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftpveui71w1b14lgidxeg.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ftpveui71w1b14lgidxeg.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;OLX is a Polish platform for classified ads. They have an open API (more about that in the next article) that allowed me to easily scrape and parse data about apartment listings for sale or rent. This enables users to search for new offers and access other interesting statistics.&lt;br&gt;
I also used APScheduler to automatically send emails with new offers to users.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Wallpaper Manager
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack: Python, Tkinter&lt;/strong&gt;&lt;br&gt;
&lt;a href="https://github.com/DEENUU1/Wallpaper-Manager" rel="noopener noreferrer"&gt;Source code&lt;/a&gt;&lt;br&gt;
&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F8fedya4vcv5njcdf4cxl.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F8fedya4vcv5njcdf4cxl.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;I never finished this app, but it has all the necessary functionalities.&lt;br&gt;
It was inspired by "wallpaper engine," and that's why I'm including this project here.&lt;br&gt;
You don't need to create something entirely new. Just take a moment, look at the apps you currently use, and try to recreate them. The point of personal projects is to practice, encounter new challenges, and learn new things.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Facebook Spy
&lt;/h2&gt;

&lt;p&gt;I've already written an article about this project, so check it out here -&amp;gt; &lt;a href="https://dev.to/deenuu1/facebook-spy-python-cli-tools-for-scraping-facebook-1bkk"&gt;Article&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  6. FJob
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Stack: Python, Selenium, BeautifulSoup, Rest Framework, Celery, Django Celery Beat, PostgreSQL, Docker, Docker Compose, React (JavaScript + Vite)&lt;/strong&gt;&lt;br&gt;
&lt;a href="https://github.com/DEENUU1/fjob/" rel="noopener noreferrer"&gt;Source code&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Well... 356 commits, but there's still a lot of work to be done.&lt;/p&gt;

&lt;p&gt;I'm including this project here to show you how different programming elements can be combined.&lt;br&gt;
The core of this project consists of web scrapers that collect data from websites such as OLX, Pracujpl, nofluffjobs, justjoinit, etc.&lt;br&gt;
This data is then stored in a database and accessed via a REST API created using the Django Rest Framework.&lt;br&gt;
The user interface is built using React and JavaScript.&lt;/p&gt;

&lt;p&gt;This application may seem extensive, but it's essentially a collection of interconnected pieces.&lt;br&gt;
If you're interested in data collection from websites, think about the various ways you can use that data. It can be used for training artificial intelligence, conducting analyses, or helping people find jobs by consolidating all the data in one place.&lt;/p&gt;

</description>
      <category>python</category>
      <category>webdev</category>
      <category>tutorial</category>
      <category>programming</category>
    </item>
    <item>
      <title>The only Celery &amp; Django Tutorial you need</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Wed, 18 Oct 2023 15:18:00 +0000</pubDate>
      <link>https://dev.to/deenuu1/the-only-celery-django-tutorial-you-need-4m0</link>
      <guid>https://dev.to/deenuu1/the-only-celery-django-tutorial-you-need-4m0</guid>
      <description>&lt;p&gt;In my life, I watched a lot of tutorials about Celery &amp;amp; Django configuration but I was always getting some errors.&lt;/p&gt;

&lt;p&gt;Today I am gonna show you the easy way to add Celery and Redis to your Django project.&lt;br&gt;
This tutorial is based on one of my projects available on GitHub &lt;a href="https://github.com/DEENUU1/MyFridge" rel="noopener noreferrer"&gt;MyFridge&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;
  
  
  About Celery and Redis
&lt;/h2&gt;
&lt;h3&gt;
  
  
  Celery
&lt;/h3&gt;

&lt;p&gt;Celery is a distributed task queue system that allows you to run time-consuming tasks asynchronously. With Celery, you can decouple long-running or resource-intensive processes from your main application, making your application more responsive and efficient. Celery supports various message brokers like Redis, RabbitMQ, and more, but in this article, we'll focus on using Celery with Redis.&lt;/p&gt;
&lt;h3&gt;
  
  
  Redis
&lt;/h3&gt;

&lt;p&gt;Redis is an open-source, in-memory data structure store that can be used as a message broker for Celery. It excels at handling high-throughput, low-latency tasks and provides persistent data storage. In the context of Celery, Redis acts as a message broker, passing messages between the main application and the worker processes that execute tasks in the background.&lt;/p&gt;
&lt;h2&gt;
  
  
  Final project structure (this structure includes other content, we gonna focus only on a few elements)
&lt;/h2&gt;

&lt;p&gt;As you can see my project in Django is called - myfridge but yours probably will have another name so you need to change all 'myfridge' to your project name (inside files: &lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;celery.py&lt;/li&gt;
&lt;li&gt;Dockerfile&lt;/li&gt;
&lt;li&gt;docker-compose.yml
)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fyzumkjl1paahe2istgm5.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fyzumkjl1paahe2istgm5.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Installation and configuration
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Install packages&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;First, you need to install celery and Redis packages&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip &lt;span class="nb"&gt;install &lt;/span&gt;celery Redis
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Add this packages to requirements.txt&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip freeze &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; requirements.txt
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ol&gt;
&lt;li&gt;Create a celery.py file inside your Django project (the same place where settings.py is)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F31fpbdr1shmx12cy7k6p.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2F31fpbdr1shmx12cy7k6p.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;celery.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;__future__&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;absolute_import&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;unicode_literals&lt;/span&gt;
&lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;celery&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Celery&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;Django.conf&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;settings&lt;/span&gt;


&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;setdefault&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;DJANGO_SETTINGS_MODULE&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;myfridge.settings&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name 
&lt;/span&gt;&lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;Celery&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;myfridge&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name
&lt;/span&gt;&lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;config_from_object&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;settings&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;namespace&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;CELERY&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;app&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;autodiscover_tasks&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ol&gt;
&lt;li&gt;Configure __ init __ .py 
In the same directory where you created celery.py, there is a file called '__ init__ .py'
You need to paste this code inside this file&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fki4tjs7hpyttc1vn0bkr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fki4tjs7hpyttc1vn0bkr.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;__ init __ .py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;.celery&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;app&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;celery_app&lt;/span&gt;

&lt;span class="n"&gt;__all__&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;celery_app&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ol&gt;
&lt;li&gt;Edit settings.py
Inside settings.py file you need to configure your Celery broker, result backend, and other important things.
For me, it looks like this&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;settings.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;CELERY_BROKER_URL&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;redis://redis:6379/0&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="n"&gt;CELERY_RESULT_BACKEND&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;redis://redis:6379/0&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="n"&gt;CELERY_ACCEPT_CONTENT&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;application/json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;
&lt;span class="n"&gt;CELERY_TASK_SERIALIZER&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;span class="n"&gt;CELERY_RESULT_SERIALIZER&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;json&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;ol&gt;
&lt;li&gt;Docker configuration
Now to run Celery, redis, and Django we need to use Docker.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;First of all, in your main directory create a 'config' directory and inside it create a directory called 'Django' and then inside it create a file called Dockerfile &lt;strong&gt;Note that D must be an uppercase&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fl9c84nyiz4cnq9uglhsr.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fl9c84nyiz4cnq9uglhsr.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Dockerfile&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="s"&gt;FROM python:3.11&lt;/span&gt;

&lt;span class="s"&gt;ENV PYTHONUNBUFFERED &lt;/span&gt;&lt;span class="m"&gt;1&lt;/span&gt;

&lt;span class="s"&gt;WORKDIR /app&lt;/span&gt;

&lt;span class="s"&gt;COPY requirements.txt.&lt;/span&gt;

&lt;span class="s"&gt;RUN pip install -r requirements.txt&lt;/span&gt;

&lt;span class="s"&gt;COPY ./myfridge /app/&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then inside the main directory (where config and Django project is) create a file called docker-compose.yml&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fa9l3kcawg0on3icuo945.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fa9l3kcawg0on3icuo945.png" alt="Image description"&gt;&lt;/a&gt;&lt;br&gt;
&lt;strong&gt;In my example it's called docker-compose.dev.yml but you need to name it docker-compose.yml&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;docker-compose.yml&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;
&lt;span class="na"&gt;version&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;3"&lt;/span&gt;

&lt;span class="na"&gt;services&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;db&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;hostname&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;db&lt;/span&gt;
        &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;postgres:15&lt;/span&gt;
        &lt;span class="na"&gt;container_name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;myfridge-db-dev&lt;/span&gt; 
        &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;./data/db:/var/lib/postgresql/data&lt;/span&gt;
        &lt;span class="na"&gt;environment&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;POSTGRES_DB=myfridge&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;POSTGRES_USER=myfridge&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;POSTGRES_PASSWORD=myfridge123&lt;/span&gt;
        &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;5432:5432"&lt;/span&gt;

    &lt;span class="na"&gt;web&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;hostname&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;web&lt;/span&gt;
        &lt;span class="na"&gt;build&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;context&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;.&lt;/span&gt;
            &lt;span class="na"&gt;dockerfile&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;./config/django/Dockerfile&lt;/span&gt;
        &lt;span class="na"&gt;container_name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;myfridge-web-dev&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name&lt;/span&gt;
        &lt;span class="na"&gt;command&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;python manage.py runserver 0.0.0.0:8000&lt;/span&gt;
        &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;./myfridge:/app&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name&lt;/span&gt;
        &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;8000:8000"&lt;/span&gt;
        &lt;span class="na"&gt;depends_on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;db&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;redis&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;celery&lt;/span&gt;
        &lt;span class="na"&gt;env_file&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;.env&lt;/span&gt;

    &lt;span class="na"&gt;redis&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;redis:latest&lt;/span&gt;
        &lt;span class="na"&gt;container_name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;myfridge-redis-dev&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name&lt;/span&gt;
        &lt;span class="na"&gt;ports&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="s"&gt;6379:6379"&lt;/span&gt;

    &lt;span class="na"&gt;celery&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;build&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="na"&gt;context&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;.&lt;/span&gt;
            &lt;span class="na"&gt;dockerfile&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;./config/django/Dockerfile&lt;/span&gt;
        &lt;span class="na"&gt;container_name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;myfridge-celery-dev&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name&lt;/span&gt;
        &lt;span class="na"&gt;command&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;celery -A myfridge.celery worker -l info&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name&lt;/span&gt;
        &lt;span class="na"&gt;volumes&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;./myfridge:/app/&lt;/span&gt; &lt;span class="c1"&gt;# Change 'myfridge' to your project name&lt;/span&gt;
        &lt;span class="na"&gt;depends_on&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;db&lt;/span&gt;
            &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="s"&gt;redis&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;And thats all for our configuration, note that you need to change all 'myfridge' to you project name&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;To run Django project with Docker you need to use PostgreSQL database&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;Check this tutorial&lt;/strong&gt; &lt;a href="https://www.youtube.com/watch?v=jyapP2Yy0AQ" rel="noopener noreferrer"&gt;Docker With Django And PostgreSQL Tutorial&lt;/a&gt;&lt;/p&gt;
&lt;h2&gt;
  
  
  Example 1
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Inside your django application create a file called task.py&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;task.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;celery&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;shared_task&lt;/span&gt;

&lt;span class="nd"&gt;@shared_task&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;x&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;x&lt;/span&gt; &lt;span class="o"&gt;+&lt;/span&gt; &lt;span class="n"&gt;y&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;As you can see to make a task you need to add @shared_task() decorator to your function &lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Call this task inside your view&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;views.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;.task&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;add&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;add_view&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;add&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;delay&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;4&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nc"&gt;HttpResponse&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To run 'add' function as a task add .delay() method &lt;/p&gt;

&lt;h2&gt;
  
  
  Example 2
&lt;/h2&gt;

&lt;ol&gt;
&lt;li&gt;Inside your django application create a file called task.py &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmtvrfq5zsrz29ybpplpw.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fmtvrfq5zsrz29ybpplpw.png" alt="Image description"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Create some function that should do somethings &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;send_email.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;send_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;email&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;subject&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="nf"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sa"&gt;f&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Sending email to &lt;/span&gt;&lt;span class="si"&gt;{&lt;/span&gt;&lt;span class="n"&gt;email&lt;/span&gt;&lt;span class="si"&gt;}&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Of course this function doesn't do anything but it's just an example&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Then in task.py &lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;task.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;.send_email&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;send_email&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;celery&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;shared_task&lt;/span&gt;

&lt;span class="nd"&gt;@shared_task&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;send_email_task&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;email&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;subject&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="nf"&gt;send_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;email&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;subject&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Before our function task, we need to add the decorator&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="nd"&gt;@shared_task&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;And then inside that task call our function&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Running a task
You can you this task inside your forms, views, commands, etc
In this example, I am gonna use it inside the registration form
This code is not import for you, you don't need to understand it.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;forms.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;django&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;django.contrib.auth.forms&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;AuthenticationForm&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;.models&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;CustomUser&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;.task&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;send_email_task&lt;/span&gt;


&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;CustomUserRegistration&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;ModelForm&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;username&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;CharField&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;widget&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;TextInput&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;attrs&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;class&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;form-control&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}))&lt;/span&gt;
    &lt;span class="n"&gt;email&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;EmailField&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;widget&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;EmailInput&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;attrs&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;class&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;form-control&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;}))&lt;/span&gt;
    &lt;span class="n"&gt;password&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;CharField&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;widget&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;PasswordInput&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;attrs&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;class&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;form-control&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;password_repeat&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;CharField&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;widget&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;PasswordInput&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;attrs&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;class&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;form-control&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;Meta&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;model&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;CustomUser&lt;/span&gt;
        &lt;span class="n"&gt;fields&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;username&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;email&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;password&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;password_repeat&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;clean_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;email&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cleaned_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;email&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;CustomUser&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;objects&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;filter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;email&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;email&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;exists&lt;/span&gt;&lt;span class="p"&gt;():&lt;/span&gt;
            &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;ValidationError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Email already exists&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;email&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;clean_password_repeat&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;password&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cleaned_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;password&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;password_repeat&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cleaned_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;password_repeat&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;password&lt;/span&gt; &lt;span class="o"&gt;!=&lt;/span&gt; &lt;span class="n"&gt;password_repeat&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
            &lt;span class="k"&gt;raise&lt;/span&gt; &lt;span class="n"&gt;forms&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;ValidationError&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Passwords don&lt;/span&gt;&lt;span class="sh"&gt;'&lt;/span&gt;&lt;span class="s"&gt;t match&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;password_repeat&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;send_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;send_email_task&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;delay&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cleaned_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;email&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;subject&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Activate your account&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Take a look at the last method 'send_email'&lt;br&gt;
Inside that method I am calling for our task and to run it I am using .delay method &lt;br&gt;
This allows us to run it as a celery task&lt;/p&gt;

&lt;p&gt;views.py&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;RegisterUserView&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;FormView&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
    &lt;span class="n"&gt;form_class&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;CustomUserRegistration&lt;/span&gt;
    &lt;span class="n"&gt;template_name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;register.html&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="n"&gt;success_url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;reverse_lazy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;users:success_register&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;form_valid&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;form&lt;/span&gt;&lt;span class="p"&gt;):&lt;/span&gt;
        &lt;span class="n"&gt;user&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;form&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;save&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;commit&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;False&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;is_active&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="bp"&gt;False&lt;/span&gt;
        &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;set_password&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;form&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;cleaned_data&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;password&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;])&lt;/span&gt;
        &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;save&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="n"&gt;form&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;send_email&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;message&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nf"&gt;render_to_string&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
                &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;acc_activate_email.html&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                &lt;span class="p"&gt;{&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;user&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;domain&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;get_current_site&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;request&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="n"&gt;domain&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;uid&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nf"&gt;urlsafe_base64_encode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nf"&gt;force_bytes&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;pk&lt;/span&gt;&lt;span class="p"&gt;)),&lt;/span&gt;
                    &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;token&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;account_activation_token&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;make_token&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;user&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
                &lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nf"&gt;super&lt;/span&gt;&lt;span class="p"&gt;().&lt;/span&gt;&lt;span class="nf"&gt;form_valid&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;form&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Inside RegisterUserView I am using this form and I am able to use the method send_email &lt;/p&gt;

</description>
      <category>django</category>
      <category>celery</category>
      <category>python</category>
      <category>programming</category>
    </item>
    <item>
      <title>Facebook SPY - Python CLI Tools for scraping Facebook</title>
      <dc:creator>Kacper Włodarczyk</dc:creator>
      <pubDate>Fri, 13 Oct 2023 20:26:42 +0000</pubDate>
      <link>https://dev.to/deenuu1/facebook-spy-python-cli-tools-for-scraping-facebook-1bkk</link>
      <guid>https://dev.to/deenuu1/facebook-spy-python-cli-tools-for-scraping-facebook-1bkk</guid>
      <description>&lt;h3&gt;
  
  
  I would like to share with you one of the biggest projects I have ever worked on.
&lt;/h3&gt;

&lt;p&gt;Facebook Spy is a tool for scraping data from Facebook profiles, pages, and search results.&lt;br&gt;
For example, you can scrape all posts from a given Facebook account, and download images and videos.&lt;/p&gt;

&lt;h3&gt;
  
  
  Info!
&lt;/h3&gt;

&lt;p&gt;For more details and visualization check README or Documentation&lt;/p&gt;

&lt;p&gt;Repository: &lt;a href="https://github.com/DEENUU1/facebook-spy"&gt;Facebook Spy on GitHub&lt;/a&gt;&lt;br&gt;
Documentation: &lt;a href="https://deenuu1.github.io/facebook-spy/"&gt;Facebook Spy Documentation&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--sm9QbkLu--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ahexcpb7djyzvgkqzqwc.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--sm9QbkLu--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ahexcpb7djyzvgkqzqwc.png" alt="Image description" width="800" height="437"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--EZqQsmF8--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f526p7vxvbp5suwrtd03.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--EZqQsmF8--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f526p7vxvbp5suwrtd03.png" alt="Image description" width="800" height="441"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--0xHnHlWg--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2vczcrrb7j3h2keg91io.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--0xHnHlWg--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2vczcrrb7j3h2keg91io.png" alt="Image description" width="800" height="426"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--zssekJmv--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4zvumwkjoove2jpx182x.png" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--zssekJmv--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_800/https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4zvumwkjoove2jpx182x.png" alt="Image description" width="800" height="427"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Facebook Spy offers the following functionalities:
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Login: Perform a secure two-step verification process to log in to your Facebook account with enhanced security settings or log in to your Facebook account using the default login method.&lt;/li&gt;
&lt;li&gt;Data Scraping: Retrieve details such as friend lists, images, recent locations, videos, reels, reviews, posts, likes, groups, events, and more.&lt;/li&gt;
&lt;li&gt;Local Web Application: Run a local web application to browse scraped data, create notes for specific user profiles, and easily search for details on the web.&lt;/li&gt;
&lt;li&gt;Video Downloader: Download all scraped videos from a Facebook account or simply by adding the provided URL.&lt;/li&gt;
&lt;li&gt;Visualize Connections: Create a graphical representation of connections between different Facebook profiles based on their friendships.&lt;/li&gt;
&lt;li&gt;Utilize an Open-Source Language Model: Use a free open-source language model to generate a concise summary of a Facebook user's information based on the scraped data.&lt;/li&gt;
&lt;li&gt;Friend Crawler: Initiate a friend crawler for a specified Facebook account, gather data about friends, and continue the process for subsequent users in the queue.&lt;/li&gt;
&lt;li&gt;Data Export: Save scraped data for a specified Facebook user to a PDF file for documentation and analysis.&lt;/li&gt;
&lt;li&gt;Parallel Scraping: Run scrapers in parallel to expedite the data collection process.&lt;/li&gt;
&lt;li&gt;Data Storage: Save all scraped data to JSON files and a database.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>python</category>
      <category>scraper</category>
      <category>selenium</category>
      <category>facebook</category>
    </item>
  </channel>
</rss>
