The latest articles on DEV Community by Brendan White (@br-white). https://dev.to/br-white https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3062299%2F833574c2-63f0-4663-a904-e24264a6f18c.png https://dev.to/br-white en Brendan White Fri, 18 Apr 2025 09:58:34 +0000 https://dev.to/br-white/nova-act-python-sdk-analysis-1ao5 https://dev.to/br-white/nova-act-python-sdk-analysis-1ao5 <h2> Overview </h2> <p>Nova Act is an experimental SDK from Amazon that enables developers to build browser automation agents. It combines natural language instructions with direct browser manipulation capabilities.</p> <h2> Key Features </h2> <ol> <li> <p><strong>Hybrid Automation Approach</strong>:</p> <ul> <li>Combines natural language instructions (<code>act()</code> method) with direct Playwright browser control</li> <li>Allows breaking complex workflows into smaller, more reliable steps</li> </ul> </li> <li> <p><strong>Information Extraction</strong>:</p> <ul> <li>Supports structured data extraction using Pydantic models</li> <li>Includes convenience schemas like <code>BOOL_SCHEMA</code> for simple yes/no questions</li> </ul> </li> <li> <p><strong>Parallel Execution</strong>:</p> <ul> <li>Enables running multiple browser sessions concurrently using ThreadPoolExecutor</li> <li>Useful for tasks like scraping multiple pages simultaneously</li> </ul> </li> <li> <p><strong>Authentication Handling</strong>:</p> <ul> <li>Supports persistent browser state through Chrome user data directories</li> <li>Allows pre-authenticated sessions</li> </ul> </li> <li> <p><strong>Sensitive Data Handling</strong>:</p> <ul> <li>Recommends using Playwright APIs directly for password entry</li> <li>Provides warnings about screenshot collection of sensitive information</li> </ul> </li> </ol> <h2> Technical Implementation </h2> <ul> <li>Built on Playwright for browser automation</li> <li>Requires Python 3.10+</li> <li>Supports MacOS and Ubuntu</li> <li>Uses temporary directories for isolated browser sessions by default</li> </ul> <h2> Best Practices </h2> <ol> <li> <p><strong>Prompt Design</strong>:</p> <ul> <li>Be prescriptive and specific in instructions</li> <li>Break large tasks into smaller steps</li> <li>Avoid high-level, vague prompts</li> </ul> </li> <li> <p><strong>Error Handling</strong>:</p> <ul> <li>Check <code>matches_schema</code> when using structured responses</li> <li>Handle potential schema mismatches gracefully</li> </ul> </li> <li> <p><strong>Performance</strong>:</p> <ul> <li>First run requires Playwright browser installation (1-2 minutes)</li> <li>Subsequent runs start quickly</li> </ul> </li> </ol> <h2> Limitations </h2> <ul> <li>Currently doesn't support IPython</li> <li>Cannot interact with non-browser applications</li> <li>Struggles with hidden elements (mouseover menus)</li> <li>Doesn't handle browser window dialogs</li> <li>Early research preview with many expected limitations</li> </ul> <h2> Use Cases </h2> <p>The SDK demonstrates several practical applications:</p> <ol> <li> <p><strong>E-commerce workflows</strong>:</p> <ul> <li>Product search and cart operations</li> <li>Order history management</li> </ul> </li> <li> <p><strong>Data Collection</strong>:</p> <ul> <li>Scraping structured data from websites</li> <li>Aggregating information from multiple sources</li> </ul> </li> <li> <p><strong>Task Automation</strong>:</p> <ul> <li>Food ordering</li> <li>Travel booking</li> <li>Research tasks</li> </ul> </li> </ol> <h2> Security Considerations </h2> <ul> <li>API keys must be protected</li> <li>Sensitive data should be entered via Playwright, not natural language prompts</li> <li>Screenshots may capture sensitive information visible in the browser</li> <li>Includes Acceptable Use Policy requirements</li> </ul> <h2> Comparison to Alternatives </h2> <p>Compared to other automation tools:</p> <ul> <li>More structured than pure Playwright scripting</li> <li>More controllable than end-to-end LLM automation</li> <li>Combines benefits of programmatic control with natural language flexibility</li> </ul> <h2> Getting Started </h2> <p>Basic requirements:</p> <ol> <li>Python 3.10+</li> <li>MacOS or Ubuntu</li> <li>API key from nova.amazon.com/act</li> </ol> <p>Installation:<br> </p> <div class="highlight js-code-highlight"> <pre class="highlight shell"><code>pip <span class="nb">install </span>nova-act </code></pre> </div> <p>The SDK particularly suited for developers who need to automate web-based workflows while maintaining control over the automation process. Its hybrid approach offers a balance between flexibility and reliability that could make it useful for prototyping and certain production use cases.</p> <p>Would you like me to help you using the Nova Act SDK?</p>