<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Orcraft</title>
    <description>The latest articles on DEV Community by Orcraft (@orcraft).</description>
    <link>https://dev.to/orcraft</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4241846%2F0d20e457-df4c-4b34-8462-73ce20da8bf2.png</url>
      <title>DEV Community: Orcraft</title>
      <link>https://dev.to/orcraft</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/orcraft"/>
    <language>en</language>
    <item>
      <title>Give your AI agent your docs and Orcraft, and watch it run your testing</title>
      <dc:creator>Orcraft</dc:creator>
      <pubDate>Mon, 16 Nov 2026 16:47:11 +0000</pubDate>
      <link>https://dev.to/orcraft/give-your-ai-agent-your-docs-and-orcraft-and-watch-it-run-your-testing-1o66</link>
      <guid>https://dev.to/orcraft/give-your-ai-agent-your-docs-and-orcraft-and-watch-it-run-your-testing-1o66</guid>
      <description>&lt;p&gt;Orcraft comes out of nearly two decades of validation work: software, hardware, firmware, drivers, applications and whole systems. The idea behind it is simple. You and your AI agents write the test logic, in plain Python, and nothing else. The Orcraft framework takes care of the rest: the GUI, the configuration, the machines the test runs on, getting the code to them, the logs, the results and the CI run.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgynejimeax4ey3rv2f9h.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fgynejimeax4ey3rv2f9h.png" alt="The testing operation" width="800" height="414"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;This article goes around that loop once, with code from the &lt;a href="https://orcraft.app/tutorial/" rel="noopener noreferrer"&gt;tutorial suite&lt;/a&gt;. Every line of code below is from it.&lt;/p&gt;

&lt;h2&gt;
  
  
  What you give your AI agent
&lt;/h2&gt;

&lt;p&gt;Two things. The first is what your product already has: its documentation. A README, a user manual, an API specification or the code itself will do, and later its issues and feature requests. The second is one link to the Orcraft instructions, which tell the agent how a suite is laid out.&lt;/p&gt;

&lt;p&gt;Then you ask in plain words:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Create a suite for the maze application from its README.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The agent writes an ordinary suite. To run it, it also gets the list of your lab's machines, a file the controller saves, and the requests stay as plain:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Run TCP Transmit on any available Linux endpoint.
Explain why TCP Receive failed on lab-linux-2 in the last run.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Any agent that can read a folder, write Python and run a command works with it, such as Claude Code, OpenAI Codex or Gemini CLI. The &lt;a href="https://orcraft.app/ai-agents/" rel="noopener noreferrer"&gt;AI Agents&lt;/a&gt; pages list the inputs and more example prompts. There is no AI inside Orcraft, and it calls no AI provider.&lt;/p&gt;

&lt;p&gt;Every part of this you can do yourself as well: write the suite, configure a run, run it and read the results. A person has the GUI and the command line for it, an agent the code and the command line. Either way the suite is the same plain Python, and the rest of this article opens one up.&lt;/p&gt;

&lt;h2&gt;
  
  
  A step is a Python class
&lt;/h2&gt;

&lt;p&gt;The smallest unit is a step. This one runs the &lt;code&gt;hostname&lt;/code&gt; command and checks the answer:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;StepHostname&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;OrcraftTestStep&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nc"&gt;StepHostnameConfig&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="nb"&gt;super&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;__init__&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;config&lt;/span&gt; &lt;span class="ow"&gt;or&lt;/span&gt; &lt;span class="nc"&gt;StepHostnameConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;_run_step&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nc"&gt;OrcraftTestStepResult&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;StepHostnameResult&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="c1"&gt;# The step's body runs on the endpoint it was sent to, so the command&lt;/span&gt;
        &lt;span class="c1"&gt;# answers with that machine's name.&lt;/span&gt;
        &lt;span class="n"&gt;completed&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;subprocess&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"hostname"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;capture_output&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="kc"&gt;True&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;hostname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;completed&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;stdout&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;checkpoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;completed&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;returncode&lt;/span&gt; &lt;span class="o"&gt;==&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt; &lt;span class="ow"&gt;and&lt;/span&gt; &lt;span class="kt"&gt;bool&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;hostname&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"hostname answered: {hostname}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"hostname failed with exit code {completed.returncode}: {completed.stderr.strip()}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="nc"&gt;Logger&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;log&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"Hello from {hostname}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;color&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;Logger&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Color&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="no"&gt;BLUE&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;hostname&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;hostname&lt;/span&gt;

        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;result&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Nothing in it says where it runs. A flow sends the step out, and a test owns the flow. The whole structure is suites, groups, tests, flows and steps, and it is the same in every suite.&lt;/p&gt;

&lt;h2&gt;
  
  
  The GUI comes from the code
&lt;/h2&gt;

&lt;p&gt;The test that uses this step declares what can be configured:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;TestHelloEndpointConfig&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nc"&gt;OrcraftTestConfig&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;name&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Text&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Hello Endpoint"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

    &lt;span class="n"&gt;test_endpoint&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"localhost"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;label&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Test Endpoint"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That one &lt;code&gt;Endpoint&lt;/code&gt; field is all it takes. The controller reads the class and builds the run panel from it, with a Test Endpoint field that lists the machines of the lab, Windows, Linux and macOS alike. There is no configuration file to write and no plugin to register.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyr4a3ur41fznq97i499z.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fyr4a3ur41fznq97i499z.png" alt="The Hello Endpoint run panel with Test Endpoint set to a Windows machine" width="800" height="731"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;With the default, &lt;code&gt;localhost&lt;/code&gt;, the step runs on your own computer. Pick another machine and press Run: the controller sends the step there, and the log comes back with that machine's name.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9nop0853plg62i0deftd.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F9nop0853plg62i0deftd.png" alt="The controller after the run, with a Pass status in the tree and the line Hello from GMG-TEST-1 in the log" width="800" height="428"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The test code was never installed on that machine.&lt;/strong&gt; The controller &lt;a href="https://orcraft.app/tutorial/packages-uploading/" rel="noopener noreferrer"&gt;uploads the test package&lt;/a&gt; to every endpoint a run uses, before the run starts. An endpoint machine needs only the Orcraft endpoint application, installed once; your suites never have to be copied, cloned or installed there.&lt;/p&gt;

&lt;h2&gt;
  
  
  Every machine has its role
&lt;/h2&gt;

&lt;p&gt;Orcraft works with a lab both ways: with machines that all do the same job, and with a setup where every machine has its own role. Roles first. Where something produces and something consumes, a server and a client, a host and the device attached to it, a test declares one &lt;code&gt;Endpoint&lt;/code&gt; field per role. This is the tutorial's Multiple Endpoints and Files Transfer test, with a generator and a solver:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;endpoint_1&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"localhost"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;label&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Endpoint 1"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;endpoint_2&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"localhost"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;label&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Endpoint 2"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The test fixes the roles. Which machine plays which role you choose in the run panel, for each run, and the code stays as it is: today the Windows machine generates and the Mac solves, tomorrow the other way round.&lt;/p&gt;

&lt;p&gt;Inside the flow, each step names the role it belongs to, and the framework sends it to whichever machine holds that role. Moving a file from one role to the other is a single call:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;transfer_result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;transfer&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;src_endpoint&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;endpoint_1&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;dst_endpoint&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;endpoint_2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;paths&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;step_2_result&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;maze_file_path&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The step after it runs on the second role, with the file that arrived:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;step_3_result&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;run_step&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nc"&gt;StepLoadSolve&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;endpoint&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;endpoint_2&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;maze_executable_folder&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nb"&gt;self&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;config&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;maze_executable_folder&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;maze_file&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;transfer_result&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;dst_paths&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The results record the split, step by step:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbd6zyllefqhsfkyqkwuj.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbd6zyllefqhsfkyqkwuj.png" alt="The Details view of the test, with the generate steps marked GMG-TEST-1 and the solve steps marked GMG-TEST-5" width="800" height="467"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  The same test on every machine
&lt;/h2&gt;

&lt;p&gt;In the other mode the machines all do the same job, and one test runs on each of them. For that, declare the field as an &lt;code&gt;EndpointFanout&lt;/code&gt; in place of an &lt;code&gt;Endpoint&lt;/code&gt;. This line is from the tutorial's Endpoint Fanout test:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;test_endpoint&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;EndpointFanout&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;default&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"localhost"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;label&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"Test Endpoint"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The same test also declares a matrix of cases:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="n"&gt;matrix&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Matrix&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Dimension&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"width"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;5&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nc"&gt;OrcraftConfigField&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Dimension&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"height"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;8&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;12&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two widths and two heights make four cases. In the run panel the Test Endpoint field now takes several machines. Here it holds three, so the counter reads twelve cases:&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnp8mfr9btg2svnbum8vk.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fnp8mfr9btg2svnbum8vk.png" alt="The Endpoint Fanout run panel with three machines in the Test Endpoint field and Total Cases 12" width="800" height="731"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Press Run, and the test starts once per machine, all three at the same time. Each machine gets its own run row, its own log and its own results.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5cy3bqie9c2gcepqqujq.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F5cy3bqie9c2gcepqqujq.png" alt="The execution tree with three passing runs of the same test, one per machine" width="800" height="605"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  From the GUI to CI
&lt;/h2&gt;

&lt;p&gt;A run that works in the GUI is exported as a snapshot, and the snapshot runs unattended from the command line or a CI pipeline, with the same configuration and the same results. Every run also writes a history file beside its logs, which the controller opens, so a run made in CI can be looked at in the same tree as a run made by hand. The tutorial page &lt;a href="https://orcraft.app/tutorial/exporting-test-batch-for-ci/" rel="noopener noreferrer"&gt;Exporting a test batch for CI&lt;/a&gt; shows the whole path.&lt;/p&gt;

&lt;h2&gt;
  
  
  What else it does
&lt;/h2&gt;

&lt;p&gt;The walkthrough shows one path through the controller. The rest is the framework's side of the work too, with no code of yours behind it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/lambda-configuration-parameters/" rel="noopener noreferrer"&gt;Lambda values&lt;/a&gt;.&lt;/strong&gt; A parameter or a &lt;a href="https://orcraft.app/tutorial/matrix-lambda-dimensions/" rel="noopener noreferrer"&gt;matrix dimension&lt;/a&gt; takes its choices from a function that runs when the test is configured, such as the releases published at that moment.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/matrix-dimensions-randomization/" rel="noopener noreferrer"&gt;Randomization&lt;/a&gt;.&lt;/strong&gt; Matrix dimensions draw their values at random while the number of cases stays fixed, and are redrawn from the controller.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/endpoint-pools/" rel="noopener noreferrer"&gt;Endpoint pools&lt;/a&gt;.&lt;/strong&gt; A test takes any free machine that matches a label, such as an operating system or attached hardware.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/endpoints-config-types/#queued-runs" rel="noopener noreferrer"&gt;Queues&lt;/a&gt;.&lt;/strong&gt; A run that needs a busy machine waits for it, and starts by itself when the machine is free.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/packages-uploading/" rel="noopener noreferrer"&gt;Endpoint updates&lt;/a&gt;.&lt;/strong&gt; The controller updates the endpoint software on a machine whenever its version differs from its own.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/details-dialog/" rel="noopener noreferrer"&gt;Step-level results&lt;/a&gt;.&lt;/strong&gt; Live logs and results per machine, down to the single step, with export to a spreadsheet.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/sessions/" rel="noopener noreferrer"&gt;Sessions&lt;/a&gt;.&lt;/strong&gt; A session holds the suites, the plan and the runs. Save it to a file, reopen it later, share it, and run it again with the same plan.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/tutorial/editing-run-configuration/#rerunning-test-runs" rel="noopener noreferrer"&gt;Rerun&lt;/a&gt;.&lt;/strong&gt; Each test run keeps the exact configuration it ran with, and reruns with it at a click.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/setup/endpoints/#secure-endpoint-connection" rel="noopener noreferrer"&gt;Secure connection&lt;/a&gt;.&lt;/strong&gt; Optional mutual TLS between the controller and the endpoints, with token-based access control.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;&lt;a href="https://orcraft.app/setup/licensing/#free-and-team" rel="noopener noreferrer"&gt;Shared endpoints&lt;/a&gt;.&lt;/strong&gt; On the Team Plan, endpoint machines are shared by every team member, CI runner and AI agent.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Try it
&lt;/h2&gt;

&lt;p&gt;Install with:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip install orcraft
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then start the controller:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;orcraft-controller run
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;It needs Python 3.14 or newer, on Windows, Linux or macOS. Running tests on your own computer needs no account.&lt;/p&gt;

&lt;p&gt;To run tests on another machine of your lab, install Orcraft on it the same way and start the endpoint application there:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;orcraft-endpoint run
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then add the machine in the controller, sign in and activate its license. The Free Plan covers two remote machines with no card and no time limit. &lt;a href="https://orcraft.app/setup/endpoints/" rel="noopener noreferrer"&gt;Adding endpoints&lt;/a&gt; shows each step, and the &lt;a href="https://orcraft.app/tutorial/" rel="noopener noreferrer"&gt;tutorial&lt;/a&gt; goes through the suite used here, test by test.&lt;/p&gt;

&lt;p&gt;The 16-minute demo shows all of it in motion:&lt;/p&gt;

&lt;p&gt;  &lt;iframe src="https://www.youtube.com/embed/xryvR4MKq0E" width="710" height="399"&gt;
  &lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;Orcraft is a commercial, closed-source product; the plans and prices are on the &lt;a href="https://orcraft.app/pricing/" rel="noopener noreferrer"&gt;pricing page&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;We would like to hear from people who run test or validation labs.&lt;/p&gt;

</description>
      <category>python</category>
      <category>testing</category>
      <category>automation</category>
      <category>ai</category>
    </item>
  </channel>
</rss>
