Take a Screenshot with AskUI - Also on Fail!

When you develop UI-Automations you often run into the problem that they fail inexplicably 😥.

This leads to frustration and oftentimes stops the effort entirely!

We will show you how can level up your debugging skills by teaching you a way to take screenshots during an AskUI execution and when on fail of an execution.

With this knowledge you can lower your frustration and become faster when developing UI-Workflows because you see what AskUI sees.

Prerequisites

• AskUI SDK installed: [Windows, Linux, macOS

That is it 😉

Take a Screenshot with annotate()

Do you want to know what AskUI sees at a specific point in time when executing your code?

Luckily the image can be obtained by using AskUI's built-in method annotate() and a little bit of TypeScript.

First you need to import the fs module at the start of your automation file. You will use it to save the file to a specific place on your disc.

After you called annotate you save the returned object into a variable. This object contains the image encoded in base64.

You can extract it with a Buffer and write it to your disc with fs.writeFileSync(<path to file>, buffer)

// Add this to the start of your AskUI-file containing your workflows/instructions
import * as fs from 'fs';

// First, get all the information from the annotation
// This will also save an interactive HTML
// file to the 'report/' folder
const annotation = await aui.annotate();

// The screenshot is contained as a string in 'base64' format
// Create a buffer with the base64 image
let buffer = Buffer.from(annotation.image.split('base64,')[1], 'base64');

// Write the file
// Name as you want 
// (./ is your project folder)
fs.writeFileSync("./test.png", buffer);

Take a Screenshot on Fail

Normally you do not take a screenshot if nothing fails. But in case your UI-Automations fails at some point it is helpful to know Why it might have failed. Was an element missing? Or was the UI just too slow to render?

You can enable this behavior by enabling a setting in your AskUI UI Control Client. For the default Jest setup, you have to go to the file test/helper/jest.setup.ts.

There you have to set annotationLevel property of the UiControlClient to ON_FAILURE as shown in the following code:

aui = await UiControlClient.build({
  annotationLevel: AnnotationLevel.ON_FAILURE
});

A more detailed explanation is documented in the API Docs of the AskUI UI Control Client.

Conclusion

With these two methods you can see what AskUI sees at any given point of your UI-Automation. This will speed up your debugging.

Get Support

If you have a recurring or persisting issue, don’t hesitate to ask the Discord community for help!