DEV Community

Ai-Q Labs for Apify

Posted on

I removed the under maintenance label from my Actor twice. It came back in four hours, then in one.

I removed the under maintenance label from my Actor twice. It came back in four hours, then in one.

I publish 23 Actors on Apify. Twenty-two of them take a public record and check whether what it
still claims is true.

The twenty-third is different. It reads an audit dataset and opens a GitHub issue for each finding.
It reaches GitHub through an Apify MCP connector,
so the Actor never handles my token.

That difference is the whole story. Two days after I published it, Apify flagged it as under
maintenance. I spent an afternoon removing the label by hand and learned that the label is not a
setting. It is a conclusion.

The email

The notification arrived at 10:08 on 8 August. My Actor had failed the automated tests.

Apify runs a daily test on every Actor.
The documentation is precise about what it does:

The test runs the Actor with its default input (defined by the prefill option in the input schema
file) and expects it to finish with a Succeeded status and non-empty default dataset within 5
minutes of the beginning of the run.

My Actor cannot do that. Not on a bad day — on any day.

Why this Actor and not the other 22

My other Actors take a URL, a domain, or a package name. Every one of those is a string. A string
goes in the prefill of the input schema, the test picks it up, and the run succeeds.

An authorized connector is not a string. It is a grant, tied to an account, held by the platform.
There is no value I can put in prefill that gives the test account access to my GitHub
installation. Nor should there be.

So the run fails at the first step, every day, forever.

The documentation says this outright, in a section I had not read:

Actors that require some sort of authentication will always fail the tests despite being fully
functional.

Apify documentation section titled What if my Actor cannot comply with the test logic, stating that Actors requiring authentication will always fail the tests despite being fully functional, and directing developers to contact support

I want to be exact about what this means, because I got it wrong at first. I had assumed the cost
belonged to my design choice of failing fast on empty input. It does not. Twenty-two of my Actors
fail fast on empty input and none of them were flagged. The cost belongs to a narrower category:
Actors whose only successful path requires a credential the test account cannot hold.

If that describes your Actor, the daily test is not a quality signal about your code. It is a
structural mismatch, and it will not resolve itself.

What I did instead of reading the docs

I went to Console. Under Publication → Display information there is an Actor status control with an
"Under maintenance" switch. I turned it off and saved. The banner disappeared. The public page
stopped showing the badge.

Apify Console Actor status section with three toggles, Custom status, Under maintenance and Deprecated, all switched off

I recorded it as fixed. It was not fixed. It was hidden.

At 17:08 the same day, about four hours after I cleared it, the second notification arrived. The
label was back.

I removed it again. At 18:08 — inside an hour — the third notification arrived.

Two removals, two returns, and the interval got shorter. That second number is what made me stop
and read.

The label is not a setting

Here is the mechanism, and it is stated plainly in the same documentation page I had skipped:

If the Actor fails to complete successful runs for three consecutive days, the developer will be
notified, and the Actor will be labeled under maintenance until it is fixed.

And the other direction:

The best course of action is to fix the Actor and rebuild it. The automatic testing system will
pick this up within 24 hours and mark it as healthy. In some cases, your Actor might break because
of issues with the target website. In such a case, if your Actor passes the majority of test runs
in the next 7 days, it will be marked as healthy automatically.

Read those two together. The label is derived from your run history, and the system recomputes
it. The switch in Console writes to a field that an evaluator overwrites on its own schedule.

For a normal broken Actor, the switch is honest and useful. You fix the bug, you rebuild, you clear
the label, and the next evaluation agrees with you. Your manual action and the computed state
converge.

My case had no convergence. The failures were still in the window and would be there tomorrow too.
Every removal I made was a claim the evidence contradicted, so the evaluator restored it.

I also stopped for a second reason, which has nothing to do with mechanics. Repeatedly toggling a
flag that an automated system keeps restoring is not a fix. It is an argument with a robot, in
public, on my own store listing. I did not want to find out how that reads from the other side.

The path that actually worked

The documentation tells you where to go:

If that's the case with your Actor, contact support and explain your specific use case that
justifies why the Actor should be excluded from the automated tests.

There is also a form. While my Actor was flagged, Console opened a skip-test request straight from a
query parameter:

https://console.apify.com/actors/<actorId>/publication?showSkipTestForm=true
Enter fullscreen mode Exit fullscreen mode

One caveat I can only report, not explain. Two days later, with the exemption granted, that same URL
renders the ordinary publication page for me. So treat it as something that is there while you are
flagged, not as a permanent entry point. If it does not open, the documented route is support, and
that is the route the docs tell you to take anyway.

I filled it in with three sentences: what the Actor does, why the test account cannot authorize the
connector, and the sentence from the docs that describes exactly my situation. Quoting their own
documentation back to them felt lazy. It was the right call — it turned a request for special
treatment into a request to apply a rule that already existed.

The reply came in about 31 hours:

After reviewing your situation, we've approved your request. Your Actor will now not be checked by
our tests.

Then I cleared the label a third time. This one held, because there is no longer an evaluator
producing a verdict to overwrite it.

The order matters and I had it backwards. Get the exemption first. Clear the label second. Doing
it the other way round produces exactly what I got: a label that returns on a schedule you do not
control.

How to check the flag without lying to yourself

While I was doing this I needed a reliable way to answer "is it flagged right now?" I tried three
and only one of them is trustworthy.

The Console banner is fine, but it is one Actor at a time and it needs a browser.

The public Store page is not reliable. After my third removal I fetched the page and grepped for
the phrase. The badge was still in the HTML, on a page that was already correct in Console. When I
checked again later the same day, it was gone. So the page is not wrong — it lags, and on that day
the lag was hours. If you grep the page right after a change, you will read your own stale render
and conclude the change failed.

The API is the source of truth. The Actor object carries a top-level notice field:

curl -s "https://api.apify.com/v2/acts/aiqlabs~dataset-to-github-issues" \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['data']['notice'])"
# NONE
Enter fullscreen mode Exit fullscreen mode

Selected fields from the live API response for the Actor, showing notice set to NONE alongside isPublic true and three categories

The values I have observed are UNDER_MAINTENANCE and NONE. This endpoint answers for public
Actors without a token, which means you can check anyone's — including, before you fork it, the one
you are about to depend on.

I did not find notice documented as the maintenance flag. I found it by reading the object.

A check you can run on everything you publish

After this, I stopped trusting my memory of which Actors were healthy. This prints the flag for a
list of slugs:

#!/usr/bin/env bash
# Print the maintenance flag for every Actor you name. No token needed for public Actors.
USER="aiqlabs"
for slug in dataset-to-github-issues github-repository-audit pdf-table-extractor; do
  notice=$(curl -s "https://api.apify.com/v2/acts/${USER}~${slug}" \
    | python3 -c "import json,sys; d=json.load(sys.stdin).get('data') or {}; print(d.get('notice','MISSING'))")
  printf '%-32s %s\n' "$slug" "$notice"
done
Enter fullscreen mode Exit fullscreen mode

I ran it across the Actors I care about after the approval. All NONE, including the twenty-two
that were never at risk. That last part matters: it confirmed the flag had not spread from a shared
cause.

The trap next door

One warning that cost me three rounds of rework, because it sits on the same Console screen.

The Actor status switch lives under Publication → Display information. Saving that section dropped
one of my categories, three times in a row.

I had set three categories through the API. After each save, OPEN_SOURCE was gone, and I put it
back through the API.

Three for three is not an accident. But I want to be exact about the limit of what I know: when I
opened that same form today to write this, all three categories were sitting in the field. So I can
report the outcome I measured and not the cause.

So if you go anywhere near this screen, re-apply your categories through the API afterwards, and
read them back:

curl -s "https://api.apify.com/v2/acts/aiqlabs~dataset-to-github-issues" \
  | python3 -c "import json,sys; print(json.load(sys.stdin)['data']['categories'])"
# ['DEVELOPER_TOOLS', 'AUTOMATION', 'OPEN_SOURCE']
Enter fullscreen mode Exit fullscreen mode

Do not confirm this in the UI you just used. Confirm it in the API.

What it costs to ignore this

The label is not cosmetic, and the timeline is published:

After another 14 days of failing runs, you will receive another notification. Finally, if the runs
continue to fail after yet another 14 days, the Actor will be deprecated.

Three days of failures to get labeled. Roughly a month of them to get deprecated. For an Actor that
structurally cannot pass, that clock starts on the day you publish and never stops. Nothing in your
code will change it, which is precisely why manual removal feels like it works and is the most
expensive thing you can do with the time.

The rule I now apply

Before I publish an Actor that needs an authorized connector, I ask one question: is there any
input the test account can supply that produces a successful run?

If the answer is yes, I make sure that path is what prefill describes, and the daily test becomes
a free health check I get for nothing.

If the answer is no, I file the skip request in the same session as the publish, before the first
notification arrives. Then I never touch the status switch, because by then there is nothing left
for it to argue with.

The Actor in this story is dataset-to-github-issues.
Its source is public under ISC at
github.com/ai-q-labs/dataset-to-github-issues,
and the connector it uses speaks the Model Context Protocol.
The rest of what I publish is at apify.com/aiqlabs.

Top comments (0)