How we reduced our review time by 90%

Adrian Brown — Thu, 11 May 2023 21:30:47 +0000

At Apollo, we use a no-code automation platform to allow users to manage their integrations & tasks. There are many debates over whether automating content moderation is a one-size-fits-all approach, it's not. AI powered content moderation has become table stakes in protecting and enhancing user experience at scale, but for us, visibility into other areas of enterprise/personal decision making led us to use tools like automation rules & workflow platforms.

TLDR;

One case study for content moderation from YikYak, we migrated from Retool.

The bigger, the slower

With all the advantages, there are a few drawbacks to using retool and a standalone automation system for task monitoring/completion. We noticed a particular drawback when scaling the number of users and amount of activity with each one: The time it takes to detect bad actors accurately and then investigate. So a typical task for a investigation would run anywhere between 3 minutes to 4 days. And that's for each time an automated flag was created or a user report was sent.

More than that, truly having accurate data was a nightmare. Standalone automation systems (environments with one AI model) would score correctly only a fraction of the time.

This amount of time spent investigating and analyzing to make an informed decision reduced the moderator efficiency and wasted collectively so much talented people's time. Being a social app dedicated to curating safer community, this was unacceptable.

Debugging the slowest tasks

Inspecting a typical 4 day case/support ticket, it was clear that two specific steps took almost 70-80% of the overall time:

Gathering metadata from existing platform policy, account information and community guidelines: 4 Hours
Connecting historical data to recent events: ~1 day

Migrating from Standalone Automation Systems to Integrated Automation System

Retool coupled with our standalone automation system provided a fast, efficient infrastructure as a service toolkit, and from some of the benchmarks, there was a massive improvement in active response time when migrated off of a independent automation system to an integrated automation system like Apollo.

Moving from Retool + Standalone automation took around 2 minutes, the migration to integrated automation (Apollo) was effortless: Just adding metadata about your internal databases and a click of a button, that's all. The aggregation and integration for each database/api resource were efficiently synced, in... wait for it... just milliseconds! And that's without any code at all! After integrated automation tools like Apollo increases the accuracy of the scored content, it takes less than 40 seconds to deploy and integrate multiple AI models under one API.

Reducing ~3 minutes to 13 seconds from the standalone's system time for every investigation and accuracy for moderators is a HUGE win.

Summary