Facial recognition + good faces bot = best faces 🏆

@good_faces_bot is probably the best Twitter account there is. It's pretty self-explanatory: just a lot of good faces posted regularly. I wanted to sift through the endless stream to find the best faces and this was the result: bryce.io/best-faces.

Get the faces 🐦

This part doesn't need much explaining, everyone with basic coding experience knows how to and probably has fetched from Twitter's API. Borrowing from the timelines quick-start guide I got an API token and downloaded all the tweets for the account.

After that, it sorts them and filters for tweets above a certain threshold of likes (1000). Then the last thing to do is to download the image associated with each tweet with a slight staggering to avoid getting rate-limited and it's ready to be analyzed!

Enter: rustface 🔬

Put simply, this is a Rust-y wrapper around the SeetaFace Engine, an excellent open-source tool for real-time facial recognition. It works surprisingly well given applications like this generally need massive amounts of processing power. Can read more about it here.

The included example was very helpful; I modified it to loop over the list of images and made it so instead of drawing a rectangle over the face it outputs a cropped image of the face. With some minor error-handling it was making it through all of the images in no time!

Unfortunately it only recognizes a face in about 1/3 of all the images currently (and some of the "faces" aren't actually faces, like the first one on the page 😂). With some tweaking this could be improved though.

Putting it all together 🪢

Now that the Rust part is complete, all that's left is to display the faces somehow. I used Svelte to build a simple site since I like using it for small projects like this.

The trickiest part here was that although the Rust step generated a bunch of images, there isn't an easy way to connect them back to each tweet. I used fs.exists to loop over the output directory and generate a bunch of imports & exports. It's hacky but ended up working well! But beware 👻 you shouldn't be generating raw javascript like this unless there are no other options, here's a snippet:

const hasOne = await fsExists(generatePath(tweet));

if (hasOne) {
    facePaths.push(generatePath(tweet));
}

facePaths.forEach(facePath => {
  output += `import ${faceFileName} from "../${facePath}";\n`;
});

output += `export const facesFor${tweet.id} = [${faceFileNames.join(',')}];\n`;

await fs.appendFile(outputFile, output);

Anyways once that was done I could loop over the list of tweets from the first step and display them in Svelte, referencing the facesForX import directly:

{#each tweets as tweet}
    {#if faces[`facesFor${tweet.id}`].length > 0}
        {#each faces[`facesFor${tweet.id}`] as face, i}
            <Face face={face} />
        {/each}
    {/if}
{/each}

Note that it checks if length > 0: since it isn't able to detect a face in every image, unfortunately some get left behind. It's sad too since a lot of the really good faces are the stranger looking ones.

Then I added a simple hover state to show the original image that the face was detected in:

<script>
    export let tweet;
    export let face;

    export let show = () => showOriginal = true;
    export let hide = () => showOriginal = false;

    let showOriginal = false;
</script>

<a href={tweet.tweet.text}>
    <img src={showOriginal ? tweet.media.url : face} on:mouseenter={show} on:mouseleave={hide} />
</a>

That's pulled directly from the Svelte component too! No wrappers or boilerplate, part of the reason why it's so satisfying to use for development.

---

Et voilà! The last kicker is that since all of these are just scripts, they can run sequentially in a GitHub Action! Although the facial recognition part takes a while (~20m) the simpler ones can fetch/generate the necessary data and run entirely in CI. There are some optimizations to be done here around uploading/downloading artifacts but in the meantime I'm happy it all works remotely.

Check out the code here:

brycedorn / best-faces

Using rustface to determine good_faces_bot's best faces

best-faces

@good_faces_bot is probably the best Twitter account there is. It's pretty self-explanatory: just a lot of good faces posted regularly. I wanted to sift through the endless stream to find the best faces and this was the result!

Requirements

• node
• rust
• Twitter API key, can get one here

Running locally

First create a .env file with your BEARER_TOKEN for accessing Twitter's API and the USER_ID for the account you want to fetch tweets for.

Then, in the following order:

npm install
npm run fetch:tweets
npm run sort:tweets
npm run fetch:images

And for rust:

cargo install
cargo run

Finally for the site:

npm run generate:imports
npm run dev

View on GitHub