DEV Community: Murph Murphy

The Hidden Dangers of Face Embeddings: Unmasking the Privacy Risks

Murph Murphy — Fri, 26 Jan 2024 21:55:31 +0000

One of the classic applications of machine learning is facial recognition, where a model is trained to find faces in images or videos and capture identifying information about them. Facial recognition has many applications, but some of the most useful ones involve taking facial features and matching them against other useful data previously collected and tied to the same face, which requires creating an embedding. Unless your facial recognition system is simply marking where to find faces in pictures or video, it’s probably generating an embedding from each face it identifies.

What are embeddings?

An embedding is simply a big list of numbers, typically represented as a vector of 300 to 4096 numbers between -2 and 2. That list of numbers represents a machine learning model’s complex interpretation of the meaning in the input. Inputs can be text, images, audio, photos, and more. It’s a distillation of the extracted meaning – the model itself may have tens of thousands or millions of traits and categories – so each value in the embedding has a lot of information packed into it. This density of information isn’t something a human can follow, and because of that, embeddings are frequently (incorrectly!) thought of as meaningless if exposed.

Embeddings are then mathematically used to find similar meanings across stored data, cluster data, find anomalies, and so forth.

Once an embedding has been created for a face, it can be compared to other embeddings that were generated for other faces. Ones that are “near” each other (mathematically) are similar faces and ones further away are different faces.

This technique enables looking someone up using an image of their face, linking it to information about that person, and using that for some of the most exciting (and scary) applications of facial recognition.

Privacy and embeddings

In this blog post, we’re going to reverse embeddings back into recognizable faces. We’ll demonstrate how we could take a database of face embeddings and convert them back to privacy-invading photos. We’ll then show you how to prevent attacks like this one.

Protecting embeddings is important because the number of places capturing and storing this information is growing. There’s already been tremendous abuse in at least one case where the FTC stepped in. As more and more stores, governments, and others add facial recognition functionality, the likelihood of abuse or accidental exposure of these embeddings grows as well.

Attacking embeddings

If an intruder stole embeddings from your database, they could immediately compare them with their own database of embeddings to identities for a match. Then they could associate whatever information you had connected with that identity (customer bases, medical data, buying habits, etc). An encrypted embedding (discussed below) wouldn't also passively protect against this. That attack requires the intruder to already have their own database of identities to embeddings, instead we'll talk about attacks that involve only the stolen embedding.

There are several different forms of attacks we could mount on facial recognition embeddings that attempt to get as close as possible to the input face (called an inversion attack). Some of the more sophisticated attacks involve training a machine learning model on the embeddings themselves, so it can produce a face when given an embedding as an input. Those attacks require a significant amount of work to create, but there’s at least one attack that doesn’t even require us to train a model while still getting good results, which is what we’ll be doing here.

We’re going to take a facial recognition embedding and find the closest pre-generated face we have to start from. This is purely a speedup step as the starting face could be random. Then we make small tweaks to that face, generate a new embedding, and see if the face (embedding) is more like the target or less like it. Rinse and repeat as many times as we’d like.

from “Realistic Face Reconstruction from Deep Embeddings” by Edward and Joshua Vendrow

This approach comes from the paper “Realistic Face Reconstruction from Deep Embeddings” by Edward and Joshua Vendrow and makes use of a modified version of their published implementation of that paper. The code we used to do this is on Github and you can try it yourself in a browser using this Colab notebook.

Results of the attack

We ran our attack with a picture of Tom Cruise as the input for 400 iterations. This run took 10 minutes and resulted in this:

We upped the iterations to 2000 because each try was still getting closer and ended up with even better results after 40 min:

While neither is exactly the face that was input, they’re both pretty close, especially in facial structure and features. Close enough that if someone had stolen this embedding they’d have a very good idea of what the input face looked like.

How do we prevent it?

Encrypting the vectors with IronCore Labs’ open source (AGPL) vector encryption library, Cloaked AI, prevents attacks like this as well as the ones that train an inversion model.

We ran the same attack as before on the same picture of Tom Cruise, but this time we encrypted the embedding with Cloaked AI before attacking it. We still ran it for 400 iterations for comparison’s sake, even though the results weren’t improving across iterations. This run took 10 minutes and resulted in this:

As you can see, the reversed image isn’t even close to the original picture this time.

The encrypted vectors use much larger numbers than the original vectors, but even if we normalize those values to bring them into the expected ranges, the face still doesn’t get close. The attack essentially produces a random face with no relation to the original.

Isn't the embedding useless when encrypted? No!

Even though we’ve made the embedding much more secure against attack, we’ve used encryption that mathematically preserves its approximate relative distance from other embeddings encrypted with the same key. That means that all the things you’d want to use the embeddings for such as similarity based retrieval, matching, classification, anomaly detection, etc., will keep working! If you don’t have the right encryption key, you can’t meaningfully do any of those things, and you can’t reverse the embeddings anymore either.

Embeddings are sensitive

Embeddings are an enormously important source of information. Faces can be used to track where people have been, what they’ve purchased, who they associate with, or even what they’ve paid attention to in a store. This is incredibly invasive information, and protecting that data is critical. Luckily, it’s also easy to do with an inexpensive open-source software library.

This same process can apply to any embeddings, not just facial recognition ones. Up next in this series we’ll take a look at attacking text embeddings, which have been made very popular due to hybrid search and retrieval augmented generation (RAG). RAG makes it possible for natural language queries to find relevant information to ground and inform an AI that can then answer the question.

Windows Subsystem for Linux Java Setup

Murph Murphy — Thu, 12 Mar 2020 15:54:27 +0000

The tools are now available to make development on Windows as easy and standard as development on Unix has been. I couldn't find a great guide that covered a full setup for a Java developer, and I needed to do it a few times myself recently. This is a quick guide to get Windows 10 users with no devtools on their system set up with Windows Subsystem for Linux, JDK 11, maven, and the editor of their choice (with slight VSCode favoritism).

Set Up WSL

This section and the next is pretty much ripped from the official guide.

Press the Windows key and type "Windows Features" and open the "Turn Windows Features On or Off" option. Scroll down to the "Windows Subsystem for Linux" feature and check the box to enable it. You'll need to reboot your computer after this step.

Now hit that Windows key again and type "Microsoft Store" and open that bad boy up. Search the store for "Linux" and click the "Get the apps" option that comes up. Choose "Ubuntu" from the options that show up and click "Install" on that page.

Set up Ubuntu

Press the Windows key again and type/open "Ubuntu". This will trigger a bit of additional setup the first time.

It'll ask you to create a username and password for the subsystem, do so. Once you've done that you'll be dumped into bash in your Ubuntu subsystem!

Note that your whole Windows C drive will be mounted at /mnt/c in Ubuntu, so ls /mnt/c should look familiar.

Set up Java

Now we need a couple more tools to get the system Java ready.

sudo apt update will pull down updated mirrors for downloading applications from.

sudo apt install openjdk-11-jdk maven will install the Java Development Kit (necessary for any Java work) and Maven (a common build tool, feel free to substitute gradle or something else).

Set up an editor

I recommend using VSCode because it has a nice server set up so you can run VSCode in Windows while having it interact with your files in Ubuntu, but any editor will work. If you use something other than VSCode (vim, Notepad++, SublimeText, Atom, etc) you'll likely want to do your development somewhere on your C drive such as C:\code\project and access it from Ubuntu at /mnt/c/code/project. With that said I'll move forward with VSCode setup.

Download VSCode for Windows and install it.

Start up VSCode in Windows and press ctrl + shift + p, type "Install Extensions" and press enter. Type "windows subsystem" and click install on "Remote - WSL" by Microsoft. Once that's finished, close VSCode in Windows.

Close your Ubuntu bash window if you still have it open, and re-open it. Type code .. This will set up VSCode's WSL server and launch VSCode back in Windows for you. Press ctrl + shift + p, type "install extensions", press enter, and search for "java red hat", and install "Java Language Support" by Red Hat.

Done! You're all set up to develop a Java project in Ubuntu in WSL!

Tips and Tricks

Use code . in the future to open VSCode in whatever directory you want, so you'll always open it with the Ubuntu <-> Windows connection set up.
With your Ubuntu bash window open you can right click the title bar and select Properties. Here you can turn on "QuickEdit mode" and "copy/paste with Ctrl+Shift+C/Ctrl+Shift+V" to enable easy selection and copy/paste between Ubuntu and Windows. Now if you pin Ubuntu to your task bar it'll always start up with those options set.
You should be able to get away with a lot just using ls (list files), cd (change directory), rm (remove a file), git, java/javac/jar, and mvn (java build tool). If you throw -h on any of those commands you'll get a quick set of options. You can search for manpages for any of them for more detail.
This free MIT resource covers a lot of computing basics that everyone should know. It's useful if you feel uncomfortable in Unix.
apt is the application installer on Ubuntu. You can apt search for programs and apt install to install them. Searching for help with apt should be easy, there's nothing different about the WSL Ubuntu version.

Deserializing Doubly Tagged JSON in Rust

Murph Murphy — Fri, 15 Nov 2019 19:22:04 +0000

I needed to deserialize a JSON data structure that contained versioning and type information.

[
  {
    "provider": "AZURE",
    "version": 1,
    "credentials": {
      "clientId": "",
      "clientSecret": ""
    }
  },
  {
    "provider": "AWS",
    "version": 1,
    "credentials": {
      "accessKeyId": "",
      "secretAccessKey": ""
    }
  }
]

My Rust representation had to represent that the datatypes could change with subsequent versions and capture (preferably at the type level) the differences between the sub-structs. I tried a few different patterns for representing this data in Rust but settled on one that seemed be the nicest to work with later.

pub enum ProviderType {
    AZURE,
    AWS,
}

// How do I deserialize this?
pub enum ProviderConfiguration {
    AzureConfigV1(AzureConfigV1),
    AwsConfigV1(AwsConfigV1),
}

// AZURE
pub struct AzureConfigV1 {
    pub credentials: AzureCredentialsV1,
}

impl AzureConfigV1 {
    pub const TYPE: ProviderType = ProviderType::AZURE;
    pub const VERSION: u16 = 1;
}

pub struct AzureCredentialsV1 {
    pub client_id: String,
    pub client_secret: String,
}

// AWS
pub struct AwsConfigV1 {
    pub credentials: AwsCredentialsV1,
}

impl AwsConfigV1 {
    pub const TYPE: ProviderType = ProviderType::AWS;
    pub const VERSION: u16 = 1;
}

pub struct AwsCredentialsV1 {
    pub access_key_id: String,
    pub secret_access_key: String,
}

I wanted to be able to deserialize these types with as little effort as possible. Serde provides some pretty great attributes for deserializing enums but none of them covered the combination of version and type going on here. The manual deserialization in the serde documentation was pretty confusing to me as someone new to Rust, it's presented without much commentary.

I tried writing Deserialize using a Visitor like the example, but didn't have access to the original deserializer inside the visit_map.

impl<'de> Deserialize<'de> for ProviderConfiguration {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        #[derive(Deserialize)]
        #[rename_all = "camelCase")]
        enum Fields {
            Provider,
            Version
        };

        struct ProviderConfigurationVisitor;

        impl<'de> Visitor<'de> for ProviderConfigurationVisitor {
            type Value = ProviderConfiguration;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("ProviderConfiguration")
            }

            fn visit_map<V>(self, mut map: V) -> Result<ProviderConfiguration, V::Error>
            where
                V: MapAccess<'de>,
            {
                let mut provider = None;
                let mut version = None;
                while let Some(key) = map.next_key()? {
                    match key {
                        Field::Provider => {
                            if provider.is_some() {
                                return Err(de::Error::duplicate_field("provider"));
                            }
                            provider = Some(map.next_value()?);
                        }
                        Field::Version => {
                            if version.is_some() {
                                return Err(de::Error::duplicate_field("version"));
                            }
                            version = Some(map.next_value()?);
                        }
                    }
                }
                let provider = provider.ok_or_else(|| de::Error::missing_field("provider"))?;
                let version = version.ok_or_else(|| de::Error::missing_field("version"))?;
                match (provider, version) {
                    ("AWS", 1) => AwsConfigV1::deserialize(deserializer).map(ProviderConfiguration::AwsConfigV1) // the fn we're in doesn't close over the original deserializer
                }
            }
        }

        const FIELDS: &'static [&'static str] = &["provider", "version"];
        deserializer.deserialize_struct("ProviderConfiguration", FIELDS, ProviderConfigurationVisitor)
    }
}

So I thought I may be able to deserialize the whole original value into a helper that only cares about provider and version then use the resulting helper to deserialize the value as the correct struct.

impl<'de> Deserialize<'de> for ProviderConfiguration {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        #[derive(Deserialize)]
        struct Helper {
            provider: ProviderType,
            version: u16,
        };

        let helper = Helper::deserialize(deserializer)?;
        match helper {
            Helper {
                provider: AwsConfigV1::TYPE,
                version: AwsConfigV1::VERSION,
            } => Ok(ProviderConfiguration::AwsConfigV1(
                AwsConfigV1::deserialize(deserializer)?,
            )),
            _ => Err(de::Error::duplicate_field("nothing")),
        }
    }
}

This didn't work because I couldn't move the deserializer and then use it again. This felt like the right track though, and much closer semantically to the way I expected this to work.

I tried a few other pathways that didn't pan out (multiple flattened internally tagged enums), and in trying to figure out why they didn't work eventually stumbled across an old question that very closely mapped to my use case.

impl<'de> Deserialize<'de> for ProviderConfiguration {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        #[derive(Deserialize)]
        struct Helper {
            provider: ProviderType,
            version: u16
        };

        let v = Value::deserialize(deserializer).map_err(de::Error::custom)?;
        let helper = Helper::deserialize(&v).map_err(de::Error::custom)?;
        match helper {
            Helper {
                provider: AwsConfigV1::TYPE,
                version: AwsConfigV1::VERSION,
            } => Ok(ProviderConfiguration::AwsConfigV1(
                AwsConfigV1::deserialize(&v).map_err(de::Error::custom)?,
            )),
            _ => Err(de::Error::custom("")),
        }
    }
}

Perfect! Here's a playground with the final working code.