Enriching a Knowledge Graph using Connectors and Functions

I was recently developing a React web app designed by one of the product designers on my team. She created a clean looking home screen design that featured some image carousels where each image appears over top of a light orange background. After I wrote the necessary components, I realized they didn’t look the way I wanted.

Each image returned by the Answers API had a white background when I needed them to have a transparent background. I knew that Cloudinary offered an image transformation API that I could use to remove the white background, but I needed to transform every image for over 1000 entities in my Knowledge Graph. In this article, I will show how I transformed every image in my Knowledge Graph data with the Cloudinary Image Transformation API, a function plugin, and the Data Connector framework.

Writing My Function Plugin

All of the images in my Knowledge Graph were JPEGs which do not support transparent backgrounds. I needed to use Cloudinary chained transformations to first convert my images to PNGs and then remove the white background. I also chose to use Eager Transformations to receive the transformed asset immediately after storing it in my Cloudinary account.

After creating my plugin folder structure, added uploadImageTransform to mod.ts to make the transformation request and return the new URL.

// Chained, eager tranformation 
const eager = "f_png,e_bgremoval";

const uploadImageAndTransform = async (
  file: string,
  public_id: string,
  signature: string,
  timestamp: number
): Promise<string> => {
  console.log(`Uploading ${public_id} to Cloudinary and transforming ${file}`);

  try {
    const response = await axiod.post(
      "https://api.cloudinary.com/v1_1/yext/image/upload",
      {
        file,
        public_id,
        api_key: CLOUDINARY_API_KEY,
        eager,
        signature,
        timestamp,
      },
      {
        headers: {
          "Content-Type": "application/x-www-form-urlencoded",
        },
      }
    );

    return response.data.eager[0].url;
  } catch (error) {
    if (error.response) {
      console.error(`${error.response.status}: ${error.response.data}`);
    }
    throw error;
  }
};

Cloudinary’s transformation API requires a digital signature to provide authentication for each request. Since I am not using a Cloudinary SDK, I had to write my my own function for generating a signature using the web crypto API that is included with Deno.

const generateCloudinarySignature = async (params: string) => {
  const data = new TextEncoder().encode(params);
  const digest = await crypto.subtle.digest("sha-1", data.buffer);
  return encodeToString(new Uint8Array(digest));
};

I then wrote the removeBackground function for my Data Connector to call. The function accepts a string that is expected to be delineated with a ‘|’. The public_id is intended to be the Knowledge Graph entityId and url is the original url of the photo that needs transformed.

export const removeBackground = async (input: string) => {
  const timestamp = Math.floor(Date.now() / 1000);
  const [public_id, url] = input.split("|");

  const signature = await generateCloudinarySignature(
    `eager=${eager}&public_id=${public_id}&timestamp=${timestamp}${CLOUDINARY_API_SECRET}`
  );

  if (public_id && url) {
    const imageUrl = await uploadImageAndTransform(
      url,
      public_id,
      signature,
      timestamp
    );

    return imageUrl;
  }
};

I wrote a function in test.ts to make sure the transformation worked for one of the images in my Knowledge Graph.

const testInput =
  "beer_60|https://a.mktgcdn.com/p-sandbox/rrK5bxcoVAgkGckDnA7GlhyC1VOpV6eEf4KjjlFumQs/400x600.jpg";

Deno.test("test remove background", async () => {
  const newUrl = await removeBackground(testInput);
  console.log(newUrl);
});

Applying the Function Plugin to a Data Connector

After uploading my function plugin to my Yext account, I created a new Data Connector with “Pull from API” as the source to pull the entities from my own account using the Knowledge Graph Entities: Get API. I also needed to create an app in the Developer Console to get an API Key to allow my Data Connector to make requests to the Entities API.

I autogenerated my selectors and then applied a Merge Columns Transformation to combine entityId and primaryPhoto.image.url into a new column called Transformed Image.

I took the new column and applied my removeBackground function to it.

Finally, I mapped the meta.id back to entityId to let the Connector know that I am editing existing entities and replacing the Primary Photo Image URL with a new URL.

I ran my newly created Connector and it completed in less than 5 minutes. When I refreshed my app home screen, the white backgrounds were gone.

Conclusion

I had the option of making a request to Cloudinary from my React application each time I wanted to fetch the transformed image. However, I am already fetching the entities from my Knowledge Graph via the Answers API. Making the additional request to Cloudinary would eat into my month bandwidth allotment. If this were a production application, I would go over my free account allotment pretty quickly. This way, the resources are only fetched when I make the initial request for each image on the Connector run.

This example could be extended to apply more complex image transformations. I could also upload a separate function to call a different API for further Knowledge Graph enhancements.

You can see the complete code for the function here and read more about Data Connectors here.