Validating API keys in ChatCraft!

Picking up from my last post, I will show the progress I've made in my code contributions to ChatCraft, a developer-oriented version of ChatGPT.

Issue Link

https://github.com/tarasglek/chatcraft.org/issues/230

Issue Description

Add validation for OpenRouter API key

---

Identifying the file or area of code to work on

For this issue, since it is about validating the OpenRouter API key, I used VSCode's search to help me locate where to get started.

I searched the line Please choose your API provider and enter your API Key below to get started! in VSCode and was able to find the React form responsible for submitting the API key which is located in the Instructions.tsx file.

This form, upon submission, triggered a function called handleApiKeySubmit. To go to this function, I simply CTRL+clicked the function name in VSCode.

It was hard to figure out which function or if statement was supposed to be in charge of validating the API keys. After adding some console.log messages and testing each scenario (valid OpenAI key, invalid OpenAI key, valid OpenRouter key, invalid OpenRouter key), I determined the following:

• validateOpenAiApiKey() always returned for OpenRouter key no matter if it is valid or invalid
• validateOpenAiApiKey() only returned for OpenAI key when the key is valid, and threw when the key is invalid

Now I know that there is validation happening in the validateOpenAiApiKey function for OpenAI keys, and that is probably where the I need to add the validation for OpenRouter keys.

Investigating validateOpenAiApiKey() in ai.ts:

I had to google what !! means here - it converts an object to boolean. If queryModels threw, it would return false; otherwise it would return true.

Investigating queryModels() in ai.ts:

After adding some more console.log messages and re-testing the scenarios, I deduced that for await (const page of openai.models.list()) would throw when the OpenAI key was invalid. For valid OpenAI keys, or a valid or invalid OpenRouter key, it would not throw.

Now I have ascertained that this is where I should add my validation code for OpenRouter keys.

---

Using OpenRouter Limits API for key validation

Following the official OpenRouter docs, I sent a GET request to to OpenRouter to check the rate limit or credits left on an API key. In theory, this request should also be able to help me to check whether an API key is valid or not.

const res = fetch("https://openrouter.ai/api/v1/auth/key", {
  method: 'GET',
  headers: {
    'Authorization': 'Bearer ${apiKey}'
  },
});

To test my theory, I printed out the response using console.log

Response of entering valid key:

Response of entering invalid key:

From these responses I could see that res.ok is true for valid OpenRouter api keys and false for invalid ones.

---

Implementing my validation code

---

Testing

I did testing with both valid and invalid OpenRouter keys. I also did regression testing with OpenAI keys to ensure I did not affect existing validations for OpenAI. See PR for all testing screenshots.

---

Pull Request

Added validation for OpenRouter API key #306

kliu57 posted on Dec 13, 2023

Closes #230

Added validation for OpenRouter API key

Current code (before PR):

• In src/lib/ai.ts, queryModels() function, when OpenAI API key is invalid, throws an Error
• No validation for OpenRouter API key

PR Code changes:

• In src/lib/ai.ts, queryModels() function, use response from https://openrouter.ai/docs#limits to check if OpenRouter API key is valid and throws an Error if response.ok is false

Testing:

Invalid OpenRouter API key:

Valid OpenRouter API key:

Regression Testing:

Invalid OpenAI API key:

Valid OpenAI API key:

View on GitHub

I fixed prettier issues, pushed my code to my branch, and created a pull request with details of my changes and screenshots of my testing.

---

Code review and fixes

What I did was check if we are using OpenAI and in the else clause, added the validation for OpenRouter. However, the repo owner preferred for me to specifically check that we are using OpenRouter. I changed my code accordingly.

Another code change I was asked to do was move the validation code out of the queryModels() function, since that function is specifically used by OpenAI. I created a new generic function validateApiKey() which validates validates both OpenAI and OpenRouter keys, by calling validateOpenAiApiKey() or validateOpenRouterApiKey() respectively. I moved my validation code out of queryModels() and into validateOpenRouterApiKey().

[Update] My PR has been merged!

---

Follow up issue

Next, I plan to follow up and work on a related issue to clear stored API keys when the user changes their API provider in the User Settings.

I was given a few choices on what the expected behaviour of this issue could be, and I decided to go with the easiest one, since I was low on time. When the user selects a different provider from the dropdown, we will clear the stored API key.

In the future, when I get more time, I would like to implement the functionality of being able to store both OpenRouter and OpenAI keys separately, with no need to clear it when we select a different provider.