Katie Liu

Posted on Apr 6

ChatCraft week 12: PR merged and follow ups!

#opensource #chatcraft #typescript #chatgpt

Landed PR

This week I finally landed the custom providers feature I have been working on! See my last blog for more details about it. Now if you have an Open AI compatible API url and key, you can use ChatCraft as your user interface to communicating with the API!

Support custom providers #530

kliu57 posted on Mar 27, 2024

Closes #503 Closes #440

Users are now able to add custom providers and the user settings modal now shows a table of providers.

Implemented functionalities:

Changed the display of the providers section to include a user-friendly and responsive table

Add Provider button:

Add custom providers by clicking "Add", typing info, clicking "Save" button
Users are allowed to add any provider that we currently support (openai, openrouter, and any future ones we implement - excluding FreeModelProvider) as well as any custom provider that is open ai compatible.
If the provider is not compatible, key or url is invalid, provider will not get added and user will see an error message.
Allowing the users to create multiples of the same provider (i.e. openai) since they may want to manage multiple keys.
Not allowing users to create FreeModelProvider since that does not have a key so you wouldn't need multiples of it.

Other buttons/fields:

Delete custom providers using "Delete" button (only custom ones can be deleted)
Change the current provider using "Set as Current Provider" button
Change the Api Key any of provider in the list by editing the Api Key field, automatically key validation will occur (error message will display if the key is invalid)

Code changes (only summary of important points):

PreferencesModal.tsx (core code is here)

focusedProvider state var tracks which provider row we are editing. This is what allows us to display the "Key is invalid" message on the correct corresponding row.
selectedProvider state var tracks which provider we have clicked the checkbox on
newCustomProvider stat var stores the data user typed in the new provider row

SmallRevealablePasswordInput.tsx created this so I don't mess with the styles of RevealablePasswordInput.tsx since we still use that in Instructions page and we may bring back Instructions page someday

use-alert.ts

added two more types of toast alert in here since the existing types don't disappear fast enough for my liking

Had to make changes to OpenAiProvider.ts and OpenRouterProvider.ts to take in a name param so that users will be able to have multiple Open AI or Open Router providers with different names

Had to add defaultModel member var to ChatCraftProvider class to store the defaultModel, since for the custom providers users will add themselves we have no way of hardcoding the defaultModel. So we will query the first model from list of models and store it in this ChatCraftProvider member var.

Follow ups issues:

https://github.com/tarasglek/chatcraft.org/issues/524 https://github.com/tarasglek/chatcraft.org/issues/542 https://github.com/tarasglek/chatcraft.org/issues/548 https://github.com/tarasglek/chatcraft.org/issues/550 https://github.com/tarasglek/chatcraft.org/issues/551 https://github.com/tarasglek/chatcraft.org/issues/552 https://github.com/tarasglek/chatcraft.org/issues/553

How to test:

Currently I don't have any custom provider urls/keys to use for testing. If anyone has any please help me to test if those providers are open ai compatible.

You may add new providers using the providers we currently support:

https://api.openai.com/v1 (you can create multiple with diff names if you want to manage more than one api key)
https://openrouter.ai/api/v1 (you can create multiple with diff names if you want to manage more than one api key)
https://free-chatcraft-ai.deno.dev/api/v1 (not allowing them to create one since there is no api key so you would not need multiples)

View on GitHub

Before this got merged I did a lot of small refactoring and bug fixes, such as consolidating components, making sure the new provider row disappears after we close the modal, etc.

And as with any major feature, we can't forget about the follow up issues and hotfixes.

Bug fixes

Some bug fixes I completed this week:

Hotfix for user settings provider table #556

kliu57 posted on Apr 03, 2024

This bug was missed when we landed #530

Closes #555 Details of the bug can be found in the issue linked above

View on GitHub

Logo bg color fix for all models that don't have logo #562

kliu57 posted on Apr 04, 2024

Bug:

Currently some logos in OpenRouter and custom providers appear as white, since they have a default white logo on a white bg.

Fix:

Set the default Avatar to have green bg color.

code change:

View on GitHub

Follow up issues

There are many follow up issues, but here are the two major ones I am currently focusing on.

Make Options dialog more dense/compact #561

tarasglek posted on Apr 04, 2024

Lets make it look something like:

Models (put the temperature setting and add free provider button here)
- free
- openai
- openrouter
WebHandlers
UI Customization (all the other settings?)

ui would look like vs code:

View on GitHub

This issue aims at revamping the UI of the User Settings modal, to make sure it encompasses all our settings in a user-friendly way.

With taras' guidance, I have created some mock up UI designs, and I am waiting to get feedback on them. I also spoke with mingming about how to create this two-column modal, as this task may need to be divided up among more people.

Update Settings model option to be per provider #524

humphd posted on Mar 23, 2024

The Settings modal used to be the place where you changed the model you were using:

However, this is no longer true, since we have the "Ask" button and menu.

Let's either remove it or convert the "GPT Model" part of the form to be "Default Model for Provider" and make it so that you can specify a default model when you switch between provider.

View on GitHub

The second issue I have my focus on is this issue which would allow the user to store a "current model" for each provider. Currently whenever the user switches between providers (let's say they switch from OpenAI to OpenRouter), the model will stay the same but when the switched provider doesn't have the model it will just change to some default model. This is not very convenient for the user because most likely this default model is not what they want to use.

I plan to change the way we store the current modal information. We currently store one "current modal" in the localStorage as settings.model. I plan to add a new member variable to the ChatCraftProvider class so that we store the model in the settings.currentProvider object. This way when the user switches between providers, they can have back the model they were last using with that provider! For those users who have more than one providers, I would assume that they have a preferred go-to model for each of their providers.

Code Reviews

I did many code reviews this week. Here are the PRs I reviewed which were later merged!

Replace "Cancel Adding Provider" SVG with an IconButton #554

Amnish04 posted on Apr 03, 2024

As discussed in https://github.com/tarasglek/chatcraft.org/pull/530#discussion_r1545481317, we should not use Svg's directly as an alternative of a button since it is semantically incorrect.

https://www.reddit.com/r/webdev/comments/r72zq6/comment/hmwvobm/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

I tried preserving the look of the button, and was successful for the most part.

Normal: