DEV Community: Nikola Cosic

Use Bard API (Palm API) without an API key

Nikola Cosic — Wed, 13 Sep 2023 10:15:02 +0000

Google's Bard is eagerly expected to join the LLM API race so we can all use it in our AI related products. Problem is, currently the access to Palm API Beta is severely limited. Early US based applicants got the key. For non-US applicants and now also for anybody from the US, it's next to impossible to get the key.

Many of us don't have the opportunity to explore how Bard and Palm API functions. Even if you have Bard web access - API access and regular web based use differ so you really need to query the Palm API to experience what it provides. Also, web based use isn't available for everyone. This is where APIScout.AI comes to play.

APIScout.AI compares ChatGPT API and Palm API output. It is displaying response to your prompts side by side, querying simultaneously the two biggest LLM APIs out there. It allows full API request configuration, and is free, can be used without registration. This way you can check out Bard without having a key or web access.

There's a limit to the number of prompts you can enter though. But it's enough to understand what Bard API offers so you can plan ahead to use it when it becomes commercially available. When you run out of prompts you can enter, you can continue comparing ChatGPT and Bard if you have your own API keys to enter into the APIScout.AI's GUI.

Please let us know in the comments if you find any other way to access Palm API at this time, or if Beta registrations continue.

ChatGPT API vs Google's Palm API

Nikola Cosic — Sun, 10 Sep 2023 05:32:18 +0000

The two major LLMs expose their functionality via API to be used in 3rd party products. ChatGPT API has been around for a while, but Google's Palm API, which taps into the Bard functionality, is just emerging in beta. This analysis aims to provide insights into the strengths and weaknesses of both, based on extensive research and hands-on experience.

Palm API (Bard):

Status: Currently in beta testing and free.
Bandwidth: Provides a much higher bandwidth at no cost during the beta phase. If you have a lot of prompts to run now, this may be the decisive factor in choosing your LLM API.
Speed: Appears to have greater computing resources and generally offers faster responses.
Accuracy: Tends to produce lower-quality responses in certain areas compared to ChatGPT. In other areas it seems to be better. So you really have to test and decide for yourself. I've created a comparative testing tool for the two LLM APIs, https://apiscout.ai - let me know what you think.
Geographical Restrictions: Primarily accessible in the US; other regions face limitations.

ChatGPT API:

Cost: Can become costly if you have a lot of prompts to run.
Speed: Has faced some performance issues and outages, also it takes more time to produce an output than Google's solution.
Accuracy: A mixed bag, but it does seem to deliver more accurate responses in a number of areas. You will have to compare the two for your particular use.
Reliability: Currently more reliable for long-term projects as it's in live commercial use.
Geographical Reach: Globally accessible without heavy restrictions.

Key Insights:

Performance and Cost: Palm API's lack of cost and high bandwidth can be enticing for projects requiring a significant number of prompts. ChatGPT's pricing structure can make it prohibitive for large-scale applications.
Reliability: Palm API's beta status implies uncertain longevity, potential pricing changes, and potential performance challenges in the future. In contrast, ChatGPT currently offers a more reliable solution for long-term projects.
Documentation Quality: Palm API's documentation, at present, seems inadequate, with some request parameters being particularly challenging to comprehend. This could signal broader issues with the project's maturity.
Geographical Considerations: Palm API has significant geographical restrictions. A US-based key cannot be used from a server in Canada, and non-US regions might face challenges in obtaining beta testing keys.
API vs. Web Access Discrepancies: It's important to note that there can be variations in the responses received from direct API access as opposed to web user interface access, even with identical parameters. As a solution, I've introduced a tool, https://apiscout.ai, which allows users to view side-by-side responses from both APIs while customizing parameters. The tool is especially handy for non-developers designing prompts.

Conclusion:

While both APIs offer unique strengths, developers must weigh the pros and cons based on the specific requirements of their projects. The rapid development in the AI industry further underscores the importance of staying updated on these platforms.

Should you have any feedback, updates, or insights on the current state of LLM APIs, your input would be invaluable to the community.

ChatGPT vs Palm API (Bard) for developers

Nikola Cosic — Thu, 07 Sep 2023 17:35:03 +0000

In these early phases of LLM competition it is becoming increasingly hard to decide what is the right solution for a particular AI-related project - ChatGPT API or Palm API (Bard). I've been spending a lot of time recently designing prompts for various uses, analyzing both. Let me share some of my findings on the subject.

Palm API is Google's API which essentially accesses Bard functionality. It is currently free, and in beta testing mode. This is all the more reason to use it in your projects provided you don't expect long term reliability.

Palm API has much higher bandwidth than ChatGPT, at no charge. This allows applications that require lots of prompts. ChatGPT becomes pretty expensive when you get into millions of prompts which makes some applications impossible and/or impractical.

Another point is speed - these days ChatGPT is struggling to deliver all what's required of it. According to my experience, it can become very slow, it has outages, and it has a lower rate limit than Bard. Palm API seems to have much more computing resources, providing responses faster. This is another point where Bard seems to be a better solution for applications with lots of prompts.

On the other hand, Bard clearly seems to deliver worse answers on many subjects - namely taxonomy has a lot more garbage than ChatGPT, but I'm sure there are many other areas. It's up to you to decide which is better for your particular us.

Another thing to consider is there's a notable difference in responses between API access and web user access. I've been getting different response to the same prompts on web and API, provided the same parameters. This means you've got to have a tool to design prompts that will communicate directly with APIs instead of using the GUI of LLMs. This tool should be able to provide access to non-developers as they will likely be interested to communicate directly with APIs.

I've developed a free tool to do so. It is able to post responses from both APIs, side by side, while allowing you to setup the parameters in detail. It's called APIScout.AI - you can check it out here: https://apiscout.ai . Let me know what you think. Everything is free except for bulk processing of multiple prompts, as that may consume server resources.

Long term, if you need reliability, ChatGPT API is currently the way to go for all your new applications. Palm API is in beta testing, so you don't really have any long term forecast on whether it will available, in what form, and at what pricing. You don't know if it start to have hiccups due to overloads like ChatGPT does now - maybe worse. And you don't have any information on the future commercial rate limits.

The state of Palm API's documentation tells us a lot about the state of the whole project - it seems incomplete. It's very hard to figure out what to do, there are request parameters which are next to impossible to understand. I've included many of those on my APIScout.AI project so you can experiment yourself with parameters without any coding.

Last but not least - consider ChatGPT is available globally, while Palm API is still under heavy geo restrictions. You can't, for example, take a key from a US based Google account, and work with it from a server in Canada. Server has to be in the US too. I've applied for Palm API beta testing key from Europe a month ago and haven't got the key while applications from the US resulted in providing the key the following day. But now the application process seems to be closed for the time being - hope you've got your key in time. Please let me know in the comments if you have any news on the availability of beta keys, that should be interesting for all the readers.

My project, APIScout.AI will work for you only if you provide a US based key as the server it's running on is in USA. Let me know if you have any interest to enable other countries.

Hope you can use this information to make better AI based applications. Please let me know what you think or if you find out about any updates to the current condition of the LLM APIs.