Google's Duet AI, their AI-powered tool suite for Workspace, has been available for a few months now and I thought it was time to give it a try at our company. There is a trial available, after which the cost is $30/month per user.
Note that there is also Duet AI for Developers, which is a separate subscription through Google Cloud instead of Workspace and, as the name implies, is software development-oriented.
Back to Duet AI for Workspace, it still feels a bit "beta" as new features are added gradually - e.g. not all Duet AI features in Google sheets have yet rolled out to our organization. However, the Create Image with Duet AI function in Google Slides is available, and as I use the app for presentations/talks, I was interested. I thought I'd put it to the test to see if it can generate useful images and also use a freely available option for a direct comparison: the Image Creator from Microsoft Designer, aka Bing Image Creator, which is based on OpenAI's DALL-E 3.
The Comparison Tests
I revisited some of my previous slide decks used in various talks, aiming to assess whether using AI image generators could have saved time compared to searching in stock libraries, editing images, or drawing from scratch. Additionally, I attempted to recreate a couple of my app icons and header images for my dev.to articles.
In total, 12 comparison image rounds were conducted. For each round, both Duet AI and Image Creator received the same prompt, and variations were explored based on the results. Variations that yielded a significant impact were included in the results. I was a novice user of both tools, it is possible that someone with more experience may have gotten more out of them.
Each round was rated from 0 to 10 based on the proximity of the result to expectations, suitability for the intended purpose and adherence to instructions. The ratings are subjective, but the images are included so you can draw your own conclusions.
The Camels
For the last few years I've been giving talks at the Perl and Raku conference. My presentations always feature some sort of camel, as that's the most recognised Perl symbol, so I tried to recreate a couple of camel images I have used in the past.
- Running Camel
For a Software Performance Optimization talk, an image of a running camel was used (an inexpensive stock photo):
The simplest prompt I could think of:
a happy camel running
And the results (remember, both solutions always try to offer 4 options):
Duet AI needs to learn how many humps camels have. Bing, despite what some 3rd-party sources claim, does not default to photography style, it had to be specified. To make the camel easy to extract, complex background should also be avoided:
a photo-realistic happy camel running, single colour background
That's more like it, Duet AI is surprisingly close to what I was going for! I guess the upper-right from Bing is quite a nice image for a fun presentation, but it's not actually what I'd call photo-realistic and the background is more complex than specified.
| Duet AI | 10/10 |
| Image Creator | 6/10 |
- Camel with Glasses
A camel with glasses was used for the Fast Perceptual Image Hashing talk, created from two stock images:
It's a bit rudimentary as I am neither a designer, nor did I want to spend much time on it. Perhaps an AI generator could have managed this with an appropriate prompt:
A smiling camel looking at us through big blue glasses, single colour background, photo-realistic
Like previously, Bing gets the idea, but can't generate a photo-realistic result. Maybe because of the cartoon-y quality that the glasses add to the scene anyway, it is probably overall more appropriate than the running camel example. The backgrounds are simpler too on 3 out of 4 results, so that's an improvement. However, Google is pretty much bang-on with what was expected - using it instead of manually editing would have saved time and given a better result too.
| Duet AI | 10/10 |
| Image Creator | 8/10 |
Astronomy
I used to do talks for my local astronomy club, so I picked some of the slides I used for a "Choosing Binoculars" talk.
- Photos of Astro Objects
This slide with examples of objects as you'd view them with binoculars was going to be a long-shot:
Without the "how would they look through binoculars" element, I tried to give a list of objects to see if I could get the sort of "astrophotos on canvas" style above:
A compilation of photos one each of the astronomical objects: the moon, pleiades, orion nebula, andromeda galaxy, the Double Cluster, comet Lovejoy, arranged randomly on a canvas with slight overlaps
It's worse than expected, mainly because neither AI seems to be able to count. Duet AI produces some realistic images, but also adds random objects. Bing is worse in that it makes up some really bizarre looking galaxies and comets, nothing is realistic.
One more attempt was made, explicitly stating the number of objects and trying to force photo-realism on Bing.
A compilation of 6 photo-realistic photos, one each of the astronomical objects: the moon, pleiades, orion nebula, andromeda galaxy, the Double Cluster, comet Lovejoy, arranged randomly on a bigger canvas, some overlap is allowed
Duet AI seems to be having a hard time, only giving 3 results, and none of them feature exactly 6 objects - never mind the correct ones. I guess if you pick and choose from the 3 variations it does manage to produce most of the individual objects listed. Bing managed to give a count of 6 in two attempts, but completely lost the plot otherwise. I did not think it was worth pursuing this more.
| Duet AI | 2/10 |
| Image Creator | 1/10 |
- Stack of Binoculars
This photo is from the same presentation:
4 binoculars stacked on top of each other from largest (bottom) to smallest (top), with their lens pointed towards our viewpoint, photo-realistic
You will note that Duet AI still does not know how to count, but was close to what was requested otherwise. Bing though seems to only have a vague idea of what binoculars are.
An alternative prompt mentioning "pairs of binoculars" was attempted in case the hint helped Bing get what a binocular is. An "of various types" hint was also added in an effort to get some variety of types from Duet AI, to better match the target image.
4 pairs of binoculars of various types, stacked on top of each other from largest (bottom) to smallest (top) with their lens pointing to viewer, photo-realistic
Duet AI managed to get a single result with 4 binoculars. They aren't really "various types", but still usable. Bing managed to veer even further off the prompt and the whole idea of binoculars.
| Duet AI | 9/10 |
| Image Creator | 1/10 |
- Lawn Chair Binocular Mount
Again on the same slide deck, there was an image of one of the various DIY "lawn chair binocular mounts" that can be impressive and sort of amusing:
They are sometimes called "bino-chairs" and require some design creativity and ingenuity, so it was a different type of test for the AI engines.
man on lawn chair using hands-free binocular mount
Duet AI refused to even try. Variations of the language did not help, it currently cannot make a man-chair-binos scene (it would only manage some sort of result if I removed the man). Note that I had originally rated Bing at 0/10 for the spectacular failures with astro objects and binoculars above, but at this point Duet AI redefined zero. Bing gave it a good try here, didn't get the hands-free element of the prompt, but that was probably asking for too much.
| Duet AI | 0/10 |
| Image Creator | 6/10 |
Technical Drawings
For technical talks, explanatory illustrations are often required. E.g. the following image from the Perceptual Image Hashing talk shows which cells of a 6x6 matrix are used for a specific hash:
draw a symmetric 6x6 square matrix with white lines, make the top-left cell black, also the cells that are below the bottom-left to top-right diagonal also black, and the rest blue, 2d art style
Google Image search knows very well what a "6x6 square matrix" is (and so does OpenAI's ChatGPT), but both AI image generators are nowhere close to what the prompt described.
Variations in the wording to do with styles, replacing 6x6 with "6 by 6" etc did not help at all. Going to the very basics, replacing even the word matrix with "grid" and not asking for anything else, sort of helped.
plain 6x6 square grid, solid colour background, vector drawing
Duet AI seems to give something that's a cropped 6x6 grid amid useless variants and Bing does some wild grids none of which is even the right size.
Many other variations were tried like "empty square grid consisting of 6 rows and 6 columns, solid colour background, vector drawing", with Bing not improving at all (still avoiding the number 6 in its drawing attempts) and even Duet AI was producing only 4x4 and 5x5 grids, indicating that the previous "almost" 6x6 was probably just luck. At this point it's clear there is no possibility of any useful geometric illustration generation.
| Duet AI | 2/10 |
| Image Creator | 1/10 |
Logo/Splash Graphic Design
Moving on to a couple of logos / splash screens I designed for the iOS apps I develop as a hobby. It's not typical slide deck graphics, but it could still be relevant if you are putting together a new project / product and creating a presentation for it.
- Polar Scope Align Icon
Polar Scope Align is an iOS app for amateur astronomers & astrophotographers. It's quite popular in its niche and it is often praised for a well-designed UI with a focus on functionality. The image assets themselves, such as icons, are rather simple as I am not a designer. Here is the older (left) and newer (right) icon of the app:
Hopefully, with the right prompt, something that resembles the older & simpler icon on the left could be within the abilities of the AI engines.
red crosshairs with circle around them, centered on the middle of the 7 stars of the Little Dipper, the Little Dipper should barely fit the circle, clip-art style
Not a great result. Google is a bit closer to what was anticipated, Bing is a bit more visually interesting but deviates further from the prompt. They both got the stars very wrong and no prompt variation (e.g. using "Ursa Major" etc) could get them to draw anything resembling the Little Dipper.
| Duet AI | 3/10 |
| Image Creator | 2/10 |
- Xasteria Icon
Next up, the icon / splash screen for the app Xasteria Astronomy Weather:
solid dark blue sky, having several yellow 4-pointed stars of various sizes, each designed using hyperbolic curves, but all with their points at top/bottom/left right orientation, a third of the sky covered by a dark grey mountainous range silhouette and a big white X that is the same shape as the 4-pointed stars but rotated 45 degrees and takes up 80% of the width of the scene, clip-art style
Google did better here, producing images that get almost everything specified, except for the shape of the X. Bing generated some interesting icons that perhaps could be used for Xasteria, but they either deviate wildly, or, in the case of the top right, which is the one following the prompt closer, cannot get shape or colour of the X. In fact, there seemed to be no way to convince Bing to make the central star white (or the right shape), despite numerous tries at writing it differently.
I even tried to be clever and use ChatGPT (which is also made by OpenAI) to produce a better prompt. So, after briefly lecturing me about how Dall·E would benefit from the prompt being more detailed and explicit, ChatGPT helpfully generated this one:
create an image of a serene landscape with a solid dark blue sky. Populate the sky with several yellow 4-pointed stars of various sizes, each designed using hyperbolic curves. Ensure that all stars have their points at top/bottom/left/right orientations. Dedicate one-third of the sky to a dark grey mountainous range silhouette. Additionally, include a prominent white X shape in the scene. The X should be the same shape as the 4-pointed stars but rotated by 45 degrees, taking up 80% of the width of the scene. The X should maintain the hyperbolic curve design.
Despite ChatGPT's confidence, the result is actually worse. It also completely broke Duet AI (which is not a hard thing to do currently).
The last attempt involved going back to the prompt that does not break Duet AI, but tweaking the description of the "X" to make it more explicit.
solid dark blue sky, having several yellow 4-pointed stars of various sizes, each designed using hyperbolic curves, but all with their points at top/bottom/left right orientation, a third of the sky covered by a dark grey mountainous range silhouette, at the foreground a big white X that is also designed using hyperbolic curves and takes up 80% of the width of the scene, clip-art style
Duet AI did even better with this prompt, it's very close to what was expected. The central X finally has both the colour and the shape that was specified.
| Duet AI | 9/10 |
| Image Creator | 4/10 |
Blog Header Image
After all these quite specific images, I thought I'd try more creative generation and see what the AI engines can come up with when given titles of articles - dev.to articles I've posted to be exact.
- Google Cloud & AMD EPYC
First, is the performance review of the latest AMD EPYC powered GCP instances. Using the full title confuses Duet AI, so the prompt was simplified. Bing gave similar style results in both cases.
An image that can serve as a title for a Google Cloud and AMD EPYC presentation
Duet AI gave a single, and not very inspired result. Image Creator tried to impress, but spoiled two of the results with some terrible text artifacts (top right is something like Ccgoke Cloud, bottom right Googc|oud). Judging these results is very subjective, but I think at least one of Bing's images could be considered suitable.
| Duet AI | 2/10 |
| Image Creator | 6/10 |
- Compute Cloud Provider Comparison
Next was the Compute Cloud Provider performance and price comparison.
an image that can be used as a title in a compute cloud provider price & performance comparison
I have to admit I had to rephrase the prompt a couple of times before Duet AI gave me something that was not JUST a cloud. It's still not good. Bing went with a tech + cloud concept that may be considered suitable. Again the ratings will be very subjective.
| Duet AI | 2/10 |
| Image Creator | 7/10 |
- This Article
For this article a generic prompt was attempted:
drawing competition Google Duet AI vs Microsoft Bing Image Creator
I was not crazy about either, even though the Duet AI was clearly a lesser attempt, a very generic image.
So I gave instructions instead, for a painter writing a different phrase for each engine on a canvas:
painter writing "Google Duet AI"/"Microsoft Image Creator" on a canvas.
Duet AI missed the word "Duet" and produced a single image, which was used for the header. Bizarrely, since generating this image, it refuses to generate any image of a painter writing anything on canvas. Any way I phrase it, I get the dreaded "try something else" and I've tried several times at a span of more than a week - so it is not a temporary glitch. The trigger seems to be including a painter/man/etc: prompts that ask for "paintbrush painting something" work fine (and include a human hand), but if a human is explicitly requested the prompt fails.
Bing gave a nice variety, except 3 out of 4 have these subtle misspellings. The only correct one was used for the header image.
| Duet AI | 5/10 |
| Image Creator | 8/10 |
Sloths with Headphones
Finally, I tried something fun for my videoconferencing background. First thing that came to my mind was:
sloths wearing headphones, photo-realistic
Note that in this case "photography style" had to be selected from the Duet AI drop-down list, as the sloths were not realistic enough otherwise. Bing still came out looking better in my opinion, and paid attention to my use of plural. The more sloths the merrier!
| Duet AI | 8/10 |
| Image Creator | 10/10 |
Conclusion
At the start of this comparison I planned 10 tests for the two engines. In the middle of conducting the tests, I added a couple more that I felt I wanted to see their performance on. This is to make it clear that I was not adding tests to tilt the scales towards one or the other based on their performance, so this result was not by design:
| Test | Duet AI | Image Creator |
|---|---|---|
| Running Camel | 10 | 6 |
| Camel Glasses | 10 | 8 |
| Astrophotos | 2 | 1 |
| Binoculars | 9 | 1 |
| Bino Chair | 0 | 6 |
| 6x6 Matrix | 2 | 1 |
| PS Align Icon | 3 | 2 |
| Xasteria Icon | 9 | 4 |
| GCP / EPYC | 2 | 6 |
| Cloud Comparison | 2 | 7 |
| Drawing Phrase | 5 | 8 |
| Sloths | 8 | 10 |
| Total | 62/120 | 60/120 |
Given that a lot of the ratings were subjective, this is essentially a draw. Both of them got around 50%, which means they lived up to my expectations about half the time.
However, they are by no means equal or even similar. Now that I know, I can easily design a test that can make either of them the winner.
Google's Duet AI is indeed better at "factual" results. This makes sense, as it was launched as an office productivity tool. It still gets some things very wrong, but it could help you generate some useful images for presentations like it is supposed to. It's a bit frustrating how it often tells you to "try something else" instead of giving it a go - you can easily "break" it like that currently, but hopefully that will improve as Google continues to develop it. What I would like most to see for the future is the ability to iterate over a design: you should be able to give corrections over the previous result as a prompt, instead of just starting over with a modified prompt.
Microsoft's Image Creator, powered by OpenAI's DALL·E 3 engine, can understand a wider variety of prompts. You'd hope it can "understand" prompts near as well as the same developer's ChatGPT, but it does not come close. It still feels as a more mature product than Duet AI, which is as expected in it's third major release. Apart from interpreting a wider variety of prompts, the results are also much more "imaginative". While Duet AI can put elements together, Image Creator will as likely create new elements, often to your detriment if you expected something realistic. It's definitely a more artistic-oriented engine, so I can see how it has its uses, but can't fully replace Duet AI.
Overall, the ~50% scores reflect my mixed feelings about how good these engines are right now. Technically, Bing Image Creator is the best value, especially if you want to play around, as its free (with a daily limit to "boosts" you can use for fast generation). However, in a professional context, Duet AI's image generation is indeed more appropriate, so Google does offer additional value. At $30/month you'd probably have to find more Duet AI features across the Workspace apps useful enough to justify the cost - especially with the image generation feature still in the development stage.
Top comments (2)
Thanks for the comparison, out of all those image generator apps, I perfer Grok! I am actually making a post on discord message deleter scripts. Hopefully I can make an image generator app directly on discord as I seen a few worth mentioning if I can find them
It's fascinating to see how AI tools like Google's Duet AI and Microsoft's Bing Image Creator are enhancing productivity and creativity in various fields, much like the Solara Executor app does for Roblox enthusiasts. Both AI image generators offer unique advantages depending on the user's needs, whether it's generating realistic images for professional presentations or more imaginative creations for artistic projects.
Similarly, the Solara Executor app provides Roblox users with advanced features for script management and gameplay enhancement, prioritizing user experience and efficiency. Just as you would choose between Duet AI and Bing Image Creator based on your specific requirements and the nature of your project, selecting a tool like Solara Executor depends on your gaming needs and how you wish to enhance your Roblox experience.
This comparison not only highlights the capabilities and limitations of each AI tool but also underscores the importance of choosing the right tool for the right job, ensuring that whether you're enhancing presentations or gaming experiences, you're equipped with the best technology to support your objectives.