Tried Bard a couple of times recently and was not very impressed tbh. Seemed to forget the context of the conversation very often. Like I had to repeat again and again to not show external links with previews and not give explanations to every little thing.
While I have found that ChatGPT pretty regularly outperforms Bard, I still run my questions through both, because sometimes Bard has a different angle that I like better.
I kinda liked the name “Bard”. It fit. Gemini is going to make me think of ancient rockets and ostentatious claims.
I've been using Gemini in Bard since the launch, with respect to coding it is outperforming GPT4 in my opinion. There is some convergence in the answers,but Bard is outputting really good code now.
In my experience, Bard is not comparable to GPT-3.5 in terms of instruction following and it sometimes gets lost in complex situations and then the response quality drops significantly. While GPT-3.5 is a much better feel, if that is a word for evaluating LLMs. And Bard is just annoying if it can't complete a task.
Also hallucinations are wild in Gemini pro compared to GPT-3.5.
In the 20 minutes of experimentation, I'm really impressed with the quality of Bard Advanced (Gemini Ultra). The results are as good as GPT-4, and in some cases is better. So far:
pros:
- better at translation (tried Chinese and Japanese idioms to English)
- better at incorporating search results in its answer vs gpt-4 bing
cons:
- slightly worse at coding
- censorship is more annoying (have to ask multiple times about medical topics)
- Worse at logic (e.g. it contradicts itself in a single sentence, and is unable to figure it out)
- Hallucinates a lot when asked to describe an image
from what I remember bard should be able to browse the internet and write code internally to better answer queries. I feel like these abilities are just improved with Gemini as a better language model.
I think there’s bias in the types of prompts they’re getting. In my personal experience, Bard is useful for creative use cases but not good with reasoning or facts.
While the other response was unflattering but true, a better answer would be that Bard ___ is trained to be a general purpose chat bot, specific for the Bard user experience. While Gemini ___ API is a developer focused API for LLM use. Similar to how ChatGPT performs different to the OpenAI APIs or Copilot in Bing, or whatever. Basically, they’re fine tuned and prompted to respond differently.
That's interesting, because I have had exactly the opposite experience testing GPT vs Bard with coding questions. Bard/Gemini far outperformed GPT on coding, especially with newer languages or libraries. Whereas GPT was better with more general questions.
I reran a few of the queries that Bard had failed on in Gemini/Ultra and didn't see any improvement. Made the same, or new, logical errors, hallucinated facts, failed to recognize things I was describing etc. It did do better on recognizing an image I uploaded but it went from accurate to nonsensical in the same response.
I've tested bard/gemini extensively on tasks that I routinely get very helpful results from GPT-4 with, and bard consistently, even dramatically underperforms.
It pains me to say this but it appears that bard/gemini is extraordinarily overhyped. Oddly it has seemed to get even worse at straightforward coding tasks that GPT-4 manages to grok and complete effortlessly.
The other day I asked bard to do some of these things and it responded with a long checklist of additional spec/reqiurement information it needed from me, when I had already concisely and clearly expressed the problem and addressed most of the items in my initial request.
It was hard to say if it was behaving more like a clerk in a bureaucratic system or an employee that was on strike.
At first I thought the underperformance of bard/gemini was due to Google trying to shoehorn search data into the workflow in some kind of effort to keep search relevant (much like the crippling MS did to GPT-4 in it's bingified version) but now I have doubts that Google is capable of competing with OpenAI.
The latest Bard is good, but at this time (Dec 2023), Bard is still not better than paid ChatGPT 4 in my experience.
I use both daily (I use ChatGPT on my phone and at home, Bard at work because work doesn't allow ChatGPT). ChatGPT 4 still provides the highest quality answers to the range of questions I'm interested in. Bard's getting better with Gemini but it's still not good.
Claude is great at summarizing PDFs and has one of the longest context windows, but the quality of answers it has to general questions is still lacking. Outside of summarization, I have little confidence in Claude's Q&A abilities.
I've also been using Perplexity.ai and I would say it hits a sweet spot and is impressive. It's essentially a RAG-based search engine that's fast and snappy and it works much better than Bing or ChatGPT Bing. Because it's primarily RAG driven, the content is primarily external and you get links to original sources so you can verify the answer. This means the level of hallucination is lower than foundational models. It's become a practical tool for me. It's CoPilot feature, which asks clarifying questions about your query, has been helpful for finding new products. For instance, I was looking for luggage packing cubes that convert to hanging cubes. It's a known product class but I didn't know the lingo, but over a few clarifying iterations, it was able to help me find the right product.
Tried Bard a couple of times recently and was not very impressed tbh. Seemed to forget the context of the conversation very often. Like I had to repeat again and again to not show external links with previews and not give explanations to every little thing.
reply