Yes all LLMs have problems. But some are better than others, and there are benchmarks used for comparing the models. Also, in terms of PR, I don't remember OpenAI having messed up like this -- having one really bad incident is worse than not giving perfect answers.
The “insert an ethnic minority into every group photo” was the straw that broke the camels back as it was clearly biased.( for anyone who cares I am an ethnic minority).
But I do remember, and I mean this, my first ever query asking it to draw a picture by a Japanese artist resulted in a “call this number for mental health problems”
I was like wtf. The prompt was benign “water color sunbathing beach sunset” by some random normal Japanese artist. Sure potentially nsfw but the reply was completely hamfisted, insulting and really crazy.
I probably wouldn’t have cared but it was literally my first prompt.
Before anyone says “well it could have been degenerate” you have heard of the Michelangelo David right? The greatest ever sculpture? I haven’t tried but I’m pretty sure there isn’t a hope in hell the llm would reproduce anything like it.
Like you say, I think what's being discovered is the lack of utility in these things. These techbro companies with Gemini, ChatGPT, etc. are just rediscovering that human to human communication is hard and cannot be automated and that human illustrators, artists, animators, writers, etc. have real, actualizable value.
And it's rediscovering that we suck at prompting even humans, much less systems that have no idea what they're doing and just statistically stitch and glue data together.
These LLMs are just exposing that humans are fairly bad at communication and they are encoding that into automated systems.
Thats disturbing even happens when searching from Europe so its a global manipulation of results. At this point I have started to tell everyone i know to avoid google products.
The question is any of the other search engines any better ?
I use DuckDuckGo primarily and hear good things about Kagi on HN, but haven’t used it much myself.
DDG is fine I suppose, although my most frequent use case of looking up (nearly wrote “googling”) code solutions is mostly handled by ChatGPT these days.
I've started just running models locally for the most part. If I want an apple cake recipe written in the speech style of Stalin who is OpenAI to deny me.
I like presearch.com wholly because they are too young/small to apply these sorts of filters, the results feel "raw" in a good way.
I have no affiliation with them and dislike their cryptocurrency based snake oil. But as far as a search engine that hasn't been tinkered with and is free, not bad, and I recommend them on that basis alone.
Though this raises a question for me. If searching "American inventors" threw down pages and pages of white people, would the Internet raise a stink about it?
If it's anything like what happened to the female human computers that made the ENIAC and UNIVAC work, my bet is no. [0]
Sorry, but who cares? He owns the company, but he isn't contributing or leading. He's just hanging around and poking a bit, making uniformed obvious comments.
„Over the last week, in communication with a flood of Googlers eager to speak on the issues facing their company — from management on almost every major product, to engineering, sales, trust and safety, publicity, and marketing — employees painted a far bleaker portrait of the company than is often reported: Google is a runaway, cash-printing search monopoly with no vision, no leadership, and, due to its incredibly siloed culture, no real sense of what is going on from team to team. The only thing connecting employees is a powerful, sprawling HR bureaucracy that, yes, is totally obsessed with left-wing political dogma. But the company’s zealots are only capable of thriving because no other fount of power asserts, or even attempts to assert, any kind of meaningful influence. The phrase “culture of fear” was used by almost everyone I spoke with, and not only to explain the dearth of resistance to the company’s craziest DEI excesses, but to explain the dearth of innovation from what might be the highest concentration of talented technologists in the world.“
I haven’t heard a single Google employee say they wanted it to be like that. Do you know who specifically did? It sounds like a conspiracy theory more than anything else.
What would strain credulity is if Google released this, after 10s of thousands of highly-paid dev and PM hours, plus thousands of full-time tester hours, without having identified the problems the internet found within 60 seconds.
Evidently (i.e. evidenced by what we got) Google’s testers tested only what they wanted to test.
It's been two weeks since Google turned off the generation of images with people. Still no word on when it's coming back. This tells me it wasn't just system prompts and prompt injection, it was done via RLHF and it's going to take awhile to un-gimp the model.
I just tried "generate an image of a man camping in the woods" and it came back with "Sure, here is an image of a man camping in the woods" followed by a 10-15 second pause followed by "We are working to improve Gemini’s ability to generate images of people. We expect this feature to return soon and will notify you in release updates when it does." This would seem to indicate they're letting it generate an image and then running some sort of human detector on the output instead of filtering the prompt on the front end.
Given the degree of the controversy, I think even if it was just simple fixes in system prompts, they would want to take a few weeks of testing and red teaming before re-releasing.
>they would want to take a few weeks of testing and red teaming before re-releasing.
One wonders why they haven't done this kind of testing before going public with such an important feature of such an important new flagship product. It's not like the prompts were some super niche stuff that required a lot of imagination to conjure.
And they could have blocked the people image generation feature from the start if they saw it malfunctioning like that in tests and didn't have enough time to fix it on such a short notice before launch.
I find it hard to believe nobody internal at Google bothered to play with Gemini to generate people for shits and giggles and saw the hilarious wrong outputs and raise the alarm.
Or they did ran tests, saw the issues but they decided internally that there's nothing wrong with those generated outputs, which is even worse.
Either way, whichever reason it was, why would you use or even want to pay for a Google product if this is the level of QC and broken functionality you can expect from Google products?
To further my point. I use ChatGPT to translate stuff from English to German a lot and now I used it to write a legal complaint in German and I tried getting Gemini to translate it as well to compare it to ChatGPT, and instead of doing that, it just said it won't do it because it can't offer legal advice. FFS, I didn't ask you for legal advice, I already did my research on that topic, I just asked you to translate verbatim from English to German.
Gemini's "safety" guardrails make it beyond useless at this point that I wouldn't even use it even for free. How does Google plant to make money from it? Or is it gonna be another side-project they'll cancel in 2-5 years.
> I find it hard to believe nobody internal at Google bothered to play with Gemini to generate people for shits and giggles and saw the hilarious wrong outputs and raise the alarm.
The alarm was drowned out by the "Shit OpenAI is miles ahead in this game, we need to launch ASAP to try to close the gap" alarm.
Sure, but the moment you as a big company go live with something you have to automatically assume people will fuck with it. Surely someone out of Google's 130k staff who wasn't overworked with getting Gemini ready for launch had the chance to fuck with it and see where it fall short.
Per The Verge, it looks like the image generation model isn't even Gemini, but some older image generation model they found on a shelf in their basement and just made Gemini make a call to it for any image requests.
So here's my hypothesis as to how it works now - and yes I believe this "problematic bias" is baked into the core model - I think they have a "dumb" simple "is this a person" detector stitched onto the end of the pipeline. If it detects a person, drop the generated images.
This suggests a jailbreak, if you can fool the "dumb" person detector (likely shallow vs deep learning) with your deep learning generator, the images pass through. I had some success last week with "upside down" or "standing on their head" though it may have been patched. You could consider occlusion "wearing a mask" etc. Basically find phrasing that would fool a naive person/face detector in the corresponding image.
As for the model, I truly believe it to be paradoxically pro-white racist. Imagine any stereotype for any race. You say "generate <stereotype> of a <historically white character/role>" and the model swaps out the white race for the stereotyped race, seemingly embracing racist stereotypes. On the other hand, for white stereotypes, they are much harder to produce since the model is hesitant to render white folks.
Gemini is just a representation of a much larger issue at Google and other tech companies. These places are run by people who think they know better then you and will try to condition you towards their political beliefs one way or the other. The only thing they learnt here is that they have to be more discreet about it. The same companies have long ago already shown they are happy to turn all their platforms into a skinner box and A/B test the shit out of different features until they have maximised their objective functions at your expense.
Idk if it's really conditioning you to their political beliefs with Gemini Images as much as "trying to follow all the modern takes on being non discriminatory AND advertiser AND good PR friendly in every way possible blindly".
LLMs are like employees of the company except if the employee says/does something stupid that causes bad PR, you fire them. If the LLM causes something stupid, you are stuck with it and your company appears inept at it.
I dunno, Bud Lite continues to slide into the abyss because the company is more afraid of going against the woke tide and apologizing- it would rather effectively lose in the market than dare to appear insufficiently enlightened.
Anheuser-Busch (Bud Lite) is actively course-correcting. For instance they hired Shane Gillis as their spokesperson/mascot/brand ambassador or something.
Think of that what you will, but it does signal they see the problem and understand it's costing them money until they fix it.
> Which advertisers find the existence of white people undesirable?
It's so well understood that a white guy can't be the protagonist or audience surrogate in an ad that even mainstream SNL and Family Guy make throwaway jokes about it.
Is there a white judge or doctor in an ad after the year 2000? Or even a non-incompetent white dad? Nobody ever got fired for making the protagonist not-a-white-man.
Can you imagine a Canadian government PSA poster at a bus stop where the doctor on the poster is a white man?
Largely, I don't mind this. Up until the 70s/80s, the reverse was true. Now the pendulum has swung too far in the other direction. Eventually it'll balance somewhere towards the center, where people of any color, gender or sexual orientation are allowed to be the bumbling fool in an ad.
I'm sorry but I just don't buy that. If you say show me a picture of person from background X and it refuses to oblige or gives you a misrepresentation , isn't that the definition of discrimination? Is denying the existence of some groups of people somehow friendly to advertisers? Who in the world is asking to show uniform distribution of some racial/gender groups in every image?
> If nothing else, bear in mind that an image generator that has its thumb so heavily on the scale is less useful for users of all races. (A Black kid who wants an image of a typical Scandinavian Viking for a history paper is not helped here.) ... But… what racism does it actually fight? Which Black person’s life is improved by pretending that there were Black Vikings? And this points to far broader and more important questions. We live in a world where fighting racism has gone from fighting for an economy where all Black families can put food on the table to white people acknowledging the land rights of dead Native Americans before they give conference panels about how to maximize synergy in corporate workflow. In a world of affinity groups, diversity pledges, and an obsession with language that tests the boundaries of the possible, we have to ask ourselves hard questions about what any of it actually accomplishes. Who is all of this shit for?
No, it's because they are lazy, not because of political beliefs. The product is just a cash grab.
Put it this way: the mindset of a company like google in their approach to moderation is to automate as much of it as possible. They hate using human labor to do manual daily tasks.
So they don't want to test and rate the quality output of Gemini over time.
This seems believable to me. Same way they fought spam in the search results. They didn't care about false positives, just whatever algorithm made for less spam, regardless of the end quality. Overweight big companies in the results and toss out good stuff? Sure. It's lazy, but the pure spam was mostly tamped down.
Clearly racial biases are embedded in the model. That wasnt simply the fastest way to build the model. They may have been too fast in releasing, but that’s not the actual cause.
> No, it's because they are lazy, not because of political beliefs.
You don't get a system that will happily output text summaries of the achievements of any ethnic group except Caucasians without political beliefs being a factor.
Either allow the query for all ethnicities, or none. Don't give a straight answer unless the query is for Caucasians, whereupon you lecture the user on how wrong they are for even asking that question.
It's probably that too, but if you look a little deeper you will see they put their thumb on the scale in all of their products. Google News media bias representation, manually curated search suggestions, even Google images has very strange results if you look up anything where race or intersectionalism can be injected.
Close... it's because they're a monopoly. They're not really lazy, plenty of people at Google work hard, but monopolies over time lose their ability to innovate or to compete on quality.
They are only able to focus and organize around one thing: protecting the cash cow. So with Google that's the search advertising business and as a calcifying monopoly they will struggle to do anything else. They know that AI's potentially a threat but in an abstract, marketing-mutating sort of way their ad sales juggernaut can't understand. So they will throw a lot of money at AI, execs will make announcements stuffed with words like Bold and Transformation, and then they'll get sub-par results. This time around they ended up chasing random internal DEI goals or something instead of building a good product, and who knows what flub it will be next time.
The direct comparison is Microsoft's inability to win at mobile, no matter how much cash they burned. Similarly mobile was kind of this existential threat at the perimeter of the cash cow, which was enterprise Windows. So they threw gobs of money at it, but because they had become dysfunctional outside of directly preserving the cash cow, they couldn't make stuff stick. Your Pocket PC (literally they called it a PC!) is a clunker but hey it can connect to an Exchange server! Now let's buy Nokia and run it into the ground!
>They are only able to focus and organize around one thing: protecting the cash cow. So with Google that's the search advertising business and as a calcifying monopoly they will struggle to do anything else.
Google is now in it's own "Steve Balmer Era", where Windows and Office were the major cash cows and the company was throwing money without focus on projects failing left and right not caring since the main products were bringing in all the guacamole.
> These places are run by people who think they know better then you and will try to condition you towards their political beliefs one way or the other. The only thing they learnt here is that they have to be more discreet about it.
Correct. The worst part is Google (and others) know it and will do everything they can to make sure that their reality distortion spells is still conditioning you.
Once the issues are so widespread in the public such as Gemini's image generator the spell breaks and there is little room for them to spin it.
We're now seeing what happens when reality kicks in and all the perks and goodies are taken away when it is wartime - exposing lots of slackers and coasters at Google.
So according to that article, you're not allowed to use the word "ninja" at Google?
Is it because they assume Japanese people would find that offensive or what?
Personally I find the overuse of the word "Ninja" in tech cringe, same as the use of the word "Rockstar". But just because something is cringe, doesn't mean it's offensive.
No, that's cultural appropriation! There's a long list of things you apparently can't do or say at Google that all strike me as completely mundane things that normal humans engage in on a daily basis. It's been years since the James Damore debacle yet it still seems like the asylum is run by the inmates.
"Cultural appropriation" being defined loosely as "doing things while white", since those same people will also let you know that white people have no culture.
It is not even so much that these people at Google and in other positions in society think they know better but that they are convinced that those who do not follow their - rather vaguely defined and quite 'fluent' - narrative are morally inferior people who need to be either re-educated (hence their motto of 'educate yourself') or labelled as such: -ist, -phobe, etc.
This is an important distinction because people who think they know better can often be brought around while those who thing they are better are much harder to convince of the error of their ways because those who do not agree are - according to their own doctrine - morally inferior and as such can be ignored or shouted down. You can gather 40 people who think they know better in a college hall and show them why they are wrong. The quicker ones to take up the facts will help pull the slower ones over the line and by the time the session is over many if not most will be convinced of the error of their ways. Try the same with a group of 40 people who are convinced they are morally superior and the result tends to be very different with the group closing up against the heresy from those deplorables. If someone in the group tries to 'defect' he'd not only have to confront his own doubts but he'd be castigated by his own group.
I still use Google for StackOverflow questions (it crawls the Exchanges super well), but otherwise I use DuckDuckGo for search and Tutanota for email. For AI? My calculator and stuff. :p I don't believe in Dark Age black magic monopolized by whoever can buy the most GPUs, so I don't use LLMs.
> “We definitely messed up on the image generation,” Brin said Saturday. “I think it was mostly due to just not thorough testing. It definitely, for good reasons, upset a lot of people.”
This tells me he doesn’t get it, or at least wont say it publicly. It’s a cultural problem. It clearly was trained or aligned with some massive biases.
My hypothesis: tell a LLM to flip a coin, then tell it the last flip was heads. It will almost certainly tell you the results was tails. It doesn't understand random means random (actually ChatGPT will now execute Python for this).
When you tell an LLM to simply include diverse people, it will do that, emphasis on the "diverse" part.
Gemini "free" - Pro 1.0 I think - responded heads on my first ask there and noted the coin remains unbiased and the next flip is still 50/50. I asked to tally 100 more coin flips and it returned 49/51 and gave the Python it was using.
I remember a half-assed roll-out of funny filters on Hangouts, like adding ears to somebody's head. Didn't work on our black team members. It was probably never tested on black people before they released it? You can't hide a subconscious racist Asian-White bias by using counter racism.
The problem is systemic and it starts at the very beginning when they vet the people they hire. If you didn't go to particular American, European or Asian universities or have a non-super talent career path you're not going to get in.
Which is exactly what happens to people from disadvantaged backgrounds or countries.
You cannot preach diversity and at the same time not even represent the diversity that exists among white people from the United States alone at the same time.
Yeah, messed up by making it too obvious. Don't blame the computer, the algorithm dutifully complied. It turns out the input was garbage, and by extension the ideology behind it. The model is just a mirror, and in this case it was too clear of a mirror for their liking.
> the model became way more cautious than we intended and refused to answer certain prompts entirely — wrongly interpreting some very anodyne prompts as sensitive
Sounds like the same effect they've had on society ;D
The problem is not image generation, is the entire thing.
reply