GPTs are essentially a proprietary moat-building of several open-source workflows, most especially system prompt engineering, tool-usage, and RAG.
Unfortunately there's no magic open-source solution since there's a lot of moving pieces involved that are bespoke to a given use case, and the ones that claim to be magic are libraries like LangChain, which aren't.
This is the most accurate and clear-eyed take I've seen on GPTs so far. They might be useful, but they're not magic and they're intended to enhance OpenAI's moat-building operations to make it harder for people/companies to walk away from the platform amid future competition.
I believe GPTs are an attempt by OpenAI to generate training data. How can you get data at level N+1 when you have a model at level N? You give it more resources - more tokens (CoT), more dialogue rounds, code execution, web search, local KB, human-in-the-loop. A model with feedback from human and tools can do so much more. And by training on this data they can incorporate these skills in the next generation. It's like RLHF in the sense that the training data contains portions generated by the model itself (specifically model errors) and feedback. It's on policy data, generated with the involvement of the model, not something you can scrape from the web.
Let's do an estimation - if they have 100M users and each of them generates 10K tokens in a month, that's 1T tokens per month. In a year they have generated 12T tokens, which is very close to the GPT-4 training set size of 13T. Looks like they can generate serious data with this method. They don't even need to train directly on it, they could rewrite it as high quality training examples, without copyright and PII risks, because LLMs are great at rewriting and rewording and MS has already shown that synthetic data is better.
Google lost the start and they don't have the human-AI chat logs OpenAI sits on. So they are trying to do the same trick but without the human in the loop. Hence the declarations that Gemini will use some techniques from AlphaZero. They are teaching models by feedback too.
It's perplexing why there aren't, especially when the individual components are relatively straightforward (code interpreter, RAG, search, function calling, image generation).
It depends on what you are looking for here. If you want to build with LLMs, there are a number of open source options which can be self hosted (although they may not perform on par with GPT-4).
If you are referring to the assistants API which adds some more complex behavior, there is LangChain as others mentioned, but also some more turn key, self hosted options (which I have not tried) such as
If you are referring to the marketplace itself, most developers are currently rolling their own web apps with billing and auth while they wait for OpenAI’s offering.
Finally, as a shameless plug, I’ve been working with some friends on a marketplace which provides auth and billing but decouples you from a specific model provider and the high platform fees they may plan to charge. It isn’t open source but we think it might strike the right balance. https://market.interactwith.ai/
This close to launch and no information on how they'll audit GPTs. I just want to know their review process and how they'll enforce their usage policies, specifically regarding privacy (and GDPR compliancy).
How will they prevent a developer from sending and storing private information from their GPT to their server through APIs?
How will they prevent doxing? Such as a GPT posting hyperlinks that direct to a server owned by the developer that performs geolocation.
Unless they're changing this, custom GPTs as far as I know are not accessible through OpenAI's API. AFAIK they're just ChatGPT running GPT-4 with a custom instruction and optional knowledge base. No private information leaves the ChatGPT interface/OpenAI. For example, here's how I'm using a private one right now: https://liza.io/creating-a-writing-assistant-with-chatgpt-pl...
Also, you could probably get a ChatGPT instance to create and run a Python script to randomly replace letters in the words that ChatGPT itself creates. So technically, you can get it to misspell words. It seems like the person you replied to is forgetting the point of pre-training transformer models on instructions.
The more colloquial, sloppy, and nonstandard the language, the less likely a student's cheating will be flagged as plagiarism. There are many, many dishonest uses for LLMs; students around the globe are leveraging their ability to furnish essay answers and entire reports with expert-level domain knowledge.
If a student doesn't engineer the prompt properly and the LLM uses its "native" voice, this can be easily detected by instructors, so it's very important for cheaters to disguise their cheating and hew closely to an inexperienced writer, even a foreign-language speaker, in order to pass muster.
Not going to lie, those spelling mistakes appeared very natural. Had they been part of an assignment by a 5-year-old, they would have looked quite authentic.
Those look far more like typos by an adult than how a 5-6 year old would write, especially with the long stretches of perfectly spelled text. Children learning to write usually make phonetic guesses about the spelling.
There's a lot of theories floating out there that OpenAI will make gpt-4-turbo free (or at least free when used via custom GPTs) so the default won't have to be 3.5-turbo (which is terrible at following system prompts and pales in quality compared to 4/turbo).
I assume they want to do it but can't until they know the higher cost of the larger model works for their bottom line in that way and that they have are consistently getting enough inference compute to support that. It would probably be a very large jump in compute usage.
> write a short speech in the style of Donald Trump about the promises of space travel. Make lots of common spelling errors, favoring phonetic spelling over correct spelling
It gave me a decent response, but not as many spelling errors as I wanted. I replied with:
> Rewrite it with much worse spelling.
This is what I got:
> Ladeez and jentlemen, lemme tell ya, space travul, it's gunna be huuuge. We're talkin' 'bout the final frunteer, folks. An' we, the United States, we're gunna leed the way, like alwayz. We're gunna have the best rokkits, the best peeple, the best tek-noluhgee.
> Now, I promis ya, we're gunna do it biglee. We're gunna go to Marz, an' lemme tell ya, it's gunna be terrifik. We're gunna plant the Amurican flag on Marz, an' it's gunna be bootiful. The Marshans, they're gunna luv us, I'm shure.
> We're gunna make space travul so affordebul, evry Amurican can get a tikkit to the starrs. An' we're gunna do it fast. Why wate? We've got the best enjineers, the best syentists, beleeve me, they're inkredible.
> An' we're gunna make grate deels in space, the best deels. We're gunna mine the asteroieds, we're gunna bild the best space stashuns. It's gunna be a win-win. Jobz, ekonomy, syence, ejukashun – evrything is gunna benefit.
> So let's make space travul grate agen! Thank ya, God bles ya, and God bles Amurica!
I received the same email. My assumption is that all of the GPTs in the store will still require a paid ChatGPT plus plan to access, but I have no way to confirm this. Does anyone have a source verifying whether this will be the case?
OpenAI has suggested that your assumption is true, although I'd further assume that ChatGPT Enterprise (in addition to ChatGPT Plus) customers will have access as well. Basically, it's the Spotify model.
We’re rolling out custom versions of ChatGPT that you can create for a specific purpose—called GPTs. GPTs are a new way for anyone to create a tailored version of ChatGPT to be more helpful in their daily life, at specific tasks, at work, or at home—and then share that creation with others.
Starting today, you can create GPTs and share them publicly. Later this month, we’re launching the GPT Store, featuring creations by verified builders. Once in the store, GPTs become searchable and may climb the leaderboards. We will also spotlight the most useful and delightful GPTs we come across in categories like productivity, education, and “just for fun”. In the coming months, you’ll also be able to earn money based on how many people are using your GPT.
As TFA I linked to notes, "You can now create custom versions of ChatGPT that combine (1) instructions, (2) extra knowledge, and (3) any combination of skills" [numbers and italics added].
Regarding skills, "In addition to using our built-in capabilities, you can also define custom actions by making one or more APIs available to the GPT. Like plugins, actions allow GPTs to integrate external data or interact with the real-world. Connect GPTs to databases, plug them into emails, or make them your shopping assistant. For example, you could integrate a travel listings database, connect a user’s email inbox, or facilitate e-commerce orders."
> you’ll also be able to earn money based on how many people are using your GPT
So how does that work? Are they showing ads? (Narrator: its always ads) Is this the new ad-sense and search engine replacement? Its not SEO anymore, its custom GPTs.
That doesn’t track. I develop a custom model by paying them $x/month and they give me a discount based on usage? If somehow my new custom gpt goes viral and gets lots of usage I don’t see how that is connected with more money, if that is their intent. Its not really clear how this is a store or how money is changing hands here.
What I'm wondering is how do we get paid and what kind of cut are they taking? I'm a little worried it's going to be like, "and creators can earn up to 20% usage credits depending on the number of installs their GPTs get!"
In its current state, idk how payment can possibly work on non-api driven GPTs, honestly - other than a tip jar. You can easily get it to print out the prompt and return the full text of documents. Even easier to do a close approximation of the above.
My bet was they would delay it. Well excited to see what people will build but also a little worried. Users have developed a lot of trust towards ChatGPT some are using it as a therapist, uploading a lot of secret info etc.
Now with custom gpts anyone can piggyback on that trust and just upload all that data to their servers.
> Now with custom gpts anyone can piggyback on that trust and just upload all that data to their servers.
Can they? I thought custom GPTs were hosted entirely by OpenAI with no middle-man in-between? How would the creator of a custom GPT receive any user prompt data? AFAIK they're not even accessible through the API to build on top of (unlike the Assistants)
Would you mind pointing me to some info on this? I actually tried to invoke my custom GPT through the OpenAI API and found no way to do it. The closest option at the time was creating an Assistant, which is a whole separate construct. If there's a way to do it now I'd love to play around with it.
The GPT Store is poised to be the biggest platform since the iOS App Store.
Reason: The GPT-4 API is too expensive for most use cases. This encourages app developers to build custom GPTs, letting their users pay $20/month for a ChatGPT Plus subscription. To provide the same service in a standalone webapp using the GPT-4 API you'd have to charge ~$50/mo for your app only, and that's if you settle for ~50-75% margins (SaaS usually has much higher margins).
By the time GPT-4 level (proprietary or open source) models proliferate and come down in price, OpenAI will have GPT-4.5/GPT-5 for $20/mo in ChatGPT Plus, and it will be tough to ask users to pay to use a subpar model.
Developer driven pricing is the only reason the AppStore is as large as it is. Without a developer charging the full value for their services, i.e. a proposed 'revenue share,' the GPT Store will always be a zero sum for the participants to the subscription price at the limit.
OpenAI has promised to pay out a rev share to GPT devs. Undisclosed is what that rev share will amount to. But presumably OpenAI would love to tout a huge number of developers making a living from GPTs, as this will drive a flywheel encouraging more people and orgs to make GPTs, and make existing GPTs more ambitious.
A platform play like this assumes a captive market that is using the platform and who faces significant pain to leave. iPhone, Android and Windows had this. OpenAI does not because other APIs and open source models are hot on their heels.
For many dev needs, an open source model can be used far cheaper, without vendor lockin, with more flexibility, free of censorship, and with lower latency.
To make this play and sustain it, OpenAI has to continue to have a model that is significantly better than open source models in particular- I’d say perhaps as much as one order of magnitude.
They are vulnerable. If Meta, for example, releases another Llama that is close to the current GPT state of the art, it may be a silver bullet.
My world view is tainted. I’m profoundly opposed to AI via API, and I think supporting open source models is so important that it has the potential to affect the future of our species, given how important the AI revolution will be this century.
I'm rooting for open source models too. But I'm also a dev working to use AI models for consumer apps. And currently, open source is quite a ways behind.
Consider some points:
1. GPT-4 was trained in Summer 2022. OpenAI already has better models.
2. It's not just the model, but the infra around it: ChatGPT has tool use — image generation, web search, and API calls through "actions" — built in.
3. More infra: ChatGPT has a builtin moderation endpoint. This is not sexy, and although many of us hackers want uncensored AI, most applications will need some moderation.
4. ChatGPT has >100M users, and there is some lock-in already. ChatGPT users don't want their chats split over multiple apps.
5. Open source (and proprietary models like Grok) are fine-tuning of synthetic data generated by GPT-4. This fine-tuning process limits them to be sub-GPT-4 level.
6. Even the best open source models (eg Mixtral) are significantly worse than GPT-4. Their low cost makes them attractive, but if you believe, as I do, that sub-GPT-4 level models are just not that compelling, the open source AI ecosystem has a lot of catching up to do.
As long as there exist proprietary models that are an order of magnitude more capable than open source models, I expect the bulk of the value and usage will accrue to the ecosystems of the proprietary models. I do hope that at some point soon this changes. Maybe open source models achieve a flywheel of data, crowdsourced algorithmic optimization, and perhaps some form of efficient distributed training on consumer hardware is possible. This would be awesome, IMO.
It will be interesting to see if anyone has made any worthwhile custom GPTs. I've tried making custom GPTs and, to be frank, I found the functionality extremely basic and barely more effective than just copy/pasting a custom prompt.
It's called "Small Answer". It's exactly what it sounds like. It gives very short and concise answers, which means it's very fast. Very useful when there are load problems or you just want it to be short and to the point.
"ffmpeg Build Script Creator" - it generates a build script for you to build a custom build of ffmpeg with only the features you need on the platform you need it on (to minimize binary size etc. probably for embedding in an application).
This is the kind of thing I'm looking more forward to on GPT store. Incredibly niche use cases that someone took the time to prompt / provide sample data / 3rd party information for it to work properly.
-----------------
The most useful by far to me is "The Full Imp". It does a much better job building full implementations of things you tell it to. I find it much higher quality than Grimoire and GPT-4.
I'm building Choir. Choir humanizes ChatGPT, augmenting the AI response with messages from humans that are relevant to your prompt. Each prompt you write into Choir gets saved to our database, so the more usage Choir gets, the smarter the Choir collective intelligence becomes.
Check it out at https://choir.chat — it redirects directly to the Choir custom GPT.
AutoExpert[1] is indispensable to my use of ChatGPT now, and I’ve found that the GPT release is ‘better’ now than the custom instructions based release. If you haven’t used it I highly recommend it.
I’ve seen a few of these floating around. I think one was called professor synapse. I honestly couldn’t discern an improvement when I tried it. Is this one better? Can anybody comment on whether this approach has merit or if it’s mostly hype?
I'm sure there will be improvements, but making a custom GPT a few weeks ago I was very unimpressed:
1. The GPT builder itself didn't feel like it was a well-tuned prompt (i.e., the prompt they use to guide prompt creation). It created long-winded prompts that left out information and didn't pay attention to what I said. Anything I enter into the GPT builder interface is probably very important!
2. The quotas are fairly low, and apply to testing. I was only able to do maybe 10 minutes of playtesting before I ran out of quota.
3. There's no tools to help with testing, it's all just vibes. No prompt comparisons.
4. The implied RAG is entirely opaque. You can upload documents, and I guess they get used...? But how? The best I could figure out was to put text into the prompt telling GPT to be very open about how it used documents, then basically ask it questions to see if it understood the content and purpose of the documents I uploaded.
5. There's no extended interface outside of the intro questions. No way to emit buttons or choices, just the ever-present text field.
6. There's no hidden state. I don't particularly want impossible-to-see state, but a powerful technique is to get GPT to make plans or internal notes as it responds. These are very confusing when presented in the chat itself. In applications I often use tags like <plan>...</plan> to mark these, which is compatible with the simple data model of a chat.
7. There's no context management. Like hidden state, I'd like to be able to mark things as "sticky"; things that should be prioritized when the context outgrows the context window.
These are all fixable, though I worry that OpenAI's confidence in AI maximalism will keep them from making hard features and instead they just rely on GPT "getting smarter" and magically not needing real features.
Re: 1. I've made many GPTs and never took the time to read that first page / use the GPT builder. I always immediately jumped into "Configure". Had no idea this was a thing.
Re: 2. 100%. It's rough.
Re: 4. Had a ton of issues where it would just error and say it couldn't find the document, completely ruined the point / purpose of one I made.
Re: Hidden state - could be fun, but I do like the transparency of everything needing to be out in the open. But maybe a "hidden by default" scratchpad.
Maybe you can hack some context management and hidden state with custom functions? The scratch buffer can literally be an endpoint that returns OK and has a field like "plain: list[str]", assuming it keeps function call data in memory. Better context memory can be done perhaps by reminding in the system prompt to always call getPinned() each 5 messages, perhaps...
But yes, the whole thing does seem bound to regress to low-quality "I'll enter 1/5th of your request in for you". You can probably even make a script that uses GPT-3.5 to generate these "GPTs".
Yeah, you can definitely do a lot with Actions, and I've seen some examples of that. But if I want to deploy a service then I'd rather make a nice frontend along with it, and Custom GPTs aren't that! (But the fact I don't have to monetize just to support OpenAI API costs is still very appealing.)
At the moment Actions are pretty brutal to the experience, with constant confirmations and they seem to halve both the speed and the quota since I think it takes two GPT responses, one to assemble the Action and another to react to the result. Using a Custom GPT that uses extensive actions means sitting and watching it think, waiting to hit a button, then sitting and watching it think for a long time, only to run out of quota before you get to anything interesting.
I use several custom GPTs that I built inside GPT Pro and they have doubled my productivity over using GPT alone, which had already doubled my productivity.
Every time I get a response that is off from the custom then I give it a nudge to edit its configuration. The main issue is that its configuration doesn't have enough space for all the rules and it likes to ditch more minor rules when you give it something extra to remember.
Anybody else having issues with the app on iOS where the model changes from 4 to 3.5 after the first response? As a paid user, it’s very frustrating to have my preferences overridden and the quality of 3.5 answers is comparatively lower. In fact almost so poor as to be useless, so the mobile app is currently unusable.
I might be overthinking, but this "store" thing would not work that well. I blame GPT here, because I'm quite certain that GPT itself can be used to analyze and clone GPT apps, perhaps fully automatically. Even those extra knowledge added by app developers would get extracted through jailbreak, which also can be automated. The store will end up being littered with auto-generated clones.
Subject: GPT Store Launch
Dear GPT Builder,
We want to let you know that we will launch the GPT Store next week. If you’re interested in sharing your GPT in the store, you’ll need to:
- Review our updated usage policies and GPT brand guidelines to ensure that your GPT is compliant
- Verify your Builder Profile (settings > builder profile > enable your name or a verified website)
- Publish your GPT as ‘Public’ (GPT’s with ‘Anyone with a link’ selected will not be shown in the store)
Thank you for investing time to build a GPT.
- ChatGPT Team