One of aspects is putting latest knowledge to the model. They could, given enough requests, provide a special subscription service, where you get the latest model every N DURATION time.
Or, maybe a finetuned version for your particular dataset?
Of course I have no idea, just speculating
EDIT: I'm speculating they might be just investing some marketing budget into this model, hoping, it would allow for capturing enough target audience to upsell related services in the future
I wonder if there might already be enough datasets laying around that even if we stopped making new ones, models can generate entire new datasets on their own, process/label them on their own, and generate new and improved models in a closed feedback loop. The only data that's really needed going forward is information on current events.
This is probably naive, but I’m imagining something like the US Library of Congress providing these models in the future. E.g., some federally funded program to procure / create enormous data sets / train.
This is a great question. Actually, I have two opinions about this:
- It is not economically useful for foundational model providers to build this, or at least integrate it with their products, because it will cost them double. They would prefer RLHF and something intrinsic to their models.
- Our dataset is proprietary. We have to collect and write a huge number of prompts to fine-tune it well. And our model is not 100% LLM, we use a version of the evolutionary algorithm to control the quality.
I would say we have a tiny moat of data and algorithm. But at the end of the day, it is about delivering something people find useful. If we could be replaced, let it be. But it works for now.
That's amazing! This model is a huge opportunity to create annotated data (with decent quality) for just a few dollars. People will iterate more quickly with this kind of foundation model.
I don't think they have been doing it for 4 years as AutoML was quite recent. The idea maybe was there before but noone publish any paper about it before.
Bear in mind that this service creates the architecture of the model for you. I think ClarifyAI has a predefined model that is fine tune with the data, which it is not even similar.
They probably won't share how they did it, but there's been a lot of research over the past 6 months showing how you don't have to retrain the entire model to add in new sources. I know nothing about this stuff, but my limited understanding from blog posts is it's easier than anyone had thought to add in new data to a pre-existing model.
Because having large datasets allows me to merge the data with proprietary sets, or increase my own predictive models.
I have real estate models, Energy Trading models, cryptocurrency models, mechanical failure predictive models, insurance models, sentiment models and clustering models
Do we need for the model to be be continuously updated from data sources or is it good enough that they can now figure out either by themselves or with some prompting when they need to search the web and find current information?
I feel like within 6 months the models will have adapted to not need these "clever" tricks. Presumably, if for many cases the trick is to say "Let's think step by step", that's something the model can learn to do on its own without the prompt.
The real interesting thing will be feeding alternative data into these models. Whether it's certain structured corpus, silo'd enterprise data, or personal data.
Maybe the real game changer in the future will be the ability to train the same model on very different kind of inputs like video, images, text, audio... Imagine also all these data cleaning tasks are already automated, you just need to feed the model PDFs and automatically a support model will extract all the relevant metadata... or probably you'll just be able to select a set of books from an online library and your model will train on them as well (of course for a non trivial subscription lol)
Awesome, can’t wait to try this. I wish the big AI labs would make more frequent model improvements, like on a monthly cadence, as they continue to train and improve stuff. Also seems like a good way to do A/B testing to see which models people prefer in practice.
That's exactly the target use case. Models make online predictions as part of Postgres queries, and can be periodically retrained in a cadence that makes sense for the particular data set. In my experience the real value of retraining at a fixed cadence is so that you can learn when your data set changes, and have fewer changes to work through when there is some data bug/anomaly introduced into the eco system. Models that aren't routinely retrained tend to die in a catastrophic manner when business logic changes, and their ingestion pipeline hasn't been updated since they were originally created.
Stability.ai and probably others too already working on video and audio models too and also I think I heard a service to train / finetune the model with your own dataset.
Or, maybe a finetuned version for your particular dataset?
Of course I have no idea, just speculating
EDIT: I'm speculating they might be just investing some marketing budget into this model, hoping, it would allow for capturing enough target audience to upsell related services in the future
reply