Maybe two models? One like current LLMs, generating the usual bullshit. A second model trained to map output from the first model to reliable citations or mapping to some value from 0 to 1 predicting the confidence of the models accuracy.
Clearly I am just bullshitting myself here, I don't know how to train the second model. Something mapping text to reliable sources...(waves hands)
> Something mapping text to reliable sources...(waves hands)
You mean basically Google search? What you want is an intelligent search engine, no such search engine exist today but not due to lack of trying, this is a trillion dollar problem.
Not to say that it is easy in absolute terms, but I'd argue that true/false'ing a statement, e.g. "humans should eat 1 rock a day" is a categorically easier problem than answering "What should humans eat"?
For fun/example I asked gpt3.5 "What percent of dieticians would suggest eating one rock a day is good for your health?" And got a pretty solid if wordy 'none'.
But how do you do "reliable citations" with the current architecture? You still have the problem that it is at its core a pattern recognition engine. It will just be "looks similar to all the reliable citations in the training set for similar subjects" not "this is the correct citation for your specific query."
Clearly I am just bullshitting myself here, I don't know how to train the second model. Something mapping text to reliable sources...(waves hands)
reply