Quote:
Originally Posted by oska
Indeed. And buzzword compliant too  |
Just wanted to follow up to make it clear that this earlier description of how you might think ChatGPT works is incorrect.
Quote:
Originally Posted by oska
This "AI" is not more than fancy pattern matching ie: these phrases are associated with these other phrases.
The filtered phrases are passed on to another pattern matcher that makes reasonable sentences. Done.
|
On the contrary, where it describes itself as a "state-of-the-art language
model, trained on vast amounts of text and equipped with deep learning
algorithms", these are not a set of buzzwords but instead the
precise technical terms used by computer scientists and engineers to
characterize this type of system.
Deep learning language models such as ChatGPT are neural networks
using a probabilistic approach to derive the output sequence from the
input sequence.
By comparison, pattern matching systems in the way you describe are
relatively simple algorithms.
And the punctuation of 'Done' as a kind of QED belies the fact that
this system is vastly more complex and sophisticated than you might
appreciate. This system is not just large, how it works is non-trivial.
In fact the description you provided showcases one of the challenges for
training systems such as these. They reply on a large corpus of text for
their training and that overwhelming came from the web.
And of course most of the web is written by humans. So the old saying
of GIGO - garbage in, garbage out - is absolutely true. I am trying to
coin this in the politest way without offending, but the description you
provided might be a good showcase for garbage. If a language model
were to parse it, it would add to its number of incorrect facts. It is a tough
problem, hence ChatGPT was trained on sources that tended to have
a higher probability of being factually correct compared to say opinions
on a chat forum.
As an engineer and a member of the Institute of Electrical and Electronics Engineers
and its associated Computer Society for 45 years, I've watched AI systems evolve over time.
I've been posting on these systems in the Science section since June of
2021 and you can follow one of those threads there if you would like to
learn more. :-
https://www.iceinspace.com.au/forum/...d.php?t=192039