How Does ChatGPT Actually Work
@larissajeanniton invited you
For beginners who use ChatGPT: what happens to your words after you press send, from the tokens it reads to the next token it predicts.
Read moreShow less
The first idea is tokens. ChatGPT does not read your message as whole words. The text is first divided into tokens, and a token can be a whole word, part of a word, a single character or a punctuation mark. The model works on those tokens and writes its reply as tokens too.
Tokens are also how length is measured. OpenAI’s rule of thumb for English is that one token is about four characters, or three quarters of a word, so 100 tokens come to roughly 75 words. The limit on how much a model can take in at once is counted in tokens, not words.
A common misconception is that ChatGPT looks up stored answers. OpenAI says its models do not keep copies of the text they were trained on. They learn patterns and predict a likely next token, one token at a time. Because several tokens can fit and one is picked with some randomness, the same question can get different answers.
“How Does ChatGPT Actually Work” is a 4-lesson course at Teen level, created with soclever, a personal AI teacher. Each lesson takes a few minutes and ends with a check-in question; finish the curriculum and you can take a certificate test to earn a diploma. Try the first lesson without an account — or make your own course on any topic. Shared by @larissajeanniton.
What you'll learn
- Tokens: The Building Blocks. You type "What's the weather?" into ChatGPT. Before it can answer, it has to break your question into pieces it can actually work with. Those pieces are called tokens.
- Prediction: The Next-Word Game. You have probably played a game where someone starts a sentence and you finish it, like "Peanut butter and..." and you say "jelly."
Questions this course answers
What is a token?
A token is roughly a word, but not exactly, consisting of pieces that text splits into for ChatGPT to work with.
How does ChatGPT generate a reply?
ChatGPT guesses which word or small piece of a word should come next, adds that piece, and guesses again until a full reply appears.
What is machine learning?
Machine learning is a form of training where a program improves from examples instead of following hand-written rules.
More courses to explore
- How Does ChatGPT Actually Work — 4 lessons (Teen level).
- How Does Bitcoin Actually Work — 4 lessons (Teen level).
- How Does Weight Loss Actually Work — 4 lessons (Teen level).
- Fall of Rome — 4 lessons (Teen level).