A large language model is a statistical system trained to predict the next token in a sequence of text. By training on trillions of words, it learns grammar, facts, reasoning patterns, and even some coding ability.
Transformers: the core architecture
Modern LLMs use a transformer architecture with attention mechanisms. Attention lets the model weigh the importance of different words in a sentence when predicting the next word. This is why models like GPT and LLaMA can handle long-range context.
Training and fine-tuning
First, the model is pretrained on general internet text to learn language. Then it is fine-tuned with human feedback to be helpful, harmless, and honest. Reinforcement learning from human feedback is the method used by most chat models today.
Limitations
LLMs do not truly understand text. They can hallucinate, repeat biases, and confidently state false information. Always verify critical claims from authoritative sources.