Definition
Optimization algorithm
The method a model uses during training to adjust its parameters and make fewer mistakes, such as gradient descent.
Learn the language of AI-native founders
Every week: the new terms, tools, and workflows founders are using, straight from the podcast.
Related terms in AI Fundamentals
Open source modelsAn AI model whose weights, code, and details of its training data are released so anyone can run, inspect, modify, and build on it.Open weights and open source modelsModels whose parameters are published, so anyone can run and fine-tune them in their own environment.Open-Source LLM StrategyThe decision to release AI model weights and architecture publicly, enabling community contribution, local deployment, trust verification, and developer adoption at scale. China's approach: all major LLMs released open source. US approach (except Meta): all major frontier models remain closed. Open source compounds via the scientific method — thousands of contributors find flaws, optimize, build smaller deployment-specific variants, and expand use cases. The strategic argument: whoever wins developer adoption wins the long-term market, and open source is how you build with developers rather than selling to them.Open-weight modelA model whose trained weights are publicly downloadable, even if its training data and code are not.Opus-class vs Sonnet-classShorthand for top-tier versus the still-excellent, cheaper tier most workloads can live in.Over-Provisioned LLMA large language model deployment in which the model is being fed significantly more raw, unclean, or poorly structured data than necessary - compensating for data quality problems with raw compute power rather than fixing the underlying data. The result is higher cloud infrastructure costs, increased latency, and often lower output quality than a well-structured smaller-context approach would produce. David Carmel uses the metaphor of 'dropping bunker busters to break up data when a scalpel would work.' An over-provisioned LLM is typically a symptom of an upstream data quality problem that has not been addressed: rather than cleaning the data, the organization throws more model capacity at it and hopes the model can compensate.OverfittingWhen a model memorizes its training data, noise and all, so it performs well in testing but poorly on anything new.ParameterAn internal value a model learns during training. Large models have billions of them, and together they hold what the model knows.