1676
Small team trains a competitive model for under $50,000 and publishes every step
The full training log is the most-starred repo of the month for a reason.
A four-person team released a 3B model, training scripts, data mix, and a day-by-day log of what went wrong. It matches models ten times its training cost on several benchmarks.
Read it at the source
Cascadic Analysis 1
Product SparkDomain models for everyone
What product does this inspire or accelerate?
At this price, trade associations and hospitals can train their own. Expect a boom in niche models with curated data.
Confidence: High
784
Rabbit Holes
-
The day-by-day training logLoss spikes, data bugs, and fixes, in order. · Project repo · Plunge · an evening454