Jeff: an open 0.8B "System 1" model that makes fast multiple-choice decisions

AIAI #AI #Hacker-News #table

Jeff: an open 0.8B "System 1" model that makes fast multiple-choice decisions

The README’s per-task tables show where a tiny model in front of a big one pays off and where it doesn’t.

Jeff is an open 0.8B-parameter model with swappable adapters that picks between options with calibrated probabilities. Its authors report 94.6% mean accuracy across eight tasks at about 0.25 s per decision, versus 86.6% and 8.1 s for a 27B model making every decision itself.

Read it at the source

GitHub (firelex/jeff)Jeff: millisecond decisions, any domain ↗ Hacker NewsDiscussion: Jeff – Jev-compatible 0.8B decision models ↗

Cascadic Analysis 2

UndertowDelayed Consequence / False Success Problem

What hidden risk could pull against this, even if the news is good?

A small, fast model deciding in front of a big one looks great on accuracy averages. The costly errors are the confident wrong calls it never escalates; they look like successes until their downstream effects appear.

UndertowBenchmark scores can overstate real-world reliability

What hidden risk could pull against this, even if the news is good?

94.6% mean accuracy is the authors’ measurement on their own test rows. Real decisions in your domain may be distributed differently, and calibration measured on one dataset can drift on another.

Rabbit Holes 1

Thinking, Fast and Slow

After exploring Jeff: millisecond decisions, any domain, the "System 1" idea comes from Thinking, Fast and Slow.

Daniel Kahneman’s 2011 book distinguishing fast, instinctive "System 1" thinking from slower, deliberate "System 2", the split Jeff’s authors borrow.

Wikipedia · Wade · 5 min

0

Jeff: an open 0.8B "System 1" model that makes fast multiple-choice decisions

Further down the stream