Magnitude: an open-source inference engine that tunes its kernels on your own hardware · Cascadity

← Maker Views

0

MVMaker Views #AI #Hacker-News #maker

Magnitude: an open-source inference engine that tunes its kernels on your own hardware

Speed claims are the company’s own; the repo has the benchmark setup.

Magnitude (YC S25) launched an Apache-2.0 inference engine for local agents that compiles and tunes kernels on the user’s device; it claims up to 2x faster decoding than llama.cpp on Apple Silicon.

Read it at the source

GitHub (magnitudedev/magnitude)Magnitude ↗ Hacker NewsDiscussion: Launch HN: Magnitude ↗

Rabbit Holes 1

Llama.cpp

After exploring Magnitude, the baseline it compares with is Llama.cpp.

The open-source inference library Magnitude benchmarks itself against.

Wikipedia · Wade · 5 min

0