LFM2 vs Qwen3.5 on an Apple M4: Every Small Model, Both Backends

Liquid AI's LFM2 model card claims 2x faster decode and prefill on CPU than Qwen3. I wanted to check that on my own hardware before repeating it. So: every LFM2 checkpoint under 2B, the two smallest Qwen3.5 models, llama-bench on my M4 MacBook (24GB), once on the Metal GPU and once CPU-only.

Raw JSONL from every run is in the post's asset folder: llama-bench-results.jsonl.

TL;DR

• At equal parameter count (LFM2-700M vs Qwen3.5-0.8B, both around 745M params), LFM2 decodes 1.2x faster on CPU and 1.6x faster on Metal. Not the 2x on the model card, but the gap is real and it showed up in every run.
• On CPU, Q8_0 prefill is about 2.5x faster than Q4_K_M for every model here. LFM2-1.2B does 175 tok/s prefill at Q8_0 and 68 tok/s at Q4_K_M. I did not expect this. If you run on CPU and care about time to first token, the smaller file is the slower file.
• LFM2-350M at Q8_0 is the only configuration that reads a 512-token prompt in under one second on CPU. Everything else takes 2 to 8 seconds before the first token appears.
• Quality: Q4_K_M costs 2.4% to 4.5% perplexity over Q8_0, so on Apple Silicon CPU Q4 is slower and worse. At matched size Qwen3.5-0.8B and LFM2-700M are within error on wikitext-2; LFM2 wins the vendors' IFEval, Qwen3.5-2B wins knowledge.
• Metal decode tops out at 146 tok/s (LFM2-350M Q4_K_M) and bottoms out at 31 tok/s (Qwen3.5-2B Q8_0). All of these are usable for chat. The interesting numbers are the CPU ones.

Setup

Two workloads per model: pp512 (read a 512-token prompt, the time-to-first-token number) and tg128 (generate 128 tokens, the chat speed number). Five repetitions each.

The lineup

Both families are hybrids that swap most attention layers for something cheaper, so this is a fair fight.

Two numbers in that table predict the speed results before you run anything.

The vocabulary: Qwen3.5's 248k-token embedding is 3.8x larger than LFM2's 65k. On Qwen3.5-0.8B that is 254M parameters, a third of the model, sitting in the embedding and output projection, and the output projection is a full matmul on every decoded token. LFM2-700M spends 100M there.

The depth: LFM2 runs 16 layers to Qwen3.5's 24 at the same parameter count. Fewer, wider layers means fewer kernel launches per token, which matters most on Metal.

Decode speed (tg128)

The head-to-head at matched size:

The model card's 2x does not reproduce here. The card compared against Qwen3, whose small models were plain transformers; Qwen3.5 moved to a Gated DeltaNet hybrid and closed much of the gap. What holds up: LFM2 is the faster architecture at equal parameter count, on both backends, at both quant levels.

LFM2-1.2B vs Qwen3.5-2B, the biggest model in each family: 35 vs 23 tok/s on CPU, 70 vs 37 on Metal. Qwen3.5-2B has 1.6x the parameters though, so read that as a per-watt tradeoff, not an architecture verdict.

Prefill speed (pp512) and time to first token

This is where the quantization result from the TL;DR comes from.

Prefill is compute-bound: 512 tokens go through every weight matrix in one batched matmul, so bytes-per-weight barely matters and unpacking 4-bit nibbles dominates. Q8_0 skips most of that. A 4-bit model on CPU pays a 2.5x tax on every prompt it reads.

As time to first token on a 512-token prompt (a short system prompt plus a paragraph of input):

Nothing here hits 100 ms on CPU with a 512-token prompt; that is a short-prompt or GPU/NPU story. But LFM2-1.2B at Q8_0 reads a prompt faster than Qwen3.5-0.8B at Q4_K_M, despite 1.5x the parameters.

One oddity: Qwen3.5-0.8B at Q4_K_M is the only place Qwen beats LFM2 at matched size, 129 vs 107 tok/s prefill. At Q8_0 the order flips back (274 vs 224). My guess is that Gated DeltaNet's chunked prefill kernel is well-tuned and LFM2's short-conv path does more dequant work at Q4.

On Metal, prefill is a different world:

Every model reads a 512-token prompt in under one second on Metal; LFM2-350M does it in 160 ms. The Q4 vs Q8 gap mostly vanishes on GPU, which has compute to burn.

Footprint vs speed

For a CPU-only deployment with a memory budget, this is the chart:

• Under 500 MiB: LFM2-350M Q8_0 (359 MiB, 119 tok/s) is the pick. LFM2-700M Q4_K_M (445 MiB, 53 tok/s) fits too but you pay the 2.5x prefill tax.
• Under 800 MiB: LFM2-700M Q8_0 (753 MiB, 59 tok/s, 1.9 s TTFT) beats Qwen3.5-0.8B Q8_0 (764 MiB, 45 tok/s, 2.3 s TTFT) on every axis at almost identical size.
• Under 1.2 GiB: LFM2-1.2B Q8_0 (1,186 MiB) at 37 tok/s and 2.9 s TTFT, versus Qwen3.5-2B Q4_K_M (1,211 MiB) at 23 tok/s and 8.4 s TTFT. Qwen3.5-2B is the more capable model on paper (1.9B params, 262k context, vision), so this is the one tradeoff where "slower but smarter" is a real option.

Does speed cost quality?

Fast is only useful if the model is still worth running. Three checks, cheapest first.

Vendor benchmarks, where they overlap

Both model cards report IFEval and MMMLU in non-thinking mode. Nothing else lines up (Liquid reports MMLU, Qwen reports MMLU-Pro and MMLU-Redux). Different labs, different harnesses, so treat the gap as a hint, not a measurement: both cards score Qwen3-1.7B on IFEval, and Liquid gets 73.98 where Qwen gets 68.2 for the same model.

Source: each model's Hugging Face card. Qwen reports GPQA only in thinking mode, so that column is not apples to apples. The picture: LFM2 wins instruction following at every size, Qwen3.5-2B wins knowledge, and the Qwen numbers come from a model that also does vision and 201 languages with the same weights.

Perplexity: what quantization actually costs

This one I can measure myself. llama-perplexity on wikitext-2, context 512, 120 chunks, all 10 GGUFs.

At matched size, Qwen3.5-0.8B (18.99) edges LFM2-700M (19.23), inside the error bars. Wikitext is English Wikipedia and Qwen has the bigger vocabulary, so I expected a wider gap. The useful result is the Q4 column: on this hardware Q4_K_M is a strictly worse point, slower on CPU and a few percent worse on perplexity, unless the file size itself is the constraint.

Six-prompt smell test

Three writing prompts, three coding prompts, every model at Q8_0, temperature 0, one shot. Coding answers were run against asserts. Writing answers were checked for the stated constraint (no "hydration", exactly three lines) and read by me. Raw outputs are in smell_results.json.

Six prompts is not a benchmark, but the failures were telling. The two sub-500M models both wrote "hydrated" or "hydration" after being told not to. LFM2-700M's fizzbuzz never emitted "FizzBuzz" and returned ints instead of strings. Qwen3.5-0.8B's oldest returned the whole dict instead of the name. LFM2-700M also closed its mitochondria answer with "Let me know if you'd like another simpler way to explain it!", which is a chatbot habit leaking into a one-shot task. Both 1B-class models went clean.

Full results

Tokens/sec, mean of 5 runs. CPU stddev under 5% on every row. Metal pp512 stddev 10 to 25% on the two smallest models (runs so short that scheduling noise shows), under 15% elsewhere.

What I would actually deploy

On Apple Silicon CPU, default to Q8_0, not Q4_K_M. Decode is the same speed or faster, prefill is 2.5x faster, and you skip the Q4 quality loss. The only reason left to pick Q4 is a hard memory ceiling.

LFM2 is the faster architecture per parameter, by 1.2x to 1.6x. The 2x on the model card was against Qwen3's plain transformers; against Qwen3.5's hybrid the gap narrowed but did not close. I suspect the smaller vocabulary does a good chunk of the work.

If you need TTFT under a second on CPU, you need a sub-500M model. LFM2-350M at Q8_0 is the only one here that does it, at 0.9 s for 512 tokens. From there the levers are shorter prompts, prompt caching, or moving prefill to the GPU or NPU.

Reproduce it

Copybrew install llama.cpp   # need a build new enough for the qwen35 arch
pip install -U huggingface_hub
 
for m in 350M 700M 1.2B; do
  for q in Q4_K_M Q8_0; do
    hf download LiquidAI/LFM2-$m-GGUF LFM2-$m-$q.gguf --local-dir models
  done
done
for m in 0.8B 2B; do
  for q in Q4_K_M Q8_0; do
    hf download unsloth/Qwen3.5-$m-GGUF Qwen3.5-$m-$q.gguf --local-dir models
  done
done
 
cd models
for f in *.gguf; do
  llama-bench -m $f -ngl 99 -t 4 -p 512 -n 128 -r 5 -o jsonl   # Metal
  llama-bench -m $f -ngl 0  -t 4 -p 512 -n 128 -r 5 -o jsonl   # CPU only
done

Set -t to your performance core count (sysctl hw.perflevel0.physicalcpu). Run it on an idle machine or the CPU numbers will lie to you.