MPS out of memory: the failure that doesn't raise an error
If you came here from RuntimeError: MPS backend out of memory, read the first section — you may be chasing the wrong failure. Mine never raised an exception at all.
First, an honest scoping note. I did not hit RuntimeError: MPS backend out of memory. I hit the quieter neighbour of it: memory exhaustion that never raised anything, on the same hardware, during the same kind of workload. If your process crashed with a stack trace, most of this won't apply. If your process is still running and doing nothing, it will.
What actually happened, twice
I ran two small language-model training jobs on Apple Silicon — an 11.5M-parameter model, a fixed five-minute wall-clock training budget, then an evaluation pass over a fixed token count. Both runs trained fine. Both died in evaluation. Neither raised an exception.
| Run | Eval batch | System free memory when it stalled | Outcome |
|---|---|---|---|
| first | 16 | 8%, ~2M pageouts | killed, no metric |
| second | 4 | 19% | killed, no metric |
The second run trained further than the first — 42 optimizer steps instead of 30, a lower training loss — and then failed at exactly the same phase, with more than twice as much memory free.
How to tell thrashing from a real OOM
A genuine MPS backend out of memory raises and the process exits. You get a stack trace, and you know. Swap-thrashing gives you none of that: the process stays alive, and everything about it looks like work is happening until you check closely.
ps -p <pid> -o pid,etime,%cpu,state # UN = uninterruptible I/O (blocked on paging), low %cpu -> thrashing # SN / RN + healthy %cpu -> actually computing vm_stat | head -5 # Pageouts climbing into the millions = spilling to disk memory_pressure | tail -1 # system-wide free percentage
U" with "output file size unchanged over 60 seconds". Either signal alone is ambiguous — a slow step looks like a stalled one, and a process can briefly enter U during normal I/O.The fix that didn't work
The obvious move is to shrink the evaluation batch, and it is the right instinct: evaluation often allocates more than training because it runs a forward pass over a large fixed token budget. In my case evaluate_bpb computed steps = EVAL_TOKENS // (batch_size * seq_len), so cutting the batch from 16 to 4 covers the identical token count in four times as many, four times smaller steps. Same metric, roughly a quarter of the peak allocation.
It did not fix it. The second run got further and then thrashed anyway, at 19% free rather than 8%. Lowering peak allocation doesn't help if the total working set still exceeds what the machine can hold alongside everything else you have open. On unified memory there is no separate VRAM to run out of — your model is competing with your browser.
Why a free-memory threshold is the wrong alarm
I built a watchdog after the first failure and keyed it on free memory below 10%, because that run had thrashed at 8%. The second run thrashed at 19%, so my watchdog could not fire. I had generalised a threshold from a single observation and built a detector guaranteed to miss the next case.
The threshold moves with whatever else is running. Process state and log progress do not. Watch those.
What I would do differently
Close everything else first, and measure free memory before the run rather than reacting during it — the same job thrashed at two different thresholds purely because of what else was open. Treat evaluation as a separate memory budget from training, since it is usually the larger of the two and it runs last, when you have already spent twenty minutes. And put the watchdog on state plus log progress, not on a percentage.
I did not get my number. Two runs, roughly fifty minutes of compute, no held-out metric. That is the honest outcome, and the diagnosis above is the part that was actually worth having.