The Laptop Was Already a Server

This started as a chore, not a plan.

I had a pile of voice recordings I needed transcripts of, and then I wanted to do something with the transcripts. Search them, pull themes out, ask questions against them. The transcription was the boring part standing between me and the interesting part.

The obvious move was to send it all to an API. Then I read around a bit and found out there are speech models that run locally on Apple Silicon. So I tried that first.

MLX

Apple has a machine learning framework called MLX, built specifically for Apple Silicon. I ran the transcription through mlx_whisper, using Whisper large-v3 converted for MLX. No API key, no upload, no per-minute cost, and the recordings never left the machine.

It worked. Not "worked for a demo" but actually produced usable transcripts of real audio.

I later ran the same job through a hosted model, and I want to be straight about the result: the hosted one was better. Cleaner output, fewer things I had to fix by hand. If the only thing I cared about was the transcripts, that is the one I would keep using.

But the local run is the one that changed something, because of what it implied about the machine rather than about the transcripts.

The machine, not the laptop

All of that ran on my everyday laptop. Which raises a question I never asked, because there is a second Mac in this flat with twice the memory and it did not cross my mind once.

It is a spare MacBook Pro, M1 Max, 64 GB. And it had quietly stopped being a laptop a while ago without me noticing: since February it has been sitting closed on a shelf running an agent gateway as a background service. I did not write the gateway, it is open source. What I built is the system on top of it: ten scheduled jobs that run every morning, each in its own isolated session, writing notes into a vault. There is a rolling window so it never picks the same source twice, and a tracker that rotates topics so it does not drift onto one subject for a week.

It has run unattended for months. The prompts turned out to be the small part. What made it hold was keeping the state outside the model, in files the agent has to read before it decides anything.

So the thing was already a server. I just kept calling it "the old laptop" because of its shape.

To be fair to myself, it would not have transcribed any faster. Whisper large-v3 is about 3 GB, and both laptops are M1 Max, so both read memory at the same speed. The spare is not better at this job. It is better at the jobs I cannot run yet.

What makes this particular hardware interesting is unified memory.

On the RTX 4090, the box I started with, there are two separate pools. The GPU has 24 GB of its own very fast memory, and the system has its own RAM on the motherboard. They are physically different chips, connected over PCIe, and moving data between them costs real time. A model's weights have to live in the GPU's 24 GB. The system RAM on the motherboard cannot stand in for it.

Apple Silicon does not work that way. There is one pool of memory and the CPU and GPU both address it directly. On the spare that pool is 64 GB. I have not worked out yet how much of it a model is actually allowed to claim, but it is a much bigger number to start from than 24.

Ten CPU cores, thirty-two GPU cores, and 64 GB that either of them can reach.

So on paper the laptop holds a considerably larger model than the desktop with the expensive graphics card in it.

The catch, and why it is interesting anyway

Bigger does not mean faster. That same unified memory is slower than the GPU's dedicated memory, and generating text is bound by how fast the weights can be read.

The 4090's memory runs at roughly 1 TB/s. The Mac's runs at 400 GB/s. So the card pulls the weights through about two and a half times faster, and that ratio lands more or less directly on how fast the words come out.

More capacity, less bandwidth.

They are different shapes:

• The 4090 is a small fast box. It runs what fits, and it runs it quickly.
• The Mac is a large slower box. It runs things the 4090 cannot hold at all, and it runs them at a gentler pace.

That tradeoff is the thing I actually want to understand, and I cannot understand it by reading about it. I need both machines and the same test.

So it joins

The Mac becomes a node in the lab rather than a laptop I occasionally open.

Two reasons beyond the hardware. First, MLX is a genuinely different stack from CUDA, and I want to know where each one wins rather than assume. Second, Apple Silicon with large unified memory is turning into a serious option for running models locally, and that is not a trend I want to read about second-hand.

The thing I did not expect from a transcription chore was a question about memory architecture. But that is usually how it goes. You do a small practical job, and the interesting part is the thing the job accidentally showed you about the machine.