Apple’s big AI week ends with Mac + Muse AI supercomputers
if ( !emtpy($headline_subheadline ) ) : ?>
Apple and Meta bring Muse Spark Open to Mac Studio, MLX gets an upgrade and more.
Apple’s big AI week ends with Mac + Muse AI supercomputers
if ( !emtpy($headline_subheadline ) ) : ?>
Apple and Meta bring Muse Spark Open to Mac Studio, MLX gets an upgrade and more.
endif; ?>
Credit: Apple
These improvements include:
Software support for the Neural Accelerators in the GPU of the M5 family of chips
Low-latency communication across several Macs using Remote Direct Memory Access (RDMA) over Thunderbolt
Efficient implementations of recent popular architectures such as mixture of experts (MoE)
The release leans heavily into what Apple’s teams have been doing with Muse, and once again demonstrates how the very architecture of Apple silicon supports AI inferencing. That’s no surprise to anybody at this point: not only have I been banging on that drum regularly, but almost anyone who is anyone in AI development and research already works on a Mac.
LLM inference with MLX on Mac Studio
A new white paper from Apple’s machine learning team shares what it has been doing with MLX, along with significant statistics concerning how well its systems support Muse Spark Open, Meta’s new open-weight frontier model.
“On a cluster of four Mac Studios, Muse Spark Open generates over 36 tokens per second, well above average human reading speed,” they said.
That’s impressive. It means for the cost of four Mac Studios, a business can run a full-strength, frontier-class model with almost a trillion parameters, on-premises, securing company data while extracting whatever benefits there are to be extracted from the use of it.
Mac Meta matters
Apple and Meta collaborated on making it possible to run Muse Spark Open AI models on Macs. It’s a shrewd move for both companies, given the growing desire to run powerful AI models locally rather than in the cloud. As a result of this work, it’s possible to use Muse Spark Open on two or four Mac Studios to engage in all kinds of agentic tasks.
Commenting on this on X, new Apple CEO John Ternus wrote: “Amazing to run a model of Muse Spark Open’s caliber and scale this efficiently, right from your desk. Mac Studio keeps pushing the boundaries of on-device AI!”
Meta’s AI chief Alexandr Wang pointed out that this is in part made possible by Apple’s Unified Memory architecture, which can provide sufficient memory to run these models.
Your Macs. Your supercomputer
“Apple’s unified memory architecture lets the M5 Ultra’s integrated GPU use up to 512 GB per Mac Studio, so two machines can hold an entire model,” Meta explained. “And the chip’s 1.2 TB/s of memory bandwidth is fast enough to read the small slice each token needs. The result is two Mac Studios behave like one supercomputer with a terabyte of unified memory.”
Meta says it tested the system using agentic workflows running entirely on-device, including applying bug fixes and new features to an open-source Mastodon client and driving DaVinci Resolve to review footage and transact tasks. The use of DaVinci Resolve makes this commentator curious, given that Apple is expected to introduce new pro Macs this month. The idea that you may be able to run tasks like this on other Macs will surely be something Apple’s developers are thinking about.
One more thing
Step by step, Apple is defining the playground for AI. In doing so it is attempting, as Greg Joswiak, Apple’s senior VP of worldwide marketing, told TIME, not to make AI for “the lunatic fringe,” but to create an ecosystem for artificial intelligence that will actually help people — a strategic approach that, if it comes together, will help maintain Apple’s position at the intersection of computing and the liberal arts.
(Read my Mac Studio review here.)
Now please subscribe to my daily, human-curated Apple-related news headline feed at The Core, or follow me on BlueSky, LinkedIn, or Mastodon.
AppleArtificial IntelligenceMacComputers and PeripheralsVendors and ProvidersIT Operations
About Author
