The YouTube video reviews the new M3 Ultra Mac Studio, highlighting its AI capabilities, especially for running large language models (LLMs) locally. Here’s a summary of the key points:

Specifications: The Mac Studio, equipped with the M3 Ultra chip, features a 32-core CPU, an 80-core GPU, and 512GB of unified RAM. While powerful for tasks like 4K video editing, its strength lies in gaming and AI applications.

AI Performance and DeepSeek R1: The M3 Ultra excels in AI tasks, particularly due to its 512GB RAM, which allows for the local operation of large AI models like DeepSeek R1 (671 billion parameters, 404GB). This local operation is beneficial for privacy, especially in sensitive sectors like healthcare.

VRAM Management: Initially, VRAM allocation limitations caused crashes when loading DeepSeek R1. This was resolved by manually allocating 448GB of VRAM using terminal commands.

Efficiency and Speed: The M3 Ultra efficiently runs the 4-bit quantized DeepSeek R1 at a practical speed of 17-18 tokens per second, consuming less than 200W of power. This is more power-efficient compared to PC setups that require multiple GPUs.

Price and Value: The Mac Studio is expensive, especially with the 512GB RAM configuration (starting at $10,000). However, it provides a local LLM solution, making it attractive for users who prioritize privacy and specific workflows, unlocking new possibilities previously limited by technology and cost.

In conclusion, the M3 Ultra Mac Studio is presented as a high-performing machine for AI, offering exceptional local LLM capabilities and power efficiency, despite its premium price.

At USD 10k+ and 200W for 18 tokens/s, nothing comes close to this.

hellothere131495

This post has been edited by ipohps3: Mar 12 2025, 04:48 PM

wait for madani ai chip

still cheaper than one H100 chip

can run CP2077 path tracing?