Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

Hacker News
Published
Show HN: Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s

The short version

  • Easy to install/update, and mac-native using MLX and Swift.
  • It ships with auto-mode, which makes a good tradeoff between memory usage and speed.
  • I'll be implementing and porting the MTP module for speculative decoding next Comments URL: https://news.ycombinator.com/item?id=49524447 Points: 11 # Comments: 0

The story

I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift.

It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next

Comments URL: https://news.ycombinator.com/item?id=49524447

Points: 11

# Comments: 0

Read the full story at Hacker NewsOriginal

Powered by Gab AI

The Story At A Glance

Reading this article now — analysis appears below

Reading the article

💡 AI analysis provides alternative perspectives on current events

More to read

Recent stories from across the outlets Alto indexes.

The VergeGoogle Find Hub will soon help you locate things without a tracker6 minutes ago
Discuss on GabFree to join, no account needed to readAsk Gab AIGet a second read on this story, privately

Questions Alto can answer

From this story — each link opens a live data page or a tool already filled in.

  1. What is $100 from 1990 worth today?CPI-adjusted dollars — result on the next page
  2. Where does a $75,000 household income rank nationally?Census percentile — national and state
  3. What's Alto covering on the Tech & AI desk?Latest headlines on this beat

All toolsAll topicsSource directoryStory timelinesHeadline comparisonSearchMost read

From Gab Shop

Official merchandise. Every order funds free speech infrastructure.

Gab Emblem America 250 T-Shirt

Gab Emblem America 250 T-Shirt

America 250 Liberty Bell T-Shirt

America 250 Liberty Bell T-Shirt

Go Home America for Americans T-Shirt

Go Home America for Americans T-Shirt

Deport Them All Garden Flag

Deport Them All Garden Flag

Shop all products

Install Alto on your phone

Add Alto to your home screen for breaking news — no app store, no account.

  1. Step 1Open alto.gab.com in SafariMust be Safari — not Chrome or in-app browsers
  2. Step 2Tap the Share buttonSquare with an arrow, at the bottom of Safari
  3. Step 3Tap "More"If you don’t see Add to Home Screen yet
  4. Step 4Tap "Add to Home Screen"Scroll the share sheet if you need to
  5. Step 5Tap "Add"Alto appears on your home screen like any other app.
gab

Talk Big Tech Where Big Tech Can't Reach

AI, surveillance, and censorship, covered by the people the platforms removed first.

g
Login to GabSign Up

What Makes Gab Different

We're not just another social network. We're a platform built on principles that matter.

Freedom of Speech & Reach

All First Amendment protected speech is welcome. No algorithmic throttling or shadow banning.

Family-Friendly Platform

We maintain a clean environment. Explicit adult content is strictly prohibited.

Western Nations Only

Third-world IPs are blocked. No scammers, no spam farms. Built for Western civilization.

Funded By Users

Our users are our investors and customers. You're not the product being sold.

Battle Tested

A decade of standing strong. Banned from app stores, banks—and still here.

American Owned & Operated

We reject foreign censorship demands. Built by Americans, for free people.

Support Alto & Gab

Alto is funded entirely by readers like you. Your donation helps us continue delivering curated news from a right-wing Christian Nationalist perspective, powered by Gab AI.