An SLM trained on $8 ESP32-S3
Hacker News
Published
0
0
Read the full story at Hacker NewsOriginal
Article URL: https://github.com/Carloscodix/qapla
Comments URL: https://news.ycombinator.com/item?id=49178464
Points: 4
# Comments: 1
Reader Reactions
The Story At A Glance
This project represents a significant scale increase over previous tiny LLM ports by utilizing specialized memory layouts. It demonstrates that generative AI can function on extremely low-cost, memory-constrained hardware.
- • A 28.9M-parameter language model runs on an $8 ESP32-S3 microcontroller.
- • The model operates entirely offline using 4-bit quantization and Per-Layer Embeddings.
- • It generates simple stories at approximately 9.5 tokens per second.
This project represents a significant scale increase over previous tiny LLM ports by utilizing specialized memory layouts. It demonstrates that generative AI can function on extremely low-cost, memory-constrained hardware.
Christian Perspective
Decentralized technology allows for the preservation of truth without reliance on centralized, secular gatekeepers.
What is your reaction to this story?
Reader Reactions
Want to join the conversation about this story?
Join our community at Gab.com→
Alto is powered by
Gab AI
The one AI they can't control. Our exclusive AI model trained to uphold Christian values and traditional principles in every interaction.
