←
Vik | Practical AI & Tech4,602 posts
Vik | Practical AI & Tech avatar

Vik | Practical AI & Tech

@uctive

Private local AI server for small businesses: your data stays inside your company. github.com/ViktorIXAI/clarye…

◷ Joined December 2011▧ 318 Photos and videos
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·Nov 14

🚨 €135 FOR A COMPLETE LOCAL AI COMPUTER. NOT JUST THE GPU. THE ENTIRE MACHINE! 🤯 And the story is even crazier. 🖥️ HP Z220, Xeon E3-1270 v2, 32GB DDR3 — €50. Bought 5 years ago. Collecting dust in my garage ever since. 🔥 AMD RX 580 2048SP, 8GB VRAM — €49 delivered this week. 💾 256GB NVMe — €30 delivered this week. 🔧 PCIe-to-NVMe adapter — €6 on Amazon. TOTAL: €135! We installed Linux, built llama.cpp b12318 with Vulkan, and benchmarked five quantized AI models. 📊 REAL MEASURED RESULTS Prefill / Decode (tokens/sec): • Qwen2.5 3B Q4_K_M — 397 / 56.8 • Nemotron 3 Nano 4B Q4_K_M — 270 / 39.1 • Nemotron 3.5 Bolt E4B Q4_K_XL — 441 / 29.3 • Ternary Bonsai 2 9B Q2_0 — 172 / 26.9 • Ternary Bonsai 2 27B Q2_0 — 48 / 9.3 Yes, even a 27B model runs on this €135 machine! Part of it is offloaded to system RAM. Test protocol: 512 prompt tokens, 128 generated tokens, maximum GPU offload, flash attention OFF. These are benchmark workloads, not maximum context-window measurements. GPU peak: 120W, 46°C. No thermal throttling. 🤯 AND HERE’S THE SURPRISE: We moved the same RX 580 from a much more powerful 12-core, 64GB quad-channel DDR3 workstation into this old 4-core, 32GB dual-channel machine. Performance barely changed! For these small models, the GPU matters far more than the expensive workstation around it. No cloud. No subscriptions. Your hardware. Your data. This is a real, affordable experimental AI laboratory. And possibly a useful starting point for practical local AI applications. 🎄 Here’s an idea: build one for your kids as a New Year’s gift. Their very own computer for experimenting with language models, learning AI, and building things! And now imagine what models we might be running on just 8GB of VRAM with llama.cpp by New Year’s Eve. THE LOCAL AI REVOLUTION MIGHT ALREADY BE COLLECTING DUST IN YOUR GARAGE. 🚀

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·1h

The last time I played video games was in 2023, when I played Call of Duty. Before that, I probably hadn’t played anything for 10 or maybe even 20 years. I honestly can’t remember. I recently thought about playing it again, but this guy talked me out of it. He saved me a ton of time! 😂

GTA 6 是游戏史上最让人失望的作品。 我等了13年。我玩了。我想退钱。 下面是那些拿了赞助的人不会告诉你的真相 👇 现在是2026年,Rockstar 卖给你的是一款30帧的游戏。 没有60帧模式。一个都没有。在一台500美元的主机上,普通版 PS5 连30帧都稳不住。街上人一多就掉帧。开快车的感觉就像在泥里拖购物车。 然后是剧情。 几个小时都在从A点开到B点,听两个角色进行无聊的对话。你得到的不是抢劫,而是杂活。烤汉堡。迷你高尔夫。陪狗玩丢球。我等了13年不是为了来遛狗的。 Jason 和 Lucia 毫无个性。没有一个角色你下周还会记得。GTA 5 有 Trevor。GTA 6 谁都没有。 世界很美,但完全空洞。几百万个多边形,却没有一件值得做的事。这是一个70美元的屏保。 可它居然在 Metacritic 拿了99分。 评测媒体提前拿到游戏,夸了画面。然后又悄悄承认剧情缓慢重复、帧数卡在30。回去再读一遍那些评测,全都写在里面。 GTA 5 是13年前的游戏,至今仍然更好玩。3个角色。真正的抢劫。真正的混乱。真正的乐趣。 我的评分:2/10。 画面给2分。其他全部0分。 别信炒作。回去重玩 GTA 5 吧。 只有我一个人觉得被骗了吗?👇 关注我,看没人花钱买的真实评测 🔔

0:28
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·2h

A quick progress update from my local AI experiments. Earlier today, I went down to the basement and found the lights on for absolutely no reason. I simply asked my AI agent, out loud, in natural language: “What’s going on? Figure out why this happened and make sure it doesn’t happen again.” No technical instructions, no commands, no configuration details. Just a normal conversation. And here’s the agent’s report.

Post attachmentPost attachmentPost attachmentPost attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·3h

For the few of you who actually made it to the end of this thread, let’s have a laugh together. 😂 The illustrations show my dream home in its ideal form. Reality? Well… let’s just say that everything I’ve described sometimes looks downright comical in real life. 😂 But there’s one important detail: everything technical I’ve described has actually been built and really works. And it will work even better over time! These aren’t fantasies or concepts — they’re real systems that I’m gradually integrating into one complete setup. One day, I’ll gather the courage to publish actual photos of this engineering masterpiece. Until then, please enjoy the AI-generated version of my reality. 🤣

Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·3h

Meanwhile, I gave my local AI agent another practical job: securing the basement where the servers now live. I plugged in a USB camera, wired up some lights and relays, and left all the software configuration to the agent. Now motion automatically triggers lighting and video recording, with footage sent to external storage. Even if someone steals the servers, the evidence survives. Dear burglars, please wear masks. 😂🥷 Although getting inside won’t be easy. With ChatGPT’s help, I finally found a suitable lock for our armored basement door — something I’d been unable to find for years. I’m also making the house increasingly energy-independent. The solar power system already has a 5 kWh battery connected. Another 15 kWh of backup batteries are waiting to be connected. Literally just a couple of wires left! And I’ve already built triple internet redundancy: 🛰️ Starlink — satellite internet. 📡 5G — via a fixed external antenna. ☎️ ADSL — good old copper telephone wires, our third backup connection. Unfortunately, fiber still hasn’t reached the outskirts of our little Italian village. So my increasingly intelligent home is becoming more energy-independent, connected, resilient, and autonomous. The next step is bringing everything together under local AI control — including reverse engineering devices, building custom integrations, and letting agents develop the software themselves. Fewer benchmarks, more real-world local AI. 😁

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·3h

We slowed down our AI benchmarks for a couple of days. Why? We moved the servers to the basement… and suddenly the house got cold. 😂 Turns out, servers were part of our heating system! So I installed two heat pumps in the bedrooms, worked on our solar power infrastructure, and let a local AI agent handle some household automation. The bigger plan? Replace gas heating with modern heat pumps, integrate everything into a locally controlled smart home, and let AI optimize heating zone by zone. We’ll measure actual energy efficiency, log everything, and publish the results. This project is secondary for now, but it’s definitely happening. Meanwhile, we’re also experimenting with reverse engineering, connecting incompatible devices, and letting AI agents write custom software. So we haven’t really stopped testing local AI. We’ve just temporarily switched from tokens per second to degrees Celsius. 😂

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·8h

BONUS 6/5 — My actual hardware downgrade: DDR5 → DDR4 Here’s the configuration I’m moving my entire home and business infrastructure to. BEFORE — CLARYEL01: * CPU: Intel Core Ultra 7 270K Plus, 24 cores * Motherboard: ASUS Pro Q870M-C * RAM: 64 GB DDR5 * Storage: 4× Lexar NM1090 PRO 2 TB, PCIe 5.0 * RAID 0: 30–35 GB/s read throughput * Additional SSD: Samsung 990 PRO 2 TB * GPU: none AFTER — P1005 (DDR4): * CPU: Intel Xeon E5-2690 v4, 14 cores / 28 threads * Motherboard: ASUS Z10PA-U8 * RAM: 64 GB DDR4 ECC * GPU: NVIDIA RTX 3050 8 GB * SSD: Patriot Viper VP4300L 1 TB NVMe * SAS controller: LSI SAS3008 12G * Storage: 2×20 TB SAS HDD, RAID 1 * OS: Ubuntu 24.04 LTS WHAT’S MOVING? Around 40 Docker containers and supporting services: Home Assistant, Zigbee, Mosquitto MQTT, ERPNext, Odoo, OpenProject, PostgreSQL, MariaDB, MongoDB, Qdrant, Meilisearch, SeaweedFS, Valkey, Redis, LibreChat, LiteLLM, Activepieces, Ollama, ClamAV, agent infrastructure, knowledge APIs, integrations, monitoring, document storage, backups, and business automation. Plus Hermes Agent, with 40 scheduled jobs. The primary AI inference engine remains on a separate machine. I’m moving from a modern 24-core DDR5 platform with extremely fast PCIe 5.0 storage to an older 14-core Xeon with DDR4 and a modest 8 GB GPU. This isn’t about saving money. I already own the hardware. I want to find out whether a €1,000-class local AI platform can reliably handle my actual home and business workloads. And I’m going to find out by using it myself, every single day.

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·8h

5/5. I want to use what I’m going to sell Today, I’m migrating all my main services, databases, Home Assistant, Docker containers, RAG, accounting, and business workflows to DDR4. I’ve published the hardware configuration before, but I’ll share it again separately. The only significant difference from the entry-level customer system is my relatively expensive SAS controller and two 20 TB HDDs in RAID 1 for data storage. My four-drive PCIe 5.0 RAID 0 array, delivering 30–35 GB/s, will be gone. And I’m genuinely curious whether I’ll even notice the difference in everyday work. I don’t want to sell someone a €1,000 server while continuing to use an expensive machine myself because the cheaper one supposedly isn’t good enough. I want to live and work on the same class of hardware I’m offering my customers. If something breaks, I’ll be the first to experience it. If everything works beautifully, that’s far more convincing evidence than any synthetic benchmark. So at the end of 2026, I’m voluntarily moving from DDR5 to DDR4. Not to save money. To test my business idea in real life.

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·8h

4/5. I’m no longer interested in just tokens per second I’ve conducted countless tests measuring generation speed, prefill, long contexts, power consumption, and memory utilization. But customers aren’t buying benchmark records. They need document processing, accounting, search across their own data, automation, AI agents, and dozens of other processes to work reliably. Preferably simultaneously, without constant maintenance. So I’ve decided to move beyond laboratory benchmarks and test these systems under real production workloads. In the coming weeks, I plan to start offering local AI servers for small businesses, starting at €1,000. I’ve chosen DDR4 as the foundation of the entry-level configuration. There will also be more powerful options, for example around €2,500, supporting faster models, more complex workloads, and more simultaneous users. But the €1,000 system must be genuinely useful, not merely capable of launching an AI model. That’s why I’m moving my entire home and business infrastructure onto this class of hardware.

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·8h

3/5. Why did I choose DDR5 in the first place? When I started developing local AI servers for small businesses in winter 2026, the situation was completely different. Local models were larger, less capable, and much slower. GPUs were expensive, and building a practical system without one seemed like a reasonable way to reduce costs. My goal was a €1,500–2,000 system capable of running entirely without a GPU. Small local models would handle documents, automation, filtering information before sending requests to cloud AI, and other relatively simple tasks. Even CPU inference at around 15 tokens per second was acceptable for certain workloads. Customers could always add a GPU later. But everything changed in September and October. Strata arrived, opening new possibilities for distributing model workloads across VRAM, system RAM, and SSD storage. Meanwhile, RAM and GPU prices increased significantly. I started testing DDR4, including relatively old hardware. Then I went even further and tested DDR3. The results confirmed something important: for many practical AI workloads, an expensive modern platform simply isn’t necessary.

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·8h

2/5. Eight months on DDR5 Since winter 2026, my home and business infrastructure has been running on a modern Intel-based server with 64 GB of fast DDR5 memory and four PCIe 5.0 NVMe SSDs. Each drive delivers up to 14 GB/s, and together in RAID 0, they achieve around 30–35 GB/s of actual read throughput. The server runs Home Assistant, AI agents, around 40 Docker containers, ERP, accounting, databases, a vector database for RAG, document storage, backups, and numerous other services. Essentially, it’s the digital brain of my home and business. Only the AI model runs on another machine because this server has no GPU. Everything works beautifully. Except for one thing: despite all these workloads, the CPU is practically idle. Utilization usually doesn’t even reach 5%. An enormous amount of computing power simply goes unused.

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·8h

1/5. Why am I switching from DDR5 to DDR4 at the end of 2026? And it’s not about money. @coldniko Strata changed my understanding of what hardware is actually needed to run local AI. So much so that I’m now moving my entire home and business infrastructure from a modern DDR5 server with ultra-fast PCIe 5.0 NVMe drives to a much older DDR4 platform. I already own both DDR5 and DDR4 hardware. I’m not trying to save money. My current server works beautifully, and I’m completely satisfied with it. So why am I voluntarily downgrading? Here’s the story.

Post attachment
Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·15h

I completely agree — give it a try! I’d just add that Q4 is great, but Q3 is already good enough for many practical tasks. For about three weeks now, maybe even longer, I’ve been using just one model locally: Qwen 3.8 Next IQ3_S. And I’ve been running it on all kinds of hardware, from €500 systems to €10,000 machines.

dear 8gb, 12gb, 16gb and 24gb vram owners, i know people in the local ai space might discourage you, saying don't use q4 and q4 kv, it trades away quality and sucks and all that, but hear me out loud, i'm going to be very clear the ones saying that own datacenter grade chips and can afford full precision, and as a small gpu owner you don't have a choice i'm out here in the field running everything from 8gb cards to 256gb of unified memory, and i know what i'm saying don't get sold, try it and find out, it's not that bad at all

Vik | Practical AI & Tech avatar
Vik | Practical AI & Tech@uctive·23h

He’s coming back. This is a huge sign for us. And I don’t think the board of directors — those nobodies nobody has ever heard of — will dare stand in his way.

Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen

Jensen Huang avatar
⟳ Vik | Practical AI & Tech retweeted
Jensen Huang@JensenHuang·Nov 17

Over the past week, I’ve been overwhelmed by the love and support of friends, industry colleagues, and people I've never met. Thank you to everyone who has shown kindness to my family and me. It means the world to me. I came to the United States 53 years ago, and becoming an American citizen is one of the greatest gifts and proudest moments of my life. NVIDIA has been my life’s work. Building it into the company it is today alongside my colleagues has been a privilege and an honor. Nowhere but in America could a company like NVIDIA be possible. I love our country, and I love NVIDIA. I did not live up to the standards expected of everyone at NVIDIA, and I have made serious mistakes in my personal life. I created problems for the Board that I deeply regret. I own my mistakes completely, and I am committed to cooperating with the Board’s independent investigation. Since my departure, I’ve seen an outpouring of calls from NVIDIA’s employees, investors, business partners, and customers for me to return as CEO. Many continue to reach out to me directly as the company navigates complex technology, market, competitive, and geopolitical dynamics. NVIDIA is a once-in-a-generation company. If the Board of Directors decides that my return is in the best interest of our shareholders, our employees, the company, and America, I would be honored to answer these calls and return as CEO. Thank you again to everyone who has shared their good wishes. I am profoundly humbled by your support. To my family: I love you, and I will spend the rest of my life being worthy of yours. Jensen

Raffaele Mancini avatar
⟳ Vik | Practical AI & Tech retweeted
Raffaele Mancini@RaffoMancio·Nov 17

It started as a weekend project to learn local inference. Now, it's a growing community. Thank you for 1,000 GitHub stars and 100+ forks! A special thanks to all the contributors and everyone who spent time testing and sharing ideas. This is just the beginning!

Post attachment