colibri runs the 744B GLM-5.2 model on a 25 GB laptop by streaming it from disk, at 10 to 20 seconds per token. Here's what it really does, why its secure-by-default server matters for Australian businesses keeping client data in-house, and when local AI is practical.
colibri, an open source engine by Italian developer Vincenzo Fornaro, runs Z.ai's 744-billion-parameter GLM-5.2 model on a laptop with 25 GB of RAM and no graphics card. It works by streaming most of the model from disk. It's also very slow: 10 to 20 seconds per token on that laptop. Here's what it actually does, why its secure-by-default server matters for Australian businesses that want client data kept in-house, and when local AI like this is practical.
A TikTok with more than 400,000 views made the rounds last month claiming someone had run a frontier-sized AI model on a laptop, proving you don't need NVIDIA anymore. Half of that is true.
The project is real. It's called colibri (Italian for hummingbird), it's on GitHub under the Apache 2.0 licence, and as of 2 November 2026 it has just over 46,000 stars. It really does run GLM-5.2 on a 25 GB laptop with no GPU.
What the video leaves out is speed, and speed is the whole story for a business.
We're an AI agency in Sydney. We build private AI systems for firms that can't send client files to an offshore chatbot. So we read the repo, the code and the benchmarks to answer one question: is this useful to an Australian business, or just impressive? Our answer: it's a genuinely useful idea with a narrow set of business uses today, and one of the better examples we've seen of an AI tool that's secure by default.
We haven't deployed colibri for a client. This is our read of the project, not a case study.
Large models like GLM-5.2 use a design called Mixture-of-Experts (MoE). The model is huge, but for each word-piece (token) it only uses a small slice of itself.
According to Fornaro's Show HN post and the README:
So instead of needing hundreds of gigabytes of RAM or a rack of GPUs, you need a modest amount of RAM and a big, fast SSD. Fornaro describes it in an interview with AITalk as a huge warehouse and a small workbench: you don't put the whole warehouse on the table, you bring over what you need when you need it.
The core engine is written in C with no dependencies, plus a small Python server for the API. A one-step setup script checks your hardware, recommends a model that fits, builds the engine, downloads the model and starts a local server.
Here are GLM-5.2's decode speeds from the colibri README, measured by the author and by community testers on their own machines:
Some quick arithmetic to make that concrete. A token is roughly three quarters of an English word, so a 500-token answer is a few paragraphs. Counting generation time alone (reading the prompt adds more):
That laptop figure is not chat. It's a proof that the thing works at all, and Fornaro says as much. In the AITalk interview he agrees with an outside analysis calling early colibri "a serious proof of architecture, not yet a drop-in production server", and says he wouldn't promise 20 or 30 tokens per second on a laptop today.
Two more caveats. These speeds are self-reported by the author and community testers, and they depend heavily on your SSD, because streaming speed is your token speed. Your machine may be faster or slower.
No. The fastest GLM-5.2 result in the README is the six-GPU machine, and colibri itself supports CUDA, Vulkan and Apple's Metal. GPUs still make it faster.
What colibri proves is narrower and still interesting: a very large model can run on hardware you already own, slowly. For a business, that turns "can we run a frontier-class model in-house?" from a hard no into "yes, if you can live with the speed, or spend on a bigger box."
GLM-5.2 is an open-weight model released by Z.ai (formerly Zhipu AI) under the MIT licence, which allows commercial use. Z.ai's own benchmark table, published on its blog and Hugging Face model card, is mixed:
These are vendor-reported numbers. Treat them as a sign that GLM-5.2 is a strong open model, not proof that it beats the paid ones. And whatever quality the model has, colibri runs it compressed to int4, so test it on your own work before trusting it.
The easy mistake with local AI is a server that anyone on your network can talk to, with no password. Whoever can reach it can use it, and feed it whatever they like.
colibri's server, which speaks the same API format as OpenAI so existing tools can point at it, is careful here. We read the server code (c/openai_server.py) today. Three defaults stand out:
That's the right way round. The safe setting is the default, and the risky one takes deliberate effort. It's the same standard we hold our own builds to, and the same thing we look for in open source AI security tools.
One clarification. "Local" means generation happens on your machine. The setup still downloads the engine from GitHub and the model from Hugging Face. Once that's done, prompts and answers don't need to leave the box.
The reason a Sydney accounting practice or a Melbourne law firm would care about any of this is privacy, not cost.
When staff paste client information into a cloud AI tool, that information goes to the provider, usually overseas. Under the Privacy Act 1988, sending personal information offshore brings in Australian Privacy Principle 8 (cross-border disclosure), and you generally stay accountable for what the overseas recipient does with it. We cover this in more depth in our Privacy Act and AI guide.
Running the model on your own hardware takes the offshore AI provider out of that picture. That's a real benefit. It doesn't make you compliant on its own. You still have to secure the machine, control who can use it, check the model's output for accuracy, and handle the data properly. A badly run server in your office is still a badly run server.
Where colibri-style local inference fits today:
Where it doesn't fit:
The most useful thing in the colibri README for a normal business isn't GLM-5.2. It's the smaller models the same project runs.
The README lists Qwen3.6-35B-A3B as a 23 GB download that measured 6.0 tokens a second on an 8-core desktop CPU, and 9.9 tokens a second using the integrated GPU. That's usable for a single person reading along. A 500-token answer is under a minute and a half on the CPU figure.
A smaller model won't match GLM-5.2 on hard reasoning, but plenty of business tasks (summarising, drafting, classifying, pulling fields out of documents) don't need a frontier model. Pairing a small local model with document tools is often where private AI starts to pay its way, a bit like the self-hosted setups in our open source OCR tools shortlist.
If you or your IT person want to test colibri, here's the list we'd work through:
colibri is a good piece of engineering, honestly presented by its author. The secure-by-default server is something every local AI tool should copy.
For most Australian businesses, it's not a product you'd put in front of staff today. On a laptop it's a demonstration. On a well-specced desktop it's an overnight worker. The real value is what it shows: frontier-class open models can run on hardware you control, and that gap is closing.
If keeping client data onshore matters to you, the decision isn't "cloud AI or a laptop". There's a middle ground: smaller local models, or larger ones on proper hardware hosted in Australia. That's the kind of setup our private AI in Australia work is built around.
Can colibri really run a 744B model on a laptop without a GPU? Yes. The README documents GLM-5.2 running on a 12-core laptop with 25 GB of RAM and no GPU, at 0.05 to 0.1 tokens per second from a cold start. That's 10 to 20 seconds per token, far too slow for chat.
How fast is colibri on better hardware? The README lists 1.83 tokens per second on a 128 GB Ryzen AI Max+ 395 desktop running on the CPU alone, and 9.0 to 9.2 tokens per second on a machine with six RTX 5090 GPUs. These are self-reported, and your SSD speed makes a big difference.
Is colibri free for business use? colibri is Apache 2.0 and GLM-5.2's weights are MIT, both of which allow commercial use. Other models it runs have their own licences, and some, like Qwen-Image-2.1, are non-commercial. You still pay for hardware, disk and power.
Is it safe to run colibri on an office network? By default it only listens on the machine it's running on, refuses to open to the network without an API key, and guards against DNS-rebinding attacks. Keep those defaults, use a strong key if you share it, and treat the machine like any server holding client data.
Does running AI locally make my business Privacy Act compliant? No tool does that on its own. Keeping prompts and data on your own hardware avoids sending them to an offshore AI provider, which helps with cross-border disclosure under APP 8. You still need security, access control, accuracy checks and a clear policy. This is general information, not legal advice.
What should a small business try first? A smaller model, not the 744B one. In the colibri README, Qwen3.6-35B-A3B is a 23 GB download that measured about 6 tokens per second on an 8-core desktop CPU, which is workable for one person.
We build AI agents and automations for Australian businesses, with security as the default rather than an add-on. If you're weighing local AI against cloud tools, we'll help you work out what your data, your hardware and your budget actually support. See how we approach business automation and AI consulting.
Whether you're in the Sydney CBD, Parramatta, North Sydney or anywhere else in Australia, email info@ailabaustralia.com with the workflow you're thinking about, and we'll tell you honestly whether local AI fits it.
Sources: colibri on GitHub (README, benchmarks, quickstart, server code); Z.ai GLM-5.2 announcement; GLM-5.2 model card on Hugging Face; Fornaro's Show HN post; AITalk interview with Vincenzo Fornaro; OAIC: Australian Privacy Principles. Figures checked 2 November 2026.
Get expert guidance on implementing ai tools solutions for your Australian business
