In preparation for the upcoming [Android lockdown](https://keepandroidopen.org/) I've started vibe-coding some PWAs for apps that I like so I can have a webby version of them with no ads and no Google/iOS lock-in. I like this, it feels like a good direction.
This. Haven't seen this in the news in years. A few years ago (2022?), they got some $x00 million in investment money. They already had IPFS/IPNS at that point so it was some deep push into Filecoin. But it felt odd at the time. Bitcoin was starting to jump the shark at that point and none of the smarter cryptos had truly found a market fit either.
Unfortunately, camera white balance seems to diminish the effect compared to actually being there. But it's something. Now imagine that the weather got 10 deg (F) colder, the birds are chirping because they think it's bedtime and your circadian rhythm is nagging you in the back of your head: "this ain't right"
Django makes simple things easy and complex things hard. Django breaks the zen of Python with much "magic". I like DjangoORM but prefer FastAPI for the webby bits.
Discontinuing a product breaks the cost-comparision chain. It makes it easier to increase prices or reduce quality without direct backlash. Releasing new products also makes for marketing announcements that work like advertisement (or is advertisement directly).
> The 1.8-bit (UD-TQ1_0) quant will run on a single 24GB GPU if you offload all MoE layers to system RAM (or a fast SSD). With ~256GB RAM, expect ~10 tokens/s. The full Kimi K2.5 model is 630GB and typically requires at least 4× H200 GPUs.
If the model fits, you will get >40 tokens/s when using a B200.
To run the model in near full precision, you can use the 4-bit or 5-bit quants. You can use any higher just to be safe.
For strong performance, aim for >240GB of unified memory (or combined RAM+VRAM) to reach 10+ tokens/s. If you’re below that, it'll work but speed will drop (llama.cpp can still run via mmap/disk offload) and may fall from ~10 tokens/s to <2 token/s.
We recommend UD-Q2_K_XL (375GB) as a good size/quality balance. Best rule of thumb: RAM+VRAM ≈ the quant size; otherwise it’ll still work, just slower due to offloading.
I'm running the Q4_K_M quant on a xeon with 7x A4000s and I'm getting about 8 tok/s with small context (16k). I need to do more tuning, I think I can get more out of it, but it's never gonna be fast on this suboptimal machine.
you can add 1 more GPU so you can take advantage of tensor parallel. I get the same speed with 5 3090's with most of the model on 2400mhz ddr4 ram, 8.5tk almost constant. I don't really do agents but chat, and it holds up to 64k.
That is a very good point and I would love to do it, but I built this machine in a desktop case and the motherboard has seven slots. I did a custom water cooling manifold just to make it work with all the cards.
I'm trying to figure out how to add another card on a riser hanging off a slimsas port, or maybe I could turn the bottom slot into two vertical slots.. the case (fractal meshify 2 xl) has room for a vertical mounted card that wouldn't interfere with the others, but I'd need to make a custom riser with two slots on it to make it work. I dunno, it's possible!
I also have an RTX Pro 6000 Blackwell and an RTX 5000 Ada.. I'd be better off pulling all the A7000s and throwing both of those cards in this machine, but then I wouldn't have anything for my desktop. Decisions, decisions!
reply