It absolutely does, and will be used and enforced by those who want to protect their IP. Within the existing frameworks.
Let's no go down the rabbit hole of "since it's easier now, we need new laws..." to make new laws and regulations about what AI coding can and can't do when the existing copyright laws work perfectly well.
edit: Want to decomp/recomp a game and modify it and change it so you have more fun? Go right ahead. Have a blast. Want to then publish that? You don't have the right to republish their materials without permission. If you do, the owner can run the math of how much it damages them to decide if they pursue it. We don't need the AI generator to police that for us.
Funny ... once upon a time, I can remember the US getting hyperventilated about China and their lack of concern about copyright. I guess, not so much now.
Disclaimer: I'm in neither camp, but just making a note of yet another hypocrisy. I'm into my 17th volume.
Copyright is not really criminal law, so companies have to sue people and it's relatively easy to be anonymous enough on the internet for that threat model, and you can't successfully sue someone if you don't know who they are.
Yeah, piracy is doing better than ever. FMHY shows the entire ecosystem—it's huge: https://fmhy.net/
This will be taken down though if Rockstar tries to because dot coms are not DMCA resistant.
At the end of the day, copyright is trying to constrain information which is not easy on the internet.
Obviously there are a variety of sizes and configurations, but if you assumed 3.5MWh per 20ft container, that's 5771 containers, and if you assumed a PCS container per two battery container, that's 8656 containers, just for an idea of the scale.
I've always wondered why they don't stack them in deployment -- the containers are obviously fine with it! -- but I guess land is cheap enough not to bother.
Horizontally they need plenty of room for airflow, as each container has some HVAC-or-whatever module on the side. But vertically...
Can share a good way of running Qwen 3.8 27B (or similar) on 16GB VRAM? I've been trying but I end up with either a heavily quantized model or a small context or a slow speed.
Dense models are not well suited to this. Depends on your system, but most inference engines let you put the KV cache on the host. They should let you use "Q8" KV cache format as well which takes half the storage of bf16, the default format. You essentially are stuck at having to fit the entire dense model into VRAM because it must compute across the entire model every token. Without it being in VRAM it is slow. MoE models such as Qwen 35B with 3B active parameters are much friendlier to limited VRAM systems. This is why you see systems with 16GB of VRAM running something like DeepSeek Flash if they have a bit of spare system RAM. Usually not at any good token rate, but tolerable.
Dense models also don't like quantization as much and are less tolerant of it. There are some very interesting smaller models like https://huggingface.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4... Dory that I would check out. You need system RAM or to stream a larger MoE model from disk and it won't be fast. You need to specialize the models some to get quality results if you have very limited system resources.
I asked Claude to come up with 7 problems to solve that were not part of any benchmarks. Besides speed, I asked it to put together a scoring that would reflect models that perform well on the 7 problems and in a reasonable time-frame (RWS, the definition is in the artifact). I valued models that produced good answers in a reasonable time better than models that had very high throughput but kept spitting useless stuff. Overall, IIRC most of them still had a >30% chance of producing wrong/non-working replies.
I then tested Strata ( https://github.com/niko1221/strata ) and the results are not even close. You can basically ignore everything in that table and use Strata instead, you can use bigger/less-quantized models and have way better results. Basically any otherwise-decent model degrades noticeably at Q < 5, and KV cache at Q4 also contributes a lot.
Still, a 20$/mo Claude subscription is way cheaper/more efficient if you code 2-3h in the evening than running stuff locally, unless you really have hardware than can hold models that get most things right at the first try. You can iterate way faster and cover a lot more ground in 2-3h using a Claude subscription, or OpenRouter if you prefer to change models. You get faster replies that are 99% correct at the first try, whereas using the local model you'd be spending 30% of your time fixing things that don't work and prompting it to do better.
Nowadays I use the local model to delegate tasks that would burn a lot of tokens, like sifting through big data archives for indexing and such.
QFN Q4 K XL Unsloth is basically identical to the bf16 model. It’s KL divergence is very very small. It is competitive with the full model. Also I benchmarked iq3 xxs on Strata for 3 days against deepswe and it scored better than the published benchnarks for the model. If the gsq rco quant prettain tasks fit well it is a great model quant. My data shows it takes 20% more tokens basically than q4 xl. A4 xl is within noise on all my repeated task benchmarks over the public model serving and my own runs of q8. I published the deeswe results. You have to be very specific about model choice and know these models and their quirks.
This is not about if a frontier AI provider is cheaper. It is about having control over your data and use of AI.
> QFN Q4 K XL Unsloth is basically identical to the bf16 model.
Yes, but like original poster I have 16GB of VRAM and 64GB of RAM, I was referring to non-Strata benchmarks :)
I did quickly run a couple things on Strata but haven't gotten around yet to benchmark bigger models/customize them with Strata, but will do rather soon. So far Strata/Swift IQ2_XS performed really well compared to non-Strata/Qwen3.8 quantized variants.
> This is not about if a frontier AI provider is cheaper. It is about having control over your data and use of AI.
Yep, totally get it, and to be honest getting to run ablated models sometimes can also be useful.
> Also YMMV in “faster” — I have less than 5K into my home AI lab shenanigans and it’s pretty fast :)
My rig is... ~1.5k, maybe 3k if we count the CPU and GPU upgrade since 2019 :-D I really wanted to grab one of the latest mini-PCs (Halo, Spark, and other minor assemblers) for local setup, but to be honest I can't justify a 6k+ purchase for me and my wife alone. Perhaps in 3 years these mini-PCs will get to the ~2k ballpark (perhaps used), and although older by then, I might be able to run things 24/7 locally as well.
With 32 GB more of RAM you can run ISTA GSQ RCO IQ3 XXS fast. You can make it work now but it is tight. You can stream experts from disk with builds now. You will get an okay token rate. Try it out if you have some disk space. It is a very good local model. I benched it at that quant. Sonnet 5 class at coding benchmarks.
Get a couple or V100S GPU. Figure out cooling. They will run it /fast/ with the right engine. :)
The easiest thing to do (takes 5 minutes) is to use LMStudio and then search for the model inside LMStudios UI.
It will show you on the model selection IF a model fits and if you use a "Qwen 3.8-35B-A3B" type of model, it should just work out of the box. A3B stands for Active which means you need a lot less fast ram.
You will run into issues fast though: If you have big tasks for it to do, your pc/mac will get warm very fast and noisy and you will cosume relevant more energy.
My setup is a Ryzen 3600, 90GB DDR4 allocated to a VM (don’t even really need that much, just had it available, special thanks to the year 2024), and an RX 9060XT 16GB ($500 GPU).
I get around 40-60 tok/s depending on the quantization level I choose.
As you can see, the CPU is nothing special, don’t even need DDR5.
Yes it runs on all levels of hardware - with the latest Strata my 5090 and 64GB DDR5 gets 200-240 tok/sec decode, 6500 tok/sec prefill. Around $0.09/M tokens in power usage (have excess solar capacity).
Jealous, lol. But I did spend a grand total of $500 for my setup. The rest of the server was built purchased with the use case of running game servers.
AI harnesses like Claude will set it all up for you, you can just say "I want to run a localllm on this machine" and it will find the best one for your hardware and even help you fine tune it etc
You will end up with a sub-par result. The space for small models is more specialized and specific and you should optimize to the specific task and or tasks you want to do. It will get some local model working, but there are a lot of interesting small models in the 4-10B space to look at and play with for your specific tasks. If you expect a full agentic coding agent it's just not going to happen without a larger model like Qwen 35B and ... that is not that great by today's standards, even Muse Glimmer is better.
After arguing and back and forth with Claude (not straight away), it found a kv streaming fork of llama https://github.com/achyudh/llama.cpp and an interesting ascii only model Qwen3.8-27B-ASCII-Condensed-UD-IQ4_XS.gguf . It's running, i'll see how it goes.
It's cool that there's an in-browser skin classifier...but it says "It is not a state-of-the-art model". The obvious question is, is there a free in browser state-of-the-art model?
Would be difficult to fit in a browser as state of the art requires multiple vision transformers. Also it would not be sensible to provide access to models without human oversight as at this point in time all models have the potential to miss skin cancer.
Makes sense. My RTX 5090 can locally take a text prompt, generate a 2D image from that, feed it into a 3D generation model, rig up the animations, and make me assets that are better than a lot of video games on the market. Not Cyberpunk 2077 level m, sure, but pretty good and this is just what I can clobber together in my free time from open models.
I'm a dev not a video or 3D guy, but I'm looking at people I used to know and also some youtube doom-scrolling, the blender MCP, cinema 4D MCP, and seeing that it's a big impact
Yep, and you can prompt something like "the function was a decompiled version, translate it into a readable version", if you don't like reading [probably] ghidra output.
Solar is great. I'm highly skeptical that a battery + inverter (w/o solar), using time-of-use for payback would ever make pure financial sense.
In fact, if it DID make financial sense, wouldn't companies already be buying up empty/cheap space and then turning a profit after the payback period? Why aren't they?
There are other reasons to do this, but purely on the financial side, I'd want to see the numbers on this one. Again, this isn't about solar, the claim was about battery + inverter + time-of-use paying for itself.
Grid scale BESS (battery energy storage system) in the UK is growing fast, because it makes financial sense. There's one that just expanded from 100MWh/100Mw to 475MWh/350MW: https://www.pacificgreen.com/energy-parks/energy-park-case-s... . It's easy to run the numbers, easier to make a payback calculator with AI. Take a look at the Anker E5000 / Ecoflow Stream 5000, and run the numbers yourself, I'd be interested to see what the payback would be for you.
> I'm highly skeptical that a battery + inverter (w/o solar), using time-of-use for payback would ever make pure financial sense.
Ever? Batteries are 10x cheaper today than they were ~a decade ago. If that trend continues for even a little longer, they'll be absolute no-brainer like solar is today within the next 5 years.
2026 seems to be roughly the break-even point. And whether it's worth it depends on where you live (greatly affects the cost of both installing a system, and buying energy from the grid).
> In fact, if it DID make financial sense, wouldn't companies already be buying up empty/cheap space and then turning a profit after the payback period? Why aren't they?
Home batteries are much quicker to pay off than solar at the moment. This is largely due to being able to do strong arbitrage between import and export prices at different times of day, and some high guaranteed consumer export tariffs.
There’s no guarantee things will stay that way though, not least because the more storage added to the grid, the more those arbitrage opportunities get soaked up.
Other people have made the point, but: the ability of companies to deploy batteries is limited by planning permission (ie the indigenous British NIMBY). Home solar and batteries are planning exempt, so they can be deployed more quickly and cheaply.
Yeah, I bought an inverter from eBay, and a battery from AliExpress. Also signed up to axle energy to reduce payback period. Most people would need an electrician to connect it though. If you just asked an electrician for a new circuit with a simple route to an isolator, it wouldn't be expensive.
I import more than I export in December and January, but for every other month of the year it’s the reverse by some margin. My annual electricity bill is net negative, and the installation cost will be paid off in 7 years. The numbers would be even better in, say, Spain, but they are perfectly adequate here, despite suboptimal latitude.
Incidentally, the grid only knows about my exports. Self-consumption manifests as demand reduction, so UK solar production and electricity consumption is probably higher than aggregate figures suggest.
It's helped a lot by the fact that solar panels have become very cheap indeed, so the fact that we don't get as much sun as other places doesn't matter so much - it will still pay off, as long as you can actually use that energy effectively (which may require batteries or a decent export payment rate).
True, but if you get cheap power from solar for the rest of the year, and charge the battery from the grid off peak when there's no solar to charge, you should end up paying less than just the standard rate.
reply