The evidence should now point to the contrary, i.e. that the code was not AI-generated. There are many people on forums like this one saying that at their (big, although not-named) big companies they haven't touched actual code when programming for the last 6 months or so, and judging by what I encounter on my search for a job right now this is what the general "spirit" of the industry is.
Haven't you heard that the age of manual coding is over? You can check almost any comment section here and see people loudly proclaiming that no one writes code by hand anymore - the entire industry has gone to genAI coding.
I don't buy that narrative for the whole industry, but I have seen management at a lot of tech companies aggressively pushing LLMs for coding, and in the process they often expect increased delivery speed. That means generated code that has not been properly reviewed or designed is often being shipped and more frequent outages would not be a surprising result.
Up until about a week ago, I had no sympathy for people complaining about "AI writing style" in articles - and then all of a sudden it hit me. Kind of like unexpectedly developing an allergy to sawdust.
It feels pretty duplicitous that the staff technology writer for The New Yorker is publishing their work behind a paywall, but doesn't see the value in paying for great search results (via kagi).
You should do that. I am also summoning the motivation to start on my own. I guess the worse will be that I fail. Might as well fork sth more basic, or build on top of sth SDK/library like.
I don't know if you'll be interested, but there is an excellent book on the topic: Build an AI Agent (From Scratch), by Jungjun Hur and Younghee Song (Manning Publication).
By chapter four you have a minimal working codebase in python. Later chapters add out features like tool calling, planning, and reflection.
I am thinking about it but I think forking will miss the point. I'd rather do in a way when there is a prevalence of various types of agentic/harness SDKs for people to make their own apps (aka harnesses) from.
That’s what pi.dev exist for but I’m not touching it because like all of the other harnesses it’s built in JS and they’re all terrible when it comes to cpu and memory use. Maki is the only one I found the be the most efficient harness. It uses 1% cpu idle vs 20-30%.
A few bucks will realistically last you forever.
I also bought a DGX Spark at launch. I've learned a ton. but it's not usually my daily driver. I do run overnight tasks on it, like "read this book and write up notes on each chapter..."
In retrospect, given the dirt cheap prices of apis and or renting hardware, buying hardware for inference was likely a mistake (but not enough of a mistake to sell it now).
I think building a OpenAI compatible RAG-pipeline that also perhaps includes the anthropic sdk let's you have the best of both worlds. You can build it in a way that a local agent can escalate to a frontier model when needed and also substitute prompts data that you want to hide from being "public". For example. I think that is meant by harness here.
And with some smart feedback loops, you will see that for home assistant type tasks local AI can pretty quickly fulfil all requirements. For quick responses or voice analysis you might want to start with an external provider.
Otherwise, I think for complex home automation around 24-32gb vram are a realistic house number, but more experienced users will probably need less here.
People say that tools are called by the model. That isn't entirely correct. The caller calls the tools, the model just formulates the syntax for it. So the model part can be externalised without you needing to compromise you system fully (just to the degree the tools would allow anyway)
The author’s discussion of their personal use case starts on the seventh sentence in TFA. And frankly, it sounds like a pretty solid use case:
“Being able to run things locally and mess around with RL, quantisation and fine-tuning on a laptop would be sick. I already publish my own GGUF quants on Hugging Face, with over 160,000 downloads in the past month.”
reply