Cool project. A team at work was building something similar to internal use.
I'm curious how this compares to just using Claude Code directly and giving it a dump of the agent traces? It seems like Claude could probably do some of the same diagnostics / trace grouping to identify failure patterns. Why use a custom harness?
Yeah, fair question. For a small number of traces just dumping them into Claude Code can work well.
However, once you're at production scale the problem changes. You can't always fit 10,000+ traces in Claude Code and still have it be effective especially when the relevant pattern of agent failures may only become apparent when you pass that many in. That's where the RLM based methodology helps. HALO recursively decomposes the trace data into smaller investigations, analyzes those sub-pieces, and then synthesizes those up to determine the recurring harness-level failure modes better than Claude Code or Codex ever could at a large scale.
I'm very curious to see a benchmark for this - have toyed with the idea myself but haven't put in the hard work to test these hypothesis on extracting learning signal from deep-agent traces.
Hi all, we built a super easy way to train small language models from your production data - install a gateway to save your request/response data from a frontier provider, and use the traces to train a open source model that can be hosted privately on your machine.
We've seen some pretty impressive results already, including training an HTML->JSON model that competes with frontier quality at 5% the cost called [schematron](https://huggingface.co/inference-net/Schematron-8B)
It's great for data extraction, chat, search, image classification, sentiment analysis, and more.
Trainings are free for the next 30 days as we test and experiment. We're also going to make it possible to launch and manage training runs on your own GPUs, so everything is private.
We'll release the full data explorer soon, with more info.
At the core of this project is a structured-extraction task using a custom Qwen 14B model, which we distilled from larger closed-source models. We needed a model we could run at scale on https://devnet.inference.net, which is comprised mostly of idle consumer-grade NVIDIA devices.
Embeddings were generated using SPECTER2, a transformer model from AllenAI specifically designed for scientific documents. The model processes each paper's title, executive summary, and research context to generate 768-dimensional embeddings optimized for semantic search over scientific literature.
The visualization uses UMAP to reduce the 768D embeddings to 3D coordinates, preserving local and global structure. K-Means clustering groups papers into ~100 clusters based on semantic similarity in the embedding space. Cluster labels are automatically generated using TF-IDF analysis of paper fields and key takeaways, identifying the most distinctive terms for each cluster.
There is growing emphasis on efficiency as more companies adopt and scale with LLMs in their products.
Developers might be fine paying GPT-5-Super-AGI-Thinking-Max prices to use the very best models in Cursors, but (despite what some may think about Silicon Valley), businesses do care about efficiency.
And if you can fine-tune an 8b-parameter Llama model on GPT-5 data in < 48 hours and save $100k/mo, you're going to take that opportunity.
OP here. I wanted a dead-simple way to quickly generate CLI commands without the overhead of Claude Code or Cursor, so I built it in an afternoon.
The project uses some zsh magic to allow for quick editing of the model's response before running the command. Commands show up in your shell history, just like any other command.
The project is intended to be very simple and hackable -- the whole project is one TypeScript file.
Hate those ads. "Inference isn't just a buzzword". Who thought it was? (No comment on whether the linked post is a useful tool, I haven't played with it enough to know)
This is cool! I don’t see many people doing write ups on their tech stack as much any more. It’s nice to see the inside of a production-grade app like this.
I’m curious, why command+r for the model? What benefits does it have over other SOTA models?
It’s unlikely an individual would need this much capacity. Folks who need tokens at this level are apps with lots of users that don’t have their own GPUs. Think character.ai type apps.
Kuzco, Inc. (https://kuzco.xyz) | Full-stack SWEs and MLE | Full-time | San Francisco, CA
We're building a serverless LLM inference network that makes use of underutilized capacity from GPU data centers. Our product is a scheduler for running LLM inference workloads on GPUs located all over the world. We currently have over 6,000 GPUs on our network, and are growing quickly.
Things we need help with:
* Improve core scheduling algorithms
* Optimize vLLM inference runtime
* Improve logging and observability stack
* Building our user-facing dashboard and APIs
* New products
We're well-funded and have a clear path to profitability. We're currently a four person team of staff-level engineers and looking to add two more engineers to our team. Our office is near the Ferry Building in downtown San Francisco. We pay well and offer significant equity.
I'm curious how this compares to just using Claude Code directly and giving it a dump of the agent traces? It seems like Claude could probably do some of the same diagnostics / trace grouping to identify failure patterns. Why use a custom harness?