My understanding is that forks on GitHub share the same storage in the GH backend.
It's not unreasonable to imagine thousands of commits per second to the kernel, especially if you consider that bots might be watching LKML and applying patches to various forks.
> Handling thousands of git pushes per second implies thousands of issues, thousands of PRs, tens of thousands of comments, thousands of Actions runs, and dozens of releases. How's that part of their infra going to scale?
For each issue my agent might raise 5 independent PRs as experiments.
Further, the more changes there are, the more important it is that they don't conflict, which means smaller changes, which means even more changes.
Bots interleaving typos, single line whitespace fixes, single dependency bumps, etc.
There's not enough context for me (or other commenters) to really evaluate this, but every change has risk, and some projects aren't in a position to be accepting that risk.
A "real performance bottleneck that I observed during testing" is a little bit unclear whether it occurred outside of test conditions, or whether speeding up that particular bit improves anything anyone cares about.
> It's pleasant and I don't need to worry about how snarky I get when going over bad code.
Yes, reviewing an LLM code is at times infuriating, but at least I can be brief and direct, rather than spend 2x the effort trying to make sure it won't be interpreted as harsh.
I spend about 90% of at-computer time in a herdr-like tool.
It feels like playing factorio: whatever is manual needs to be automated away.
LLM writes the test. The test is crap. Update the skill. Ask an LLM to review your feedback on the last 5 PRs and write a skill for a review bot to look for similar problems and post. Now you only see PRs after they've gone through a few rounds of review.
The LLM has stopped picking up new issues. Ask an LLM to diagnose why. Ask an LLM to write a cron to regularly check if the LLM PR pipeline has halted and ping you. Get an LLM to try to diagnose it first.
I'm using a lot of LLMs! Ask an LLM to diagnose why. Write a tool to audit your usage, and produce a weekly report. Have it raise issue to reduce spending.
And like a base building game there are many competing facets, coming all at once.
Haha, I have been building my own software factory exactly like that. I consider it the ultimate factory game- but I sure as fuck would not want that to be my dayjob... Though I'm aware it will likely be that in a few years when the industries in my nation normalize around that.
No offense, but this feels like the most un-Factorio thing imaginable.
In factory building games, the systems in play can be understood and are fully deterministic. You're given a set of tools and a problem, and then craft together a solution. Your reward isn't just seeing the item you want on a conveyor belt, but looking at the factory you built ticking like clockwork.
What you're describing is beating out endless nondeterministic fires. It's like you were forced to play Factorio exclusively through an algorithm that designs incomprehensible factories that don't work sometimes, but designs them much faster than you ever could. So you click the magic button and see a mass of buildings appear in front of you. They may work perfectly fine, but if they don't, you need to use the same statistical factory estimator to build an in-game factory fixer mechanism and factory tester mechanism. And then a few updates later the factory fixer breaks so you need to deploy a factory fixer fixer, and a belt clean-up machine, and so on ad infinitum. To me this doesn't seem satisfying in the slightest.
I don't really see how it is as different as you say. Even if a system is stochastic, it can still be controlled, it's just a different type of engineering. Much of traditional engineering works exactly like this, with tolerances and probability rather than absolute determinism.
I think we (software engineers) are suffering from partial blindness to this aspect of engineering given that software has always been so extremely deterministic. We've trained ourselves that solving a problem means composing small fully deterministic parts into deterministic systems, and the prospect of having to work more with statistical confidence and managing failure modes feels scary and wrong.
No matter what your opinion on determinism is, the point I made that what OP is doing is very unlike a predictable and deterministic factory game still stands.
But beside that, I don't think that LLM non-determinism is really the same thing as what you see in engineering. Engineering has things like tolerances and redundancy, and while they accommodate random events, they're known, estimated and expected. They also deal with external factors, not the potential of shoddy engineering in the same system they're working on. The best way I can compare it is building a bridge that can withstand twice the normal expected full load and with enough structural redundancy to not collapse even after a prolonged lack of maintenance. These things account for many types of natural degradation and edge cases that are expected. And then there's building a bridge that you know will collapse in the next 10 years at a completely random moment, and instead of engineering a more robust solution, tacking on more gadgets to make the collapse less bad - automatic gates that close the entrances to the bridge, lake drainage system that will suck out the debris, a drone system that drops a brand new randomized bridge on the same spot, and so on. It doesn't feel very much like engineering.
If that doesn't sound convincing to you, consider one other thing. If you gave an engineer a non-deterministic solution and a deterministic solution to the exact same problem, which do you think they would prefer? Would they willingly remove determinism from a system? Maybe it's not the chaos of the real world that scares software developers so much - lots of them have to work on systems that directly interact with the real world. What if it's the fact that there is now a fight to take away their nice, understandable, satisfying deterministic solutions and inject probabilistic chaos into things that never had to have it in the first place - into absolutely everything?
> No matter what your opinion on determinism is, the point I made that what OP is doing is very unlike a predictable and deterministic factory game still stands.
I didn't mention determinism or predictability.
I said "It feels like playing factorio: whatever is manual needs to be automated away.".
I'm sure you can find many differences between Factorio and what I'm doing.
> If you gave an engineer a non-deterministic solution and a deterministic solution to the exact same problem, which do you think they would prefer? Would they willingly remove determinism from a system?
We choose non-determinism in our systems a lot, because we get better outcomes in other aspects.
* Non-deterministic network latency: Packet switching rather than circuit switching.
* Multi-tasking operating contending for the same resources.
* Multi process programs whose interleaving/execution is non-deterministic.
* Profile guided optimisation, depends on the profile.
* The entire field of randomised algorithms.
I prefer deterministic processes, but if the benefits of non-deterministic processes are significant enough, I pick non-deterministic.
If, for example, you could trick a client into thinking "com" was not a public suffix, then from malicious.com you could set cookies on paypal.com
DNSSEC is not widely deployed, and so we cannot rely on the authenticity of DNS.
Maybe if we had DNSSEC, then we could rely on it for publicsuffix. Maybe lots of others things could be more easily solved if we had that too.
DNSSEC automation is coming, e.g. https://www.rfc-editor.org/info/rfc10026/. It might never become mainstream, but don't let anyone tell you that we couldn't benefit from having DNS authenticity!
My work GitHub account got suspended because Claude ran a bunch of gh commands in parallel. (Btw why not rate limit if it’s a problem?) For days now they won’t respond to my support ticket. Guess I just don’t get to have a job anymore because GitHub doesn’t like me? Burned a lifetime worth of goodwill from me this week.
It is so mid compared to the source control ecosystem I was exposed to at Meta. It really opened my eyes to how little innovation there has been in the public space and around git pretty much since its inception.
At its core, Meta uses a modified version of mercurial. Most of the cool stuff I was referring to is not related to the core "hg" operations but related to the server-side hosting (monanoke) and the platform for operations like code review (diffusion), CI/CD (sandcastle), and all kinds of other tooling (notable mention: interactive smartlog for its fantastic GUI). I see no functional reason why a company like GitHub could not make the "Hub" part of GitHub better. It seems like they basically just haven't because they don't want to.
Microsoft has abandoned Azure Devops so we've had to move to Github for any 'new' (read AI) features.
Github is so bad I don't even know where to start.
It's fine for small personal projects and the odd project with a few collaborators.
But it's just not built for managing large software projects. Even the PR process is awful compared to Azure Devops (and Azure Devops was, in my opinion, ok bordering on good). Diffs are somehow worse than any tool I've ever used.
I am now questioning what Microsoft has been doing all these years they've owned it.
It's not unreasonable to imagine thousands of commits per second to the kernel, especially if you consider that bots might be watching LKML and applying patches to various forks.
> Handling thousands of git pushes per second implies thousands of issues, thousands of PRs, tens of thousands of comments, thousands of Actions runs, and dozens of releases. How's that part of their infra going to scale?
For each issue my agent might raise 5 independent PRs as experiments.
Further, the more changes there are, the more important it is that they don't conflict, which means smaller changes, which means even more changes.
Bots interleaving typos, single line whitespace fixes, single dependency bumps, etc.
reply