Hacker Newsnew | past | comments | ask | show | jobs | submit | jmaker's commentslogin

It judging by many microbenchmarks and my prior experience with RabbitMQ and a few Erlang systems, performance has often been a bottleneck. I’m happy that it’s got some typing. I’ll take it to ChatGPT about that. But I’m wondering about real world experience. Could you share more? I don’t have any meaningful experience with Erlang hands on and have never used Elixir. But I’d be fancy to try.

In real world, we currently serve 20M+ visitors and 500M+ requests with Elixir. Previously the client was on PHP, let's just say they have the lowest cost compared to any of the competitors in the region just by switching to Elixir.

All that gloating aside, a lot of it depends on the architecture itself, which Elixir allows you to do eloquently. Just the fact that there are no object oriented concepts alone should be reason enough to use Elixir.


Appreciate that. What’s the infrastructure that it runs on? Is it about delegating the workloads to BEAM as opposed to something like Node.js scaled on AWS Lambdas or a single large JDK container or VM instance for a Spring project where the JVM would serve the requests?

$10k for Sonnet per week? How did you arrive at that number? It’s quite a huge inference volume. No consumer hardware will give you that level of latency at comparable cost.

$6k is not the only cost - electricity adds up over time. And hardware can fail, redundancy is priced into the token prices and subscription plans. Maintenance is mostly negligible, but some updates/upgrades might regress. There’s a lot to SRE to keep it running at that scale.


> How did you arrive at that number?

I run "npx ccusage@latest weekly" and also spot-check Claude Code sessions via "/usage".

> No consumer hardware will give you that level of latency at comparable cost.

You're right. My belief is the frontier AI companies are currently subsidizing real costs for the purpose of rapid growth customer sign ups, and that in a couple years or so local costs will become reasonable, at least for full-throttle work that's queueable.


Even if one believes the price of models will only go up from frontier AI companies AND believe the local AI frontier will be fast enough at a good enough profile, it still doesn't explain how you're going to beat the price of a place hosting open weight models at scale on shared hardware always maxed out 100% in a place with cheap electricity by buying a single platform up front for 3 years.

Data sovereignty always makes sense in this case because it's explicitly not about cost, but I've not seen a convincing "it's actually cheaper to do it yourself" analysis that came to that conclusion by answering all of these things up front instead of just hoping it'll somehow get better by then.


I did the math once, to run Kimi K3 on a Mac, you'd need to daisy chain 5 Mac Studio M3 Ultras. So $125K. And the latency would be bad.

For good latency, you would need a multi GPU server around $3-600K.

I agree the hardware will catch up and local is the future. For the average person who is sharing there privacy data out willy nilly all the time without much regard, i.e. most people, it makes no sense right now to go local vs. API / subscription given the price differential.


> no sense right now

Local hw may get cheaper in the future, but so will dc hw, and DCs still have the advantage of 24x7 use, more efficient DC transformers, etc


Unless the grid goes down, my computer can turn on and is on 24x7.

And I don't need to be connected to the internet to use a local model.

I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.

Local makes sense only when the rate of improvement flattens and the pricing goes way up.

I'd bet that at least the latter is going to happen, soon.


>> DCs still have the advantage of 24x7 use, more efficient DC transformers

> my computer can turn on and is on 24x7.

1. Unless you are running AI queries 24x7 to max out your hardware, a piece of hardware that is idle the majority of the time (even if you had a full-time jobs that used the card 100% literally every single second of your workday, that would still be less than 1/3 utilized) is going to necessarily have way worse $ cost for you than cloud.

2. An idle PC does not use anywhere near the same power as inference running 100%, so I'm unsure why mention that your PC is on 24x7 - does that change things?

3. But even if we ignore both of the above and you had 4 or 5 40-hour per-week jobs that utilized local hardware ~100% every second, it would still be worse in terms of electricity costs, etc. than cloud due to worse AC/DC transformers for a local machine than a datacenter.

> I'm looking at the rate at which the models continue to improve and the pricing / rate limits in the the OpenAI/Codex/Claude/Grok subscriptions.

4. Last year you could have made the case that open models were cheaper, but now it's the frontier labs that are often the cheapest on the cost/task pareto curve.

> … the pricing goes way up. I'd bet that at least the latter is going to happen, soon.

5. Even if we were only comparing open cloud models vs frontier cloud models, is your argument that frontier labs will stop trying to be competitive with the open models?

6. When comparing to local, an obvious alternative would be cloud-hosted open models available from large numbers of providers already - why would their pricing go way up?


When I hear Apple and AI, I think of Siri. And then of my attempts to use it in my smart home where it would turn off all smart sockets instead of turning on the lights in one room. Or its response latency. Or how I was excited about Apple Intelligence and then removed it completely and am hesitant to upgrade to OS 27.

I understand that we’re talking about local inference with an Apple inference engine for Apple Silicon uniform memory. And I know it’s a stretch of an analogy. But somehow I’ve grown skeptical of Apple’s AI capabilities. Not to say Microsoft’s or Google’s weren’t disappointing.


For me Siri hasn’t improved a bit since release and it’s just the fact that she can’t understand my voice/words. Constantly mishears or asks me to repeat things. I have a pretty “regular” or “generic” US English accent and other voice agents have no problem so it’s definitely Siri. So I have never really been able to ascertain if it’s any good without an acceptable interface. I bounce immediately when I encounter the friction of having to yell, try to enunciate differently, think how to word my statements, etc. It’s like ordering food at a drive thru that has a broken mic and speaker and the employee is distracted not even trying to pay attention to my order. All to say, the spirit of your analogy seems appropriate to me.

Since that’s “likely” correlated (and a “likely” is often enough), a product manager will take the most optimal decision for themselves and the company - reject such use cases as unsupported. This is easier than to manage such an edge case for “a few” legitimate users.

Few orgs understand compliance. While there might be some compliance people at the org level, individual teams implementing features will need to have a good line to them. In the end, knowledge is siloed and everyone needs to deliver on time. So it’s easier to forbid with some controlled collateral damage than to permit with an uncontrolled downside variance. Just a derisking hedge.

Serving use cases that drift away a bit “too far away” from the average (+ some sigmas) use case is not economically justified for a product manager - their KPIs would deteriorate from such “altruism”. No one will care if a few low-margin customers can’t use the product.


Not using remote attestation requires exactly zero effort, but some jerk put it on a checklist and now middle managers want it.


I couldn’t find a way to decline model training and reduce data retention for antigravity or Gemini. Apparently it’s only available on a business/enterprise plan, not personal. Did you manage to solve it? That’s the only reason I don’t use Gemini or antigravity.


I still love Haskell, had dedicated several years to it while at the university. Was impossible to land a job with Haskell. Did some Scala but it’s in demise. F# and OCaml are extremely niche. Ever fewer opportunities and engineering management is convinced it’s impossible to hire functional programmers.

In this new AI-driven world, is there still place for such a luxury as functional programming?

I mean few people still code by hand, few read the generated code, models aren’t trained on functional languages, it’s inefficient token wise to use functional languages - while a lot become self-proclaimed software engineers overnight by just prompting LLMs.


Hiring for Haskell is very easy, unless you need to hire more than 100 people on the spot, or don't offer remote. I've been doing Haskell hiring for 10 years, with more than 50 interviews, the applicants are always great, each time we had to pick the best of multiple candidates.

Cannot confirm functional programming being inefficient token-wise, or worse at being generated than other paradigms. Claude is great at Haskell.

The difficult parts about Haskell, such as understanding type checker error messages, are gone thanks to LLMs.

Type-safe, side-effect free code degrades correctness a lot less under heavy LLM action in my experience.


I’m really happy to hear that. I wish Haskell were more widely adopted and more generally accepted as a solid enterprise choice without the bus factor. I can confirm that the applicants are usually great in every respect. I tried to introduce Haskell in multiple enterprises but failed at engineering managers’ reluctance to admit Haskell as an option - every time because of the bus factor argument. Scala used to be easy to advocate for as an alternative. To me, with LLMs there’s really no need to care about the choice of the runtime language unless there are SLOs to meet or library choices due to domain peculiarities.

As for token-wise inefficiency, this is based on my sporadic observations and discussions with friends and colleagues from the past three years. I can’t give you a fresh benchmark with latest models in a reproducible way. But I’m happily willing to accept your assertion at face value. This makes me curious to see for myself how the latest models perform, will perhaps set up a quick evals just out of curiosity.

From my past experience, the latest I’ve seen were Opus-4.6 and GPT-5.5 struggle a lot with standard GHC Haskell, no extensions, nothing fancy. Not that they produced impeccable Python or Rust. But it appeared to take several turns for obvious expressions, while at Java and TypeScript they were much better - fewer turns, time, and cost for the same verified results. I had assumed that the training set wasn’t large and diverse enough for Haskell and Scala, and this was the go-to explanation with everyone I talked about it. Generated F# code was good enough but not OCaml. With C++ there’s still quite some struggle, expectedly.

To me it resorted to the question, if we’re mostly generating code through LLMs now, which ones and what will it cost in total terms per task/capability completed. I’m wondering now if DeepSeek/MiMo or GPT-6 Luna can benefit from the terser, more (forgive the pun) load-bearing expressions.


There are companies like IOHK or Mercury that hired large swathes (50?) of Haskell programmers in relatively short time, so I don't think bus factor of the programmer market is an issue. Of course it's still an issue if a company hires 1 or 2 Haskell devs only, and those depart.

> from the past three years [..] the latest I’ve seen were Opus-4.6 and GPT-5.5 struggle a lot with standard GHC Haskell

If your experience is from years ago, I totally get that. For my standards, LLMs were pretty bad before 2026.

I haven't tried any serious LLM Haskell before 2026-02, because it was just not a timesaver.

I had good experiences with Opus-4.6 already on Haskell, such as it having detail knowledge on async exceptions, allowing it to immediately pinpoint difficult bugs that me and other experts missed, e.g. https://github.com/snoyberg/conduit/pull/530

But most of my experience is from Opus >= 4.8. I found it excellent at writing difficult TemplateHaskell and Generics implementations, such as oneshotting Postgres JsonPath queries derived from lens-like field accessors on Haskell types.

Note it's still not great at writing general high-quality "engineering" stuff, e.g. Opus 5.0 writes functions that refer to things out of their scope, such as:

    requireFooOr400 :: MyMonad m => Maybe Foo -> m Foo
    requireFooOr400 x = case x of
      Nothing -> throwHttp400 "Cannot perform [very specific action from the caller]: the inputs have no foo"
      Just foo -> return foo
However, that's not a lack of Haskell understanding, and it does that for all languages, e.g. in Python it keeps writing implementation details into API docstrings instead of function body comments.


> I haven't tried any serious LLM Haskell before 2026-02, because it was just not a timesaver.

Same, and now seven months later a GBP 20 per month Plus subscription gets me basically all the GPT-6-Luna-xhigh I can eat, which does a tremendous job of Haskell. It's even helped me with some really gnarly things like this, which require deep knowledge of the GHC RTS: https://github.com/tomjaguarpaw/bluefin/commit/c298733bdd67d...

It's possible it's much slower when working on Haskell than it is on other languages. I wouldn't know because I've only tried it on Haskell. But I'm very happy (and extremely impressed) by it.


For what it’s worth I recently decided to learn Haskell for fun and to keep my brain alive since work turned into a prompt factory.

Fable has made an excellent tutor. Maybe it’s just because the exercises I have it generating for me are relatively simple, but I have yet to catch it suggesting non functional or non compiling Haskell code.


Please note the "we had to pick the best of multiple candidates". Easy to hire haskellteers, but Haskell shops are not many. You would probably find more Clojure roles, but they are in decline.


My full time job is in scala and I find LLMs produce pretty solid code with it.


there are a lot of strong claims here, but to just pick one:

> inefficient token wise to use functional languages

where does that idea come from?


With LLMs, I'd expect if something appears more frequently in the training data, then the LLM would be more effecient or smarter at dealing with it somehow.

Surely there's way more content related to programming in Python, JavaScript, Go, etc. than in Haskell.

So you'd expect some things like: an LLM is likely able to come up with an approach that's suited to Python/etc., and an LLM is likely able to work its way through Python/etc.

My experience has been: when using an LLM with a nice language (Nickel-lang, a modular configuration language with types and contracts) that it frequently guesses slightly wrong as to what works (e.g. guessing wrong about how stuff like { x = x + 1 } would work) that it spends more tokens than it otherwise might.


Ooh, you're saying you need to ask it twice to get the same real-world result. I thought you meant the tokenization was less efficient (as if `Endofunctor` were tokenized `Endo`+`Functor` while `AbstractFactory` was a single token or something like that)


Interesting, most functional languages are quite terse and heavily whitespace, so you would expect raw token counts like that to be lower simply because source documents are on average shorter. Even having to make up for disadvantage things like C++ style operators like &&= all having single tokens from heavy use across a variety of languages but fancy functional operators like <=> being much more rare and in worst cases of undertrained models needing multiple tokens to express.

But yes, terser source documents and low raw token counts could mean lower prediction rates and more prediction attempts needed to get intended results (again, especially if the model was undertrained in that particular language).


This is an interesting hypothesis to test. Depending on the embedding model, such less frequently encountered character sequences in the coding domain could indeed produce less optimal tokens. A dedicated model trained from scratch on Haskell would be optimal but wouldn’t generalize well beyond that, but can serve as a deliberately overfitted baseline.

What I meant was really about the output tokens, on the API pricing level of interaction with an LLM.


Opposite experience here. SuperGrok allowance is drained extremely fast. About 20-30 simple coding or code review prompts in grok build, and it’s over for the week. When using it via the Grok app, it can last longer, unless you need to edit photos. I didn’t bother to properly measure grok’s burn rate on the SuperGrok plans. It’s just not there yet.


For me Sol is less efficient than Astra - Sol makes many avoidable mistakes and has issues with context compaction - sometimes it goes haywire after a couple compactions.

Agreed on the “being dumbed down” observation. It appears they’re most powerful at release time and then are gradually “optimized” so every new model feels more powerful. But there’s no evidence on routing to a deployment with other weights. It would be plausible to do so though at least at peak times.


What really is fun is when the opposite of the public outcry about purported distillation happens - when Astra in Codex suddenly responds in Chinese. Now that is fun. Wonder what it’s routing to.


I have done the same. I hesitated for way too long. I shouldn’t have.

I get way more usage for way less money without any quality or performance degradation. My $200 Codex Pro plan allowance is depleted in 2-3 days. Sometimes Tibo announces a usage reset. But GPT-5.6 models are really not good for coding. Sol has been making increasingly more mistakes in the past two weeks even in the reviewer and advisor roles. Astra is usable for coding but slow and very expensive. In the past two days I’ve used up over 70% on simple copy editing, with dedicated short specs and short sessions. Really little one can do to make it more efficient. Similar work took 20% at most just a month ago. I’m looking to use Astra for milestone reviews/advisory. Perhaps a $100 Pro downgrade will be enough. But my main work is now on open-weight models. And you don’t need to depend on someone to send you a reset. And it’s cheaper by the end of the month too.

With Claude the limits are not even fun anymore - my weekly $100 Max plan quota is gone in one day on merely review invocations, no coding. And my $200 Pro quota is gone in two with some coding. Sonnet 5 is not usable for coding. And Opus 5 tends to always make a couple avoidable mistakes on every task. Fable 5.1 is ok but tends to ignore skills and to work around explicit instructions. Completely canceled all my Claude subscriptions.

With Qwen 3.8, DeepSeek 4.1 Flash, GLM 5.3 I’ve been getting Opus 5-level performance, with less blah blah and no overengineered churn. Public benchmarks are really not telling the real story. The models are more dependable and more predictable. They have their own failure modes. Sometimes DeepSeek 4.1 Flash is quite stubborn but it fails in a good way. Bad for full autonomy - I need to intervene, but it sticks to the rails and instructions - other than Fable and Opus that try to outsmart you and your harness.

Grok is interesting but has been a bit underwhelming on Grok plans - my SuperGrok allowance is depleted in a single session overnight. SuperGrok+ gives more but it’s still about the same as with OpenAI, Claude is way less now.

Since the allowance volume has been shrinking with the major model providers, to me, open-weight alternatives are really necessary now to at least maintain the momentum and budget.

But at work it’s really an uphill challenge - it’s become impossible to convince the tech leadership once they got hooked on Anthropic. They No facts will help. Some people underestimate how expensive Claude really is after getting used to the subscription plans with allowance resets. OpenAI models are expensive too.


Why not the other way around?


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: