What makes it frustrating is that the information, assuming it’s accurate, is actually useful and interesting, but it’s swaddled and interleaved in these bulky layers of unnecessary verbiage.
AI prose really is the thick crust pizza of writing.
I think hate of plastics is an emergent form of elitism.
Upwardly mobile middle/upper class people who've sort of "maxed out" the amount of personal identity they can buy with regular plastic things can unlock a new level of identity by deciding that plastics are bad for them and eliminating plastics from their life, a process which conveniently requires buying a whole new set of things that distinguish them from their peers.
This is the only way I can explain how irrational and inconsistent plastic-haters behavior is. There is so much invisible plastic in their life that they don't seem to care about.
> requires buying a whole new set of things that distinguish them from their peers
No, it requires buying a whole new set of things to fit in with and be accepted by their peers, to distinguish themselves from the outgroup, the plastic users.
I don’t necessarily believe this is some emergent elitism; I see it more as a modern religion with many many rules about eating and consumption (using plastic is now a sin).
Like any religion, sinners (for example plastic users) are mostly pitied because they are ignorant, but those who know and choose to use plastic anyway, well, it’s OK to hate them.
I don't get this whole attitude. Moving away from plastics in our kitchen was basically zero extra cost. Something busted, and we replaced it with non-plastic. Even bamboo scrubbers don't cost more than plastic (maybe even a little less) and I can't see any particular longevity difference.
I think the fact that I volunteer to clean up trash on public lands and know that weathered plastic is the period worst period to remove makes me move away from plastic in general.
Plus, a solid $3 wooden spoon is just a joy to cook with. They outlast the plastic ones, too.
Microplastic ingestion? Well, I'm not sure the effects or the relative quantity compared to tire shed and other industrial factors. But if I were forced by some diety to bet my life on if plastics in the kitchen or on clothing had a negative health effect, I'd make that bet.
But the main thing I don't get about the attitude stems from the fact that I don't really care what other people use in their kitchens. I recommend it.
Things like glass food storage containers are really expensive compared to plastic. And they still have plastic lids; like, I don't know what you'd even do without at least the seal-part being some kind of plastic, I guess you'd need to use natural-sourced wax to make it seal, or something?
And on the topic of cost, I'm certain my kids have broken between 50 and 100 glass and ceramic drinking cups, storage containers, plates, and bowls in a little over a decade. They destroy plastic items at a way, way lower rate. Consider the use case of packing a kid's lunchbox. Plastic is... very tempting, for practical reasons. And cheaper.
Last I checked, plastic vs. wood on an otherwise identical stamped metal Victoronix knife costs you an extra $15-$20, which is a notable percentage of the total cost of the item. I sprung for the wood on my latest replacement just for the aesthetics, but it cost enough more that I did give it a good think first.
> Even bamboo scrubbers don't cost more than plastic (maybe even a little less) and I can't see any particular longevity difference.
Are those actually just bamboo? Maybe they are, I dunno, I can't recall seeing one. Lots of the "bamboo" materials I've encountered have turned out to contain (at least some) plastic.
> Plus, a solid $3 wooden spoon is just a joy to cook with. They outlast the plastic ones, too.
That's just true. Plastic spoons for cooking suck, wood and (where it makes sense and won't damage other things) metal are way better. Wooden ones aren't even expensive. The popularity of plastic ones is baffling.
One thing that's surprised me is the cost and/or total lack of availability of glass blender jars, even on fairly high-end brands (both the fake-high-end ones that are just expensive, and the actually-good ones). I remember my parents' assuredly cheapest-thing-in-the-store blender that they probably bought in the 70s or 80s had a glass jar, because that was just... standard. Meanwhile my as-awesome-as-I'd-hoped-thank-god expensive-ass Vitamix came with a plastic jar, and they do not make glass replacements. (I'm just checking and it looks like they might finally make one in stainless, though? Still, I'd prefer glass because being able to see what's going on in there is very nice, but I'm gonna have to look into that...)
I don't have a good answer for kids breaking stuff.
And I do wonder what glues are used in the bamboo scrubbers, but I have no way of finding out. The scrapers are at least a solid single piece. They're either definitely bamboo or some very skillfully-engineered plastic to look exactly like it. ;)
There are a lot of cheap glass blenders on [ONLINE STORE]. But yeah, the high end ones--maybe they're trying to avoid that 50s look.
We just run with the plastic lids with glass tupperware. The only other sensible replacement we've found is reused jars and Ball jars, but all those lids have plastic liners, too. We don't cook the lids and food contact is limited. Would be nice to have something else. Silicone lids? We have some of those we picked up for Ball jars at some point. But this seems like a lower potential issue than cooking with plastic materials.
For really inexpensive stuff like blenders and glass containers is the second-hand store.
Around the time that I became a parent, I remember a friend’s wife emphatically telling me why one should never ever give fruit juice to a child. Later in the evening, she recounted an anecdote where a bout of constipation was cured with prune juice. I could not help myself from commenting “but isn’t that fruit?”
somehow this thread reminds me of that conversation.
> That's just true. Plastic spoons for cooking suck, wood and (where it makes sense and won't damage other things) metal are way better.
If you have ever used nice commercial high-temp silicone spatula, it's an incredibly versatile and easy to clean spoon for cooking. A bit expensive at like $20 though. Pair with nice nonstick pan and polycarbonate cutting board (dishwashable) for the easiest and most out of fashion cooking and cleaning experience.
> One thing that's surprised me is the cost and/or total lack of availability of glass blender jars
My cheapest in the store oyster blender is glass, I think they mostly still are.
> If you have ever used nice commercial high-temp silicone spatula, it's an incredibly versatile and easy to clean spoon for cooking. A bit expensive at like $20 though. Pair with nice nonstick pan and polycarbonate cutting board (dishwashable) for the easiest and most out of fashion cooking and cleaning experience.
Those, I do use! My wife insists on keeping one non-stick pan, mostly for eggs (I just cook them in stainless, whatever) so we've got a couple around for that specific use case, but I grab them sometimes for other things, too. They're great for scraping little bits of sauce out of the edge of a pan, things like that.
> My cheapest in the store oyster blender is glass, I think they mostly still are.
Ha! Really? I killed one blender before upgrading (the old "buy a cheap one, and if you wear out out, buy the expensive one" approach) and that was also plastic, but it probably wasn't Oyster. Hm.
All of those were glass when I was a kid, it seems really weird to me that the pricey ones are usually plastic now. I'm not even (that) worried about the health effects of it, I mostly just like the way the contents move & pour in glass better, the plastic's too "sticky" (though I do cringe a little when we blend a near-boiling sauce in the plastic jar)
Sorry if I offended you; fwiw I was describing a phenomenon that I observe. My own attitude about this whole situation was left unspecified, so I’ll add a few notes here: I do actively pick up litter (sometimes bigtime: I cleared a derelict boat from a lake today!) and I try to not litter. I try to respect the planet. I do use a lot of plastic.
I've addressed this idea in a sibling comment. I think at least some superstition is inevitable in any subculture. Consider how many tech 'holy wars' might involve baseless beliefs about how a text editor or programming language or whatever being not only superior because of personal preference but because it's inherently more optimized. Treating anti-micro-plastics as a "religion" rather than a subculture based on a meme deserves a bit more nuance.
1. Is it based on inherently irrational, unfactual beliefs, e.g. anti-vaccination or anti-5G myths?
2. If we consider religion as a way to explain complex phenomena using just-so stories (the pop anthropology / layman idea of primitive man inventing Zeus to explain lightning), then what intellectual or emotional need does anti-microplastics belief validate?
> Treating anti-micro-plastics as a "religion" rather than a subculture based on a meme deserves a bit more nuance.
Agreed, and I think we satisfy your two conditions and many more.
If I read you right, question 1 pertains to irrationality, and question 2 pertains to powerful storytelling.
OK here’s one example that shows you are on the right track, but there is still more.
My ex-wife is gluten free. This single choice never comes alone. in my ex-wife’s case, she also believes in the healing power of crystals and essential oils. (Irrationality checkbox); I cannot describe her vision of cosmic creation but I guarantee it involves witchcraft/folk
magic and differs from
traditional dogma (Storytelling checkbox).
BUT THERE IS AT LEAST ONE MORE VERY IMPORTANT CHECKBOX! A good religion also prescribes a proper way to behave and act in the world for maximum benefit in the afterlife. In these newer religions, the afterlife is “the health of the planet”, so one must do everything in this life to ensure you do not hurt that future.
To be fair, that wasn't meant to be an exhaustive list of criteria for what constitutes a religion, nor is the second one really a criterion for inclusion, just a couple of points I was thinking of. For the first, I would hesitate to label microplastics skepticism as baseless as the other two examples I listed, as there seems to be some studies that suggest they're not great for the human body? But sure, forming an entire belief system about how they're the anti-panacea for all of modern ills is certainly unfounded.
The second is more prescriptive-suggesting than descriptive. What motivates the desire to create single scapegoats for all of modern ills? Maybe a breakdown in public trust in science, lack of comprehensive medical education, spread of misinformation, political polarization (lots of these health fads end up embroiled in culture wars), etc. So that's a host of issues to work on. And at the same time, the anxieties they are preoccupied with- widespread changes to people's bodies, for one - do need explanations for, otherwise people will flock to just-so stories.
Your introduction of a third point about how a modern pseudo-religion provides a call to action to achieve a desirable afterlife that actually exists in this life (e.g. some unspecified healthy future) is rather inspired. Reminds me of the phenomenon of how secular cults of political personality now appear to be moving in on traditional spirituality. Superstition, it would seem, can exist even without belief in the overtly supernatural.
All culture is shared ignorance. These comparisons to religion are inverted. Religion was born out of cultures needing to herd their people.
You're right that the debate about plastics is mostly meaningless noise by people who don't really care. Taking advantage of uncertainty while it still exists is a lucrative game.
None of this is comparable to software. Writing software is a choice and the users don't have to care beyond the UI. It's apathy, not ignorance, that holds software back. Text editors and programming languages are not usually the highest priority choice to make. The majority of software tends to be specialized one-off solutions. We don't exactly have chemists cooking up their own kitchenware materials on the weekend.
Software development creates subcultures, just like any other occupation, craft, human activity. Tech holy wars is a thing, and I'm sure a lot of it dealt with articles of faith just like any other source of controversy.
I brought them up to illustrate that any endeavor will end up pockets of irrationality as part of the general culture. Doubtless in academia, scientific research, other forms of engineering, etc. there are little superstitions as part of the subculture.
That said, sure; the movement against microplastics is a pop health fad, which is different because it's a consumer-oriented activity whose actual effects are probably impossible to quantify. (As pointed out in a different comment in this discussion thread: https://news.ycombinator.com/item?id=48836691) Building software produces tangible, measurable outcomes. Though likely far less measurable compared to other forms of engineering!
> It's apathy, not ignorance, that holds software back.
That's somewhat debatable, in that compared to the physical sciences, it sure seems like software engineering involves a fair amount of following traditions of preexisting practices and there's a lot of cargo culting that ends up happening. More of a craft than a science. But that's tangential to this discussion.
> is different because it's a consumer-oriented activity whose actual effects are probably impossible to quantify
Your perspective on software is that of a consumer, so you're not necessarily wrong. You're in the majority of people using software along with all those people having their "holy wars".
I'm just saying it's not intrinsic to software. The majority of people who write software for a living are silent about this because it's completely irrelevant to their lives.
It's like this with all other creative work too. As they say "a poor craftsman blames his tools", but more generally improvisation is expected when you're supposed to know what you're doing. Professionals can't afford to be helpless. In fact, that's why we have so many competing standards in technology to begin with. Constant reinvention is the most boneheaded way to progress, but my point is that this is in direct opposition to everything you're saying.
You seem to be insisting that there's ignorance where there is just apathy. For every one person whining, the internet has a chorus of hundreds. On the other side of the fence, there are dozens of people who could fix it in their sleep just ignoring it because it doesn't bother them that bad. That's what makes software so different from manufacturing plastic doodads at scale. You at least don't need a factory, but there are probably countless other reasons.
> Was this LLM-generated? Because the intent is garbled, each sentence going in a different direction from the previous. (So perhaps not LLM after all.)
Quoting the entirety of your comment verbatim in case you edit it. It's poor form to overplay your hand and troll instead of discuss.
I flagged your reply, and I suggest anyone reading these should do the same. HN is really going off the rails allowing this, but I know it's hard to rein it all in. At the risk of becoming even more of a hive mind, I think we need users to pitch in some more.
> Your perspective on software is that of a consumer, so you're not necessarily wrong. You're in the majority of people using software along with all those people having their "holy wars".
What do either of those statements mean? You neither say what I am not necessarily wrong about, and the second sentence means that - everyone is using software? The majority and the minority are both doing something?
> The majority of people who write software for a living are silent about this because it's completely irrelevant to their lives.
I’m not sure if this is accurate. Programmers have opinion, taste-based arguments all the time, sometimes based on dubious assertions. People pay attention to the proclamations of Linus Torvalds or John Carmack even when sometimes they’re not necessarily based on evidence, and parrot it as prophecy. Though sure, they are also not the norm, and becoming a public figure changes one’s behavior. (Uncle Bob Martin appears to have turned into a Twitter crank in recent years, for example.)
> It's like this with all other creative work too. As they say "a poor craftsman blames his tools", but more generally improvisation is expected when you're supposed to know what you're doing.
Like what? What are you referring to? What does improvisation have to do with what you were just writing about?
> Professionals can't afford to be helpless.
What is making them helpless? What are they doing or not doing?
> In fact, that's why we have so many competing standards in technology to begin with.
Okay this might be relevant. So you’re saying what I was talking about- tech holy wars based on opinions and superstition- causes competing standards?
> Constant reinvention is the most boneheaded way to progress, but my point is that this is in direct opposition to everything you're saying.
What does constant reinvention have to do with anything? What is an example of reinvention?
> You seem to be insisting that there's ignorance where there is just apathy.
I am not saying that all of engineering is ignorant, I am simply making the banal and obvious observation that any field of human activity will contain pockets of emotion-based conclusions that might not reflect reality.
The original point is that even the most seemingly empirical, scientific, whatever, domain might still involve facets that aren’t so scientific.
What does apathy have to do with it? If you were to rebut my point, you can simply say, “no there is no ignorance.” Why are you contrasting it with apathy?
> For every one person whining, the internet has a chorus of hundreds.
What does this sentence mean? A chorus of hundreds not what, not whining? The internet is full of whining choruses.
> On the other side of the fence, there are dozens of people who could fix it in their sleep just ignoring it because it doesn't bother them that bad.
Fix what? Ignoring what? What are you referring to?
> That's what makes software so different from manufacturing plastic doodads at scale. You at least don't need a factory, but there are probably countless other reasons.
??? Countless other reasons for what? Why are you contrasting software with physical engineering in a way completely irrelevant to my prior comparison?
Finally, I haven’t even touched upon your original response, which starts with “All culture is shared ignorance.” Which, while sounding both profound and impressive at first, is meaningless. All culture is shared knowledge as well. All culture is share norms, mores, values. What does it all mean?
> HN is really going off the rails allowing this, but I know it's hard to rein it all in.
Allowing what? Allowing people to write things? Last I checked people flag plenty of posts, including trolling. Rein what all in? Trolling?
First of all, yes, performatively following health fads has always been a thing for people with money.
But do you honestly believe this is what's going on at a large scale? What an absurd reaction to other people doing something that you don't like or don't understand. Nobody who isn't a paid influencer is sneering at you for using the wrong materials in your kitchen.
If you feel sneered that, that's your problem and not anyone else's. Are you going to start plasticmaxing, like rolling coal but for food containers?
I think you're doing the thing that you are accusing other people of doing. Creating an artificial distinction between "us" and "them", so that you can feel like you are part of the "us". It's just that in this case you want to feel like some kind of rational rebel, one of the final chosen few with common sense, in an increasingly irrational world.
Even more so: plastics are not a specific chemical and they are not a specific material. Plastic is a category of materials that is very broad and very wide. You can make plastics out of almost anything. Therefore, to hate on plastics is to basically hate on an entire category of engineering and material design but not to actually know what a plastic is... sheer ignorance.
Yes, it's a category of materials that is overwhelmingly populated by a much more specific, ultra-cheap and therefore ultra-pervasive, set of chemicals which are shown in study after study to have worrying characteristics.
Anything that shows evidence of omnipresence, endocrine disruption, bioaccumulation, and inter-generational transmission should be extremely, extremely closely scrutinized.
To think otherwise is absolute braindead contrarianism, full-stop.
Personally, I think that the Microplastics Moral Panic is a textbook study of F.U.D.
There is practically nothing that ordinary people can do for prevention, mitigation, diagnosis or treatment of microplastics in our bodies, so I therefore conclude that it is futile and wasteful to worry or argue about it, unless you have abundant free time and resources to get paranoid strangers all in a frenzy, for no good reason.
I half-assed try to avoid plastic in contact with (especially very) hot food or drink, and avoid it in long-term food containers, in no small part because I've seen things like plastic cooking spoons losing non-microscopic parts of themselves in food, and I find the stains plastic storage containers acquire after a little use kinda worrisome (I'd rather my reusable storage containers not be that permeable, thank you very much), but otherwise agree that any real amount of effort to avoid microplastics would probably do more good if it took the form of a 20-minute jog per week, i.e. "don't even consider worrying about it unless you've really, really got all your other health stuff sorted out"
Like I'm pretty sure the bigger health risk with a plastic soda straw is the soda, not the straw, you know?
This is a fine argument against banning plastics. This is unrelated to "Microplastics Moral Panic is a textbook study of F.U.D." though.
If microplastics are a case of FUD, then there's nothing to look into. How and why would anyone discover anything surrounding a case of moral panic/FUD? By definition, there's nothing to discover.
You could just liken it to any pop health, dietary, or environmental fad instead of trying to portray a banal "people turn consumer choices into personal identity/lifestyle" trend as a whole new class of phenomenon. Crunchy hippies shop organic and audiophiles buy gold-plated premium wires; every subculture has at least a little bit of superstition.
Health in general has always worked like this, from supplements, to organic food, to avoiding ultra-processed foods, to gluten-free diets, pretty much every popular health fad has no provable causative effect on health.
Most of human behavior is irrational. If we were all perfectly rational, we would have healthy diet and exercise habits from the get go, and we'd have plenty of time to prepare food and exercise, because we wouldn't waste any time on entertainment.
>pretty much every popular health fad has no provable causative effect on health.
Isn't this almost true by definition? If it actually works, it's not a "fad", just "science" or whatever. Advice like "eat more vegetables" and "don't drink alcohol" probably do work, but they're ingrained enough that nobody thinks they're "fads".
The late 1800's health craze centered around Battle Creek, Michigan has had long lasting cultural and dietary impact, but I think most people would still call it a fad, because the core tenants were only popular for a short time.
Besides, the breakfast cereal movement it created quickly devolved from the Kellogg's and Post's original healthy cereals, Corn Flakes and Grape Nuts respectively, into the sugar-laden craze of 1950's cereals. Even Graham crackers were healthy and bland at the time, but we now associate them with cinnamon and sugar.
If someone comes out with a study that shows, all other things being equal, decreasing or increasing the amount of ultra-processed foods causes a respective increase or decrease in health, then I will rethink my stance.
Until then, I'll keep eating foods consisting of proven healthy ingredients, even if they are classified as Nova 4 ultra-processed, such as lasagna made using my great grandmother's recipe or ancient Mayan dishes like pavo con achiote and also cochinita pibil, while avoiding foods that are proven unhealthy despite the not-ultra-processed Nova 3 and lower status, like Cronuts, potato chips, and bacon cheeseburgers.
I don't know why their ingredients are classified as Nova 4, or why the processing level is defined by ingredients, in the first place. Lasagna contains ricotta which is made from whey, and whey automatically makes something Nova 4. Many ancient Mayan dishes contain achiote seeds, also known as annatto, which also automatically makes something Nova 4.
Achiote seeds add a bright red to yellow color, depending on the concentration, and add a mild earthy flavor. This puts it in the Nova 4 ultra-processed foods category, ostensibly because the color can make food more attractive. This is as opposed saffron stigmas, used in some traditional European dishes, which add a bright red to yellow color, depending on the concentration, and add a mild earthy flavor. Why adding achiote makes something ultraprocessed and adding saffron doesn't isn't explained, but I wonder if there's a predilection toward traditional European foods.
The Nova classification only counts fresh fruits and vegetables as unprocessed, which is Nova 1, so almost all foods are processed foods, Nova 3, or processed ingredients, Nova 2.
I eat healthy food and avoid unhealthy food, but the Nova classification system is so poorly done that it is meaningless. Even going off of actual levels of processing, instead of the ingredients, doesn't help much. One of the earliest processed foods, hominy, is much healthier than the unprocessed counterpart, and when Italians relied on polenta made from unprocessed corn, they became malnourished and suffered from pellagra, while those in America relying on the otherwise identical grits, made from hominy, weren't malnourished.
That's why I go off of actual research proving which ingredients are healthy based on the effects of adding them to or removing them from a diet that otherwise stays consistent in volume and calories, instead of the vibes-based categories the Nova classifications use, and the vague research showing that rich healthy people who exercise eat some things and poor unhealthy people who don't exercise eat other things, but never showing that eating some things actually makes you more healthy and eating other things actually makes you less healthy.
If similar levels of research into ultra-processed foods were applied to other statistics, they'd find that ice cream sales lead to drowning and owning a Ferrari makes you rich.
Why was that we used to be able to ban environmental toxins such as leaded gasoline without this weird psycho-social-political analysis that has become the fashion in some circles?
I've picked up enough plastics off the beaches of the world already. Plastic ruins ecosystems and is a scourge on this planet. It's too late to fix, it's everywhere. 8 million tonnes of plastics enter the oceans every year, and it just keeps accumulating. We're ruining marine ecosystems.
Are you sure it's all irrational? I, for one, prefer other materials for many things because they are more durable, hygenic or simply feels nicer. Seems perfectly rational to me. An irrational choice would be something like carrying a heavy canvas tent with wooden poles just because you hate plastic. Do you know anyone who does that?
Interesting theory but doesn't really coincide with on the ground facts. We know there has been an increase in hormonal issues. We can barely even get anyone to stop heating up and microwaving plastic to be ingested, which we know causes issues. The European Food Safety Authority completed a re-evaluation into the risks of BPA in 2023, concluding that its tolerable daily intake should be greatly reduced. BPA was everywhere, every can and receipt. Just because people can't identify all the ways plastic is ubiquitously ingested in their life, it means they're hypocritical and don't care about the unknowns?
Part of the value of doing an ETL pipeline via streaming replication is you get the full history of data in a table. An SCD type 2 table where each row also has a valid_from and valid_to timestamp column.
How would someone do the same thing with this architecture?
because they are making an extra step: instead of the snapshot being actually an object manifest, they store the snapshot metadata in a relation database.
I have a use case where we do 100k changes a second. I bet you "their unlimited snapshots because we are cool and using dbs for metadata" will fail in such a use case.
Every commit in Delta is time travel-able. Defaults in Delta clean up logs in 30 days (and data in 7 days but requires a VACUUM be issued.) You can set those retention periods to any arbitrary time period, but it's typically advised against due to the cost to store all the historical data.
It wouldn't be possible to do this with LTAP architecture since (I'm assuming) the individual logical changes are not visible. But honestly I've always seen SCD type 2 table as a workaround due to lack of data modeling experience in the source database. If you design your tables correctly, you shouldn't need SCD type 2 downstream.
For example, if you know your user can change emails, and there might be events from another source that is keyed by user email (e.g. marketing-related events), then naturally you will need some sort of email_history table that has historical mapping of user id to email (you probably need it for audit purposes too). Then in this case there is no need to build SCD type 2 table of user from CDC, it's already there.
I think you have a point, and SCD type 2 feels like a workaround, but there is also something to be said for the ability to query every row as it was at any given version.
I’m not saying that SCD type 2 is the best solution given there might be a more domain-specific way to do it, but I see it a lot like file-based version control.
It’s convenient to be able to examine all files as they existed at any point in time, without having to “model” the ways in which those files might change directly into the domain of the individual files.
If you have something like dolt (not affiliated), a version controlled database, you wouldn’t have to slap change dates on anything OR create your historical table. The changes would be implicit in the version history.
> If you have something like dolt (not affiliated), a version controlled database, you wouldn’t have to slap change dates on anything OR create your historical table. The changes would be implicit in the version history.
Nope, even if I have the ability to see the exact changes of each row, I would still add timestamps everywhere, because timestamp of row change does not equal event timestamp. For example, if I have an order table with status column, and I see a CDC event where status changed from in_progress to completed, I cannot simply assume that the CDC timestamp is the timestamp when order was completed. It's possible that the source database received the event late a few minutes late due to delay upstream, or it's backfilling some missed orders a few days ago. Having a completed_at timestamp (and a bunch of other timestamps for each order lifecycle) would eliminate any ambiguities, and your data analyst will thank you for it.
It's the same thing with row history. You cannot simply assume that your row changes are aligned with the logical history of your entity.
Rather than answering directly, I'm thinking about this problem from the other end altogether ever since I saw the dbricks rt demo. Apologies for the rambling response, as I haven't yet finished thinking about this problem...
We ended up with 'hot' data in oltp and 'cold/archival' data in olap because the storage size of oltp has always been limited.
(1) Limited by computation - there's only so much data that we can store on disks and nvme
(2) Limited by wallet - disks and nvme are EXPENSIVE
Also, the tight coupling of compute and data didn't help. It limited the size of databases on the individual expensive compute nodes.
So, another question will be -
What's currently stopping me from keeping the scd history tables right in my oltp db? what's forcing me to copy state into my etl/elt pipeline and the process it into scd into a dedicated olap db?
To some extent,the answer is still the same - the oltp cannot scale for the storage size required for keeping historical data. So, I've had to take out the 'cold' historical data and keep it in my olap freezer.
Now, if oltp itself is scaling, I'm not gonna bother with the copying step. I'll just prefer to store the history in oltp itself.
In my perspective (majorly from handling IoT systems), I need olap for 2 reasons - (1) storage scalability, and (2) analytical processing speed
I now consider (1) to be a solved problem
As for (2), I'm still not sure how this architecture ends up matching the query processing speeds of column-oriented storages. But again, I need to study more.
The SCD pipeline still remains in some form. Either in the form of (1) scd rows that we currently keep (etl pipeline)
, or (2) as older lsn rows that simply don't get deleted (existing db engine).
I've done quite a lot of experimentation with (2), and it is a pretty solid concept to work with.
I've spent quite a lot of years hammering my brain at databases and datastores in general. And I've now got a feeling that this is it.
Finally.
I have an imperfect "canary in the coalmine" test for whether/where the principled/idealist nerds have the power vs the mercenary/cynical nerds have the power:
Advertisers vs Ad Blockers - as long as the Ad Blockers are blocking the vast majority of junk, I feel like team principles is winning. Like they have more nerd power to outsmart the lesser nerds working on team mercenary.
But there are places like Facebook where that is not the case. And even in Youtube, team mercenaries has recently won some territory from team principles.
Is it possible that different regions have different ad experiences? I've not seen a single ad nor hint of one in a long time using the Brave browser on both android and desktop. Primarily in Europe.
I’ve been very involved with the DNS blocking scene- and it’s extremely easy to circumvent. I always wonder if there are some principled nerds on the architecture side purposely designing things to be easily blocked with DNS blocking. Or perhaps the mercenary nerds are just that inept. Maybe a mix. I also don’t think the numbers of users with DNS blocking are large enough for the mercenary nerds to care about. However, the math for LoE to bypass DNS blocking has certainly changed recently.
It's not. You forget the disadvantage the mercenary nerds have - they do not, and can not, trust each other. So having the same company serve ads as is serving the content can never happen because mercenaries can't resist cheating and so advertisers won't trust them (after being burned way too many times)
If we go this route, they are just going to embed the ads into the video itself. Clearly the solution is to create an alternative to Youtube. This war is not going to bode well to either side.
> just going to embed the ads into the video itself.
This has already happened, and a counter measure for this first iteration has already been developed - SponserBlock: https://smarttubenext.com/sponsorblock/
Next up, embedded floating ads over the watchable desired content.
I would happily, delightedly, see the entire internet disappear in a puff of smoke, to remove ads from the world. Honestly, I'd be happy to see everything on the internet except the wikipedia vanish regardless, at this point.
Much of the mundane world - tv, bus stops, and so on seems to have ads. While in my nerd domain, my computer, I don't see ads at all. So it would seem that open source software is letting me live without ads on the internet, from home.
I was even annoyed at the ads in a paper newspaper I was leafing through, I'm not just used to them and I think some of them were a bit dishonest and scummy.
I was wandering around the carcass of Reddit the other day, and it’s crazy how it’s like 70 percent AI now commenting to each other in a lot of subs that used to be at least nominally interesting… and then a few clueless humans getting all riled up with an AI lol.
They’re getting 200m a year to share that garbage pit with ai training.
Reddit was already heavily astroturfed before AI. There's no space on the internet where you can get as much bang for a buck with an influence campaign due to its centralized frontpage. The most obvious example is /r/worldnews, with millions of readers, a few thousand commenters and maybe 40k voters. To skew any discussion, you need at most 6k accounts, and that's giving you the kind of influence on American politics as would the frontpage of the NYT. You could hire real people for each account, and it would still be worth it.
I'm wondering how it goes with a lot of social media actually.
There were problems with the "algorithm"[0] before being intentionally gamed... but I feel like it's easier than ever to have an army of bots intentionally pointing a conversation.
I'll give you a brief example (we should not debate this case on HN but I'm talking specifically about a hypothesis that can be drawn).
In the UK, the streets are alight with the news of the murder of a young white boy at the hands of a Sikh boy. In the UK, historically, the Sikh community has been seen as a bit of an outlier in terms of how well they integrated and how tolerant they are. They are, for all intents and purposes "model immigrants".
The issue was, when the police were called to the scene of the stabbing, they immediately arrested the barely conscious boy on the ground, who had been stabbed and was bleeding from the mouth- handcuffing him and dismissing his pleas on the basis that he might have been making racist remarks.
The people in my circles are furious at the police for doing this.
However, it seems like there is an army of bots who keep trying to paint it as if it's a problem with Sikhs, or that their religious rights were the problem.
(for context, British law permits baptised Sikhs to carry a dull Kirpan knife - but this knife was not used for the incident above, it was a separate and still illegal knife to carry).
So now, online, instead of people talking about police indifference or the issues we have when we see race before trying to assess the situation: we are instead caught in a conversation about banning Kirpans for Sikhs (they have never been used in any crime from what I can tell) or how the Sikhs should feel bad or whatever.
Which is wonderful for a politician who wants to dismiss the criticism against the police as racism, but not so good for holding the police and media to account for the situation and the attempted coverup.
By being able to control the aggression, they are able to control the response, which drowns everything else out, and with LLMs the barrier is stupidly low to do this.
Creating division for your own reasons has never been cheaper or easily accessible.
I think you missed the part of this where I talked about specifically not litigating this case on this forum.
I was gesturing that the sentiment online is being reframed from one of the police handling of the situation (whatever the outcome you personally think it should have gone down as) and the media coverage- to instead the "issue" of Sikhism and religious rights in the kingdom, despite those aspects playing literally no role in the situation that transpired.
I was speaking specifically about how little cost it is to do this reframing now.
I don't care whether you're litigating it or not, I care about reporting the facts so that whoever reads the thread is better inform than they would be otherwise
There’s a bunch of conflicting information, for example your fact would be further clarified by saying:
A) He was described by the father of the killer as having a mouthful of blood when police arrived (per bodycam footage).
B) He stated he could not breathe 9 times and that he had been stabbed 5 times (as per bodycam footage)
C) He was immobilised
D) During the 999 call an ambulance was requested. (as per the recording to 999)
His statements were dismissed by police and he was handcuffed before any later assessment of his claims, the handling of the officer greatly accelerated the death of Nowak (as per the coroner report).
"we could find a nail for this hammer... or we could just hammer everything until we find the nail or make too much money hammering for a bit then sell in may and go away".
Since we just asked you to stop breaking the site guidelines (https://news.ycombinator.com/item?id=48388447) and you've continued to do it, I've banned this account. If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. They're here: https://news.ycombinator.com/newsguidelines.html.
Edit: I suppose I'd better add that no, this is not because of your views. Thoughtful criticism is fine, and HN hosts plenty of thoughtful criticism of AI. Rather, it's because you've been crossing into personal attack repeatedly as well as breaking the site guidelines in many other ways (snark, flamebait, etc.).
Also, single-purpose accounts aren't allowed here, regardless of their agenda, and your account has unfortunately been turning into something close to that.
Yes, and to add, in case it's not obvious: in my experience the maintenance, mental (and emotional costs, call me sensitive) cost of bad code compounds exponentially the more hacks you throw at it
Now with AI, you're not only dealing with maintenance and mental overhead, but also the overhead of the Anthropic subscription (or whatever AI company) to deal with this spaghetti. Some may decide that's an okay tradeoff, but personally it seems insane to delegate a majority of development work to a blackbox, cloud-hosted LLM that can be rug pulled from underneath of you at any moment (and you're unable to hold it accountable if it screws up)
Call me naive, but I don't believe that I'm going to wake up tomorrow and ChatGPT.com and Claude.ai are going to be hard down and never come back. Same as Gmail, which is an entirely different corporation. I mean, they could, but it doesn't seem insane to use Gmail for my email, and that's way more important to my life functioning than this new AI thing.
I'm pretty sure that will be true with AI as well.
No accounting for taste, but part of makes code hard for me to reason about is when it has lots of combinatorial complexity, where the amount of states that can happen makes it difficult to know all the possible good and bad states that your program can be in. Combinatorial complexity is something that objectively can be expensive for any form of computer, be it a human brain or silicon. If the code is written in such a way that the number of correct and incorrect states are impossible to know, then the problem becomes undecidable.
I do think there is code that is "objectively" difficult to work with.
There are a number of things that make code hard to reason about for humans, and combinatorial complexity is just one of them. Another one is, say, size of working memory, or having to navigate across a large number of files to understand a piece of logic. These two examples are not necessarily expensive for computers.
I don't entirely disagree that there is code that's objectively difficult to work with, but I suspect that the Venn diagram of "code that's hard for humans" and "code that's hard for computers" has much less overlap than you're suggesting.
Certainly with current models I have found that the Venn diagram of "code that's hard for humans" and "code that's hard for computers" has actually been remarkably similar, I suspect because it's trained on a lot of terrible code on Github.
I'm sure that these models will get better, and I agree that the overlap will be lower at that point, but I still think what I said will be true.
I wouldn't expect so. These machines have been trained on natural language, after all. They see the world through an anthropomorphic lens. IME & from what I've heard, they struggle with inexpressive code in much the same way humans do.
What do you think about the argument that we are entering a world where code is so cheap to write, you can throw the old one away and build a new one after you've validated the business model, found a niche, whatever?
I mean, it seems like that has always been true to an extent, but now it may be even more true? Once you know you're sitting on a lode of gold, it's a lot easier to know how much to invest in the mine.
It hasn't always been true, it started with rapid development tools in the late 90's I believe.
And some people thought they were building "disposable" code, only to see their hacks being used for decades. I'm thinking about VB but also behemoth Excel files.
I guess the question is, are the issues not worth fixing because implementing a fix is extremely expensive, or because the improvements from fixing it were anticipated to be minor? I assume the answer is generally a mix of the two.
Someone has to figure out how to make the experiences of the two generations consistent in the ways it needs to be and differ only in the ways it doesn't still.
The tl;dr of this is that I don't think that the code itself is what needs to be preserved, the prompt and chat is the actual important and useful thing here. At some point I think it makes more sense to fine tune the prompts to get increasingly more specific and just regenerate the the code based on that spec, and store that in Git.
> At some point I think it makes more sense to fine tune the prompts to get increasingly more specific and just regenerate the the code based on that spec, and store that in Git.
Generating code using a non-deterministic code generator is a bold strategy. Just gotta hope that your next pull of the code slot machine doesn’t introduce a bug or ten.
We're already merging code that has generated bugs from the slot machine. People aren't actually reading through 10,000 line pull requests most of the time, and people aren't really reviewing every line of code.
Given that, we should instead tune the prompts well enough to not leave things to chance. Write automated tests to make sure that inputs and outputs are ok, write your specs so specifically that there's no room for ambiguity. Test these things multiple times locally to make sure you're getting consistent results.
> Write automated tests to make sure that inputs and outputs are ok
Write them by hand or generate them and check them in? You can’t escape the non-determinism inherent in LLMs. Eventually something has to be locked in place, be it the application code or the test code. So you can’t just have the LLM generate tests from a spec dynamically either.
> write your specs so specifically that there's no room for ambiguity
Using English prose, well known for its lack of ambiguity. Even extremely detailed RFCs have historically left lots of room for debate about meaning and intention. That’s the problem with not using actual code to “encode” how the system functions.
I get where you’re coming from but I think it’s a flawed idea. Less flawed than checking in vibe-coded feature changes, but still flawed.
> Write them by hand or generate them and check them in?
Yes, written by hand. I think that ultimately you should know what valid inputs and outputs are and as such the tests should be written by a human in accordance with the spec.
> Less flawed than checking in vibe-coded feature changes, but still flawed.
This is what I'm trying to get at. I agree it's not perfect, but I'm arguing it's less evil than what is currently happening.
Observability into how a foundation model generated product arrived to that state is significantly more important than the underlying codebase, as it's the prompt context that is the architecture.
Yeah, I'm just a little tired of seeing these pull requests of multi-thousand-line pull requests where no one has actually looked at the code.
The solution people are coming up with now is using AI for code reviews and I have to ask "why involve Git at all then?". If AI is writing the code, testing the code, reviewing the code, and merging the code, then it seems to me that we can just remove these steps and simply PR the prompts themselves.
You don't actually need source control to be able to roll back to any particular version that was in use. A series of tarballs will let you do that.
The entire purpose of source control is to let you reason about change sets to help you make decisions about the direction that development (including bug fixes) will take.
If people are still using git but not really using it, are they doing so simply to take advantage of free resources such as github and test runners, or are they still using it because they don't want to admit to themselves that they've completely lost control?
> are they still using it because they don't want to admit to themselves that they've completely lost control?
I think this is the case, or at least close.
I think a lot of people are still convincing themselves that they are the ones "writing" it because they're the ones putting their names on the pull request.
It reminds me of a lot of early Java, where it would make you feel like you were being very productive because everything that would take you eight lines in any other language would take thirty lines across three files to do in Java. Even though you didn't really "do" anything (and indeed Netbeans or IntelliJ or Eclipse was likely generating a lot of that bootstrapping code anyway), people would act like they were doing a lot of work because of a high number of lines of code.
Java is considerably less terrible now, to a point where I actually sort of begrudgingly like writing it, but early Java (IMO before Java 21 and especially before 11) was very bad about unnecessary verbosity.
> If people are still using git but not really using it, are they doing so simply to take advantage of free resources such as github and test runners,
does it have to be free to be useful? the CD part is is even more important than before, and if they still use git as their input, and everyone including the LLM is already familiar with git, whats the need to get rid of it?
there's value in git as a tool everyone knows the basics of, and as a common interface of communicating code to different systems.
passing tarballs around requires defining a bunch of new interfaces for those tarballs which adds a cost to every integration that you'd otherwise get for about free if you used git
A series of tarballs is really unwieldy for that though. Even if you don't want to use git, and even if the LLM is doing everything, having discrete pieces like "added GitHub oauth to login" and "added profile picture to account page" as different commits is still valuable for when you have to ask the LLM "hey about the profile picture on the account page".
Also, the approach you described is what a number of AI for Code Review products are using under-the-hood, but human-in-the-loop is still recognized as critical.
It's the same way how written design docs and comments are significantly more valuable than uncommented and undocumented source.
Because LLMs are designed as emulators of actual human reasoning, it wouldn't surprise me if we discover that the things that make software easy for humans to reason about also make it easier for LLMs to reason about.
It’s also possible to sell chairs that are uncomfortable and food that tastes terrible. Yet somehow we still have carpenters and chefs; Herman Miller and The French Laundry.
Some business models will require “good” code, and some won’t. That’s how it is right now as well. But pretending that all business models will no longer require “good” code is like pretending that Michelin should’ve retired its list after the microwave was invented.
Those high end restaurants are more like art and exploration of food then something practical like code. The only similarity is maybe research in academia. There's not real industry uses of code that's like art.
I used the extreme of the spectrum, I can’t imagine you’re arguing that food is binary good / bad? There’s a litany of food options and quality, matching different business models of convenience and experience.
Research in academia seems less appropriate because that’s famously not really a business model, except maybe in the extractive sense
There's no equivalent of experience and art in code. Writing code is not expressing your self, and you don't pay for pushing the limits and experimenting with it. That's what high end restaurants are along with service they provide.
As far as good or bad, how food is made is irreverent to the outcome if it's enjoyable.
Still, talk about "good" code exist for a reason. When the code is really bad, you end up paying the price by having to spend too more and more time and develop new features, with greater risk to introduce bugs. I've seen that in companies in the past, where bad code meant less stability and more time to ship features that we needed to retain customers or get new ones.
Now whether this is still true with AI, or if vibe coding means bad code no longer have this long term stability and velocity cost because AI are better than humans at working with this bad code... We don't know yet.
Not only true but I would guess it's the normal case. Most software is a huge pile of tech debt held together by zip-ties. Even greenfield projects quickly trend this way, as "just make it work" pressure overrides any posturing about a clean codebase.
A cornerstone of this community is "if you're not embarrassed by the first release you've waited too long", which is a recognition that imperfect code is not needed to create a successful business. That's why ShowHN exists.
It depends on the urgency. Not every product is urgent. CC arguable was very urgent; even a day of delay meant the competitors could come out with something slightly more appealing.
Most of their products are so large that you can easily find parts with very bad and parts with excellent code. I am not sure a whole ERP product could work with all very bad code, though.
