Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Everything you highlight is basic level knowledge to work with databases. If you don't know that, you should either learn or not go near them.

However I will agree that "scalable" is starting to get a little complicated with all the various scale up/out options and strategies out there.



You can buy, off the shelf from any number of vendors now, a single box with 512G or more main memory, 48 or more processor cores, and several teras of storage, with the option of SSDs. Like, click click click and a courier delivers it a couple days later, that easy. Running Postgres say, the workload these things can support is insane. Far, far more than 99% of people worrying about "scalability" will ever actually need. This is a solved problem.


True, most corporations are under 1TB of data (per project) so not a big deal.

However scalability is about more than just a big box though. Scale up has always been easy, and should be the first thing to do, but there's still a new class of companies/projects that need HA for 100% uptime and scale-out so they can have relational access across PBs of data.

This is usually done with Hadoop or other "big data" tech but there's no reason regular relational databases can't scale up to this and offer all the benefits and tooling they come with. Closing that gap is still very tough these days without 3rd party vendors or custom extensions.


Everyone even contemplating Hadoop should read this first http://aadrake.com/command-line-tools-can-be-235x-faster-tha...


That's a pointless article. When you have a few GB of data, you can use anything. Command line tools or SQLite or anything in between would've worked fine.

Realistically, anyone contemplating Hadoop or anything bigger than traditional relational databases is dealing with the hundreds of TBs to PBs range which is not going to work with some unix tools.


No, what's pointless is the typical Hadoop workload. Sure there are some people who need it but I'll wager they're not even 1% of the people using it.


Ok, but this is a random tangent. People will always use things they shouldnt, that's their problem.

The point of the thread is that there's still no easy scale-out solution that has come along for relational databases to provide for data that typically gets put into proprietary data warehouse or hadoop installations. Citus data and memsql might get close but the whole industry is still far behind where it should be for this.


I also would guess that for 95% of databases it would be enough to just run a query/performance analyzer and follow the advice it gives you.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: