Well, O(100GB) is where I start thinking of "big data". Processing 100 GB is slow in a single "normal" machine (depending on what, sure). Between 10 and 100 I usually do it in a cluster since it is less hassle (even if I could process it locally with some tweaks or patience). Less than 10 is usually locally run unless I'm already computing something else
100GB can be handled without problems by mysql or postgre on a single node (yeah maybe not a laptop).
You need to tune your database and take the right design decisions, but it's still less work than setting up a distributed system.
Is significantly easier than making PG work easily at the 100GB scale in my experience. spark-ec2 is a script that ships with Spark to make it easy to set up a cluster in AWS.
I have a custom EMR launcher to get a cluster with Spark, Zeppelin and Hive. So yes, much faster for ad-hoc than tuning Postgres, basically it is a matter of if I need ad-hoc reporting/analysis or need it for normal operations. For 100 GB, regular, I'd just drop it into our running Redshift and let it eat it.
