Do you have a write-up of your experience by any chance? We're primarily using D... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

mziel on Nov 7, 2016 | parent | context | favorite | on: Apache Spark: A Unified Engine for Big Data Proces...

Do you have a write-up of your experience by any chance?

We're primarily using DataFrames (since we're doing lots of ML processing) and Pipelines API, but I was looking forward to switching to Dataset API.

sandGorgon on Nov 8, 2016 [–]

We don't do a lot of ML processing..so my experience is very limited.

Do you see yourself getting better performance with the Dataframes+ Pipelines infra...or is it simply a better mental model.

Consider applying for YC's Winter 2027 batch! Applications are open till November 2.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact