There's a great quote by some bodybuilder. "Everybody wants to get big, but no one wants to lift heavy weights."
Paraphrasing this to data science: "Everybody wants to have software provide them insights from data, but no one wants to learn any math."
The top two comments here illustrate this perfectly. Anyone who is serious about learning data science will read this book and will not shy away from learning math. You can also learn about data pipelines, but that's not a substitute for what's in this book.
There are also a variety of other algorithmically focused machine learning books. They are also not a substitute for this book.
Also consider that those who shy away from learning this material aren't exactly wrong. They're making an optimal choice given their skillset.
I've been a DS for the past 4 years, used to be a risk quant at the bank before that. So the territory is familiar, especially as a math major. But my colleagues who are enamoured by this stuff often underestimate how dry and boring it can get to a non-math person. I took 2 of my colleagues under my wings and after getting their feet wet with a few DS projects, asked them what they'd like to do. Their response was basically - we want to get back to engineering. The thing with software engineering is that you can iteratively make progress. You can come in knowing zero Ruby and zero JS on day one, and cut and paste from stackoverflow and get by, and slowly, in the next month or so, figure out enough to do a good job. Unfortunately for DS that's not a hot option. The fundamentals are best learnt as theorems sitting in a classroom, not as Spark APIs. So the ability to invoke a classifier api or do an svd with Spark isn't actually giving you any intuition as to what's going on under the hood. Soon people realize this, and say hey this stuff is fine but it isn't really my thing. Which is ok.
I feel that what you describe is largely a pedagogical fault of the community though: most of the people who understand the topic gained their education in a particular way, and just haven't thought very hard about how they might make that process better using new technology. This isn't to call anyone out, of course, because that exact process happens to literally every field, and even within mature fields as new paradigms or technologies are developed.
I think there's a lot of room for mathematics education to be revamped to be more iterative and exploratory, which I think would make it appeal to a wider audience. I think that the mathematics community is actually beginning to pick up on this, with the publishing of python notebooks which contain increasingly sophisticated models as you read in to them and provide controls for readers to manipulate the parameter space of the models to explore the consequences of claims, theorems, etc. I think this kind of exploration of the numbers is a better way to learn a lot of ideas than the way I was taught in class, particularly things like statistics, where you can see distributions change as you manipulate parameters.
I figure that in a few years, as people my age who were mostly taught using older methods, but got a little exposure to explore-via-computer learning in university mature, we'll see an increasingly "internet-y" version of teaching methods appear, where there's a mix of video lectures, reading materials, discussion posts/blogs, and interactive models.
Computer programming and hobby crafts are among some of the first fields to make that transition, traditional academics are somewhere in the middle of the pack, and specialized topics like data science are keeping pace with the wider academic community.
tl;dr: I think your point is really just a temporary pedagogical fluke, as we take time to transition our teaching methods.
I complete disagree. Dr.Neal Koblitz (inventor of elliptical curve cryptography)spent a good bit of his life fighting this obsession with bringing the computer into math so things can be visualized & manipulated. You should check out what he has to say.
https://www.math.washington.edu/~koblitz/mi.html
btw, I used to think like you before I got into math. Its a very valid albeit flawed viewpoint - to think that hey if only there was a javascript doodad somewhere so I could move a slider and manipulate the parameter space, I will instantly "feel" CLT in my blood instead of working through CLT as a dry formal math proof. The reality is, this sort of dumbing things down works maybe in 2D and best case 3D, but beyond that, its utility is rapidly diminished. Also, the results you get in the lower dimensions don't map into higher space. As yummyfajitas points out elsewhere, in a reasonably high dim space, all pairs of random points will be fairly equidistant, which is simply not the case for say 2D. So even the little bit of useful intuition you learn in the lower dims becomes quite useless as you scale up.
Proofs are best learnt by doing proofs. Math is not cinema.
You responded to a strawman of what I said, made no points that even remotely address or refute what I said, and further, decided that you were going to personally attack me. In fact, your link generally agreed with my point! (In summary: traditional reading and lecture methods have benefits; videos and other newer media can help (some) people, and give a new perspective; computers have a few useful behaviors; engaging imagination and exploration are better than rote learning.)
That's really kind of pathetic, and a disservice to your view.
I'd be happy to speak with you, though, if you want to actually read my comment and address a (charitable) interpretation of what I said, rather than your strawman one.
Traditional math education: a professor explains proofs of selected theorems on a blackboard and students are then required to memorize the proofs and reconstruct them during the examination. They are also required to solve a lot of problems that involve proving some consequences of the main theorems or doing some rote calculations using nothing but pencil and paper.
The advantage of this approach is that it works. You can acquire solid intuition this way. But it is arduous as hell and also pretty one-sided - problems are limited to what an average undergraduate can do by hand.
Now, when I think about using computers in math education I don't think of some "javascript doodad" with moving sliders. I imagine instead implementing an algorithm or doing a numerical experiment, learning about possible pitfalls in the implementation, running the algorithm on some real data, seeing it fail etc. I think this experience is pretty valuable too and can nicely complement traditional approach.
That's why that approach has been offered in courses traditionally called "Numerical analysis" that have been available to math students for... how long?
Taking the sarcasm out, your point is that such thing exists. Agreed. But usually it is just a single course, it does not cover everything. And why it is a separate course at all? Why should you study some field with pencil and paper and then (possibly, after a couple of years) study a subset of that field "numerically"?
As an example of what I am talking about, the book "Structure and Interpretation of Classical Mechanics" (a cousin to SICP) tries to teach classical mechanics with the help of computer programs. But this approach remains unconventional.
I should probably not have been sarcastic, especially when I agree completely about the importance of looking at math with a more constructive/computational point of view.
Now, I'm not a mathematician or a scientist of any kind, merely a dilettante textbook reader, but I'm not sure that approach is sufficient, or even most efficient to teach all subjects.
I think it arises naturally when you approach subjects where it's directly relevant: engineering, complexity theory, type theory, etc. and people who will need those insights will get them in due time.
But yeah, broadening the audience could help people get a better appreciation of the connections between pure and applied math.
This is one of the reasons that I really dislike the term "data scientist." Essentially the job is Analyst, which is something we've had for many years. It's not a bad thing, but there's no reason to embellish it into something it's not.
The primary reason I dislike this term, though, is because all scientists are data scientists. Data is how the whole thing works.
The problem is that industry has coopted the term data analyst to mean something more like data reporter or data descriptor. There often isn't a difference between business intelligence and data analyst. Many data analysts are using tools like Excel, and don't have good programming skills. A data scientist will generally be involved with prediction, and use more sophisticated tools than Excel. This isn't to knock on data analysts (or Excel), they do good and useful work. The proof is in the numbers - a data scientist makes a significantly higher salary than a data analyst. Similarly quantitative analyst got taken over by finance. The term statistician is actually pretty accurate (data science is basically applied statistics), but usually people don't call themselves statisticians unless they have a degree in statistics. With that said, the term data scientist is not at all well-defined, and the field is quite diverse.
I'm sorry, but I'm not sure that you're correct that a data scientist is a statistician who can write distributed systems. Problems have been solved for decades by people with quantitative training, from well before distributed systems (or computers) existed. Very roughly, a data scientist is a person with a combination of quantitative training and some programming skills, who is able to effectively analyze (and explain) different kinds of data, be they audio, text, census information, or whatever it might be. What we call data scientists today are the engineers, computer scientists, and mathematicians of yesterday.
I have no experience in Data Science. How much need is there for a distributed system? How many data points would one need to necessitate it? What if we had a billion data points. Would it be sufficient to run on a fast system overnight to crunch data?
According to me (note zintinio's reply - he disagrees), the etymology of the term is the following.
Back in the day, if you had small data you didn't need a data scientist. You needed a statistician. He'd do some shit in SAS/Python/etc, reading one CSV and writing another. Then the developers could run that on a beefy server with cron and push the CSV output someplace else.
At some point during the 2000's this stopped working - things became complicated and tangled enough that you couldn't just munge CSVs like this. You needed folks who understood the math well enough to come up with algorithms, and who also understood the computer science well enough to scale out. These folks were termed "data scientists". Banks call them "quant developers".
Very often you don't need a distributed system, or anything that isn't trivially parallelizable. Most of the time when I make money it's based on a CSV < 100GB - often it fits in ram. Nowadays I don't think it's misleading to call yourself a "data scientist" if you can only handle such data sets.
I recently learned that Facebook calls their business analysts data scientists. I guess it's just sexier.
Science is more than just data. A theoretical physicist is a scientist who does not necessarily care about data collection. Some anthropologists focus on case studies, rather than data, especially if they're just doing descriptive study rather than predictive.
100% agree. When I made the switch from technical business analyst to head down the road of data science, I had the option to use a year of postbac study to study computer science or math. the CS dept at the school wasn't fantastic, it was mostly in Java, and DS is moving so fast I wasn't certain what percentage of what I studied would be of any use in 5 years time. So I studied math. All the math I learned is still good as new. Still "true. Still useful. Many DS tools have come out since I finished that year. Many are Python based.
One thing I am constantly frustrated by is the lack of mathematical rigor that pervade DS tools, and programming in general. Seems like everyone is happy to be given some programmatic tool and all the mistakes that where programmed into it. "You just need to deal with that." "You'll get used to the syntax." Yeah, but, I'd rather not, since it's often completely arbitrary and specific to a given tool. And, well, sometimes just doesn't make sense in the way something that has mathematical rigor built in might.
But, I often feel when I bring this issue up I'm shouting down an empty hallway.
Part of the problem with DS is that there are known engineering solutions without good theoretical explanation, that is, we just brute forced some basic engineering solutions and keep hacking at them to get more performance, but don't have a good theoretical model of why those things do what they do. (Or have a theoretical model too complex to compute useful insight, which is another kind of problem.)
I think that this kind of approach is going to get us pretty far -- see how far we got on building bridges with no more sophisticated insight in to gravity than "things fall down; some things are heavy" or material science than "some things hard and not bend" -- but I think that ultimately, our longer term goals like AGI will require that we build theoretical models capable of explaining our current engineering progress and predict a new, deeper valley of progress to move to.
Of course, that's a lot like saying making progress on physics paradigms requires explaining high temperature superconductors (or other currently open problems). It's actually pretty normal for a field to be a mix of engineers being ahead and theorists being ahead.
Mathematical rigor is really a sign that you're in established territory and explorers have moved on, and much of data science is still under heavy exploration.
There are tons, tons, of examples of "established territory" in CS that do not exhibit "mathematical rigor". When something works in CS people seem to say "good enough" and use it. In mathematics (or as you pointed out, in physics) it becomes an "open question" and people begin the hard work of explaining the phenomenon by applying rigor. In maths, it's a constant march to push the boundaries, where the boundary is defined as that which is interesting but not yet well defined. In CS it seems that the boundary is that which is not yet a solved problem, where "problem" is something that needs getting done. Once, in CS, we can get it done, people move on. It's not often that in the enterprise CS process (academic is a whole other beast) it is assumed that there should be an application of rigor or an attempt to well define "solutions" before moving on. There is simply the accumulation of technical debt. But, in my opinion, technical debt has amassed at the systemic level to such a point where it's starting to look like a ponzi scheme. Sure, it works, so long as we keep throwing good money after bad. But, stop investing, and one can see it for what it is. In maths on the other hand, step away from it all for a year or two, and when you go back everything is still just as valuable and beautiful.
edit: what i would love to see is a "category theory" for programming languages / paradigms such that moving between them is well defined. it boggles my mind that translating between two programming languages isn't trivial. if both are well defined, one should be able to translate one to the other precisely given one is willing to define the translations. there should be zero guess work or heuristics in the process.
It is baffling that machine translation on human language is approaching human accuracy but it is still far behind on programming language, given that it is supposed to be more closed/matching to metal/machine.
I find that the opposite of baffling: human language is information sparse and redundant; computer languages are information dense and, by choice, are made as unredundant as possible. It just seems like trying to hit a big fluffy cloud on one side and a bald tree on the other. Obviously the big fluffy thing is easier than the spindly, narrow one.
How does machine translation do at poetry, in the sense of capturing the figurative meaning and stylistic elements, not just the literal meaning of the tokens?
I love this quote. For accuracy purposes, it is from Ronnie Coleman, former Mr. Olympia and goes more like
"Everybody wants to be a bodybuilder, but nobody wants to lift no heavy-ass weights...I do it though"
I never said to not read the book or learn the maths. They are obviously important. I apologize if I sounded dismissive. But I've met people in my field that are great at the maths (which should be a given), but can't execute the analysis because they can't do simple "janitorial" tasks. Perhaps this is a problem more prevalent in academia. I think there should be equal attention paid to both aspects of the job.
I absolutely agree that a data scientist must be a good software engineer also. But there are lots of books on software engineering. There are lots of books on algorithms.
As far as I'm aware this book is one of a kind. This book covers ground that nothing else does, at least not in any comprehensive way.
> Anyone who is serious about learning data science will read this book and will not shy away from learning math.
The problem is not math. The problem is the way math is explained. Most books do not go into the intuition and are very dry. People generally shy away due to the dryness of the material, not the content.
One of the major points that this book is trying to convey is that your intuition is wrong. You need to burn it down, learn the formalism, and maybe develop new intuition based on that. Even then you need to recognize that intuition can be wrong.
An N-dimensional vector space is not like R^3. A monad is not like a burrito, and >>= is not like chipotle forgetting to put chicken in your burrito, opening it up, adding chicken and wrapping in a new tortilla.
Math is a new thing rather than the same old thing with new syntax. It needs to be understood on it's own terms. If you are serious you'll learn it.
I agree with your sentiment that math is important, but it is also true that math is not taught well. Just briefly glancing at chapter 2 the first part, the author presents the law of large numbers out of the blue, then just goes on to present proofs of it. There is no clear discussion of why this law is important, when you can use it and so on. If you contrast how wikipedia explains it:
> In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed.
This book:
> If one generates random points in d-dimensional space using a Gaussian to generate coordinates, the distance between all pairs of points will be essentially the same when d is large. The reason is that the square of the distance between two points y and z ...
I always get a eerie feeling for people who excessively stress on formalism. It almost goes to the point where I feel they wouldn't want to share their knowledge or intuition.
They are trying to help you get a correct intuition. If you have never approached things this way, there are almost certainly very large holes in your understanding. Not to say that everyone needs to be formal all the time, but you need to sometimes.
> If you have never approached things this way, there are almost certainly very large holes in your understanding.
When anyone tells me that X is the ONLY way to do it, almost exclusively I have found them wrong - beyond data science. Formalism is a means of communication, not the end. You can always communicate ideas without formalism.
FYI, I am a Ph.D Computer Science and a practicing Data Scientist for many years now.
You are essentially saying - the way I think of math is how it should be learned. Sorry but if you're serious, you will find a way to explain your mathematical intuition.
The problem is detailed in this wonderful essay by Paul Lockdhart
We can explain mathematical intuition. We just can't explain it as quickly or in as interesting way as you seem to want.
Here's a simple concept: all polynomials with coefficients in a number system (field) have a solution (possibly in a larger field).
Do you know how hard it is to convey what exactly this means and what the consequences are? I like Lockhart's essay. It resonates with me. However, I don't see his ideas being useful in terms of learning advanced topics. At some point one has to get their hands dirty and slog through the material. Intuition will come with experience.
It is shared. It's just that everyone has to go through to mental work to truly understand. It takes me a semester to convince my students that 3x+5x=8x and that this is true because of the distributive property. And this is why ax+2x = (a+2)x. And the reason we can't simplify further is because our language does not have a word for (a+2) but it does have a word for (3+5). But I know that after a semester of teaching this most still don't actually understand the distributive property. It takes a lot of effort on the part of students for the concept to click.
Most math textbooks focus too much on proofs, as does this book. Data science is a broad field, but for most use cases formal proofs are not really needed. I would prefer a text that focuses on examples and how/when to apply the theorems, rather than here is a theorem and here is a proof. Oh and here is another theorem and here is another proof. That isn't to say proofs are not important, but the focus of data science is usually on applied statistics.
I would prefer a text that focuses on the java standard library and shows how/when to apply a java.util.List<T>. That's far better than a Data Structures and Algorithms book that teaches you how you'd implement a linked list or other complex data structure.
90% of the time I use the java standard lib. Sometimes I need something almost the same, but a bit different. The difference between a middling developer and a good one is what happens in that 10% of the time.
The same applies to proofs. What happens when you need something that's almost like the theorem, but not exactly? Can you tweak the proof to apply to your case, or alternately recognize that it can't be fixed and you need to do something different?
I agree that knowing and understanding the fundamental math is important, but I think proof based texts are not a good way to learn math. A lot of people get turned off from math because of how it is taught. Personally I learn best by doing problems and seeing examples. Proofs are useful, but I want to know why the theorem is important before I learn how to prove it is true.
I don't think anybody is saying that proofs don't matter, or that they aren't important. But it looks like you're choosing to focus on uncommon edge cases. I think a lot of people would question that line of thinking. It's almost like the old saw about how you don't need a degree in mechanical engineering in order to effectively use a car. But if you need something "almost like a car, but also kinda like a bulldozer" then you're out of luck. OK, how often does that scenario come up? Enough to care about? For most people, the answer is a resounding "no".
Likewise, one can certainly apply a lot of mathematical techniques in a lot of situations, without proving new theorems. So there are edge cases... not the end of the world. Deal with it when it comes up.
Most math textbooks focus too much on proofs, as does this book.
My problem with (most) maths texts isn't the proofs in and of themselves, it's just that they make so many assumptions about what you already know, and don't always state those assumptions clearly. And then too many maths texts, to me, fail to include enough expository text to explain the math and provide more of an intuition around what's going on.
That said, I also agree with you that more examples in terms of applications would be nice. Learning really abstract stuff in isolation is OK, but it's always nice to have a few (or more than a few) examples of how to apply the technique to something concrete.
> Most well-written textbooks list the assumed background knowledge in the preface/introduction.
Most actually-existing books simply write that the student should be "mathematically mature", by which they mean that the student should know all of multivariable and vector calculus, most of differential equations, much of optimization, and some amount of analysis. And also be able to do proofs, and also have good intuition.
In short, most "applied" math textbooks aim themselves at Master's degree students or beginning PhD students in math itself, to teach material which can actually be taught to anyone with a bachelor's in science or engineering (except possibly computer scientists, whose continuous maths degrade because we never use them).
Ironically, the first math textbook I've ever picked up that didn't assume far too much background was my real analysis textbook, because analysis teachers assume that they are teaching the gateway course to "real math" and have to educate little babbies who only just got done with calculus.
If you want to learn something difficult that's above your level, and learning that something includes learning a prerequisite subject that's also above your level, you simply start at the node in the DAG that's at your level and work through the prerequisites if you're serious.
If you understand a mathematical theory, then you should be able to figure out on your own when that theory is applicable and how to apply it. So I can't help but read your comment as “I don't want to have to learn”.
You are being condescending and over simplifying the issues, try to not rush to judgment. I have spent a lot of work and time learning how to apply machine learning, so your comment was offensive and wrong. Applying mathematical theory is not just a matter of understanding the theory - in the real world there are a whole lot of reasons why you won't be able to apply the theory, even if you understand it perfectly. You will always have to make assumptions and approximations. I like learning, but I prefer learning for a purpose. Proofs don't really give you a practical understanding of the theory.
> You will always have to make assumptions and approximations.
The ability to determine when an approximation is good enough comes from a solid understanding of the underlying theory. For example, we understand that special relativity doesn't invalidate classical mechanics, because, in the limit, as speeds and energies become arbitrarily small, their predictions coincide. But, if all you have is a bunch of computational recipes, even the tiniest unexpected thing renders you unable to solve problems.
Anyway, I'm not saying that examples of applications are a bad thing. But proofs are indispensable. Even for “intuitive” people, proofs are necessary confirmation that their calculations will match their intuitions.
You bring up an interesting example in special relativity. I am a physicist, and there isn't really a "proof" in the mathematical sense of special relativity. Googling it provides this quora answer:
https://www.quora.com/What-is-the-proof-for-Special-Relativi...
There is no proof of special relativity, we believe it because it makes experimentally verified predictions. Your example of limiting cases of special relativity is how I wish statistics texts were taught - it isn't based on the "proof" of special relativity. Of course I know when special relativity is applicable and when it isn't (when beta = v / c is close to one, special relativistic effects are important, and gamma = 1 / sqrt(1 - beta*beta) indicates how good the approximation is).
But I do agree that proofs are useful and needed, I think that complicated proofs could easily be placed in appendices and looked at after understanding why the theorem is relevant. Of course this is just my personal preference.
> There is no proof of special relativity, we believe it because it makes experimentally verified predictions.
Right, that's because special relativity is a scientific theory. A scientific theory has two components: a mathematical theory (in which you can perform a priori physically meaningless calculations) and a physical interpretation (which turns the results of such calculations into predictions).
However, statistics isn't a scientific theory. It's just math, and like all math, it's about itself and nothing else. It doesn't make sense to “experimentally confirm a mathematical theory”, because, without a physical interpretation, math doesn't make any predictions about the real world.
> I think that complicated proofs could easily be placed in appendices and looked at after understanding why the theorem is relevant.
Then you're looking for books on applications. That's fine. But a book whose subject matter is a mathematical theory (it even has “foundations” in the title!) can't relegate proofs to appendices.
Are there any MOOCs that cover this math? I suspect there's only so far away from introductory material Coursera and Edx can get and have enough students to justify the costs.
I have a PhD in Biomedical Engineering, and am working as a DS for the past several years, and I can tell you that there is a hard ceiling for many of us out there, because we don't have enough of a math background (I'm looking at you "Real Analysis") to ever really understand the "deep theory" that someone with a PhD in Statistics or Math (or hell even a BS in Math) can have.
However, having said that, I still consider myself a very productive DS, and I get stuff done. I'm not going to ever be on a research team at Google, but those aren't the only kinds of DS jobs out there.
to your point about analysis, the folks with a CS education here (myself included) should ponder this quote from the intro: "...we have written this book to cover the theory likely to be useful in the next 40 years, just as an understanding of automata theory, algorithms and related topics gave students an advantage in the last 40 years. One of the major changes is the switch from discrete mathematics to more of an emphasis on probability, statistics, and numerical methods." some of the key stuff in statistical learning isn't even real analysis, it's functional analysis (usually two courses later).
You are right, but an important part of providing insights from data is actually providing the insights, meaning the interface problem between the data science and users is crucial.
For many industries, this problem is not well-solved at all. And it's not trivial. It's one thing to have great models, it's another to have a good way of making people smarter with it.
I really appreciate your viewpoint, a friend and I had a discussion about this exact problem last night. Nobody wants to dirty themselves in the details, but that's precisely where you need to go. Also, you have a very interesting blog, you've now gained another reader.
THANK YOU for posting this comment. I absolutely LOVE these two quotes, and can't tell you how many times I could have used them over the years.
These two quotes are now a permanent part of my arsenal for responding to busy individuals who say they "want to understand" a new technology or idea, but what they actually want is for the new technology or idea to be explained to them using only simple concepts with which they are already familiar, so they don't have to learn anything new.
Paraphrasing this to data science: "Everybody wants to have software provide them insights from data, but no one wants to learn any math."
The top two comments here illustrate this perfectly. Anyone who is serious about learning data science will read this book and will not shy away from learning math. You can also learn about data pipelines, but that's not a substitute for what's in this book.
There are also a variety of other algorithmically focused machine learning books. They are also not a substitute for this book.