I had a few questions about the actual implementation of this stuff. I took the coursera ML course, so much of the terminology and techniques are familiar after that, but Professor Ng structured the exercises in the course around Matlab/Octave and suggested using one of these tools for a first-pass solution when implementing machine learning problems.
Have you used Matlab much in your work? How does the performance (and libraries available) compare with Python?
Also, does anyone know a good resource for finding good, high performance ML libraries for other languages (Ruby, C++, etc.)?
matlab and python + numpy + scipy + matplotlib give the same kind of performance for the same level of features and ability to do quick exploratory prototyping with high level, vector based datastructures.
However one is free (beer and freedom) and multipurpose (you can preprocess string data and expose your model with a HTTP API) while the other is much less so.
In python + numpy you should always profile and if the bottleneck is in python interpreted code (rather than a low level numpy call) you can always rewrite the offending python loop in cython.
I used to prototype with Octave but I actually find I"m much faster in python. I push my tests to EC2 and just use cProfile to make sure I'm not doing anything silly before hand. I parallelize with celery, but I've also used pp with some limited success. Performance is sufficient for my needs currently but not for production or real-time.
One of the most famous collections/frameworks for development and usage of (traditional) machine learning algorithms is weka [1]. It is written in java, and actively developed by the machine learning group of the University of Waikato. They also seem to have a forum to discuss, which should be a good resource given that a lot of their users are actually researchers.
The very same group is also developing another framework (also in java), MOA [2], that is oriented towards "big data": the algorithms used in there are specifically tailored for huge datasets and/or realtime (streaming) machine learning.
Have you used Matlab much in your work? How does the performance (and libraries available) compare with Python?
Also, does anyone know a good resource for finding good, high performance ML libraries for other languages (Ruby, C++, etc.)?