This is a harder problem than it sounds, and something I've given a lot of thought to. I think the underlying issue is that all machine learning was discovered through a combination of applied math and intuitive ideas/models. Without the intuitive model no one would have thought to discover the method, and without the math the intuitive idea would be a pipe dream. Both are fundamentally linked, and it's a bad idea to separate them.

For example, one of the most basic and oldest statistical methods is linear regression. The first thing anyone will tell you when they are teaching it is the basic "idea and model" - finding a line that fits a scatter plot (and then extending that idea). But this doesn't give you a real understanding of linear regression: where does that line come from, and why does a clean, algorithmic solution exist? Why does the standard solution often lead to numerical errors, and why is regularization a valid solution?

These questions require an increasing amount of math, but they are essential to really understanding linear regression. I agree that some textbooks just throw you a wall of math, but many actually do a solid job of explaining what is going on as they do so.

If you don't want to know the details of machine learning methods, which are inherently mathematical, you might as well as just remember the names of libraries that implement the solutions for you.

Completely agree. However one thing that I think is missing in most books with a lot of math is the opportunity to use programming to help teach and communicate the math in question.

A great example of a book that communicates abstract mathematical concepts via algorithms is the Little Schemer. Ironically, it doesn't even set out to communicate the math, but actually just uses the math to communicate other programming ideas like recursion. That, however, doesn't diminish the fact that it demonstrates ways to teach math through programming and algorithms.

Another book which also does a good job at using programming to demonstrate more concrete math is Allen Downey's "Think Stats" book. All through the book, you learn the mathematical concepts of statistics through hands on programming activities.

There definitely is a chicken and egg problem in areas like machine learning because unlike the above resources which have only one layer of abstraction to cross, machine learning presents two layers of programming<->math abstractions to cross for most people who decide to learn it. To really understand and apply machine learning you need to understand the math and models behind it. However the math and models are presented in pure form that makes it difficult to grok unless you arrived at the resource with a classically-trained mathematical background. I would hope that given that the target market for such learning resources it not mathematicians but programmers, that such learning resources would present content to help you arrive at the math from a programmer's point of view.

You are right, it is indeed difficult and I never meant to say it is easy. My tripe was with the people who intentionally make it difficult. The thing is that you can learn ML with minimal (not minimum) math and that is what should be tried to achieved. I would be curious to know the textbooks you talk about - I have read quite a lot of those and struggled with them.

For example - Linear regression is just brilliantly tackled by Andrew Ng in his ML course (CS229 at Stanford Lecture Notes - not Coursera).

One trouble is that the "idea" you've just communicated isn't actually the idea behind linear regression. The idea behind regression is this.

You're trying to make a prediction for some number you care about - let's say the value of a given stock price. You've come up with a set of hypotheses about which characteristics might help you make that prediction. Moreover, you have a set of examples that you've witnessed in the past, and you want to learn from that experience.

Using linear regression, you can test those hypotheses. You turn those characteristic features into a quantifiable number themselves. Linear regression is simply the name we give to the process of testing whether there's any validity to your hypothesis. If that hypothesis is true and you've discovered what makes the stock go up, then the corresponding feature will be given a high absolute coefficient. You'll also know whether it's an indicator of the stock going up or down, based on the sign of that coefficient. There's no math involved - you're testing your own intuition about how to make predictions.

The idea behind linear regression isn't "finding a line that fits a scatter plot." That's still math and it's still unhelpful to many people with serious, real-world applications. It's just an abstraction of the math that happens to leave out the formal representation.

To really communicate ideas in application, you need to move past the math entirely, and get to how it ties into people's judgments about data that they know well and have experience with, and show them that the intuition they can bring to the table is valuable (for feature determination). Otherwise, even scatter plots will often shut people out, because they "aren't good at math."

Hey there, I'd like to follow up on this point (and some other comments you've posted on HN). Do you mind if I email you?

Personally, I find Bishop's book to hit on a good balance between developing intuition and presenting the math to give precision to understanding (the opposite of this would be Murphy's book--the math seems too esoteric and disjointed to be useful). I have a hard time feeling confident I thoroughly understand a concept without having precision in the presentation of the math behind the concept.

There is certainly something to be said for courses and books that can present a complex idea without requiring a graduate-level degree of math literacy. But at the end of the day, ML is a subfield of mathematics so not having a thorough grasp of the math underlying it will definitely hinder your understanding.

Yeah, of what use can mathematics be, when you can just study "ideas and models"...