(I'm no expert at software patents so take what follows with a healthy dose of NaCl)
It seems to me the difference is a question of "real world implications". For sorting integers using quicksort, yes that's just a mathematical algorithm: an algorithm that performs operations on mathematical entities. The moment you apply that algorithm to something that affects the real world, that's when it becomes patentable. E.g. sorting restaurant recommendations by applying quicksort to the individual rating. This is no longer simply a mathematical algorithm; the entities being manipulated have a 1:1 correspondence with entities in the real world. It is recognizing the usefulness of this particular correspondence between mathematical entities and real world results that is patentable.
I'm no expert either, but I will take offense at your statement. It's everything I think is wrong with the patent system.
Sorting is a mostly solved problem. You take a group of numerical things and end up with a group of numerical things that are now sequential. You can either do it slow, fast, or stupidly.
If you have the example of ratings of restaurants, you're going to have some kind of group of numbers. If you want them sorted, you use a predefined and should-be unpatentable algorithm and sort the damned list. But your business model shouldn't be sorting a predefined list, because that's a really stupid business model and any fucking person can do it. Your business model should be to find a unique and novel way to rate restaurants (social graph, Zagat, number of cockroaches found in soup, etc.). The outwards facing UI of that rating system should be entirely disconnected from what your patent does. The order of that list is, yes, directly connected to how you rate restaurants, but the end result itself is not exactly how you rate them, it's just representative of the end result of your rating.
Think of Page Rank. Anyone can crawl the web and get a list of pages on the internet. Anyone can then sort that list by whatever means they want. It's not the sorting that is patentable, it's the association of some qualifier into a numerical representation of which gets sorted by another un-patentable system that is the "secret sauce".
I think you misunderstood my comment. My point wasn't to justify patenting sorting (that would be stupid). It was just an example to clearly show the difference between an algorithm and an application of an algorithm. In the case of sorting (replace with arbitrary algorithm X if you prefer), its discovering a unique and non obvious mapping from the mathematical entities that are manipulated to the real world that is patentable, rather than the algorithm itself. The point I'm making is simply that patenting an application of an algorithm is fundamentally different than patenting the algorithm itself, so claiming software patents are invalid "because math" is missing the point.
It seems to me the difference is a question of "real world implications". For sorting integers using quicksort, yes that's just a mathematical algorithm: an algorithm that performs operations on mathematical entities. The moment you apply that algorithm to something that affects the real world, that's when it becomes patentable. E.g. sorting restaurant recommendations by applying quicksort to the individual rating. This is no longer simply a mathematical algorithm; the entities being manipulated have a 1:1 correspondence with entities in the real world. It is recognizing the usefulness of this particular correspondence between mathematical entities and real world results that is patentable.