Is not about vision, is about emerging patters that arise from noise. If just one piece (deemed important) piece of the pattern differs from the expectation, then the classification fails. Therefore surface blurring does fix the issue, because the number of patterns is reduced (when applied in both the source samples and the user input).
Here is a diff of the borders of the pictures (instead of pixels, which is less useful), the one in the left with surface blur and the one in the right without: http://i.imgur.com/3bihGVA.png (The parts marked as completely different are the white ones)
Yeah, dude, I know. But why is that piece deemed important? Is it truly important? If it is not truly important, then the algorithm is overtrained -- that is, it has found "patterns" that exist in the training set but only by random chance, and which would not exist in a sample set.
But as I mentioned, there seem like there are reasonable reasons to suspect that this is not just overtraining.
What does adding surface blur really fix? Why could you not add similarly subtle network-breaking noise to a blurred image? If you think of blurring as downsampling, it is obvious that you also need downsampled noise to trick the NN.
The noise is not random, one can see exactly where the artifacts were added (the white parts of the image). And for this solution to work the down sampling must happen inside the program, not in the file system, therefore all network-braking noise would have been already discarded.
And for this solution to work the down sampling must happen inside the program, not in the file system, therefore all network-braking noise would have been already discarded.
I see, maybe I just didn't get your point the first time.
Anyway, what I was trying to say is that if you do view the downsampling as part of the network/program, you could apply the optimization procedure mentioned in the paper to a network that blurs its input. I assume that this would then generate network-breaking patterns that are imperceptible to the human eye, in the same way as happens in the paper.
Here is a diff of the borders of the pictures (instead of pixels, which is less useful), the one in the left with surface blur and the one in the right without: http://i.imgur.com/3bihGVA.png (The parts marked as completely different are the white ones)