Who are considered holy not because they aren't social, but because they are rejecting something fundamentally human in the name of $diety.

A number of schools of buddhism hold no 'diety', yet still have monks who isolate themselves.

Skim milk is diety.

I read an article by an East Orthodox priest once - evidently the idea was the hermit life was popular for wrestling with the Self and for the pursuit of reflection and personal betterment.