It's definitely a pretty small world, and to make things worse there are sub-niches -- between which there's certainly cross-pollination, but that's still a barrier to people looking to change jobs. Frontend language semantics (where most PL papers focus) vs. middle-and-back end optimization and hardware support; AoT compilers vs. JITs; CPU targets vs. a blossoming array of accelerators, etc.

Beyond that, I've definitely interviewed people who seemed like they could have been smart + capable but who couldn't cut it when it came to systems programming questions. Even senior developers often struggle with things like memory layouts and hardware behavior.