The one single fail point they prevent is infected build hosts.

That might be some reasonable benefit for the company if it is building it on public architecture, but for projects like Debian that insist build hosts are basically offline (package in, package out with no internet access during build process) it is very fringe benefit.