I should point out that you're conflating anarcho-libertarians with old-school statist libertarians. Most people who self-identify as libertarian believe in a state to protect property, prosecute crime, etc.
Even the self-described anarcho-capitalist libertarians of my acquaintance usually see the use of force as legitimate, as long as it's a non-governmental actor such as a private police force, mutual defense pact, enforcer of contracts, publicly traded corporation, etc. The people who take on the somewhat more difficult task of imagining a society without the organized use of force at all tend to call themselves anarchists, in my experience.