Libertarianism is basically about negative rights - not so much about "do what you want", but more about "nobody is going to force you to do something you don't want".
As an ethical system, think of it as utilitarianism which assigns infinite value to freedom from violent coercion. Everything else follows from that.

ā€” Source HN discussion on Theil supporting Trump

If we could pick and choose concepts from Libertarianism, I think most people would easily find it agreeable.