By Right or by Permission

From "The Fountainhead":

"My dear fellow, who will let you?"
"That's not the point. The point is, who will stop me?"