Here’s the full quote — just a bit different from what we have above (emphasis added), emphasizing the difference between taxes and penalties:

The plaintiff’s reliance is upon Allgeyer v. Louisiana, 165 U.S. 578 , 17 S. Ct. 427, in which it was held that a fine could not be imposed by the State for sending a notice similar to the present to an insurance company out of the State. But it seems to me that the tax was justified and that this case is distinguished from that of Allgeyer and from St. Louis Cotton Compress Co. v. Arkansas, 260 U.S. 346 , 43 S. Ct. 125, by the difference between a penalty and a tax. It is true, as indicated in the last cited case, that every exaction of money for an act is a discouragement to the extent of the payment required, but that which in its immediacy is a discouragement may be part of an encouragement when seen in its organic connection with the whole. Taxes are what we pay for civilized society, including the chance to insure. A penalty on the other hand is intended altogether to prevent the thing punished. It readily may be seen that a State may tax things that under the Constitution as interpreted it can not prevent. The constitutional right asserted in Allgeyer v. Louisiana to earn one’s livelihood by any lawful calling certainly is consistent, as we all know, with the calling being taxed.

Compania General de Tabacos de Filipinas v. Collector of Internal Revenue, 275 U.S. 87 (1927) (here from FindLaw)

See also the website with photos from the late Lachlan Cranswick.

It makes you wonder, though: Did Holmes later add that part about “I like to pay taxes?”