Now, I personally think everyone should have a passport, but this misses an important detail that so far no Australasian I have heard say this has known:
Americans can probably go to twenty countries or political units without a passport.
I pointed this out to one of them and it really took the wind out of their sails. I encourage anyone who gets this factoid tossed at them to provide this list, which I compiled by looking at the Foreign Entry Requirements list, June 2003.
ANTIGUA AND BARBUDA, ARUBA, BAHAMAS, BARBADOS, BERMUDA, CANADA, DOMINICA, DOMINICAN REPUBLIC, GRENADA, JAMAICA, MEXICO, MICRONESIA, NETHERLANDS ANTILLES, PALAU, PANAMA, SAINT KITTS AND NEVIS, SAINT LUCIA, ST. PIERRE, SAINT VINCENT AND THE GRENADINES, VIRGIN ISLANDS, British, WEST INDIES, British, WEST INDIES, French.
I was wrong. It's 22. To make the comparison meaningful to an Australasian, let them imagine the following as an analogy: they wouldn't need a passport to go to most of the South Pacific islands (Tonga, Samoa, French Polynesia, etc.), Australia, New Zealand, Indonesia, and possibly Thailand.
I still think everyone should get a passport, but the "only 20% of Americans have a passport" factoid is a cheap shot, lacking an understanding of just how many places (North America, most of the Caribbean, and parts of the Pacific) Americans can go with no more than photo ID and proof of citizenship.