If you are giving a complete sentence, you should use the article. "The plane arrived in the USA." Etc.

If you are giving the names of countries in a list or chart or some such where you are not using complete sentences, then you simply put "USA", like you would put "Germany" or "France".

I have always (since my earliest memories) considered the official name of the nation to be:

• The United States of America

For instance: I am a citizen of The United States of America. (Notice that I capitalized "The", even though it appeared mid-sentence, because it is part of the name.)

However, I also recognized the following as also being valid shortcuts:

• United States of America
  
• United States
  
• America (although I later learned that this is somewhat unclear: referring to "the Americas" is a way to specify both North America and South America, and "American" might refer to something related to "the Americas", particularly if the discussion is making contrasts to European or Eastern cultures)
  
• USA (abbreviation for United States of America)
• The United States
• U.S. ("you ess")
• U.S. of A

Based on the above, saying "the USA" (starting out lowercase) is appropriate, because it is a shortcut referring to the informal name of "United States of America", while "The USA" (starting out uppercase) is also appropriate, because it is a shortcut referring to the full authentic name. I don't typically consider either of these abbreviations to be wrong.

This nation also has multiple names that are valid names (even if they are not the one single official name), just like I have a full name (which includes a "first name", and a "middle name", and a "last name"), but I also have a shorter name that I typically go by. (I'm referring to my first name, which my parents have frequently called me. I'm not even referring other pseudonyms/names/nicknames, like "TOOGAM", that I may have used over the years.)

Reasoning

I wouldn't typically say, "he traveled to United States of America" (entirely leaving off the word "the"), just like I probably wouldn't be very likely to say "he traveled to Hawaiian islands" or "he traveled to Twin Cities". Maybe I could say some of those things (and not completely sound grammatically incorrect), but I would be more inclined to say "he traveled to the Hawaiian islands" or "he traveled to the Twin Cities". In these cases, the names are Hawaii and "Twin Cities", and so I leave the word "the" in lowercase because it isn't part of the actual name. Similarly, if "the" is not being treated as part of the shorter "United States of America" name, then it should be lowercase because it is not part of the name that is being used. However, if "The" is being treated as part of the longer full name, which is "The United States of America", then it should be uppercase because it is part of the name.

Both names are widely supported, and so I would say that you should be able to get away with either choice (regarding whether to capitalize the letter). One way, "The" is part of the title. One way, "the" is not part of the title. Both ways sound the same when being pronounced. When speaking, it can be impossible to tell which way the person had in mind, but that's okay because it really doesn't make any difference to what the person was trying to say. (Also, the speaker probably wasn't bothering to even think of such a distinction.) If two people are writing down the words spoken by one person, and each of those two people capitalized that word differently, I would consider each of the resulting documents to be an accurate record of what was spoken, word-for-word.

More tidbits

In truth, the term "The United States of America" is not used in common speech nearly as frequently as the shorter names. We typically leave off the "of America", or leave off "The United States of". We will sometimes use "US" or "U.S." as an abbreviation. (For instance: "He went to Europe, and then got back to the U.S.") When actually writing things down, I've found that to be getting to be less common than "USA". I guess the single extra letter is considered worthwhile, just to be less likely to conflict with a possible two-letter abbreviation for something else. So, when writing, we will often use "USA" because it is shorter than "America" or "United States". We use the abbreviation based on the full name, although when we don't abbreviate then we tend to use a shorter name, unless we really want the entire name (probably because we are trying to be as formal as possible).

The purpose of the word "the" is to denote (a) specific noun(s). Eg. "the apple(s) on the table" - there are a lot of apples in the world, but I am talking about the specific/understood/defined apple(s) on the table. Same thing happens with USA. "The" is used to define specific states, as there are many states around the world - "The (specific/defined/understood) United States of America". America in isolation deos not need "the" because, like England, it is a proper noun. In the same way, we have the Netherlands (specific/defined/understood lands), the "United Kingdom" (defining/specifying which Kingdom), the People´s Republic of China (defining which Republic), etc. The Ukraine uses "the" because Ukraine is a definition, meaning "the edge" of slav territory. Therefore, it seems the reason we use "the" with USA is the obvious and same reason we always use "the" - when we define a specific object understood by everyone.

I think the key lies in whether or not "States" is being included in the name.

For example:

"I am going to the United States of America."

vs

"I am going to America."

You wouldn't say, "I am going to States," but rather, "I am going to the States."

Similarly, you wouldn't say, "I am going to the America," but rather, "I am going to America."

The exception to this rule (welcome to America, the land of exceptions...) being with the initials "USA", such as when referring to The United States as a team (Team USA, USA have won, etc.) or when something is made in The United States (Made in USA, Made in U.S., etc.)