Tagging is adding pieces of information describing the song. They are stored in the header of the audio file. This makes the file self-documenting.
Windows Audio has its on tagging schema.
MP3's are tagged with IDV3
Tagging in WAV is not supported. Something to keep in mind if you want to transfer WAV-files to another computer. In case of WAV you better don't rely on tagging but on a file structure with descriptive file names.

More about tagging can be found here