Removing duplicate files in your iTunes library

One of the problems with a networked drive acting as the iTunes library for multiple computers in the house is that I often end up with duplicate songs in the library. I think this comes from importing the library directory when you also have "Copy files to the iTunes library" set in the iTunes advanced prefs. ITunes imports a song and then tries to copy it over itself. Seeing there’s already a file there by that name it creates a file called "songname 1.mp3"

At least that’s my theory.

To clean up a library full of duplicate files, here’s what I did. I installed Duff, a unix utility that quickly finds duplicate files. Duff works by comparing the actual files of any two files that have identical sizes. I sent Duff’s output to a text file with the command

duff -r /Volumes/music/ > duplicatemusic.txt

Once Duff was done running, I ran a short command to grab all the lines from the output that end in 1.mp3 and delete them.

cat duplicatemusic.txt | grep 1.mp3 | tr '\012' '\000' | xargs -0 rm

If you’ve done whatever it is that causes duplicates a number of times, you might have *2.mp3, *3.mp3, etc. Just run that command again, replacing 1.mp3 with 2.mp3 and so on. The one liner above could probably be improved to grep for any single digit followed by .mp3, but it’s quick enough to run it a few times that I didn’t bother.