I was reading this post by Matt Tinsley on TUAW this morning, talking about charing an iPad with the correct charger. After hunting around the house (same experience at Matt by the sounds) I too have a load of iPhone/iPad chargers that I thought were all the same.
Have a look at the image opposite, click it to view larger, they are all labeled either 5W (iPhone) and 10W (iPad). In Matt’s post he said that his 5W chargers were not labeled clearly instead saying they were “(5V 1A)”, I couldn’t fine one like this, all of mine were 5W.
So there you go, solves the mystery as to why your iPad sometimes charges quickly and other times takes literally hours to get to 100% charge.