I don't know Utrecht too well, but here are some record shops worth visiting in Amsterdam:
Concerto in the Utrechtse straat. They have second hand CDs and I suppose vinyl as well.
Boudisque near the Central Station, first or second alley to the right when you're at the Damrak. Used to be the place for underground music. In an alley next to Boudisque I saw a "vinly only" shop some time ago, but I don't know if it's still around.
Blue Note behind the New Church on the Dam square is a jazz only shop.
Fame near the Dam square is one of the biggest stores and has a handy computer system where you can see for yourself what is in stock. Their classical section is also not bad.
There used to be a shop called de Nieuwe Muziekhandel in de Leidsestraat with classical music and another shop with "classical only" next to the Heineken brewery, but I don't know if they still exist.
Have fun!
Willem