Sandpaper comes in grit numbers, e.g. 100, 400, etc. About 100 grit is pretty rough and will remove more material quickly. Once you get it generally where you want it and want to smooth it out, it's time to go up in numbers. Move up gradually, e.g. 200 next, maybe some 400 after, and finer and finer until you like how it is. Ace Hardware has a sanpaper assortment that comes with fine, medium, and coarse, which seems to do the job nicely and only costs like $2.50.