Jitter is measurable, and apparently audible. Here are some informative links I found by searching Google:

http://www.birdland.com/papers/jitter92.pdf (research article)
http://home.earthlink.net/~bobkatz24bit/jitteressay.html
http://www.stereophile.com/showarchives.cgi?112:7 (measuring jitter)
http://www.pcabx.com/technical/jitter_power/index.htm (listen to jitter)

-Cooper