William, I must say that Satie is an unusual place to begin. He was notably eccentric(even for a Frenchman), and for example, when critics complained that his music "lacked shape", he titled his next composition "Trois Morceaux en forme de poire"(Three Pieces in the shape of a pear; actually there were five). If you've heard some Satie that you liked, I'll specifically suggest this collection
by Anne Queffelec, joined on some four-hand pieces by Catherine Collard.
Now, if you give more info on what you're looking for, I'll be happy to suggest a small starter collection of piano and/or orchestral music. I've made quite a few suggestions in the last couple years of items that I consider essential in the "Stunning Recordings" thread in the Stereo forum.