Pros
The people at Caltech are amazing. The other graduate students are not stereotypical nerds, but simply very smart and interested in the world around them. The campus is beautiful and the weather is great, if you like constant sunshine. :) Caltech has a very small student population, and so pretty much every face on campus is familiar, which can be a good or bad thing. The research opportunities are top noch and unbeatable. Did I mention that everyone is really, really nice? I have never encountered such a concentration of intelligent and kind people anywhere else. But what Caltech is know for most of all is its reserach.
Cons
All of the budget cuts are annoying and bad for morale. Advisors seem to be very self-absorbed and very few are good mentors. Caltech is so small that if you're not interested in working on one of the very few topics that are the specialties of the professors there, you're sort of screwed. (This is particularly troublesome if you *think* you know what you want to work on, and then change your mind after the first year. The possibilities for a new advisor are very limited.) They also keep reducing the health insurance benefits for grad students.