In tenth grade, I made my first "official" college visit. My destination was Berry College. The day was rather grey and cloudy. Boy, were there a lot of deer! And there was something about Chick-fil-A, too. Did you know Berry College has the world's largest contiguous campus? These few bullet points were the main things I carried with me from my visit. Oh, and Rome, Georgia, is tiny. Based on this wealth of knowledge, I decided then, in 10th grade, not to apply.
Was Berry hampered by the fact that I visited so early in my high school career? Perhaps. But I think if I had visited a year and a half later, my final response would still have been the same. While, as a sophomore, I didn't have clear academic goals for my college years, nor did I know what kind of school I would like (big/little, rural/urban, arts/sports, and so forth), I did know myself. And I knew that Berry wasn't the right fit for me.
Like I tell students when working SAT Math questions, "You have to start somewhere, even if you don't quite know where to start." The same goes for campus visits. Start them, and start them early. I've spoken to students who were trying to figure out the big/little question or the rural/urban question without setting foot on a campus. I've spoken to others who were going to apply to all big schools because those are the places their peers were applying, again without seeing any of those big schools or getting a feel for the little schools.
You have to start somewhere, and I think the place to start is on a campus. Pick one that doesn't revolt you on paper, and go see it. If you get there and hate/love it, figure out what factors are contributing to these emotions and apply that knowledge to your next visit. My Berry visit, if nothing else, informed me that I wanted to be at a college that was not quite so rural. Now, where will you start?