A Mixed Bag...
I relocated to Ventura about a year and a half ago for a job after having spent over ten years living in the Los Angeles area. So first I will give the good. The weather, as other have stated is purely awesome. While it's true that we do get a lot of fog in the Spring and Summer I would take this any day over 90 plus degree days in Los Angeles and the fact is that many coastal areas in Southern California are very similar. And although I am not a beach person I would definitely say the beaches in Ventura are very nice. Unlike LA beaches it is never very difficult to find parking and the crowds are way better than they are in Santa Monica or Manhattan Beach.
Another positive, again in comparison to most areas in Southern California, is that Ventura is quite bit cheaper than most decent places to live in Southern California. In Ventura is it possible to purchase a three, or even four bedroom house, for between $350K-$450K in a good area. You will not be living four blocks from the beach but you will most certainly be living in a great area to raise a family.
One last positive is the location of Ventura. While Ventura doesn't have all of the action of Los Angeles the fact is that you are located within thirty minutes of Santa Barbara to the North and Thousand Oaks to the South. Santa Barbara is great for a quick weekend getaway, where you might even see a celebrity or two;-), and Thousand Oaks is great if your are looking for your "suburban mall fix." I also should mention Camarillo, even closer than Thousand Oaks, has a pretty decent Outlet mall.
The main "negatives" I see with Ventura I can sum up pretty quickly. First, the population in general seems to skew a bit older. I say this being almost 42 so I am not exactly a young one but I do miss the young hip feeling of LA. Secondly, there seems to be somewhat of a conservative vibe as well. I am a complete liberal and LA obviously skews more this direction. My final issue, which some may perceive as a positive, is that even though Ventura is not what I would call necessarily a small town there are a lot of natives here and it seems a bit hard to break into on the social front.
Overall I would say Ventura has more positives than negatives compared to other areas in Southern California if you can make it work.
Skip | Ventura, CA