The Denim Jacket
You can live in jeans all year round and that denim jacket might suit cooler climates. Okay, so that is not a sweater per se, but that is casual enough to wear like you would a sweater. Wear it over a summer dress when you go to a restaurant. Bring that along for your vacation to keep you warm on cold nights.
If you find denim to hot for you, or if you prefer something more sophisticated, cashmere is your best bet. The fabric is soft and luxurious. The look, totally elegant. You can even wear it with your more sophisticated dresses.
In between denim and cashmere, you have the more affordable jersey tops.
Jersey drapes beautifully and are excellent light-weight sweaters if you are on a budget. They are supercomfortable and extremely flattering. These sweaters can be worn during the colder months. Layered over your favourite dress or your jeans and t-shirts.