I went right to a 4-year school after high school (University of Wash.), I was lucky enough to know what I wanted to do with life. I was pretty much destined to be an engineer, I was good with math and science, but horrible at english and related classes. I also knew that I didn't want a career field that was a fad or live as a struggling artist, my one other skill. But at college I watched so many others come in right after high school and struggle to no end. I had friends drop out, some that are still in school (over 6 years) with no degree in sight and no passion for any field just living on Daddy's check book. The only people that continuously impressed me were the ones that took time off before coming back to school. I was friends with a couple of guys in their 30's who came back for an engineering degree and I never saw anyone work harder or with more determination. They were also so much better at managing their time, not like the other people in their early 20's trying to figure out how to manage their hang-overs during the week.
So don't be deterred, if you can handle the extra financial burden it will definitely be worth it, and it is very achieveable, just don't go in looking for a degree in anything, find what you like and what will give you a job when you're done.