Feeling guilty but don't know if it's THAT bad?!?

Honestly I would dump the "kind of" guy. If after a year you only see each other a couple times a month then that isn't much of a relationship. And honestly I would be willing to bet you are his only "kind of" girl. As far as how you've invested time....well sometimes you have to cut your losses and go with something better.