I hate zippers. I hate them on sweaters, on shoes, on denim, on pretty much anything. Whenever I see a zipper, I always am forced to think how much better it would be if there were buttons instead.
But my personal biases aside, you could easily put a button-down collar shirt with a tie under a zip-up sweater for a more layered look.
My one piece of warning would be that if you have a tie with an open silk weave or knit (like a silk knit or grenadine), I’d be careful about the loops of silk getting caught in the zipper — which goes back to why I hate zippers on things.