I’ll agree on fit. Fit and fit alone can set you way apart. Best thing is that it only takes a modification of your current wardrobe versus getting a new one.
Also, I totally can sympathize with people giving you shit for wearing a tie! Happened to me when I first started to try and dress better. Work through that!
getting shit for wearing a tie lasts about 2 days. getting shit for NOT wearing a tie once you start wearing one is eternal.