1. You need more than jeans. In an interview published earlier this month, Luciano Barbera said he objected to men wearing jeans for any occasion. That’s a stronger position than I would take, but I do think men (Americans especially) rely far too much on jeans. Don’t be afraid to wear other kinds of trousers.
For someone just starting to build a nice wardrobe, I recommend starting with a pair of khaki chinos and grey mid-weight wool flannels. After that, get some seasonal trousers. Brown corduroys and charcoal moleskins work well for fall/ winter. Grey tropical wool and tan linens work well for spring/ summer. You can wear any of these with your basic blue or white button up shirts, or their respective seasonal counterparts such as flannel plaids and madras.
2. Your pants should flatter you. Most men wear pants that are too big and too long. Your trousers shouldn’t be so large that extra cloth is folding around your pelvis when the waist is cinched. You should also take them to a tailor and ask for them to be hemmed with a slight break or no break at all, depending on your taste and how large your trousers’ leg openings are.
Other men wear their pants too tight. This is often a more fashionable look, but I think it lacks elegance. These trousers often have low rises and overly tapered, skinny legs. I prefer mine slightly higher waisted, a bit fuller in the leg, and go down with just a bit of taper, but not too much. I think this gives more appealing, classical proportions, as demonstrated here by Luciano Barbera.
3. Know what you can tailor. Make sure your trousers come up to where you like them to and fit your seat well. Most everything else can be altered. The waist can be let out if there is the fabric allowance, or taken in. The legs can also be tapered. You want to do this only to a degree, however. Reducing the waist more than two inches, for example, will cause the side seams to come too far back. These days, I personally just make sure pants fit in the seat, rise, and thighs well, and adjust everything no more than a half an inch (except the hemming, of course). If you need to alter a bit more, it’s probably not a problem.
Note that if you let out trousers on a fabric such as cotton, you may be able to see the holes from the previous seam. It’s a bit safer to do this with wool.