That's not entirely true. Most of the Imperial personnel we see in the OT are obviously Navy. The one time a General shows up, he's leading an entirely different class of soldiers who are never actually identified as "Stormtroopers" onscreen, and again, it doesn't make any sense for the entire ground combat arms of the Imperial military to consist entirely of "Stormtroopers," which historically and linguistically are highly specialized and/or elite shock infantry. People just got it into their heads somehow that all Imperial soldiers are Stormtroopers, just like all Rebel soldiers dress like either the Endor commandos or the security personnel on the Tantive IV/Yavin 4 (which is another kettle of fish all together).

I don't think the progression of events in ESB provides much basis to argue that Vader stopped off at the nearest fleet depot or whatever and picked up Veers and his troops specifically prior to attacking Echo Base. Veers is already present aboard the Executor (which appears to be in the middle of deep space) when the probe droid data comes in, and Vader gives the order to set course for the Hoth system immediately upon reviewing it.

Frankly, they make more sense as Navy personnel.