This question requires consideration of the number of factors that conspire to produce the wind.
Large-scale weather systems are perhaps the best-known features that impact the winds at a location.
Unsettled weather is nearly always associated with low sea-level pressure and relatively strong gradients of pressure (differences from one location to another in a region) that drive the winds associated with major storms.
Regions of high sea-level pressure often characterize fair weather and such weather systems are associated with small gradients in pressure and, consequently, light winds.
Aside from the influence of highs and lows, which can visit any location, proximity to a coast (whether it be the ocean or one of the Great Lakes) can also be a major influence on the windiness.
The local topography also exerts a major influence on average wind speeds with sheltered valleys being less windy than the open plains, for instance.
Thus, it is not surprising that inland locations, away from the coasts of large lakes and near or within valleys are among the least windy.
Among the largest 50 cities in the U.S., Phoenix has the lowest average annual wind speed at 6.2 mph.
For comparison, Milwaukee has an average breeze of 11.5 mph.
Oak Ridge, Tennessee, appears to be the least windy city in the U.S. with an average annual wind speed of 4.1 mph.
Interestingly, Anchorage, Fairbanks, and Valdez, Alaska, all have average annual wind speeds around 4 mph even though it can occasionally be very windy in those cities.
Light winds in interior Alaska are a consequence of the terrain.