Microbial communities play a major role in disease, biogeochemical cycling, agriculture, and bioremediation. However, identifying the ecological processes that govern microbial community assembly and disentangling the relative impacts of those processes has proven challenging. Here, we propose that this discord is due to microbial systems being studied at different spatial, temporal, and phylogenetic scales. We argue that different processes dominate at different scales, and that through a more explicit consideration of spatial, temporal, and phylogenetic grains and extents (the two components of scale) a more accurate, clear, and useful understanding of microbial community assembly can be developed. We demonstrate the value of applying ecological concepts of scale to microbiology, specifically examining their application to nestedness, legacy effects, and taxa-area relationships of microbial systems. These proposed considerations of scale will help resolve long-standing debates in microbial ecology regarding the processes determining the assembly of microbial communities, and provide organizing principles around which hypotheses and theories can be developed.