The link between the gut microbiota and type 2 diabetes (T2D) warrants further investigation because of known confounding effects from antidiabetic treatment. Here, we profiled the gut microbiota in a discovery (n = 1,011) and validation (n = 484) cohort comprising Swedish subjects naive for diabetes treatment and grouped by glycemic status. We observed that overall gut microbiota composition was altered in groups with impaired glucose tolerance, combined glucose intolerance and T2D, but not in those with impaired fasting glucose. In addition, the abundance of several butyrate producers and functional potential for butyrate production were decreased both in prediabetes and T2D groups. Multivariate analyses and machine learning microbiome models indicated that insulin resistance was strongly associated with microbial variations. Therefore, our study indicates that the gut microbiota represents an important modifiable factor to consider when developing precision medicine approaches for the prevention and/or delay of T2D.