The human gut is inhabited by trillions of microorganisms composing a dynamic ecosystem implicated in health and disease. The composition of the gut microbiota is unique to each individual and tends to remain relatively stable throughout life, yet daily transient fluctuations are observed. Diet is a key modifiable factor influencing the composition of the gut microbiota, indicating the potential for therapeutic dietary strategies to manipulate microbial diversity, composition, and stability. While diet can induce a shift in the gut microbiota, these changes appear to be temporary. Whether prolonged dietary changes can induce permanent alterations in the gut microbiota is unknown, mainly due to a lack of long-term human dietary interventions, or long-term follow-ups of short-term dietary interventions. It is possible that habitual diets have a greater influence on the gut microbiota than acute dietary strategies. This review presents the current knowledge around the response of the gut microbiota to short-term and long-term dietary interventions and identifies major factors that contribute to microbiota response to diet. Overall, further research on long-term diets that include health and microbiome measures is required before clinical recommendations can be made for dietary modulation of the gut microbiota for health.