Hsp70 and Hsp90 chaperones are critical for protein quality control in the cytosol, whereas organelle-specific Hsp70/Hsp90 paralogs provide similar protection for mitochondria and the endoplasmic reticulum (ER). Cytosolic Hsp70/Hsp90 can operate sequentially with Hsp90 selectively associating with Hsp70 after Hsp70 is bound to a client protein. This observation has long suggested that Hsp90 could have a preference for interacting with clients at their later stages of folding. However, recent work has shown that cytosolic Hsp70/Hsp90 can directly interact even in the absence of a client, which opens up an alternative possibility that the ordered interactions of Hsp70/Hsp90 with clients could be a consequence of regulated changes in the direct interactions between Hsp70 and Hsp90. However, it is unknown how such regulation could occur mechanistically. Here, we find that the ER Hsp70/Hsp90 (BiP/Grp94) can form a direct complex in the absence of a client. Importantly, the direct interaction between BiP and Grp94 is nucleotide-specific, with BiP and Grp94 having higher affinity under ADP conditions and lower affinity under ATP conditions. We show that this nucleotide-specific association between BiP and Grp94 is largely due to the conformation of BiP. When BiP is in the ATP conformation its substrate-binding domain blocks Grp94; in contrast, Grp94 can readily associate with the ADP conformation of BiP, which represents the client-bound state of BiP. Our observations provide a mechanism for the sequential involvement of BiP and Grp94 in client folding where the conformation of BiP provides the signal for the subsequent recruitment of Grp94.