Star anise essential oil (SAEO) is a plant essential oil with good antibacterial activity, but its applications are limited due to its high volatility, strong smell, and unstable physical and chemical properties. The effect of selective encapsulation of SAEO by hydroxypropyl-β-cyclodextrin (HPCD) on its compositions, volatility stability and antibacterial activity was investigated. The GC-MS results indicated that the compositions reduced and content of the compositions of SAEO changed after encapsulation. Most of the components in SAEO were successfully encapsulated by HPCD, which can be supported by data from FTIR and 1H NMR. According to the molecular modeling results, the three guest molecules (trans-anethole, estragole and trans-foeniculin) were all docked in the cavity of HPCD on the isoallyl (or allyl) side. The volatile stability of SAEO before and after encapsulation was evaluated by electronic nose, and the results confirmed that encapsulation significantly reduced the irritating smell of SAEO and makes the clathrate have a sustained release effect. Furthermore, in the antibacterial test, the selective encapsulation of HPCD improved the inhibition effect of SAEO on Rhizopus stolonoifer, Saccharomyces cerevisiae, and E. coli and its antibacterial stability in 24 h.