In recent years, wearable devices have been popularly applied in the health care field. The electrocardiogram (ECG) is the most used signal. However, the ECG is measured under a body-motion condition, which is easily coupled with some noise, like as power line noise (PLn) and electromyogram (EMG). This paper presents a grey spectral noise cancellation (GSNC) scheme for electrocardiogram (ECG) signals where two-stage discrimination is employed with the empirical mode decomposition (EMD), the ensemble empirical mode decomposition (EEMD) and the grey spectral noise estimation (GSNE). In the first stage of the proposed GSNC scheme, the input ECG signal is decomposed by the EMD to obtain a set of intrinsic mode functions (IMFs). Then, the noise energies of IMFs are estimated by the GSNE. When an IMF is considered as noisy one, it is forwarded to the second stage for further check. In the second stage, the suspicious IMFs are reconstructed and decomposed by the EEMD. Then the IMFs are discriminated with a threshold. If the IMF is considered as noisy, it is discarded in the reconstruction process of the ECG signal. The proposed GSNC scheme is justified by forty-three ECG signal datasets from the MIT-BIH cardiac arrhythmia database where the PLn and EMG noise are under consideration. The results indicate that the proposed GSNC scheme outperforms the traditional EMD and EEMD based noise cancellation schemes in the given datasets.