Non-orthogonal multiple access (NOMA) systems are being considered as candidates for 5G wireless systems due to their promise of improved spectral efficiency. NOMA schemes are being combined with popular multicarrier schemes such as orthogonal frequency division multiplexing (OFDM) to take advantage of the benefits of multicarrier signals. A variant of the power domain NOMA is Layer Division Multiplexing (LDM). The most commonly deployed power domain LDM scheme involves successive interference cancellation (SIC) based decoding at the receiver. Fast convolution based filtered-OFDM (FC-F-OFDM) systems are becoming popular among 5G wireless access technologies due to their ability to process 5G physical layer signals efficiently. In this work, firstly, a cognitive multicarrier non-orthogonal multiplexed system based on the concept of LDM is discussed, which uses FC-F-OFDM and conventional OFDM as its component layers. Secondly, cyclostationary FREquency SHift (FRESH) filter based SIC decoding is used at the receiver side, which also utilizes artificial neural network (ANN) processing. Computer simulations indicate that the system provides good bit error rate (BER) performance under frequency selective Rayleigh fading channels.