In this paper we propose a novel spectrum sharing protocol for overlay cognitive radio networks using nonorthogonal
multiple access (NOMA), spatial modulation (SM) and antenna selection (AS). The proposed protocol
allows a secondary transmitter (ST) to transmit simultaneously to both a primary receiver (PR) and a secondary
receiver (SR) using SM. The usage of NOMA and SM will increase the spectral efficiency for both PR and SR with
reduced detection complexity than the case without NOMA in which the detectors are required to jointly detect
both SM symbols at each receiver. The application of AS at ST with regards to PR provides higher quality
transmission for PR without affecting the performance of SR. The performance of the proposed protocol is
investigated by derivations of upper bounds on the average symbol error probabilities at PR and SR and by
Monte Carlo simulations. Analytical and simulation results show that the proposed protocol offers efficient
spectrum utilization over spectrum sharing protocols proposed recently that uses SM – to convey the primary
data to PR through the amplitude phase modulation technique and the secondary data to SR through the index of
the active antenna.