Radio over fiber communication system technology modulation techniques are presented in this study in order to enhance the performance behavior of optical fiber communication systems. The proposed modulation techniques used to meet this aim mainly consist from a mixture of traditional modulation technologies. The proposed modulation technologies include merging modulation techniques such as pulse amplitude frequency modulation, quadrature amplitude frequency modulation, differential phase shift keying amplitude modulation, offset quadrature phase shift keying amplitude modulation and frequency phase modulation are applied on different optical communication system models. The different types of modulation techniques are illustrated and discriminated in current research paper that provides measured quantities details that namely received power, maximum Q-factor and minimum bit error rate regarding to simulation results. Optiwave Simulation Version 13 software is the basic tool which is used to study proposed models.