This paper deals with the design of a hybrid optimal Genetic-Sliding Mode Control (GA-SMC) approach for VSC- HVDC transmission systems for improving the system's dynamic stability over a wide range of operating conditions considering different parameter variations and disturbances. For this purpose, a comprehensive state of the art of the VSC-HVDC stabilization dilemma is discussed. The nonlinear VSC-HVDC model is developed. The problem of de- signing a nonlinear feedback control scheme via two control strategies is addressed seeking a better performance. For ensuring robustness and chattering free behavior, the conventional SMC (C-SMC) scheme is realized using a boundary layer hyperbolic tangent function for the sliding surface. Then, the Genetic Algorithm (GA) is employed for determining the optimal gains for such SMC methodology forming a modified nonlinear GA-SMC control in order to conveniently stabilize the system end enhance its performance. The simulation results verify the enhanced performance of the VSC-HVDC transmission system controlled by SMC alone compared to the proposed optimal GA-SMC control. The comparative dynamic behavior analysis for both SMC and GA-SMC control schemes are presented.