The complex modal approach is introduced for the optimal vibration control (Linear Quadratic Regulator) of high-order nonsymmetric discrete systems. An LQ regulator is designed based on a reduced-order model obtained by neglecting high-frequency complex modes of the original system. The matrix transformations between physical coordinates and complex coordinates are derived. A 52 degree-of-freedom finite element based rotordynamic system is used for illustration. Simulation results show that an LQ regulator based on a reduced-order system obtained by using normal modes of a high-order system with asymmetric models can possibly destabilize the original system i.e., the spill-over problem (Ulsoy, 1984), however, this problem might be avoided by applying complex modes which provides a more accurate reduced-order model than obtained by normal modes. Comparison of the reduced-order models using normal modes and complex modes is presented. Frequency, time transient and steady state responses of the controlled and uncontrolled systems are also shown.