We consider the problem of determining an optimal semi-active damping of vibrating systems. For this damping optimization we use a minimization criterion based on the impulse response energy of the system. The optimization approach yields a large number of Lyapunov equations which have to be solved. In this work we propose an optimization approach that works with reduced systems which are generated using the parametric dominant pole algorithm. This optimization process is improved with a modal approach while the initial parameters for the parametric dominant pole algorithm are chosen in advance using residual bounds. Our approach calculates a satisfactory approximation of the impulse response energy while providing a significant acceleration of the optimization process. Numerical results illustrate the effectiveness of the proposed algorithm.