In recent years, with application of optimization technology and computer technology in the field of optimal design, it has become an essential course for mechanical students. How to effectively apply multimedia optimal methods to the optimal design course is the problem that we need to solve. The teaching mode of optimal design course is explored by multimedia software simulation analysis. The existing problems in the teaching of optimal design are introduced in this dissertation. And the case of the optimal design teaching according to the teaching method of multimedia software is realized. The finite element analysis of reducer and the application of genetic algorithm in it, for example, emphasize how multimedia simulation can be effectively applied to optimal design. We take the gear reducer as a case to carry out the modal analysis and gear meshing analysis and optimize its structure by using genetic algorithm. Making full use of multimedia software, combined with the course knowledge points and simulation software to cultivate the actual design and application capabilities, paying attention to the acquisition of optimization design methods, could improve the quality of optimization design personnel training. The results show that integrating multimedia simulation into optimization design teaching could be a good way to improve the practice of teaching and to promote knowledge learning and optimal design capabilities. It has a great significance for the development and application of optimal design based on multimedia.