Silicon carbide fiber-reinforced silicon carbide ceramic matrix composites (SiCf/SiC composites) are widely used in aerospace, defense, and transportation applications because of their excellent properties, such as high hardness, light weight, high-temperature resistance, and wear resistance. However, the anisotropic, heterogeneous, and hard-and-brittle nature of SiCf/SiC composites poses challenges for their high-efficiency and high-quality machining. In this study, the tool trajectory, cutting force characteristics, axial thrust force, hole entrance damage, hole diameter accuracy, and hole wall roughness during both longitudinal–torsional ultrasonic vibration drilling (LTUVD) and conventional drilling (CD) of SiCf/SiC composites were investigated. The results indicate that, compared with CD, LTUVD of SiCf/SiC composites reduces the axial thrust force by 35.7%, minimizes hole entrance and exit damage, decreases hole diameter deviation by 32.5%, improves hole wall surface roughness (Sa) by 21.7%, and significantly reduces linear roughness (Ra) by 96.9%. Using a three-factor, four-level orthogonal experimental design, a significance analysis and a range analysis were performed on the axial drilling force, tear factor, and hole diameter deviation. The optimal machining parameters for ultrasonic vibration drilling of SiCf/SiC composites were obtained, corresponding to the minimal axial thrust force, minimal exit damage, and minimal hole diameter deviation. These findings provide a valuable reference for selecting machining parameters in ultrasonic vibration drilling of SiCf/SiC composites.