4. Cross-Modal Transmission

In this chapter, the cross-modal transmission techniques are described in detail, which can be divided into two main categories. Schemes in the first category belong to the preemptive-resume scheduling pattern, where haptic streams have higher priority than audio-visual streams during transmission. On one hand, low-latency and high-reliability demands for haptic streaming should be guaranteed. Considering this, the intra-modal correlation-based haptic streaming scheme is proposed, which explores correlation within the haptic modality to employ efficient transmission. On the other hand, due to the occasional arrival of haptic streams, the throughput of audio-visual streams may well be reduced or even interrupted. The inter-modal correlation-based audio-visual streaming and content-aware stream scheduling schemes are respectively designed, handling the fluctuant audio-visual transmission quality issues with the aid of the haptic modality. Schemes in the second category belong to the non-preemptive-resume scheduling pattern. In other words, the transmission resources allocated to haptic and audio-visual modalities are under comprehensive and coordinated consideration. Among them, the modal-aware stream scheduling scheme flexibly settles the transmission priority of the modal stream instead of the data flow, achieving a tradeoff between various requirements. The adaptive prediction-aware scheme predicts and transmits the multi-modal signals in advance to reduce the whole latency during stream delivery. The dynamic transmission mode selection scheme tries to improve resource utilization efficacy by optimizing both the chosen transmission strategy and the allocated resource. Finally, the edge intelligence-empowered transmission decision scheme carefully considers factors such as communication, caching, computation, and control capacities, and realizes autonomous transmission decisions through AI technology.