Anzeige
Erschienen in:
2023 | OriginalPaper | Buchkapitel
Pose Driven Deep Appearance Feature Learning for Action Classification
verfasst von : Rejeti Hima Sameer, S. Rambabu, P. V. V. Kishore, D. Anil Kumar, M. Suneetha
Erschienen in: International Conference on Innovative Computing and Communications
Verlag: Springer Nature Singapore
Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.
Wählen Sie Textabschnitte aus um mit Künstlicher Intelligenz passenden Patente zu finden. powered by
Markieren Sie Textabschnitte, um KI-gestützt weitere passende Inhalte zu finden. powered by
Abstract
In this work, we propose to learn the fusion process between the dominant skeletal features and the RGB features. This is in contrast to the previous fusion methods that simply fused these multimodal features, without learning the fusion process to exploit the semantic relationship between them. Here, we propose a gated feature fusion (GFF) of multimodal feature data which provides attention to the appearance stream of RGB data using the temporal skeletal data. Initially, the features from RGB and skeletal frames are extracted using CNN models. Subsequently, the gated fusion network fuses the features from pose and appearance domains using temporal convolutions which are further combined into a latent subspace. Finally, the latent subspace features are classified using fully connected layers with the combined loss embeddings. The proposed architecture has performed better than the state-of-the-art models on RGB-D action datasets.