Skip to main content

2025 | OriginalPaper | Buchkapitel

A Multi-model Approach for Video Data Retrieval in Autonomous Vehicle Development

verfasst von : Jesper Knapp, Klas Moberg, Yuchuan Jin, Simin Sun, Miroslaw Staron

Erschienen in: Product-Focused Software Process Improvement. Industry-, Workshop-, and Doctoral Symposium Papers

Verlag: Springer Nature Switzerland

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Autonomous driving software generates enormous amounts of data every second, which software development organizations save for future analysis and testing in the form of logs. However, given the vast size of this data, locating specific scenarios within a collection of vehicle logs can be challenging. Writing the correct SQL queries to find these scenarios requires engineers to have a strong background in SQL and the specific databases in question, further complicating the search process. This paper presents and evaluates a pipeline that allows searching for specific scenarios in log collections using natural language descriptions instead of SQL. The generated descriptions were evaluated by engineers working with vehicle logs at the Zenseact on a scale from 1 to 5. Our approach achieved a mean score of 3.3, demonstrating the potential of using a multi-model architecture to improve the software development workflow. We also present an interface that can visualize the query process and visualize the results.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Alibeigi, M., et al.: Zenseact open dataset: a large-scale and diverse multimodal dataset for autonomous driving (2023) Alibeigi, M., et al.: Zenseact open dataset: a large-scale and diverse multimodal dataset for autonomous driving (2023)
2.
Zurück zum Zitat Carballo, K.V., et al.: Tabtext: a flexible and contextual approach to tabular data representation. arXiv preprint arXiv:2206.10381 (2022) Carballo, K.V., et al.: Tabtext: a flexible and contextual approach to tabular data representation. arXiv preprint arXiv:​2206.​10381 (2022)
3.
Zurück zum Zitat Chen, K., Yang, Y., Chen, B., López, J.A.H., Mussbacher, G., Varró, D.: Automated domain modeling with large language models: a comparative study. In: 2023 ACM/IEEE 26th International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 162–172. IEEE (2023) Chen, K., Yang, Y., Chen, B., López, J.A.H., Mussbacher, G., Varró, D.: Automated domain modeling with large language models: a comparative study. In: 2023 ACM/IEEE 26th International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 162–172. IEEE (2023)
4.
Zurück zum Zitat Chen, X., et al.: Automatic labeling to generate training data for online lidar-based moving object segmentation. IEEE Robot. Autom. Lett. 7(3), 6107–6114 (2022)CrossRef Chen, X., et al.: Automatic labeling to generate training data for online lidar-based moving object segmentation. IEEE Robot. Autom. Lett. 7(3), 6107–6114 (2022)CrossRef
5.
Zurück zum Zitat Fan, A., et al.: Large language models for software engineering: survey and open problems. arXiv preprint arXiv:2310.03533 (2023) Fan, A., et al.: Large language models for software engineering: survey and open problems. arXiv preprint arXiv:​2310.​03533 (2023)
6.
Zurück zum Zitat Hussain, R., Zeadally, S.: Autonomous cars: research results, issues, and future challenges. IEEE Commun. Surv. Tutor. 21(2), 1275–1313 (2018)CrossRef Hussain, R., Zeadally, S.: Autonomous cars: research results, issues, and future challenges. IEEE Commun. Surv. Tutor. 21(2), 1275–1313 (2018)CrossRef
7.
Zurück zum Zitat Lebret, R., Grangier, D., Auli, M.: Neural text generation from structured data with application to the biography domain. arXiv preprint arXiv:1603.07771 (2016) Lebret, R., Grangier, D., Auli, M.: Neural text generation from structured data with application to the biography domain. arXiv preprint arXiv:​1603.​07771 (2016)
9.
Zurück zum Zitat Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Advances in Neural Information Processing Systems, vol. 36 (2024) Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Advances in Neural Information Processing Systems, vol. 36 (2024)
10.
Zurück zum Zitat Muennighoff, N., Tazi, N., Magne, L., Reimers, N.: Mteb: massive text embedding benchmark. arXiv preprint arXiv:2210.07316 (2022) Muennighoff, N., Tazi, N., Magne, L., Reimers, N.: Mteb: massive text embedding benchmark. arXiv preprint arXiv:​2210.​07316 (2022)
11.
Zurück zum Zitat Pandey, S.K., Staron, M., Horkoff, J., Ochodek, M., Mucci, N., Durisic, D.: Transdpr: design pattern recognition using programming language models. In: 2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 1–7. IEEE (2023) Pandey, S.K., Staron, M., Horkoff, J., Ochodek, M., Mucci, N., Durisic, D.: Transdpr: design pattern recognition using programming language models. In: 2023 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM), pp. 1–7. IEEE (2023)
12.
Zurück zum Zitat Sharma, R.: Big data for autonomous vehicles. In: Deep Learning and Big Data for Intelligent Transportation: Enabling Technologies and Future Trends, pp. 21–47 (2021) Sharma, R.: Big data for autonomous vehicles. In: Deep Learning and Big Data for Intelligent Transportation: Enabling Technologies and Future Trends, pp. 21–47 (2021)
15.
16.
Zurück zum Zitat Xiao, S., Liu, Z., Zhang, P., Muennighof, N.: C-pack: packaged resources to advance general Chinese embedding. arXiv preprint arXiv:2309.07597 (2023) Xiao, S., Liu, Z., Zhang, P., Muennighof, N.: C-pack: packaged resources to advance general Chinese embedding. arXiv preprint arXiv:​2309.​07597 (2023)
17.
Zurück zum Zitat Zhao, Y., Zhang, H., Si, S., Nan, L., Tang, X., Cohan, A.: Investigating table-to-text generation capabilities of large language models in real-world information seeking scenarios. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 160–175 (2023) Zhao, Y., Zhang, H., Si, S., Nan, L., Tang, X., Cohan, A.: Investigating table-to-text generation capabilities of large language models in real-world information seeking scenarios. In: Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pp. 160–175 (2023)
18.
Zurück zum Zitat Ziegler, L., Krämer, F., Haustein, N.: Ki und big-data-management für autonomes fahren. ATZelektronik 15(4), 40–45 (2020)CrossRef Ziegler, L., Krämer, F., Haustein, N.: Ki und big-data-management für autonomes fahren. ATZelektronik 15(4), 40–45 (2020)CrossRef
Metadaten
Titel
A Multi-model Approach for Video Data Retrieval in Autonomous Vehicle Development
verfasst von
Jesper Knapp
Klas Moberg
Yuchuan Jin
Simin Sun
Miroslaw Staron
Copyright-Jahr
2025
DOI
https://doi.org/10.1007/978-3-031-78392-0_3