Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Machine Learning and Cybernetics
Published in: International Journal of Machine Learning and Cybernetics 2/2023

02-08-2022 | Original Article

Complete interest propagation from part for visual relation of interest detection

Authors: You Zhou, Fan Yu

Published in: International Journal of Machine Learning and Cybernetics | Issue 2/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Visual relation detection (VRD) is proposed to describe an image with visual relation triplets in the form of <subject, predicate, object>. As a further extension of the traditional VRD task, visual relation of interest detection (VROID) is proposed to obtain visual relations of interest, i.e., visual relations are semantically important for expressing the main content of an image. In this paper, we propose a complete interest propagation from part (CIPFP) method for VROID, which exploits semantic parts and propagates interest along part-instance-relation. Specifically, the interest in CIPFP is propagated from parts to part pairs, from parts to instances, from part pairs to instance pairs, from instances to instance pairs, from parts to relation triplets and from instance pairs to relation triplets. We conduct substantial experiments to validate the effectiveness of the CIPFP method and the components in CIPFP.
previous article A multi-stage fusion instance learning method for anomalous event detection in videos
next article D2TS: a dual diversity tree selection approach to pruning of random forests

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Show more products
Literature
1.
Abdelkarim S, Achlioptas P, Huang J, Li B, Church K, Elhoseiny M (2020) Long-tail visual relationship recognition with a visiolinguistic hubless loss. arXiv preprint arXiv:​2004.​00436
2.
Baier S, Ma Y, Tresp V (2017) Improving visual relationship detection using semantic modeling of scene descriptions. In: International semantic web conference. Springer, pp 53–68
3.
Chen L, Zhang H, Xiao J, He X, Pu S, Chang SF (2018) Scene dynamics: Counterfactual critic multi-agent training for scene graph generation. arXiv preprint arXiv:​1812.​023473
4.
Chen T, Yu W, Chen R, Lin L (2019) Knowledge-embedded routing network for scene graph generation. In: IEEE conference on computer vision and pattern recognition, pp 6163–6171
5.
Chen X, Mottaghi R, Liu X, Fidler S, Urtasun R, Yuille A (2014) Detect what you can: detecting and representing objects using holistic models and body parts. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1971–1978
6.
7.
Cui Z, Xu C, Zheng W, Yang J (2018) Context-dependent diffusion network for visual relationship detection. In: Proceedings of the 26th ACM international conference on Multimedia, pp 1475–1482
8.
Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338CrossRef
9.
Fang, H.S., Cao, J., Tai, Y.W., Lu, C.: Pairwise body-part attention for recognizing human-object interactions. In: European conference on computer vision, pp 51–67 (2018)
10.
Farhadi A, Redmon J (2018) Yolov3: an incremental improvement. computer vision and pattern recognition, cite as
11.
Girshick R (2015) Fast r-cnn. In: IEEE international conference on computer vision, pp 1440–1448
12.
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 580–587
13.
Gkanatsios N, Pitsikalis V, Koutras P, Maragos P (2019) Attention-translation-relation network for scalable scene graph generation. In: Proceedings of the IEEE international conference on computer vision workshops
14.
Han C, Shen F, Liu L, Yang Y, Shen HT (2018) Visual spatial attention network for relationship detection. In: ACM international conference on multimedia, pp 510–518
15.
He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 37(9):1904–1916CrossRef
16.
He T, Gao L, Song J, Cai J, Li YF (2020) Learning from the scene and borrowing from the rich: tackling the long tail in scene graph generation. arXiv preprint arXiv:​2006.​07585
17.
Jae Hwang S, Ravi SN, Tao Z, Kim HJ, Collins MD, Singh V (2018) Tensorize, factorize and regularize: robust visual relationship learning. In: IEEE conference on computer vision and pattern recognition, pp 1014–1023
18.
Jung J, Park J (2020) Improving visual relationship detection using linguistic and spatial cues. ETRI J 42(3):399–410CrossRef
19.
Kirillov A, He K, Girshick R, Rother C, Dollár P (2019) Panoptic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 9404–9413
20.
Li Y, Chen X, Zhu Z, Xie L, Huang G, Du D, Wang X (2019) Attention-guided unified network for panoptic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 7026–7035
21.
Li Y, Ouyang W, Zhou B, Shi J, Zhang C, Wang X (2018) Factorizable net: an efficient subgraph-based framework for scene graph generation. In: European conference on computer vision, pp 335–351
22.
Li YL, Xu L, Liu X, Huang X, Xu Y, Wang S, Fang HS, Ma Z, Chen M, Lu C (2020) Pastanet: toward human activity knowledge engine. In: IEEE conference on computer vision and pattern recognition, pp 382–391
23.
Liang X, Lee L, Xing EP (2017) Deep variation-structured reinforcement learning for visual relationship and attribute detection. In: IEEE conference on computer vision and pattern recognition, pp 848–857
24.
Liang, Y., Bai, Y., Zhang, W., Qian, X., Zhu, L., Mei, T (2019) Vrr-vg: Refocusing visually-relevant relationships. In: Proceedings of the IEEE international conference on computer vision, pp 10403–10412
25.
Lin TY, Dollár P, Girshick R, He K, Hariharan B, Belongie S (2017) Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2117–2125
26.
Lu, C., Krishna, R., Bernstein, M., Fei-Fei, L (2016) Visual relationship detection with language priors. In: European conference on computer vision. Springer, pp 852–869
27.
Michieli U, Borsato E, Rossi L, Zanuttigh P (2020) Gmnet: graph matching network for large scale part semantic segmentation in the wild. In: European conference on computer vision. Springer, pp 397–414
28.
29.
Plesse F, Ginsca A, Delezoide B, Prêteux F (2018) Learning prototypes for visual relationship detection. In: International conference on content-based multimedia indexing. IEEE, pp 1–6
30.
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. In: IEEE conference on computer vision and pattern recognition, pp 779–788 (2016)
31.
Redmon J, Farhadi A (2017) Yolo9000: better, faster, stronger. In: IEEE conference on computer vision and pattern recognition, pp 7263–7271
32.
Ren S, He K, Girshick R, Sun J (2016) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149CrossRef
33.
Tajrobehkar M, Tang K, Zhang H, Lim JH (2021) Align r-cnn: a pairwise head network for visual relationship detection. In: IEEE transactions on multimedia
34.
Tang K, Niu Y, Huang J, Shi J, Zhang H (2020) Unbiased scene graph generation from biased training. In: IEEE conference on computer vision and pattern recognition, pp 3716–3725
35.
Tang K, Zhang H, Wu B, Luo W, Liu W (2019) Learning to compose dynamic tree structures for visual contexts. In: IEEE conference on computer vision and pattern recognition, pp 6619–6628
36.
Tian H, Xu N, Liu AA, Zhang Y (2020) Part-aware interactive learning for scene graph generation. In: ACM international conference on multimedia, pp 3155–3163
37.
Tian Z, Shen C, Chen H, He T (2019) Fcos: fully convolutional one-stage object detection. In: IEEE international conference on computer vision, pp 9627–9636
38.
Wan B, Zhou D, Liu Y, Li R, He X (2019) Pose-aware multi-level feature network for human object interaction detection. In: IEEE international conference on computer vision, pp 9469–9478
39.
Wang P, Shen X, Lin Z, Cohen S, Price B, Yuille AL (2015) Joint object and part segmentation using deep learned potentials. In: IEEE international conference on computer vision, pp 1573–1581
40.
Wang W, Liu R, Wang M, Wang S, Chang X, Chen Y (2020) Memory-based network for scene graph with unbalanced relations. In: ACM international conference on multimedia, pp 2400–2408
41.
Wang W, Wang M, Wang S, Long G, Yao L, Qi G, Chen Y (2020) One-shot learning for long-tail visual relation detection. AAAI Conf Artif Intell 34:12225–12232
42.
Wang W, Wang R, Shan S, Chen X (2020) Sketching image gist: human-mimetic hierarchical scene graph generation. In: European conference on computer vision, pp 222–239
43.
Wen B, Luo J, Liu X, Huang L (2020) Unbiased scene graph generation via rich and fair semantic extraction. arXiv preprint arXiv:​2002.​00176
44.
Xiong Y, Liao R, Zhao H, Hu R, Bai M, Yumer E, Urtasun R (2019) Upsnet: a unified panoptic segmentation network. In: IEEE conference on computer vision and pattern recognition, pp 8818–8826
45.
Xu D, Zhu Y, Choy CB, Fei-Fei L (2017) Scene graph generation by iterative message passing. In: IEEE conference on computer vision and pattern recognition, pp 5410–5419
46.
Yang J, Lu J, Lee S, Batra D, Parikh D (2018) Graph r-cnn for scene graph generation. In: European conference on computer vision, pp 670–685
47.
Yao Q, Gong X (2018) Exploiting lstm for joint object and semantic part detection. In: Asian conference on computer vision. Springer, pp 498–512
48.
Yin G, Sheng L, Liu B, Yu N, Wang X, Shao J, Change Loy C (2018) Zoom-net: mining deep feature interactions for visual relationship recognition. In: Proceedings of the European conference on computer vision (ECCV), pp 322–338
49.
Yu F, Wang H, Ren T, Tang J, Wu G (2020) Visual relation of interest detection. In: ACM international conference on multimedia, pp 1386–1394
50.
51.
Yu R, Li A, Morariu VI, Davis LS (2017) Visual relationship detection with internal and external linguistic knowledge distillation. In: IEEE international conference on computer vision, pp 1974–1982
52.
Zellers R, Yatskar M, Thomson S, Choi Y (2018) Neural motifs: scene graph parsing with global context. In: IEEE conference on computer vision and pattern recognition, pp 5831–5840
53.
Zhan Y, Yu J, Yu T, Tao D (2019) On exploring undetermined relationships for visual relationship detection. In: IEEE conference on computer vision and pattern recognition, pp 5128–5137
54.
Zhang J, Elhoseiny M, Cohen S, Chang W, Elgammal A (2017) Relationship proposal networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5678–5686
55.
Zhang J, Zhang Y, Wu B, Fan Y, Shen F, Shen HT (2020) Dual resgcn for balanced scene graphgeneration. arXiv preprint arXiv:​2011.​04234 (2020)
56.
Zhao,Y, Li J, Zhang Y, Tian Y (2019) Multi-class part parsing with joint boundary-semantic awareness. In: IEEE international conference on computer vision, pp 9177–9186
57.
Zheng S, Chen S, Jin Q (2019) Visual relation detection with multi-level attention. In: ACM international conference on multimedia, pp 121–129
58.
Zhou Y, Fan Y (2021) Visual relation of interest detection based on part detection. In: International symposium on artificial intelligence and robotics
59.
Zhu Y, Jiang S, Li X (2017) Visual relationship detection with object spatial distribution. In: IEEE international conference on multimedia and expo. IEEE, pp 379–384
60.
Zhuang B, Liu L, Shen C, Reid I (2017) Towards context-aware interaction recognition for visual relationship detection. In: IEEE international conference on computer vision, pp 589–598
Metadata
Title
Complete interest propagation from part for visual relation of interest detection
Authors
You Zhou
Fan Yu
Publication date
02-08-2022
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 2/2023
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-022-01603-w

Other articles of this Issue 2/2023

International Journal of Machine Learning and Cybernetics 2/2023 Go to the issue

Original Article

Learning relations in human-like style for few-shot fine-grained image classification

Original Article

A multi-stage fusion instance learning method for anomalous event detection in videos

Original Article

A survey on federated learning: challenges and applications

Original Article

A novel adaptive methodology for removing spurious components in a modified incremental Gaussian mixture model

Original Article

Small target deep convolution recognition algorithm based on improved YOLOv4

Original Article

Disentangling latent space better for few-shot image-to-image translation