Skip to main content
Erschienen in:

13.11.2021 | Original Research

DeepRhole: deep learning for rhetorical role labeling of sentences in legal case documents

verfasst von: Paheli Bhattacharya, Shounak Paul, Kripabandhu Ghosh, Saptarshi Ghosh, Adam Wyner

Erschienen in: Artificial Intelligence and Law | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The task of rhetorical role labeling is to assign labels (such as Fact, Argument, Final Judgement, etc.) to sentences of a court case document. Rhetorical role labeling is an important problem in the field of Legal Analytics, since it can aid in various downstream tasks as well as enhances the readability of lengthy case documents. The task is challenging as case documents are highly various in structure and the rhetorical labels are often subjective. Previous works for automatic rhetorical role identification (i) mainly used Conditional Random Fields over manually handcrafted features, and (ii) focused on certain law domains only (e.g., Immigration cases, Rent law), and a particular jurisdiction/country (e.g., US, Canada, India). In this work, we improve upon the prior works on rhetorical role identification by proposing novel Deep Learning models for automatically identifying rhetorical roles, which substantially outperform the prior methods. Additionally, we show the effectiveness of the proposed models over documents from five different law domains, and from two different jurisdictions—the Supreme Court of India and the Supreme Court of the UK. Through extensive experiments over different variations of the Deep Learning models, including Transformer models based on BERT and LegalBERT, we show the robustness of the methods for the task. We also perform an extensive inter-annotator study and analyse the agreement of the predictions of the proposed model with the annotations by domain experts. We find that some rhetorical labels are inherently hard/subjective and both law experts and neural models frequently get confused in predicting them correctly.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
We use only the publicly available full text judgement. All other proprietary information had been removed before performing the experiments.
 
8
The word embeddings \(x_i\) can be obtained using random initialization or Law2Vec or Google News embeddings, as discussed earlier in Sect. 5.2.
 
9
Note that, during the 5-fold cross-validation, we ensured that at least one document from each domain is present in the training set (40 documents) as well as the test set (10 documents) in each fold.
 
Literatur
Zurück zum Zitat Artstein R, Poesio M (2008) Inter-coder agreement for computational linguistics. Comput Linguist 34(4):555–596CrossRef Artstein R, Poesio M (2008) Inter-coder agreement for computational linguistics. Comput Linguist 34(4):555–596CrossRef
Zurück zum Zitat Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:14090473 Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:​14090473
Zurück zum Zitat Bhattacharya P, Hiware K, Rajgaria S, Pochhi N, Ghosh K, Ghosh S (2019a) A comparative study of summarization algorithms applied to legal case judgments. In: European conference on information retrieval, Springer, pp 413–428 Bhattacharya P, Hiware K, Rajgaria S, Pochhi N, Ghosh K, Ghosh S (2019a) A comparative study of summarization algorithms applied to legal case judgments. In: European conference on information retrieval, Springer, pp 413–428
Zurück zum Zitat Bhattacharya P, Paul S, Ghosh K, Ghosh S, Wyner A (2019b) Identification of rhetorical roles of sentences in Indian legal judgments. In: legal knowledge and information systems–JURIX, pp 3–12 Bhattacharya P, Paul S, Ghosh K, Ghosh S, Wyner A (2019b) Identification of rhetorical roles of sentences in Indian legal judgments. In: legal knowledge and information systems–JURIX, pp 3–12
Zurück zum Zitat Bhattacharya P, Ghosh K, Pal A, Ghosh S (2020) Hier-spcnet: a legal statute hierarchy-based heterogeneous network for computing legal case document similarity. In: proceedings of the ACM SIGIR conference on research and development in information retrieval, pp. 1657–1660 Bhattacharya P, Ghosh K, Pal A, Ghosh S (2020) Hier-spcnet: a legal statute hierarchy-based heterogeneous network for computing legal case document similarity. In: proceedings of the ACM SIGIR conference on research and development in information retrieval, pp. 1657–1660
Zurück zum Zitat Chalkidis I, Androutsopoulos I (2017) A deep learning approach to contract element extraction. In: legal knowledge and information systems–JURIX, pp. 155–164 Chalkidis I, Androutsopoulos I (2017) A deep learning approach to contract element extraction. In: legal knowledge and information systems–JURIX, pp. 155–164
Zurück zum Zitat Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:1412.3555 Chung J, Gulcehre C, Cho K, Bengio Y (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv:​1412.​3555
Zurück zum Zitat Farzindar A, Lapalme G (2004) Letsum, an automatic legal text summarizing system. In: legal knowledge and information systems–JURIX, pp. 11–18 Farzindar A, Lapalme G (2004) Letsum, an automatic legal text summarizing system. In: legal knowledge and information systems–JURIX, pp. 11–18
Zurück zum Zitat Graves A, Fernández S, Schmidhuber J (2005) Bidirectional LSTM networks for improved phoneme classification and recognition. In: proceedings of the international conference on artificial neural networks (ICANN), pp. 799–804 Graves A, Fernández S, Schmidhuber J (2005) Bidirectional LSTM networks for improved phoneme classification and recognition. In: proceedings of the international conference on artificial neural networks (ICANN), pp. 799–804
Zurück zum Zitat Hachey B, Grover C (2006) Extractive summarisation of legal texts. Artif Intell Law 14(4):305–345CrossRef Hachey B, Grover C (2006) Extractive summarisation of legal texts. Artif Intell Law 14(4):305–345CrossRef
Zurück zum Zitat Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: proceedings of the eighteenth international conference on machine learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, ICML 01, pp. 282–289 Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: proceedings of the eighteenth international conference on machine learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, ICML 01, pp. 282–289
Zurück zum Zitat Liu CL, Chen KC (2019) Extracting the gist of chinese judgments of the supreme court. In: proceedings of the seventeenth international conference on artificial intelligence and law, pp. 73–82 Liu CL, Chen KC (2019) Extracting the gist of chinese judgments of the supreme court. In: proceedings of the seventeenth international conference on artificial intelligence and law, pp. 73–82
Zurück zum Zitat Nejadgholi I, Bougueng R, Witherspoon S (2017) A semi-supervised training method for semantic search of legal facts in canadian immigration cases. In: legal knowledge and information systems–JURIX, pp. 125–134 Nejadgholi I, Bougueng R, Witherspoon S (2017) A semi-supervised training method for semantic search of legal facts in canadian immigration cases. In: legal knowledge and information systems–JURIX, pp. 125–134
Zurück zum Zitat Pagliardini M, Gupta P, Jaggi M (2018) Unsupervised learning of sentence embeddings using compositional n-gram features. In: proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, Vol. 1, pp 528–540 Pagliardini M, Gupta P, Jaggi M (2018) Unsupervised learning of sentence embeddings using compositional n-gram features. In: proceedings of the 2018 conference of the North American chapter of the association for computational linguistics: human language technologies, Vol. 1, pp 528–540
Zurück zum Zitat Sanchez G (2019) Sentence boundary detection in legal text. In: proceedings of the natural legal language processing workshop 2019:31–38 Sanchez G (2019) Sentence boundary detection in legal text. In: proceedings of the natural legal language processing workshop 2019:31–38
Zurück zum Zitat Saravanan M, Ravindran B, Raman S (2008) Automatic identification of rhetorical roles using conditional random fields for legal document summarization. In: proceedings of the international joint conference on natural language processing: Vol. 1 Saravanan M, Ravindran B, Raman S (2008) Automatic identification of rhetorical roles using conditional random fields for legal document summarization. In: proceedings of the international joint conference on natural language processing: Vol. 1
Zurück zum Zitat Savelka J, Ashley KD (2018) Segmenting us court decisions into functional and issue specific parts. In: legal knowledge and information systems–JURIX, pp. 111–120 Savelka J, Ashley KD (2018) Segmenting us court decisions into functional and issue specific parts. In: legal knowledge and information systems–JURIX, pp. 111–120
Zurück zum Zitat Savelka J, Westermann H, Benyekhlef K, Alexander CS, Grant JC, Amariles DR, Hamdani RE, Meeùs S, Troussel A, Araszkiewicz M, Ashley KD, Ashley A, Branting K, Falduti M, Grabmair M, Harašta J, Novotná T, Tippett E, Johnson S (2021) Lex Rosetta: transfer of predictive models across languages, jurisdictions, and legal domains. In: proceedings of the international conference on artificial intelligence and law (ICAIL), pp. 129–138 Savelka J, Westermann H, Benyekhlef K, Alexander CS, Grant JC, Amariles DR, Hamdani RE, Meeùs S, Troussel A, Araszkiewicz M, Ashley KD, Ashley A, Branting K, Falduti M, Grabmair M, Harašta J, Novotná T, Tippett E, Johnson S (2021) Lex Rosetta: transfer of predictive models across languages, jurisdictions, and legal domains. In: proceedings of the international conference on artificial intelligence and law (ICAIL), pp. 129–138
Zurück zum Zitat Shao Y, Mao J, Liu Y, Ma W, Satoh K, Zhang M, Ma S (2020) Bert-pli: Modeling paragraph-level interactions for legal case retrieval. In: proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI-20, pp. 3501–3507 Shao Y, Mao J, Liu Y, Ma W, Satoh K, Zhang M, Ma S (2020) Bert-pli: Modeling paragraph-level interactions for legal case retrieval. In: proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI-20, pp. 3501–3507
Zurück zum Zitat Shulayeva O, Siddharthan A, Wyner AZ (2017) Recognizing cited facts and principles in legal judgements. Artif Intell Law 25(1):107–126CrossRef Shulayeva O, Siddharthan A, Wyner AZ (2017) Recognizing cited facts and principles in legal judgements. Artif Intell Law 25(1):107–126CrossRef
Zurück zum Zitat Venturi G (2012) Design and development of temis: a syntactically and semantically annotated corpus of italian legislative texts. In: proceedings of the workshop on semantic processing of legal texts (SPLeT 2012), pp. 1–12 Venturi G (2012) Design and development of temis: a syntactically and semantically annotated corpus of italian legislative texts. In: proceedings of the workshop on semantic processing of legal texts (SPLeT 2012), pp. 1–12
Zurück zum Zitat Walker VR, Pillaipakkamnatt K, Davidson AM, Linares M, Pesce DJ (2019) Automatic classification of rhetorical roles for sentences: comparing rule-based scripts with machine learning. In: proceedings of the workshop on automated semantic analysis of information in legal texts (with ICAIL) Walker VR, Pillaipakkamnatt K, Davidson AM, Linares M, Pesce DJ (2019) Automatic classification of rhetorical roles for sentences: comparing rule-based scripts with machine learning. In: proceedings of the workshop on automated semantic analysis of information in legal texts (with ICAIL)
Zurück zum Zitat Wang P, Yang Z, Niu S, Zhang Y, Zhang L, Niu S (2018) Modeling dynamic pairwise attention for crime classification over legal articles. In: the 41st international ACM SIGIR conference on research & development in information retrieval, pp. 485–494 Wang P, Yang Z, Niu S, Zhang Y, Zhang L, Niu S (2018) Modeling dynamic pairwise attention for crime classification over legal articles. In: the 41st international ACM SIGIR conference on research & development in information retrieval, pp. 485–494
Zurück zum Zitat Wang P, Fan Y, Niu S, Yang Z, Zhang Y, Guo J (2019a) Hierarchical matching network for crime classification. In: proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp. 325–334 Wang P, Fan Y, Niu S, Yang Z, Zhang Y, Guo J (2019a) Hierarchical matching network for crime classification. In: proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp. 325–334
Zurück zum Zitat Wang P, Fan Y, Niu S, Yang Z, Zhang Y, Guo J (2019b) Hierarchical matching network for crime classification. In: proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp. 325–334 Wang P, Fan Y, Niu S, Yang Z, Zhang Y, Guo J (2019b) Hierarchical matching network for crime classification. In: proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp. 325–334
Zurück zum Zitat Wyner A (2010) Towards annotating and extracting textual legal case elements. In: CEUR workshop proceedings vol. 605, pp. 9–18 Wyner A (2010) Towards annotating and extracting textual legal case elements. In: CEUR workshop proceedings vol. 605, pp. 9–18
Zurück zum Zitat Wyner AZ, Peters W, Katz D (2013) A case study on legal case annotation. In: legal knowledge and information systems–JURIX, pp. 165–174 Wyner AZ, Peters W, Katz D (2013) A case study on legal case annotation. In: legal knowledge and information systems–JURIX, pp. 165–174
Zurück zum Zitat Wyner AZ, Gough F, Lévy F, Lynch M, Nazarenko A (2017) On annotation of the textual contents of scottish legal instruments. In: legal knowledge and information systems–JURIX, pp. 101–106 Wyner AZ, Gough F, Lévy F, Lynch M, Nazarenko A (2017) On annotation of the textual contents of scottish legal instruments. In: legal knowledge and information systems–JURIX, pp. 101–106
Zurück zum Zitat Yamada H, Teufel S, Tokunaga T (2019) Neural network based rhetorical status classification for Japanese judgment documents. In: legal knowledge and information systems–JURIX, pp. 133–142 Yamada H, Teufel S, Tokunaga T (2019) Neural network based rhetorical status classification for Japanese judgment documents. In: legal knowledge and information systems–JURIX, pp. 133–142
Zurück zum Zitat Zhong H, Xiao C, Tu C, Zhang T, Liu Z, Sun M (2020) How does nlp benefit legal system: a summary of legal artificial intelligence. In: proceedings of the 58th annual meeting of the association for computational linguistics, pp. 5218–5230 Zhong H, Xiao C, Tu C, Zhang T, Liu Z, Sun M (2020) How does nlp benefit legal system: a summary of legal artificial intelligence. In: proceedings of the 58th annual meeting of the association for computational linguistics, pp. 5218–5230
Metadaten
Titel
DeepRhole: deep learning for rhetorical role labeling of sentences in legal case documents
verfasst von
Paheli Bhattacharya
Shounak Paul
Kripabandhu Ghosh
Saptarshi Ghosh
Adam Wyner
Publikationsdatum
13.11.2021
Verlag
Springer Netherlands
Erschienen in
Artificial Intelligence and Law / Ausgabe 1/2023
Print ISSN: 0924-8463
Elektronische ISSN: 1572-8382
DOI
https://doi.org/10.1007/s10506-021-09304-5