Skip to main content
Erschienen in: Minds and Machines 1/2020

23.02.2020 | Original Paper

A Puzzle concerning Compositionality in Machines

verfasst von: Ryan M. Nefdt

Erschienen in: Minds and Machines | Ausgabe 1/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper attempts to describe and address a specific puzzle related to compositionality in artificial networks such as Deep Neural Networks and machine learning in general. The puzzle identified here touches on a larger debate in Artificial Intelligence related to epistemic opacity but specifically focuses on computational applications of human level linguistic abilities or properties and a special difficulty with relation to these. Thus, the resulting issue is both general and unique. A partial solution is suggested.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
Janssen (2012) argues that Frege was not the source (nor an adherent) of the PoC. In fact, he argues that Frege subscribed to a quite different principle for natural language semantics. Its true origins can actually be traced further back than Frege to Lotze, Wundt and Trendelenburg, according to Janssen. Hodges (2012) goes further to trace the concept to the works of the tenth century Arab scholar Al-F\(\overline{a}\)r\(\overline{a}\)b\(\overline{i}\) who could have in turn found it in 3rd century commentaries on Aristotle.
 
2
Propositional logic is a good example of a formal language with a simple compositional semantics. The meaning of a formula is a truth value and the meaning of a complex formula is a function of the meanings/truth values of its components. Predicate logic is not as simple a matter. Following Pratt (1979), we know that “there is no function such that meaning of \(\forall x\phi\) can be specified with a constraint of the form \(\mathcal {M}(\forall x\phi )=F(\mathcal {M}(\phi ))\)” (Janssen 1997: 498). In other words, the meaning of a universally quantified formula is not straightforwardly given in terms of a function from the meaning of its parts, at least not by means of the standard Tarskian interpretation.
 
3
Parallelism has other weaknesses though. For instance, it strongly suggests a building metaphor of a step-by-step procedure mapping syntactic combination with semantic interpretation. Possible world semantics does not respect this constraint, nor do semantic formalisms with intermediary representations like Montague’s Type 2.
 
4
I neglected to give an interpretation of what is meant by “syntactic rule” here. This is a matter of theoretical perspective to a large extent. Traditionally, categorial grammars have been used as well as phrase structure grammars. However, the options are without obvious limit.
 
5
However, the precise definition of word-hood assumed from isolating languages such as English and Chinese is not generalisable to agglutinating languages such as Turkish, Yupik and Nguni languages, partly due to the vague lines between morphology and syntax in these latter families. See Nefdt (2019) for a philosophical view on the difficulty of defining words and Haspelmath (2011) for a linguistic discussion.
 
6
In some literature, parthood is defined as a partial ordering, i.e. reflexive, antisymmetric and transitive. This allows a part to be a part of itself which when viewed from the point of view of set theory seems to invite inconsistencies.
 
7
Inferentialism’s “top-down” notion of compositionality might not naturally dovetail with some of the remarks made here. They tend to take the sentence as the primary unit of meaning and derive subsentential semantic value from there. Specifically, Brandom’s account sees language as recursively structured but doesn’t see meaning as compositional. See Brandom (2007) for more. I thank Bernard Weiss for this observation.
 
8
Consider Jabberwocky sentences or Chomsky’s Colorless Green Ideas Sleep Furiously, even in the absence of knowing what the meaning is, we can still identify what the meaningful parts are (or should be).
 
9
I thank an anonymous reviewer for drawing my attention to this possibility and guiding me to seek out more general examples.
 
10
I thank an anonymous reviewer for pressing me on this point.
 
11
It might help to think of storage here. Words might be stored as units and brought up or recalled during composition independently of their internal structures. According to Baggio et al. (2012: 656) “psychologically speaking, the real issue is about ‘the balance between storage and computation’, and the role compositionality plays there”. Martin and Baggio (2019: 1) even suggest that “human behaviour, including language use and linguistic data, indicates that composing parts into complex structures does not threaten the existence of constituent parts as independent units in the system: parts and wholes exist simultaneously yet independently from one another in the mind and brain.”
 
12
There is a tendency in the classical connectionist and current machine learning literature to take compositionality to only involve a recursive relationship between primitive and compound types of some kind (van Gelder 1990, 1994; Baroni 2019). The ways in which this abstract procedure is instantiated are then the particular types of compositionality which are implemented. I think these kinds of definitions run the risk of confusing semantic compositionality with computability and/or combinatoriality. One major difference between the latter concepts and the former is that they can operate on pure strings or syntax without semantic representation. Some experiments in machine learning adopt this confusion and test for compositionality on nonce words or ungrammatical strings. However, the PoC is a semantic principle which is essentially bound up in the syntax-semantics interface and discussions which neglect this aspect can therefore fail to capture its nature.
 
13
For comparisons between AlphaGo and Deep Blue of the previous AI generation, see Schubbach (2019).
 
14
Take Schelling’s famous model of segregation. With a minor preference function (30% satisfaction) and two kinds of agents distributed randomly in a population, a macro-level segregation effect is produced. But this equilibrium is explicable in terms of features of the simulation despite the effect only showing itself after a few generations have been run.
 
15
I thank Eduoard Machery for pointing this worry out to me.
 
16
Humphrey’s does go on to define essentially epistemic opacity or “a process is essentially epistemically opaque to X if and only if it is impossible, given the nature of X, for X to know all of the epistemically relevant elements of the process” (2009: 650). It is unclear what is meant exactly by “epistemically relevant elements” here. Durán and Formanek (2018) interpret it in terms of some sort of surveyability of steps in finite time. Nevertheless, one worries about the historical applicability of some such definition in times before a particular scientific advance. Surely relativity might have seemed epistemically opaque to Newtonians? The definition assumes we have a clear grasp of the limits of our natures and knowledge.
 
17
Weisberg (2007) calls this modelling technique “multiple models idealization”.
 
18
Again see Duran and Formanek (2018) for a computational version of reliabilism as a tool to capture surveyability and epistemic access in the service of grounding trust in complex systems.
 
19
Ananny and Crawford (2016) question the ideal of transparency in computational systems itself. They discuss a number of issues with the ideal and conclude that a larger “sociotechnical” appreciation of the interaction between machines and humans is necessary in order to reconstruct the notion of accountability in computational settings. Robbins (2019) also questions transparency but offers “envelopment” of AI systems as an approach to their uncertainty or opacity, in which we contain or limit their impact on and potential harm to humans.
 
20
Technically, dynamics or updates should not preclude the possibility of transparency. Dynamic semantics based as it is on dynamic logic is not epistemically opaque in any sense relevant here and although static concepts of meaning are jettisoned for context change potentials or updates, meaningful parts are clearly identifiable. See Groenendjik and Stokhof (1990) and Veltman (1991) for clear descriptions of the general framework.
 
21
Sullivan interprets this situation as one of “link uncertainty” in which understanding the intricacies of model is not paramount but rather the epistemic opacity is generated by a lack of understanding the link between the model and target phenomenon.
 
22
Many ethical discussions have centred around the possibility or necessity of “opening the black-boxes” or the “right to explanation” (such as the EU’s General Data Protection Regulation legislation). These discussions are of course beyond the present scope but see Robbins (2019) for an alternative approach to the ethical issues around black boxes in AI.
 
23
Similarly, for the salience based methods of describing image classifier tasks discussed in Ribeiro et al. (2016).
 
24
Of course, compositionality could apply in the visual domain similarly. The argument could go as follows: people seem to interpret visual stimuli they have never encountered before and they do so in a systematic way; the best explanation is that they accomplish this by relying on the smallest interpretable parts of the stimuli and the way those parts are combined. So, visual interpretation must be compositional. I thank Zoltán Szabó for this observation.
 
25
More direct approaches to identifying structure in networks do exist. One famous example is Smolensky’s (1990) tensor product representations which aimed at capturing variable binding and symbolic processing while remaining true to the neural net architecture of classical connectionism. See McCoy et al. (2019) for a more recent adaptation of this idea on RNNs.
 
26
I thank an anonymous reviewer for pointing me in the direction of this research.
 
27
There have been some interesting comparisons between Alpha Go and Alpha Go Zero (which was not trained on human data). See Silver et al. (2017a, b) for more game comparisons.
 
Literatur
Zurück zum Zitat Ananny, M., & Crawford, K. (2016). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3), 973–989. Ananny, M., & Crawford, K. (2016). Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media & Society, 20(3), 973–989.
Zurück zum Zitat Andreas, J. (2019). Measuring compositionality in representation learning. ICLR. Andreas, J. (2019). Measuring compositionality in representation learning. ICLR.
Zurück zum Zitat Baggio, G., van Lambalgen, M., & Hagoort, P. (2012). The processing consequences of compositionality. In M. Werning, W. Hinzen, & E. Machery (Eds.), The Oxford handbook of compositionality (pp. 655–672). Oxford: Oxford University Press. Baggio, G., van Lambalgen, M., & Hagoort, P. (2012). The processing consequences of compositionality. In M. Werning, W. Hinzen, & E. Machery (Eds.), The Oxford handbook of compositionality (pp. 655–672). Oxford: Oxford University Press.
Zurück zum Zitat Barker, C., & Jacobson, P. (Eds.). (2007). Direct compositionality. Oxford: Oxford University Press. Barker, C., & Jacobson, P. (Eds.). (2007). Direct compositionality. Oxford: Oxford University Press.
Zurück zum Zitat Baroni, M. (2019). Linguistic generalization and compositionality in modern artificial neural networks. Retrieved from ArXiv preprint arXiv:1904.00157, to appear in the Philosophical Transactions of the Royal Society B. Baroni, M. (2019). Linguistic generalization and compositionality in modern artificial neural networks. Retrieved from ArXiv preprint arXiv:​1904.​00157, to appear in the Philosophical Transactions of the Royal Society B.
Zurück zum Zitat Blutner, R., Hendriks, P., De Hoop, H., & Schwartz, O. (2004). When compositionality fails to predict systematicity. In S. D. Levy, & R. Gayler (eds.), Compositional connectionism in cognitive science. papers from the AAAI fall symposium (pp. 6–11). Arlington: The AAAI Press. Blutner, R., Hendriks, P., De Hoop, H., & Schwartz, O. (2004). When compositionality fails to predict systematicity. In S. D. Levy, & R. Gayler (eds.), Compositional connectionism in cognitive science. papers from the AAAI fall symposium (pp. 6–11). Arlington: The AAAI Press.
Zurück zum Zitat Brandom, R. (1994). Making it explicit. Harvard: Harvard University Press. Brandom, R. (1994). Making it explicit. Harvard: Harvard University Press.
Zurück zum Zitat Brandom, R. (2007). Inferentialism and some of its challenges. Philosophy and Phenomenological Research, 74(3), 651–676. Brandom, R. (2007). Inferentialism and some of its challenges. Philosophy and Phenomenological Research, 74(3), 651–676.
Zurück zum Zitat Chomsky, N. (1982). Some concepts and consequences of the theory of government and binding. Cambridge: MIT Press. Chomsky, N. (1982). Some concepts and consequences of the theory of government and binding. Cambridge: MIT Press.
Zurück zum Zitat Cooper, R. (1975). Montague’s semantic theory and transformational syntax. Ph.D. Thesis, University of Massachusetts, Amherst. Cooper, R. (1975). Montague’s semantic theory and transformational syntax. Ph.D. Thesis, University of Massachusetts, Amherst.
Zurück zum Zitat Croft, W. (2001). Radical construction grammar. Oxford: Oxford University Press. Croft, W. (2001). Radical construction grammar. Oxford: Oxford University Press.
Zurück zum Zitat Davidson, D. (1967). Inquiries into truth and interpretation: Philosophical essays. Oxford: Oxford Clarendon Press. Davidson, D. (1967). Inquiries into truth and interpretation: Philosophical essays. Oxford: Oxford Clarendon Press.
Zurück zum Zitat Dever, J. (1999). Compositionality as methodology. Linguistics and Philosophy, 22(3), 311–326. Dever, J. (1999). Compositionality as methodology. Linguistics and Philosophy, 22(3), 311–326.
Zurück zum Zitat Dever, J. (2012). Compositionality. In The Routledge handbook to the philosophy of language (pp. 91–102). Dever, J. (2012). Compositionality. In The Routledge handbook to the philosophy of language (pp. 91–102).
Zurück zum Zitat Dowty, D. (1979). Word meaning and montague grammar: The semantics of verbs and times in generative semantics and in Montague’s PTQ. Dordrecht: Reidel. Dowty, D. (1979). Word meaning and montague grammar: The semantics of verbs and times in generative semantics and in Montague’s PTQ. Dordrecht: Reidel.
Zurück zum Zitat Dowty, D. (2007). Compositionality as an empirical problem. In C. Barker & P. Jacobson (Eds.), Direct compositionality (pp. 23–101). Oxford: Oxford University Press. Dowty, D. (2007). Compositionality as an empirical problem. In C. Barker & P. Jacobson (Eds.), Direct compositionality (pp. 23–101). Oxford: Oxford University Press.
Zurück zum Zitat Durán, J., & Formanek, N. (2018). Grounds for trust: Essential epistemic opacity and computational reliabilism. Minds and Machines, 28, 645–666. Durán, J., & Formanek, N. (2018). Grounds for trust: Essential epistemic opacity and computational reliabilism. Minds and Machines, 28, 645–666.
Zurück zum Zitat Elman, J. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–225. Elman, J. (1991). Distributed representations, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–225.
Zurück zum Zitat Evans, G. (1981). Semantic theory and tacit knowledge. Collected papers (pp. 322–342). Oxford: Clarendon Press. Evans, G. (1981). Semantic theory and tacit knowledge. Collected papers (pp. 322–342). Oxford: Clarendon Press.
Zurück zum Zitat Fodor, J., & Pylyshyn, Z. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1–2), 3–71. Fodor, J., & Pylyshyn, Z. (1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1–2), 3–71.
Zurück zum Zitat Frege, G. (1908). Über Sinn und Bedeutung. Zeitschrift fir Philosophie und philosophische Kritik 100 (1892) 25–50; translated as ’On Sense and Reference’ in P. T. Geach and M. Black, Translations from the Philosophical Writings of Gottlob Frege, Blackwell, Oxford, 1960. Frege, G. (1908). Über Sinn und Bedeutung. Zeitschrift fir Philosophie und philosophische Kritik 100 (1892) 25–50; translated as ’On Sense and Reference’ in P. T. Geach and M. Black, Translations from the Philosophical Writings of Gottlob Frege, Blackwell, Oxford, 1960.
Zurück zum Zitat Frege, G. (1919). Notes for Ludwig Darmstaedter (Logik in der Mathematik), in Frege 1979: 253–257. Frege, G. (1919). Notes for Ludwig Darmstaedter (Logik in der Mathematik), in Frege 1979: 253–257.
Zurück zum Zitat Frigg, R., & Reiss, J. (2009). The philosophy of simulation: Hot new issues or same old stew? Synthese, 169, 593–613.MathSciNet Frigg, R., & Reiss, J. (2009). The philosophy of simulation: Hot new issues or same old stew? Synthese, 169, 593–613.MathSciNet
Zurück zum Zitat Fodor, J. (1983). The modularity of mind. Cambridge: MIT Press. Fodor, J. (1983). The modularity of mind. Cambridge: MIT Press.
Zurück zum Zitat Goldberg, A. (2015). Compositionality. In N. Reimer (Ed.), The Routledge handbook of semantics (pp. 419–433). London: Routledge. Goldberg, A. (2015). Compositionality. In N. Reimer (Ed.), The Routledge handbook of semantics (pp. 419–433). London: Routledge.
Zurück zum Zitat Goldberg, Y. (2017). Neural network methods for natural language processing. San Francisco: Morgan & Claypool. Goldberg, Y. (2017). Neural network methods for natural language processing. San Francisco: Morgan & Claypool.
Zurück zum Zitat Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. Cambridge: MIT Press.MATH Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. Cambridge: MIT Press.MATH
Zurück zum Zitat Groenendijk, J., & Stokhof, M. (1990). Dynamic Montague grammar. In L. Kalman & L. Polos (Eds.), Papers from the second symposium on logic and language (pp. 3–48). Akademiai Kiadoo: Budapest. Groenendijk, J., & Stokhof, M. (1990). Dynamic Montague grammar. In L. Kalman & L. Polos (Eds.), Papers from the second symposium on logic and language (pp. 3–48). Akademiai Kiadoo: Budapest.
Zurück zum Zitat Groenendijk, J., & Stokhof, M. (2005). Why compositionality? In G. Carlson & J. Pelletier (Eds.), Reference and quantification: The partee effect (pp. 83–106). Stanford: CSLI Press. Groenendijk, J., & Stokhof, M. (2005). Why compositionality? In G. Carlson & J. Pelletier (Eds.), Reference and quantification: The partee effect (pp. 83–106). Stanford: CSLI Press.
Zurück zum Zitat Gulordava, K., Bojanowski, P., Grave, E., Linzen, T. & Baroni, M. (2018). Colorless green recurrent networks dream hierarchically. In Proceedings of NAACL, pp 1195–1205, New Orleans, LA. Gulordava, K., Bojanowski, P., Grave, E., Linzen, T. & Baroni, M. (2018). Colorless green recurrent networks dream hierarchically. In Proceedings of NAACL, pp 1195–1205, New Orleans, LA.
Zurück zum Zitat Haspelmath, M. (2011). The indeterminacy of word segmentation and the nature of morphology and syntax. Folia Linguistica, 45(1), 31–80. Haspelmath, M. (2011). The indeterminacy of word segmentation and the nature of morphology and syntax. Folia Linguistica, 45(1), 31–80.
Zurück zum Zitat Heim, I., & Kratzer, A. (1998). Semantics in generative grammar. Oxford: Blackwell. Heim, I., & Kratzer, A. (1998). Semantics in generative grammar. Oxford: Blackwell.
Zurück zum Zitat Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.
Zurück zum Zitat Hodges, W. (2012). Formalizing the relationship between meaning and syntax. In M. Werning, W. Hinzen, & E. Machery (Eds.), The oxford handbook of compositionality (pp. 245–261). Oxford: Oxford University Press. Hodges, W. (2012). Formalizing the relationship between meaning and syntax. In M. Werning, W. Hinzen, & E. Machery (Eds.), The oxford handbook of compositionality (pp. 245–261). Oxford: Oxford University Press.
Zurück zum Zitat Humphreys, P. (2009). The philosophical novelty of computer simulation methods. Synthese, 169, 615–626.MathSciNet Humphreys, P. (2009). The philosophical novelty of computer simulation methods. Synthese, 169, 615–626.MathSciNet
Zurück zum Zitat Hupkes, D., Dankers, V., Mul, M., Bruni, E. (2019). The compositionality of neural networks: Integrating symbolism and connectionism. Retrieved from arXiv:1908.08351. Hupkes, D., Dankers, V., Mul, M., Bruni, E. (2019). The compositionality of neural networks: Integrating symbolism and connectionism. Retrieved from arXiv:​1908.​08351.
Zurück zum Zitat Jackendoff, R. (1990). Semantic structures. Cambridge: MIT Press. Jackendoff, R. (1990). Semantic structures. Cambridge: MIT Press.
Zurück zum Zitat Jackendoff, R. (2002). The foundations of language: Brain, meaning, grammar, evolution. Oxford: Oxford University Press. Jackendoff, R. (2002). The foundations of language: Brain, meaning, grammar, evolution. Oxford: Oxford University Press.
Zurück zum Zitat Jacobson, P. (2002). The (dis)organization of the grammar: 25 years. Linguistics and Philosophy, 25, 601–26. Jacobson, P. (2002). The (dis)organization of the grammar: 25 years. Linguistics and Philosophy, 25, 601–26.
Zurück zum Zitat Jacobson, R. (1958/1984). Morphological observations on Slavic declension (the structure of Russian case forms). In L. R. Waugh & M. Halle (eds.), Roman Jakobson. Russian and Slavic grammar: Studies 1931–1981 (pp. 105–133). Berlin: Mouton de Gruyter. Jacobson, R. (1958/1984). Morphological observations on Slavic declension (the structure of Russian case forms). In L. R. Waugh & M. Halle (eds.), Roman Jakobson. Russian and Slavic grammar: Studies 1931–1981 (pp. 105–133). Berlin: Mouton de Gruyter.
Zurück zum Zitat Janssen, T. (1997). Compositionality. In J. van Benthem & A. ter Meulen (Eds.), Handbook of logic and language (pp. 417–473). Amsterdam: Elsevier Science. Janssen, T. (1997). Compositionality. In J. van Benthem & A. ter Meulen (Eds.), Handbook of logic and language (pp. 417–473). Amsterdam: Elsevier Science.
Zurück zum Zitat Janssen, T. (2012). Compositionality: Its historic context. In M. Werning, W. Hinzen & E. Machery (eds.) (pp. 19–46). Janssen, T. (2012). Compositionality: Its historic context. In M. Werning, W. Hinzen & E. Machery (eds.) (pp. 19–46).
Zurück zum Zitat Johnson, K. (2004). On the systematicity of language and thought. Journal of Philosophy, 101, 111–139. Johnson, K. (2004). On the systematicity of language and thought. Journal of Philosophy, 101, 111–139.
Zurück zum Zitat Johnson, K. (2015). Notational variants and invariance in linguistics. Mind and Language, 30(2), 162–186. Johnson, K. (2015). Notational variants and invariance in linguistics. Mind and Language, 30(2), 162–186.
Zurück zum Zitat Kay, P., & Michaelis, L. (2011). Constructional meaning and compositionality. In C. Maienborn, K. von Heusinger, & P. Portner (Eds.), Semantics: An international handbook of natural language meaning. Berlin: Mouton de Gruyter. Kay, P., & Michaelis, L. (2011). Constructional meaning and compositionality. In C. Maienborn, K. von Heusinger, & P. Portner (Eds.), Semantics: An international handbook of natural language meaning. Berlin: Mouton de Gruyter.
Zurück zum Zitat Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems (pp. 1097–1105). Krizhevsky, A., Sutskever, I., & Hinton, G. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems (pp. 1097–1105).
Zurück zum Zitat Lake, B., & Baroni, M. (2018). Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In Proceedings of ICML, pp 2879–2888, Stockholm, Sweden. Lake, B., & Baroni, M. (2018). Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In Proceedings of ICML, pp 2879–2888, Stockholm, Sweden.
Zurück zum Zitat LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521, 436–444.
Zurück zum Zitat Lei, T., Barzilay, R., & Jaakkola, T. (2016). Rationalizing neural predictions. In Proceedings of the 2016 conference on empirical methods in natural language processing. Association for Computational Linguistics. Lei, T., Barzilay, R., & Jaakkola, T. (2016). Rationalizing neural predictions. In Proceedings of the 2016 conference on empirical methods in natural language processing. Association for Computational Linguistics.
Zurück zum Zitat Lenhard, J., & Winsberg, E. (2010). Holism, entrenchment, and the future of climate model pluralism. Studies in History and Philosophy of Modern Physics, 41, 253–262. Lenhard, J., & Winsberg, E. (2010). Holism, entrenchment, and the future of climate model pluralism. Studies in History and Philosophy of Modern Physics, 41, 253–262.
Zurück zum Zitat Leśniewski, S. (1916). Podstawy ogólnej teoryi mnogości. I, Moskow: Prace Polskiego Kola Naukowego w Moskwie, Sekcya matematyczno-przyrodnicza; Eng. trans. by D. I. Barnett: ‘Foundations of the General Theory of Sets. I’, in S. Leśniewski, Collected Works (ed. by S. J. Surma et al.), Dordrecht: Kluwer, 1992, vol. 1, (pp. 129–173). Leśniewski, S. (1916). Podstawy ogólnej teoryi mnogości. I, Moskow: Prace Polskiego Kola Naukowego w Moskwie, Sekcya matematyczno-przyrodnicza; Eng. trans. by D. I. Barnett: ‘Foundations of the General Theory of Sets. I’, in S. Leśniewski, Collected Works (ed. by S. J. Surma et al.), Dordrecht: Kluwer, 1992, vol. 1, (pp. 129–173).
Zurück zum Zitat Liang, P., & Potts, C. (2015). Bringing machine learning and compositional semantics together. Annual Reviews of Linguistics, 1(1), 355–376. Liang, P., & Potts, C. (2015). Bringing machine learning and compositional semantics together. Annual Reviews of Linguistics, 1(1), 355–376.
Zurück zum Zitat Marcus, G. (2003). The algebraic mind. Cambridge: MIT Press. Marcus, G. (2003). The algebraic mind. Cambridge: MIT Press.
Zurück zum Zitat Marr, D. (1982). Vision. New York: W.H. Freeman and Company. Marr, D. (1982). Vision. New York: W.H. Freeman and Company.
Zurück zum Zitat Martins, A., & Baggio, G. (2019). Modelling meaning composition from formalism to mechanism. Philosophical Transactions of the Royal Society B 375. Martins, A., & Baggio, G. (2019). Modelling meaning composition from formalism to mechanism. Philosophical Transactions of the Royal Society B 375.
Zurück zum Zitat McCoy, T., Linzen, T., Dunbar, E., & Smolensky, P. (2019). RNNs implicitly implement tensor product representations. ICLR. McCoy, T., Linzen, T., Dunbar, E., & Smolensky, P. (2019). RNNs implicitly implement tensor product representations. ICLR.
Zurück zum Zitat Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016). Deep patient: An unsupervised representation to predict the future of patients from the electronic health records. Scientific Reports, 6(26094), 1–10. Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016). Deep patient: An unsupervised representation to predict the future of patients from the electronic health records. Scientific Reports, 6(26094), 1–10.
Zurück zum Zitat Montague, R. (1974). The proper treatment of quantification in ordinary English. Approaches to natural language (pp. 221–242). Dordrecht: Springer. Montague, R. (1974). The proper treatment of quantification in ordinary English. Approaches to natural language (pp. 221–242). Dordrecht: Springer.
Zurück zum Zitat Morgan, J. (1969). On arguing about semantics. Papers in Linguistics, 1, 49–70. Morgan, J. (1969). On arguing about semantics. Papers in Linguistics, 1, 49–70.
Zurück zum Zitat Müller, V. (2019). Ethics of AI and robotics. In E. Zalta (Ed.), Stanford encyclopedia of philosophy. Palo Alto: CSLI, Stanford University. Müller, V. (2019). Ethics of AI and robotics. In E. Zalta (Ed.), Stanford encyclopedia of philosophy. Palo Alto: CSLI, Stanford University.
Zurück zum Zitat Nefdt, R. (2019). The ontology of words: A structural approach. Inquiry, 62(8), 877–911. Nefdt, R. (2019). The ontology of words: A structural approach. Inquiry, 62(8), 877–911.
Zurück zum Zitat Newman, J. (2016). Epistemic opacity, confirmation holism and technical debt: Computer simulation in the light of empirical software engineering. In F. Gadducci & M. Tavosanis (Eds.), History and philosophy of computing—third international conference, HaPoC 2015, Pisa, Italy, October 8–11, 2015, Revised Selected Papers (pp. 256–272). Dordrecht: Springer. Newman, J. (2016). Epistemic opacity, confirmation holism and technical debt: Computer simulation in the light of empirical software engineering. In F. Gadducci & M. Tavosanis (Eds.), History and philosophy of computing—third international conference, HaPoC 2015, Pisa, Italy, October 8–11, 2015, Revised Selected Papers (pp. 256–272). Dordrecht: Springer.
Zurück zum Zitat Pagin, P., & Westerstahl, D. (2010). Compositionality I: Definitions and variants. Philosophy Compass, 5(3), 250–264. Pagin, P., & Westerstahl, D. (2010). Compositionality I: Definitions and variants. Philosophy Compass, 5(3), 250–264.
Zurück zum Zitat Partee, B. (2004). Compositionality in formal semantics. Oxford: Blackwell. Partee, B. (2004). Compositionality in formal semantics. Oxford: Blackwell.
Zurück zum Zitat Pelletier, J. (2012). Holism and compositionality. In M. Werning, W. Hinzen & E. Machery (eds.) (pp. 149–174). Pelletier, J. (2012). Holism and compositionality. In M. Werning, W. Hinzen & E. Machery (eds.) (pp. 149–174).
Zurück zum Zitat Pietroski, P. (2018). Conjoining meanings: Semantics without truth values. Oxford: Oxford University Press. Pietroski, P. (2018). Conjoining meanings: Semantics without truth values. Oxford: Oxford University Press.
Zurück zum Zitat Pinker, S. (1984). Language learnability and language development. Cambridge: Harvard University Press. Pinker, S. (1984). Language learnability and language development. Cambridge: Harvard University Press.
Zurück zum Zitat Plebe, A., & Grasso, G. (2019). The unbearable shallow understanding of deep learning. Minds and Machines, 29, 515–553. Plebe, A., & Grasso, G. (2019). The unbearable shallow understanding of deep learning. Minds and Machines, 29, 515–553.
Zurück zum Zitat Pratt, V. R. (1979). Models of program logics. In 20th Annual Symposium on Foundations of Computer Science (sfcs 1979), San Juan, Puerto Rico, USA, pp. 115–122. Pratt, V. R. (1979). Models of program logics. In 20th Annual Symposium on Foundations of Computer Science (sfcs 1979), San Juan, Puerto Rico, USA, pp. 115–122.
Zurück zum Zitat Pustejovsky, J. (1995). The generative lexicon. Cambridge: The MIT Press. Pustejovsky, J. (1995). The generative lexicon. Cambridge: The MIT Press.
Zurück zum Zitat Pylyshyn, Z. (1984). Computation and cognition. Cambridge: MIT Press. Pylyshyn, Z. (1984). Computation and cognition. Cambridge: MIT Press.
Zurück zum Zitat Rambow, O., & Joshi, A. (1992). A formal look at dependency grammars and phrase structure grammars, with special consideration of word-order phenomena. In International workshop on the meaning-text theory. Darmstadt. Arbeitspapiere der GMD, 671, 47–66. Rambow, O., & Joshi, A. (1992). A formal look at dependency grammars and phrase structure grammars, with special consideration of word-order phenomena. In International workshop on the meaning-text theory. Darmstadt. Arbeitspapiere der GMD, 671, 47–66.
Zurück zum Zitat Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I Trust You? Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144). Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I Trust You? Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1135–1144).
Zurück zum Zitat Rumelhart, D., McClelland, J., & Research Group, P. D. P. (Eds.). (1986). Parallel distributed processing: Explorations in the microstructure of cognition: Foundations (Vol. 1). Cambridge: MIT Press. Rumelhart, D., McClelland, J., & Research Group, P. D. P. (Eds.). (1986). Parallel distributed processing: Explorations in the microstructure of cognition: Foundations (Vol. 1). Cambridge: MIT Press.
Zurück zum Zitat Schubbach, A. (2019). Judging machines: Philosophical aspects of deep learning. Synthese (online first). Schubbach, A. (2019). Judging machines: Philosophical aspects of deep learning. Synthese (online first).
Zurück zum Zitat Silver, D., Huang, A., Maddison, C., Guez, A., Sifre, L., van den Driessche, G., et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489. Silver, D., Huang, A., Maddison, C., Guez, A., Sifre, L., van den Driessche, G., et al. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484–489.
Zurück zum Zitat Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2017a). Mastering chess and Shogi by self-play with a general reinforcement learning algorithm. Retrieved from arXiv preprint arXiv:1712.01815. Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2017a). Mastering chess and Shogi by self-play with a general reinforcement learning algorithm. Retrieved from arXiv preprint arXiv:​1712.​01815.
Zurück zum Zitat Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Arthur Guez, A., et al. (2017b). Mastering the game of go without human knowledge. Nature, 550, 354–359. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Arthur Guez, A., et al. (2017b). Mastering the game of go without human knowledge. Nature, 550, 354–359.
Zurück zum Zitat Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1–2), 159–216.MathSciNetMATH Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1–2), 159–216.MathSciNetMATH
Zurück zum Zitat Stöckler, M. (2000). On modelling and simulations as instruments for the study of complex systems. In M. Carrier (Ed.), Science at century’s end: Philosophical questions on the progress and limits of science. Pittsburgh: University of Pittsburgh Press. Stöckler, M. (2000). On modelling and simulations as instruments for the study of complex systems. In M. Carrier (Ed.), Science at century’s end: Philosophical questions on the progress and limits of science. Pittsburgh: University of Pittsburgh Press.
Zurück zum Zitat Sullivan, E. (2019). Understanding from machine learning models. British Journal of the Philosophy of Science. (forthcoming). Sullivan, E. (2019). Understanding from machine learning models. British Journal of the Philosophy of Science. (forthcoming).
Zurück zum Zitat Sutskever, I., Vinyals, O., & Le, Q. (2014). Sequence to sequence learning with neural networks. In Proceedings of NIPS (pp. 3104–3112). Montreal, Canada. Sutskever, I., Vinyals, O., & Le, Q. (2014). Sequence to sequence learning with neural networks. In Proceedings of NIPS (pp. 3104–3112). Montreal, Canada.
Zurück zum Zitat Szabó, Z. (2000). The Problem of compositionality. Abingdon: Routledge Press. Szabó, Z. (2000). The Problem of compositionality. Abingdon: Routledge Press.
Zurück zum Zitat Szabó, Z. (2012). The case for compositionality. In M. Werning, W. Hinzen & E. Machery (eds.) (pp. 64–80). Szabó, Z. (2012). The case for compositionality. In M. Werning, W. Hinzen & E. Machery (eds.) (pp. 64–80).
Zurück zum Zitat Tarski, A. (1933). The concept of truth in the languages of the deductive sciences. Reprinted in Zygmunt 1995 (pp. 13–172); expanded English translation in Tarski 1983 [1956] (pp. 152–278). Tarski, A. (1933). The concept of truth in the languages of the deductive sciences. Reprinted in Zygmunt 1995 (pp. 13–172); expanded English translation in Tarski 1983 [1956] (pp. 152–278).
Zurück zum Zitat van Gelder, T. (1990). Compositionality: A connectionist variation on a classical theme. Cognitive Science, 14, 355–384. van Gelder, T. (1990). Compositionality: A connectionist variation on a classical theme. Cognitive Science, 14, 355–384.
Zurück zum Zitat van Gelder, T. J., & Port, R. (1994). Beyond symbolic: Towards a Kama-Sutra of compositionality. In V. Honavar & L. Uhr (Eds.), Artificial intelligence and neural networks: Steps toward principled integration (p. 1071–25). San Diego: Academic Press. van Gelder, T. J., & Port, R. (1994). Beyond symbolic: Towards a Kama-Sutra of compositionality. In V. Honavar & L. Uhr (Eds.), Artificial intelligence and neural networks: Steps toward principled integration (p. 1071–25). San Diego: Academic Press.
Zurück zum Zitat Veltman, F. (1991). Defaults in update semantics. In Hans Kamp (Ed.), Conditionals, defaults and belief revision. Dyana Deliverable R2.5A: Edinburgh. Veltman, F. (1991). Defaults in update semantics. In Hans Kamp (Ed.), Conditionals, defaults and belief revision. Dyana Deliverable R2.5A: Edinburgh.
Zurück zum Zitat Weisberg, M. (2007). Three kinds of idealization. The Journal of Philosophy, 104(12), 639–659. Weisberg, M. (2007). Three kinds of idealization. The Journal of Philosophy, 104(12), 639–659.
Zurück zum Zitat Werning, M. (2005). Right and wrong reasons for compositionality. In M. Werning (Ed.), The Compositionality of Meaning and Content (vol. 1, Foundational Issues, pp. 285–309). Frankfurt: Ontos Verlag. Werning, M. (2005). Right and wrong reasons for compositionality. In M. Werning (Ed.), The Compositionality of Meaning and Content (vol. 1, Foundational Issues, pp. 285–309). Frankfurt: Ontos Verlag.
Zurück zum Zitat Werning, M. (2012). Non-symbolic compositional representation and its neuronal foundation: Towards an emulative semantics. In M. Werning, W. Hinzen, & E. Machery (Eds.), The Oxford handbook of compositionality (pp. 633–654). Oxford: Oxford University Press. Werning, M. (2012). Non-symbolic compositional representation and its neuronal foundation: Towards an emulative semantics. In M. Werning, W. Hinzen, & E. Machery (Eds.), The Oxford handbook of compositionality (pp. 633–654). Oxford: Oxford University Press.
Zurück zum Zitat Wittgenstein, L. (1953). Philosophical investigations. In G. Anscombe & R. Rhees (Eds.), G.E.M. Anscombe (trans.). Oxford: Blackwell. Wittgenstein, L. (1953). Philosophical investigations. In G. Anscombe & R. Rhees (Eds.), G.E.M. Anscombe (trans.). Oxford: Blackwell.
Metadaten
Titel
A Puzzle concerning Compositionality in Machines
verfasst von
Ryan M. Nefdt
Publikationsdatum
23.02.2020
Verlag
Springer Netherlands
Erschienen in
Minds and Machines / Ausgabe 1/2020
Print ISSN: 0924-6495
Elektronische ISSN: 1572-8641
DOI
https://doi.org/10.1007/s11023-020-09519-6

Weitere Artikel der Ausgabe 1/2020

Minds and Machines 1/2020 Zur Ausgabe