Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2017 | OriginalPaper | Chapter

Chemical Boltzmann Machines

Authors: William Poole, Andrés Ortiz-Muñoz, Abhishek Behera, Nick S. Jones, Thomas E. Ouldridge, Erik Winfree, Manoj Gopalkrishnan

Published in: DNA Computing and Molecular Programming

Publisher: Springer International Publishing

share
SHARE

Abstract

How smart can a micron-sized bag of chemicals be? How can an artificial or real cell make inferences about its environment? From which kinds of probability distributions can chemical reaction networks sample? We begin tackling these questions by showing three ways in which a stochastic chemical reaction network can implement a Boltzmann machine, a stochastic neural network model that can generate a wide range of probability distributions and compute conditional probabilities. The resulting models, and the associated theorems, provide a road map for constructing chemical reaction networks that exploit their native stochasticity as a computational resource. Finally, to show the potential of our models, we simulate a chemical Boltzmann machine to classify and generate MNIST digits in-silico.
Appendix
Available only for authorised users
Literature
1.
go back to reference Bray, D.: Protein molecules as computational elements in living cells. Nature 376(6538), 307 (1995) CrossRef Bray, D.: Protein molecules as computational elements in living cells. Nature 376(6538), 307 (1995) CrossRef
2.
go back to reference Bray, D.: Wetware: A Computer in Every Living Cell. Yale University Press, New Haven (2009) Bray, D.: Wetware: A Computer in Every Living Cell. Yale University Press, New Haven (2009)
3.
go back to reference McAdams, H.H., Arkin, A.: Stochastic mechanisms in gene expression. Proc. Natl. Acad. Sci. 94(3), 814–819 (1997) CrossRef McAdams, H.H., Arkin, A.: Stochastic mechanisms in gene expression. Proc. Natl. Acad. Sci. 94(3), 814–819 (1997) CrossRef
4.
go back to reference Elowitz, M.B., Levine, A.J., Siggia, E.D., Swain, P.S.: Stochastic gene expression in a single cell. Science 297(5584), 1183–1186 (2002) CrossRef Elowitz, M.B., Levine, A.J., Siggia, E.D., Swain, P.S.: Stochastic gene expression in a single cell. Science 297(5584), 1183–1186 (2002) CrossRef
5.
go back to reference Perkins, T.J., Swain, P.S.: Strategies for cellular decision making. Mol. Syst. Biol. 5(1), 326 (2009) Perkins, T.J., Swain, P.S.: Strategies for cellular decision making. Mol. Syst. Biol. 5(1), 326 (2009)
6.
go back to reference Muroga, S.: Threshold Logic and Its Applications. Wiley Interscience, New York (1971) MATH Muroga, S.: Threshold Logic and Its Applications. Wiley Interscience, New York (1971) MATH
7.
go back to reference Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982) MathSciNetCrossRef Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. 79(8), 2554–2558 (1982) MathSciNetCrossRef
8.
go back to reference Hinton, G.E., Sejnowski, T.J., Ackley, D.H.: Boltzmann Machines: Constraint Satisfaction Networks that Learn. Department of Computer Science, Carnegie-Mellon University, Pittsburgh (1984) Hinton, G.E., Sejnowski, T.J., Ackley, D.H.: Boltzmann Machines: Constraint Satisfaction Networks that Learn. Department of Computer Science, Carnegie-Mellon University, Pittsburgh (1984)
9.
go back to reference Bray, D.: Intracellular signalling as a parallel distributed process. J. Theor. Biol. 143(2), 215–231 (1990) CrossRef Bray, D.: Intracellular signalling as a parallel distributed process. J. Theor. Biol. 143(2), 215–231 (1990) CrossRef
10.
go back to reference Hellingwerf, K.J., Postma, P.W., Tommassen, J., Westerhoff, H.V.: Signal transduction in bacteria: phospho-neural network(s) in Escherichia coli. FEMS Microbiol. Rev. 16(4), 309–321 (1995) CrossRef Hellingwerf, K.J., Postma, P.W., Tommassen, J., Westerhoff, H.V.: Signal transduction in bacteria: phospho-neural network(s) in Escherichia coli. FEMS Microbiol. Rev. 16(4), 309–321 (1995) CrossRef
11.
go back to reference Mjolsness, E., Sharp, D.H., Reinitz, J.: A connectionist model of development. J. Theor. Biol. 152(4), 429–453 (1991) CrossRef Mjolsness, E., Sharp, D.H., Reinitz, J.: A connectionist model of development. J. Theor. Biol. 152(4), 429–453 (1991) CrossRef
12.
go back to reference Mestl, T., Lemay, C., Glass, L.: Chaos in high-dimensional neural and gene networks. Physica D: Nonlin. Phenom. 98(1), 33–52 (1996) MathSciNetCrossRefMATH Mestl, T., Lemay, C., Glass, L.: Chaos in high-dimensional neural and gene networks. Physica D: Nonlin. Phenom. 98(1), 33–52 (1996) MathSciNetCrossRefMATH
13.
go back to reference Buchler, N.E., Gerland, U., Hwa, T.: On schemes of combinatorial transcription logic. Proc. Natl. Acad. Sci. 100(9), 5136–5141 (2003) CrossRef Buchler, N.E., Gerland, U., Hwa, T.: On schemes of combinatorial transcription logic. Proc. Natl. Acad. Sci. 100(9), 5136–5141 (2003) CrossRef
16.
go back to reference Hjelmfelt, A., Weinberger, E.D., Ross, J.: Chemical implementation of neural networks and turing machines. Proc. Natl. Acad. Sci. 88(24), 10983–10987 (1991) CrossRefMATH Hjelmfelt, A., Weinberger, E.D., Ross, J.: Chemical implementation of neural networks and turing machines. Proc. Natl. Acad. Sci. 88(24), 10983–10987 (1991) CrossRefMATH
17.
go back to reference Hjelmfelt, A., Ross, J.: Chemical implementation and thermodynamics of collective neural networks. Proc. Natl. Acad. Sci. 89(1), 388–391 (1992) CrossRef Hjelmfelt, A., Ross, J.: Chemical implementation and thermodynamics of collective neural networks. Proc. Natl. Acad. Sci. 89(1), 388–391 (1992) CrossRef
18.
go back to reference Kim, J., Hopfield, J.J., Winfree, E.: Neural network computation by in vitro transcriptional circuits. In: Advances in Neural Information Processing Systems (NIPS), pp. 681–688 (2004) Kim, J., Hopfield, J.J., Winfree, E.: Neural network computation by in vitro transcriptional circuits. In: Advances in Neural Information Processing Systems (NIPS), pp. 681–688 (2004)
19.
go back to reference Napp, N.E., Adams, R.P.: Message passing inference with chemical reaction networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 2247–2255 (2013) Napp, N.E., Adams, R.P.: Message passing inference with chemical reaction networks. In: Advances in Neural Information Processing Systems (NIPS), pp. 2247–2255 (2013)
20.
21.
go back to reference Hjelmfelt, A., Schneider, F.W., Ross, J.: Pattern recognition in coupled chemical kinetic systems. Science 260, 335–335 (1993) CrossRef Hjelmfelt, A., Schneider, F.W., Ross, J.: Pattern recognition in coupled chemical kinetic systems. Science 260, 335–335 (1993) CrossRef
22.
go back to reference Kim, J., White, K.S., Winfree, E.: Construction of an in vitro bistable circuit from synthetic transcriptional switches. Mol. Syst. Biol. 2, 68 (2006) CrossRef Kim, J., White, K.S., Winfree, E.: Construction of an in vitro bistable circuit from synthetic transcriptional switches. Mol. Syst. Biol. 2, 68 (2006) CrossRef
23.
go back to reference Kim, J., Winfree, E.: Synthetic in vitro transcriptional oscillators. Mol. Syst. Biol. 7, 465 (2011) CrossRef Kim, J., Winfree, E.: Synthetic in vitro transcriptional oscillators. Mol. Syst. Biol. 7, 465 (2011) CrossRef
24.
go back to reference Qian, L., Winfree, E., Bruck, J.: Neural network computation with DNA strand displacement cascades. Nature 475(7356), 368–372 (2011) CrossRef Qian, L., Winfree, E., Bruck, J.: Neural network computation with DNA strand displacement cascades. Nature 475(7356), 368–372 (2011) CrossRef
25.
go back to reference Lestas, I., Paulsson, J., Ross, N.E., Vinnicombe, G.: Noise in gene regulatory networks. IEEE Trans. Autom. Control 53, 189–200 (2008) MathSciNetCrossRefMATH Lestas, I., Paulsson, J., Ross, N.E., Vinnicombe, G.: Noise in gene regulatory networks. IEEE Trans. Autom. Control 53, 189–200 (2008) MathSciNetCrossRefMATH
26.
go back to reference Lestas, I., Vinnicombe, G., Paulsson, J.: Fundamental limits on the suppression of molecular fluctuations. Nature 467(7312), 174–178 (2010) CrossRef Lestas, I., Vinnicombe, G., Paulsson, J.: Fundamental limits on the suppression of molecular fluctuations. Nature 467(7312), 174–178 (2010) CrossRef
27.
go back to reference Veening, J.W., Smits, W.K., Kuipers, O.P.: Bistability, epigenetics, and bet-hedging in bacteria. Annu. Rev. Microbiol. 62, 193–210 (2008) CrossRef Veening, J.W., Smits, W.K., Kuipers, O.P.: Bistability, epigenetics, and bet-hedging in bacteria. Annu. Rev. Microbiol. 62, 193–210 (2008) CrossRef
28.
go back to reference Balázsi, G., van Oudenaarden, A., Collins, J.J.: Cellular decision making and biological noise: from microbes to mammals. Cell 144(6), 910–925 (2011) CrossRef Balázsi, G., van Oudenaarden, A., Collins, J.J.: Cellular decision making and biological noise: from microbes to mammals. Cell 144(6), 910–925 (2011) CrossRef
29.
30.
go back to reference Eldar, A., Elowitz, M.B.: Functional roles for noise in genetic circuits. Nature 467(7312), 167–173 (2010) CrossRef Eldar, A., Elowitz, M.B.: Functional roles for noise in genetic circuits. Nature 467(7312), 167–173 (2010) CrossRef
31.
go back to reference Mansinghka, V.K.: Natively probabilistic computation. Ph.D. thesis, Massachusetts Institute of Technology (2009) Mansinghka, V.K.: Natively probabilistic computation. Ph.D. thesis, Massachusetts Institute of Technology (2009)
32.
go back to reference Wang, S., Zhang, X., Li, Y., Bashizade, R., Yang, S., Dwyer, C., Lebeck, A.R.: Accelerating Markov random field inference using molecular optical Gibbs sampling units. In: Proceedings of the 43rd International Symposium on Computer Architecture, pp. 558–569. IEEE Press (2016) Wang, S., Zhang, X., Li, Y., Bashizade, R., Yang, S., Dwyer, C., Lebeck, A.R.: Accelerating Markov random field inference using molecular optical Gibbs sampling units. In: Proceedings of the 43rd International Symposium on Computer Architecture, pp. 558–569. IEEE Press (2016)
33.
go back to reference Fiser, J., Berkes, P., Orbán, G., Lengyel, M.: Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14(3), 119–130 (2010) CrossRef Fiser, J., Berkes, P., Orbán, G., Lengyel, M.: Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14(3), 119–130 (2010) CrossRef
34.
go back to reference Pouget, A., Beck, J.M., Ma, W.J., Latham, P.E.: Probabilistic brains: knowns and unknowns. Nat. Neurosci. 16(9), 1170–1178 (2013) CrossRef Pouget, A., Beck, J.M., Ma, W.J., Latham, P.E.: Probabilistic brains: knowns and unknowns. Nat. Neurosci. 16(9), 1170–1178 (2013) CrossRef
35.
go back to reference Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for Boltzmann machines. Cogn. Sci. 9(1), 147–169 (1985) CrossRef Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for Boltzmann machines. Cogn. Sci. 9(1), 147–169 (1985) CrossRef
36.
go back to reference Tanaka, T.: Mean-field theory of Boltzmann machine learning. Phys. Rev. E 58(2), 2302 (1998) CrossRef Tanaka, T.: Mean-field theory of Boltzmann machine learning. Phys. Rev. E 58(2), 2302 (1998) CrossRef
37.
go back to reference Tang, Y., Sutskever, I.: Data normalization in the learning of restricted Boltzmann machines. Department of Computer Science, University of Toronto, Technical report UTML-TR-11-2 (2011) Tang, Y., Sutskever, I.: Data normalization in the learning of restricted Boltzmann machines. Department of Computer Science, University of Toronto, Technical report UTML-TR-11-2 (2011)
38.
go back to reference Taylor, G.W., Hinton, G.E.: Factored conditional restricted Boltzmann machines for modeling motion style. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML), pp. 1025–1032. ACM (2009) Taylor, G.W., Hinton, G.E.: Factored conditional restricted Boltzmann machines for modeling motion style. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML), pp. 1025–1032. ACM (2009)
39.
go back to reference Casella, G., George, E.I.: Explaining the Gibbs sampler. Am. Stat. 46(3), 167–174 (1992) MathSciNet Casella, G., George, E.I.: Explaining the Gibbs sampler. Am. Stat. 46(3), 167–174 (1992) MathSciNet
40.
go back to reference Gillespie, D.T.: Stochastic simulation of chemical kinetics. Annu. Rev. Phys. Chem. 58, 35–55 (2007) CrossRef Gillespie, D.T.: Stochastic simulation of chemical kinetics. Annu. Rev. Phys. Chem. 58, 35–55 (2007) CrossRef
41.
go back to reference Qian, H.: Phosphorylation energy hypothesis: open chemical systems and their biological functions. Annu. Rev. Phys. Chem. 58, 113–142 (2007) CrossRef Qian, H.: Phosphorylation energy hypothesis: open chemical systems and their biological functions. Annu. Rev. Phys. Chem. 58, 113–142 (2007) CrossRef
42.
go back to reference Beard, D.A., Qian, H.: Chemical Biophysics: Quantitative Analysis of Cellular Systems. Cambridge University Press, Cambridge (2008) CrossRefMATH Beard, D.A., Qian, H.: Chemical Biophysics: Quantitative Analysis of Cellular Systems. Cambridge University Press, Cambridge (2008) CrossRefMATH
43.
go back to reference Ouldridge, T.E.: The importance of thermodynamics for molecular systems, the importance of molecular systems for thermodynamics. arXiv preprint arXiv:​1702.​00360 (2017) Ouldridge, T.E.: The importance of thermodynamics for molecular systems, the importance of molecular systems for thermodynamics. arXiv preprint arXiv:​1702.​00360 (2017)
44.
go back to reference Joshi, B.: A detailed balanced reaction network is sufficient but not necessary for its Markov chain to be detailed balanced. arXiv preprint arXiv:​1312.​4196 (2013) Joshi, B.: A detailed balanced reaction network is sufficient but not necessary for its Markov chain to be detailed balanced. arXiv preprint arXiv:​1312.​4196 (2013)
45.
46.
go back to reference Anderson, D.F., Craciun, G., Kurtz, T.G.: Product-form stationary distributions for deficiency zero chemical reaction networks. Bull. Math. Biol. 72(8), 1947–1970 (2010) MathSciNetCrossRefMATH Anderson, D.F., Craciun, G., Kurtz, T.G.: Product-form stationary distributions for deficiency zero chemical reaction networks. Bull. Math. Biol. 72(8), 1947–1970 (2010) MathSciNetCrossRefMATH
47.
go back to reference LeCun, Y., Cortes, C., Burges, C.J.C.: The MNIST database of handwritten digits (1998) LeCun, Y., Cortes, C., Burges, C.J.C.: The MNIST database of handwritten digits (1998)
Metadata
Title
Chemical Boltzmann Machines
Authors
William Poole
Andrés Ortiz-Muñoz
Abhishek Behera
Nick S. Jones
Thomas E. Ouldridge
Erik Winfree
Manoj Gopalkrishnan
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-66799-7_14