Martin Schoeberl
ABSTRACT
Requirements to embedded systems increase steadily. In parallel, also the performance of the processors used in these systems is improved leading to multithreaded and/or multicore architectures. Depending on the type of the embedded system, using Java is a more and more popular way for software development. In this paper, we present a Java benchmark suite that enables the comparison of different embedded Java platforms while solely assuming the availability of a CLDC API, the minimal configuration defined for the J2ME. The core of the benchmark suite consists of adapted real-world applications. Furthermore, the suite contains benchmarks to explore multi-core/multi-threaded systems. Hence, it is possible to determine the gain of a parallel execution platform compared to sequential execution. Additionally, the penalty of a sequential program running on a parallel platform can be measured. Our benchmarks are structured in micro, kernel, application, parallel, and streaming benchmarks.
- aJile. aj-100 real-time low power Java processor. preliminary data sheet, 2000.Google Scholar
- Aonix. Perc pico 1.1 user manual. http://research.aonix.com/jsc/pico-manual.4-19-08.pdf, April 2008.Google Scholar
- Austin Armbruster, Jason Baker, Antonio Cunei, Chapman Flack, David Holmes, Filip Pizlo, Edward Pla, Marek Prochazka, and Jan Vitek. A real-time Java virtual machine with applications in avionics. Trans. on Embedded Computing Sys., 7(1):1--49, 2007. Google ScholarDigital Library
- S. M. Blackburn, R. Garner, C. Hoffman, A. M. Khan, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, A. Phansalkar, D. Stefanović, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-Oriented Programing, Systems, Languages, and Applications, pages 169--190, New York, NY, USA, October 2006. ACM Press. Google ScholarDigital Library
- Stephen M. Blackburn, Kathryn S. McKinley, Robin Garner, Chris Hoffmann, Asjad M. Khan, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony Hosking, Maria Jump, Han Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanovik, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. Wake up and smell the coffee: evaluation methodology for the 21st century. Commun. ACM, 51(8):83--89, 2008. Google ScholarDigital Library
- Greg Bollella, James Gosling, Benjamin Brosgol, Peter Dibble, Steve Furr, and Mark Turnbull. The Real-Time Specification for Java. Java Series. Addison-Wesley, June 2000. Google ScholarDigital Library
- G. Chen, M. Kandemir, N. Vijaykrishnan, and M. J. Irwin. Pennbench: A benchmark suite for embedded java. In in the IEEE 5th Annual Workshop on Workload Characterization, 2005.Google Scholar
- Brian P. Doherty. A real-time benchmark for java#8482;. In JTRES '07: Proceedings of the 5th international workshop on Java technologies for real-time and embedded systems, pages 35--46, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- Sudheendra Hangal and Mike O'Connor. Performance analysis and validation of the picojava processor. IEEE Micro, 19:66--72, 1999. Google ScholarDigital Library
- Trevor Harmon, Martin Schoeberl, Raimund Kirner, and Raymond Klefstad. A modular worst-case execution time analysis tool for Java processors. In Proceedings of the 14th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS 2008), pages 47--57, St. Louis, MO, United States, April 2008. IEEE Computer Society. Google ScholarDigital Library
- John Hennessy and David Patterson. Computer Architecture: A Quantitative Approach, 3rd ed. Morgan Kaufmann Publishers Inc., Palo Alto, CA 94303, 2002. Google ScholarDigital Library
- Thomas Henties, James J. Hunt, Doug Locke, Kelvin Nilsen, Martin Schoeberl, and Jan Vitek. Java for safety-critical applications. In 2nd International Workshop on the Certification of Safety-Critical Software Controlled Systems (SafeCert 2009), York, United Kingdom, Mar. 2009.Google Scholar
- Tomas Kalibera, Jeff Hagelberg, Filip Pizlo, Ales Plsek, Ben Titzer, and Jan Vitek. Cdx: a family of real-time java benchmarks. In JTRES '09: Proceedings of the 7th International Workshop on Java Technologies for Real-Time and Embedded Systems, pages 41--50, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- Andreas Krall and Reinhard Grafl. CACAO -- A 64 bit JavaVM just-in-time compiler. In Geoffrey C. Fox and Wei Li, editors, PPoPP'97 Workshop on Java for Science and Engineering Computation, Las Vegas, June 1997. ACM.Google Scholar
- Mälardalen Real-Time Research Center. WCET benchmarks. Available at http://www.mrtc.mdh.se/projects/wcet/benchmarks.html, accessed 2009.Google Scholar
- Caffeinemark 3.0 benchmark. Available at http://www.benchmarkhq.ru/cm30/info.html, 1997.Google Scholar
- Christof Pitter and Martin Schoeberl. A real-time Java chip-multiprocessor. Trans. on Embedded Computing Sys., accepted for publication, 2010. Google ScholarDigital Library
- Filip Pizlo, Lukasz Ziarek, and Jan Vitek. Real time java on resource-constrained platforms with fiji vm. In JTRES '09: Proceedings of the 7th International Workshop on Java Technologies for Real-Time and Embedded Systems, pages 110--119, New York, NY, USA, 2009. ACM. Google ScholarDigital Library
- Martin Schoeberl. Evaluation of a Java processor. In Tagungsband Austrochip 2005, pages 127--134, Vienna, Austria, October 2005.Google Scholar
- Martin Schoeberl. Application experiences with a real-time Java processor. In Proceedings of the 17th IFAC World Congress, pages 9320--9325, Seoul, Korea, July 2008.Google ScholarCross Ref
- Martin Schoeberl, Wolfgang Puffitsch, Rasmus Ulslev Pedersen, and Benedikt Huber. Worst-case execution time analysis for a Java processor. Software: Practice and Experience, 40/6:507--542, 2010. Google ScholarDigital Library
- Fridtjof Siebert and Andy Walter. Deterministic execution of Java's primitive bytecode operations. In Proceedings of the Java Virtual Machine Research and Technology Symposium (JVM '01): April 23--24, 2001, Monterey, California, USA. Berkeley, CA, pages 141--152. USENIX, 2001. Google ScholarDigital Library
- Doug Simon, Cristina Cifuentes, Dave Cleal, John Daniels, and Derek White. Java on the bare metal of wireless sensor devices: the squawk Java virtual machine. In Proceedings of the 2nd international conference on Virtual execution environments (VEE 2006), pages 78--88, New York, NY, USA, 2006. ACM Press. Google ScholarDigital Library
- Jose Solorzano. leJOS: Java based os for lego RCX. Available at: http://lejos.sourceforge.net/.Google Scholar
- Sun Microsystems. J2ME Building Blocks for Mobile Devices, White Paper on KVM and the Connected, Limited Device Configuration (CLDC), May 2000.Google Scholar
- Sun Microsystems. Connected Limited Device Configuration Specification, Version 1.1, March 2003.Google Scholar
- Sascha Uhrig. Evaluation of different multithreaded and multicore processor configurations for sopc. In SAMOS '09: Proceedings of the 9th International Workshop on Embedded Computer Systems: Architectures, Modeling, and Simulation, pages 68--77, Berlin, Heidelberg, 2009. Springer-Verlag. Google ScholarDigital Library
- Sascha Uhrig and Jörg Wiese. jamuth -- an IP Processor Core for Embedded Java Real-Time Systems. In The 5th International Workshop on Java Technologies for Real-time and Embedded Systems - JTRES 2007, Vienna, Austria, September 2007. Google ScholarDigital Library
- Martin Zabel, Thomas B. Preusser, Peter Reichel, and Rainer G. Spallek. Secure, real-time and multi-threaded general-purpose embedded Java microarchitecture. In Prceedings of the 10th Euromicro Conference on Digital System Design Architectures, Methods and Tools (DSD 2007), pages 59--62, Lübeck, Germany, Aug. 2007. Google ScholarDigital Library
- Martin Zabel and Rainer G. Spallek. SHAP --- scalable multi-core Java bytecode processor. Technical report, Fakultät Informatik, Technische Universität Dresden, 2009. ftp://ftp.inf.tu-dresden.de/pub/berichte/tud09-13.pdf.Google Scholar
- The embedded Java benchmark suite JemBench
Recommendations
A Comprehensive Java Benchmark Study on Memory and Garbage Collection Behavior of DaCapo, DaCapo Scala, and SPECjvm2008
ICPE '17: Proceedings of the 8th ACM/SPEC on International Conference on Performance EngineeringBenchmark suites are an indispensable part of scientific research to compare different approaches against each another. The diversity of benchmarks is an important asset to evaluate novel approaches for effectiveness and weaknesses. In this paper, we ...
A parallel java grande benchmark suite
SC '01: Proceedings of the 2001 ACM/IEEE conference on SupercomputingIncreasing interest is being shown in the use of Java for large scale or Grande applications. This new use of Java places specific demands on the Java execution environments that can be tested using the Java Grande benchmark suite [5], [6], [7]. The ...
Subsetting the SPEC CPU2006 benchmark suite
On August 24, 2006, the Standard Performance Evaluation Corporation (SPEC) announced CPU2006 -- the next generation of industry-standardized CPU-intensive benchmark suite. The SPEC CPU benchmark suite has become the most frequently used suite for ...
Comments