research-article

Public Access

AOT vs. JIT: impact of profile data on code quality

Authors:

Prasad A. Kulkarni,

Michael R. JantzAuthors Info & Claims

LCTES 2017: Proceedings of the 18th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems

Pages 1 - 10

https://doi.org/10.1145/3078633.3081037

Published: 21 June 2017 Publication History

Abstract

Just-in-time (JIT) compilation during program execution and ahead-of-time (AOT) compilation during software installation are alternate techniques used by managed language virtual machines (VM) to generate optimized native code while simultaneously achieving binary code portability and high execution performance. Profile data collected by JIT compilers at run-time can enable profile-guided optimizations (PGO) to customize the generated native code to different program inputs. AOT compilation removes the speed and energy overhead of online profile collection and dynamic compilation, but may not be able to achieve the quality and performance of customized native code. The goal of this work is to investigate and quantify the implications of the AOT compilation model on the quality of the generated native code for current VMs.

First, we quantify the quality of native code generated by the two compilation models for a state-of-the-art (HotSpot) Java VM. Second, we determine how the amount of profile data collected affects the quality of generated code. Third, we develop a mechanism to determine the accuracy or similarity for different profile data for a given program run, and investigate how the accuracy of profile data affects its ability to effectively guide PGOs. Finally, we categorize the profile data types in our VM and explore the contribution of each such category to performance.

References

[1]

Dacapo batik benchmark fails. https://github.com/RedlineResearch/OLDOpenJDK8/issues/1.

[2]

Dacapo eclipse benchmark fails. https://github.com/RedlineResearch/OLD-OpenJDK8/issues/2.

[3]

M. Arnold and D. Grove. Collecting and exploiting high-accuracy call graph profiles in virtual machines. In Proceedings of the Symposium on Code Generation and Optimization, CGO ’05, pages 51–62, 2005.

Digital Library

[4]

M. Arnold and D. Grove. Collecting and exploiting high-accuracy call graph profiles in virtual machines. In Proceedings of the Symposium on Code Generation and Optimization, pages 51–62, 2005.

Digital Library

[5]

M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. A survey of adaptive optimization in virtual machines. Proceedings of the IEEE, 92(2):449–466, February 2005.

[6]

M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. Adaptive optimization in the Jalapeno JVM. SIGPLAN Notices, 46(4):65–83, May 2011. ISSN 0362-1340.

Digital Library

[7]

S. Blackburn, D. Frampton, R. Garner, and J. Zigman. dacapo-9.12bach. http://dacapobench.org/RELEASE NOTES.txt, 12 2009.

[8]

S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, B. Moss, A. Phansalkar, D. Stefanovi´c, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, OOPSLA ’06, pages 169–190. ACM, 2006.

Digital Library

[9]

W. J. Bowman, S. Miller, V. St-Amour, and R. K. Dybvig. Profileguided meta-programming. In Proceedings of the Conference on Programming Language Design and Implementation, pages 403–412, 2015.

Digital Library

[10]

P. P. Chang, S. A. Mahlke, and W. mei W. Hwu. Using profile information to assist classic code optimizations. Software Practice and Experience, 21:1301–1321, 1991.

Digital Library

[11]

M. Cierniak, G.-Y. Lueh, and J. M. Stichnoth. Practicing judo: Java under dynamic optimizations. In Proceedings of the Conference on Programming Language Design and Implementation, pages 13–26, 2000.

Digital Library

[12]

E. Duesterwald and V. Bala. Software profiling for hot path prediction: Less is more. SIGPLAN Notices, 35(11):202–211, Nov. 2000.

Digital Library

[13]

A. Georges, D. Buytaert, and L. Eeckhout. Statistically rigorous Java performance evaluation. In Proceedings of the conference on Objectoriented programming systems and applications, pages 57–76, 2007.

Digital Library

[14]

S. L. Graham, P. B. Kessler, and M. K. Mckusick. Gprof: A call graph execution profiler. SIGPLAN Notices, 17(6):120–126, 1982.

Digital Library

[15]

U. Hölzle and D. Ungar. Reconciling responsiveness with performance in pure object-oriented languages. ACM Trans. Program. Lang. Syst., 18(4):355–400, 1996. ISSN 0164-0925.

Digital Library

[16]

A. Homescu, S. Neisius, P. Larsen, S. Brunthaler, and M. Franz. Profile-guided automated software diversity. In Proceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO), CGO ’13, pages 1–11, 2013.

Digital Library

[17]

S. Hong, J.-C. Kim, J. W. Shin, S.-M. Moon, H.-S. Oh, J. Lee, and H.-K. Choi. Java client ahead-of-time compiler for embedded systems. In Proceedings of the Conference on Languages, Compilers, and Tools for Embedded Systems, LCTES ’07, pages 63–72, 2007.

Digital Library

[18]

X. Huang, S. M. Blackburn, K. S. McKinley, J. E. B. Moss, Z. Wang, and P. Cheng. The garbage collection advantage: Improving program locality. In Proceedings of the Conference on Object-oriented Programming, Systems, Languages, and Applications, OOPSLA ’04, pages 69–80, 2004.

Digital Library

[19]

W.-M. W. Hwu, S. A. Mahlke, W. Y. Chen, P. P. Chang, N. J. Warter, R. A. Bringmann, R. G. Ouellette, R. E. Hank, T. Kiyohara, G. E. Haab, J. G. Holm, and D. M. Lavery. The superblock: An effective technique for vliw and superscalar compilation. J. Supercomput., 7 (1-2):229–248, 1993. ISSN 0920-8542.

Digital Library

[20]

M. R. Jantz, F. J. Robinson, P. A. Kulkarni, and K. A. Doshi. Crosslayer memory management for managed language applications. In Proceedings of the Conference on Object-Oriented Programming, Systems, Languages, and Applications, pages 488–504, 2015.

Digital Library

[21]

D.-H. Jung, S.-M. Moon, and H.-S. Oh. Hybrid compilation and optimization for java-based digital tv platforms. ACM Trans. Embed. Comput. Syst., 13(2s):62:1–62:27, Jan. 2014.

Digital Library

[22]

C. Krintz, D. Grove, V. Sarkar, and B. Calder. Reducing the overhead of dynamic compilation. Software: Practice and Experience, 31(8): 717–738, December 2000.

[23]

P. A. Kulkarni. JIT Compilation policy for modern machines. In Proceedings of the Conference on Object Oriented Programming Systems Languages and Applications, pages 773–788, 2011.

Digital Library

[24]

M. Mock, C. Chambers, and S. J. Eggers. Calpa: A tool for automating selective dynamic compilation. In Proceedings of the Symposium on Microarchitecture, pages 291–302, 2000.

Digital Library

[25]

T. Moseley, A. Shye, V. J. Reddi, D. Grunwald, and R. Peri. Shadow profiling: Hiding instrumentation costs with parallelism. In Proceedings of the Symposium on Code Generation and Optimization, CGO ’07, pages 198–208, 2007.

Digital Library

[26]

T. Mytkowicz, A. Diwan, M. Hauswirth, and P. F. Sweeney. Evaluating the accuracy of java profilers. In Proceedings of the Conference on Programming Language Design and Implementation, PLDI ’10, pages 187–197, 2010.

Digital Library

[27]

H.-S. Oh, J. H. Yeo, and S.-M. Moon. Bytecode-to-c ahead-of-time compilation for android dalvik virtual machine. In Proceedings of the Design, Automation & Test in Europe Conference & Exhibition, pages 1048–1053, 2015.

Digital Library

[28]

M. Paleczny, C. Vick, and C. Click. The Java HotSpot™server compiler. In Proceedings of the Symposium on Java Virtual Machine Research and Technology Symposium, pages 1–12, 2001.

Digital Library

[29]

K. Pettis and R. C. Hansen. Profile guided code positioning. In Proceedings of the ACM SIGPLAN 1990 Conference on Programming Language Design and Implementation, PLDI ’90, pages 16–27, 1990.

Digital Library

[30]

F. J. Robinson, M. R. Jantz, and P. A. Kulkarni. Code cache management in managed language vms to reduce memory consumption for embedded systems. In Proceedings of the Conference on Languages, Compilers, Tools, and Theory for Embedded Systems, pages 11–20, 2016.

Digital Library

[31]

S. Rubin, R. Bod´ık, and T. Chilimbi. An efficient profile-analysis framework for data-layout optimizations. In Proceedings of the 29th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’02, pages 140–153. ACM, 2002.

Digital Library

[32]

M. Serrano, R. Bordawekar, S. Midkiff, and M. Gupta. Quicksilver: A quasi-static compiler for java. In Proceedings of the Conference on Object-oriented Programming, Systems, Languages, and Applications, pages 66–82, 2000.

Digital Library

[33]

SPEC2008. Specjvm2008 benchmarks. http://www.spec.org/jvm2008/, 2008.

[34]

T. Suganuma, T. Yasue, M. Kawahito, H. Komatsu, and T. Nakatani. Design and evaluation of dynamic optimizations for a java just-intime compiler. ACM Transactions on Programming Languages and Systems, 27(4):732–785, July 2005.

Digital Library

[35]

C.-S. Wang, G. Perez, Y.-C. Chung, W.-C. Hsu, W.-K. Shih, and H.-R. Hsu. A method-based ahead-of-time compiler for android applications. In Proceedings of the Conference on Compilers, Architectures and Synthesis for Embedded Systems, pages 15–24, 2011.

Digital Library

[36]

Y. Wu and J. R. Larus. Static branch frequency and program profile analysis. In Proceedings of the Symposium on Microarchitecture, pages 1–11, 1994.

Digital Library

Cited By

Burchell HLarose OMarr SErtl MKirsch C(2024)Towards Realistic Results for Instrumentation-Based Profilers for JIT-Compiled SystemsProceedings of the 21st ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3679007.3685058(82-89)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3679007.3685058
Mueller RCarpen-Amarie MAslandukov MTovletoglou KErtl MKirsch C(2024)The Cost of Profiling in the HotSpot Virtual MachineProceedings of the 21st ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3679007.3685055(112-126)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3679007.3685055
Jia HWen MXie ZGuo XWu RSun MChen KJin HGrundy JPollock LPenta M(2023)Detecting JVM JIT Compiler Bugs via Exploring Two-Dimensional Input SpacesProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00016(43-55)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00016
Show More Cited By

Index Terms

AOT vs. JIT: impact of profile data on code quality
1. Software and its engineering
  1. Software notations and tools
    1. Compilers
      1. Just-in-time compilers
      2. Runtime environments
  2. Software organization and properties
    1. Software system structures
      1. Embedded software

Recommendations

Exploring Impact of Profile Data on Code Quality in the HotSpot JVM
Special Issue on LCETES, Part 2, Learning, Distributed, and Optimizing Compilers

Managed language virtual machines (VM) rely on dynamic or just-in-time (JIT) compilation to generate optimized native code at run-time to deliver high execution performance. Many VMs and JIT compilers collect profile data at run-time to enable profile-...
AOT vs. JIT: impact of profile data on code quality
LCTES '17

Just-in-time (JIT) compilation during program execution and ahead-of-time (AOT) compilation during software installation are alternate techniques used by managed language virtual machines (VM) to generate optimized native code while simultaneously ...
Profile inference revisited

Profile-guided optimization (PGO) is an important component in modern compilers. By allowing the compiler to leverage the program’s dynamic behavior, it can often generate substantially faster binaries. Sampling-based profiling is the state-of-the-art ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

LCTES 2017: Proceedings of the 18th ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems

June 2017

120 pages

ISBN:9781450350303

DOI:10.1145/3078633

General Chair:
Vijay Nagarajan
University of Edinburgh, UK
,
Program Chair:
Zili Shao
Hong Kong Polytechnic University, China

ACM SIGPLAN Notices Volume 52, Issue 5
LCTES '17
May 2017
120 pages
ISSN:0362-1340
EISSN:1558-1160
DOI:10.1145/3140582
Editor:
Matthew Fluet
Issue’s Table of Contents

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

LCTES '17

Sponsor:

LCTES '17: SIGPLAN/SIGBED Conference on Languages, Compilers and Tools for Embedded Systems 2017

June 21 - 22, 2017

Barcelona, Spain

Acceptance Rates

Overall Acceptance Rate 116 of 438 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
960
Total Downloads

Downloads (Last 12 months)243
Downloads (Last 6 weeks)35

Reflects downloads up to 19 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Burchell HLarose OMarr SErtl MKirsch C(2024)Towards Realistic Results for Instrumentation-Based Profilers for JIT-Compiled SystemsProceedings of the 21st ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3679007.3685058(82-89)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3679007.3685058
Mueller RCarpen-Amarie MAslandukov MTovletoglou KErtl MKirsch C(2024)The Cost of Profiling in the HotSpot Virtual MachineProceedings of the 21st ACM SIGPLAN International Conference on Managed Programming Languages and Runtimes10.1145/3679007.3685055(112-126)Online publication date: 13-Sep-2024
https://dl.acm.org/doi/10.1145/3679007.3685055
Jia HWen MXie ZGuo XWu RSun MChen KJin HGrundy JPollock LPenta M(2023)Detecting JVM JIT Compiler Bugs via Exploring Two-Dimensional Input SpacesProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00016(43-55)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00016
Yu XTang WXiong TChen WHe JYang BQi Z(2023)Enhancing embedded systems development with TSAutomated Software Engineering10.1007/s10515-023-00404-x31:1Online publication date: 6-Dec-2023
https://dl.acm.org/doi/10.1007/s10515-023-00404-x
Anthimopulos TKeramidas GKelefouras VStamoulis I(2023)A Comparative Study of Neural Network Compilers on ARMv8 ArchitectureArchitecture of Computing Systems10.1007/978-3-031-42785-5_2(18-33)Online publication date: 26-Aug-2023
https://doi.org/10.1007/978-3-031-42785-5_2
Moreira AOttoni GQuintão Pereira F(2021)VESPA: static profiling for binary optimizationProceedings of the ACM on Programming Languages10.1145/34855215:OOPSLA(1-28)Online publication date: 15-Oct-2021
https://dl.acm.org/doi/10.1145/3485521
Serrano M(2021)Of JavaScript AOT compilation performanceProceedings of the ACM on Programming Languages10.1145/34735755:ICFP(1-30)Online publication date: 19-Aug-2021
https://dl.acm.org/doi/10.1145/3473575
Effler TKammerdiener BJantz MSengupta SKulkarni PDoshi KJones T(2019)Evaluating the effectiveness of program data features for guiding memory managementProceedings of the International Symposium on Memory Systems10.1145/3357526.3357537(383-395)Online publication date: 30-Sep-2019
https://dl.acm.org/doi/10.1145/3357526.3357537
Leopoldseder DStadler LRigger MWürthinger TMössenböck HKell SMarr S(2018)A cost model for a graph-based intermediate-representation in a dynamic compilerProceedings of the 10th ACM SIGPLAN International Workshop on Virtual Machines and Intermediate Languages10.1145/3281287.3281290(26-35)Online publication date: 4-Nov-2018
https://dl.acm.org/doi/10.1145/3281287.3281290
Leopoldseder DSchatz RStadler LRigger MWürthinger TMössenböck HTilevich EMössenböck H(2018)Fast-path loop unrolling of non-counted loops to enable subsequent compiler optimizationsProceedings of the 15th International Conference on Managed Languages & Runtimes10.1145/3237009.3237013(1-13)Online publication date: 12-Sep-2018
https://dl.acm.org/doi/10.1145/3237009.3237013
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents