Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8176889

AOT: aot compilation hangs while compiling jdk.localedata module

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P2
    • Resolution: Won't Fix
    • Affects Version/s: 9, 10
    • Fix Version/s: 10
    • Component/s: hotspot
    • Labels:

      Description

      try to compile jdk.localedata module via jaotc. Compilation can't finish(at least in reasonable time)

        Activity

        Hide
        dlong Dean Long added a comment -
        Location objects are used in jvmciCodeInstaller.cpp to create oopmaps for CallSites and Safepoints, so they cannot be cleaned up early in the current design.

        Because we retain all compilation results until the end, the total heap required is going to be proportional to the module size. We can use tricks to increase memory efficiency by some percent, but the heap requirement is still going to be O(bytecode_size). If the module size grows, we need a new trick to keep the heap size constant. So Jamsheed's suggestion is to do incremental compilation. For example, create separate .o files for each class and then combine at the end into a single .so. Or (my idea) maybe we can write compilation result objects to disk using Serialization.
        Show
        dlong Dean Long added a comment - Location objects are used in jvmciCodeInstaller.cpp to create oopmaps for CallSites and Safepoints, so they cannot be cleaned up early in the current design. Because we retain all compilation results until the end, the total heap required is going to be proportional to the module size. We can use tricks to increase memory efficiency by some percent, but the heap requirement is still going to be O(bytecode_size). If the module size grows, we need a new trick to keep the heap size constant. So Jamsheed's suggestion is to do incremental compilation. For example, create separate .o files for each class and then combine at the end into a single .so. Or (my idea) maybe we can write compilation result objects to disk using Serialization.
        Hide
        kvn Vladimir Kozlov added a comment -
        We do free this data after AOT metadata is generated by jvmciCodeInstaller.
        An other approach is to convert Graal's compiled data (code, metadata, etc) into AOT code and data immediately after compilation and do not wait when all compilations are finished.
        Separate object files will not work since we need info about offsets in one continues section. I like your idea about dumping compilation results and reading it back when we generate fine output.
        I agree that such work can be done late as separate RFE.

        What confused me is that there are proposed changes which can help: "reusing Location saves memory for aot compilation, and give compilation time improvement (30%) ". Why we are not doing it now? 30% is huge.
        Show
        kvn Vladimir Kozlov added a comment - We do free this data after AOT metadata is generated by jvmciCodeInstaller. An other approach is to convert Graal's compiled data (code, metadata, etc) into AOT code and data immediately after compilation and do not wait when all compilations are finished. Separate object files will not work since we need info about offsets in one continues section. I like your idea about dumping compilation results and reading it back when we generate fine output. I agree that such work can be done late as separate RFE. What confused me is that there are proposed changes which can help: "reusing Location saves memory for aot compilation, and give compilation time improvement (30%) ". Why we are not doing it now? 30% is huge.
        Hide
        dlong Dean Long added a comment -
        >> Why we are not doing it now? 30% is huge.

        It was only for some methods in jdk.localedata. We may be better off excluding those methods and just interpreting them, or increasing the heap size by 30% for this module.

        Finally, I think Jamsheed was not happy with my idea to cache Location objects as a solution. We went through several iterations, starting with synchronized HashMaps and finally ending up with unsynchronized ThreadLocal arrays, but the design is still not very clean because, for example, frameSize and wordSize needs to be made available to Location somehow. [~kvn] However, if you think we should not abandon this improvement, perhaps Jamsheed would be happy to let someone else take it on.
        Show
        dlong Dean Long added a comment - >> Why we are not doing it now? 30% is huge. It was only for some methods in jdk.localedata. We may be better off excluding those methods and just interpreting them, or increasing the heap size by 30% for this module. Finally, I think Jamsheed was not happy with my idea to cache Location objects as a solution. We went through several iterations, starting with synchronized HashMaps and finally ending up with unsynchronized ThreadLocal arrays, but the design is still not very clean because, for example, frameSize and wordSize needs to be made available to Location somehow. [~kvn] However, if you think we should not abandon this improvement, perhaps Jamsheed would be happy to let someone else take it on.
        Hide
        kvn Vladimir Kozlov added a comment -
        Okay. Thank you for explanation. So it was not general improvement.
        I agree with closing this now.
        Thanks!
        Show
        kvn Vladimir Kozlov added a comment - Okay. Thank you for explanation. So it was not general improvement. I agree with closing this now. Thanks!
        Hide
        jcm Jamsheed C M added a comment - - edited
        to add to what Dean said, Location object is value type candidate by graal design, so saving memory by sharing wouldnt best fit to current design..
        Show
        jcm Jamsheed C M added a comment - - edited to add to what Dean said, Location object is value type candidate by graal design, so saving memory by sharing wouldnt best fit to current design..

          People

          • Assignee:
            jcm Jamsheed C M
            Reporter:
            dpochepk Dmitrij Pochepko
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved: