Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8168503

JEP 297: Unified arm32/arm64 Port

    Details

    • Author:
      Edward Nevill
    • JEP Type:
      Feature
    • Exposure:
      Open
    • Subcomponent:
    • Scope:
      Implementation
    • Discussion:
      aarch32 dash port dash dev at openjdk dot java dot net
    • Effort:
      M
    • Duration:
      S
    • Alert Status:
       Green
    • JEP Number:
      297

      Description

      Summary

      Integrate the unified port of HotSpot for arm32 and arm64, contributed by Oracle, into the JDK.

      Motivation

      The main motivation is to provide arm32/aarch32 support in the JDK. Although there have been some efforts to support arm32/aarch32, the only maintained option in the JDK today is the Zero port. The contribution from Oracle provides full C1 and C2 support for ARM, putting it on par with other architectures.

      The port also provides support for arm64/aarch64, however this is less of a motivation since the JDK already includes an aarch64-only port.

      Description

      The contribution from Oracle provides C1 and C2 support for both arm32 and arm64. The code has been merged into a JDK 9 tree in a separate repository in the aarch32 project area.

      Oracle's intention to open source the ARM port was announced on the aarch32 mailing list on Aug 23, 2016 and there have been several discussion threads on the aarch32 mailing list. Please follow the links below to see these discussions.

      On an open conference call on Fri, Oct 21, 2016, it was agreed that we should push for this to be included in the JDK. No formal minutes were kept but notes from the meeting were published to the aarch32 list.

      The webrev for the Oracle contribution is available at http://cr.openjdk.java.net/~bobv/arm3264/webrev and a merged jdk9 tree at http://hg.openjdk.java.net/aarch32-port/jdk9-arm3264.

      The merged tree is capable of building a minimal, client, or server VM for arm32 or arm64. The tree can also build the existing aarch64 port.

      There is an additional ARM specific option, --with-cpu-port, which can be used to specify the new aarch64 build --with-cpu-port=arm64 or the existing aarch64 build --with-cpu-port=aarch64. If no option is specified the build defaults to the existing aarch64 build.

      It is intended that the default build for aarch64 will remain the existing aarch64 build.

      Testing

      The port has been tested internally by Oracle since it is part of Oracle's proprietary VM. As such it has been tested with JPRT and other internal tests.

      The JDK 9 tree in the aarch32 project has been tested using jtreg in a variety of configurations including hard/soft FP, release/debug builds and client/server builds. A couple of issues were found with the merged sources which have been fixed.

      Risks

      The changes to shared code have been kept to a minimum. The contributed port shares #ifdefs with the aarch64 port. In addition the #ifdefs for ARM are already in the shared code. This means that the changes to shared code are limited to build changes and a single #ifdef in libproc.h.

      Since the sources originate from the closed source port within Oracle they are a known quantity within Oracle and this significantly reduces the associated risks of merging this into the JDK.

      There is a risk of confusion amongst people building JDK by the fact that there will be two aarch64 ports, the existing aarch64 port and the unified arm32/arm64 port.

        Issue Links

          Activity

          Hide
          bobv Bob Vandette added a comment - - edited
          FC Extension Request:
          Justification: We just recently received agreement with aarch32 project team to accept our ARM ports. Moving these files to the open will allow a larger community to contribute to the enhancement and support of these ARM ports.
          Risk: Low. Work is primarily just moving files from closed repositories to open which has been completed.
          Proposed Integration Date: November 18
          Proposed Due Date: December 8
          Show
          bobv Bob Vandette added a comment - - edited FC Extension Request: Justification: We just recently received agreement with aarch32 project team to accept our ARM ports. Moving these files to the open will allow a larger community to contribute to the enhancement and support of these ARM ports. Risk: Low. Work is primarily just moving files from closed repositories to open which has been completed. Proposed Integration Date: November 18 Proposed Due Date: December 8
          Hide
          enevill Ed Nevill added a comment - - edited
          Engineering plan:

          According to the JEP process "a credible engineering plan should be documented in subtasks of the JEP issue" before a JEP can be "proposed to target". However, in the case of this JEP all development work is already complete as it is based on a set of sources which have been tested and deployed in commercial product.

          The remaining tasks consist of

          - merging to the latest JDK 9
          - testing final merge
          - integration

          A subtask has been created which reflects the engineering work done to date (items 1-5) and those remaining (items 6-7).
          Show
          enevill Ed Nevill added a comment - - edited Engineering plan: According to the JEP process "a credible engineering plan should be documented in subtasks of the JEP issue" before a JEP can be "proposed to target". However, in the case of this JEP all development work is already complete as it is based on a set of sources which have been tested and deployed in commercial product. The remaining tasks consist of - merging to the latest JDK 9 - testing final merge - integration A subtask has been created which reflects the engineering work done to date (items 1-5) and those remaining (items 6-7).
          Hide
          enevill Ed Nevill added a comment -
          An open conference call was held on Thus, 1/12/2016 at which the status of the JEP were discussed.

          No formal minutes of the call were kept, however the following are notes from the call.

          Present:

          Bob Vandette
          Andrew Haley
          Alexey Kashchenko
          Derek White
          Joel Jones
          Stuart Monteith
          Edward Nevill

          The current status of the JEP is 'Candidate' and the timescales for moving to 'Proposed to Target', 'Targetted' & 'Complete' were discussed.

          Bob Vandette pointed out that there is a deadline of 22/12/16 which is expiry date of the fc exemption.

          The following timescale was proposed

          5/12/16 - Move to 'Proposed to Target' followed by one week open review
          12/12/16 - Move to 'Targetted' allowing one week for integration into JDK 9.
          19/12/16 - Move to 'Integrated'
          22/12/16 - Move to 'complete'

          The move to complete stage is defined in the JEP process as

          "By the JEP’s owner, after functional, performance, and conformance tests have been delivered and the feature has reached the point where only bug fixes are expected"

          Because this JEP adds no new functionality and the source base is already at a stage where 'only bug fixes are expected' the move to 'Complete' could follow almost immediately from 'Integrated'.

          There was broad consensus among the participants on this plan.

          There was a discussion regarding what testing needed to be performed in the community and Ed Nevill requested help from the participants in testing once Bob has merged up to the latest JDK 9 sources.

          The following tests were identified:-

          - Full JTreg run on arm32 & arm64 (Ed)
          - JCStress on arm32 (Stuart to investigate)
          - Retarget Linaro CI loop from hs repo to merged arm32/arm64 repo (Stuart to investigate)
            (Linaro CI Loop runs JTReg, JCStress, Hadoop/Terasort & SpecJBB2015)
          - Build RedHat using merged arm32/arm64 repo. Andrew will investigate whether the team ad RedHat can do this.
          - Build and test softfloat version. Problem with lack of Hardware. Various offers/suiggestions.

          It was agreed that once people complete any testing they should append details of the test and results to the JEP so that a record may be kept.

          Thank you to the participants of the call and apologies if I have omitted any points. Please feel free to add any points which you feel I have omitted.
          Show
          enevill Ed Nevill added a comment - An open conference call was held on Thus, 1/12/2016 at which the status of the JEP were discussed. No formal minutes of the call were kept, however the following are notes from the call. Present: Bob Vandette Andrew Haley Alexey Kashchenko Derek White Joel Jones Stuart Monteith Edward Nevill The current status of the JEP is 'Candidate' and the timescales for moving to 'Proposed to Target', 'Targetted' & 'Complete' were discussed. Bob Vandette pointed out that there is a deadline of 22/12/16 which is expiry date of the fc exemption. The following timescale was proposed 5/12/16 - Move to 'Proposed to Target' followed by one week open review 12/12/16 - Move to 'Targetted' allowing one week for integration into JDK 9. 19/12/16 - Move to 'Integrated' 22/12/16 - Move to 'complete' The move to complete stage is defined in the JEP process as "By the JEP’s owner, after functional, performance, and conformance tests have been delivered and the feature has reached the point where only bug fixes are expected" Because this JEP adds no new functionality and the source base is already at a stage where 'only bug fixes are expected' the move to 'Complete' could follow almost immediately from 'Integrated'. There was broad consensus among the participants on this plan. There was a discussion regarding what testing needed to be performed in the community and Ed Nevill requested help from the participants in testing once Bob has merged up to the latest JDK 9 sources. The following tests were identified:- - Full JTreg run on arm32 & arm64 (Ed) - JCStress on arm32 (Stuart to investigate) - Retarget Linaro CI loop from hs repo to merged arm32/arm64 repo (Stuart to investigate)   (Linaro CI Loop runs JTReg, JCStress, Hadoop/Terasort & SpecJBB2015) - Build RedHat using merged arm32/arm64 repo. Andrew will investigate whether the team ad RedHat can do this. - Build and test softfloat version. Problem with lack of Hardware. Various offers/suiggestions. It was agreed that once people complete any testing they should append details of the test and results to the JEP so that a record may be kept. Thank you to the participants of the call and apologies if I have omitted any points. Please feel free to add any points which you feel I have omitted.
          Hide
          enevill Ed Nevill added a comment -
          The following are the results of JTReg testing on the 'hs' repository with the unified arm32/64 patch.

          hs repo: http://hg.openjdk.java.net/jdk9/hs
          patch: http://cr.openjdk.java.net/~bobv/8168503/jdk9-opensource-openjdk.patch

          The tests were run on three systems x86, aarch64 (arm64) and armv7 (arm32).

          In all cases a release build was tested.

          For x86 the tests were run on the original repo and then the patched repo for comparison. For aaarch64 two sets of tests were run, one on the existing aarch64 port (aarch64) and one on the unified port (arm64).

          For the jdk tests the options -k:\!headful\&\!intermittant\&\!randomness to suppress any tests requiring X11 and to remove random and intermittant tests.

          Here are the test results

          x86(original):

          hotspot: passed: 1,337; failed: 5; error: 44 (0 fatal errors)
          langtools: passed: 3,757; failed: 5; error: 8 (0 fatal errors)
          jdk: passed: 6,348; failed: 194; error: 11 (1 fatal error)

          x86(patched):

          hotspot: passed: 1,337; failed: 5; error: 44 (0 fatal errors)
          langtools: passed: 3,757; failed: 5; error: 8 (0 fatal errors)
          jdk: passed: 6,346; failed: 196; error: 10 (0 fatal errors)

          arm32

          hotspot: passed: 1,244; failed: 5; error: 44 (0 fatal errors)
          langtools: passed: 3,761; failed: 1; error: 8 (0 fatal errors)
          jdk: passed: 6,339; failed: 197; error: 11 (0 fatal errors)

          aarch64

          hotspot: passed: 1,325; failed: 9; error: 44 (0 fatal errors)
          langtools: passed: 3,757; failed: 2; error: 8 (0 fatal errors)
          jdk: passed: 6,389; failed: 155; error: 9 (0 fatal errors)

          arm64

          hotspot: passed: 1,327; failed: 7; error: 44 (0 fatal errors)
          langtools: passed 3,760; failed: 1; error: 8 (0 fatal errors)
          jdk: passed: 6,408; failed: 136; error: 9 (0 fatal errors)

          * 'fatal' errors are those where the test failed with the message "A fatal error has been detected" in the .jtr file.
          Show
          enevill Ed Nevill added a comment - The following are the results of JTReg testing on the 'hs' repository with the unified arm32/64 patch. hs repo: http://hg.openjdk.java.net/jdk9/hs patch: http://cr.openjdk.java.net/~bobv/8168503/jdk9-opensource-openjdk.patch The tests were run on three systems x86, aarch64 (arm64) and armv7 (arm32). In all cases a release build was tested. For x86 the tests were run on the original repo and then the patched repo for comparison. For aaarch64 two sets of tests were run, one on the existing aarch64 port (aarch64) and one on the unified port (arm64). For the jdk tests the options -k:\!headful\&\!intermittant\&\!randomness to suppress any tests requiring X11 and to remove random and intermittant tests. Here are the test results x86(original): hotspot: passed: 1,337; failed: 5; error: 44 (0 fatal errors) langtools: passed: 3,757; failed: 5; error: 8 (0 fatal errors) jdk: passed: 6,348; failed: 194; error: 11 (1 fatal error) x86(patched): hotspot: passed: 1,337; failed: 5; error: 44 (0 fatal errors) langtools: passed: 3,757; failed: 5; error: 8 (0 fatal errors) jdk: passed: 6,346; failed: 196; error: 10 (0 fatal errors) arm32 hotspot: passed: 1,244; failed: 5; error: 44 (0 fatal errors) langtools: passed: 3,761; failed: 1; error: 8 (0 fatal errors) jdk: passed: 6,339; failed: 197; error: 11 (0 fatal errors) aarch64 hotspot: passed: 1,325; failed: 9; error: 44 (0 fatal errors) langtools: passed: 3,757; failed: 2; error: 8 (0 fatal errors) jdk: passed: 6,389; failed: 155; error: 9 (0 fatal errors) arm64 hotspot: passed: 1,327; failed: 7; error: 44 (0 fatal errors) langtools: passed 3,760; failed: 1; error: 8 (0 fatal errors) jdk: passed: 6,408; failed: 136; error: 9 (0 fatal errors) * 'fatal' errors are those where the test failed with the message "A fatal error has been detected" in the .jtr file.
          Hide
          enevill Ed Nevill added a comment -
          The following are the results of JCStress testing on am32 and arm64/aarch64.

          The tests were run in 'quick' mode because of the excessive time to run them in 'default' mode on the systems available to us.

          arm32: http://cr.openjdk.java.net/~enevill/jcstress_results/results_arm32/
          arm64: http://cr.openjdk.java.net/~enevill/jcstress_results/results_arm64/
          aarch64: http://cr.openjdk.java.net/~enevill/jcstress_results/results_aarch64/

          In all cases there was 100% pass rate.

          There were some 'formally acceptable but surprising results' for arm32. This is probably due to 64 bit values being non atomic on arm32 and therefore having 'schrodinger cat' values.

          There were also some formally acceptable but surprising results for aarch64 which did not occur with the unified arm32/arm64 port.
          Show
          enevill Ed Nevill added a comment - The following are the results of JCStress testing on am32 and arm64/aarch64. The tests were run in 'quick' mode because of the excessive time to run them in 'default' mode on the systems available to us. arm32: http://cr.openjdk.java.net/~enevill/jcstress_results/results_arm32/ arm64: http://cr.openjdk.java.net/~enevill/jcstress_results/results_arm64/ aarch64: http://cr.openjdk.java.net/~enevill/jcstress_results/results_aarch64/ In all cases there was 100% pass rate. There were some 'formally acceptable but surprising results' for arm32. This is probably due to 64 bit values being non atomic on arm32 and therefore having 'schrodinger cat' values. There were also some formally acceptable but surprising results for aarch64 which did not occur with the unified arm32/arm64 port.
          Hide
          enevill Ed Nevill added a comment - - edited
          The following is the latest patchset for proposed integration into the JDK9 hotspot forest.

          This merges in the latest AOT and Jigsaw changes

          http://cr.openjdk.java.net/~bobv/8168503-01/hotspot/webrev/ <http://cr.openjdk.java.net/~bobv/8168503-01/hotspot/webrev/>
          http://cr.openjdk.java.net/~bobv/8168503-01/jdk/webrev/ <http://cr.openjdk.java.net/~bobv/8168503-01/jdk/webrev/>
          http://cr.openjdk.java.net/~bobv/8168503-01/top/webrev/ <http://cr.openjdk.java.net/~bobv/8168503-01/top/webrev/>

          The following are the jtreg results from a release build of the hs repo using the above patches on x86, arm64 and arm32

          x86:

          hotspot: passed: 1,372; failed: 30; error: 55
          langtools: 3,773; failed: 12; error: 9
          jdk: passed: 6,378; failed: 202; error: 10

          arm64:

          hotspot: passed: 1,315; failed: 31; error: 49
          langtools: 3,772; failed: 10; error: 10
          jdk: passed: 6,433; failed: 148; error: 8

          arm32:

          hotspot: 1,212; failed: 50; error: 48
          langtools: passed: 3,771; failed: 14; error: 9
          jdk: passed: 6,043; failed: 442; error: 18

          No fatal errors were detected.

          The results for jcstress testing with the above changesets are available here:-

          arm64: http://cr.openjdk.java.net/~enevill/jcstress/results_arm64
          arm32: http://cr.openjdk.java.net/~enevill/jcstress/results_arm32

          There are 2 failures on arm32

             o.o.j.t.accessAtomic.fields.volatiles.LongTest
             o.o.j.t.atomicity.primitives.perbyte.VolatileLongAtomicityTest

          These will need to be fixed but it is proposed that this can be done post integration.
          Show
          enevill Ed Nevill added a comment - - edited The following is the latest patchset for proposed integration into the JDK9 hotspot forest. This merges in the latest AOT and Jigsaw changes http://cr.openjdk.java.net/~bobv/8168503-01/hotspot/webrev/ < http://cr.openjdk.java.net/~bobv/8168503-01/hotspot/webrev/ > http://cr.openjdk.java.net/~bobv/8168503-01/jdk/webrev/ < http://cr.openjdk.java.net/~bobv/8168503-01/jdk/webrev/ > http://cr.openjdk.java.net/~bobv/8168503-01/top/webrev/ < http://cr.openjdk.java.net/~bobv/8168503-01/top/webrev/ > The following are the jtreg results from a release build of the hs repo using the above patches on x86, arm64 and arm32 x86: hotspot: passed: 1,372; failed: 30; error: 55 langtools: 3,773; failed: 12; error: 9 jdk: passed: 6,378; failed: 202; error: 10 arm64: hotspot: passed: 1,315; failed: 31; error: 49 langtools: 3,772; failed: 10; error: 10 jdk: passed: 6,433; failed: 148; error: 8 arm32: hotspot: 1,212; failed: 50; error: 48 langtools: passed: 3,771; failed: 14; error: 9 jdk: passed: 6,043; failed: 442; error: 18 No fatal errors were detected. The results for jcstress testing with the above changesets are available here:- arm64: http://cr.openjdk.java.net/~enevill/jcstress/results_arm64 arm32: http://cr.openjdk.java.net/~enevill/jcstress/results_arm32 There are 2 failures on arm32    o.o.j.t.accessAtomic.fields.volatiles.LongTest    o.o.j.t.atomicity.primitives.perbyte.VolatileLongAtomicityTest These will need to be fixed but it is proposed that this can be done post integration.
          Hide
          kvn Vladimir Kozlov added a comment -
          [~enevill] Ed, please close this JEP - press "Delivered" button.
          Show
          kvn Vladimir Kozlov added a comment - [~enevill] Ed, please close this JEP - press "Delivered" button.

            People

            • Assignee:
              enevill Ed Nevill
              Reporter:
              enevill Ed Nevill
              Owner:
              Ed Nevill
              Reviewed By:
              Bob Vandette, Mikael Vidstedt, Vladimir Kozlov
              Endorsed By:
              Mikael Vidstedt, Vladimir Kozlov
            • Votes:
              1 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Due:
                Created:
                Updated:
                Resolved:
                Integration Due: