Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8153711

[REDO] JDWP: Memory Leak: GlobalRefs never deleted when processing invokeMethod command

    Details

    • Subcomponent:
    • Resolved In Build:
      b139
    • CPU:
      generic
    • OS:
      generic

      Backports

        Description

        The fix of:
          JDK-4858370 JDWP: Memory Leak: GlobalRefs never deleted when processing invokeMethod command

        was backed out with:
          JDK-8153673 [BACKOUT] JDWP: Memory Leak: GlobalRefs never deleted when processing invokeMethod command

        because of the following regressions:
          JDK-8152686 com/sun/jdi/OomDebugTest.java fails with java.lang.reflect.InvocationTargetException
          JDK-8152586 com/sun/jdi/InterfaceMethodsTest.java fails in hs nightly
          JDK-8152985 nsk/jdi tests fail with unexpected com.sun.jdi.InvocationException

        A corrected fix for the original problem with GlobalRefs memory leak is needed.
        1. JDK-8153711-jdk9-jdk.export.patch
          21 kB
          Severin Gehwolf
        2. JDK-8153711-jdk9-jdk.export.patch
          14 kB
          Severin Gehwolf
        3. JDK-8153711-jdk9-jdk.export.patch
          14 kB
          Severin Gehwolf

          Issue Links

            Activity

            Hide
            sgehwolf Severin Gehwolf added a comment -
            It appears the initial fix for JDK-4858370 deleted global references while *not* holding the invoker lock, but I think it should. That is, the callsite of deleteGlobalRefs() needs to move up to to before the invoker lock is released. I'm currently having issues with the test (OomDebugTest), but I'll report back once I have that issue resolved.
            Show
            sgehwolf Severin Gehwolf added a comment - It appears the initial fix for JDK-4858370 deleted global references while *not* holding the invoker lock, but I think it should. That is, the callsite of deleteGlobalRefs() needs to move up to to before the invoker lock is released. I'm currently having issues with the test (OomDebugTest), but I'll report back once I have that issue resolved.
            Hide
            sspitsyn Serguei Spitsyn added a comment -
            [To Severin G.: ]
            I think, you spotted the issue.
            Feel free to re-assign the bug to yourself if you want to persist.
            Show
            sspitsyn Serguei Spitsyn added a comment - [To Severin G.: ] I think, you spotted the issue. Feel free to re-assign the bug to yourself if you want to persist.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            @Serguei
            This isn't the whole story, unfortunately.

            First, deleteGlobalRefs() needs to be called after the reply has been sent. Otherwise some variables needed for the reply get cleared. So my comment above is incorrect.

            Second, when we do clear global references after the reply has been sent this also clears global references created via JDI for example with ClassType.newInstance. This will make the code more susceptible to run into the issue as described in JDK-4257193.

            Finally, all invoker* incarnations, such as invokeConstructor, don't hold the invokelock either, even though, they are all saving global refs pointed to by the current request, which a fix for this issue would delete. There seems to be a possibility for races.
            Show
            sgehwolf Severin Gehwolf added a comment - @Serguei This isn't the whole story, unfortunately. First, deleteGlobalRefs() needs to be called after the reply has been sent. Otherwise some variables needed for the reply get cleared. So my comment above is incorrect. Second, when we do clear global references after the reply has been sent this also clears global references created via JDI for example with ClassType.newInstance. This will make the code more susceptible to run into the issue as described in JDK-4257193 . Finally, all invoker* incarnations, such as invokeConstructor, don't hold the invokelock either, even though, they are all saving global refs pointed to by the current request, which a fix for this issue would delete. There seems to be a possibility for races.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            Current WIP fix.
            Show
            sgehwolf Severin Gehwolf added a comment - Current WIP fix.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            With the attached patch, com/sun/jdi/InterfaceMethodsTest.java passed ~1250 times. But there also was one failure. Not sure if it's related to the third issue described above or the second.

            Also, I seem to get fairly frequent failures in OomDebugTest (mostly SIGSEGV in the jvm) when JDI's newInstance() is being called. I'll see if acquiring the locks in invoke* functions fixes that.
            Show
            sgehwolf Severin Gehwolf added a comment - With the attached patch, com/sun/jdi/InterfaceMethodsTest.java passed ~1250 times. But there also was one failure. Not sure if it's related to the third issue described above or the second. Also, I seem to get fairly frequent failures in OomDebugTest (mostly SIGSEGV in the jvm) when JDI's newInstance() is being called. I'll see if acquiring the locks in invoke* functions fixes that.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            Improved fix, with locking during do_invoke()
            Show
            sgehwolf Severin Gehwolf added a comment - Improved fix, with locking during do_invoke()
            Hide
            sgehwolf Severin Gehwolf added a comment -
            With the latest patch com/sun/jdi/InterfaceMethodsTest.java and com/sun/jdi/InvokeTest.java did not fail for 1300 runs.
            Show
            sgehwolf Severin Gehwolf added a comment - With the latest patch com/sun/jdi/InterfaceMethodsTest.java and com/sun/jdi/InvokeTest.java did not fail for 1300 runs.
            Show
            sgehwolf Severin Gehwolf added a comment - Posted patch for review: http://mail.openjdk.java.net/pipermail/serviceability-dev/2016-April/019410.html
            Hide
            sgehwolf Severin Gehwolf added a comment -
            Some clarifications.

            1.) The original patch for JDK-4858370 did not hold the eventHandler lock and neither the invoker lock as the rest of invoker_completeInvokeRequest() does. This caused test failures in com/sun/jdi/InterfaceMethodsTest.java and others. Grabbing the locks again in invoker_completeInvokeRequest() before clearing global references fixes this.

            2.) Even though there are cases where global references are saved while not holding the invoker lock (see invoker_doInvoke() and related impls such as invokeStatic()) this seems to be OK since invokes need to go through invoker_requestInvoke() and finish via completeInvokeRequest() which grabs the lock again. Grabbing the invoker lock too coarsely while doing the invoke may cause deadlocks. As seen during review of the first version of the fix for this bug.
            Show
            sgehwolf Severin Gehwolf added a comment - Some clarifications. 1.) The original patch for JDK-4858370 did not hold the eventHandler lock and neither the invoker lock as the rest of invoker_completeInvokeRequest() does. This caused test failures in com/sun/jdi/InterfaceMethodsTest.java and others. Grabbing the locks again in invoker_completeInvokeRequest() before clearing global references fixes this. 2.) Even though there are cases where global references are saved while not holding the invoker lock (see invoker_doInvoke() and related impls such as invokeStatic()) this seems to be OK since invokes need to go through invoker_requestInvoke() and finish via completeInvokeRequest() which grabs the lock again. Grabbing the invoker lock too coarsely while doing the invoke may cause deadlocks. As seen during review of the first version of the fix for this bug.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            Note that I'm still seeing fairly rare cases (1 out of 1000) where OomDebugTest would timeout. The debugger waits for the reply from the debuggee which is in process of fulfilling a newInstance() request of a primitive array. However that reply never seems to come, because for some reason the call to jni_NewByteArray never completes? This looks like an issue in GC to me and not related to this bug. Here is the jstack snippet of the hung OomDebugTestTarg JVM process:

            [...]
            ----------------- 28829 -----------------
            0x00007efeb8e9cb10 __pthread_cond_wait + 0xc0
            0x00007efeb7f3419f Monitor::IWait(Thread*, long) + 0xef
            0x00007efeb7f3541e Monitor::wait(bool, long, bool) + 0x22e
            0x00007efeb7fff7aa ReferencePendingListLockerThread::receive_and_handle_messages() + 0x4a
            0x00007efeb7fff879 ????????
            0x00007efeb80dd227 JavaThread::thread_main_inner() + 0x1e7
            0x00007efeb7f6f430 java_start(Thread*) + 0xf0
            Locked ownable synchronizers:
                - None
            [...]
            ----------------- 28837 -----------------
            0x00007efeb8e9ceb9 __pthread_cond_timedwait + 0x129
            0x00007efeb7f56bf3 ObjectMonitor::EnterI(Thread*) + 0x3c3
            0x00007efeb7f572c0 ObjectMonitor::enter(Thread*) + 0x320
            0x00007efeb7fff3ea ReferencePendingListLocker::lock() + 0xca
            0x00007efeb813af3f VM_GC_Operation::doit_prologue() + 0x6f
            0x00007efeb8149be1 VM_G1IncCollectionPause::doit_prologue() + 0x11
            0x00007efeb814818e VMThread::execute(VM_Operation*) + 0x22e
            0x00007efeb7bd4fb5 G1CollectedHeap::collect(GCCause::Cause) + 0x95
            0x00007efeb7bdb28a G1CollectedHeap::attempt_allocation_humongous(unsigned long, unsigned int*, unsigned int*) + 0x2ea
            0x00007efeb7bdb519 G1CollectedHeap::mem_allocate(unsigned long, bool*) + 0x219
            0x00007efeb80f6501 TypeArrayKlass::allocate_common(int, bool, Thread*) + 0xa1
            0x00007efeb7ce5713 jni_NewByteArray + 0xa3
            0x00007efeb641ab56 newInstance + 0x496
            0x00007efeb6428fd8 debugLoop_run + 0x268
            0x00007efeb643b075 attachThread + 0x25
            0x00007efeb7dbaa00 JvmtiAgentThread::start_function_wrapper(JavaThread*, Thread*) + 0xb0
            0x00007efeb80dd227 JavaThread::thread_main_inner() + 0x1e7
            0x00007efeb7f6f430 java_start(Thread*) + 0xf0
            Locked ownable synchronizers:
                - None

            Interestingly enough it always seems to get stuck in test2() which instantiates larg-ish primitive byte arrays in the debuggee.
            Show
            sgehwolf Severin Gehwolf added a comment - Note that I'm still seeing fairly rare cases (1 out of 1000) where OomDebugTest would timeout. The debugger waits for the reply from the debuggee which is in process of fulfilling a newInstance() request of a primitive array. However that reply never seems to come, because for some reason the call to jni_NewByteArray never completes? This looks like an issue in GC to me and not related to this bug. Here is the jstack snippet of the hung OomDebugTestTarg JVM process: [...] ----------------- 28829 ----------------- 0x00007efeb8e9cb10 __pthread_cond_wait + 0xc0 0x00007efeb7f3419f Monitor::IWait(Thread*, long) + 0xef 0x00007efeb7f3541e Monitor::wait(bool, long, bool) + 0x22e 0x00007efeb7fff7aa ReferencePendingListLockerThread::receive_and_handle_messages() + 0x4a 0x00007efeb7fff879 ???????? 0x00007efeb80dd227 JavaThread::thread_main_inner() + 0x1e7 0x00007efeb7f6f430 java_start(Thread*) + 0xf0 Locked ownable synchronizers:     - None [...] ----------------- 28837 ----------------- 0x00007efeb8e9ceb9 __pthread_cond_timedwait + 0x129 0x00007efeb7f56bf3 ObjectMonitor::EnterI(Thread*) + 0x3c3 0x00007efeb7f572c0 ObjectMonitor::enter(Thread*) + 0x320 0x00007efeb7fff3ea ReferencePendingListLocker::lock() + 0xca 0x00007efeb813af3f VM_GC_Operation::doit_prologue() + 0x6f 0x00007efeb8149be1 VM_G1IncCollectionPause::doit_prologue() + 0x11 0x00007efeb814818e VMThread::execute(VM_Operation*) + 0x22e 0x00007efeb7bd4fb5 G1CollectedHeap::collect(GCCause::Cause) + 0x95 0x00007efeb7bdb28a G1CollectedHeap::attempt_allocation_humongous(unsigned long, unsigned int*, unsigned int*) + 0x2ea 0x00007efeb7bdb519 G1CollectedHeap::mem_allocate(unsigned long, bool*) + 0x219 0x00007efeb80f6501 TypeArrayKlass::allocate_common(int, bool, Thread*) + 0xa1 0x00007efeb7ce5713 jni_NewByteArray + 0xa3 0x00007efeb641ab56 newInstance + 0x496 0x00007efeb6428fd8 debugLoop_run + 0x268 0x00007efeb643b075 attachThread + 0x25 0x00007efeb7dbaa00 JvmtiAgentThread::start_function_wrapper(JavaThread*, Thread*) + 0xb0 0x00007efeb80dd227 JavaThread::thread_main_inner() + 0x1e7 0x00007efeb7f6f430 java_start(Thread*) + 0xf0 Locked ownable synchronizers:     - None Interestingly enough it always seems to get stuck in test2() which instantiates larg-ish primitive byte arrays in the debuggee.
            Hide
            sspitsyn Serguei Spitsyn added a comment -
            I see similar behavior.
            Below is a jstack output fragment:

            ----------------- 9131 -----------------
            0x00007f82c9063d84 __pthread_cond_wait + 0xc4
            0x00007f82c84e83c7 Monitor::IWait(Thread*, long) + 0x127
            0x00007f82c84ea3ed Monitor::wait(bool, long, bool) + 0x1fd
            0x00007f82c877b114 _ZN10JavaThread17java_suspend_selfEv.part.195 + 0xb4
            0x00007f82c82cd919 JvmtiRawMonitor::raw_wait(long, bool, Thread*) + 0x229
            0x00007f82c82a2a82 JvmtiEnv::RawMonitorWait(JvmtiRawMonitor*, long) + 0xa2
            0x00007f82c69d9b49 debugMonitorWait + 0x29
            0x00007f82c69cad6e enqueueCommand + 0x11e
            0x00007f82c69c6a22 reportEvents.part.3 + 0xa2
            0x00007f82c69c7046 event_callback + 0x446
            0x00007f82c69c9d55 cbBreakpoint + 0xc5
            0x00007f82c82bc78f JvmtiExport::post_raw_breakpoint(JavaThread*, Method*, unsigned char*) + 0x1ef
            0x00007f82c80bf662 InterpreterRuntime::_breakpoint(JavaThread*, Method*, unsigned char*) + 0xa2
            0x00007f82a89af712 Locked ownable synchronizers:
                - None
            . . . . . . . . . . . . . . . . . .

            ----------------- 9169 -----------------
            0x00007f82c9063d84 __pthread_cond_wait + 0xc4
            0x00007f82c84e83c7 Monitor::IWait(Thread*, long) + 0x127
            0x00007f82c84ea98f Monitor::wait(bool, long, bool) + 0x79f
            0x00007f82c861d380 ReferencePendingListLockerThread::receive_and_handle_messages() + 0xb0
            0x00007f82c861d8a9 ????????
            0x00007f82c878c991 JavaThread::thread_main_inner() + 0x1d1
            0x00007f82c878cb3a JavaThread::run() + 0x15a
            0x00007f82c854a3b2 java_start(Thread*) + 0x142
            Locked ownable synchronizers:
                - None
            . . . . . . . . . . . . . . . . . .

            ----------------- 9202 -----------------
            0x00007f82c90640fe __pthread_cond_timedwait + 0x13e
            0x00007f82c85228f1 ObjectMonitor::EnterI(Thread*) + 0x581
            0x00007f82c8523147 ObjectMonitor::enter(Thread*) + 0x307
            0x00007f82c872fb11 ObjectSynchronizer::slow_enter(Handle, BasicLock*, Thread*) + 0xa1
            0x00007f82c872fdcb ObjectSynchronizer::fast_enter(Handle, BasicLock*, bool, Thread*) + 0x8b
            0x00007f82c861d9ef ReferencePendingListLocker::lock() + 0x13f
            0x00007f82c87ffdef VM_GC_Operation::doit_prologue() + 0xbf
            0x00007f82c8832221 VM_G1IncCollectionPause::doit_prologue() + 0x11
            0x00007f82c882d49e VMThread::execute(VM_Operation*) + 0x2ee
            0x00007f82c7f4a5ee G1CollectedHeap::collect(GCCause::Cause) + 0x1ce
            0x00007f82c7f56c41 G1CollectedHeap::attempt_allocation_humongous(unsigned long, unsigned int*, unsigned int*) + 0x3e1
            0x00007f82c7f56f1d G1CollectedHeap::mem_allocate(unsigned long, bool*) + 0x26d
            0x00007f82c7abe0d7 CollectedHeap::array_allocate(KlassHandle, int, int, Thread*) + 0x2d7
            0x00007f82c87b0864 TypeArrayKlass::allocate_common(int, bool, Thread*) + 0x1d4
            0x00007f82c8179c3d jni_NewByteArray + 0x16d
            0x00007f82c69b5d20 newInstance + 0x430
            0x00007f82c69c3fd8 debugLoop_run + 0x298
            0x00007f82c69d661e attachThread + 0x2e
            0x00007f82c82c6093 JvmtiAgentThread::call_start_function() + 0x153
            0x00007f82c878c991 JavaThread::thread_main_inner() + 0x1d1
            0x00007f82c878cb3a JavaThread::run() + 0x15a
            0x00007f82c854a3b2 java_start(Thread*) + 0x142
            Locked ownable synchronizers:
                - None
            Show
            sspitsyn Serguei Spitsyn added a comment - I see similar behavior. Below is a jstack output fragment: ----------------- 9131 ----------------- 0x00007f82c9063d84 __pthread_cond_wait + 0xc4 0x00007f82c84e83c7 Monitor::IWait(Thread*, long) + 0x127 0x00007f82c84ea3ed Monitor::wait(bool, long, bool) + 0x1fd 0x00007f82c877b114 _ZN10JavaThread17java_suspend_selfEv.part.195 + 0xb4 0x00007f82c82cd919 JvmtiRawMonitor::raw_wait(long, bool, Thread*) + 0x229 0x00007f82c82a2a82 JvmtiEnv::RawMonitorWait(JvmtiRawMonitor*, long) + 0xa2 0x00007f82c69d9b49 debugMonitorWait + 0x29 0x00007f82c69cad6e enqueueCommand + 0x11e 0x00007f82c69c6a22 reportEvents.part.3 + 0xa2 0x00007f82c69c7046 event_callback + 0x446 0x00007f82c69c9d55 cbBreakpoint + 0xc5 0x00007f82c82bc78f JvmtiExport::post_raw_breakpoint(JavaThread*, Method*, unsigned char*) + 0x1ef 0x00007f82c80bf662 InterpreterRuntime::_breakpoint(JavaThread*, Method*, unsigned char*) + 0xa2 0x00007f82a89af712 Locked ownable synchronizers:     - None . . . . . . . . . . . . . . . . . . ----------------- 9169 ----------------- 0x00007f82c9063d84 __pthread_cond_wait + 0xc4 0x00007f82c84e83c7 Monitor::IWait(Thread*, long) + 0x127 0x00007f82c84ea98f Monitor::wait(bool, long, bool) + 0x79f 0x00007f82c861d380 ReferencePendingListLockerThread::receive_and_handle_messages() + 0xb0 0x00007f82c861d8a9 ???????? 0x00007f82c878c991 JavaThread::thread_main_inner() + 0x1d1 0x00007f82c878cb3a JavaThread::run() + 0x15a 0x00007f82c854a3b2 java_start(Thread*) + 0x142 Locked ownable synchronizers:     - None . . . . . . . . . . . . . . . . . . ----------------- 9202 ----------------- 0x00007f82c90640fe __pthread_cond_timedwait + 0x13e 0x00007f82c85228f1 ObjectMonitor::EnterI(Thread*) + 0x581 0x00007f82c8523147 ObjectMonitor::enter(Thread*) + 0x307 0x00007f82c872fb11 ObjectSynchronizer::slow_enter(Handle, BasicLock*, Thread*) + 0xa1 0x00007f82c872fdcb ObjectSynchronizer::fast_enter(Handle, BasicLock*, bool, Thread*) + 0x8b 0x00007f82c861d9ef ReferencePendingListLocker::lock() + 0x13f 0x00007f82c87ffdef VM_GC_Operation::doit_prologue() + 0xbf 0x00007f82c8832221 VM_G1IncCollectionPause::doit_prologue() + 0x11 0x00007f82c882d49e VMThread::execute(VM_Operation*) + 0x2ee 0x00007f82c7f4a5ee G1CollectedHeap::collect(GCCause::Cause) + 0x1ce 0x00007f82c7f56c41 G1CollectedHeap::attempt_allocation_humongous(unsigned long, unsigned int*, unsigned int*) + 0x3e1 0x00007f82c7f56f1d G1CollectedHeap::mem_allocate(unsigned long, bool*) + 0x26d 0x00007f82c7abe0d7 CollectedHeap::array_allocate(KlassHandle, int, int, Thread*) + 0x2d7 0x00007f82c87b0864 TypeArrayKlass::allocate_common(int, bool, Thread*) + 0x1d4 0x00007f82c8179c3d jni_NewByteArray + 0x16d 0x00007f82c69b5d20 newInstance + 0x430 0x00007f82c69c3fd8 debugLoop_run + 0x298 0x00007f82c69d661e attachThread + 0x2e 0x00007f82c82c6093 JvmtiAgentThread::call_start_function() + 0x153 0x00007f82c878c991 JavaThread::thread_main_inner() + 0x1d1 0x00007f82c878cb3a JavaThread::run() + 0x15a 0x00007f82c854a3b2 java_start(Thread*) + 0x142 Locked ownable synchronizers:     - None
            Hide
            sspitsyn Serguei Spitsyn added a comment -
            The jprt push of the fix failed with the NullPointerException:

            TEST: com/sun/jdi/InvokeTest.java
            TEST JDK: /opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug

            ACTION: build -- Passed. All files up to date
            REASON: User specified action: run build TestScaffold VMConnection TargetListener TargetAdapter
            TIME: 0.0 seconds
            messages:
            command: build TestScaffold VMConnection TargetListener TargetAdapter
            reason: User specified action: run build TestScaffold VMConnection TargetListener TargetAdapter
            elapsed time (seconds): 0.0

            ACTION: compile -- Passed. Compilation successful
            REASON: User specified action: run compile -g InvokeTest.java
            TIME: 0.106 seconds
            messages:
            command: compile -g /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi/InvokeTest.java
            reason: User specified action: run compile -g InvokeTest.java
            Mode: agentvm
            elapsed time (seconds): 0.106
            configuration:
            Boot Layer (javac execution environment)
              class path: /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/javatest.jar
                          /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/jtreg.jar
              patch: java.base /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/patches/java.base

            javac compilation environment
              class path: /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi
                          /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi
                          /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun

            rerun:
            DISPLAY=scaaa563.us.oracle.com:514 \
            HOME=/opt/jprt/jprtadm \
            JDK8_HOME= \
            LANG=C \
            LC_ALL=C \
            LC_CTYPE= \
            LD_LIBRARY_PATH=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \
            PATH=/bin:/usr/bin \
            TZ=localtime \
                /opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug/bin/javac \
                    -J-ea \
                    -J-esa \
                    -J-d64 \
                    -J-server \
                    -J-Xmx512m \
                    -J-Duser.home=/opt/jprt/T/P1/100525.sspitsyn \
                    -J-Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity \
                    -J-d64 \
                    -J-server \
                    -J-Djava.library.path=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \
                    -J-Dtest.class.path.prefix=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \
                    -J-Dtest.src=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi \
                    -J-Dtest.src.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun \
                    -J-Dtest.classes=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi \
                    -J-Dtest.class.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \
                    -J-Dtest.vm.opts='-ea -esa -d64 -server -Xmx512m' \
                    -J-Dtest.tool.vm.opts='-J-ea -J-esa -J-d64 -J-server -J-Xmx512m' \
                    -J-Dtest.compiler.opts= \
                    -J-Dtest.java.opts='-Duser.home=/opt/jprt/T/P1/100525.sspitsyn -Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity -d64 -server' \
                    -J-Dtest.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \
                    -J-Dcompile.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \
                    -J-Dtest.timeout.factor=4.0 \
                    -J-Dtest.modules=jdk.jdi \
                    -J-Dtest.nativepath=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \
                    -d /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi \
                    -sourcepath /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun \
                    -classpath /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \
                    -g /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi/InvokeTest.java
            direct:
            Note: /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi/InvokeTest.java uses unchecked or unsafe operations.
            Note: Recompile with -Xlint:unchecked for details.

            ACTION: build -- Passed. All files up to date
            REASON: Named class compiled on demand
            TIME: 0.0 seconds
            messages:
            command: build InvokeTest
            reason: Named class compiled on demand
            elapsed time (seconds): 0.0

            ACTION: driver -- Failed. Execution failed: `main' threw exception: java.lang.NullPointerException
            REASON: User specified action: run driver InvokeTest
            TIME: 0.541 seconds
            messages:
            command: driver InvokeTest
            reason: User specified action: run driver InvokeTest
            Mode: agentvm
            elapsed time (seconds): 0.541
            configuration:
            Boot Layer
              class path: /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/javatest.jar
                          /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/jtreg.jar
              patch: java.base /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/patches/java.base

            Test Layer
              class path: /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi
                          /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi
                          /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun
                          /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun

            rerun:
            DISPLAY=scaaa563.us.oracle.com:514 \
            HOME=/opt/jprt/jprtadm \
            JDK8_HOME= \
            LANG=C \
            LC_ALL=C \
            LC_CTYPE= \
            LD_LIBRARY_PATH=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \
            PATH=/bin:/usr/bin \
            TZ=localtime \
                /opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug/bin/java \
                    -Dtest.class.path.prefix=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \
                    -Dtest.src=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi \
                    -Dtest.src.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun \
                    -Dtest.classes=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi \
                    -Dtest.class.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \
                    -Dtest.vm.opts='-ea -esa -d64 -server -Xmx512m' \
                    -Dtest.tool.vm.opts='-J-ea -J-esa -J-d64 -J-server -J-Xmx512m' \
                    -Dtest.compiler.opts= \
                    -Dtest.java.opts='-Duser.home=/opt/jprt/T/P1/100525.sspitsyn -Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity -d64 -server' \
                    -Dtest.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \
                    -Dcompile.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \
                    -Dtest.timeout.factor=4.0 \
                    -Dtest.modules=jdk.jdi \
                    -Dtest.nativepath=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \
                    -classpath /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun:/opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/javatest.jar:/opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/jtreg.jar \
                    InvokeTest
            STDOUT:
            vmOpts: '-ea -esa -d64 -server -Xmx512m'
            javaOpts: '-Duser.home=/opt/jprt/T/P1/100525.sspitsyn -Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity -d64 -server'
            JVM version:9-internal
            JDI version: 9.0
            JVM description: Java Debug Interface (Reference Implementation) version 9.0
            Java Debug Wire Protocol (Reference Implementation) version 9.0
            JVM Debug Interface version 9.0
            JVM version 9-internal (Java HotSpot(TM) 64-Bit Server VM, mixed mode, sharing)
            Howdy!
            STDERR:
            [2ms] run args: [InvokeTarg]
            [373ms] return val = "[Z@6adca536"
            [374ms] toString return value : "[Z@6adca536"
            [376ms] return val = true
            [376ms] invokeVoid return value matches: true
            [378ms] return val = true
            [378ms] invokeBoolean return value matches: true
            [379ms] Got Exception: com.sun.jdi.InvocationException: Exception occurred in target VM
            com.sun.jdi.InvocationException: Exception occurred in target VM
            at com.sun.tools.jdi.ObjectReferenceImpl.invokeMethod(jdk.jdi@9-internal/ObjectReferenceImpl.java:436)
            at InvokeTest.invoke(InvokeTest.java:330)
            at InvokeTest.invoke(InvokeTest.java:365)
            at InvokeTest.invoke(InvokeTest.java:372)
            at InvokeTest.runTests(InvokeTest.java:457)
            at TestScaffold.startTests(TestScaffold.java:431)
            at InvokeTest.main(InvokeTest.java:310)
            at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
            at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
            at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:531)
            at com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:226)
            at java.lang.Thread.run(java.base@9-internal/Thread.java:843)
            [383ms] return val = null
            java.lang.NullPointerException
            at InvokeTest.invoke(InvokeTest.java:338)
            at InvokeTest.invoke(InvokeTest.java:365)
            at InvokeTest.invoke(InvokeTest.java:372)
            at InvokeTest.runTests(InvokeTest.java:457)
            at TestScaffold.startTests(TestScaffold.java:431)
            at InvokeTest.main(InvokeTest.java:310)
            at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(java.base@9-internal/Native Method)
            at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(java.base@9-internal/NativeMethodAccessorImpl.java:62)
            at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(java.base@9-internal/DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(java.base@9-internal/Method.java:531)
            at com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:226)
            at java.lang.Thread.run(java.base@9-internal/Thread.java:843)

            JavaTest Message: Test threw exception: java.lang.NullPointerException
            JavaTest Message: shutting down test


            TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.NullPointerException
            Show
            sspitsyn Serguei Spitsyn added a comment - The jprt push of the fix failed with the NullPointerException: TEST: com/sun/jdi/InvokeTest.java TEST JDK: /opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug ACTION: build -- Passed. All files up to date REASON: User specified action: run build TestScaffold VMConnection TargetListener TargetAdapter TIME: 0.0 seconds messages: command: build TestScaffold VMConnection TargetListener TargetAdapter reason: User specified action: run build TestScaffold VMConnection TargetListener TargetAdapter elapsed time (seconds): 0.0 ACTION: compile -- Passed. Compilation successful REASON: User specified action: run compile -g InvokeTest.java TIME: 0.106 seconds messages: command: compile -g /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi/InvokeTest.java reason: User specified action: run compile -g InvokeTest.java Mode: agentvm elapsed time (seconds): 0.106 configuration: Boot Layer (javac execution environment)   class path: /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/javatest.jar               /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/jtreg.jar   patch: java.base /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/patches/java.base javac compilation environment   class path: /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi               /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi               /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun rerun: DISPLAY=scaaa563.us.oracle.com:514 \ HOME=/opt/jprt/jprtadm \ JDK8_HOME= \ LANG=C \ LC_ALL=C \ LC_CTYPE= \ LD_LIBRARY_PATH=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \ PATH=/bin:/usr/bin \ TZ=localtime \     /opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug/bin/javac \         -J-ea \         -J-esa \         -J-d64 \         -J-server \         -J-Xmx512m \         -J-Duser.home=/opt/jprt/T/P1/100525.sspitsyn \         -J-Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity \         -J-d64 \         -J-server \         -J-Djava.library.path=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \         -J-Dtest.class.path.prefix=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \         -J-Dtest.src=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi \         -J-Dtest.src.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun \         -J-Dtest.classes=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi \         -J-Dtest.class.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \         -J-Dtest.vm.opts='-ea -esa -d64 -server -Xmx512m' \         -J-Dtest.tool.vm.opts='-J-ea -J-esa -J-d64 -J-server -J-Xmx512m' \         -J-Dtest.compiler.opts= \         -J-Dtest.java.opts='-Duser.home=/opt/jprt/T/P1/100525.sspitsyn -Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity -d64 -server' \         -J-Dtest.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \         -J-Dcompile.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \         -J-Dtest.timeout.factor=4.0 \         -J-Dtest.modules=jdk.jdi \         -J-Dtest.nativepath=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \         -d /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi \         -sourcepath /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun \         -classpath /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \         -g /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi/InvokeTest.java direct: Note: /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi/InvokeTest.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. ACTION: build -- Passed. All files up to date REASON: Named class compiled on demand TIME: 0.0 seconds messages: command: build InvokeTest reason: Named class compiled on demand elapsed time (seconds): 0.0 ACTION: driver -- Failed. Execution failed: `main' threw exception: java.lang.NullPointerException REASON: User specified action: run driver InvokeTest TIME: 0.541 seconds messages: command: driver InvokeTest reason: User specified action: run driver InvokeTest Mode: agentvm elapsed time (seconds): 0.541 configuration: Boot Layer   class path: /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/javatest.jar               /opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/jtreg.jar   patch: java.base /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/patches/java.base Test Layer   class path: /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi               /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi               /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun               /opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun rerun: DISPLAY=scaaa563.us.oracle.com:514 \ HOME=/opt/jprt/jprtadm \ JDK8_HOME= \ LANG=C \ LC_ALL=C \ LC_CTYPE= \ LD_LIBRARY_PATH=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \ PATH=/bin:/usr/bin \ TZ=localtime \     /opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug/bin/java \         -Dtest.class.path.prefix=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \         -Dtest.src=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi \         -Dtest.src.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun \         -Dtest.classes=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi \         -Dtest.class.path=/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun \         -Dtest.vm.opts='-ea -esa -d64 -server -Xmx512m' \         -Dtest.tool.vm.opts='-J-ea -J-esa -J-d64 -J-server -J-Xmx512m' \         -Dtest.compiler.opts= \         -Dtest.java.opts='-Duser.home=/opt/jprt/T/P1/100525.sspitsyn -Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity -d64 -server' \         -Dtest.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \         -Dcompile.jdk=/opt/jprt/T/P1/100525.sspitsyn/testproduct/solaris_x64_5.11-fastdebug \         -Dtest.timeout.factor=4.0 \         -Dtest.modules=jdk.jdi \         -Dtest.nativepath=/opt/jprt/T/P1/100525.sspitsyn/bundles/testBundle/jdk/jtreg/native \         -classpath /opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun/jdi:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/testoutput/jdk_svc_sanity/JTwork/classes/com/sun:/opt/jprt/T/P1/100525.sspitsyn/s/jdk/test/com/sun:/opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/javatest.jar:/opt/jprt/jib-data/install/java/re/jtreg/4.2/promoted/all/b02/bundles/jtreg_bin-4.2.zip/jtreg/lib/jtreg.jar \         InvokeTest STDOUT: vmOpts: '-ea -esa -d64 -server -Xmx512m' javaOpts: '-Duser.home=/opt/jprt/T/P1/100525.sspitsyn -Djava.io.tmpdir=/opt/jprt/T/P1/100525.sspitsyn/io/solaris_x64_5.11-fastdebug-c2-jdk_svc_sanity -d64 -server' JVM version:9-internal JDI version: 9.0 JVM description: Java Debug Interface (Reference Implementation) version 9.0 Java Debug Wire Protocol (Reference Implementation) version 9.0 JVM Debug Interface version 9.0 JVM version 9-internal (Java HotSpot(TM) 64-Bit Server VM, mixed mode, sharing) Howdy! STDERR: [2ms] run args: [InvokeTarg] [373ms] return val = "[ Z@6adca536 " [374ms] toString return value : "[ Z@6adca536 " [376ms] return val = true [376ms] invokeVoid return value matches: true [378ms] return val = true [378ms] invokeBoolean return value matches: true [379ms] Got Exception: com.sun.jdi.InvocationException: Exception occurred in target VM com.sun.jdi.InvocationException: Exception occurred in target VM at com.sun.tools.jdi.ObjectReferenceImpl.invokeMethod( jdk.jdi@9-internal /ObjectReferenceImpl.java:436) at InvokeTest.invoke(InvokeTest.java:330) at InvokeTest.invoke(InvokeTest.java:365) at InvokeTest.invoke(InvokeTest.java:372) at InvokeTest.runTests(InvokeTest.java:457) at TestScaffold.startTests(TestScaffold.java:431) at InvokeTest.main(InvokeTest.java:310) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0( java.base@9-internal /Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke( java.base@9-internal /NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke( java.base@9-internal /DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke( java.base@9-internal /Method.java:531) at com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:226) at java.lang.Thread.run( java.base@9-internal /Thread.java:843) [383ms] return val = null java.lang.NullPointerException at InvokeTest.invoke(InvokeTest.java:338) at InvokeTest.invoke(InvokeTest.java:365) at InvokeTest.invoke(InvokeTest.java:372) at InvokeTest.runTests(InvokeTest.java:457) at TestScaffold.startTests(TestScaffold.java:431) at InvokeTest.main(InvokeTest.java:310) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0( java.base@9-internal /Native Method) at jdk.internal.reflect.NativeMethodAccessorImpl.invoke( java.base@9-internal /NativeMethodAccessorImpl.java:62) at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke( java.base@9-internal /DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke( java.base@9-internal /Method.java:531) at com.sun.javatest.regtest.agent.MainActionHelper$SameVMRunnable.run(MainActionHelper.java:226) at java.lang.Thread.run( java.base@9-internal /Thread.java:843) JavaTest Message: Test threw exception: java.lang.NullPointerException JavaTest Message: shutting down test TEST RESULT: Failed. Execution failed: `main' threw exception: java.lang.NullPointerException
            Hide
            sgehwolf Severin Gehwolf added a comment -
            Thanks. I'll see if I can find a solaris x64 box and will try to reproduce.
            Show
            sgehwolf Severin Gehwolf added a comment - Thanks. I'll see if I can find a solaris x64 box and will try to reproduce.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            Status update: I was able to reproduce this locally.
            Show
            sgehwolf Severin Gehwolf added a comment - Status update: I was able to reproduce this locally.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            OomDebugTest also fails on this system. One failure is the result of a crash in OomDebugTestTarget:

            DEBUG: Done invoking method via debugger.
            # To suppress the following error report, specify this argument
            # after -XX: or in .hotspotrc: SuppressErrorAt=/jniHandles.hpp:197
            #
            # A fatal error has been detected by the Java Runtime Environment:
            #
            # Internal Error (/export/home/sgehwolf/Documents/openjdk9-hs/hotspot/src/share/vm/runtime/jniHandles.hpp:197), pid=20440, tid=2
            # assert(handle != 0L) failed: JNI handle should not be null
            #
            # JRE version: OpenJDK Runtime Environment (9.0) (fastdebug build 9-internal+0-2016-05-17-185224.sgehwolf.openjdk9-hs)
            # Java VM: OpenJDK 64-Bit Server VM (fastdebug 9-internal+0-2016-05-17-185224.sgehwolf.openjdk9-hs, mixed mode, tiered, compressed oops, g1 gc, solaris-amd64)
            # Core dump will be written. Default location: /export/home/sgehwolf/Documents/openjdk9-hs/JTwork/scratch/core or core.20440
            #
            # An error report file with more information is saved as:
            # /export/home/sgehwolf/Documents/openjdk9-hs/JTwork/scratch/hs_err_pid20440.log
            #
            # If you would like to submit a bug report, please visit:
            # http://bugreport.java.com/bugreport/crash.jsp
            #


            The corresponding stack trace from hs_err.log is:
            V [libjvm.so+0x17b6b94] oop JNIHandles::resolve_non_null(_jobject*)+0x54
            V [libjvm.so+0x1d8e766] instanceOop alloc_object(_jclass*,Thread*)+0x26
            V [libjvm.so+0x1d8f1d2] jni_NewObjectA+0x252
            C [libjdwp.so+0x3a130] invokeConstructor+0x70
            C [libjdwp.so+0x3b3e3] invoker_doInvoke+0x113
            C [libjdwp.so+0x32250] reportEvents+0x130
            C [libjdwp.so+0x32804] event_callback+0x324
            C [libjdwp.so+0x32e75] cbBreakpoint+0x145
            V [libjvm.so+0x1f8c026] void JvmtiExport::post_raw_breakpoint(JavaThread*,Method*,unsigned char*)+0x996
            V [libjvm.so+0x1d2aa8b] void InterpreterRuntime::_breakpoint(JavaThread*,Method*,unsigned char*)+0x19b
            j OomDebugTestTarget.entry()V+0
            j OomDebugTestTarget.main([Ljava/lang/String;)V+15
            Show
            sgehwolf Severin Gehwolf added a comment - OomDebugTest also fails on this system. One failure is the result of a crash in OomDebugTestTarget: DEBUG: Done invoking method via debugger. # To suppress the following error report, specify this argument # after -XX: or in .hotspotrc: SuppressErrorAt=/jniHandles.hpp:197 # # A fatal error has been detected by the Java Runtime Environment: # # Internal Error (/export/home/sgehwolf/Documents/openjdk9-hs/hotspot/src/share/vm/runtime/jniHandles.hpp:197), pid=20440, tid=2 # assert(handle != 0L) failed: JNI handle should not be null # # JRE version: OpenJDK Runtime Environment (9.0) (fastdebug build 9-internal+0-2016-05-17-185224.sgehwolf.openjdk9-hs) # Java VM: OpenJDK 64-Bit Server VM (fastdebug 9-internal+0-2016-05-17-185224.sgehwolf.openjdk9-hs, mixed mode, tiered, compressed oops, g1 gc, solaris-amd64) # Core dump will be written. Default location: /export/home/sgehwolf/Documents/openjdk9-hs/JTwork/scratch/core or core.20440 # # An error report file with more information is saved as: # /export/home/sgehwolf/Documents/openjdk9-hs/JTwork/scratch/hs_err_pid20440.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # The corresponding stack trace from hs_err.log is: V [libjvm.so+0x17b6b94] oop JNIHandles::resolve_non_null(_jobject*)+0x54 V [libjvm.so+0x1d8e766] instanceOop alloc_object(_jclass*,Thread*)+0x26 V [libjvm.so+0x1d8f1d2] jni_NewObjectA+0x252 C [libjdwp.so+0x3a130] invokeConstructor+0x70 C [libjdwp.so+0x3b3e3] invoker_doInvoke+0x113 C [libjdwp.so+0x32250] reportEvents+0x130 C [libjdwp.so+0x32804] event_callback+0x324 C [libjdwp.so+0x32e75] cbBreakpoint+0x145 V [libjvm.so+0x1f8c026] void JvmtiExport::post_raw_breakpoint(JavaThread*,Method*,unsigned char*)+0x996 V [libjvm.so+0x1d2aa8b] void InterpreterRuntime::_breakpoint(JavaThread*,Method*,unsigned char*)+0x19b j OomDebugTestTarget.entry()V+0 j OomDebugTestTarget.main([Ljava/lang/String;)V+15
            Hide
            dholmes David Holmes added a comment -
            Severin: are you still actively working on this issue? As we approach Rampdown Phase 1 (RDP1) and head towards Zero-bug-bounce (ZBB) we need to track all open issues. Thanks.
            Show
            dholmes David Holmes added a comment - Severin: are you still actively working on this issue? As we approach Rampdown Phase 1 (RDP1) and head towards Zero-bug-bounce (ZBB) we need to track all open issues. Thanks.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            @David: It's been on the back-burner for a while, but I'll work on this again this or next week. I don't have an ETA for proper resolution yet, though.
            Show
            sgehwolf Severin Gehwolf added a comment - @David: It's been on the back-burner for a while, but I'll work on this again this or next week. I don't have an ETA for proper resolution yet, though.
            Hide
            kbarrett Kim Barrett added a comment - - edited
            With JDK-8156500 fixed, the proposed OomDebugTest now runs quite reliably for me. However, while testing that change + the proposed webrev.02 I saw frequent / consistent failures on SPARC for ./test/com/sun/jdi/InvokeTest.java, which weren't reproducible with only the fix for JDK-8156500.
            Show
            kbarrett Kim Barrett added a comment - - edited With JDK-8156500 fixed, the proposed OomDebugTest now runs quite reliably for me. However, while testing that change + the proposed webrev.02 I saw frequent / consistent failures on SPARC for ./test/com/sun/jdi/InvokeTest.java, which weren't reproducible with only the fix for JDK-8156500 .
            Hide
            sgehwolf Severin Gehwolf added a comment -
            @Kim: Any details about the failures you were getting on SPARC. I don't have a SPARC machine available myself.
            Show
            sgehwolf Severin Gehwolf added a comment - @Kim: Any details about the failures you were getting on SPARC. I don't have a SPARC machine available myself.
            Hide
            kbarrett Kim Barrett added a comment -
            No, sorry. I hadn't investigated further, since I was focusing on JDK-8156500. I'll try to reproduce the problem, but it might take a few days.
            Show
            kbarrett Kim Barrett added a comment - No, sorry. I hadn't investigated further, since I was focusing on JDK-8156500 . I'll try to reproduce the problem, but it might take a few days.
            Hide
            kbarrett Kim Barrett added a comment - - edited
            Tried and failed to reproduce, so I went back through old test reports. Looks like I mis-remembered, and it was only once. And looking more closely, it turned out to be an ephemeral test reporting artifact. So not a problem after all. Sorry for the noise.
            Show
            kbarrett Kim Barrett added a comment - - edited Tried and failed to reproduce, so I went back through old test reports. Looks like I mis-remembered, and it was only once. And looking more closely, it turned out to be an ephemeral test reporting artifact. So not a problem after all. Sorry for the noise.
            Hide
            sgehwolf Severin Gehwolf added a comment -
            Latest webrev which has the NPE problem on solaris x86_64 fixed. Turns out, getting OomDebugTest working reliably on solaris also fixes the InvokeTest problem. I need to do some more testing, but it's looking good so far:
            http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8153711/webrev.03/
            Show
            sgehwolf Severin Gehwolf added a comment - Latest webrev which has the NPE problem on solaris x86_64 fixed. Turns out, getting OomDebugTest working reliably on solaris also fixes the InvokeTest problem. I need to do some more testing, but it's looking good so far: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8153711/webrev.03/
            Hide
            sgehwolf Severin Gehwolf added a comment - - edited
            The NPE problem in InvokeTest and the triggered assertion in OomDebugTest is caused by clearing the request's instance and clazz members *after* the IO operation. Like every other failures I've seen for this bug it's not reliably reproducible. Especially not when debugging this itself. It maybe related to the sequence of monitor_exit->perform IO->monitor_enter->toss references. Perhaps there is a schedule where the response has been sent back, the next invoke starts for the same app thread and it is just far enough along so that the tossing of the references becomes observable by the next request. I really don't know.

            However, testing showed that moving this up prevents this assertion from being triggered. Thus, I'm now clearing global references - the ones we can clear before sending back the response to the client - in the same block while still holding the invoker and event handler locks as the rest of the operations being done in completeInvokeRequest. Once the response has been sent to the client there are still two global references to clear: The one for the return value and a possible exception which might have occurred. Since they are required for sending the response we do this after it's been sent.
            Show
            sgehwolf Severin Gehwolf added a comment - - edited The NPE problem in InvokeTest and the triggered assertion in OomDebugTest is caused by clearing the request's instance and clazz members *after* the IO operation. Like every other failures I've seen for this bug it's not reliably reproducible. Especially not when debugging this itself. It maybe related to the sequence of monitor_exit->perform IO->monitor_enter->toss references. Perhaps there is a schedule where the response has been sent back, the next invoke starts for the same app thread and it is just far enough along so that the tossing of the references becomes observable by the next request. I really don't know. However, testing showed that moving this up prevents this assertion from being triggered. Thus, I'm now clearing global references - the ones we can clear before sending back the response to the client - in the same block while still holding the invoker and event handler locks as the rest of the operations being done in completeInvokeRequest. Once the response has been sent to the client there are still two global references to clear: The one for the return value and a possible exception which might have occurred. Since they are required for sending the response we do this after it's been sent.
            Show
            sgehwolf Severin Gehwolf added a comment - Posted for review: http://mail.openjdk.java.net/pipermail/serviceability-dev/2016-September/020407.html
            Hide
            hgupdate HG Updates added a comment -
            URL: http://hg.openjdk.java.net/jdk9/hs/jdk/rev/4c843eb35b8a
            User: dsamersoff
            Date: 2016-09-15 11:09:47 +0000
            Show
            hgupdate HG Updates added a comment - URL: http://hg.openjdk.java.net/jdk9/hs/jdk/rev/4c843eb35b8a User: dsamersoff Date: 2016-09-15 11:09:47 +0000
            Hide
            hgupdate HG Updates added a comment -
            URL: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/4c843eb35b8a
            User: lana
            Date: 2016-10-05 20:01:48 +0000
            Show
            hgupdate HG Updates added a comment - URL: http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/4c843eb35b8a User: lana Date: 2016-10-05 20:01:48 +0000

              People

              • Assignee:
                sgehwolf Severin Gehwolf
                Reporter:
                sspitsyn Serguei Spitsyn
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: