Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-7124710

interleaved RedefineClasses() and RetransformClasses() calls may have a problem

    Details

    • Subcomponent:
    • CPU:
      generic
    • OS:
      generic

      Description

      Thomas Wuerthinger is working in the RedefineClasses and
      RetransformClasses area so I asked him to review the fix
      for the following bug:

          7122253 2/3 Instrumentation.retransformClasses() leaks class bytes

      During our e-mail discussions, Thomas raised some possible issues
      with my fix for 7122253 and I wanted to document those here. The
      next note contains part of the e-mail thread that I think captures
      the potential issue.
      One other thing has occurred to me as I'm filing this bug.
      Prior to the fix to 7122253, we have a memory leak. I think
      we also have a problem where a RetransformClasses() call
      does not cause the proper "initial class bytes" to be passed
      to the ClassFileLoadHook event handler. I think this particular
      bug is independent of any RedefineClasses() calls and would
      occur on the second (and subsequent) RetransformClasses() calls.

      Of course, we would need a test to prove this issue and to
      verify that we don't have that problem after the fix for
      7122253 is applied.
      Thomas,

      I've finished running through scenarios. Below is what I have so far.

      I agree that the cached class bytes should have been "reset" when a
      RedefineClasses() goes through the system, but there's a problem or
      a risk that we could end up caching class bytes that have not been
      validated. We don't want to do that.

      Do you concur that this bad/unexpected cache interaction only
      happens when RedefineClasses() and RetransformClasses() are mixed?

      If that's the case, then I think we can go with my current fix for
      the memory leak and file a new bug.

      Please let me know what you think.

      Dan



      JVM/TI RetransformClasses() spec extract
      ----------------------------------------

      > When classes are initially loaded or when they are redefined, the
      > initial class file bytes can be transformed with the ClassFileLoadHook
      > event. This function reruns the transformation process (whether or not
      > a transformation has previously occurred). This retransformation follows
      > these steps:
      >
      > * starting from the initial class file bytes
      >
      > * for each retransformation incapable agent which received a
      > ClassFileLoadHook event during the previous load or redefine, the
      > bytes it returned (via the new_class_data parameter) are reused
      > as the output of the transformation; note that this is equivalent
      > to reapplying the previous transformation, unaltered. except that
      > the ClassFileLoadHook event is not sent to these agents
      >
      > * for each retransformation capable agent, the ClassFileLoadHook
      > event is sent, allowing a new transformation to be applied
      >
      > * the transformed class file bytes are installed as the new definition
      > of the class
      >
      > See the ClassFileLoadHook event for more details.
      >
      > The initial class file bytes represent the bytes passed to
      > ClassLoader.defineClass or RedefineClasses (before any transformations
      > were applied), however they may not exactly match them.


      Scenarios
      ---------

      1) Redefine first, Retransform second, Retransform...

          When Redefine happens first, class bytes are not cached on the
          instanceKlass. When the Retransform happens, it caches the class
          bytes from the previous Redefine on the instanceKlass.

          Subsequent Retransform calls will also start with the class bytes
          from the previous Redefine.

          I think this scenario works as expected.

      2) Redefine first, Retransform second, Redefine third, Redefine ...

          Same as scenario #1 until the third operation (Redefine instead of
          Retransform). The second Redefine call replaces the class bytes
          being executed, but it does not update the cached class bytes on
          the instanceKlass.

          Also note that the second (and subsequent) Redefine calls result in a
          CFLH event being posted to the Retransform capable agent. That agent
          will be given the class bytes from the Redefine call to transform.

          Subsequent Redefine operations will work as expected. Each Redefine
          will apply 'new' class bytes as transformed by the Retransform agent.

          I think this scenario works as expected.

      3) Redefine first, Retransform second, Redefine third, Retransform fourth

          Same as scenario #2 until the fourth operation (Retransform instead of
          Redefine). Again the second Redefine call replaces the class bytes
          being executed, but it does not update the cached class bytes on
          the instanceKlass. This means that second Retransform call starts
          with the class bytes from the first Redefine operation instead of
          the class bytes from the second Redefine operation.

          Also note that the second (and subsequent) Redefine calls result in a
          CFLH event being posted to the Retransform capable agent. That agent
          will be given the class bytes from the Redefine call to transform.

          Subsequent Redefine operations will work as expected; Each Redefine
          will apply 'new' class bytes as transformed by the Retransform
          agent. However, subsequent Retransform operations will always start
          with the class bytes from the first Redefine operation instead of
          the class bytes from the previous Redefine operation.

          I think this scenario shows a problem.

          When the second Redefine operation happens, the class bytes cache
          needs to be updated with the class bytes from the Redefine operation
          before they are modified by the Transform capable agent's CFLH
          handler. Note: one problem is that these class bytes haven't been
          verified yet.

      3) Retransform first, Redefine second, Retransform third, Retransform ...

          When Retransform happens first, it caches the class bytes from
          before the transform operation on the instanceKlass. The Redefine
          call replaces the class bytes being executed, but it does not update
          the cached class bytes on the instanceKlass.

          Also note that the Redefine call results in a CFLH event being posted
          to the Retransform capable agent. That agent will be given the class
          bytes from the Redefine call to transform.

          Subsequent Retransform operations will always start with the class
          bytes from before the first Retransform operation instead of the
          class bytes from the first Redefine operation.

          I think this scenario shows a problem.

          When the second Redefine operation happens, the class bytes cache
          needs to be updated with the class bytes from the Redefine operation
          before they are modified by the Transform capable agent's CFLH
          handler. Note: one problem is that these class bytes haven't been
          verified yet.


      On 12/21/11 2:25 PM, Thomas Wuerthinger wrote:
      > Dan,
      >
      > Thanks for keeping me informed on that.
      >
      > I had a quick look over the code, and I'm not sure if you may keep the cached class file bytes in the following scenario:
      > - Class C is loaded.
      > - C is retransformed to C1 (thus saving cached class bytes)
      > - C1 is redefined to C2 (but without any "transformation" on it, so there are no new cached class bytes)
      > - C2 is redefined to C3: Here will see the transformer the class bytes from C1 (because they are the last ones cached), but it should see the class bytes from C2
      > If you think that this could be problem, I can try to create a test case for it.
      >
      > In general, I'm wondering how big the performance gain from caching the class bytes is. I'd guess that the class file reconstitutor works quite fast.
      >
      > - thomas

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                sspitsyn Serguei Spitsyn
                Reporter:
                dcubed Daniel Daugherty
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Imported:
                  Indexed: