Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8181917

Refactor UL LogStreams to avoid using resource area

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P3
    • Resolution: Fixed
    • Affects Version/s: 9, 10
    • Fix Version/s: 10
    • Component/s: hotspot
    • Labels:
      None
    • Subcomponent:
    • Resolved In Build:
      b21

      Description

      LogStream, an outputStream child, is used to give us outputStream compatibility to write to UL. It assembles a log line and, once complete, sends it off to UL backend. It needs memory to hold the line in the mean time. Line size is not limited to avoid truncation, so this memory is managed dynamically. Currently this memory is allocated in resource area. Line memory is dynamically enlarged if necessary.

      The problem with that solution is that LogStream (as all outputStreams) are often handed down to sub print functions, which in turn span new Resourcemarks. if the line buffer reallocation happens in a sub print function under a new ResourceMark, code will assert (debug) or crash (ndebug).

      See:

      https://bugs.openjdk.java.net/browse/JDK-8181807
      https://bugs.openjdk.java.net/browse/JDK-8149557
      https://bugs.openjdk.java.net/browse/JDK-8167995

      At SAP we also hit the problem a number of times without opening bug reports for each occurence. So, this has been a problem.

      So far all workarounds have been to shuffle the ResourceMarks around, or disable them. But all these solutions are very fragile and do not solve the underlying problem, the fact that LogStream should not use resource area in the first place (arguably, no outputStream class should use resource area - see stringStream - but this bug issue concentrates on UL).

      The solution should be not to use resource memory for LogStream classes.

      Alternatives are:
      1) having a fixed sized char buffer member in the LogStream class, or a dynamically allocated buffer which does not enlarge. This is not a good solution and therefore was rejected in discussions with the UL authors, because we do not want to silently truncate long log lines.
      2) dynamically allocating the buffer memory on the C-Heap.

      With (2) there are some problems:

      Performance. malloc is slower than resource area allocation. Arguably this is not much of a problem since when we log, we ususally do IO anyway, and malloc only happens rarely when the LogStream line buffer is enlarged. Even so, it can be prevented in 90% of the cases by reverting to a small fixed char array in LogStream for small lines and only invoking malloc for longer log lines.

      The biggest issue is the fact that deallocation of the malloc'ed line buffer is difficult. LogStream is an outputStream which is an ResourceObj. LogStream instances are allocated on the fly via "LogImpl<>::xxxx_stream()" which returns an instance of LogStream which is allocated with "new LogStream(..)". This means we use ResourceObj::operator new(), which is an arena allocation and this means we cannot delete the LogStream instance, so the LogStream destructor is never called.

      There are a number of ways to solve this. The currently preferred way by the UL authors is to make LogStream a stack-allocatable object only. That way it will be automatically deleted when it goes out of scope.

      See the ongoing discussion: http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-June/027090.html




        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stuefe Thomas Stuefe
                Reporter:
                stuefe Thomas Stuefe
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: