Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8134396

Check upper limit of flags setting numbers of stack protection pages.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P4
    • Resolution: Duplicate
    • Affects Version/s: 9
    • Fix Version/s: 9
    • Component/s: hotspot
    • Labels:
      None
    • Subcomponent:
    • Introduced In Version:
      9

      Description

      TestOptionsWithRanges shows that illegal values for red and yellow pages are possible.

      In JavaThread::create_stack_guard_pages(),
      setting StackRedPages=92233720368547753 and StackYellowPages=1 yields

      len = (92233720368547753 + 1) * os::vm_page_size()
          = 0x8000000000000000 * os::vm_page_size()
          = 0

      Thus not protecting any pages of the stack.

      The check in os::init_2() succeeds, as there the Shadow pages are considered, too, so that the overflowed value is > 0.

      On linux, the VM stops with the misleading message
      "OpenJDK 64-Bit Server VM warning: Attempt to allocate stack guard pages failed."
      On aix, we get assertions and SIGSEGVs.

        Attachments

          Activity

            People

            • Assignee:
              goetz Goetz Lindenmaier
              Reporter:
              goetz Goetz Lindenmaier
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: