|Issue||Fix Version||Assignee||Priority||Status||Resolution||Resolved In Build|
RHEL 5 & 6 are affected, and earlier Ubuntu releases 10.04 (LTS), 11.04, and 12.04 have also taken in the patch. Whilst earlier Exec Shield patches had a switch, current incantations have no work-around, no way to disable.
Details of the problem see RH bug filed: https://bugzilla.redhat.com/show_bug.cgi?id=996149
To summarize: to emulate NX feature on X86_32 code segment is used to limit execution to the highest executable VA. There is a tiny race on SMP MM invalidation code which can cause the lazy CS update code in trap handling to think a general protection fault wasn't cause by itself. This results in sending the JVM a useless SIGSEGV with si_code:SI_KERNEL, results in JVM signal handling forcing a dump.
The suggested work around (limited to 32 bit Linux): is to enable execution (PROT_EXEC) on a high address and execute some code.
The memory layout on these distros is similar: TASK_LIMIT set to 3G (0xC0000000), and the main stack typically grows down from around here, so:
1) using a page in main stack at JVM start up, test for said layout (don't bother if its not what we expect, i.e. exotic Linux config/distro)
2) enable execution (mprotect)
3) Place a single "ret" instruction and call it.
This will cause CS to be updated to a high enough address to basically disable any further updates (due to code generation), effectively disabling the feature (which is already of questionable value, NX is the way to go).