Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-8234483

Multiple redefinitions of a class breaks reflection

    Details

    • Type: Bug
    • Status: New
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: 8u231
    • Fix Version/s: None
    • Component/s: tools
    • Labels:
    • Subcomponent:
    • CPU:
      x86
    • OS:
      linux

      Description

      ADDITIONAL SYSTEM INFORMATION :
      Intel NUC 5i7RYH, Fedora 31, OpenJDK Runtime Environment (build 1.8.0_232-b09)

      A DESCRIPTION OF THE PROBLEM :
      This issue is important because incorrect mathematical calculations are performed, but the Java VM does not generate any exceptions for the unsuspecting software engineer.

      In code below, a defect occurs with repeated calls to the Java compiler (to compile dynamic code at runtime) and reflections (to invoke the dynamic code). At first, all calls perform the correct calculation, but after 1000-2000 invocations, the calculations start to become incorrect.
       
      A possible related software defects is JDK-2174826, but this has already been resolved.

      The code below has been reduce from the original code baseline at https://github.com/peytonc. Initial documentation started at https://stackoverflow.com/questions/58927052/is-java-vm-defective-or-why-does-repeated-use-of-compilationtask-and-reflections.

      REGRESSION : Last worked in version 13.0.1

      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      Install the latest version of JDK 8 (e.g. JDK 1.8.0_232-b09). The installation must include the javac executable. Execute the code below for approximately 1000-2000 iterations.

      EXPECTED VERSUS ACTUAL BEHAVIOR :
      EXPECTED -
      The code below is only suppose to write the console output "INFO: Iteration # ..." forever. This is because the reflections invocation had successfully incremented a value by one. A specific examples is as follows:

      INFO: Iteration #0: (c0) (i0) (c1) (i1) (c2) (i2) (c4) (i4) (c3) (i3) (c5) (i5) (c6) (i6) (c7) (i7) (c8) (i8) (c9) (i9)
      ACTUAL -
      The code below will write "ERROR ..." to the console when it defects occurs. This will occur after approximately 1000-2000 iterations (10 minutes). This is because the reflections invocation did NOT correctly incremented a value by one. A specific examples is as follows:

      ERROR: On iteration #914 id #4, actualResultsFromProgram != expectedResultFromProgram, 4!=5

      ---------- BEGIN SOURCE ----------
      package minijava;

      import java.io.IOException;
      import java.util.ArrayList;
      import java.util.Arrays;
      import java.util.List;
      import java.util.Locale;
      import java.util.concurrent.ExecutorService;
      import java.util.concurrent.Executors;
      import java.util.concurrent.TimeUnit;

      import javax.tools.DiagnosticCollector;
      import javax.tools.JavaCompiler;
      import javax.tools.JavaFileObject;
      import javax.tools.StandardJavaFileManager;
      import javax.tools.ToolProvider;
      import javax.tools.JavaCompiler.CompilationTask;

      public class Programs {
          public int species = 1;
          private static final JavaCompiler JAVA_COMPILER = ToolProvider.getSystemJavaCompiler();
          private static final int MAX_POPULATION = 10;
          private List<Program> listProgram = new ArrayList<Program>(MAX_POPULATION);
          private static final int AVAILABLE_PROCESSORS = Runtime.getRuntime().availableProcessors();
          private long iteration = 0;

          // The compute function will increment the first array value by one
          private static final String sourceCodeTemplate =
                  "package species0.id0; \n" +
                  "import java.util.ArrayList; \n" +
                  "public class GeneticProgram { \n" +
                  " public static void compute(ArrayList<Long> values00) { \n" +
                  " long value = values00.get(0); \n" +
                  " System.out.print(\" (c\" + value + \") \"); \n" +
                  " values00.set(0, value + 1); \n" +
                  " } \n" +
                  "} \n";

          public Programs() {
              System.out.println(sourceCodeTemplate);

              int errorCode = 0;
              while(errorCode == 0) {
                  System.out.print("\nINFO: Iteration #" + iteration + ":");

                  errorCode = createPrograms();
                  if(errorCode == 0) {
                      compilePrograms();
                  }
                  if(errorCode == 0) {
                      executePrograms();
                  }
                  iteration++;
              }
          }

          public int createPrograms() {
              listProgram.clear();
              for(int index=0; index<MAX_POPULATION; index++) {
                  String simulatedRandomSourceCode = replacePackage(sourceCodeTemplate, species, index);
                  Program program = new Program(simulatedRandomSourceCode, species, index);
                  program.vectors.add(new Long(index)); // this number will be incremented by 1 upon execution of program
                  listProgram.add(program);
              }
              return 0;
          }

          public int compilePrograms() {
              DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();
              for(Program program : listProgram) {
                  try (StandardJavaFileManager standardJavaFileManager = JAVA_COMPILER.getStandardFileManager(diagnostics, Locale.ENGLISH, null)) {
                      Iterable<Program> javaFileObject = Arrays.asList(program);
                      ProgramClassSimpleJavaFileObject programClassSimpleJavaFileObject = null;
                      try {
                          programClassSimpleJavaFileObject = new ProgramClassSimpleJavaFileObject(Program.PACKAGE_SPECIES + species + "." + Program.PACKAGE_ID + program.ID + "." + Program.PROGRAM_CLASS);
                      } catch (Exception e) {
                          e.printStackTrace();
                          return -1;
                      }
                      ProgramForwardingJavaFileManager programForwardingJavaFileManager = new ProgramForwardingJavaFileManager(standardJavaFileManager, programClassSimpleJavaFileObject, program.programClassLoader);
                      CompilationTask compilerTask = JAVA_COMPILER.getTask(null, programForwardingJavaFileManager, diagnostics, null, null, javaFileObject);
                      if (!compilerTask.call()) {
                          System.out.println("\nERROR: compilePrograms compilerTask.call()");
                          return -1;
                      }
                  } catch (IOException e) {
                      e.printStackTrace();
                      return -1;
                  }
              }
              return 0;
          }

          public int executePrograms() {
              List<CallableMiniJava> listCallable = new ArrayList<CallableMiniJava>(MAX_POPULATION);
              ExecutorService executorService = Executors.newFixedThreadPool(AVAILABLE_PROCESSORS);

              try {
                  for(Program program : listProgram) {
                      listCallable.add(new CallableMiniJava(program));
                  }
                  for(CallableMiniJava callableMiniJava : listCallable) {
                      executorService.execute(callableMiniJava);
                  }
                  executorService.shutdown();
                  if(!executorService.awaitTermination(1000, TimeUnit.MILLISECONDS)) {
                      executorService.shutdownNow();
                      int milliseconds = 1000;
                      while(!executorService.awaitTermination(1000, TimeUnit.MILLISECONDS)) {
                          milliseconds += 1000;
                          System.out.println("\nINFO: Runaway program for " + milliseconds + " milliseconds");
                      }
                  }

                  for (Program program : listProgram) {
                      long actualResultsFromProgram = program.vectors.get(0);
                      long expectedResultFromProgram = program.ID + 1;
                      if(actualResultsFromProgram != expectedResultFromProgram) {
                          System.out.println("\nERROR: On iteration #" + iteration + " id #" + program.ID + ", actualResultsFromProgram != expectedResultFromProgram, " + actualResultsFromProgram + "!=" + expectedResultFromProgram);
                      }
                  }
              } catch (InterruptedException e) {
                  e.printStackTrace();
                  return -1;
              }
              return 0;
          }

          public static String replacePackage(String source, int species, int packageNumber) {
              return source.replaceFirst("package species[0-9][^;]*;", "package " + Program.PACKAGE_SPECIES + species + "." + Program.PACKAGE_ID + packageNumber + ";");
          }

          public static void main(String[] args) {
              Programs programs = new Programs();
          }
      }


      package minijava;

      import java.net.URI;
      import java.util.ArrayList;

      import javax.tools.SimpleJavaFileObject;

      public class Program extends SimpleJavaFileObject {
      public static final String PROGRAM_CLASS = new String("GeneticProgram");
      public static final String PACKAGE_SPECIES = new String("species");
      public static final String PACKAGE_ID = new String("id");
      public String source;
      public ArrayList<Long> vectors;
      public int species;
      public int ID;
      public ProgramClassLoader programClassLoader = null;

      Program(String source, int species, int ID) {
      super(URI.create("string:///" + PACKAGE_SPECIES + species + '/' + PACKAGE_ID + ID + '/' + PROGRAM_CLASS + Kind.SOURCE.extension), Kind.SOURCE);
      this.source = new String(source);
      this.species = species;
      this.ID = ID;
      vectors = new ArrayList<Long>(1);
      programClassLoader = new ProgramClassLoader(ClassLoader.getSystemClassLoader());
      }

      @Override
      public CharSequence getCharContent(boolean ignoreEncodingErrors) {
      return source;
      }
      }

      package minijava;

      import java.lang.reflect.Method;
      import java.util.ArrayList;

      public class CallableMiniJava implements Runnable {
      private Program program = null;
      private Class<?> cls = null;
      private Method method = null;

      public CallableMiniJava(Program program) {
      if(program.vectors != null) {
      this.program = program;
      try {
      cls = program.programClassLoader.loadClass(Program.PACKAGE_SPECIES + program.species + "." + Program.PACKAGE_ID + program.ID + "." + Program.PROGRAM_CLASS);
      } catch (ClassNotFoundException e1) {
      e1.printStackTrace();
      }
      try {
      method = cls.getMethod("compute", ArrayList.class);
      } catch (NoSuchMethodException e1) {
      e1.printStackTrace();
      } catch (SecurityException e1) {
      e1.printStackTrace();
      }
      }
      }

      @Override
      public void run() {
      try {
      method.invoke(null, program.vectors);
      System.out.print(" (i" + program.ID + ") ");
      } catch(Exception e) {
      e.printStackTrace();
      }
      }
      }


      package minijava;

      import java.util.HashMap;
      import java.util.Map;

      public class ProgramClassLoader extends ClassLoader {

          public Map<String, ProgramClassSimpleJavaFileObject> mapProgramClass = new HashMap<>();

          public ProgramClassLoader(ClassLoader classLoader) {
              super(classLoader);
          }

          @Override
          protected Class<?> findClass(String className) throws ClassNotFoundException {
              ProgramClassSimpleJavaFileObject programClassSimpleJavaFileObject = mapProgramClass.get(className);
              if(programClassSimpleJavaFileObject != null) {
               byte[] byteCode = programClassSimpleJavaFileObject.byteArrayOutputStream.toByteArray();
                  return defineClass(className, byteCode, 0, byteCode.length);
              } else {
                  return super.findClass(className);
              }
          }
      }


      package minijava;

      import javax.tools.SimpleJavaFileObject;
      import java.io.ByteArrayOutputStream;
      import java.io.IOException;
      import java.io.OutputStream;
      import java.net.URI;

      public class ProgramClassSimpleJavaFileObject extends SimpleJavaFileObject {
      public ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();

      public ProgramClassSimpleJavaFileObject(String className) throws Exception {
      super(new URI(className), Kind.CLASS);
      }

      @Override
      public OutputStream openOutputStream() throws IOException {
      return byteArrayOutputStream;
      }
      }


      package minijava;

      import javax.tools.FileObject;
      import javax.tools.ForwardingJavaFileManager;
      import javax.tools.JavaFileManager;
      import javax.tools.JavaFileObject;
      import javax.tools.JavaFileObject.Kind;

      import java.io.IOException;

      public class ProgramForwardingJavaFileManager extends ForwardingJavaFileManager<JavaFileManager> {
      private ProgramClassLoader programClassLoader;
          private ProgramClassSimpleJavaFileObject programClassSimpleJavaFileObject;

      protected ProgramForwardingJavaFileManager(JavaFileManager javaFileManager, ProgramClassSimpleJavaFileObject programClassSimpleJavaFileObject, ProgramClassLoader programClassLoader) {
           super(javaFileManager);
           this.programClassSimpleJavaFileObject = programClassSimpleJavaFileObject;
              this.programClassLoader = programClassLoader;
              this.programClassLoader.mapProgramClass.put(programClassSimpleJavaFileObject.getName(), programClassSimpleJavaFileObject);
      }

      @Override
      public JavaFileObject getJavaFileForOutput(JavaFileManager.Location location, String className, Kind kind, FileObject fileObject) throws IOException {
           return programClassSimpleJavaFileObject;
      }

      @Override
      public ClassLoader getClassLoader(Location location) {
           return programClassLoader;
      }
      }

      ---------- END SOURCE ----------

      CUSTOMER SUBMITTED WORKAROUND :
      The defect does not occur in Java 13.0.1. This was validated by a 24 hour run on the Intel/Linux computer stated above in addition to Windows 10/AMD (16 core).

      FREQUENCY : always


        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              webbuggrp Webbug Group
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: