Uploaded image for project: 'JDK'
  1. JDK
  2. JDK-4705093

Use /dev/urandom rather than /dev/random if it exists.

    Details

    • Subcomponent:
    • Resolved In Build:
      tiger
    • CPU:
      x86, sparc
    • OS:
      linux, solaris_8

      Description



      Name: gm110360 Date: 06/20/2002


      FULL PRODUCT VERSION :
      > java -version
      java version "1.4.0"
      Java(TM) 2 Runtime Environment, Standard Edition (build 1.4.0-b92)
      Java HotSpot(TM) Client VM (build 1.4.0-b92, mixed mode)


      FULL OPERATING SYSTEM VERSION :
      [fjo@maren jObj]$ uname -r -s -p
      Linux 2.4.9-34 unknown

      On intel pIII

      A DESCRIPTION OF THE PROBLEM :
      I'm using SSL to access a webbserver and https:// urls,
      which is now available by default and use the
      com.sun.net.ssl package I believe.

      The problem is that the SecureRandom number generator which
      is used to create the connection is seeded from the blocking
      random device /dev/random. This causes an application using
      SSL urls to hang anything from seconds to several minutes
      depending on how much randomness is available on the first
      access of a https url.

      I believe all linuxes has /dev/urandom today, and hopefully
      so will Solaris in the next versions (there is a patch for
      it). If SecureRandom was seeded from /dev/urandom instead
      the performance would increase incredibly.

      So basically, please use /dev/urandom prior to /dev/random
      if it exists. There are too many asking for randomness
      nowadays to use /dev/random. The increased security of using
      /dev/random for seeding must be rather negligible and right
      now the performance of setting up a connection is so bad
      that I may not be able to use SSL at all, which doesn't
      really improve security.


      STEPS TO FOLLOW TO REPRODUCE THE PROBLEM :
      1.Use any api that accesses an url and access a https one.
      It is a documented "feature", I simply ask it to be resolved.



      REPRODUCIBILITY :
      This bug can be reproduced always.
      (Review ID: 153694)
      ======================================================================

        Issue Links

          Activity

          Hide
          andreas Andreas Sterbenz added a comment -
          BT2:WORK AROUND

          Edit the jre/lib/security/java.security file to point to /dev/urandom or any other URL or set the System property java.security.egd when invoking Java, e.g. "java -Djava.security.egd=file:/dev/urandom com.foo.MyApp"

          ###@###.### 2002-07-24
          Show
          andreas Andreas Sterbenz added a comment - BT2:WORK AROUND Edit the jre/lib/security/java.security file to point to /dev/urandom or any other URL or set the System property java.security.egd when invoking Java, e.g. "java -Djava.security.egd=file:/dev/urandom com.foo.MyApp" ###@###.### 2002-07-24
          Hide
          andreas Andreas Sterbenz added a comment -
          BT2:EVALUATION

          We read 20 bytes from /dev/random to seed our internal PRNG, which is used to generate an arbitrary amount of pseudo random data. Therefore, it is important that we use a high entropy seed.

          I would not expect the kernel entropy pool to be drained when just 160 bits are read. Of course, each invocation of a Java application that uses SecureRandom will read 160 bits. If a new Java process is started e.g. for each HTTPS URL retrieved, the pool can eventually drain, but starting a new Java process for such small tasks is inefficient irrespective of SecureRandom.

          Note that you can always edit the jre/lib/security/java.security file to point to /dev/urandom or any other URL.

          ###@###.### 2002-06-20

          We are considering changing more aspects of the SecureRandom implementation in a future release, in which case it may make sense to use /dev/urandom.

          ###@###.### 2002-07-24
          Show
          andreas Andreas Sterbenz added a comment - BT2:EVALUATION We read 20 bytes from /dev/random to seed our internal PRNG, which is used to generate an arbitrary amount of pseudo random data. Therefore, it is important that we use a high entropy seed. I would not expect the kernel entropy pool to be drained when just 160 bits are read. Of course, each invocation of a Java application that uses SecureRandom will read 160 bits. If a new Java process is started e.g. for each HTTPS URL retrieved, the pool can eventually drain, but starting a new Java process for such small tasks is inefficient irrespective of SecureRandom. Note that you can always edit the jre/lib/security/java.security file to point to /dev/urandom or any other URL. ###@###.### 2002-06-20 We are considering changing more aspects of the SecureRandom implementation in a future release, in which case it may make sense to use /dev/urandom. ###@###.### 2002-07-24
          Hide
          defectconv Defect Conversion BT2 (Inactive) added a comment -
          BT2:CONVERTED DATA

          BugTraq+ Release Management Values

          COMMIT TO FIX:
          tiger

          FIXED IN:
          tiger

          INTEGRATED IN:
          tiger
          tiger-b20

          Show
          defectconv Defect Conversion BT2 (Inactive) added a comment - BT2:CONVERTED DATA BugTraq+ Release Management Values COMMIT TO FIX: tiger FIXED IN: tiger INTEGRATED IN: tiger tiger-b20
          Hide
          wetmore Bradford Wetmore added a comment - - edited
          BT2:EVALUATION

          I'm working on a cleanup of the SecureRandom implementations, and found the previous evaluation to be not very helpful.

          The fix for this bug was:

          Added a new algorithm in the Sun provider called NativePRNG.java on Solaris/Linux (no Windows), which calls into /dev/random and /dev/urandom depending on whether seed or nextBytes() are needed (respectively). It currently (Jan 2012) does not take into account java.security.egd/securerandom.source, except when starting up the Sun provider: if the value is "file:/dev/urandom", this provider is placed before SHA1PRNG. If not, then it goes in after SHA1PRNG.

          engineSetSeed(byte[] seed): First tries to to write to /dev/random. This file is typically owned by root, so in addition, it also (re)creates SHA1PRNG (seeding with /dev/urandom if needed), then with seed parameter. This SHA1PRNG is used by later nextBytes calls.

          engineGenerateSeed(int numbBytes): Does a direct read on /dev/random.

          engineNextBytes(byte[] bytes): Using the same code called by setSeed (getMixRandom), it (re)creates a SHA1PRNG (seeding from /dev/urandom if needed). It then reads from /dev/urandom and XOR's with data from the SHA1PRNG mixRandom.

          ------

          In SHA1PRNG, there is a SeedGenerator which does various things depending on the configuration.

          1. If java.security.egd or securerandom.source point to "file:/dev/random" or "file:/dev/urandom", we will use NativeSeedGenerator, which calls super() which calls SeedGenerator.URLSeedGenerator(/dev/random). (A nested class within SeedGenerator.) The only things that changed in this bug was that urandom will also trigger use of this code path.

          2. If those properties point to another URL that exists, we'll initialize SeedGenerator.URLSeedGenerator(url). This is why "file:///dev/urandom", "file:/./dev/random", etc. will work.

          3. If neither were successful, then we'll fall back to the ThreadedSeedGenerator, which does the measurement heuristic.

          Calls to SeedGenerator.URLSeedGenerator will then read from the file and provide bytes.

          ----

          Since we're here, what does SHA1PRNG do?

          engineSetSeed(byte[] seed): If there is existing state, it will put that state back into the SHA1 digest, add the seed to it, cdigest() and store new value to the state.

          engineGenerateSeed(int numBytes): Calls directly to SeedGenerator.generateSeed(b), which reads from from whatever the seed generator points to. Recall that if it's one of the two reserved strings, it will go to /dev/random.

          engineNextBytes(int num): If there has been no state assigned yet, it needs to call the SeedGenerator to get something. It does this by calling SeedGenerator.getSystemEntropy(), which creates a SHA1, then adds in a bunch of system dependent values (time, properties, hostname, tmpdir file names, memory amounts, etc).

          This SHA value is then used to create/seed a *DIFFERENT* SecureRandom implementation. This *DIFFERENT* SecureRandom impl is then additionally seeded again by a call to *THIS* instance's engineGenerateSeed(b) call, which goes to whatever URL/method was established in the logic above. So if an app calls "new SecureRandom().nextBytes()", it will get into the /dev/random here. If an app calls "new SecureRandom().setSeed(b).nextBytes()" it will short circuit the need to go initialize the seeder, and thus won't go off to /dev/random.

          Then it adds the existing state back to SHA1 digest, then digests it again, doing some manipulations between the old and new state, and then outputs the result.

          What a confusing mess.

          Show
          wetmore Bradford Wetmore added a comment - - edited BT2:EVALUATION I'm working on a cleanup of the SecureRandom implementations, and found the previous evaluation to be not very helpful. The fix for this bug was: Added a new algorithm in the Sun provider called NativePRNG.java on Solaris/Linux (no Windows), which calls into /dev/random and /dev/urandom depending on whether seed or nextBytes() are needed (respectively). It currently (Jan 2012) does not take into account java.security.egd/securerandom.source, except when starting up the Sun provider: if the value is "file:/dev/urandom", this provider is placed before SHA1PRNG. If not, then it goes in after SHA1PRNG. engineSetSeed(byte[] seed): First tries to to write to /dev/random. This file is typically owned by root, so in addition, it also (re)creates SHA1PRNG (seeding with /dev/urandom if needed), then with seed parameter. This SHA1PRNG is used by later nextBytes calls. engineGenerateSeed(int numbBytes): Does a direct read on /dev/random. engineNextBytes(byte[] bytes): Using the same code called by setSeed (getMixRandom), it (re)creates a SHA1PRNG (seeding from /dev/urandom if needed). It then reads from /dev/urandom and XOR's with data from the SHA1PRNG mixRandom. ------ In SHA1PRNG, there is a SeedGenerator which does various things depending on the configuration. 1. If java.security.egd or securerandom.source point to "file:/dev/random" or "file:/dev/urandom", we will use NativeSeedGenerator, which calls super() which calls SeedGenerator.URLSeedGenerator(/dev/random). (A nested class within SeedGenerator.) The only things that changed in this bug was that urandom will also trigger use of this code path. 2. If those properties point to another URL that exists, we'll initialize SeedGenerator.URLSeedGenerator(url). This is why " file:///dev/urandom ", "file:/./dev/random", etc. will work. 3. If neither were successful, then we'll fall back to the ThreadedSeedGenerator, which does the measurement heuristic. Calls to SeedGenerator.URLSeedGenerator will then read from the file and provide bytes. ---- Since we're here, what does SHA1PRNG do? engineSetSeed(byte[] seed): If there is existing state, it will put that state back into the SHA1 digest, add the seed to it, cdigest() and store new value to the state. engineGenerateSeed(int numBytes): Calls directly to SeedGenerator.generateSeed(b), which reads from from whatever the seed generator points to. Recall that if it's one of the two reserved strings, it will go to /dev/random. engineNextBytes(int num): If there has been no state assigned yet, it needs to call the SeedGenerator to get something. It does this by calling SeedGenerator.getSystemEntropy(), which creates a SHA1, then adds in a bunch of system dependent values (time, properties, hostname, tmpdir file names, memory amounts, etc). This SHA value is then used to create/seed a *DIFFERENT* SecureRandom implementation. This *DIFFERENT* SecureRandom impl is then additionally seeded again by a call to *THIS* instance's engineGenerateSeed(b) call, which goes to whatever URL/method was established in the logic above. So if an app calls "new SecureRandom().nextBytes()", it will get into the /dev/random here. If an app calls "new SecureRandom().setSeed(b).nextBytes()" it will short circuit the need to go initialize the seeder, and thus won't go off to /dev/random. Then it adds the existing state back to SHA1 digest, then digests it again, doing some manipulations between the old and new state, and then outputs the result. What a confusing mess.
          Hide
          wetmore Bradford Wetmore added a comment -
          BT2:EVALUATION

          Adding some additional discussion to explain what is going on:

          This is a long story that we hope to address in JDK 8.

          Because SHA1PRNG is a MessageDigest-based PRNG, it historically has always used /dev/random for initial seeding if seed data has not been provided by the application. Since all future values depend on the existing state of the MessageDigest, it's important to start with a strong initial seed.

          Changing that behavior was troubling to the original developer. So he did created a new SecureRandom impl called NativePRNG, which does respect the java.security.egd value.

          If you call:

          o new SecureRandom() on Linux and the default values are used, it will read from /dev/urandom and not block. (By default on Solaris, the PKCS11 SecureRandom is used, and also calls into /dev/urandom.)

          o SecureRandom.getInstance("SHA1PRNG") and do not specify a seed, *OR* new SecureRandom() but have specified an alternate java.security.egd besides "file:/dev/urandom", it will use the SHA1PRNG which calls into /dev/random and may potentially block.

          o SecureRandom.getInstance("NativePRNG"), it will depend on what java.security.egd is pointing to.

          Hope this helps a bit. I know, it's confusing as heck, and we hope to somehow make this clearer in JDK8.
          Show
          wetmore Bradford Wetmore added a comment - BT2:EVALUATION Adding some additional discussion to explain what is going on: This is a long story that we hope to address in JDK 8. Because SHA1PRNG is a MessageDigest-based PRNG, it historically has always used /dev/random for initial seeding if seed data has not been provided by the application. Since all future values depend on the existing state of the MessageDigest, it's important to start with a strong initial seed. Changing that behavior was troubling to the original developer. So he did created a new SecureRandom impl called NativePRNG, which does respect the java.security.egd value. If you call: o new SecureRandom() on Linux and the default values are used, it will read from /dev/urandom and not block. (By default on Solaris, the PKCS11 SecureRandom is used, and also calls into /dev/urandom.) o SecureRandom.getInstance("SHA1PRNG") and do not specify a seed, *OR* new SecureRandom() but have specified an alternate java.security.egd besides "file:/dev/urandom", it will use the SHA1PRNG which calls into /dev/random and may potentially block. o SecureRandom.getInstance("NativePRNG"), it will depend on what java.security.egd is pointing to. Hope this helps a bit. I know, it's confusing as heck, and we hope to somehow make this clearer in JDK8.

            People

            • Assignee:
              andreas Andreas Sterbenz
              Reporter:
              gmanwanisunw Girish Manwani (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:
                Imported:
                Indexed: