FAQ
Content
John the Ripper FAQ. The latest version of this FAQ may be viewed online at: http://www.openwall.com/john/doc/FAQ.shtml Help!
I can't run John.
If you're not familiar with your OS, you should probably not be using John in the first place since John is a tool for system administrators. However, here are the answers to a few (not very) common questions to avoid having them asked over and over and for amusement. Q: When I type "john" (or "john passwd", etc.), it says "command not found" (or equivalent)?! A: The examples given in John the Ripper documentation assume that you know how to invoke newly-built programs from your shell. On Unix-like systems, it is typical to not have "." (the current directory) in your $PATH (the list of directories to search for programs). In that case, you need to type "./john" (dot, slash, and "john", without the quotes) to invoke the John binary executable located in the current directory. Q: ...but I am on a Unix-like system and I don't seem to readily have a John binary executable. A: Please follow the instructions in INSTALL. Q: When I double-click on "john.exe", a window flashes and disappears?! A: You're not supposed to click. You're supposed to run John from a command-line shell. On Windows, some of those shells would be cmd.exe, command.com, or bash (the latter is available with Cygwin). Other trivial matters. Q: How do I start John on my password file, use a specific cracking mode, see the passwords it cracked, etc? A: See README and EXAMPLES. :-) Q: How do I "unshadow"? A: See EXAMPLES on how to combine your passwd and shadow files, provided that you have root access to the target system. Q: Why doesn't John load my password file? It says "No password hashes loaded", "No password hashes loaded (see FAQ)", or "No password hashes left to crack (see FAQ)". A: Your password file might be shadowed. You need to get both /etc/passwd and the shadow file (typically /etc/shadow), and combine them into one file for use with John. Please refer to EXAMPLES. A: All of the password hashes found in the file (that are of the same type as the very first recognized hash in the file unless you're using the "--format=..." option) might be already cracked by previous invocations of John. (The message printed in that case has been changed to "No password hashes left to crack (see FAQ)" starting with version 1.7.7.) To display cracked passwords, use "john --show" on your password hash file(s). To force John to crack those same hashes again, remove the john.pot file. A: With PWDUMP-format files, John focuses on LM rather than NTLM hashes by default, and it might not load any hashes at all if there are no LM hashes to crack. To have JtR Pro or a build of JtR with the jumbo patch
focus on NTLM hashes instead, you need to pass the "--format=nt" option. A: The file you're trying to run John on might in fact not be a password file at all. A: Your command line syntax might be wrong, resulting in John trying to load a wrong file. A: Your password file format or hash type(s) might not be supported by John, or at least by the version and build of John that you're using. If you're positive that this is the case, you may want to check the contributed resources list on John the Ripper homepage for a suitable patch and, if unsuccessful with that, post a note to the mailing list (see CONTACT) including a sample password file line that John does not load (please make sure that the password is already changed by the time you post). Q: I am getting the error "fopen: ./all.chr: No such file or directory" (or "fopen: ./lanman.chr: No such file or directory"). Q: Where are the charset files? A: Development versions of John the Ripper might not include the charset files. You're supposed to take them out of the latest official release. Q: Where do I get wordlists for use with John? A: http://www.openwall.com/wordlists/ Q: Where do I get new versions of John the Ripper? Q: Where do I get the source code for John? Q: I only have the source code for John the Ripper, where do I get it pre-compiled for my OS (if supported)? Q: What is the primary website for John the Ripper? A: http://www.openwall.com/john/ Q: How can I contact you (the author)? A: See CONTACT. (Semi-)advanced topics. Q: I've recently switched my system to MD5-based (or Blowfish-based) password hashes, but there are still some DES-based hashes in the password file. How do I handle multiple hash types in one file? A: Use the "--format=..." option to tell John which hashes you would like it to load. Unfortunately, you will have to run John for each hash type separately. This requirement may sometimes be avoided with the use of "--format=crypt", but this is not recommended. Please see the description of the "--format" option in OPTIONS for more detail. Q: I have 10 users, but John said it loaded 15 password hashes. What's going on? A: Some extremely poorly designed hash types (Windows LM hashes and DES-based crypt(3) hashes known as "bigcrypt") have a property that allows John to split their encodings into two separate hashes (corresponding to "halves" of plaintext passwords) on load. John then proceeds to crack those hashes separately, so at a given time it might have only one of two halves of some passwords cracked. If interrupted and restarted, it would need to only load the hashes that correspond to uncracked password halves, so the number of such hashes is what John reports (in all cases, for consistency). Q: Are the strings tried with "-i" ("incremental" mode) random? They certainly look like they are almost random. A: No, they are not. No single candidate password will be tried for a
second time and the order in which they are tried is in fact very smart: it is based on frequencies of different trigraphs, stored and processed separately for each character position and for each password length. Q: Why doesn't John display a progress indicator for the "incremental" mode? A: Do you really want to see a 0% all the time? As explained in MODES, "incremental" mode is not supposed to terminate in a reasonable time. (There are a few exceptions to this, so a progress indicator might be added at some point.) Q: I am running John for 10 days and it is still not finished?! Q: How long should I expect John to run? A: It primarily depends on the cracking mode(s) and on your password files (in particular, the type of hashes and the number of different salts, if applicable). Most importantly, you should note that the "incremental" mode, which a default John run (with no command line options) proceeds with after being done with the quicker checks, is not supposed to terminate in a reasonable time. It is up to you to decide how long you're going to let it run, then consider any uncracked passwords strong enough. "Single crack" mode runs typically take from under a second to one day (depending on the type and number of password hashes). Wordlist mode runs may also be quick (under a second) for tiny wordlists and fast hashes or they may take multiple days with large wordlists, with word mangling rules, and with slow hash types and substantial numbers of different salts. The status line John reports whenever you hit a key includes a progress indicator (percent complete) for "single crack" and wordlist modes. With no cracking mode requested explicitly, John will start with "single crack" mode (pass 1), then proceed with wordlist mode (pass 2), and finally with "incremental" mode (pass 3). The pass numbers are reported on the status line, too. It is reasonable to let John reach "incremental" mode (pass 3) and run that for a while (some days). You will notice that John's success rate (the number of passwords cracked per hour or per day) will be dropping rapidly. When you determine that the success rate is low enough, you interrupt John. Q: Why does John display meaningless c/s values while cracking, instead of real "crypts per second" rate? A: The values displayed by John mean combinations (of username and password) per second, not crypts per second. This is the effective cracking speed that you get on a particular set of password hashes, and it may be useful, for example, to tune the "--salts=..." threshold and other settings. If you want a benchmark of the low-level password hashing routines only, use "--test". (Future versions of John the Ripper might report effective and raw c/s rates for different time intervals. These won't fit on the current status line, though.) Q: I just noticed that the c/s rate reported while using "incremental" mode is a lot lower than it is with other cracking modes. Why? A: You're probably running John for a few seconds only. The current "incremental" mode implementation uses large character sets which need to be expanded into even larger data structures in memory each time John switches to a different password length. Fortunately, this is only noticeable when John has just started since the length switches become rare after a few minutes. For long-living sessions, which is where we care about performance the most, this overhead is negligible. This is a very low price for the better order of candidate passwords tried. Q: What are the "real" and "virtual" c/s rates as reported by "--test"
(on Unix-like operating systems)? A: These correspond to real and virtual (processor) time, respectively. The two results would differ when the system is under other load, with the "virtual" c/s rate indicating roughly what you could expect to get from the same machine if it were not loaded. Q: How can I test John's password hashing routines for proper operation? A: John always performs a self-test when you run it on a password file and refuses to work if an error occurs. If you need to test all of the low-level routines at once, use "--test". Q: Does John support multi-processing or distributed processing? A: There's currently built-in parallel processing support using OpenMP (to make use of multiple CPUs and/or CPU cores in a single system) for all crypt(3) hash flavors (DES-, MD5-, and Blowfish-based) supported by John natively, as well as for LM hashes and, when running on Linux or Solaris, also for the underlying system's thread-safe password hashing function. The latter is only reasonable to use for crypt(3) hash types not yet supported by John natively (that is, for glibc 2.7+ SHA-crypt hashes as used by recent versions of Fedora and Ubuntu, and for SunMD5 hashes, which may optionally be enabled on Solaris). In "community enhanced" -jumbo versions, parallelization with OpenMP is also supported for many (but not all) of the hash types added in those versions. To use John's OpenMP support, you need to make an OpenMP-enabled build by uncommenting one of the OMPFLAGS lines near the beginning of the Makefile. This requires GCC 4.2 or newer, or another OpenMP-capable C compiler. For other hash types and/or to distribute the workload between multiple machines, other approaches need to be used. For a small number of nodes (CPUs, CPU cores, and/or machines), it is reasonable to use a manual approach. One of those approaches is to have your nodes try different password lengths. This is easily accomplished with "incremental" mode's "MinLen" and "MaxLen" settings (see CONFIG). Typically, you would not really need to split the workload for "single crack" and wordlist modes since these are relatively quick, although you may dedicate one node to those initially. You may safely run multiple instances of John in the same working directory, all writing to the same "pot file" (this is a feature). You do, however, need to assign each of them a unique session name, with "--session". Other approaches, such as splitting password files naively (without regard to salts), are typically less efficient (in some cases to the extent where there's no speedup from using multiple nodes at all). Some advanced and automated approaches are listed on the wiki at: http://openwall.info/wiki/john/parallelization Q: What is the format of the crash recovery files ("john.rec", other .rec's)? What do the numbers mean? A: The format of these files is deliberately undocumented and is subject to change without notice. (However, each release of John the Ripper is likely to be able to read .rec files produced by at least the immediately preceding release. Whenever compatibility is broken, John will refuse to recover the session, leaving the .rec file intact.) Although the meaning of some of the numbers that get into .rec files is trivial to explain, it is not possible to reasonably describe some others without going into great detail on John internals. If you really need to know, read the source code. $Owl: Owl/packages/john/john/doc/FAQ,v 1.27 2011/11/21 02:36:55 solar Exp $
I can't run John.
If you're not familiar with your OS, you should probably not be using John in the first place since John is a tool for system administrators. However, here are the answers to a few (not very) common questions to avoid having them asked over and over and for amusement. Q: When I type "john" (or "john passwd", etc.), it says "command not found" (or equivalent)?! A: The examples given in John the Ripper documentation assume that you know how to invoke newly-built programs from your shell. On Unix-like systems, it is typical to not have "." (the current directory) in your $PATH (the list of directories to search for programs). In that case, you need to type "./john" (dot, slash, and "john", without the quotes) to invoke the John binary executable located in the current directory. Q: ...but I am on a Unix-like system and I don't seem to readily have a John binary executable. A: Please follow the instructions in INSTALL. Q: When I double-click on "john.exe", a window flashes and disappears?! A: You're not supposed to click. You're supposed to run John from a command-line shell. On Windows, some of those shells would be cmd.exe, command.com, or bash (the latter is available with Cygwin). Other trivial matters. Q: How do I start John on my password file, use a specific cracking mode, see the passwords it cracked, etc? A: See README and EXAMPLES. :-) Q: How do I "unshadow"? A: See EXAMPLES on how to combine your passwd and shadow files, provided that you have root access to the target system. Q: Why doesn't John load my password file? It says "No password hashes loaded", "No password hashes loaded (see FAQ)", or "No password hashes left to crack (see FAQ)". A: Your password file might be shadowed. You need to get both /etc/passwd and the shadow file (typically /etc/shadow), and combine them into one file for use with John. Please refer to EXAMPLES. A: All of the password hashes found in the file (that are of the same type as the very first recognized hash in the file unless you're using the "--format=..." option) might be already cracked by previous invocations of John. (The message printed in that case has been changed to "No password hashes left to crack (see FAQ)" starting with version 1.7.7.) To display cracked passwords, use "john --show" on your password hash file(s). To force John to crack those same hashes again, remove the john.pot file. A: With PWDUMP-format files, John focuses on LM rather than NTLM hashes by default, and it might not load any hashes at all if there are no LM hashes to crack. To have JtR Pro or a build of JtR with the jumbo patch
focus on NTLM hashes instead, you need to pass the "--format=nt" option. A: The file you're trying to run John on might in fact not be a password file at all. A: Your command line syntax might be wrong, resulting in John trying to load a wrong file. A: Your password file format or hash type(s) might not be supported by John, or at least by the version and build of John that you're using. If you're positive that this is the case, you may want to check the contributed resources list on John the Ripper homepage for a suitable patch and, if unsuccessful with that, post a note to the mailing list (see CONTACT) including a sample password file line that John does not load (please make sure that the password is already changed by the time you post). Q: I am getting the error "fopen: ./all.chr: No such file or directory" (or "fopen: ./lanman.chr: No such file or directory"). Q: Where are the charset files? A: Development versions of John the Ripper might not include the charset files. You're supposed to take them out of the latest official release. Q: Where do I get wordlists for use with John? A: http://www.openwall.com/wordlists/ Q: Where do I get new versions of John the Ripper? Q: Where do I get the source code for John? Q: I only have the source code for John the Ripper, where do I get it pre-compiled for my OS (if supported)? Q: What is the primary website for John the Ripper? A: http://www.openwall.com/john/ Q: How can I contact you (the author)? A: See CONTACT. (Semi-)advanced topics. Q: I've recently switched my system to MD5-based (or Blowfish-based) password hashes, but there are still some DES-based hashes in the password file. How do I handle multiple hash types in one file? A: Use the "--format=..." option to tell John which hashes you would like it to load. Unfortunately, you will have to run John for each hash type separately. This requirement may sometimes be avoided with the use of "--format=crypt", but this is not recommended. Please see the description of the "--format" option in OPTIONS for more detail. Q: I have 10 users, but John said it loaded 15 password hashes. What's going on? A: Some extremely poorly designed hash types (Windows LM hashes and DES-based crypt(3) hashes known as "bigcrypt") have a property that allows John to split their encodings into two separate hashes (corresponding to "halves" of plaintext passwords) on load. John then proceeds to crack those hashes separately, so at a given time it might have only one of two halves of some passwords cracked. If interrupted and restarted, it would need to only load the hashes that correspond to uncracked password halves, so the number of such hashes is what John reports (in all cases, for consistency). Q: Are the strings tried with "-i" ("incremental" mode) random? They certainly look like they are almost random. A: No, they are not. No single candidate password will be tried for a
second time and the order in which they are tried is in fact very smart: it is based on frequencies of different trigraphs, stored and processed separately for each character position and for each password length. Q: Why doesn't John display a progress indicator for the "incremental" mode? A: Do you really want to see a 0% all the time? As explained in MODES, "incremental" mode is not supposed to terminate in a reasonable time. (There are a few exceptions to this, so a progress indicator might be added at some point.) Q: I am running John for 10 days and it is still not finished?! Q: How long should I expect John to run? A: It primarily depends on the cracking mode(s) and on your password files (in particular, the type of hashes and the number of different salts, if applicable). Most importantly, you should note that the "incremental" mode, which a default John run (with no command line options) proceeds with after being done with the quicker checks, is not supposed to terminate in a reasonable time. It is up to you to decide how long you're going to let it run, then consider any uncracked passwords strong enough. "Single crack" mode runs typically take from under a second to one day (depending on the type and number of password hashes). Wordlist mode runs may also be quick (under a second) for tiny wordlists and fast hashes or they may take multiple days with large wordlists, with word mangling rules, and with slow hash types and substantial numbers of different salts. The status line John reports whenever you hit a key includes a progress indicator (percent complete) for "single crack" and wordlist modes. With no cracking mode requested explicitly, John will start with "single crack" mode (pass 1), then proceed with wordlist mode (pass 2), and finally with "incremental" mode (pass 3). The pass numbers are reported on the status line, too. It is reasonable to let John reach "incremental" mode (pass 3) and run that for a while (some days). You will notice that John's success rate (the number of passwords cracked per hour or per day) will be dropping rapidly. When you determine that the success rate is low enough, you interrupt John. Q: Why does John display meaningless c/s values while cracking, instead of real "crypts per second" rate? A: The values displayed by John mean combinations (of username and password) per second, not crypts per second. This is the effective cracking speed that you get on a particular set of password hashes, and it may be useful, for example, to tune the "--salts=..." threshold and other settings. If you want a benchmark of the low-level password hashing routines only, use "--test". (Future versions of John the Ripper might report effective and raw c/s rates for different time intervals. These won't fit on the current status line, though.) Q: I just noticed that the c/s rate reported while using "incremental" mode is a lot lower than it is with other cracking modes. Why? A: You're probably running John for a few seconds only. The current "incremental" mode implementation uses large character sets which need to be expanded into even larger data structures in memory each time John switches to a different password length. Fortunately, this is only noticeable when John has just started since the length switches become rare after a few minutes. For long-living sessions, which is where we care about performance the most, this overhead is negligible. This is a very low price for the better order of candidate passwords tried. Q: What are the "real" and "virtual" c/s rates as reported by "--test"
(on Unix-like operating systems)? A: These correspond to real and virtual (processor) time, respectively. The two results would differ when the system is under other load, with the "virtual" c/s rate indicating roughly what you could expect to get from the same machine if it were not loaded. Q: How can I test John's password hashing routines for proper operation? A: John always performs a self-test when you run it on a password file and refuses to work if an error occurs. If you need to test all of the low-level routines at once, use "--test". Q: Does John support multi-processing or distributed processing? A: There's currently built-in parallel processing support using OpenMP (to make use of multiple CPUs and/or CPU cores in a single system) for all crypt(3) hash flavors (DES-, MD5-, and Blowfish-based) supported by John natively, as well as for LM hashes and, when running on Linux or Solaris, also for the underlying system's thread-safe password hashing function. The latter is only reasonable to use for crypt(3) hash types not yet supported by John natively (that is, for glibc 2.7+ SHA-crypt hashes as used by recent versions of Fedora and Ubuntu, and for SunMD5 hashes, which may optionally be enabled on Solaris). In "community enhanced" -jumbo versions, parallelization with OpenMP is also supported for many (but not all) of the hash types added in those versions. To use John's OpenMP support, you need to make an OpenMP-enabled build by uncommenting one of the OMPFLAGS lines near the beginning of the Makefile. This requires GCC 4.2 or newer, or another OpenMP-capable C compiler. For other hash types and/or to distribute the workload between multiple machines, other approaches need to be used. For a small number of nodes (CPUs, CPU cores, and/or machines), it is reasonable to use a manual approach. One of those approaches is to have your nodes try different password lengths. This is easily accomplished with "incremental" mode's "MinLen" and "MaxLen" settings (see CONFIG). Typically, you would not really need to split the workload for "single crack" and wordlist modes since these are relatively quick, although you may dedicate one node to those initially. You may safely run multiple instances of John in the same working directory, all writing to the same "pot file" (this is a feature). You do, however, need to assign each of them a unique session name, with "--session". Other approaches, such as splitting password files naively (without regard to salts), are typically less efficient (in some cases to the extent where there's no speedup from using multiple nodes at all). Some advanced and automated approaches are listed on the wiki at: http://openwall.info/wiki/john/parallelization Q: What is the format of the crash recovery files ("john.rec", other .rec's)? What do the numbers mean? A: The format of these files is deliberately undocumented and is subject to change without notice. (However, each release of John the Ripper is likely to be able to read .rec files produced by at least the immediately preceding release. Whenever compatibility is broken, John will refuse to recover the session, leaving the .rec file intact.) Although the meaning of some of the numbers that get into .rec files is trivial to explain, it is not possible to reasonably describe some others without going into great detail on John internals. If you really need to know, read the source code. $Owl: Owl/packages/john/john/doc/FAQ,v 1.27 2011/11/21 02:36:55 solar Exp $
