bruteforce should score by frequency/rank about zxcvbn

[lowe]

Hi @tewalds, I'm going to close this as a wontfix, but thanks anyway for suggesting this approach. My issue is that this adds too big an assumption about how people choose passwords; I haven't seen data to support that this is a common strategy, and even if 10% of all passwords used this scheme, it would still be incorrect to apply to 90% of other passwords.

[tewalds]

Instead of simply closing this as wontfix, why not check the character frequencies in passwords that don't follow simple patterns. You've got a large password database so this should be pretty easy to check. If this is a valid way of reducing the entropy as shown in real passwords, then real password crackers probably use it effectively and it should be used here. If it doesn't reduce entropy much, then wontfix seems perfectly reasonable. I've read articles about password crackers downloading giant corpuses (lyrics, movie quotes, wikipedia, etc) and using sentences from there fairly effectively, though I'm having trouble finding a reference for that. What I did find is another similar idea of using markov chains, which should have similar effects: https://www.trustwave.com/Resources/SpiderLabs-Blog/Hashcat-Per-Position-Markov-Chains/ .