It is unlikely that Hillary's email server was actually hacked

It is unlikely that Hillary's email server was actually hacked
(Originally posted Aug 2, 2016 on Google+)


Update: In the meantime, the FBI released its detailled report[2]. It basically confirms all of my assumptions, that the forensic analysis was thourough, that the server admins were not clueless, that the systems were maintained, that security devices and software like firewalls and intrusion detection systems were used, that all were aware of the risks. And so on. I'm now even more confident that Hillary was not hacked. 


It is an open question: Was Hillary Clinton's private email server(s) compromised, possibly by Russian intelligence, and will the emails be published as a weapon right before the upcoming Presidential election?

My guess is that this is -- considering the facts we know so far -- possible, but unlikely. Let me explain why.
(This post is technical and quite lengthy, so if you are just interested in my conclusions, just skip to the very end)

First a quick overview on how a hacker can gain access to emails:

1. Get physical access to the server
2. Remotely compromise the server’s operating system or email software
3. Intercept email transmissions from or to other email servers
4. Intercept client connections to the server
5. Get physical access to the client
6. Remotely compromise the client
7. Guess, intercept or get otherwise the target’s password
8. Get access to backups
9. “Hijack” the email domain

So let’s have a closer look at each of these points, analyze how likely it is Hillary was successfully attacked this way, and whether the use of a government system would have provided more security here.


1. Get physical access to the server

Physical access to the server means access to the stored emails (unless the server’s data volumes are encrypted, which is very rarely used). However an attacker needs to remove the server, or at least boot another operating system, to be able to copy the emails and/or install software for later remote access. I assume it is highly unlikely this happened undetected, even not while the server was located in Clinton’s home.

Of course a government (as well as a good commercial) data center provides a much better physical protection than a private home.


2. Remotely compromise the server’s operating system or email software

That’s maybe the most interesting point, mostly because here it matters most that Hillary used a private email server, not a government one. And of course, it is clear she would have been much more secure if she had used the government's email system, that isn’t -- unlike her server -- directly and comfortably accessible over the internet (which was, most likely, the main reason why she used a private server at all).

But how likely was it that her private server (to be correct, and that makes it even more complicated, there were several servers, administrated by several people; I’m aware of this, but in the following I will speak of only one server -- in the sake of comprehensibility) was actually hacked?

“Hacked” here means that an attacker found a vulnerability in the server's operating system or email server software (or other running software), exploit it and thus gain access to at least the email system, maybe to the entire server.

I doubt this actually happened for several reasons:

First, because from what we know, the administrators of Hillary's server were at least not completely careless. They knew about the risks, and at least they installed some intrusion detection software. This is for me a clear indication that they at least took the most basic and most important precautions: To keep the operating system and email server software up to date, to install all necessary patches. If you do this, it basically means an attacker needs a 0-day exploit (I will come back to this later) to successfully hack the server.

There is another indication that the administrators kept the server up-to-date: The involvement of the NSA. It was reported that the NSA was aware of Hillary’s private server (because they were requested for a statement), so the least I would expect from the NSA would be that they frequently scan the server for known vulnerabilities, and alarm her (or the State Department) if they found one.

The next indication that Hillary’s server was not hacked is because the FBI didn’t find any traces of a successful hack, despite thorough forensic examinations. This is again no proof, it is possible to wipe all traces after a successful hack, but not very likely. One reason are human errors (which are the more likely the more complex a task is). The other reason needs a short explanation (and should give you a clue why it is so complex): When a hacker discovers a vulnerability, and uses it to break into a system, sophisticated hackers then do two things: First, install a backdoor for themselves, and second -- and this is often overlooked in the discussion -- close the vulnerability, so that no other hacker can exploit the same vulnerability und thus thwart them. This means the hacker needs to seriously alter the system, making it quite unlikely that he will leave the system without any traces.

Let's come back to the 0-day exploits, which leads to my last reason why I think it's unlikely Hillary's server was hacked. So far, my conclusion is that a 0-day exploit was needed for a successful hack. So let's assume Russian intelligence had such a 0-day for Hillary's server, but then I ask the critical question: Why should they have used it? You may think this question is ridiculous, especially when you only look at it from today's point of view. But think back, and keep in mind that 0-days are very precious, and if you use it, there is a high risk it is (afterwards) detected and it is no 0-day anymore then. And further consider that the Russians could not knowthat Hillary will email classified, top secret content. And consider that democratic countries usually act very transparent. What should they expect from Hillary's emails? Would they really "waste" a precious 0-day exploit to read something they can read the next day in any newspaper anyway? I seriously doubt.

As a last point, it was reported that Hillary’s private email server had a worldwide open Microsoft remote desktop service running (likely for remote administration of the server). If true, this was an unnecessary and serious security risk, to say it friendly. However, what I said above is here also true: The FBI’s forensic examination didn’t show any signs of compromises, and as long as the administrator kept the system up-to-date with patches and used good passwords, the risks were minimized.


3. Intercept email transmissions from or to other email servers

This is a quite complex topic too, and to best understand my points I suggest you read this post[1], in which I explain the possibilities and limitations of internet backbone tapping.

But what does this mean for this case?

First what you should know is that we are speaking about a time pre-Snowden, which means that very less email servers supported STARTTLS, so it is fair to assume that virtually all emails to and from Hillary’s server were transmitted unencrypted (unless the email itself was encrypted with tools like PGP or GPG). So, it is true that the emails were likely vulnerable to eavesdroppers.

But the deciding point is where the eavesdropping took place. The only place that ensures to be able to capture all of Hillary’s emails was very close to her server, so this would have been the internet uplink of her home or the data center the server moved to later. And I don’t think this likely happened, at least not permanently (remember that with network captures you can only collect emails that are transmitted within the timeframe you are running the tap). Foreign intelligence agencies cannot compel American companies to assist. It would be possible to try to bribe employees (in Hillary’s home, the data center, or from the uplink communication company), but this would have been a risky operation. As well as to try to physically tap Hillary’s internet uplinks. You may think it may have happened, and yes it’s possible, but I highly doubt it actually happened.

The next possibility are internet backbone taps, but as I explained in[1], the internet is so big and messy, with so many participating routers and uplinks, that it is unlikely that a hostile intelligence agency was able to capture a serious amount of Hillary’s emails. And you should consider that virtually all domestic American traffic remains in the country, so emails that Hillary sent or received from or to American people could only be collected with assistance from American companies, or with very risky operations. Both is IMHO very unlikely. And even when Hillary sent an email to friendly governments like Germany, it is very unlikely that hostile intelligence agencies could intercept it, because there is a very low chance these emails crossed countries where the hostile county had jurisdiction.

So, to summarize, network captures of Hillary's emails were possible, but pretty unlikely to actually have happened, at least not on big scale. And, more important here, it wouldn't have been a difference here if Hillary had used the government's email system, the described vulnerabilities would have applied the same.


4. Intercept client connections to the server

This topic is similar to the former one, but still different. Different mainly because here we speak about a client-to-server communication, not about a server-to-server communication like in point 3. The difference is that it is much easier to enforce encryption between client-to-server as server-to-server. And this is, at least according to the known facts, what the administrators of Hillary's server did, at least since 2009: Do the client communications over encrypted HTTPS.

So here I disagree a bit with the FBI’s in investigations. They criticized that Hillary sent and received emails from within hostile countries without mentioning that these communications were most likely encrypted, and thus most likely unreadable for eavesdroppers. Man-in-the-middle attacks were still possible, but likely Hillary would have received a warning about the certification change, so it’s unlikely a hostile intelligence agency launched such a risky operation in the first place.

But, of course, to use the government system likely without any remote access possibility would have been much more secure.


5. Get physical access to the client

This possibility was just added for completeness, there is no indication at all that one of Hillary's phones was stolen.


6. Remotely compromise the client

I will not go into detail much on this, simply I’m not an expert about Blackberry (what Hillary reportedly used) operating system security. I didn't read so far a convincing report that this might have happened, so I rule it out with a reasonable likelihood. But again, I’m no expert here, so if you think I’m wrong, just explain me what I missed and I’ll correct this post.


7. Guess, intercept or get otherwise the target’s password

This point is interesting in some way. So far, I mostly covered highly sophisticated hacking attempts. However, if we look at the stats, successful email hacks are almost always enabled by compromised passwords. Be it that the passwords are weak or compromised in any other way (eg, the infamous Podesta emails were most likely hacked with a phishing attack, entrapping Podesta to expose his login credentials). That is why a good security policy should always include a 2-factor authentication, which Hillary's email server clearly didn't demand -- however, there are also no indications that her password was compromised. It is much more documented that she was briefed, that there were concerns that her setup is not secure, so my conclusion is that most likely she at least used a good password and was not conned by a phishing attack.

But here the use of the government system would have been much more secure, simply because it likely offered no remote access at all.


8. Get access to backups

This point is more or less for completeness. It is a possibility to steal the emails, but not very likely, because backup servers and storages are almost always located in out-of-band networks, unaccessible from the outside. It may be possible to bribe technicians to hand over backup tapes, but this didn’t happen very likely. And in any case, I don’t see here a big risk difference between using a government or a private server.

Update: It is possible that the server was backuped to a cloud-based provider (and there are news reports that this actually happened). In the worst case, the backups were transmitted unencrypted and/or a weak password/authentication was used. Each of it would make it very vulnerable. However, I still consider it very hard for foreign intelligence agencies to launch powerful network taps insider the U.S. And in the case of a compromised password, the logfiles of the cloud-based backup provider should show accesses from unexpected IP-addresses. I assume the FBI had investigated such accesses. So I think it’s not very likely Hillary's emails were compromised this way.
(It is also possible that the backup provider itself was compromised, but I will leave this possibility aside, for now)


9. “Hijack” the email domain

I only add this because I read some articles that mentioned this as a possible attack. I think it is unimaginable that it happened undetected, so while it is of course a possible attack, it is fair to assume that it didn’t actually happen. (And BTW, with this kind of attack a hacker can “only” collect incoming emails, not outgoing)


Conclusion

I listed all possible attacks against Hillary’s private email server (I don’t think I missed a major one), worked through each of it, and despite each of them is more or less possible, none I consider to likely have actually succeeded.

So I come to another conclusion as most experts, and as the FBI as well. Maybe they are a little bit more “careful” than I am. The problem with my position is: You cannot prove something that didn’t happen. There will never be a proof that Hillary’s private email server was not hacked, but maybe some day a proof emerges that it was hacked. And if this happens, and you publicly stated that you don’t think it was hacked, then you are the exposed dumbass idiot. I do it anyway, because it is my honest conviction that the known facts indicate the server was not hacked.

In the wake of the “Snowden revelations”, most people got the impression that almighty intelligence agencies could always monitor and read everything, everywhere, everyone. But this is simply not true. The technical facts are that hacking is usually hard and risky, and internet backbone tapping has many, many limitations.

And please don’t get me wrong. With this post I don’t want to give you the impression that it was ok what Hillary did. I absolutely agree with the FBI that she and her staff were “extremely careless”. The only thing that I would add is that it is my impression a lot of more people in the U.S. government were not much less careless (but this should be no excuse for Hillary). Email is a security nightmare (and even today’s widespread use of STARTTLS doesn’t remedy this much), and it should be an absolutely NoGo that classified content can find its way into an email. When Hillary did sent classified content over email, why didn’t she get responses like “WTF did you send me this over email?!?”. The only explanation I have is that nobody was really aware of the risks.
I just say it was unlikely Hillary was actually hacked. No more, no less.

2016/08/04 Updated the article with adding the possibility that a cloud-based backup was hacked (thanks to hints from Matt Beebe @VoteBeebe on Twitter)

[1] https://rolfweber.blogspot.com/2019/10/the-difference-between-nsas-prism-and.html
[2] https://vault.fbi.gov/hillary-r.-clinton