@sroberts advanced persistent incident responder

The What happens when you use a browser Question

Browser Bar

Go into a tech interview, especially one for operations or security, and you’re more than likely going to get an interview question like this:

“What happens when you put a URL in the address bar of a browser and hit enter?”

I’ve been on both ends of this question, asked it and answered it. I’d like to look at what the answer is (or at least one answer), why it’s good, why it’s bad, and what could be better.

My Answer

Like many good things in life the answer to this question is a recipe in two parts.


Well… it’s sorta cooking… Source: Giphy

The Ingredients

There are concepts/acronyms you must know, name drop, and discuss:

The Recipe

Those ingredients aren’t enough. You have to know how to put them together.

To start with there is all the stuff that happens on the client. This can go pretty deep pretty quickly, even getting into processor interrupts, how the key presses lead to the letters showing up on screen, etc. It’s a deep rabbit hole, so I avoid that idea to start by making a joke about it and moving on. Getting caught up in a systems architecture discussion just means this question lasts 15 mins instead of five. No one wants that.

Internetting! Literally how do you even Internet? Source: Giphy

First real bit is jumping into a discussion of either the local network stuff like DHCP and switches or DNS. I’m split on the best move. DHCP and local networking stuff is boring but essential and lets you talk about the OSI model early. I make a quick reference that this is home network stuff (it is), ask if the asker really wants to go into that (they never have) and move on.

The Domain Name System is where I really start building up a head of steam. First I dive in with an every person description of DNS, describing it as a telephone book for the Internet, taking written names people understand and turning them into numbers that people are bad at, equating phone numbers to IP addresses. Elegant and simple. Too simple you say? Then I come right back with a deeply technical concept, discussing the recursive nature of DNS, how you check the local name server first and then it checks the next server up until you get to the root (pick a letter if your choosing, no one knows them that well), then reference the TLD server, and then down to a specific domain answer. Are you interviewing with a dotcom? This is a great place to throw in a “… but it wouldn’t go that far to get to your site, that would already be cached.” (You suck up you!) and then explain how DNS record caching works. Don’t hesitate to throw in an aside about round robin DNS for big sites… you don’t really have to explain it (unless its a DNS job) because your interviewer likely doesn’t know what it is either, they’ve just heard the network team mention it.

Now that we have an IP address we can move on to packets. Throw in the ol’ SYN, SYN/ACK, ACK TCP handshake bit (you already talked about UDP during DNS right? If you didn’t just throw it back in now). Next you’re talking about how packets get across the Internet. I’ll be honest, I don’t fully understand this myself and I really should. I know it involves the Boarder Gateway Protocol but even after being asked this question multiple times I heaven’t really learned how that works (I should). Why haven’t I? Well honestly just saying “Some BGP magic happens…” has always been enough. Once I even mentioned RIP and even though that’s totally out of date no one cared.


Does it mean anything? Naa… just looks cool. Source: Giphy

Breeze past that and now your describing your packets once they’ve arrived. Throw in some sequence & acknowledgement numbers and reassembly, good to mention even if it’s stuff the networking stack does for you. Now we’re talking HTTP and basic conventions like serving the index.html page, but no one does that anymore, we’re into crypto now, in fact that’s part of why the question got asked, so pivot quickly to an URL redirect (mentioning HTTP code 3xx just because) and move on.


Source: XKCD

Alright the main event, this is where we’ve been going all this time, to explain that little green lock in the browser. Here you’ll want to start discussing public and private key cryptography, say something about super big prime numbers and move on. Math… who knows? You can shift this to certificates without a lot of pretense (it’s just some extra metadata right? No one really understands X509) and now you’re discussing chains of trust. This means discussing Certificate Authorities and how they sign other keys (Intermediate Certs) most of which are used to sign other keys until they get to your browser. Then some more magic happens and somehow the public key for the CA (stored in your browser already which is so handy), can be used to verify the public key of the site your on. How? I don’t know… big prime numbers? The point is it works and I haven’t done real math since college. Also add something about how certificates are tied to the domain name of the site, so that DNS thing is now doubly important.

Now you’re into the bonus round and you can mention lots of things. Some attack on CAs, sure. Dodgy CAs from some bad country, why not. Perfect forward secrecy, perfectly natural. What a browser does or should do when a certificate is wrong? Gutsy! Extended Validation, crypto deceleration, revocation, hash collisions, weak algorithms. The world is your oyster.

Is my answer good? Not really. I’m sure some networking centric folks will pick out real inaccuracies beyond me being flippant. I know they’re rampant. That isn’t the point though. The point is its good enough. The reason it’s good enough isn’t because my answer is correct but because ultimately the question itself is bad.

Why Interviewers Like It

I don’t blame anyone for using this question. There are reasons it seems great and in many cases the interviewer doesn’t get a choice. To start with it’s a multilayer question that’s highly open ended. A skilled interviewer can use this question to pivot into a variety of concepts around networking, intrusion detection, cryptography, fraud & abuse, etc. It lets an interviewee cover lots of ground and focus on specific aspects they want. It also hits on topics most networking and security folks have a basic understanding of.

Why It Isn’t A Good Diagnostic

First of all because it’s asked so often. I bet you’re thinking back to the last time you got it. It’s a question everyone can and should be prepared for. It’s also an answer that can come out of a book (or a well written blog post) and isn’t based on any practical experience. You study for about 30 mins and you can answer this question solidly, even without understanding what’s going on under the hood.

This question is deeply reliant on a skilled interviewer with deep understanding at all levels. While there are a few people capable of that (I was interviewed by one when I got my job at Mandiant) many people aren’t. Instead they just listen for a few key elements, probably 4 of the 6 ingredients mentioned above, and move on.

The result is an interviewer can’t judge an answer well because in most cases they don’t understand the question entirely either. They understand it just well enough to see someone who doesn’t know any of it but not well enough to argue with a vague or inaccurate answer. Truth be told almost no one should be expected to understand it all. Its too big, too wandering, and just understanding core concepts of the big pieces is difficult let alone a deep technical discussion at every layer. In the end all it does is test the ability to give a hand wavy conceptual answer about some common Internet subsystems. It doesn’t prove you can run any of them, troubleshoot any of them, or secure any of them.

In short describing this process doesn’t prove you can do a network operations or network security job in the least.

What would be better?

I think there’s a wide variety of important questions that can fill the same role. A bit of decomposition helps a lot, both for focus and ease of developing realistic answers. Here are a few ideas for better questions:

Describe how your home network is setup? What functions does it support? What decisions did you make about performance or security?

This gets away from mere facts and piles of acronyms and into uses and practical experience. Its subjective, open for discussion, and based on a users own experience.

What’s the difference between TCP and UDP and what’s a good use for each?

I’ve used this question a lot and it is funny. It starts like a common book question and then takes a turn. I’ve actually had more than one interviewee explain to me why no one should ever use UDP.

If you want to encrypt and compress a file which do you do first and why?

Its an academic question that doesn’t have an open ended answer, but it captures the understanding of basic cryptography concepts, at least to the level most people need to know them. Its also a good question for non-technical interviewers since it has a definitive answer.

If setting up an intrusion detection system would you set it up on the inside of a firewall or the outside? Why?

A bit more security centric, this is open ended while still being far more finite.

Better than any of these in my personal opinion? Don’t ask a question and expect an answer that second. That rarely happens day to day and thus doesn’t test an interviewees ability to do the job. Go for realism. Give a scenario similar to what the interviewee would actually do in their job and give them a chance to work through it, even learn about it if they need to. Are they interviewing to analyze malware? Give them a sample and a report template to fill out. Will they work through logs? Have an ELK server while a few million logs and ask them to identify the right ones based on a scenario you’ve seen before. Don’t just see if an employee can answer questions, see if they can do the job by doing it. Then ask them about their process.

We use <SNORT/YARA> and are currently very concerned about . Can you write us a signature for it? Here’s an example of what we’re trying to detect.

Not just an answer, now the response is an actual artifact that can be tested. Also real world applicable and lets you ask follow up questions about it.

Improving Interviews There are plenty of other good questions and you should work up a list of them. More than that look at what your interview process is set to test for and make sure it aligns with the tasks the interviewee would face in their first day. Unless you’re starting a Network Traffic Trivia Team it probably won’t start with “What happens when you start typing in a browser?”.

Personal Aside: Culture Fit

This bit will be unpopular and may even bug folks, but I think if you’ve read this far it’s because you care about improving your technical interviews. Let’s do away with culture fit questions.

I admit I used to love asking these. They were fun and easy. My team wanted people we would get along with so we asked the following:

If every time you walked into a room a song played like you were walking in for WWF what would it be?

I thought it was great, clever, funny, innocuous but now I see it a lot differently.

  • First of all it implies you know what WWF is and how the walk in song works. Not everyone does.
  • Second it puts someone on the spot to name a song, a pretty unexpected departure from questions about Snort and ArcSight. It’s totally out of left field.
  • Thirdly it encourages the interviewee to given an answer they think the interviewer will like, not a song they like. Maybe the interviewee likes classical or jazz and isn’t into EDM or rock.
  • It doesn’t speak to anything about the job, common tasks, or anything day to day. Unrelated in the extreme.
  • Lastly by its very nature it implies a certain arrogance and bravado that not everyone has and honestly you might not want in a coworker (and I hesitate to think what liking the question implied about 26 year old me…).

I’m embarrassed I used to ask that question. I can’t say I’ve ever been given a culture fit question that was a better diagnostic than an our question (an investment bank I once interviewed with might be the one exception but that may not be an especially good thing) and at this point I’m adamantly against them. I get that no one likes working with a jerk, but determining that takes more than asking someone’s outside interests.

Culture fit questions, by their very nature, focus on finding other people just like the interviewer. That’s fine if you’re interviewing for a new best friend, but for a professional setting your focus shouldn’t be on someone who likes the same movies or weekend activities, it should focus on someone who can do the job. Culture fit questions also disproportionately select against diverse candidates (and then teams wonder why they’re nothing but white guys). I’ve never been happy working with someone who couldn’t do the job just because they shared my interests.

Liking a person and respecting a person are not the same thing (nor are they mutually exclusive) and while I don’t need to be BFF with all my coworkers I do need to respect them and be respected by them. We should focus on finding coworkers we respect who can do the job, not finding a Work BFF. Culture fit questions don’t help.

2017 Goals

Time Square Source: Flickr

Ahh January 4th. It’s that time of year to review 2016 and think about what’s coming in 2017. Let’s start by looking at what I kicked off 2016 with:

A Year Later - 2016 Goals

Did I get it all done or fail miserably?

Little A, Little B Source: Pinterest

Here is the breakdown:

  • Chess: I play a lot of chess on Chess.com. My rating has hovered around 1000–1100. I could do a lot better if I took the time to review my games better, but I haven’t gotten there yet.
  • Code: My coding, both in Python and Golang (even a bit of JavaScript) has improved. I’m a decent enough coder, but would love to be a developer (better architecture, writing tests, etc).
  • Cook: Well I got over that Blue Apron thing and cook 3–4 times a week. I’d like to get better at it, and plan to review some cook books, but it’s coming along. It also does feel healthier.
  • Exercise: I was on track for 1200 miles as of June but eventually a combination of travel and 95F+ degree days just sapped my will.
  • Read: One of my better wins. I read a lot this year. A combination of regular Kindle and Audible audiobooks. Some favorites were Dark Territory, Incident Response & Computer Forensics 3rd Ed, & The Cuckoo’s Egg (Yeah I know, I was late to the party). For pleasure I can’t recommend The Blue Ant Trilogy (Pattern Recognition, Spook Country, & Zero History) by William Gibson enough.
  • Write: I’ve blogged a bit, I think I did better than last year (as well as moving over to Medium, which has it’s pros and cons). The real coup however will be in 2017 with the release of Intelligence Driven Incident Response, the book I’ve been working on with Rebekah Brown.

So so but overall I was pleased. So what about 2017? I’m trying to streamline my goals even more and make them more concrete. Here’s my initial four goals:

Learn Reverse Engineering

Reversing (specifically to understand malware (vs trying to develop exploits)) is one of the most fundamental DFIR investigative skills. I’m a capable malware triage analyst, using basic static (Yara, hashes, metadata, etc) and dynamic (mostly sandboxes) techniques to understand malware. I’d like to get much deeper and capable. I’m planning to start with Practical Malware Analysis and probably a reread of the Malware Analyst’s Cookbook. I might approach this in the style of 100 Days of Code and do my own 100 Days of Reversing.

Lose At Least 30lbs.

Yeah I know, that’s what everyone says on New Years, but this is really what my cooking & bike riding have been leading to. I don’t believe it’s a one factor fix, so a combination sleeping better, cutting calories in the kitchen, and burning them on the bike should do the trick. I might also look into giving rucking a try, in part just because of my odd obsession with the GoRuck products (seriously, as a bag fiend this is the best bag I’ve ever had, full stop).

Read & Write Daily

Again a combination/merging of previous goals. Reading more improves my writing, writing makes me want to read more. Both improve my thought process, understanding, and expand my worldview.

Automate more of my Life

This is the last one added and honestly just sort of thrown in. I’m not 100% sure what it means though. I’ve had great success with automating a lot of small things using IFTTT and a few other small tools. I’d like to continue to improve that especially around organization. Life is short and there’s a lot to do. I can use all the help I can get.


2016 was a crazy year for a lot of folks. If 2016 was a battle for many I tend to think 2017 will be a slog. It’ll take patience and understanding, but I think there is tremendous opportunity. Have a very happy New Year.

United States Response to Grizzly Steppe


Kremlin from the River. Source: Wikipedia.

Here it is. After weeks of wondering if and how the United States Government might respond the United States White House, State Dept, Treasury, and US-CERT have released information on and sanctions against the Russian government’s efforts to influence the United States elections. I offer all this without too much analysis given I’ve just seen it myself and expect it will take a long time to digest.

First the technical response, the US-CERTs information which included IOCs and a Joint Analysis Report with technical descriptions of TTPs:

US-CERT: GRIZZLY STEPPE – Russian Malicious Cyber Activity

You can find the extracted IOCs at the end of the article (Note: I only did minimal clean up on these. Their usefulness may vary.). Ok now that you’re back from checking all those indicators of compromise against your own environment there’s political stuff as well. First we have the best overall government summary from the White House, detailing the whats and whys.

Whitehouse: FACT SHEET: Actions in Response to Russian Malicious Cyber Activity and Harassment

This also links to the executive order enabling all of this to take place:

Whitehouse: Executive Order – “Blocking the Property of Certain Persons Engaging in Significant Malicious Cyber-Enabled Activities”

Then we get the details, broken out by agency.


E.O. 13694 authorized the imposition of sanctions on individuals and entities determined to be responsible for or complicit in malicious cyber-enabled activities that result in enumerated harms that are reasonably likely to result in, or have materially contributed to, a significant threat to the national security, foreign policy, or economic health or financial stability of the United States.

U.S. Deptartment of the Treasury: Issuance of Amended Executive Order 13694; Cyber-Related Sanctions Designations

Federal Bureau of Investigation

Taking a slightly different tact the FBI called attention to information on two Russian hackers already wanted for financial cyber crimes. Both were already on the FBI Cyber Most Wanted List:


Evgeniy Mikhailovich Bogachev is designated today for having engaged in significant malicious cyber-enabled misappropriation of financial information for private financial gain. Bogachev and his cybercriminal associates are responsible for the theft of over $100 million from U.S. financial institutions, Fortune 500 firms, universities, and government agencies.


Aleksey Alekseyevich Belan engaged in the significant malicious cyber-enabled misappropriation of personal identifiers for private financial gain. Belan compromised the computer networks of at least three major United States-based e-commerce companies.

Quotes pulled from the White House FACT SHEET: Actions in Response to Russian Malicious Cyber Activity and Harassment. Both are also mentioned in the Treasury Issuance above. Neither seems directly implicated in the Grizzly Steppe intrusions.

Media Coverage

Some of these seem to be based on coordination with the administration, some seem like follow up stories based on what’s been release.

There were lots of other articles but these seemed most thoughtful and well prepared.

Another interesting side note is the references to the United States authorities taking over physical compounds in the United States used by Russian Intelligence:

Older References

Office of the Directory of National Intelligence & DHS

A bit of a throwback but here’s their original statement on the matters:

DHS: Joint Statement from the Department Of Homeland Security and Office of the Director of National Intelligence on Election Security

Senate Armed Services Chairman Senator John McCain

A prominent Republican senator who’s called for investigation and action.


Russian Response

This all came out quickly so there has been limited Russian response so far. The most notable quote comes from the New York Times quoting Dmitri S. Peskov, the spokesman for Russian Prime Minister Vladimir Putin.

“We regret that this decision was made by the U.S. administration and President Obama personally […] As we have said before, we believe such decisions and such sanctions are ungrounded and illegal from the point of view of international law.”


Update: 17:12 EDT:

The Russian Foreign ministry has released a statement denying the attacks and denouncing the United States Government response:

MID: Новости

Update: 21:42 EDT:

The verified (and thus I assume valid) Russian Embassy twitter added this tweet:

So what now?

Hard to say. I do know jumping to conclusions is a bad move, so I’ll be writing more after doing some research. To that end here are the IOCs shared by the US-CERT extracted using my Cacador indicator extraction tool. I hope they’re useful!

Waiting vs Passivity in DFIR

Hamilton From the New York Times: “Review: ‘Hamilton,’ Young Rebels Changing History and Theater

Give it a second, I’ll explain the Hamilton reference to DFIR, but for now let me share one of my favorite songs. Aaron Burr thinks Alexander Hamilton is a brash aggressive brute and believes Hamilton thinks him slow and unwilling to make a decision. Burr then sings this song to explain his true goals:

Wait for It by the cast of Hamilton

Burr ends up having an iconic line about 3/4 of the way through the song:

BURR: I’m not falling behind or running late ENSEMBLE: Wait for it, Wait for it, Wait for it, Wait for it BURR: I’m not standing still, I am lying in wait

I’d heard this line dozens of times (yeah I admit, I’m a bit obsessed) but it struck me a bit differently recently. In this case I was monitoring a resource being used by an espionage group in carrying out their attacks. The question came up: Do we notify the owner and effectively burn the resource making it unusable by the attacker while losing telemetry or continue to monitor it and gain intelligence but risk it still being used against victims.

It was a tough choice that ended with us setting a series of conditions for determining if it was still gaining intelligence. I ended up framing this cost vs benefit analysis in the terms Aaron Burr uses in Hamilton: In our investigation what was happening? Were we simply standing still (being passive) or lying in wait (actively avoiding action in furtherance of a more decisive action later).

Sometimes people claim investigations need to always fix things and move, but sometimes the best thing for an investigation is to take the time to see what’s happening and plan the next steps. That shouldn’t be mistaken for passivity. But when do you hold and when do you move forward?

Unfortunately I can’t really help you with this one. This takes a combination of experience and intuition. While I can’t give you a solid answer here are some key questions to ask yourself:

  • Are you gaining meaningful intelligence?
  • Is gaining more information in the best interest of victims as well as the team collecting?
  • Are you able to exploit the information you’re gathering in order to act on it in an effective and timely manner?

If you answer yes to those then the best course of action is continuing collection until you’ve gained as much value as possible. The key is to keep revisiting these questions on a regular basis until there is a change. At that point you must be prepared to act, whether that’s remediating your network, notifying a victim, or sharing the intelligence you’ve gathered with others.

The last thing to consider about balancing the collection vs. action question is the pressure that can come with it. This can come from many directions, inside your team, your boss, impacted users, law enforcement, etc. The argument is straightforward and compelling: fix the problem and move on to other things.

In the end you not only have to decide for yourself but make your case to stakeholders, especially superiors. You have to be prepared to push back against others as long as you’re getting meaningful collection and developing intelligence. If you can’t get buy in then the best choice is to hand off information as soon as possible.

Hamilton himself probably said the best thing about the need to remain strong in situations like this…

Those who stand for nothing fall for anything. ~ Alexander Hamilton

ACH Analysis of a Trump Campaign Compromise

ASIDE: This post gets political. People may agree or disagree based on their own experience or personal belief. I accept that. I’m attempting to use evidence and analytical rigor to reach my conclusions while averting my own bias. If you think I missed the mark on those aspects (evidence or rigor) feel free to reach out to me. If you just disagree with my conclusions then I’d love to see a blog post exploring your own evidence and process.

The news this past week has been filled with statements about claims and counter claims of whether or not Russia was involved in meddling in the 2016 Presidential Election. Intelligence agencies, news outlets, & security vendors have stated that Russia compromised the Democratic National Committee, Hillary for America campaign Chairman John Podesta, Democratic Congressional Campaign Committee, and others related to the campaign. During the debates Donald Trump questioned the validity of this, leading to the 400 lbs hacker line (and resulting Twitter accounts) and he and his transition team have repeatedly questioned/refuted all such claims, even suggesting that the entire intelligence community might be wrong.

What’s been more interesting to me though has been questions about whether or not the Republican National Committee and Donald J. Trump for President, Inc were victims of similar attacks (For brevity we’ll just refer to them all collectively as the Trump Campaign). Many of us working in intrusion detection and incident response have had conversations with peers from other companies in the same vertical describing identical attacks. In fact this is one of the major strengths of computer network operations: hacking represents an asymmetric threat, having relatively low variable costs (the effort to compromise a target) vs fixed costs (the development of exploits and tools). I will say I for one assumed that the same attacks directed at the Democratic institutions were also directed at the Republican institutions.

The Republican leadership has said little about any of this until this weekend:

Reince Priebus, Incoming White House Chief of Staff on Meet The Press Dec 11, 2016

I for one was hugely surprised to hear not simply the assertion that the Trump Campaign hadn’t been hacked (an absence of evidence not being the same as evidence of absence), but the unabashed confidence. I realized that this surprise (even incredulousness) was based on my own experience and assumptions, biases I have, instead of on any analytic rigor. Sunday morning with coffee is obviously the best time to apply intelligence rigor to a problem, so here goes.

One powerful analytic model for problems of conflicting information and rampant bias is Analysis of Competing Hypotheses. A system of listing important information and scoring it to weed out bias and develop relativistic scoring of likely possibilities. ACH doesn’t give the answer but helps analysts understand what is most likely. Descriptions of each step are pulled from the CIA’s Center for the Study of Intelligence article linked above.


Identify the possible hypotheses to be considered. Use a group of analysts with different perspectives to brainstorm the possibilities.

I can come up with five serious hypotheses specifically with regard to Russian intelligence gathering activity:

  • The Trump Campaign was compromised and they are unaware.
  • The Trump Campaign was compromised and they are aware but want to withhold it.
  • The Trump Campaign was not compromised because their security was so good it blocked all attempts.
  • The Trump Campaign was not compromised because no one attempted.
  • The Trump Campaign was not compromised because the adversary was adversary collecting via non-technical means (IE HUMINT).

These don’t cover 100% of all possibilities (like if the adversary has a time machine and is getting intelligence sent from the future) but covers the major possibilities.


Make a list of significant evidence and arguments for and against each hypothesis.

Next we collect evidence for and against our hypotheses. Evidence is based on publicly disclosed information and some are based on experience from 10+ years as a defender and incident handler. Some of it is circumstantial, but it’s the best we have to go off of.

The Trump Campaign was compromised and they are unaware.

The Trump Campaign was compromised and they are aware but want to withhold it.

  • Political campaigns are rich targets full of policy level information.
  • Successful compromises is seen as an organizational failure.
  • It is particularly important for Republicans to be seen as strong on National Security.

The Trump Campaign was not compromised because their security was so good it blocked all attempts.

  • Network & Host Intrusion Detections are avoidable by skilled adversaries.
  • Rince Priebus (RNC Chairman) has stated the FBI investigated the Trump Campaign networks and found nothing.

The Trump Campaign was not compromised because no one attempted.

  • The intelligence community, vendors, and FBI stated the DNC hack was compromised by RU CNO.
  • Adversaries commonly target multiple organizations in the same vertical.

The Trump Campaign was not compromised because the adversary was collecting via non-technical means (IE HUMINT or collaboration).


Prepare a matrix with hypotheses across the top and evidence down the side. Analyze the “diagnosticity” of the evidence and arguments–that is, identify which items are most helpful in judging the relative likelihood of the hypotheses.

Taking the theories and evidence above everything goes onto a matrix without any editing or scoring. Fields have been abbreviated or coloquilized layout.

  Hacked & Unaware Hacked & Aware Perfect Security No Attempt Non-CNO Collection
Most orgs are hacked          
Campaigns hard to protect          
RU CNO hits political orgs          
RU CNO for influence          
Campaigns are rich targets          
Compromises is a failure          
RNC wants strong NatSec          
HIDS & NIDS are avoidable          
RP says FBI found nothing          
DNC hit by RU CNO          
CNO attacks multiple orgs          
DJT is positive on Putin          
DJT closeness to RU          
Lots of unvetted staff          
TOTALS: 0 0 0 0 0

Looking at the matrix a handful of pieces of evidence stand out as diagnostically significant:

  • Adversaries commonly target multiple organizations in the same vertical. They often have similar information and similar vulnerabilities. There’s no reason to think the defense of one political campaign would be better or worse than another. They are all resource limited and trying to provide similar capabilities.
  • Donald Trump is accused of having many undisclosed ties to Russia. While this hasn’t been confirmed or denied by the President Elect there are many many many discussion about DJT having deep business ties to Russia. His current rumored Secretary of State Rex Tillerson is also accused of having deep financial relationships in Russia.
  • The intelligence community, vendors, and FBI stated the DNC hack was compromised by RU CNO. This is probably the biggest key to the whole issue and is confirmed by a wide variety of sources (1, 2, 3).

Refine Matrix

Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no diagnostic value.

Here we add and remove a few pieces of evidence.

  • Removed: DJT is positive on Putin as this is sort of redundant with DJT closeness to Russia and could over influence.
  • Added: Vulnerable Campaign Infra. There are reports during the campaign as researchers looked at the security of Trump Campaign systems. This includes out of date servers and lacking security measures like two factor authentication. While many of these may have been fixed this came far after the attacks against DNC and thus likely periods of actor focus on campaigns.
  Hacked & Unaware Hacked & Aware Perfect Security No Attempt Non-CNO Collection
Most orgs are hacked          
Campaigns hard to protect          
RU CNO hits political orgs          
RU CNO for influence          
Campaigns are rich targets          
Compromises is a failure          
RNC wants strong NatSec          
HIDS & NIDS are avoidable          
RP says FBI found nothing          
DNC hit by RU CNO          
CNO attacks multiple orgs          
DJT closeness to RU          
Lots of unvetted staff          
Vulnerable Campaign Infra          
TOTALS: 0 0 0 0 0

I didn’t change my hypotheses. I still feel strong about all of them.

Tentative Conclusion

Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to disprove the hypotheses rather than prove them.

Now the process moves on to scoring. The scale is +2 to -2. The +2 should indicate a very strong correlation, -2 means a strong contradiction. A score of 0 indicates evidence neither confirms or denies the hypothesis.

  Hacked & Unaware Hacked & Aware Perfect Security No Attempt Non-CNO Collection
Most orgs are hacked +2 +2 -1 -1 0
Campaigns hard to protect +2 +2 -2 0 0
RU CNO hits political orgs +2 +1 0 -1 -1
RU CNO for influence +1 0 0 -2 0
Campaigns are rich targets +2 +2 0 -2 0
Compromises is a failure -1 +2 0 -1 0
RNC wants strong NatSec 0 0 0 0 0
HIDS & NIDS are avoidable +2 0 -2 -2 -2
RP says FBI found nothing +1 -1 +2 +2 +1
DNC hit by RU CNO +2 +2 -1 -2 0
CNO attacks multiple orgs +2 +2 0 -2 0
DJT closeness to RU -1 -1 0 +2 +2
Lots of unvetted staff 0 0 0 0 +2
Vulnerable Campaign Infra +2 +2 -2 0 -1
TOTALS: 16 13 -6 -9 1

This isn’t the end yet! Now it moves on to the adjustment of bias.

Adjust for Bias

Analyze how sensitive your conclusion is to a few critical items of evidence. Consider the consequences for your analysis if that evidence were wrong, misleading, or subject to a different interpretation.

This one is tough and requires considerable self reflection.

  • Rince Priebus (RNC Chairman) has stated the FBI investigated the Trump Campaign networks and found nothing. This piece of evidence sways everything pretty far and relies heavily on taking Rince Priebus at face value. Given that the validity of his statement is at the core of the whole exercise this may be a bad piece of information to rely on.
  • The adversary has used CNO against political targets before. This is another very influential piece of information, however it should not be removed. This is backed up by multiple sources and while it moves the analysis across the board that’s because it’s an important piece.

What about consequences if evidence is incorrect? In some cases this could have a fairly dramatic swing. A particularly high amount of weight is placed on the evidence that the Democratic campaign institutions were compromised. Given this has been confirmed by multiple sources I’m comfortable with that. Similar issues arise with the assumption that campaign infrastructure is likely vulnerable, but this is also supported with considerable evidence. That said a change to either of these would cause a significant shift in probability, but would not ultimately change the outcome of the analysis.

Finalize & Synthesize Analysis

Report conclusions. Discuss the relative likelihood of all the hypotheses, not just the most likely one.

Now our final matrix. Again, it’s based on a number of experience based assumptions and facts we have available.

  Hacked & Unaware Hacked & Aware Perfect Security No Attempt Non-CNO Collection
Most orgs are hacked +2 +2 -1 -1 0
Campaigns hard to protect +2 +2 -2 0 0
RU CNO hits political orgs +2 +1 0 -1 -1
RU CNO for influence +1 0 0 -2 0
Campaigns are rich targets +2 +2 0 -2 0
Compromises is a failure -1 +2 0 -1 0
HIDS & NIDS are avoidable +2 0 -2 -2 -2
DNC hit by RU CNO +2 +2 -1 -2 0
CNO attacks multiple orgs +2 +2 0 -2 0
DJT closeness to RU -1 -1 0 +2 +2
Lots of unvetted staff 0 0 0 0 +2
Vulnerable Campaign Infra +2 +2 -2 0 -1
TOTALS: 15 14 -8 -11 0

Based on this analysis I have a moderate confidence that the Trump Campaign was the victim of offensive computer network operations, but it is unclear whether or not the leadership was aware. It seems highly unlikely that the Trump Campaign had either perfect security or was not targeted at all. Non-CNO collection is difficult to determine based on the evidence we have available.

One thing I do know is that none of these items being true are good things.

  • If the Trump Campaign was compromised and they are unaware then they have no idea how bad the damage could be. This seems most likely.
  • If the Trump Campaign was compromised and they are aware but want to withhold it then we are being lied to, either to protect their own ongoing investigation (which seems unlikely given the lack of leaks) or in an act of hubris to prevent us from knowing how bad it really is.
  • If the Trump Campaign was not compromised because their security was so good it blocked all attempts then the technical security lead should start a company and put us all out of business (I find this personally the least likely possibility by miles).
  • If the Trump Campaign was not compromised because no one attempted then either they had nothing of value to steal which seems bad or…
  • If the Trump Campaign was not compromised because the adversary was collecting via non-technical means then the campaign either had a traitor in their midst without knowing or were sharing sensitive information deliberately, neither of which are good things.

I’ve wracked my brain trying to come up with more possibilities but I’m at a loss. I’ve long subscribed to the idea “Never assume malice where ignorance will suffice” but when it comes to electing our officials and leading our country neither seems acceptable to me.

Future Analysis

Identify milestones for future observation that may indicate events are taking a different course than expected.

How could this analysis improve in the future?

  • Further technical reporting released either by vendors.
  • Leaks from inside the Trump Campaign.
  • Declassification of relevant intelligence.
  • Better insight into HUMINT or collaboration between the Trump Campaign and the Kremlin.
  • More information may be released by other political campaigns, either from the primary or 3rd parties.
  • All of this has been based on the assumption of an intrusion by Russian CNO. There are many other countries with viable CNO teams that may have also targeted various campaign entities. Activist attacks are also within scope of consideration.

Whether or not we get any of these there are plenty of unanswered questions.

My Conclusion

It’s unlikely anyone will ever know the full truth and this analysis is likely missing key information, it merely provides a framework to understand what we do know. Hopefully this demonstration shows how an organization can use ACH to understand complicated issues. This may become even more important in the future as we have to accept that political realities, domestically and internationally, are a part of the computer security world.