October 12, 2013
A few of you might have noticed the quote yesterday, the ones that is used to analysing technical threats will know what I’m thinking of there. When analysing technical threats there are features to each and every threat actor. A lot of people say that everything can be faked or spoofed, but I believe that he will leave traces he didn’t think about - that new eyes may see. Even not leaving a trace is an indicator that should be thoroughly investigated. The traces can be in any dimension from the infrastructure used, to a guy a bit sloppy in the physical world, people talk, that is as certain as there will always be bugs in computer programs. The problem here is, as always, when you have terabytes of data to structure and abstract it in a way that produces the (most) correct result. The result here being to find which group is behind the e.g. spear phish - attribution. The cost is usually time. That’s about the introduction you will get to the subject at this point.
What this post is really about is ACH (Analysis of Competing Hypotheses). There are many people that have an academic background, but few I’ve seen actually use what has been taught them. ACH is a way of re-introducing some of it into attribution. Every scientist knows that for a hypothesis to be confirmed, it must be proved and tested against other theories. E.g. in court the result must be proven against reasonable doubt. Forensics is an area of information security that I admire in an scientific aspect.
ACH was developed by the CIA in the 1970’s because analysts favoured their presumptions in their conclusions. Or in CIAs own words:
Analysis of competing hypotheses, sometimes abbreviated ACH, is a tool to aid judgment on important issues requiring careful weighing of alternative explanations or conclusions. It helps an analyst overcome, or at least minimize, some of the cognitive limitations that make prescient intelligence analysis so difficult to achieve.
There are some read-ups on it, e.g. on Wikipedia, but I would recommend you go to the source to learn the methodology. The ACH procedure found in the latter link for reference:
Step-by-Step Outline of Analysis of Competing Hypotheses
- Identify the possible hypotheses to be considered. Use a group of analysts with different perspectives to brainstorm the possibilities.
- Make a list of significant evidence and arguments for and against each hypothesis.
- Prepare a matrix with hypotheses across the top and evidence down the side. Analyze the “diagnosticity” of the evidence and arguments–that is, identify which items are most helpful in judging the relative likelihood of the hypotheses.
- Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no diagnostic value.
- Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to disprove the hypotheses rather than prove them.
- Analyze how sensitive your conclusion is to a few critical items of evidence. Consider the consequences for your analysis if that evidence were wrong, misleading, or subject to a different interpretation.
- Report conclusions. Discuss the relative likelihood of all the hypotheses, not just the most likely one.
- Identify milestones for future observation that may indicate events are taking a different course than expected.
So now you’ve read it, but what do you do next? Write it on paper? There are quite a few electronic tools to help you out, but the one that in my experience works best is the one found on competinghypotheses.org. It’s in PHP (even how bad that may sound) and the backend is a MySQL-database that makes it easy to incorporate into you existing tools portfolio. I’m not going to elaborate further on the application since the site does this very well on its own.