/ conversion

Perspectives of Using OpenIOC with a Python Example

Despite all the good stuff going on with standardization of indicators and contextual exchange in information security, the adoption rate seems rather low. I bet that most organizations isn't even close to using the standards yet.

Another aspect of the low adoption rate is integration into the necessary tools such as Google Rapid Response (GRR), popular intrusion detection systems (IDS) such as Snort, Suricata and Bro and Mandiant IR (MIR). MIR supports OpenIOC though, while GRR indirectly and in an extraordinary undocumented way supports Yara through Rekall (haven't figured how to work it yet).

In this post I'm considering STIX, CybOX and OpenIOC (the first two very briefly) in regard to usage in a practical manner. Technical interpretation (converting it to plain text) of the indicators can be done in two ways as I see it right now:

  1. By the Python ioc_writer library for e.g. inserting indicators into a graph, or outputting to the more generally accepted csv format
  2. Using e.g. openioc-to-stix from the STIX Project for converting between different standards

Both STIX, CybOX and OpenIOC is thouroughly defined through XML schemas. The reason I think adoption is low is exactly due to that. The technical implementation gets too intricate. Most incident handlers I know wouldn't ever go near XML if it wasn't convertible to plain-text easily. Since that's the reality I think that is where we should start when looking at it technically as well.

Personally I favor OpenIOC due to its simplicity and broad Python support from the start. This is of course pretty close the the goal outlined above.

Some organizations choose to do things a little different than me. Option two is also pretty interesting, and requires you to have some STIX/Cybox-based system to insert the results into. I'm going to leave it with the repository. It doesn't support Python3 and not OpenIOC 1.1.

Take an OpenIOC 1.1 indicator file from IOC Bucket for instance. It would look something like this:

<?xml version="1.0" encoding="utf-8"?>
<OpenIOC xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" id="3d6882ca-18be-48ed-8f4c-38cb1843b51d" last-modified="2014-02-19T04:18:42Z" published-date="0001-01-01T00:00:00" xmlns="http://openioc.org/schemas/OpenIOC_1.1">
  <metadata>
    <short_description>asprox - kuluoz disk only</short_description>
    <description>...</description>
    <authored_by>@herrcore</authored_by>
    <authored_date>2014-02-19T03:20:36Z</authored_date>
    <links />
  </metadata>
  <criteria>
    <Indicator operator="OR" id="5e003c92-f308-42a8-922e-8b5cc3c17714">
      <Indicator operator="AND" id="ae1d98e5-4eac-45f9-80de-ee1f69669ccd">
        <IndicatorItem id="761761ba-8143-44e4-b505-dd1552ad934d" condition="contains" preserve-case="false" negate="false">
          <Context document="RegistryItem" search="RegistryItem/KeyPath" type="mir" />
          <Content type="string">Microsoft\Windows\CurrentVersion\Run\</Content>
        </IndicatorItem>
        <IndicatorItem id="3477a3ff-abdf-4631-9929-07a29f74a61c" condition="matches" preserve-case="false" negate="false">
          <Context document="RegistryItem" search="RegistryItem/ValueName" type="mir" />
          <Content type="string">[a-z]{8}</Content>
        </IndicatorItem>
        <IndicatorItem id="460080df-2586-48a9-a431-161b6e3cf88b" condition="matches" preserve-case="false" negate="false">
          <Context document="RegistryItem" search="RegistryItem/Text" type="mir" />
          <Content type="string">\\Users\\[A-Za-z\-\.]+\\AppData\\Local\\[a-z]{8}\.exe$</Content>
        </IndicatorItem>
        <IndicatorItem id="af83238b-fc97-40d9-8114-bcc9c0052bf3" condition="matches" preserve-case="false" negate="false">
          <Context document="FileItem" search="FileItem/FullPath" type="mir" />
          <Content type="string">\\Users\\[A-Za-z\-\.]+\\AppData\\Local\\[a-z]{8}\.exe$</Content>
        </IndicatorItem>
      </Indicator>
    </Indicator>
  </criteria>
  <parameters />
</OpenIOC>

Luckily ioc_writer is pretty simple to implement by your own. Find an example below:

from ioc_writer import ioc_api
import sys

ioc=ioc_api.IOC

root, metadata_node, top_level_indicator, parameters_node=ioc.open_ioc(sys.argv[1])

data={}
data['file_fullpath'] = top_level_indicator.xpath('.//IndicatorItem[Context/@search = "FileItem/FullPath"]')
data['arp_ipv4'] = top_level_indicator.xpath('.//IndicatorItem[Context/@search = "ArpEntryItem/IPv4Address"]')
data['network_dns'] = top_level_indicator.xpath('.//IndicatorItem[Context/@search = "Network/DNS"]')
data['file_sha256'] = top_level_indicator.xpath('.//IndicatorItem[Context/@search = "FileItem/Sha256sum"]')
data['file_md5'] = top_level_indicator.xpath('.//IndicatorItem[Context/@search = "FileItem/Md5sum"]')
data['network_uri'] = top_level_indicator.xpath('.//IndicatorItem[Context/@search = "Network/URI"]')

short_description=metadata_node.findtext('short_description')
authored_date=metadata_node.findtext('authored_date')
authored_by=metadata_node.findtext('authored_by')

print("type, value, short_description, author, authored_date")

for key in data.keys():
  for item in data[key]:
    print(item[1].text+",%s,%s,%s,%s"%(key,short_description,authored_by,authored_date))

Which outputs:

type, value, short_description, authored_date
\\Users\\[A-Za-z\-\.]+\\AppData\\Local\\[a-z]{8}\.exe$,file_fullpath,asprox - kuluoz disk only,@herrcore,2014-02-19T03:20:36Z

And by that you've got a CSV representation of IP addresses, domains and full path indicators from you OpenIOC 1.1.