@sroberts advanced persistent incident responder

Minimal Packing for Maximum Travel

I started writing this at the end of March right after two trips in a row. I’ve since done another type of packing, moved, and now I’m finally catching up, so forgive some out of date thougths.

I’ve basically travelled non stop for the last two weeks, home only for last weekend. Back to back travel of fairly similar lengths makes it easy to compare, experiment, and plan a bit better.


On some level everyone has the same process for packing:

  • How long are you going to be gone?
  • What events do you think you’ll be going to?
  • What do you think you need for each of those events?

For my last two trips those amounted to:

  • Length: A week-ish in both cases
  • Events:
    • Most of what I did in both events was attend a conference as a representative of GitHub
    • General work time
    • Socializing with other Hubbers
    • Usual day to day stuff
    • It was also safe to assume at least one high end dinner
  • Needs:
    • Conference Hubbering: GitHub t-shirts (a benefit working for a company that’s based on being low key & productive)
    • A nice dinner & drinks out
    • The usual casual hanging out/living stuff


May 13-16: FOSE Conference

This was a 4 day trip to Washington DC to man GitHub’s booth at the FOSE Government IT conference. Most of our time was spent manning the booth, meaning GitHub tshirts (we wear them almost always). We also had one nice “team dinner” out and general chilling.

My Bag for FOSE

  • 6 Tshirts (2 blank grey, 4 GitHub tshirts)
  • 2 collard shirts
  • 2 pair of jeans, 2 pair of shorts
  • 5 socks
  • 4 pairs of shoes (gym shoes, flipflops, loafers, & Tom’s)
  • 2 sets of gym clothes (shorts & UnderArmor shirts)
  • 8 pairs of unmentionables

For a four day trip this was entirely too much. I never made it to the gym, only wore one collared shirt (on the trip back no less), and had extra tshirts, socks, jeans, etc.

May 19-25: BreizhCamp

This was a bit more extensive, as it was a longer trip, involved 2 cities (Rennes & Paris France) and a few more events. Hanging out at the conference was the same as booth duty, GitHub tshirts and jeans. My “buddy” coworker and I also went out for a very nice (Michelin ) lunch, so I had to have something appropriate for that. General hanging out needs also apply.

My Bag for BreizhCamp

  • 6 tshirts (2 blank grey, 4 GitHub tshirts)
  • 2 collard Shirts
  • 1 pair of Jeans
  • 4 socks
  • 2 pairs of shoes (One pair of loafers, one pair of Tom’s)
  • 6 pairs of unmentionables

I took considerably less for this trip and was considerably happier for it. While I try to make it to the gym it’s so tough while travelling it rarely happens, and ditching that space and weight made everything much more comfortable travelling wise. I even had some spare room in my bag. Same thing with minimizing on shoes. Two pairs covered basically all my needs.


For both of these two weeks I packed a single bag; a 2010 North Face Surge I backpack (the model before this one). This was my primary travel bag.

In both cases I also included a small, fraying, soft sided tweed shoulder bag from the Old Navy to use walking around. This is great because I can roll it up and either lash it to my backpack or put it in with my clothes.


Honestly this is a tough one. In 95% of hotels a certain set of things (like soap and shampoo) are always included, yet things like toothbrushes never are. Not sure where the line is. I tend to pack a small plastic bag with:

  • Toothbrush and toothpaste
  • Deodorant
  • Foot Powder (For those loafers especially)
  • Body wash (Often from a hotel I visit regularly)
  • Lint roller
  • Mouth wash


Not clothing related but certainly relevant to packing for technical folks. For the FOSE trip I took a 15” MacBook Pro & iPad Mini. This is ideal for getting things done but is a decent amount of weight and basically never left the hotel room. For BreizhCamp I took my 11” MacBook Air & iPad Mini and it was great; the Air is light enough to carry all over without ever noticing and perfect for the kind of work I typically do on the road. The iPad is great for the and some reading but not much use otherwise.


  • Always carry MacBook Air vs Macbook Pro
    • Every trip I debate taking my work 15” MacBook Pro or my personal 11” MacBook Air. This needs to stop being a debate: the MBA is the ideal traveling machine. I usually get lots of things done on trips, especially writing, catching up on emails, etc; things that the MBA does just as well as the MBP.
  • Stop carrying my iPad
    • I tend to bring my iPad on trips, mostly so I have something light to carry when dragging my 15” MacBook Pro seems like too much. But with the MacBook Air I have a real computer small enough to carry around all the time.
  • Get a GoRuck GR1 (or maybe GR2)
    • I’ve thought for awhile that moving to a more travel friendly bag. The Surge has been great, but it’s divided up a bit more for a typical school day than travelling. The GoRuck is bigger, divided less awkwardly, and a bit more understated (even if it is a bit more tacticool than I’d go for). The price is a lot to get over, so I still haven’t pulled the trigger, but it seems like for a longer trip it would make a huge difference.
  • Get a PlugBug
    • Charging is one of the biggest annoyances, especially when travelling abroad. Especially if you avoid using random untrusted USB ports (which you should). The 12South PlugBug makes it easier to charge multiple devices off one outlet, especially if you only have one adapter.


  • Less is undoubtably more. Less weight & volume is more comfort, more freedom, and just a more enjoyable trip. I’m learning to take a really critical eye.
  • Get things at the destination, it’s easier than you think. Many hotel chains even have “rental” workout gear and many things are cheap enough it’s not worth the cost to pack them.
  • Having a second “walk around” bag that’s only enough to carry my MBA makes a huge difference. You don’t want a full backpack just to go to a conference with a small laptop.

Using Robots to Fight Bad Guys

At the end of last year I was invited few places (CentralPA Open Source, BSidesDFW, & BayThreat) and gave a talk about some of the work I’ve done to adapt Hubot, GitHub’s friendly-ish chatbot, and GitHub’s Chat Ops workflow for DFIR. While it was great to get the ideas out there’s a lot to deploying, using, and customizing VTR. So this is my extended breakdown of ChatOps, Hubot, Hubot-VTR, and building modules in CoffeeScript.

My Presentation

How GitHub Uses Hubot: ChatOps

So DevOps in general and ChatOps specifically are massive topics unto themselves and I’m not sure I’m the right person to fully address them, so here’s my short version of each:

  • DevOps: An integration of software development & systems operations to increase efficiency, consistency, repeatability, and security.

  • ChatOps: Doing devops by building tools that integrate with the teams chat applications.

Both of these techniques take advantage of modern development tools for rapid tool creation and integration, tools like Puppet/Chef for automation, and a data driven approach where feelings, conventional wisdom, and “that’s just the way we do it” is replaced with collaboration and metrics.


Benefits of Chat

  • asynchronous: searchable, read back, push notifications
  • multi device: laptop, tablet, & phone
  • collaborative
  • flexbile interface: an extensible and easy to use interface to build tools around

Making Hubot a DFIR Sidekick

The idea is to make Hubot support security ops:

  • open source intelligence
  • network forensics
  • system forensics (disk & memory)
  • reverse engineering
  • penetration testing & vulnerability management

Setting Up Hubot VTR

Setting up Hubot is a simple task that’s best started by looking at the Hubot documentation itself. Go ahead, we’ll wait here.

Once you’ve got a basic Hubot setup you’ll want to install the VTR scripts with these directions. It should be pretty painless.

Using Hubot VTR

Script Description Using It
Code Name Generator Generates code names for being spooky codename
Geolocate IP Identify the physical location of an IP address geo & geo maxmind
MyWOT Look up the reputation of a website mywot example.com
Pipl Look up OSINT on a users email address pipl email example@example.com
Reputation Links Generate links for Robtext, IP/URLVoid, etc reputation ip & reputation url example.com
Reverse DNS Get the urls associated with an IP address reverse dns example.com
Shodan Search engine for server strings. shodan foo
Short URL Expander Take a shortened URL and find out where it redirects to. expand url example.com
VirusTotal Hash, URLs, IP Addresses virustotal hash abcd1234, virustotal ip, & virustotal ip example.com
Yara Generates template for creating Yara rules. yara-template

These all require addressing Hubot directly, either by using the bots given name or an alias. So for me that would be hubot shodan openssh or ! yara-template. Both your Hubots name and alias can be configured during setup.

Writing Hubot Scripts

Hubot scripts are written in CoffeeScript. CoffeeScript is basically a higher level language that compiles to JavaScript and is then interpreted by Node.js.

# Description:
#   Get domains associated with a specific IP Address based on http://hackertarget.com/reverse-dns-lookup/
# Dependencies:
#   None
# Configuration:
#   None
# Commands:
#   hubot reverse dns <ip> - Gets domains associated with an IP
# Author:
#   Scott J Roberts - @sroberts

api_url = "http://api.hackertarget.com"

module.exports = (robot) ->
  robot.respond /reverse dns (.*)/i, (msg) ->
    ip = msg.match[1].toLowerCase()
    robot.http(api_url + "/reversedns/?q=#{ip}")
      .get() (err, res, body) ->
        if res.statusCode is 200
          msg.send "Reverse DNS: #{body}"
          msg.send "Error: Couldn't access #{api_url}."

There are a few major parts of any Hubot script:

  • Documentation: This is actually quite important given that this documentation is passed along into chat when people ask for help. It’s important that this be descriptive and complete.
  • The Respond or Hear Method: Hubot can listen for anyone mentioning a specific phrase (robot.hear) addressed to anyone, or wait to be addressed directly (robot.respond). In either case this is followed with a regular expression detailing what statement to listen for. Anything in () is saved and can be used in the method body.
  • The Method Body: This is where the bulk of the work happens. Call a 3rd party API, manipulate text, anything you can think of that you can do in CoffeeScript. In nearly every case the method body has one or more msg.send "Foo" statement, which has Hubot respond back to the chat.

These are the simplest requirements, but of course there are many optional patterns as well. CoffeeScript tends to be fairly simple, so the best way to pick things up is usually just browsing through open source scripts.

My Setup

I currently run two separate Hubots with different adapters and environments:

  • Development: On my laptop I have a Hubot running with the terminal adapter. Basically it just spawns a shell that acts as chat directly with Hubot. It’s not easy to use, but it’s fast for testing without having to redeploy, just restart the server. This system has no memory (Redis database), just a brain (Node.js server).
  • Production: I have a shared chat room with some other DFIR types and we have a shared Hubot. This Hubot is deployed onto Heroku, a single dyno system, and has a RedisToGo memory. We use Slack as our chat server, we it, and their Hubot integration is excellent. This is actually a pretty ideal setup, and definitely what I’d recommend for a first deploy.

Things to Build

Take a look at “The List”. There are endless possibilities, new modules, new integrations. My current goals revolve around Yara, including automating signature generation. That said make this what will help you, and pull requests are always welcome.

ChatOps Resources

Hubot & Hubot VTR

ChatOps & DevOps

A Basic Guide to Advanced Incident Response

On 5/5 was lucky enough to be invited to speak at an education technical conference Tech Talk Live Cyber Security Symposium. I wanted to do something new, something different. I’ve long been an advocate of intelligence driven incident response, but had never seen a sufficiently useful presentation to introduce this complex but powerful work flow to others. So I tried to make one.


Overall I was pleased with how the talk was received. Schools face a myriad of cyber threats from the mundane to the sophisticated, but lack resources in terms of money, people, and time. I don’t know if central PA schools are going to see booming intel driven IR programs, but I hope there were some interesting ideas for them that may be useful.

I also learned a lot, as I always do, giving the talk live for the first time. I plan on revising it, and hopefully giving an improved version.


I mentioned a lot of tools and concepts in this talk that need further exploration. Here they are:

Presentation Design

If you made it this far I also wanted to call attention that this was the first presentation I’ve fully built and gave using Zach Holaman’s Speaking.io guide. Working at GitHub with so many amazing designers I’ve become conscious of both the importance of design in presentation and the benefit of better preparation. I recommend it highly for new speakers to learn and seasoned speakers to level up.

Installing Yara 2 on Ubuntu 14.04

One of my projects last weekend was getting Cuckoo Sandbox stood up again. I hadn’t done much more than beginning to install libraries and dependencies than it called for one of my favorite security tools: Yara.

In the past I was a heavy Yara user, but I haven’t really played with it much since the upgrade to 2.0. Version 2 includes some big improvements and Yara is a valuable tool alone as well as in conjunction with a tool like Cuckoo. That said, Ubuntu is pretty sparse out of the box and the instructions for installing Yara don’t include much information about prerequisites.

Here’s the process I followed:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install python libtool automake autoconf python-dev

I downloaded the most recent release:

wget https://github.com/plusvic/yara/archive/2.1.0.tar.gz

At this point I followed through with the given instructions:

tar -zxf yara-2.1.0.tar.gz
cd yara-2.1.0
sudo make install

Things seemed fine, I tried running yara -h.

cuckoo@cuckoo-box:~/yara-2.1.0$ yara
yara: error while loading shared libraries: libyara.so.2: cannot open shared object file: No such file or directory

Well that’s frustrating:

Sherlock Holmes is super frustrated!

Not a normal error, but a quick Google for error loading shared libraries led to a Stack Overflow article named “Error loading shared libraries”. Well isn’t that lucky? Turns out just running sudo ldconfig tracked down my missing shared object, and next thing I know:

cuckoo@cuckoo-box:~$ yara
usage:  yara [OPTION]... RULES_FILE FILE | PID
  -t <tag>                 only print rules tagged as <tag>.
  -i <identifier>          only print rules named <identifier>.
  -n                       only print not satisfied rules (negate).
  -g                       print tags.
  -m                       print metadata.
  -s                       print matching strings.
  -p <number>              use the specified <number> of threads to scan a directory.
  -l <number>              abort scanning after matching a number rules.
  -a <seconds>             abort scanning after a number of seconds has elapsed.
  -d <identifier>=<value>  define external variable.
  -r                       recursively search directories.
  -f                       fast matching mode.
  -w                       disable warnings.
  -v                       show version information.

Report bugs to: <vmalvarez@virustotal.com>

For the Mac Users

If, on the other hand, you’re just looking to use Yara on your OSX client and you happen to be a Homebrew user (and why wouldn’t you be?) then you can just use brew install yara.

OSX Airport Preferences Forensics

I’ve been a huge fan of @jipe_’s OSXAuditor. In the limited field of OS X incident response tools OSXAudtior is the best quick triage tool out there. Unfortunately recently when I started working with I hit a quick snag.

OSXAuditor stack trace.

When using the -a, --all (Analyze all) or -A, --airportprefs (Analyze Airport preferences) options in OSXAuditory.py you hit a hard error, Python stack trace and all.

This is all based on the line:

PrintAndLog(u"SSID: " + RememberedNetwork["SSIDString"].decode("utf-8") + u" - BSSID: " + RememberedNetwork["CachedScanRecord"]["BSSID"] + u" - RSSI: " + str(RememberedNetwork["CachedScanRecord"]["RSSI"]) + u" - Last connected: " + str(RememberedNetwork["LastConnected"]) + u" - Security type: " + RememberedNetwork["SecurityType"] + u" - Geolocation: " + Geolocation, "INFO")

Yuck. Turns out this is attempting to parse com.apple.airport.preferenes.plist. This is the plist file that tracks stores information about a users wireless usage. Most interesting, this plist includes information about every saved wireless network a user has accessed in the RememberedNetworks array.


The issue was clear pretty quickly (you likely already have a guess as a programmer). There was a change between OSX 10.8 and 10.9, including the removal of the BSSID & RSSI dictionary keys, which was clearly breaking that PrintAndLog statement.

I submitted a Pull Request (you always submit a PR when you find broken code right?) to OSXAuditor to fix the parsing error, but I was struck by how limited the information about many of these crucial plists.

Apple has a habit of not documenting things they don’t intend for users such as private APIs, so the lack of official documentation isn’t hard to believe. Even my go to guide, Mac OS X and iOS Internals by Jonathan Levin didn’t offer any help. Here’s my limited attempt to document com.apple.airport.preferences.plist.

com.apple.airport.preferences.plist format in OSX 10.9

At it’s root this plist contains the following:

Key Datatype Use Notes
AirPortGlobalDebug Number Is the AirPort being debugged by root.  
AirPortUserLandDebug Number Is the AirPort being debugged by a user.  
RememberedNetworks Array The list of previously a accessed wireless access points. See below for RememberedNetwork item format.
Version Number    

The RememberedNetworks array contains a number of dictionaries in the following format:

Key Datatype Use Notes
AutoLogin Boolean Is the profile for this network configured to automatically connect and auth?  
Captive Boolean Is there a captive portal for this AP?  
Closed Boolean I believe this indicates if the network is in use.  
Disabled Boolean ??  
LastConnected Date Indicates the last time this AP was connected to. This seems to be an optional item.
Passpoint Boolean Did this AP support Passpoint?  
PossiblyHiddenNetwork Boolean Is this AP is broadcasting it’s SSID?  
SPRoaming Boolean Does the AP allow authentication to a roaming SP consortium?  
SSID Data Service Set Identification  
SSIDString String The wireless network name, one of the most useful items.  
SecurityType String WPA or WPA2? WEP or open?  
SystemMode Boolean Supplicant mode is System Mode?  
TemporarilyDisabled Boolean ???  

I have guesses on a few of the others, but no hard indications of their purpose.

Going Further

If you want to look into this more the best place to start is digging into the plist itself. Just go to a Terminal and run the following:

open /Library/Preferences/SystemConfiguration/com.apple.airport.preferences.plist

This will open Xcode’s plist viewer. And be sure to let me know if you figure anything out.