1.4: It’s good to be the Warlord
Under the Mountain
The apogee of my ability to drive change with data probably came while I was living in a mud hut in Afghanistan. I was only a captain then, and I’m now about to be a colonel. In the time since I’ve commanded a battalion of over 500 people and served in headquarters that led over 25,000. Despite the rank and the scale of the units, change is usually constrained by the inertia of these organizations. However, when you live in a small valley in a corner of Afghanistan, away from the big FOBs, there’s no one else out there.1 It’s just us. This means no one’s coming to help you, but it also means no one is there to tell you ‘No’ either. I was a warrior prince with my own little fiefdom, and like the best mentors I’d served under before then, I took full advantage of the opportunity.
My detachment was given a VSO mission, which were set up in remote populations outside the reach of the Afghan government. We worked to provide security and development while building a bridge to the nearest local civic center.2 It meant living outside the wire, in a literal mud hut, among villagers who were mostly illiterate and all lacking electricity. And it was a data goldmine.
Just before we deployed, we received a brief from an obscure government office with a miserable acronym for a name. JIEDDO had been stood up to address the challenge of the roadside bombs that were killing soldiers and civilians in both Iraq and Afghanistan.3 No one had any idea how to tackle the problem, but the DoD was willing to throw money at just about anyone who might solve the problem. All sorts of vendors piled in to sell their proposed ‘solutions’. A lot of the things the DoD purchased were crap and some were straight snake oil. But, from my perspective, JIEDDO was an exception. We got a great ROI with JIEDDO, specifically the team at COIC.4,5
Barely a month before we deployed, a team from JIEDDO visited our unit in Tacoma, Washington and told us about COIC and their work with Google Earth. I’d been using Google Earth since Ar Ramadi, and was very interested in anything that leveraged the software. Programs like ArcGIS were more powerful, but Google Earth was stupid easy and it was free. The fact that it provided better quality and color imagery than the ostensibly classified black and white maps we were using on FalconView meant all of the lieutenants in Ramadi had switched to using it as soon as we had accessible internet available to us.
The work COIC did with Google Earth had two major advantages, both of which I have not seen replicated since: it gave you easy access to a ton of data streams, and it was modable.
Data Puddles into Data Streams
The same ‘throw money at the problem’ environment that spawned JIEDDO had also given rise to a myriad of competing programs all seeking to be the ‘single pane of glass’ the DoD still pines for today. Everyone wants a single app that is capable of displaying all of the data in all of the disparate databases. In 2010 programs with obscure names like TIGR Net and CIDNE were all trying.6 Some were narrow in scope, others broad. But COIC was the only team I know of that managed to tap into all of them.
COIC had a web portal where you could set a time frame, a geographic region, and then check all of the various databases you wanted to check. In 10 minutes, I was able to set up a tool which would pull the last 5 days of reports for everything that plotted within a 50km circle of my VSO team house. The file, a KMZ, wasn’t even one megabyte, since it didn’t actually hold any real data.7 It just held my query, and every time I refreshed it, the code would methodically pull the latest reports that met the criteria. That’s a data superpower, so I wanted to reinforce it. The ability to tap into more data streams alone was huge, especially since almost everything in the DoD is siloed. But being able to filter by configurable criteria was even more impactful. It’s the difference between building piles of hay and having a magnet that only grabs needles. At the time, it was revolutionary, and I have never seen any system do it that well again.
Making it Modable
COIC’s second huge innovation was the forum they setup for their users. From the outset, they set out to make their work with Google Earth modable. Government procured software is, with only the rarest exception, crap. There are a lot of reasons for this assessment, and I’ll discuss them later, but one of the biggest shortfalls is there’s no simple way to make the software better. COIC built a an easy to access forum website where any user could ask for new features or fixes. To anyone outside the military, this must sound like the easiest, lowest effort thing to do. I mean, how could you not? Welcome to the Army.
The first feature I asked for was a bearing tool. The insurgents in the valley loved to use roadside bombs, which kick up a cloud of smoke when they exploded. If two people in two different places both get a compass heading (azimuth) to the smoke, you can plot them on a map and quickly pin-point where the explosion is. It’s called intersection, and every Lieutenant in the army learns how to do it.8 You just need a compass, a map, you can do it in under five minutes. Computers can do it faster though.
Google Earth did range (distance) but not bearing, so anytime I wanted to run an intersection I had to fire up FalconView. Which. Was. Very. Slow. To. Open. Getting an intersection plotted was also the only reason I opened the software anymore, since Google Earth was faster and easier for everything else. One night, after getting frustrated waiting, I went on the COIC forum and posted an ask for a bearing feature that worked in less time than it took FalconView to open.
Three weeks later, the latest version of Google Earth had bearing. There must have been a dozen features I saw get rolled out during my ten months in Afghanistan. Google Earth did latitude and longitude, but everything Army is in MGRS.9 At first the COIC team just rolled out a widget, a KMZ you downloaded and refreshed. A few weeks later it was built right into the Google Earth app.
I was down in my tiny valley, an hour’s drive from the nearest city, and over four hours from the provincial capital. I genuinely don’t know who at COIC was making all this happen for us. I have no idea how COIC even came about. All I know is as awesome as it was, it was gone soon after. I’m not sure why.
If I had to guess, it was probably because it used free software. The DoD, or at least the contracting firms we trade with, have an aversion to free and good. I’ve seen a dozen attempts to sell the government a replacement for mIRC, the chat app that I used in Diyala as a new captain, and in Iraq later as a Lieutenant Colonel. Government contract software seems designed to try and sell us a poorer version of what we already have for free. Worse still, they set up the code so it has no APIs and no way to update it.10 Instead, you’ll have to hire a company guy to make any changes, meaning you’ll pay them twice for the privilege of fixing their shit code.
COIC was one of the first times I met other people working to leverage data to improve operations, and their work with Google Earth stands out as an exception. It’s one I would have never knew existed if the JIEDDO team hadn’t visited our unit just before we deployed. They found ways to keep integrating new data streams into a single query tool and they made sure their tools could be tweaked on the fly. Throughout the rest of my career, I’ve only encountered that sort of modability one other time.11 The COIC team even built everything on top of freely available and easy to use software, rather than building their own clunky, hard to use interface. As we set worked to map and unpack the anarchic and complex tribal world we’d just landed in, it became the backbone of how we tracked what we learned.
As a reminder, there’s a live glossary I keep updated with all the terms in case you see something unfamiliar.
Village Stability Operations.
Joint Improvised Explosive Device Defeat Organization. Joint IED Defeat Organization (JIEDDO) was founded in 2006, focused on countering the IED threat. It is now under the Defense Threat Reduction Agency (DTRA).
Return on Investment. Literally what you get back in proportion to what you spent. Your goal is always a number higher than 1.
Counter-IED Operations Integration Center
Tactial Ground Reporting System and Combined Information Data Network Exchange. Some of the DoD’s data repositories of tactical information.
Keyhole markup language, or zipped XML files formatted to display on a map.
Or is at least taught it.
Military Grid Reference System, the military’s way of turning a round sphere into rectangles.
Application Programming Interface, connections that allow data to flow in and out seamlessly from your software.
More in 2.10: The Files Are In The Computer