Profile of a Trace3 Elite Engineer: Kevin Donahoe

SEP 10, 2018

Some would contend predicting the future can be a purely right brain exercise. With the proper amount of imagination, anyone can dream up a scenario, throw it in play, and hope for the best. However, when you add in access to historical patterns and relevant data, that prediction can cross into left brain territory, with practical science influencing pure fancy. A once only imagined scenario becomes enlightened by existing form and statistical forecast. In other words, science meets fiction to become more than science fiction. Therefore, the world of analytics can be seen as the perfect marriage of right and left brain thinking. Who best to champion such a field than the progeny of a musician and a programmer. Meet the world of Kevin Donahoe – engineer by day, musician by night, and all-around advocate of using science to shape your future.

Born in Minneapolis but raised, schooled, and shaped in sunny Southern California, Kevin comes naturally to both his artistic and scientific bent. With a musician mother and engineer father – “Dad was in computers since the beginning of time” – Kevin took to the guitar and programming at a young age. As early as sixteen, Kevin had already identified Audio Engineering as a “very interesting way to create art using science”. Entering the field under the tutelage of strong mentors, Kevin learned sound engineering from the ground up, getting his hands on any technology at his disposal. “I spent time in all the three major disciplines in audio engineering: recording studios, live sound reinforcement, and broadcast.” Live sound suited him best, and “was probably why the majority of my time in IT has been in Operations”.

During this period, tumult played a prime factor in Kevin’s professional evolution as he lay witness to the seismic shift in the music industry, where digital overtook analogue as the primary vehicle for distribution. “I was working for a company that made the onboard entertainment for aircrafts…and realized I knew more than they did about how to convert it (music) to digital and serve it electronically. And with this realization, Kevin identified “the real place to make a career was in the computer itself and not in the activity”. He dug into the system architecture, exploring solution design for the modern challenges on hand. “That’s something that’s always been my passion, more systems design using computers to do something really useful”.

With the Inflight Entertainment company and through his involvement in a number of media streaming startups, over a ten-year period Kevin developed an in-depth and practiced understanding of audio/video compression. That’s when DirecTV came calling, seeing him as a good fit for their burgeoning Video OnDemand market. “I had the background and experience to put together a complete workflow to take content, convert it to digital with really high-quality compression that still left you with a really good picture but very low bandwidth, and deliver it over a public Internet.”

Kevin helped DirecTV build their early OnDemand workflow and it snowballed from there. ““I helped them launch their first over the top streaming service, then eventually moved into operating those services. Because I built them, I knew how they worked. I was a natural fit for knowing what to do when they broke.”

By 2010, Kevin realized to further his technical range and advance his career he needed to supplement his education. He earned ITIL Certification in v3 Foundation for Operational Framework, which opened a previously untapped avenue to him. “I started my journey down the Operations Analytics path, where I realized the standard way of doing things wasn’t working. We were trying to operate this service and it was a miserable failure. We were having problems right and left, couldn’t tell what was what, it was taking us a week plus to fix things, not a good situation.” By learning one of the larger Operations frameworks, Kevin and his team developed and applied a strategy, which saw immediate success. “I began to learn about the operations tools, the things that monitor IT systems and services and then also tools that monitor the network, tools that take a look at logs, all those various tools I had to get into deep.” His natural affinity to both find and attack a challenge pushed Kevin further toward his current path. “What I really got into was I recognized the need to be able to correlate all those various sources of data and correlate it to the infrastructure. I could understand where something went wrong.

“I realized there are ways to become proactive if you have enough information from the various sources about your infrastructure and services. If you’ve done a good job mapping them and creating those correlations, then you have the ability to use machine learning to start to predict when failures happen or predict when you’re going to reach an overload condition, so you can do something about those issues.”

It took a year developing the IT Operation Analytics strategy to make a major difference. Then, Kevin and his team could resolve the issues hampering DirecTV’s NFL Sunday Ticket, reducing errors by 97% the first year. “We got real value pretty quick from it and was able to take that over the course of six years at DTV and create a situation where we were actually self-healing…We could auto-scale infrastructure, we could auto-heal systems if a site goes down. Really, what happened was we focused human beings on things only human beings could do, and automation on things automation could do really well.”

By 2013, Kevin’s team produced their third iteration of their command-level dashboard, providing a clear view of how each of the core underlying services supported the customer service. 2013 also produced the first “everything went beautifully year”. Through the next couple of years, further refinements in the strategy were implemented, raising their predictive analytics to an even further level. Things were going smooth, perhaps too much so for Kevin’s innate desire to solve the next puzzle in his path. It was time for a change.

Enter Trace3. ”I had been using Trace3 at DTV, poking on their shoulder, asking had they ever seen this and that. I realized they had this amazing pool of talent who got to do really cool stuff. They got to see these really amazing use cases and challenges. I said I gotta do that.” Kevin joined Trace3 in 2017 and immediately – in his job interview, actually – began pitching an IT Operation Analytics practice, saying, “I guarantee every company has this issue where they have a proliferation of tools, no real correlation, and an unclear view of their service status, and they’re not being very successful in their Operations. And I have a plan.”

As of September of 2017, so now does Trace3. The IT Operations Analytics team was born with “a heck of a team with subject matter expertise in at least one to two tools that all have to do with Operations toolsets…Trace3 has an amazing group of Cloud engineers. We have an amazing group and I leverage them. It demonstrates to the customer it’s not these individual pockets, but an entire organization of experts brought to bear on that customer’s needs.” Now on his second large customer project, Kevin recognizes he is truly performing IT transformation. “I come in to help them make that shift from a reactive Operations stance to a proactive one.

“It’s a really exciting time in my practice because we’re getting to move forward and ramp up with our service offerings. It starts with what we call an ITOA leadership Session. We come in to a customer, learn about their business and their services, take a look at what tools they have and the staff they’re using for Operations, and get a sense for how well they’re covering the monitoring of their services and infrastructure. How effective their staff is at understanding and operating that infrastructure. Lastly, we’ll provide them analysis in the form of a gap analysis plus some recommendations. We typically offer this in a one-day session as a way to get them started on their journey to being proactive. It’s our IT Transformation Service.”

Along the way, Kevin is also redeploying the education method he benefitted from throughout his career, fostering mentorship relationships with both colleagues and customers, openly sharing the hands-on wisdom he’s acquired along the way. This, to him, forms both true partnerships and a true team. “The team makes the whole of it so much better than the individual parts…The team brings their experience together, showing what did and didn’t work to build the practice collateral to help a customer really leverage it. That’s what builds customers for life.”

The move to Trace3 and the ability to further develop his desire around IT Operations Analytics has proven fruitful. The need to both cultivate and stretch himself, both on the job and performing with the Trace3 in-house band, The Untraceables” has opened avenues and opportunities he’s been quick to take and appreciate. “These are the things that I’ve come to enjoy and the challenges that I’m after. I want to help the customer use what they have as efficiently as possible, find their gaps, fill those gaps, and come up with an efficient strategy to operate with the full set of information, and that way, work smarter not harder.” Left brain thinking with right brain desire. That’s Kevin Donahoe to a t.


Leave a Reply

Your email address will not be published. Required fields are marked *