Introduction
The Cisco Telemetry Broker celebrated its release earlier this month on April 1st. In my previous blog, The Rise of Telemetry Architecture, I discussed how the Cisco Telemetry Broker can help you develop a healthy telemetry architecture. This time around, I’ll be taking a look at what went into creating the product, how its roots in the Stealthwatch UDP Director helped to lay the groundwork for Telemetry Broker.
To do this, I wanted to provide a little more insight into what the development process for Cisco Telemetry Broker was like and take a moment to recognize the contributions and efforts of the members of the team. For this first spotlight, I had a chance to interview Sunil Amin, who was the Chief Scientist for Lancope (Stealthwatch product line as well as the UDP-Director) and discuss his work on the Cisco Telemetry Broker.
Interview
TK: Sunil, tell us a little about your background. You’ve been solving problems in the field of analytics and data science for quite some time now!
Sunil: Thanks TK. I joined Lancope back in 2003 at the dawn of the application of data science and data analytical techniques for network and computer security. Since then, there has been an explosion of products and platforms that use novel data analysis to produce interesting outcomes and the state-of-the-art has been moving rapidly. However, while the analytical techniques themselves have improved, no one has really thought about the upstream problem it presents to data acquisition and transformation.
TK: You led the team that designed and brought to market the original UDP Director back in 2006, back when it was called the Flow-Replicator. Tell us how that came to be and the story behind the product, because it speaks to the mother of invention: Necessity.
Sunil: Correct. The fundamental problem was that the data we needed to drive our analytics belonged, administratively, to another team within a potential customer’s organization. Additionally, much of the source data could only send to a single destination. So, the potential customer was at an impasse: I want to carry out data analysis on data that I can’t get my hands on. Our engineering team got creative and over a weekend, put together a piece of software that allowed the data to be “fanned” out to multiple destinations in a “transparent” way. This way, another analytics platform could be introduced without disturbing the existing consumers of the data.
TK: So, we have UDP-Director in market since 2006, 15 years later, what changes would you say Cisco Telemetry Broker needed to address to satisfy the needs of consumers in 2021?
Sunil: Well, at that time, we were solving for a single type of data source and a single type of destination analytics platform. Now take that situation and expand it to multiple data source types and many analytics platforms. It becomes a challenge that grows exponentially in complexity and creates organizational tension as teams battle for data access and try to maintain ownership.
In addition, this conditions the vendors to be “source-based.” Because data access is so “stove piped” we try and offer all possible outcomes from a single data set, which is crazy. We need to flip the script so that analytics platforms can concentrate on the outcome and, somehow, get access to all the data types it needs.
TK: If I am a customer who already has UDP Director, what features stand out to you as a reason to migrate now, versus later?
Sunil: I think the key difference between Telemetry Broker and the UDP Director is that CTB has been designed from the ground up to be agnostic of sources and destination and really be that “horizontal” piece that brokers telemetry data and makes sure everything is flowing as it should be. Out of the box, you will see a drastically improved monitoring capability that ensures the health of the brokering layer.
In addition, we are really excited about adding our first transforms that will allow traditional network analytics platforms to consume logs from public cloud datacenters. This is a great illustration of how a telemetry brokering layer can increase the value of existing investments in analytic platforms simply by making telemetry more accessible.
TK: Getting a new product to market is exciting! I know you were hands on the entire time designing, coding, and testing, but what would you say were some of the high points of the experience? Lessons learned? Something you are most proud of?
Sunil: I am so proud of this team and the way they went about getting this new product to market. It was a very small team that was instantiated, literally, days before the world-wide lockdown caused be the COVID-19 pandemic. However, we used every tool-in-the-box, both technical and social, to create what is now a very close-knit team that is distributed over the USA and Europe. Additionally, we embraced every and all opportunities to automate what we do. This is the way our very small team was able to launch with incredibly high-quality in such a short time.
TK: Without giving away too much, what is next for Cisco Telemetry Broker. What highly valued set of features will be on the roadmap? What problem are you most excited about solving?
Sunil: I think it is no coincidence that the vision of Cisco Telemetry Broker is so compelling to our customers at this time. The explosion of data sources and analytic platforms that all want to consume that telemetry is not going to slow down anytime soon, especially, as customers look to transition to the cloud. I am really looking forward to the CTB features that support those transitions at a flick-of-a-switch!
TK: It has been a real pleasure interviewing you and even more of a pleasure working with you on Cisco Telemetry Broker! Do you have any parting thoughts you’d like to share before we wrap things up? How should our audience find you on the Interwebz?
Sunil: One of the, really high-level takeaways from this journey is that the more we think in terms of a “telemetry architecture,” the more we can drive high-quality analytic outcomes. Digitization of our businesses has produced an explosion of telemetry data that we can harness to drive better business decisions in the future. This is the feedback loop that Cisco Telemetry Broker was conceived to support.
If you’d like to connect with me, you can reach me on LinkedIn.
For more information about the Cisco Telemetry Broker, please be sure to visit http://cs.co/telemetrybroker. While you’re there, be sure to watch the demo video where Sunil provides an overview and walkthrough of the product in action!
Share:
Leave a Reply