Splunk held their 2016 annual conference, .conf, in late September, with over 5,000 people in attendance. As first-time attendees, here are some high-value points that we took away from this year’s conference, presenting in David Letterman-ish “Top Ten” style:
1. Machine Learning is coming into its’ own, for predictive analytics. We were on a sales call recently when an IT Director said “Splunk is amazing for helping me pinpoint why something occurred, after it occurred. I wish it could predict these things ahead of time!” He said this half-jokingly, as if his request was a wish that might never be fulfilled. Enter Splunk’s premium app, ITSI (IT Service Intelligence), powered by Splunk’s Machine Learning Toolkit. ITSI, via machine learning, develops baselines and trends derived from historical machine data generated in your environment. It then brings forth information that system administrators wouldn’t have known to look for, before and as events are occurring. We sat through a breakout session where Carnival Cruise Lines demonstrated their use of ITSI and frankly we were amazed by the power of the app, particularly the predictive nature of the tool. It builds a repository of what happened across multiple services, just before events occur, and uses this data to generate future Really powerful stuff.
2. Splunk is offering free 50 GB dev licenses to customers and prospective customers. Why? Splunk wants to encourage experimentation with their product; not only among potential new customers, but existing customers who have ideas for new use cases. The company is sure that once people test it against a proposed use, they’ll be hooked by its’ power and usefulness. Go to https://www.splunk.com/en_us/resources/personalized-dev-test-licenses to get your Dev/Test License from Splunk. You can read more about the Dev/Test License via Splunk’s FAQ.
3. Splunk has also introduced a new, unlimited usage license. As most people know, Splunk’s cost is tied to the volume of data ingested, and in the past, after exceeding the data cap five times, an organization’s license would essentially be “turned off” – sometimes, at inopportune times! The new unlimited usage license is designed to prevent this interruption to a customer’s operations. Splunk directed attendees to contact their sales reps, to learn more.
4. Culture is just as (if not more) important to successful operations, than just market-leading tools/platforms. We sat through a couple of break-out sessions that stressed that a collaborative culture must be attained for any organization’s technology initiatives to be truly successful. Silos in organizations limit organizational performance, and despite all of the amazing things that Splunk can do, it can’t shape the culture of your organization. Do you want to run a successful DevOps operation, for instance? There needs to be true collaboration between the Development team and Ops team, including an understanding and appreciation for each other’s roles and a shared mindset on how to work well together. The same can be said for Security; a shared set of operating principles will drive a better working relationship between developers, Infosec, Release Management and otherwise different groups.
5. Automation should be embraced by engineers, and not viewed as a threat to their jobs. Splunk, and many other tools, is bringing a high level of efficiency to technology operations, yet some techies are fearful that they are being “automated” out of a job. An alternate way to look at this is as an opportunity to allow you to go out and execute higher value If things such as provisioning and building environments, or more quickly pinpointing the root cause of a failure, can be automated, it frees up engineers to focus on just that – engineering – and spend less time focusing on mundane administrative tasks. One Splunker referred to this as having an opportunity to “amplify your brain”.
6. User Behavior Analytics (UBA) is a cool tool for Security – and doesn’t need to sit on top of Splunk. Security threats can come from external attacks as well as internal breaches. Splunk’s UBA product protects from internal threats by baselining the behavior of your employees and flagging anomalies. It’s based upon Splunk’s Machine Learning Toolkit and this product is hailed as the equivalent of hiring a unicorn known as a “Security Data Scientist” – but getting one in the software. One important thing for security engineering teams to know: because Splunk purchased this product via the company’s mid-2015 acquisition of Caspida, Splunk’s UBA product does not require to be run on top of core Splunk. It can be purchased as a stand-alone product, since it was initially designed that way.
7. It’s easy to improperly configure Splunk, and there are many “low hanging fruit” opportunities to optimize your instance of Splunk. We sat at lunch with a well-known and well-respected Splunker who gave us the example of how there are 18 different date formats to choose from and many times a Splunk Admin doesn’t normalize or standardize date fields when being ingested into Splunk. When a date field is not standardized, query times are affected. This is just one simple example, and there are many others, of easy optimization opportunities that may exist within your own environment. A Splunk health check with a Splunk consulting partner might be a way to identify these opportunities.
8. Splunkers need to remove themselves from the tech talk and be able to speak “business”. It was clear that those in attendance at .conf see the power in Splunk; most Splunkers are evangelists for the product and favor and see the benefit of expanded usage within their organizations. This bumps against the reality of those people in their organizations who hold the “purse strings” of budget. Splunkers must be able to show and portray how Splunk is improving the business and not just the technical KPI’s. A CEO is not going to be as concerned with how fast the technical team was able to troubleshoot an issue, but he/she will be concerned with “how much money was lost during a typical outage vs. how much was saved/gained due to the speed of the issues being resolved.” A great example of this was from an e-commerce company, which reduced their e-commerce outage time from 3 hours per month down to 20 minutes. Each hour was equivalent to $20,000 in revenue. Executive buy-in for expanded Splunk usage (and cost) will only be derived with business-side “speak”. Communicate things in term of business results and business impact.
9. Splunk sees the need to expand the universe of Splunk engineers in the market. All .conf attendees were awarded $5,000 worth of free Splunk training, and that credit is transferable within organizations. Strategically, Splunk sees that one of the impediments to continued growth is the shortage of Splunk expertise available to their customers. The company wants to ensure that the community of Splunk “talent” grows and removes a barrier to continued adoption and expansion.
10. Opportunities are being missed to collaborate within your own four walls. We met one woman at .conf who said that she was the only Splunk Admin employed at her company. The very next day, we met a group of three Splunkers from the same company! Albeit they were from a different division, we none-the-less found it amazing that these people had a passion and responsibility for the same technology platform, yet didn’t know one another. What opportunities do you have to network within your own four walls, even if across different groups or business units, to create your own version of “Splunk Answers”, allowing you to extend your knowledge base and reach out when you hit a wall?
.conf was a week well spent. From break-out sessions, to keynotes, to casual conversation with Splunkers in attendance, it was clear how powerful the Splunk platform is and just how much potential it has, to do for an organization. Frankly, the use cases for Splunk seem almost limitless and we were left wondering “what doesn’t Splunk do?” We’re looking forward to seeing everyone in Washington, D.C. for 2017’s .conf, as well as all of the enhancements and growth to the product and community between now and then.
About the authors:
Jim Barge and Eddie Humenik are partners at Aditum, a Splunk consulting services firm. They have a combined twenty years of experience in technology services and are proud members of the Splunk community.
Aditum is a Splunk consulting firm focused on Splunk professional services including Splunk deployment, ongoing Splunk administration and Splunk development. Aditum has a separate division that also offers Splunk recruitment and the placement of Splunk professionals into direct-hire (FTE) roles for those companies that may require assistance with acquiring their own full-time staff, given the challenge that currently exists in the market today.