Loggly Q&A: Ryan Shriver Explores the Software Defined Enterprise, Cloud, and DevOps

Ryan Shriver is Director of Technology with SingleStone, an award-winning customer experience consulting firm. Ryan is also a Research Analyst with GigaOm covering the industry and topics related to DevOps, Cloud and Agile Development. Follow Ryan on Twitter: @ryanshriver.

Loggly: You’ve written about the Agile Cloud Development framework, citing cloud computing as an Agile accelerator. How do services like AWS and log management play here?

Shriver: Today time to market is really the driving factor for the cloud. The first wave of cloud computing was focused on IT ops and reducing costs with consolidation, but recently speed has become more important. For software companies, speed has always been important; now, in other industries, companies have realized that customers want to interact with them in digital channels. So to support customers there, managers want to come up with ideas and deliver to the customer as fast as possible. Companies are also creating innovation labs or R&D labs where employees have a lot more liberty in using cloud technologies. Also, marketing has a good chunk of the technology spend these days, and business people have a lot more options today than just their internal IT team. They can use software as a service tools like Loggly or they can put an entire project up on Amazon or Azure.

Loggly: Can cloud-based log analysis/management help a company be more agile?

Shriver: Yes, primarily by offloading infrastructure-related log tasks to a service and also helping teams respond quickly to problems. Traditionally apps logged to their local storage, but for highly available or scalable apps that run in the cloud, it’s better to:

• Use a scalable logging service like Loggly
• Log to a common storage space that can grow on demand (like S3 buckets in AWS)
• Or log to a database

If your app uses auto scaling or is planned to run across multiple servers, you’ll need to think through your logging architecture and related operational and auditing requirements. This is because as apps expand and shrink, it makes logs a little challenging; an individual server may get deleted, thus it’s not a great place to store a log file. Beyond providing a safer storage option, cloud-based log tools can provide some other interesting features like seamless searching across log files for diagnosing a problem. Monitoring is another area where you can see a pattern in the log files and alert someone for quicker action. Since log files contain error information and diagnostic information, you can use them to investigate a problem such as a failed customer transaction. The best solutions connect application monitoring data with log data, allowing development and operations teams to connect the dots across different data sets to see the big picture of what’s really happening.

Loggly: This is all presumably easier to do in the cloud environment?

Shriver: The service nature of cloud generally makes it easier to integrate components, but this is not always the case. On the plus side, developers and operations people don’t need to think about all the nuts and bolts. They’re not designing the mechanics of the logging framework. You can now get that using a service. That way, your engineering people can focus on solving business problems not building out infrastructure components. One thing worth mentioning is that sometimes logging is not something organizations are familiar with paying for, given the propensity of popular open source logging frameworks. So while moving to a service has advantages, expectations might have to be reset to justify the value as part of an investment.

Loggly: When did we start using the term “Software Defined Enterprise” and what does it mean in terms of change for IT departments?

Shriver: For much of the history of computing, there were software people and hardware people. Now, the three aspects of hardware—storage, computing and networking—have been virtualized, meaning you can control them with software. So the software defined enterprise is the idea that everything needed to run your apps is all done through software today. You don’t need to build your own data center; you can go to Amazon or Azure or someone else. You don’t need to submit tickets to get firewalls configured, file systems mounted and servers provisioned; you can do these through code, API’s and web portals. In the software defined enterprise, development and operations organizations operate more like an Agile software delivery organization.

Loggly: What is the intersection between software defined enterprise and DevOps?

Shriver: They are very tightly connected. Today DevOps is a loaded term that means different things to different people, but its rise is directly related to the virtualization of everything that used to be hardware and is now software. Development and operations teams have traditionally run in silos, because Ops people don’t really like changes to their environments; traditionally that’s when things break and cause problems which they had to fix in the middle of the night. Meanwhile development has always been about go faster, which means more changes, so these two groups have always been somewhat at odds. DevOps represents the idea that to get fast time to market, developers need to think more like operations and operations need to think more like developers. If you’ve got a tool like Loggly for operations people, now developers are using it too. They have a greater appreciation of when their code goes into production and something breaks, they should figure out why that happened. Conversely, operations can’t keep saying no, no, no to every change. Change is inevitable – accept it and accommodate it. The trick is to automate and streamline all the mundane tasks so we can make changes frequently and we can be more strategic with our time. The tools are important but the cultural aspect is often the hardest part.

The Loggly and SolarWinds trademarks, service marks, and logos are the exclusive property of SolarWinds Worldwide, LLC or its affiliates. All other trademarks are the property of their respective owners.