Log Management and Analytics

Explore the full capabilities of Log Management and Analytics powered by SolarWinds Loggly

View Product Info


Infrastructure Monitoring Powered by SolarWinds AppOptics

Instant visibility into servers, virtual hosts, and containerized environments

View Infrastructure Monitoring Info

Application Performance Monitoring Powered by SolarWinds AppOptics

Comprehensive, full-stack visibility, and troubleshooting

View Application Performance Monitoring Info

Digital Experience Monitoring Powered by SolarWinds Pingdom

Make your websites faster and more reliable with easy-to-use web performance and digital experience monitoring

View Digital Experience Monitoring Info

Blog Technology

Use the Source, Loggly

By Hoover J. Beaver 29 Mar 2013

Here at Loggly we make great use of open-source software to avoid remanufacturing wheels. It allows us to employ the latest technologies to build exciting, interesting and large-scale systems quickly — all without worrying that the building blocks will forever remain a black-box. But working with open-source at such an accelerated pace requires its own set of skills. Here are five key lessons learned from our Engineering team — that might help you.

1. Assume at your own risk

It’s easy to assume that the project developers implemented a certain feature in a certain way, and before you know it that assumption becomes taken for granted. Without realizing it, one is building a system on that assumption. This is insidious and one needs to deliberately check the source and test the system before making any major decisions. Time is short at a small company like ours, but it’ll be even shorter if you have to re-implement due to an invalid assumption.
 Also, never make customer adoption assumptions in a vacuum, the Steve Jobs effect is awesome but rare. Leverage your customer community for responsive feedback.

2. Community support

Look for tools that can help debug, deploy, and manage the technology. If the tools ecosystem is healthy, then the open-source technology is probably vibrant too. Your sysadmins will thank you too, if they don’t have to build everything from scratch. And take the pulse of the community. Join the mailing lists. Do the same questions come up over and over again? And do they get answered

3. The project’s strength of vision

We at Loggly always feel better when the direction of the project aligns closely with the problems we’re trying to solve. For example, ZeroMQ is a technology with a clear purpose. Problems arise when this clear direction is missing, because the items high on your list may not be high on the project maintainers’ list.

4. Gambling on the “Killer Feature”

Sometimes you just have to take a gamble because of a “killer feature“. ZeroMQ is a good example, a technology we chose during its early days. Data-driven experiments showed that it was an order of magnitude faster and less resource intensive than the alternatives, but it was a very young project at the time we chose it. But we chose it because of its performance, even though it it was immature — but it did have a strong vision.

5. Comparative Advantages

When there are multiple contenders, choose the one that is weak on the things you want to be strong on, and strong on the things you’d rather be weak on.

This is my favorite insight from the Loggly Engineering team — it’s comparitive advantage applied to software design. Engineering is about trade-offs. Imagine you need to choose from 2 competing technologies. One has great algorithm support, but is weaker when it comes to IO. The other technology is not as strong when it comes to algorithms, but has superb IO support. So if you’re planning to design and implement novel algorithms, you’re going to become strong in algorithms — in fact you want to be strong in that area. So choose the system with the strong IO support, and focus on the algorithms yourself.

Hopefully these pointers help you when you next come to evaluating an open-source technology. We believe it shows in the adoption of our world’s most popular cloud-based log management service.

Even better, why not join us and help choose the technology for Loggly’s next generation systems?

The Loggly and SolarWinds trademarks, service marks, and logos are the exclusive property of SolarWinds Worldwide, LLC or its affiliates. All other trademarks are the property of their respective owners.
Hoover J. Beaver

Hoover J. Beaver