Use the Source, Loggly

Here at Loggly we make great use of open-source software to avoid remanufacturing wheels. It allows us to employ the latest technologies to build exciting, interesting and large-scale systems quickly — all without worrying that the building blocks will forever remain a black-box. But working with open-source at such an accelerated pace requires its own set of skills. Here are five key lessons learned from our Engineering team — that might help you.

1. Assume at your own risk

It’s easy to assume that the project developers implemented a certain feature in a certain way, and before you know it that assumption becomes taken for granted. Without realizing it, one is building a system on that assumption. This is insidious and one needs to deliberately check the source and test the system before making any major decisions. Time is short at a small company like ours, but it’ll be even shorter if you have to re-implement due to an invalid assumption.
 Also, never make customer adoption assumptions in a vacuum, the Steve Jobs effect is awesome but rare. Leverage your customer community for responsive feedback.

2. Community support

Look for tools that can help debug, deploy, and manage the technology. If the tools ecosystem is healthy, then the open-source technology is probably vibrant too. Your sysadmins will thank you too, if they don’t have to build everything from scratch. And take the pulse of the community. Join the mailing lists. Do the same questions come up over and over again? And do they get answered

3. The project’s strength of vision

We at Loggly always feel better when the direction of the project aligns closely with the problems we’re trying to solve. For example, ZeroMQ is a technology with a clear purpose. Problems arise when this clear direction is missing, because the items high on your list may not be high on the project maintainers’ list.

4. Gambling on the “Killer Feature”

Sometimes you just have to take a gamble because of a “killer feature“. ZeroMQ is a good example, a technology we chose during its early days. Data-driven experiments showed that it was an order of magnitude faster and less resource intensive than the alternatives, but it was a very young project at the time we chose it. But we chose it because of its performance, even though it it was immature — but it did have a strong vision.

5. Comparative Advantages

When there are multiple contenders, choose the one that is weak on the things you want to be strong on, and strong on the things you’d rather be weak on.

This is my favorite insight from the Loggly Engineering team — it’s comparitive advantage applied to software design. Engineering is about trade-offs. Imagine you need to choose from 2 competing technologies. One has great algorithm support, but is weaker when it comes to IO. The other technology is not as strong when it comes to algorithms, but has superb IO support. So if you’re planning to design and implement novel algorithms, you’re going to become strong in algorithms — in fact you want to be strong in that area. So choose the system with the strong IO support, and focus on the algorithms yourself.

Hopefully these pointers help you when you next come to evaluating an open-source technology. We believe it shows in the adoption of our world’s most popular cloud-based log management service.

Even better, why not join us and help choose the technology for Loggly’s next generation systems?

The Loggly and SolarWinds trademarks, service marks, and logos are the exclusive property of SolarWinds Worldwide, LLC or its affiliates. All other trademarks are the property of their respective owners.