What is remote logging?

This is the first in a series of three blog posts describing some technologies that I use on a daily basis:

1. What Is Remote Logging?
2. How to Write Effective Logs with Loggly
3. How Do I Actually Use Loggly?

Why Everyone Should Be Logging Remotely

So, What Is Remote Logging?

Remote logging is not mysterious. You write log files to a remote computer instead of locally.  Before Loggly, this had its disadvantages. Worse than having to scrub a 200MB log file to reproduce a bug you don’t understand is having to scrub 15 of them. Before cell phones and the IoT, even scrubbing 15 of them wasn’t unreasonable. In today’s interconnected, global, real-time communications network, however, data has gotten somewhat out of control. Now we have 96 cell phone subscriptions for every 100 people,with potentially 2 billion smartphones, and (incredibly) 6.4 billion “things-that-communicate-over-IP,” including consumer products, business utilities, and specialized business devices [Source: Gartner].

Remote logging and indexing is quickly becoming the only way to debug all this code. Almost every server is virtual, so logging should be too.

Developers have a big job. Making things worse, the vast number of users uncovers many more corner cases than does explicit QA testing, unit testing, integration testing, end-to-end testing, and monkeys.  

A Brief Description of Remote Logging

A remote logging provider creates a secure, HTTPS-encrypted POST end point as a string dump. You, instead of writing logs to the console or to a log file, write all of your logs directly to this lovely post end point. The end point is a string dump. It doesn’t take a complicated list of arguments; it doesn’t require careful construction and deconstruction. It is a string dump, like this:

Remote Logging: The New Reality of Debug

Everyone is familiar with the use of debug statements to track code flow. In fact, before the popularity of graphical IDEs with their graphical stack traces, pointers into the current context of execution, and GDB-style step-over/step-through functionality, console logging was the most popular form of debugging. In my career, log-driven debugging has been largely unpopular. Now, however, Loggly and other remote logging services like it are definitely causing that to change.

Imagine you are a new developer for a medium-sized Ruby on Rails project. Your boss comes to you with a customer complaint. Data is being lost after the customer tries to save his profile. No one knows where the data is going or why it is lost. Being new to the project, you have only one reasonable approach to solving the bug: Connect to the server responsible for losing the data and read its logs until you can identify the point in time at which the bug occurs. As this is a medium-sized Ruby on Rails project, let’s say you have fewer than 10 servers and essentially no specialized tooling. Your solution: SSH into the servers one by one and read their logs until you’ve found a server that was responsible for executing the invalid code. I call this log-scrub. It can be easy — or it can be fruitless.

Now, using Loggly, it’s almost always easy. In fact, it’s obligatory. Loggly provides free, easy-to-use libraries that pipe the syslog (or any other file or pipe for that matter) from each server to a remote host. Instead of log-scrub, you as sysadmin/dev-ops/tech support simply log in to the Loggly log-in portal and attempt searches (just like we use stack overflow) until you’ve identified the problem. Every search gives you easy server disambiguation, easy search term-lookup, and a built-in set of JSON key/value indexes for categorizing your data.

Remote Logging Increases Accountability

When someone asks you a question or when they have experienced a use case that you have not, saying, “We will investigate that,” is not great accountability. Saying, “We have the logs and know on what line of code the error occurred,” is accountability.

With Loggly, you gain accountability and confidence in finding errors. If you’re asked a software question, especially one regarding a use case you haven’t seen, you have the ability to answer in specifics. Saying, “We’ll investigate that error,” versus saying, “Let me look at the logs and find the line of code where the error occurred,” is the difference between acceptable service and accountable, real-time problem solving.

Loggly improves communication in both directions on software projects. By reducing ambiguity with remote logging, developers, managers, and customers are free to communicate. When a manager is uncertain about the “solvability” of a bug or when a client isn’t confident that his or her contract employees are telling the whole truth, people tend to clam up. This can be the death knell of a small-scale software project. When you can say, “We have those logs — tell me when it happened,” it proves that you can fix it, and enables you to do so as soon as it is appropriate. Even bug triage is easier, since far fewer defect reports will become bottomless holes of productivity loss.

Clients want to make progress, but they understand errors will occur. Loggly makes fixing these errors much more reliable. One of my favorite feelings during a development cycle is talking to clients on the phone after they’ve discovered a bug. Instead of saying, “I don’t know why that occurred, but fixing it will be my highest priority!” I say, confidently, “Let’s talk through what happened, and I’ll read the logs for your device on my computer while it is happening. There’s a 90% chance we’ll identify the error and fix it while you’re on the phone with me.”

I had a bullet item for “resolve bugs professionally.” That’s right, gentle readers, this is professional software development.

Remote Logging Is the Future

Remote logging applies to IoT development, mobile development, web application development, hardware/firmware development, and most home electronics. Remote logging could become law as cars get more intelligent. Bandwidth use can be a concern with Loggly, especially for mobile users (which will be addressed in the next article “How to Log Effectively with Loggly”). Loggly, however, will always use a tiny fraction of the bandwidth in comparison to A/V streaming — negligible, meaningless bandwidth compared with the benefits of its use.

Next Time

How to remote-log efficiently, informatively, and securely using Loggly.

Read More: What Is Remote Logging? »

Start remote logging today, click to sign-up for a free 30-day log monitoring account.

About Thomson

Having received his M.S. in Computer Science at Colorado State University, Thomson now provides on-demand consulting services with Cardinal Peak, LLC near Boulder, Colorado. When he’s not building UIs, analyzing data, or auditing security protocols, he spends time adventuring and sailing the seven seas with his wife and four children. Cardinal Peak clients include Samsung, Beyond Oblivion, Yonder, Time Warner Cable, Dish Network, Ball Aerospace, Cable Labs, Covidien, Comcast, and a variety of big new customers up and coming in Internet of Things (IoT).  Follow him on Twitter: @thomcom.

The Loggly and SolarWinds trademarks, service marks, and logos are the exclusive property of SolarWinds Worldwide, LLC or its affiliates. All other trademarks are the property of their respective owners.