.NET Troubleshooting and Analyzing

In this section, we’ll explain the role logs play in .NET application troubleshooting and debugging (Note the samples in this section are based on the standard .NET Framework, not .NET Core.) We’ll also show how to analyze your logs using various tools—and why log analysis tools should be part of every developer’s toolkit.

Logs play a key role in root cause analysis. If your application has a problem, logs can identify the cause and help you find a solution. They contain important troubleshooting data including error messages, stack traces, and memory dumps. Logs can also provide insights on trends in application behavior. While manually reading logs can work for smaller applications, large applications can generate thousands of events per second, making manual analysis impossible.

Debugging Application Problems

Application problems can result in poor performance, unreliability, and increased downtime. The longer these problems go untreated, the more likely your users will abandon your service for a competitor’s.

Resolving an application problem typically involves these steps:

1. Gather information about the problem. Collect diagnostic data that might be relevant to the problem, such as logs, stack traces, and bug reports.
2. Identify the cause. Use details in your diagnostic data to find out where and why the problem occurred. Repeat the user’s actions, search logs for relevant events, and use debugging tools to locate the problem in your code.
3. Find and implement a permanent solution. Create a fix that not only resolves the problem, but does so without introducing new problems. Avoid quick-and-dirty fixes.

Troubleshooting Common Problems in .NET

Now, let’s look at several common troubleshooting techniques using an open-source ASP.NET MVC application created by Microsoft MVP Syed Shanu. We’ll look at resolving exceptions, failed transactions, and memory usage. While the specific problems in your own application might be different, you can apply these techniques to debug problems in your own production application.

For monitoring, we added application logging via Log4Net, Windows, and IIS logging via NXLog, and shipped the logs to SolarWinds^® Loggly^®, a cloud-based log management solution. You can also manage your logs using the Windows event viewer as shown in the section on searching .NET logs. However, we’ll show below how tools like Loggly make it easier with features such as aggregating across multiple servers and more comprehensive search.

Resolving Exceptions

Many ASP.NET developers have encountered the ASP.NET error page (also called the “yellow screen of death”). It appears when an unhandled runtime exception occurs, causing the application to halt. Not only is this a jarring experience for the user, but it can divulge sensitive information about the application and provide a poor user experience.

Here’s an example error page that’s generated when our application fails to validate a string value before writing it to a database:

An unhandled ASP.NET exception (“Yellow Screen of Death”). © 2019 Microsoft Corporation. All rights reserved.

Error pages will highlight the exact cause of the error, but it will only be confusing and frustrating to end users. Instead, we can redirect users to a better designed error page and use Log4Net to send the full exception to Loggly. We’ll override ASP.NET’s Application_Error() method to log all exceptions as Fatal events.

protected void Application_Error(object sender, EventArgs e)

{

Exception exception = Server.GetLastError();

if (exception != null)

{

log.Fatal("An unhandled exception has occurred", exception);

}

}

Now, if we open Loggly and search json.level: FATAL, we’ll see the full exception message.

Viewing an ASP.NET exception log in Loggly. © 2019 SolarWinds, Inc. All rights reserved.

Using the exception.stacktrace field, we can see the problem occurs on line 110 of the RoleController.cs file when using the Create(IdentityRole Role) method. We still need to do more information gathering and debugging to find out what parameters are causing the error, but we now know exactly where to start.

To learn more about logging exceptions in .NET, read How Logging Exceptions in C# Gets You Ahead.

Failed Transactions

A failed transaction occurs when a request can’t successfully complete. If a request fails, IIS will respond with an HTTP status code of 4XX for client-side errors, and 5XX for server-side errors. If we search Loggly for json.sc-status:[400 TO 600], we can find out which transactions have recently failed, where they failed, and what the user was trying to do.

Searching for HTTP codes in Loggly between 400 and 600.

Seeing when and what types of errors are being generated is a good start, but let’s look deeper to find the root cause.

Tracing Transactions

Errors typically result from a series of actions performed by a user. When debugging, you may need to work backwards from the error to the cause. Doing this by hand is time-consuming, but tools like Loggly make it easy to trace an application’s flow across multiple log events.

IIS logs contain many request-specific details including the URL, the user agent, and even the user’s username (if available). This data is automatically appended to logs when W3C extended logging is enabled. We can use this data to trace an individual user’s path and even reproduce individual requests sent from their browser to IIS. Let’s take a closer look at the 500 error shown in the last screenshot.

Viewing a log of an HTTP 500 Internal Server Error for a specific user in Loggly.

Looking at the json.cs-method and json.cs-uri-stem fields shows this error occurred after sending a POST request to the /Role/Create URL. This looks familiar to the exception we just troubleshooted using our application logs, only now we know which user encountered the error.

For even greater traceability, Log4Net Contexts let you create and automatically append fields to log events using name-value pairs. Contexts are especially useful for web applications because they can store unique identifiers—such as a user, session, or request ID—for each request. You can then search on a specific identifier’s value to view all related logs.

Production Monitoring and Debugging Tools

Some problems require more data than logs can provide. There are a number of tools you can use for advanced debugging, including Azure Monitor for live applications, as well as Perfmon and Log Viewer for local development.

Monitoring and Debugging on Azure

Microsoft Azure is a complete cloud computing environment for running applications, databases, virtual servers, and other assets. Using Azure Monitor, you can collect and analyze logs, metrics, and other telemetry data to effectively monitor and troubleshoot cloud resources. Azure includes several solutions for searching and analyzing this data, including Application Insights for metrics and Log Analytics for logs.

For more advanced analysis, you can also stream your logs to a service like Loggly.

For more interactive troubleshooting, you can also use Visual Studio to remotely debug Azure apps. You can set breakpoints, monitor memory usage, and view running processes. However, this can lower performance and even prevent the app from responding to requests. Therefore, this method is best used on non-production deployments.

Using the Performance Monitor

Windows includes a performance monitoring tool (Perfmon.exe) that runs on Windows Server or desktop. It displays real-time metrics of .NET applications including CPU usage, memory usage, and compiler performance. You can view combined metrics for an entire system, as well as metrics for a specific .NET application instance.

To start Perfmon.exe, open the Start menu, type “Performance Monitor,” then click the first result. By default, Perfmon.exe shows CPU usage for the host. Let’s say we want to monitor our application’s memory usage. Click the “Add” button in the toolbar and expand the “Available counters” drop-down. Expand “.NET CLR Memory,” then select “# Bytes in all Heaps.” In the Instances list, select the process that your application is running under. Click Add, then click OK.

In this screenshot, we’ve added two additional metrics: Total committed Bytes (in green), and Total reserved Bytes (in blue). We then started the application around 2:09:50 PM, which is indicated by the sudden increase and stabilization in all three metrics.

Monitoring the performance of an ASP.NET application. © 2019 Microsoft Corporation. All rights reserved.

These metrics can be accessed from within .NET, making it possible to include metrics in your logs.

To monitor your servers remotely and see aggregated metrics across your entire cluster, use an application performance monitoring (APM) solution such as SolarWinds^® AppOptics^®.

Logger and LogViewer

Logger and LogViewer are local Windows debugging tools that record every function and API call made by an application.

Logger is a lightweight debugger that attaches itself to a process and outputs calls to a file on the desktop. The file lists each function call with the time it was performed. You can choose which types of events to log, as well as which modules to include or exclude from logging. You can then open this file in LogViewer to trace the application flow, search for specific calls, or export the file to plain text.

Logger’s main limitation is that it can’t attach to existing processes, but must start the application itself. It can also cause programs to behave differently or crash unexpectedly. However, it’s useful as a standalone debugger for quickly troubleshooting applications.