Loggly gives you plenty of powerful tools to search for events and filter down to the relevant results. It helps you cut through the noise in your logs, and find exactly what you are looking for.
Four Powerful Ways to Search
- Range queries can find events that exceed certain parameters, for example database responses that took longer than 100 ms.
- Regular expressions will match patterns in your logs. For example, if you have machines named app01 through app99, you can search for /app[0-2][0-9]/ to narrow the scope of your search to the first 30 nodes in that set.
- Advanced Boolean can help you look at more than one thing at a time. Because we expose the full Lucene query language, you have a huge amount of flexibility available in how you construct your queries. Using a combination of () and AND, OR, and NOT, you can construct arbitrarily complex queries that will show you log lines from multiple applications on multiple hosts.
- Filters show you the number of events for each unique value. Clicking on one of them narrows down your result set to the events containing that filter value. Furthermore, by showing you all of the structured/semi-structured fields included in your result set at a single glance, you can quickly see what’s included in your dataset and what’s not so you can search by elimination.
Before we get to the search example, we want to break down a few concepts related to searching in Loggly.
A search can be framed with four main components:
- 1. Source groups
- Logical groupings of your log data. Configured by you, based on meta-data within the event.
- 2. Search query
- Search terms that you enter into the search box.
- 3. Time range
- Use our drop down to choose a quick pick or set a custom time range.
- 4. Filters
- Use the Field Explorer panel to quickly add to your search criteria.
Each term above links to a section that describes the concept in detail. The Anatomy of the Search Screen section reviews where to find and how to set each parameter in Loggly.
Search Strategy Comparison
Structured vs Unstructured Log Searches
Loggly accepts both structured and unstructured data. Structured data, such as JSON, is automatically parsed which means that the field and value pairs are extracted from the data. There are also some types of unstructured data that we can automatically parse. When the data is parsed, you have many more options for building granular search queries. With unstructured data, you’re generally limited to full text search of your logs. The query language is explained in the next section.
Search Queries vs. Filters
As you will see below, it’s possible to build out some pretty gnarly queries in the search box with a large number of statements joined by Boolean operators. However, this is not usually the best way to build a search since long nested queries make it harder for you to spot errors or make changes. We encourage you to use the field explorer to further narrow down search results without having to change the query.
Once you’ve narrowed down your search, Charts allow you to visualize patterns in your data. You can look for activity trends over time, compare series of data over time, and more. Go to the Trends section for more detail.
Before we get into the search query language lets take a look at the Search functionality. You can get tons of information from this screen without having to touch your search query.
- Persistent Work Areas
Each tab represents a search context to aid in multi-tasking. Work areas are persistent across browser sessions and will maintain state even across computers. Single click on the tab to change the name of the tab. Once a tab is removed it cannot be reinstated.
- Saved Searches
Clicking on the star icon opens a menu that shows all saved searches and allows you to save your current search. Click to pin a saved search to your dashboard.
- Source Groups
Set up source groups to segment your data sources through a selection event meta-data. Search queries can only be executed on one source group at a time.
- Search Box
Enter your search query in this field. See Search Query Language for further detail.
- Time selection
Set either a relative or custom time frame for your search query.
- Field Explorer
Narrow down the list of available field names.
- Histogram & Event Count
The histogram will show the search results chronologically. Select any section of the histogram to zoom to that time period.
- Field list
Selecting a field will show the top three values for that field and the number of log results with those field values. The “Show all” link at the bottom right of these three values points to a modal that will show all values that show up in the search results. Selecting a field value in either window will filter all search results by that field.
Each of these tabs will present a different view of your event data. The Events tab shows the raw view of your data. Customize the display by clicking the cog. The Grid tab is a tabular view. Limit the display to specific fields. Trends is where you can create different charts and graphs based on values in your data.
- Resize histogram
Click to close, drag to resize.