Posts about Monitoring

Read How To Get Fooled By Metrics
Backend Monitoring

How To Get Fooled By Metrics

Posted on

Metrics are one of the main building blocks in the topic of observability and we use them heavily. This story is about an incident where we tried to find and resolve a problem that we saw in these metrics. We went down a rabbit hole of potential fixes, only to discover that the metrics were correct all along.

Read the post
Read Better URL Search with Elasticsearch
Monitoring

Better URL Search with Elasticsearch

Posted on

At trivago, we generate a huge amount of logs and we have our own custom setup for shipping logs using mostly Protocol Buffers. Eventually we end up with some fields in Elasticsearch (ES) that contain partial (or full) URLs. For instance, in our specific case we store the query component of the URL in a field called query and the path component in a field named url_path. Sample values for these fields could be:

Read the post
Read The Web Performance Impact Of Lossy Network Conditions
Monitoring Frontend User Experience

The Web Performance Impact Of Lossy Network Conditions

Posted on

tl;dr: continuously monitor your CDN and origin servers on layer 3 with tools like MTR. Layer 3 issues on external middleware can have a significant impact on layer 7 web performance. In a recent rollout of a new cloud service, we monitored the impact of this service on web performance, UX and business metrics. For all cloud regions and origin servers, we had Synthetic and Real User Monitoring for our site in place.

Read the post
Read Nomad - our experiences and best practices
Monitoring Backend DevOps

Nomad - our experiences and best practices

Posted on

Hello from trivago’s performance & monitoring team. One important part of our job is to ship more than a terabyte of logs and system metrics per day, from various data sources into elasticsearch, several time series databases and other data sinks. We do so by reading most of the data from multiple Kafka clusters and processing them with nearly 100 Logstashes. Our clusters currently consists of ~30 machines running Debian 7 with bare-metal installations of the aforementioned services.

Read the post
Read Splitting a Monitoring Monolith into Separate Components
Monitoring Open Source

Splitting a Monitoring Monolith into Separate Components

Posted on

Ever heard about Microservices? Those tiny litte pieces of code that are used to split a big pile of magic into smaller pieces of magic? Well, they’re not that tiny after all and require lots of preliminary work to use them properly. Have a look at this post to hear about my journey of splitting an existing monolith written in PHP up into several microservices written in Go.

Read the post

We're Hiring

Tackling hard problems is like going on an adventure. Solving a technical challenge feels like finding a hidden treasure. Want to go treasure hunting with us?

View all current job openings