Performance at trivago

At trivago we operate on petabytes of data. In live-traffic applications that are related to the bidding business cases we use our in-house in-memory key-value storage-service written in Java to keep data as close to calculation logic as possible.

One day, Memcached ran out of free memory. The method `get` failed and all requests went directly to the database. Of course these calls also failed under the huge load, and eventually it caused downtime for the whole trivago website. Yikes!

While our company and our application were constantly growing, we often ran into some consistency issues between code and design. Because we didn't have a design/frontend system and development guidelines to follow, our UI became cluttered and unsustainable.

At trivago we rely heavily on the ELK stack for our log processing. We stream our webserver access logs, error logs, performance benchmarks and all kind of diagnostic data into Kafka and process it from there into Elasticsearch using Logstash.