trivago-logo

Kafka at trivago

Read How We Cut Kafka Consumer Deployment Costs by 83%

How We Cut Kafka Consumer Deployment Costs by 83%

This post walks through a layered performance investigation that cut PSE-kafka's infrastructure costs by 83% and ended a run of 19 P1 incidents.

PSE-kafka (price-search-engine-kafka) is an internal microservice. It consumes hotel price Kafka messages and pushes ads to an external Ads service, which drives real revenue for trivago. But the service had a few persistent problems.

ZhongLi Shen · · 6 min read
Read Better Log Parsing with Logstash and Google Protocol Buffers

Better Log Parsing with Logstash and Google Protocol Buffers

At trivago we rely heavily on the ELK stack for our log processing. We stream our webserver access logs, error logs, performance benchmarks and all kind of diagnostic data into Kafka and process it from there into Elasticsearch using Logstash. Our preferred encoding within this pipeline is Google's Protocol Buffers, short protobuf. In this blog post, we will explain with an example how to read protobuf encoded messages from Kafka using Logstash.

Inga Feick · · 4 min read