At trivago we operate on petabytes of data. In live-traffic applications that are related to the bidding business cases we use our in-house in-memory key-value storage-service written in Java to keep data as close to calculation logic as possible.

If you’ve never heard about Memcached, it is simply a high-performance, distributed memory caching system which uses a key-value store for strings and objects. Usually, it serves for saving data originally retrieved from a database or external services. As simple as it is, it can improve the performance of your website quite a bit. The API of Memcached is very simple and accessible from most of modern programming languages. A simple example:

While our company and our application were constantly growing, we often ran into some consistency issues between code and design. Because we didn't have a design/frontend system and development guidelines to follow, our UI became cluttered and unsustainable. This caused maintenance issues, slowed down our UI Development and caused us to produce technical debt with an overcomplicated CSS/DOM structure. In the end, we were afraid of spending our time on figuring out crazy rendering bugs rather than rapidly developing new UI features. I gave a lightning talk at the beginning of 2015 to present what we had learned at the LeanUX Meetup in Düsseldorf.

At trivago we rely heavily on the ELK stack for our log processing. We stream our webserver access logs, error logs, performance benchmarks and all kind of diagnostic data into Kafka and process it from there into Elasticsearch using Logstash. Our preferred encoding within this pipeline is Google's Protocol Buffers, short protobuf. In this blog post, we will explain with an example how to read protobuf encoded messages from Kafka using Logstash.