Read Nomad - our experiences and best practices

Nomad - our experiences and best practices

Hello from trivago's performance & monitoring team. One important part of our job is to ship more than a terabyte of logs and system metrics per day, from various data sources into elasticsearch, several time series databases and other data sinks. We do so by reading most of the data from multiple Kafka clusters and processing them with nearly 100 Logstashes. Our clusters currently consists of ~30 machines running Debian 7 with bare-metal installations of the aforementioned services. This summer we decided to migrate all of this to an on-premise [Nomad](https://www.nomadproject.io/ cluster) cluster.

Read the whole article ›
Read Teardown, Rebuild: Migrating from Hive to PySpark

Teardown, Rebuild: Migrating from Hive to PySpark

Machine Learning (ML) engineering and software development are both fundamentally about writing correct and robust algorithms. In ML engineering we have the extra difficulty of ensuring mathematical correctness and avoiding propagation of round-off errors in the calculations when working with floating-point representations of a number.

Read the whole article ›
Read Webpack Creator Visits trivago Headquarters

Webpack Creator Visits trivago Headquarters

It was a very special occasion for all JavaScript enthusiasts at trivago: Tobias Koppers, original creator of webpack, and now member of the webpack core team, visited us in our Düsseldorf office. Since we rely on webpack to build not only some very important internal tools, but also our core product, its importance to us cannot be overstated.

Read the whole article ›
Read Nine Nations, United in Code

Nine Nations, United in Code

Ten participants from nine countries — India, Cuba, Tunisia, England, Poland, Spain, Indonesia, Malaysia, and Brazil. Even on trivago scale, this kind of diversity was impressive.

Ten participants from nine countries — India, Cuba, Tunisia, England, Poland, Spain, Indonesia, Malaysia, and Brazil. Even on trivago scale, this kind of diversity was impressive.
Read the whole article ›
Read Efficient Image Recovery at Scale Using Amazon S3 Versioning

Efficient Image Recovery at Scale Using Amazon S3 Versioning

If you’re using Amazon Web Services, then there is a higher possibility that you’re familiar with Amazon S3. Amazon S3 ( Simple Storage Service ) is a widely used service where we can store (theoretically unlimited amount of) our data with a high availability (99.99%)[1]. That’s why we, the Visual Content team at trivago, use Amazon S3 to store the images which you see on our website and many other tools.

Read the whole article ›

We're Hiring

Tackling hard problems is like going on an adventure. Solving a technical challenge feels like finding a hidden treasure. Want to go treasure hunting with us?