Hello from trivago's performance & monitoring team. One important part of our job is to ship more than a terabyte of logs and system metrics per day, from various data sources into elasticsearch, several time series databases and other data sinks. We do so by reading most of the data from multiple Kafka clusters and processing them with nearly 100 Logstashes. Our clusters currently consists of ~30 machines running Debian 7 with bare-metal installations of the aforementioned services. This summer we decided to migrate all of this to an on-premise [Nomad](https://www.nomadproject.io/ cluster) cluster.
Machine Learning (ML) engineering and software development are both fundamentally about writing correct and robust algorithms. In ML engineering we have the extra difficulty of ensuring mathematical correctness and avoiding propagation of round-off errors in the calculations when working with floating-point representations of a number.
I’m a product designer at trivago, and would like to share some insights into one of our biggest projects we tackled in the last period. Well, from a product design perspective at least.
I’m Behrang Yarahmadi from Iran and I’m a 3rd-year Computer Engineering student at the University of Duisburg-Essen.
Sometimes, when I look back over the time I have spent working at trivago, I see how it changed my life and how lucky I’ve been to have the chance to work with this amazing community, to live and to learn with them. I look back and see a younger version of myself looking desperately for something different and, through sheer luck, getting it.
Test, test, test. If you don’t, an issue is bound to crop up in production sooner or later.
We’ve all heard this mantra in one form or another. The importance of testing your software has been covered by countless articles, books and conferences. You worked hard on your code coverage and your downtime due to regression-related bugs has severely decreased.
Ten participants from nine countries — India, Cuba, Tunisia, England, Poland, Spain, Indonesia, Malaysia, and Brazil. Even on trivago scale, this kind of diversity was impressive.
These were the software developers who were selected for the trivago Tech Camp 2018, an eight-day event taking place at the trivago campus in Düsseldorf, Germany. The event is aimed primarily at IT students, but the admission rules are not terribly strict — basic-to-intermediate coding and problem-solving skills suffice, and many candidates sent in code samples which were so advanced that we were quite impressed. In the end, we also had a physicist on board.
If you’re using Amazon Web Services, then there is a higher possibility that you’re familiar with Amazon S3. Amazon S3 ( Simple Storage Service ) is a widely used service where we can store (theoretically unlimited amount of) our data with a high availability 99.99%. That’s why we, the Visual Content team at trivago, use Amazon S3 to store the images which you see on our website and many other tools.
Technology keeps getting better and better which, at some point, makes us think "Should I migrate to the latest version/technology or not?" Well when you decide to use a better technology for your application, you have to also consider rewriting the code that your application runs on. The business logic remains the same in most of the cases but the data model would definitely change if you are switching from SQL to some NoSQL Technology for example.
Imagine a world without open source software. Pretty scary, isn't it?
There would be no free operating systems that let you take full control of your computer.
Almost six months ago, our team started the journey to replicate some of our data stored in on-premise MySQL machines to AWS. This included over a billion records stored in multiple tables. The new system had to be responsive enough to transfer any new incoming data from the MySQL database to AWS with minimal latency.
On the 11th of November 2017 trivago hosted a Hackathon at NASDAQ's MarketSite in New York. The winners were awarded with a trip to Europe to visit trivago's offices as well as conference tickets for The Next Web, Amsterdam. We invited the winners to our office where we got the chance to interview them about their Hackathon experience.