Open Source Data Science

A New Functional Approach to Complex Types in Apache Hive

Posted on

When faced with the challenge to store, retrieve and process small or large amounts of data, structured query languages are typically not far away. These languages serve as a nice abstraction between the goal that is to be achieved and how it is actually done. The list of successful applications of this extra layer is long. MySQL users could switch from MyISAM to InnoDB or use new algorithms like Multi-Range-Read without a change to their application.

Read the post
Monitoring Backend

Nomad - our experiences and best practices

Posted on

Hello from trivago’s performance & monitoring team. One important part of our job is to ship more than a terabyte of logs and system metrics per day, from various data sources into elasticsearch, several time series databases and other data sinks. We do so by reading most of the data from multiple Kafka clusters and processing them with nearly 100 Logstashes. Our clusters currently consists of ~30 machines running Debian 7 with bare-metal installations of the aforementioned services.

Read the post
Data Science

Teardown, Rebuild: Migrating from Hive to PySpark

Posted on

Machine Learning (ML) engineering and software development are both fundamentally about writing correct and robust algorithms. In ML engineering we have the extra difficulty of ensuring mathematical correctness and avoiding propagation of round-off errors in the calculations when working with floating-point representations of a number. As such, ML engineering and software development share many challenges… and some of the solutions to these. Concepts like unit testing and continuous integration rapidly found its way into the jargon and the toolset commonly used by data scientist and numerical scientist working on ML engineering.

Read the post

We're Hiring

Tackling hard problems is like going on an adventure. Solving a technical challenge feels like finding a hidden treasure. Want to go treasure hunting with us?

View all current job openings
Frontend Engineering Culture

My Journey to trivago

Posted on

Sometimes, when I look back over the time I have spent working at trivago, I see how it changed my life and how lucky I have been to get the chance to work among this amazing community, to live and to learn with them. I look back and see a younger version of myself looking desperately for something different and, by just sheer luck, getting it.

Read the post
Engineering Culture Frontend

JavaScript Workshop with Kyle Simpson

Posted on

It has been a couple of months since Kyle, author of the “You Don’t Know JS” series, visited our trivago headquarters and gave us a wonderful JavaScript workshop. It was such an enjoyable week, being able to meet Kyle in person and walk through his thinking behind the materials. trivago hosts internal workshops and meetups regularly and offers numerous learning opportunities to the employees throughout the year. This time, we decided to do it a little differently.

Read the post