How can we organize the collaboration of more than a hundred developers on a wide range of topics? How could they decide about good practices in the company? Those are some questions that drove trivago to give it a try on a different structure: the guilds.
At trivago we store a subset of our realtime metric data in InfluxDB and we are quite impressed by the load it can handle. Despite all the joy, we had to learn some lessons the hard way. It is pretty easy to overload the database or the web browser by executing queries that return too many datapoints. To prevent that, we wrote Protector - a circuit breaker for Time series databases that blocks malicious queries.
While our company and our application were constantly growing, we often ran into some consistency issues between code and design. Because we didn't have a design/frontend system and development guidelines to follow, our UI became cluttered and unsustainable. This caused maintenance issues, slowed down our UI Development and caused us to produce technical debt with an overcomplicated CSS/DOM structure. In the end, we were afraid of spending our time on figuring out crazy rendering bugs rather than rapidly developing new UI features. I gave a lightning talk at the beginning of 2015 to present what we had learned at the LeanUX Meetup in Düsseldorf.
At trivago we rely heavily on the ELK stack for our log processing. We stream our webserver access logs, error logs, performance benchmarks and all kind of diagnostic data into Kafka and process it from there into Elasticsearch using Logstash. Our preferred encoding within this pipeline is Google's Protocol Buffers, short protobuf. In this blog post, we will explain with an example how to read protobuf encoded messages from Kafka using Logstash.
When using webpack to build your assets, it's only a matter of time until you wish for targeted builds.
Whether it's the output of the library you're working on (CJS, UMD, AMD, Var, etc.), or the specific feature
set (IE8 support, no IE8 support).
parallel-webpack can run those builds in parallel, thus making full use
of the multi-core processing capabilities of modern devices.
Twice a year, the trivago software developers gather to have a 2 day internal hackathon. This December saw another round of ambitious creativity, relaxed atmosphere, and good food :-)
One of our core values at trivago is fanatic learning. A great way to learn and expand your knowledge is to creatively try new technologies, or to apply well known technologies to new situations, without having to pay attention to a strict schedule or deadline. Exactly that is what roughly 120 engineers were doing on the first Thursday and Friday in December: It was internal hackathon time, meaning each developer could spend two entire days on their own projects as he or she saw fit.
The advances and growth of our Selenium based automated testing infrastructure generated an unexpected number of test results to evaluate. We had to rethink our reporting systems. Combining the power of Selenium with Kibana's graphing and filtering features totally changed our way of working. Now we have real-time testing feedback and the ability of filtering between thousands of tests, all in one Dashboard.
tCache takes a creative approach for near lock-free evictions and supports data-aware evictions. Its key features are:
Configuration of features is individual per Cache instance, by using a cache Builder:
Last weekend, the Python Hackathon Düsseldorf took place at trivago's office. Although we were only five people we had a lot of fun. I took the chance to brush up my Python skills a little bit. Also I wanted to scratch an itch that was bugging me for a long time: our housekeeping book.
You know those bugs, like, those. Where the application state dances around you like a crazed Polynesian fire dancer. Where changing the sorting order of a search in London reverts the result list back to Paris.... Seriously? Unfortunately, a lot of us are specialists in dealing with this kind of bug.
Learn how we managed to move fast and create a new Symfony application without breaking our old legacy session handling. We write to our legacy session (which is file based) from our new project which uses PDO as the session storage.
Here at trivago we write a huge number of log messages every day that need to be stored and monitored. To handle all these messages we created Gollum, a tool that enables us to conveniently send messages from multiple sources to different services. While initially only covering log messages Gollum quickly evolved to a routing framework for all kinds of data. This blogpost is a short introduction to Gollum and how we use it at trivago.