Why should you retry all tests on failure? Why not? This article will not go into details, listing pros and cons of each approach. There are already enough resources on the Web about the topic, listing valid points for both opposing views. As trivago Hotel Search frontend QA team over the last years we tried to stay away from a brute-force retry policy for failures and we rather tried to execute test retries only in selected cases. Recently, when we switched to a Continuous Deployment approach for our new frontend Web application (which empowers developers to merge and release some pull requests autonomously), we faced a greater need than before for understandable and stable test results. Due to that, showing as few “red flags” as possible for the automated checks on pull requests became even more important to ensure enough confidence in test results and to avoid slowing down the software development life cycle. The requirements and the balance between deterministic results and success ratio shifted, at least in some cases.
Over the last few years, we completely refactored what was described in our previous article about how we use the ELK stack for an overview of our test automation results, but some core concepts remain valid and applicable.
Make was created in 1976 by Stuart Feldman at Bell Labs to help build C programs. But how can this 40+ year old piece of software help us develop and maintain our ever-growing amount of cloud-based microservices?
We do think that our tech blog is full of interesting things powered by our engineers' great stories. Let me take you on a journey of how we maintain the trivago tech blog from the technical perspective and how we recently automated its deployment process.
At trivago, we use a Cucumber based framework for end-to-end tests of our most important web applications. Cucumber stores test result as JSON files which can be turned into human-readable test reports.
We're a data-driven company. At trivago we love measuring everything. Collecting metrics and making decisions based on them comes naturally to all our engineers. This workflow also applies to performance, which is key to succeed in the modern Internet.
When I joined trivago a year ago, we had problems with our releases. The traffic was increasing each day. When we put the server back into the load balancer without warming up the OPcache it would die. From time to time the warmup failed silently. Our DCO (data center operations) crew had to log into the servers and restart a few processes manually. During this time every release was very intense.
I love to take complex and tedious processes and automate the pain out of them until they are reduced to three or four steps!
As part of the release team of trivago, one of our roles is to create tools that make the lives of our developers easier so that they can create amazing features for our website.
Configuration management tools have recently gained a lot of popularity. At trivago we use SaltStack to automate our infrastructure. As the complexity of configuration files and formulas is increasing, we need a fast, reliable way to test our changes.