With more than 500 million viewers around the world, The Voice is one of the most successful television reality formats of our time. This groundbreaking vocal competition took The Netherlands by storm on the heels of its premiere in September of 2010. The show was such an instant success that it was quickly picked up by the US network, establishing its own name for itself. Over the last seven years, The Voice’s expansion around the world has continued at record speed.
While most are familiar with the United States’ television franchise, The Voice of Holland (TVOH) not only started the series phenomenon, it would be the first talent show in which social media was integrated into the user experience.
After seven seasons now, TVOH has watched viewership hit the millions (two to be exact). TVOH has also demonstrated significant fruitful returns through social media and mobile application engagement adoption, and quickly realized the need for continuous quality assurance.
TVOH’s ownership team contends that it will not release a product or service without fully checking under the toughest conditions. So, it should be no surprise that they recently conducted performance testing through its partnership with Computest, a leading performance testing provider, to help them determine whether their online environment (the mobile app – currently utilized by viewers in >50 countries), would be able to accommodate a peak load of two million simultaneous votes during a single live broadcast.
Unlike the watch, vote, wait for outcome hallmark of the The Voice series, I will spare any further suspense with the reveal of the record-setting testing results, first. I will then review the key test design features and technology, including Computest’s collaborative engagement with Neotys, using the NeoLoad product, that made it all happen.
The short of it. Load testing for The Voice of Holland was a roaring success. After many testing iterations (including ongoing re-tests performed based on continued optimization effort), it showed that the production environment of the TVOH’s platform processed a peak load of the two million user votes within seven seconds, and proved capable of processing up to 300,000 votes/second, continuously.
“The major challenge in conducting this performance testing was not just how to process the two million simultaneous users, it was also finding a way to simulate them. Simulation at that volume requires a great deal of capacity. This is why we chose NeoLoad. While in the past, we have often found this tool invaluable in conducting large-scale performance testing, never before, even with NeoLoad, has a test been conducted with this many simultaneous users,” said Joerek Van Gaalen, CTO with Computest (follow him on Twitter @Jgaalen).
How it was done:
- The test involved a single set-up in contact with over 850 individual systems, each of which simulating >2,400 simultaneous users. Systems were scattered across various data centers in order to track performance by geo-location.
- Testing relied heavily on the cloud to access the many disparate systems simultaneously, including the use of a large set of load generators big enough to scale with the volume and timing needs as testing evolved.
- The application itself leveraged WebSocket technology – which allows the server and application to communicate bi-directionally.
- What this means is that when a contestant walks on stage, this communication produces a notification that is sent to all TVOH users updating them that their voting window is open.
- Testing then proceeded to simulate a viewer at home who is actively participating in The Voice using NeoLoad. Each time a singer came out, all app users were once again notified directly – TVOH’s way of encouraging repeated voting interaction throughout the show.
- “Within seven seconds, a vote was cast at an instant chosen at random. This allows us to watch all two million simulated users vote at the same time, finding out right away whether errors will occur, and how long it will take to process them,” added Van Gaalen.
Computest thought beyond just managing to the synchronous user metric by considering those components required to produce the most realistic load testing scenarios at large-scale volume – something that its coordination with NeoLoad was very easy to support. This meant that things like geographic and device/browser type performance testing jointly across millions of users were critical elements associated with the realistic recreation of TVOH’s user experience.
Why NeLoad was chosen:
There is a point during every enterprise web application project when the scalability of the application under development becomes a necessary measurement requirement. Having to simulate concurrent users in the millions is definitely the critical mass which will warrant this need. By choosing to conduct its testing using NeoLoad, Computest was able to take advantage of the fast, most automated performance testing tool available. It’s important to note some of NeoLoad’s product features on display as you consider TVOH’s design and performance testing results:
- High Load Support – NeoLoad’s distributed architecture enabled the extremely high loads to the TVOH server – millions of virtual users at a time.
- Ability to generate this massive load within only a few seconds (this is a technical challenge in terms of coordinating the 850 load injectors!).
- Support of WebSocket to generate complex test scenarios.
Performance testing, as a whole, is one of the most important things you can do when building out a web or mobile application. Under the performance testing umbrella – load testing ensures that your application can handle expected and unexpected user loads. When realistically simulating user activity and monitoring infrastructure behavior, you can eliminate most, if not all, of the bottlenecks in your web or mobile applications. Load testing is an important process which you will either choose to do well, or otherwise risk the ever-important availability and performance of your application.
For more information about some of the pitfalls you could experience when such extreme load testing is conducted, or what was keeping the Computest team up at night during The Voice of Holland’s testing, join “Chef” Henrik Rexed (Neotys – follow on Twitter @Hrexed) and special guest, Joerek Van Gaalen, from Computest, in the latest episode of the Neotys Test Kitchen at https://youtu.be/MBlA-ctyzSI.