Like any good customer-focused company, Neotys strives to provide a valuable service to its users. As part of our continued focus on delivering and supporting the most up-to-date and appropriate load testing and performance testing technology solutions, we are excited about WebSocket protocol. The following offers a great perspective on the technology, it’s beginning, and how one company has successfully integrated WebSocket testing into its larger load testing process. We thank INXPO for their support, partnership, and, particularly, Corey Miller, Web Developer, for his participation and co-authorship.
The Client-Server Handshake Starts with WebSocket
Introduced as computer communication protocol nearly ten years ago, WebSocket was intended to usher in native browser methods for pushing content from the server to client. Classically, browser operations are initiated by the client – the browser downloads a new web page only when the user clicks on a new link, for example.
However, some web applications work best when the server is capable of initiating contact with the client directly. Chat rooms and real-time games are two examples, but now nearly every modern app uses some form of push notification or real-time messaging to improve its user experience. Sadly, the HTTP protocol was never designed to accommodate such server-driven initiation, so developers created their own workarounds, including polling techniques and keeping expensive connections open for long periods of time. These workarounds often result in hard-to-scale apps and services or unreliable performance in production environments.
WebSocket: A Standard Approach to Two-way Communication
A WebSocket connection starts off just like a simple HTTP connection, but instead of exchanging a single, bidirectional payload, the connection upgrades itself with the WebSocket protocol to maintain an indefinite connection. This allows both server and client to hand-off payloads back and forth at any time. It is analogous to a phone call, during which either party can converse freely without hesitation or delay.
WebSocket provides a few advantages over HTTP:
- Bilateral communication. Enables messages to originate from either the client or the server.
- Bandwidth efficiency. HTTP requires header information for every message, thus increasing request size/response time. WebSocket, on the other hand, require header information for the connection handshake only. The subsequent message is 100% payload-based, which can significantly reduce total bandwidth used.
- Establishing a connection takes time. By preserving the connection, WebSocket payloads can travel to and from without repeated handshakes.
WebSocket can be a valuable time and speed champion, but what is it like to load test with it? Fortunately, we have firsthand accounts like that of INXPO, who can answer the question, and are happy to share.
WebSocket Promotes Competitiveness for INXPO
INXPO is a leading enterprise-serving video platform company. Years ago, after recognizing the impending death of Flash, INXPO chose WebSocket to backbone their next-generation video conferencing platform, INXPO Studio.
In a perfect world, virtual audiences would hear a speaker as if they were in the same room. In reality, both audience and speaker must deal with network latency (time between speaker delivery and recipient hearing). Synchronizing all webcast elements in order to present each viewer with a live experience requires multi-event timing complexity. WebSocket simplified this process significantly by allowing the server to actively push messages to clients rather than waiting for the client to poll for updates.
Corey Miller was part of the team who built the NeoLoad scripts used to load test INXPO Studio. He recently presented some of his experiences and best-practice considerations for WebSocket testing at one of our Ramp Up events, and I am excited to share his perspective. The following are Corey’s findings in his own words:
INXPO’s Load Testing with WebSocket
With the first version of our new webcasting product in testing, the team’s initial goal was to simply test whether the product worked. If we simulate a few hundred users via a NeoLoad script, then we wouldn’t need to tie up man-hours by having lots of employees logging on and off at the same time just so that we could test basic functionality. That done, it only took a few modifications to transform a user testing script into a load testing script to query the limits of our webcasting platform. Thus, we transformed from a company that load tested with typical HTTP scripts to one that could synchronize WebSocket communications within our scripts. Here are just a few of the many lessons we learned along the way:
- Understanding your communications is the first step: Modeling a WebSocket application in NeoLoad requires a different approach vs. modeling simple HTTP requests. For example, after NeoLoad processes a recording, the WebSocket connection request hosts all push request messages (the order received), but provides no indication relative receipt time, whether synchronous or asynchronous and/or triggered. It is for us to synchronize WebSocket traffic with other, standard HTTP traffic.
It was critical for us to copy all WebSocket incoming/outgoing payloads into a separate tracking document (we used Excel) so we could review and analyze. If the WebSocket traffic is in clear text, this is fairly straightforward. However, if packets are condensed or encrypted, it can be very challenging.
Because we developed both our product platform and load testing scripts in-house, accessing the product developers for clarification as to what various packets meant was easy, which saved a lot of script build time.
- Build the script around the WebSocket Channel: The central component of our WebSocket script is the WebSocket Channel transaction. The rest of the script essentially amounts to HTTP request logging, the passing of information from the server to client in order for them to produce the WebSocket request.
Configuring the WebSocket Channel transaction was surprisingly simple using NeoLoad. Inside the Request headers tab of the Advanced Parameters window, we formatted the channel to mirror what we had on our system. Of note, some of the parameters, such as Sec-WebSocket-Key, are automatically configured by NeoLoad during path execution.
- Understand how to route packets to the actions that can handle them: Attached to each WebSocket Channel action are a set of Push Message requests. If you have not used these requests before, think of them as a combination of NeoLoad’s Switch and Fork actions.
When a WebSocket packet comes down from the server, NeoLoad uses the conditions defined within the various push messages to determine which transaction will handle the message assignment. It then creates a new thread to traverse the tree contained within that push message in the same manner as it would a fork action. If the end of the tree is reached, the thread simply expires. The end result – a WebSocket Channel action combined with a set of Push Messages (acts like operator – receives the message, determines the right recipient and delivers the message (contents of Push Request)).
- Modeling WebSocket requires the use of threading: Up to this point, I haven’t touched upon the main thread which still needs to be addressed. Once the main thread creates the WebSocket connection via a WebSocket Channel transaction, it simply proceeds to the next transaction, exactly as it would if it encountered a Switch transaction in which none of the cases matched. If allowed to proceed to the End container, the user path will simply end and the virtual user will cease to operate regardless of any actions it may be undertaking. It is critical to construct a mechanism to prevent the main thread from completing until you want it to.
There are several viable approaches. Our preferred method is to use a While action with a Delay or Wait Until action inside so that the loop iterates every ten seconds. We can easily add actions within the loop to debug or simulate user behavior during the eventual webcast. A good example of this use case would be to have users send chats out at a particular rate. To accomplish this, we need only add a chat transaction with a “percent chance” of execution within our loop.
- The payoff is finding problems before your users do: Once the script was built, it was time to test. As the number of virtual users climbed into the tens of thousands, it revealed all sorts of issues we were able to address before they could impact our clients.
One notable finding was that it did not take long before chats started to hamstring us. To combat this, we created chat box scripts simulating user chatter, and as they ramped, other areas of our webcasts experienced interruptions. As a result, we re-wrote some of the message handling procedures to prioritize operational packets over less-critical packets like chats.
Other limit thresholds were hit due to sheer message volume. Again, we first encountered this as we ramped up chatbots. For any number of chats, a number of attendees, and the average size of chats, the formula for chat size per second is given as Size*(Count/Sec)*Audience. This helped us determine when we would encounter certain limits.
If a thousand users each chat every five seconds in a chatroom with 10K users (a total rate of 200 chats/s), and chat packages average 500 bytes, that would mean our servers were uploading 500B*200 chats/s*10,000 listeners ≈ 1 GB/s. Realizing this could not scale with multiple active webcasts, we implemented several safeguards to prevent the risk. This included compressing our JSON packages significantly to minimize the overhead in each packet.
What Did INXPO Takeaway from their WebSocket Testing with NeoLoad?
- “Load testing taught us to examine all of our communications more critically to help determine which payloads absolutely had to travel via WebSocket for performance reasons vs. what could use HTTP requests (many of which routed through CDNs).”
- “… load testing results made it obvious which parts of our infrastructure needed expansion in order to enable product experience across a larger INXPO audience.”
- “WebSocket has also proven to be a boon for us, but one requiring hard testing/learning to implement.”
After putting lessons learned into practice, WebSocket testing has helped INXPO create a world-class webcasting platform – success that has fueled growth and challenge. With NeoLoad, they have been able to identify and address before ever becoming an issue for their customers.