Hey everyone, my company is trying to explore a way to run load tests and we’ve tried setting up a cluster in GKE to run remote distributed testing. Before we dive head-first into this strategy though, we wanted to make sure we didn’t overlook any of the services that might be able to make the whole process easier.
Currently we’ll use JMeter scripts, so we are leaning towards services that might support that, but I’m open to exploring services that might require scripting or an API to run tests.
Here’s what I’ve looked at so far but don’t have much experience to compare between them.
GKE Cluster using our own configurations - higher side with costs but we can control more of the whole process and aren’t limited to anything for most part, as long as we are willing to pay.
Redline13 - seems to be one of the most affordable options and was pretty easy to jump into. Uses AWS so there aren’t really any limits to run time or # of users.
Blazemeter - has the most limitations, but I doubt I would hit the upper ceilings before needing to talk to a Sales team for increased limits.
Flood.io 1 - looks like they use virtual hours to bill usage, which makes it a little hard at the moment to figure out how much it costs.
Thanks, any feedback or corrections on my current findings are greatly appreciated!
Well it kinds of depends on your use case… How many requests per second , http or other protocols required. How complex is the test scenario. Where are the servers hosted.
In most cases my advice right now is to avoid clustering:-
Clustering is not needed for load scaling… WRK will do 50,000 requests per second. K6, 10,000 per second on moderate hardware.
Clustering adds significant complexity even with tools that support it.
I work for a large SaaS company all our services together globally can’t handle than many requests. If you really did need more traffic than that, then network architecture and cost would be more important than toools.
So I’ve been moving teams from JMeter to K6 because it works better with version control and continuous integration. Feedback so far is good. We deploy them to out own docker clusters locally as well as AWS.
Advantage to self hosting is lower cost but also better control over CI integration.
The one caveat is k6 is open source with commercial backer. Currently clustering is not full supported in the opensource version, so if you really need big loads you either pay or it, or build your own.
Load and performance testing helps an organization to find out if they are prepared for high sales and if you can guarantee high quality services to your clients. It is a non-functional software testing approach that examines how well is application’s stability, speed, scalability, and responsiveness hold up under a particular amount of workload. Due to this reason it is very crucial for an organization to select correct load and performance testing services as per their requirement. There are lots of load and performance testing tools right now but being a software engineer myself I can sum up some of these:
WebLoad : It is cloud based or on-premise solution. Very extensible and compatible. It creates a manageable test environment.
LoadNinja : It is used At-scale browser load testing. Create a load test and playback feature without scripts. Debugs in real time.
LoadView : It has real-time web browser testing. Creates test scripts without coding. Supports Java, Flash, HTML5, Ruby, Silverlight, and PHP.
StresStimulus : Generates multiple load generators in cloud or on-premise. The tool supports three scripting languages.
Apache JMeter : Multiple load injectors handled by single controller. Load testing infrastructure is not required. It has user friendly UI.
SmartMeter.io 1 : Builds non-scripted test environment. Extractor of advanced response bodies. GUI test runs and real-time results.