Numino Labs | Pixm Performance Testing

Overview

Pixm, a computer vision cyber security startup, protects enterprises from undiscovered, non-traditional phishing attacks.

Pixm delivers a frictionless browser solution, a browser extension that security professionals can deploy to an unlimited number of devices in minutes. It operates in the browser at the moment a link is clicked or at “point-of-click,” protecting users regardless of the channel the link is delivered. The browser extension communicates through REST APIs with a multi-tenant serverless computer vision based backend deployed on Azure cloud infrastructure to identify, record and stop targeted spear phishing attacks. The third component is an enterprise administration portal where administrators can login for management of various aspects of the installation, reports and configuration.

Our primary objective was to load test 250,000 extension users onto the product without compromising performance, stability, or user experience. We aimed to showcase our ability to handle a substantial user base while ensuring that our infrastructure remained reliable and scalable. To achieve this, we utilized the Locust tool to simulate and stress test our APIs under various workloads.

Key Infrastructure components

Our infrastructure was built on Azure cloud services to ensure high availability, scalability, and flexibility. The key components of our infrastructure included:

Azure SQL Server Elastic Pool: We leveraged Azure SQL Server Elastic Pool to optimize performance and cost-efficiency by sharing resources across multiple databases within the pool.
VMSS (Virtual Machine Scale Sets): VMSS enabled us to automatically scale our application instances based on the workload demands, ensuring seamless performance as the user base grew.
Function Apps: We utilized Azure Function Apps for serverless computing, allowing us to build and deploy scalable, event-driven applications to support different features of our product.
Application Gateway: The Azure Application Gateway acted as a load balancer and provided secure access to our product by distributing incoming traffic across multiple application instances.

Testing and Performance Analysis

During our testing phase, we conducted a comprehensive performance test to simulate the onboarding of 250,000 users generating millions of requests over a period of 10 hours of business time. The objective was to evaluate the performance and stability of our infrastructure under realistic usage conditions. We generated 200 days of data and populated it in the database. The results of our testing are as follows:

User Load: 250,000 users - We simulated a large user base by generating concurrent requests from 250,000 virtual users.
Request Generation: Millions of Requests - The 250,000 users generated millions of requests throughout the test duration.
Preconfigured Ratios: Realistic Workload Distribution - We designed the workload distribution based on preconfigured ratios to replicate real-world usage patterns. These ratios were aligned with expected user interactions and the functionality of our product.
Test Duration: 10 Hours of Business Time - The performance test was conducted over a period of 10 hours, simulating a typical business day.
Error Rate: Less than 1% - Despite the high user load and millions of requests, our infrastructure performed exceptionally well, with an error rate of less than 1%. This indicates the robustness and reliability of our system, ensuring a smooth user experience.

During the testing process, we closely monitored key performance metrics, including response times, throughput, and error rates. We made several optimizations to our infrastructure configuration and fine-tuned application components as needed to ensure optimal performance and stability.

Result

Through our meticulous testing and optimization efforts, we successfully achieved the milestone of load testing the onboarding 250,000 users using our product. Key highlights of our accomplishments include:

Scalability: Our infrastructure demonstrated exceptional scalability, effortlessly handling the increased user load as we onboarded 250,000 users. The VMSS feature allowed us to automatically scale our application instances, ensuring optimal performance even during peak usage.
Stability: Despite the significant increase in user traffic, our infrastructure remained stable throughout the onboarding process. The Azure SQL Server Elastic Pool efficiently managed the database resources, maintaining consistent performance levels.
Performance Optimization: Our thorough testing and analysis enabled us to identify and optimize performance bottlenecks, resulting in improved response times and reduced error rates. The Application Gateway efficiently distributed incoming traffic, ensuring a smooth user experience.

By leveraging the power of the Azure cloud services and employing the Locust tool for API testing, we successfully onboarded 250,000 users onto our product while maintaining exceptional performance and stability. Our infrastructure components, such as Azure SQL Server Elastic Pool, VMSS, Function Apps, and Application Gateway, played a crucial role in achieving our performance milestone. We are proud to showcase our ability to handle a growing user base and deliver an outstanding user experience. This achievement underscores our commitment to continuous improvement and the delivery of high-quality products to our customers.