Title: Stress and Load Test Web
Application and Dynamic AX:
Author: Toraj Khavari
Date: July 15, 2015
Objective: Outline practical
solutions to stress and load test of a Web Application and Dynamic AX:
Article Body: A common
thread among good system architects and engineers is finding answers to the
following questions.
- How does the system behave under load and stress?
- What is the systems' virtual and physical capacity as the number of users increases?
- When there are concurrent activities, e.g., Batch Jobs, Reports, Fail over recovery, etc., what are their effects on the End Users?
- Are there enough analysis and facts available to share with the System Stakeholders about the system behavior?
- There are so many system items to monitor in production, what should be the monitor focus? Where are the weak points?
Although, the above questions and similar ones paralyze us
with concerns, very often, stress and load tests are not performed. There are a
variety of reasons, e.g., test technologies are not available, limited time
between stable product and release time, budget limitation. Or, if stress and
load tests are performed; they are compromised. Thereafter, real production
environments will become the stress and load test laboratory. Leading to
performance degradation, system patches, loss of or unhappy customers, and
system modification on the fly, Also Known As (AKA), “Fire Fighting”.
In this article, practical solutions to stress and load test of
a Web Application and Dynamic AX are outlined. The A-dec OrderNet and CORE
System, which include Dynamic AX, are used as case studies.
Case
Description:
A-dec Inc. is an international dental equipment and dental
office furniture manufacturer head quartered in Newberg Oregon, USA. A-dec is
in the process of releasing a new version of OrderNet and change Enterprise Resource
Planning (ERP) BaaN to Microsoft Dynamic AX 2012 R2.
OrderNet is a web solution for customers to place, to
configure, and to submit orders, on-line. A-dec is among very few companies
providing a web solution for ordering dental equipment on-line. The system
supports both standard and configured quotes and sales orders.
The A-dec ERP upgrade is more than a Microsoft Dynamic AX implementation.
The solution collection, referred to as the CORE Replacement Project, CORE for
short, consists of a collection of products; Microsoft Dynamics AX, and Independent
Software Vendor (ISV) systems meeting the business needs. The CORE system
includes a collection of interfaces, e.g., Electronic Data Interchange (EDI),
Manufacturing Execution Systems (MES), Shipping Systems, Warehouse Carousel,
etc. The following picture describe the scope, pictorially.
The CORE system includes integration with the A-dec data
warehouse, and existing other business systems, while using infrastructure
technologies, inside the firewall. The solution collection is modeled to
support “A5” methodologies. It seeks to enable anyone, from anywhere, at any
time, from any device to access any application.
Stress and load testing CORE solution was prioritized based
on the business impacts. Although the stress and load testing were focused on
OrderNet and Dynamics AX, we exercised the lower priorities interfaces and
batch processes.
Load and Stress
Testing Definitions:
Let us explore what is load and stress testing and define
them.
“Load testing is a software testing technique used to examine
the behavior of a system when subject to both normal and extreme expected load
conditions.”
“Stress testing is a form of testing which deliberately
stresses the application and observes its stability.”
The best analogy I can think of is cardiovascular stress
testing. Under observation, doctors stress the patients and monitor the
patient’s vital signs.
When the solution collection is stressed, it does not really
mean to break the system. Yes, we want to know where the threshold is; however,
don’t destroy the server to find the threshold. Back to our analogy, when the
Doctor puts the patient under stress test, the Doctor does not push the patient
to a point of catastrophic event. Watch the system under stress, and monitor
the servers key stress indicators. E.g., CPU usage > 80%, memory loss over
time. The key system stress indicators
will outline the weak points and trouble areas.
A-dec OrderNet
Load and Stress Testing:
A-dec OrderNet enables end users to create quote, sales
orders, and keep track of the order. The solution supports both standard
components and configures products.
Outside the A-dec firewall, internet users interface with
OrderNet using EmpowerID (Federation Services) and A-dec Portal.
The business requirements for the A-dec stress and load
testing as follows.
·
100 Concurred Users
o
40 Users Create Sales Orders
o
40 Users Modify Sales Orders
o
20 Users Print Sales Order
We selected Microsoft Visual Studio Online (VSO) to
instantiate / create 100 concurrent virtual users. The virtual users will use
ID and Password to login to A-dec New OrderNet. The A-dec OrderNet has its own data
base using SQL cluster.
We utilized on-site local computers and created the test
scripts. Copied the scripts to a “Load Test Rig” in Azure – A cloud server. The
VSO instantiated 100 concurrent users, ran the scripts, repeatedly, after login
to A-dec OrderMet.
Let us review the sequence.
1- Created the test script in Visual Studio
2- Copied the scripts to the cloud load test rig
3- The load test web services created a few workers to instantiate
100 concurrent users
4- The agents ran the scripts
5- The VSO captures the ran time transactional data
6- The internal A-dec servers logged very specific systems
information for more analysis.
7- Ran the test scripts, capture end users and systems
experiences / performance.
Dynamic AX
Stress and Load Testing:
The CORE solution collection contains a few Microsoft ISV
solutions, including Dynamics AX. The Dynamics AX is the heart of A-dec CORE
solution. Although Dynamics AX has web interface, we use and stress the
application interface. A-dec internal employees access the Dynamics AX using Remote
Desktop Services (RDS) sessions. We have a few dedicated RDS Servers to support
Dynamics AX application users.
The A-dec Dynamics AX stress and load testing utilized
AXceptance technology. AXceptance is a great product. A-dec's partner to
implement stress and load testing is Armanino. The team successfully simulated
100s of concurrent virtual users.
Observations
and Conclusion:
There are significant lessons learned through the stress and load
testing. The following are a few highlights.
Although infrastructure architecture is out of scope of this
article, let us touch on a few highlights.
- Virtual verses physical servers – A-dec CORE systems consist of virtual and physical servers. The SQL Custer, AX AOS, and AX batch are physical servers (appliances).
- Load balancing – KEMP technology is powerful, proved very versatile, and very adequate.
ROI – A-dec’s continued commitment to quality and customer
services are paramount to the company’s success. Even one dissatisfied customer
has a large financial impact. In this case, the stress and load testing cost
justifications received a significant Sr. management support.
Technologies and implementation partners – Discovering
correct technology and implementation partner for the stress and load testing
OrderNet and AX were changing. In our case, Microsoft Visual Studio Online
(VSO) team is the best choice for OrderNet stress and load testing. Armanino stress
and load testing team, and AXceptance technology were most matched for A-dec
needs.
Commitment – There are a lot of discussion around stress and
load testing. However, very few committed to them prior to production. The
production environments are aimed to serve customers and not be a laboratory
for testing functionality, and stress and load testing. Just commit to stress
and load testing, and do them. Technologies and partners are available to bring
software releases to higher quality.
The stress and load testing has been very productive. Let me
share with you some highlights.
- Identified a few transactional enhancements.
- Improvements to end users’ experience. E.g., UI Rendering, Network Chatting, and Volume of data across Network.
- Identified the weak design spots.
- Proved the infrastructure bandwidth.
- Above all, system performance in production is no longer a mystery.
A-dec New OrderNet and AX testing are not aimed at fault
finding or challenging confidence. The goal is and will be to find a balance
between stress and load testing. Identify a confidence level so that we can
meet the business need based on the business check list and requirements.
Load and Stress testing bring to light two things; weakness
and strength. Powerful ingredients to a successful new product deployment.
Take ambiguity out of production deployment, create a test
lab, stress and load systems before production release, and commit to quality.
Keep the end users happy and satisfied.
Have fun deploying solid products to production.
Toraj
