Title: Stress and Load Test Web Application and Dynamic AX:

Author: Toraj Khavari

Date: July 15, 2015

Objective: Outline practical solutions to stress and load test of a Web Application and Dynamic AX:

Article Body: A common thread among good system architects and engineers is finding answers to the following questions.

• How does the system behave under load and stress?
• What is the systems' virtual and physical capacity as the number of users increases?
•  When there are concurrent activities, e.g., Batch Jobs, Reports, Fail over recovery, etc., what are their effects on the End Users?
•  Are there enough analysis and facts available to share with the System Stakeholders about the system behavior?
• There are so many system items to monitor in production, what should be the monitor focus? Where are the weak points?

Although, the above questions and similar ones paralyze us with concerns, very often, stress and load tests are not performed. There are a variety of reasons, e.g., test technologies are not available, limited time between stable product and release time, budget limitation. Or, if stress and load tests are performed; they are compromised. Thereafter, real production environments will become the stress and load test laboratory. Leading to performance degradation, system patches, loss of or unhappy customers, and system modification on the fly, Also Known As (AKA), “Fire Fighting”.

In this article, practical solutions to stress and load test of a Web Application and Dynamic AX are outlined. The A-dec OrderNet and CORE System, which include Dynamic AX, are used as case studies.

Case Description:

A-dec Inc. is an international dental equipment and dental office furniture manufacturer head quartered in Newberg Oregon, USA. A-dec is in the process of releasing a new version of OrderNet and change Enterprise Resource Planning (ERP) BaaN to Microsoft Dynamic AX 2012 R2.

OrderNet is a web solution for customers to place, to configure, and to submit orders, on-line. A-dec is among very few companies providing a web solution for ordering dental equipment on-line. The system supports both standard and configured quotes and sales orders.

The A-dec ERP upgrade is more than a Microsoft Dynamic AX implementation. The solution collection, referred to as the CORE Replacement Project, CORE for short, consists of a collection of products; Microsoft Dynamics AX, and Independent Software Vendor (ISV) systems meeting the business needs. The CORE system includes a collection of interfaces, e.g., Electronic Data Interchange (EDI), Manufacturing Execution Systems (MES), Shipping Systems, Warehouse Carousel, etc. The following picture describe the scope, pictorially.

The CORE system includes integration with the A-dec data warehouse, and existing other business systems, while using infrastructure technologies, inside the firewall. The solution collection is modeled to support “A5” methodologies. It seeks to enable anyone, from anywhere, at any time, from any device to access any application.

Stress and load testing CORE solution was prioritized based on the business impacts. Although the stress and load testing were focused on OrderNet and Dynamics AX, we exercised the lower priorities interfaces and batch processes.

Load and Stress Testing Definitions:

Let us explore what is load and stress testing and define them.

“Load testing is a software testing technique used to examine the behavior of a system when subject to both normal and extreme expected load conditions.”

“Stress testing is a form of testing which deliberately stresses the application and observes its stability.”

The best analogy I can think of is cardiovascular stress testing. Under observation, doctors stress the patients and monitor the patient’s vital signs.

When the solution collection is stressed, it does not really mean to break the system. Yes, we want to know where the threshold is; however, don’t destroy the server to find the threshold. Back to our analogy, when the Doctor puts the patient under stress test, the Doctor does not push the patient to a point of catastrophic event. Watch the system under stress, and monitor the servers key stress indicators. E.g., CPU usage > 80%, memory loss over time.  The key system stress indicators will outline the weak points and trouble areas.

A-dec OrderNet Load and Stress Testing:

A-dec OrderNet enables end users to create quote, sales orders, and keep track of the order. The solution supports both standard components and configures products.

Outside the A-dec firewall, internet users interface with OrderNet using EmpowerID (Federation Services) and A-dec Portal.

The business requirements for the A-dec stress and load testing as follows.

·         100 Concurred Users

o   40 Users Create Sales Orders

o   40 Users Modify Sales Orders

o   20 Users Print Sales Order

We selected Microsoft Visual Studio Online (VSO) to instantiate / create 100 concurrent virtual users. The virtual users will use ID and Password to login to A-dec New OrderNet. The A-dec OrderNet has its own data base using SQL cluster.

We utilized on-site local computers and created the test scripts. Copied the scripts to a “Load Test Rig” in Azure – A cloud server. The VSO instantiated 100 concurrent users, ran the scripts, repeatedly, after login to A-dec OrderMet.

Let us review the sequence.

1- Created the test script in Visual Studio

2- Copied the scripts to the cloud load test rig

3- The load test web services created a few workers to instantiate 100 concurrent users

4- The agents ran the scripts

5- The VSO captures the ran time transactional data

6- The internal A-dec servers logged very specific systems information for more analysis.

7- Ran the test scripts, capture end users and systems experiences / performance.

Dynamic AX Stress and Load Testing:

The CORE solution collection contains a few Microsoft ISV solutions, including Dynamics AX. The Dynamics AX is the heart of A-dec CORE solution. Although Dynamics AX has web interface, we use and stress the application interface. A-dec internal employees access the Dynamics AX using Remote Desktop Services (RDS) sessions. We have a few dedicated RDS Servers to support Dynamics AX application users.

The A-dec Dynamics AX stress and load testing utilized AXceptance technology. AXceptance is a great product. A-dec's partner to implement stress and load testing is Armanino. The team successfully simulated 100s of concurrent virtual users.

Observations and Conclusion:

There are significant lessons learned through the stress and load testing. The following are a few highlights.

Although infrastructure architecture is out of scope of this article, let us touch on a few highlights.

•  Virtual verses physical servers – A-dec CORE systems consist of virtual and physical servers. The SQL Custer, AX AOS, and AX batch are physical servers (appliances).
• Load balancing – KEMP technology is powerful, proved very versatile, and very adequate.

ROI – A-dec’s continued commitment to quality and customer services are paramount to the company’s success. Even one dissatisfied customer has a large financial impact. In this case, the stress and load testing cost justifications received a significant Sr. management support.

Technologies and implementation partners – Discovering correct technology and implementation partner for the stress and load testing OrderNet and AX were changing. In our case, Microsoft Visual Studio Online (VSO) team is the best choice for OrderNet stress and load testing. Armanino stress and load testing team, and AXceptance technology were most matched for A-dec needs.

Commitment – There are a lot of discussion around stress and load testing. However, very few committed to them prior to production. The production environments are aimed to serve customers and not be a laboratory for testing functionality, and stress and load testing. Just commit to stress and load testing, and do them. Technologies and partners are available to bring software releases to higher quality.

The stress and load testing has been very productive. Let me share with you some highlights.

•   Identified a few transactional enhancements.
•  Improvements to end users’ experience. E.g., UI Rendering, Network Chatting, and Volume of data across Network.
• Identified the weak design spots.
• Proved the infrastructure bandwidth.
• Above all, system performance in production is no longer a mystery. 

A-dec New OrderNet and AX testing are not aimed at fault finding or challenging confidence. The goal is and will be to find a balance between stress and load testing. Identify a confidence level so that we can meet the business need based on the business check list and requirements.

Load and Stress testing bring to light two things; weakness and strength. Powerful ingredients to a successful new product deployment.

Take ambiguity out of production deployment, create a test lab, stress and load systems before production release, and commit to quality. Keep the end users happy and satisfied.

Have fun deploying solid products to production.

Toraj