Working with scale down environments

Approach

One of the main elements to determine the success of performance testing an application, is to verify if the conclusions obtained from the test can be applicable immediatly in the production environment.

The perfect world is when stakeholders invest in building a test environment that can match the production environment. The truth is that building such an environment is expensive, some times difficult to allocate budget and highlight value, in addition to the team that can govern successfully the environment.

So in this scenario we are facing a scale down test environment that represents only a fraction of the production environment. Let's leave apart the fact that most of the times performance test teams have to share environments with other teams, like integration, regression or the business continuity test teams.

So from this point, we know that obtaining performance test results that match exactly the production environment will be only forecasts, or estimates, based 100% in assumptions or mathematical models that can accurately make the differences in every area of the application, from physical to logical.

The mathematical model to depict the differences in every layer of your architecture, like the brand of the servers, or the speed of the CPUs, and the type of memory installed on each machine, in addition to the logical differences on data, sounds possible, but complex and definitely not pragmatic.

Humint make use of statistical analysis to resolve this problem and maximize the benefit of your small test environments.

The solution we provide, is to reduce the number of variables in the testing model and maintain as many constants as possible to identify load patterns.

We call this process Business Process Profiling and consists in creating baselines and benchmarks for isolated business processes on your application. Each profile is built using mirror scenarios with modification only in simple variables like users, transaction rate, think time and others, over time. In this way we fit valuable test in a small test environment, we produce replicable test results for analysis , we predict application behavior based on controlled tests, and finally we collect valuable feedback for the development team that highlights potential performance defects.

Loading individual business processes during earlier stages of development increments your capability to spot performance related defects and establishes the first hypothesis for performance testing in the best case scenario for your critical business processes, like: We can perform 8000 search hits without transaction failures or system crashes. Sounds like a step forward, agree?

In this way, there is consistent and statistical evidence that allows your teams to act instead of react to performance related issues, and the most important is that helps stakeholders to develop confidence around the performance not in one single test at the moment there is an environment ready for performance test. All of this by running tests in an environment with a smaller dimension than production.

Your opinion is important

Rate

Transaction Response Time Breakdown in Practice

Showing

One of the biggest challenges in performance testing is the capability for the whole team to break down the final user’s transaction response time. With final response time we refer to the composite time that the final user of your business application has to wait to receive a successful response, including server time, network time, client time, etc.

In this field there have been some great developments from 2 different angles. The first one is something external to your application and within the domain of the monitoring tools. This mechanism is called in the monitor world as instrumentation, and its based in the installation of monitoring probes in your servers that collect metrics related to the resources influencing the performance of a particular transaction. +This approach is mostly used when using Virtual Machines for computation like in J2EE applications, or when the underlying the platform does not provide an integrated ARM (Application Response Measurements).

The second option is to use your application’s own ARM to collect this information following by the correlation of times in multiple servers and translation of binary files.

Each of the 2 have advantages and disadvantages, because in both cases we are facing elements that impact the performance of your application.

Your opinion is important

Rate

Transaction Response Time Breakdown in Practice

Showing

One of the biggest challenges in performance testing is the capability for the whole team to break down the final user’s transaction response time. With final response time we refer to the composite time that the final user of your business application has to wait to receive a successful response, including server time, network time, client time, etc.

In this field there have been some great developments from 2 different angles. The first one is something external to your application and within the domain of the monitoring tools. This mechanism is called in the monitor world as instrumentation, and its based in the installation of monitoring probes in your servers that collect metrics related to the resources influencing the performance of a particular transaction. +This approach is mostly used when using Virtual Machines for computation like in J2EE applications, or when the underlying the platform does not provide an integrated ARM (Application Response Measurements).

The second option is to use your application’s own ARM to collect this information following by the correlation of times in multiple servers and translation of binary files.

Each of the 2 have advantages and disadvantages, because in both cases we are facing elements that impact the performance of your application.

Your opinion is important

Rate

Transaction Response Time Breakdown in Practice

Showing

One of the biggest challenges in performance testing is the capability for the whole team to break down the final user’s transaction response time. With final response time we refer to the composite time that the final user of your business application has to wait to receive a successful response, including server time, network time, client time, etc.

In this field there have been some great developments from 2 different angles. The first one is something external to your application and within the domain of the monitoring tools. This mechanism is called in the monitor world as instrumentation, and its based in the installation of monitoring probes in your servers that collect metrics related to the resources influencing the performance of a particular transaction. +This approach is mostly used when using Virtual Machines for computation like in J2EE applications, or when the underlying the platform does not provide an integrated ARM (Application Response Measurements).

The second option is to use your application’s own ARM to collect this information following by the correlation of times in multiple servers and translation of binary files.

Each of the 2 have advantages and disadvantages, because in both cases we are facing elements that impact the performance of your application.

Your opinion is important

Rate