Introduction
One of the main problems with performance testing services is the overall scope and in particular applying scrutiny to an individual software component.
The first issue with scope is that if multiple components are being tested, as in a full systems performance test, it is difficult to isolate the system resources used by a given component (process). By way of example consider a Web services architecture that has Common Gateway Interfaces CGI (or asp and jsp etc) running on the same server.
When load is put on the front end (Web browser) the Web Services are called via the CGI and the resource utilization is an aggregate of both the CGI and the other Web services. This issue does go away if the components are separated out onto different servers but in most cases some shared server resource, by components, will exist.
The second issue, concerning scope, with performance testing a system, is that the complete system has to be built before a complete test can be executed through the front end (i.e. Web Service, CGI or Web Browser).
In the case of Web services both the client (usually Web browser, CGI, asp, jsp etc) and the backend data base need to be available to perform the end-to-end performance test.
Both of the issues of scope for performance testing can be addressed with mock services (stubs) and a test harness to drive the load from.
Mock Web Services.
Mock Web services are place holders for any SOA request that has to be developed. Mock services are similar to Mock objects for unit testing . There are many papers on Mock Objects, in the software testing services directory, that go into further detail.
The advantages of mock services for load testing are:-
The called service does not have to be fully complete.
The calling component (i.e. Web service) can be isolated in terms of measuring performance.
It is also easier to control the behavior of the returned values, using a Mock service, as these (canned) returned values can be defined for the given purpose.
It is important to note that this test is not a system test and other functional tests will need to be performed to validate the component but for the purposes of performance testing the mock object will be able to ‘stand in’ for the real thing.
An open source Mock service tool.
SoapUI has a web service mocking feature that allows for quick and easy building of mock services. This tool set also includes building the various responses that the anticipated calling program expects from the complete delivered web service.
The harness or driver.
Given we can isolate the server component from the Web Services; we now look to isolate that component from the client. This isolation is needed in any event in order to recreate large volumes of simulated service calls to the component under test.
If the component under test is itself a Web service then a SOAP load testing tool can be used as the driving harness.
If the component is a CGI (or other web server side component) a HTTP load driver can be used to put load on the component under test.
In either case a suitable open source load testing tool can be found.
Measuring the component under test.
When isolated there are a number of useful monitoring tools that can be used to measure the performance of the given component.
Although CPU is a useful measure to determine the overall performance of a middleware component it is essential to establish that a server side component is not leaking memory or not returning other unused resources (such as threads) back to the pool when appropriate. The first type of load test that should be performed on a middleware component, and this can be done early in the SDLC with Mock services, is stability testing.
With stability testing the memory, thread pool and CPU are monitored over an extended period, about six hours, of consistent load to determine if performance degrades over time.
One of the main problems with performance testing services is the overall scope and in particular applying scrutiny to an individual software component.
The first issue with scope is that if multiple components are being tested, as in a full systems performance test, it is difficult to isolate the system resources used by a given component (process). By way of example consider a Web services architecture that has Common Gateway Interfaces CGI (or asp and jsp etc) running on the same server.
When load is put on the front end (Web browser) the Web Services are called via the CGI and the resource utilization is an aggregate of both the CGI and the other Web services. This issue does go away if the components are separated out onto different servers but in most cases some shared server resource, by components, will exist.
The second issue, concerning scope, with performance testing a system, is that the complete system has to be built before a complete test can be executed through the front end (i.e. Web Service, CGI or Web Browser).
In the case of Web services both the client (usually Web browser, CGI, asp, jsp etc) and the backend data base need to be available to perform the end-to-end performance test.
Both of the issues of scope for performance testing can be addressed with mock services (stubs) and a test harness to drive the load from.
Mock Web Services.
Mock Web services are place holders for any SOA request that has to be developed. Mock services are similar to Mock objects for unit testing . There are many papers on Mock Objects, in the software testing services directory, that go into further detail.
The advantages of mock services for load testing are:-
The called service does not have to be fully complete.
The calling component (i.e. Web service) can be isolated in terms of measuring performance.
It is also easier to control the behavior of the returned values, using a Mock service, as these (canned) returned values can be defined for the given purpose.
It is important to note that this test is not a system test and other functional tests will need to be performed to validate the component but for the purposes of performance testing the mock object will be able to ‘stand in’ for the real thing.
An open source Mock service tool.
SoapUI has a web service mocking feature that allows for quick and easy building of mock services. This tool set also includes building the various responses that the anticipated calling program expects from the complete delivered web service.
The harness or driver.
Given we can isolate the server component from the Web Services; we now look to isolate that component from the client. This isolation is needed in any event in order to recreate large volumes of simulated service calls to the component under test.
If the component under test is itself a Web service then a SOAP load testing tool can be used as the driving harness.
If the component is a CGI (or other web server side component) a HTTP load driver can be used to put load on the component under test.
In either case a suitable open source load testing tool can be found.
Measuring the component under test.
When isolated there are a number of useful monitoring tools that can be used to measure the performance of the given component.
Although CPU is a useful measure to determine the overall performance of a middleware component it is essential to establish that a server side component is not leaking memory or not returning other unused resources (such as threads) back to the pool when appropriate. The first type of load test that should be performed on a middleware component, and this can be done early in the SDLC with Mock services, is stability testing.
With stability testing the memory, thread pool and CPU are monitored over an extended period, about six hours, of consistent load to determine if performance degrades over time.
No comments:
Post a Comment