How to Mock Service Using SOAPUI
Content
How to mock Service using SOAPUI??
Transferring values from the request to the response:
This is straightforward scripting done at the MockResponse level; the script uses properties of the mockRequest object to get hold of the incoming request, extracts the desired values via XPath and then writes the created result to a requestContext property:
def temp="IBM"; def groovyUtils=new com.eviware.soapui.support.GroovyUtils(context) def xml=new XmlSlurper().parseText(mockRequest.requestContent) xml.breadthFirst().each{ def v=it.toString() if(it.name()=="name"){ temp=it.text(); log.info("============================================="+it.text()); } } context.session=temp+' Munna' mockOperation.setDefaultResponse("R1");
a.
Created mock service as ContextResponse with script. Here will have single response, based on request will return response. From request will take name as a message will add with"Munna" string.
b.
This is the response
c. Start the server for contextService and execute request. Find the output below:
How to mock Service using SOAPUI??
Read the response from a database:
This one requires a bit more work as we need to set up and close a database connection which we can use in our scripts. This is best done in the MockService start script as follows; import groovy.sql.Sql //def mockService=mockRunner.mockService def sql=Sql.newInstance("HostName","username","password","databaseDriver"); context.dbConnection=sql def row=sql.firstRow("select * from table")
log.info "=================="+row.get("column1"); context .responseMessage = row.get("column2"); mockOperation.setDefaultResponse("R1"); if(context.dbConnection!=null){ log.info "Closing the connection"; context.dbConnection.close() }
Aim of Script: Here response is whatever we are getting data from DB. If we want to specifically want specify any condition based on request we can add like where condition with request element value. ** Connecting data base and selecting first row. From that row, based on column name we can specially get the value and set to the context.
a.
Created mock service for database example. Find the URL details for Service:
b.
This is the response for the request
c.
Now time to execute the request. Find the request execution
LOGGERS: Groovy log:Whatever loggers we are using in script, it will display in Groovy log window.
soapUI log:soapUI log for process.
http log: Here headers details and message details will display.
Error log: while running application, if we may face any problems we can verify errors in error log.
Memory log: memory details
Jetty log: whenever will start the mock service, we can identify event in jetty log.
How to mock Service using SOAPUI??
Selecting a response based on the request
To mock services based on request we can give static response. Here whatever request will pass will get same response from SOAP UI of mock Service. Here I want to show that how we can create dynamic responses using SOAP UI.
1. Selecting a response based on the request
This script is specified at the MockOperation level and uses the script code to extract the input values from the incoming request. Based on some validations it returns the name of the MockResponse to return to the client. The script is as follows:
Aim of Script:In request we have name property based name property we are returning different responses like R1, R2 and R3. * If name is "AA" then response R1, * Else if name is "BB" then response R2, * Else response is R3. def temp="XX"; def groovyUtils=new com.eviware.soapui.support.GroovyUtils(context) def xml=new XmlSlurper().parseText(mockRequest.requestContent) xml.breadthFirst().each{ def v=it.toString() log.info("==============="+it.name()+"==================="+it.text()); if(it.name()=="name"){ temp=it.text(); log.info("===================matching tag=========================="+it.text()); } } //log.info("*temp**"+temp); if(temp=='AA'){ log.info("if R1"); mockOperation.setDefaultResponse("R1"); }else if(temp=='BB'){ log.info("else if Response 1"); mockOperation.setDefaultResponse("R2"); }else{ log.info("else Response 1"); mockOperation.setDefaultResponse("R3"); }
a.
Create SOAP Project in SOAPUI
b.
Import wsdl into the project
c.
Here we can see operation is "doAction". Right click on request and create mock service.
d.
Mock service name is given as MultipleResponses. Our aim to create multiple responses for single request.� Click on operation and create responses like below.
e.
Already I have created two responses, creating one more response (R3) for operation.
f.
Responses R1, R2 and R3 are ready
g.
Now step is to write script for to handle request. Click on operation and change dispatch to SCRIPT.
h.
Copy that script whatever I have provided before creating SOAP project.
i.
Change the target URL for request to MultipleResponse
j.
Provide URL
k.
Start the mock server for "MultipleResponses".
l.
This is my R1 response
m. Execute request
Streamlined service simulation
MockServices in soapUI gives you the unique ability to mimic Web services and create/run Functional and Load Tests against them even before they are implemented. Better yet, this allows you to eliminate the expense of building full-scale replicas of your production systems, as well as provide your customers access to your services without having to wait for them to be complete or available. In soapUI, you can create standards-compliant Mocks with virtually no effort on your part : just select a WSDL from your desired location and soapUI automatically generates the MockService and its methods for you. Then populate it with pre-defined responses for requests, customize responses any way you like, and define different responses for a given operation. Use the advanced scripting features to simulate any kind of desired behavior : fixed responses, random errors, dynamic results, and much more.
1) MockService Scripting Overview
A quick recap of the MockService model in soapUI: A MockService contains an arbitrary number of MockOperations, which each mock some WSDL operation in the containing project. These MockOperations can mock operations in different WSDL services, so a single MockService can mock any number of WSDLs to any extent. Each MockOperation contains an arbitrary number of MockResponse messages, i.e. messages that are returned to the client when the MockOperation is invoked. Exactly which message to return is decided by the MockOperation dispatch method, for which several are available; sequence, random, script, etc. For the MockService itself there are number of scripting events available:
Start and Stop scripts:these are useful for opening or closing shared objects (for example collections or database connections) which can be made accessible to scripts "further down" in the mock object hierarchy.
OnRequest script:this is the main handler for simulating non-SOAP behavior (REST, etc); it is called before soapUI does any internal dispatching and has the possibility to return any kind of response to the client (thus bypassing the whole soapUI dispatch mechanism)
AfterRequest script:primarily intended for custom logging and reporting functionality
2) Mock Handler Objects
A number of objects are commonly available in most scripts; here comes a quick overview with links to their corresponding javadoc:
context:used for storing MockService-wide objects, for example database connections, security tokens, etc
requestContext:used for storing Mock Request-wide objects, for example dynamic content to be
inserted into the response message
mockRequest:an object corresponding to the actual request made to the MockService. Through this object you can get hold of the incoming message, its headers, etc. Also the underlying HttpRequest and HttpResponse objects are available for "low-level" processing
mockResult:an object encapsulating the result of the dispatched request, this is available in the MockService.afterRequest script.
mockResponse/mockOperation:objects available at the corresponding levels for accessing the underlying configuration and functionality
mockRunner:the object corresponding to the execution of the MockService; this gives you access to previous mock results, etc
A company that decides to implement a Web API must make two critical decisions. First, what protocol will they use to exchange information between the client and the server. Rest and Soap are the most popular, but there are many other options including developing a JavaScript library that hides the web services interaction from the client programmer (this is what Google Maps does). The second decision is to decide what data format to use when returning information to the client. The most popular data formats are XML and JSON, but there are many other formats supported by big players such as YouTube, which supports the Atom data format. The following table provides a view of the protocols and formats supported by the 10 most popular Web APIs (data obtained from www.ProgrammableWeb.com). Protocols Google Maps Flickr YouTube Amazon eCommerce Twitter eBay Microsoft Virtual Earth Del.icio.us Google Search Yahoo Maps JavaScript REST, SOAP, XML-RPC GData, Atom REST, SOAP REST REST, SOAP JavaScript REST SOAP REST, JavaScript, Flash Data Formats XML, VML, JSON, KML XML, JSON, PHP GData, RSS XML XML, JSON, RSS, Atom XML KML, GeoRSS XML XML XML
As part of a customer contract, over the past 6 weeks QualityLogic’s web services testing team utilized SoapUI to develop an automated functional conformance test for a REST API that provides account management services to a variety of related web sites. The 100 resource paths in the API all used http POST requests with an XML fragment contained in the body of the http message conveying the desired actions and/or response. One particular challenge with this project was that we were sharing the test environment with other groups and could make no assumptions about the availability static values in the back-end database.
Before jumping into how we approached this project, let me make a few general observations about SoapUI.
It is an amazingly high quality open source test tool. The use of groovy scripts, xPath, and dynamic property expansion created a very robust development environment. Although not being Java programmers, it was initially a bit confusing to sort out where Groovy stopped and Java began. A good book, Groovy in Action, solved that problem. We found test case development a bit more tedious than expected with a test cases consisting of a dozen steps and within each step perhaps a dozen assertions. It was just a lot of detail, with lots of mouse clicking to drill into things. We wrote some helper routines to consolidate a lot of the assertion checks which helped to some extent. We wound up buying the Pro version of SoapUI as we needed some technical support and this turned out to be a great decision. The support was excellent, we were more productive using the Pro version, and the code remained compatible with the open source version of SoapUI as long as we stayed away from certain Pro features. The SoapUI documentation was quite well written, but initially it was a challenge to sort out which things applied specifically to REST based testing.
The project proceeded as follows… 1. We created a SoapUI project, a rest service within the project, defined each of the 100 resource paths within the service, and created an empty test suite. 2. We decided to use test suite properties as a set of global variables. Some properties were used to store dynamic data that was stored and retrieved by various test steps, while other properties contained static content that represented various possible values for customer records. Referencing these properties in a variety of ways throughout the test suite was simple. A few examples… o Simple Property Expansion:
${#TestSuite#password} o
Dynamic Property Expansion where multiple comma separated values are stored in one property:
${=context.testCase.testSuite.properties["accountNumber"].value.s plit(",")[1]}
3. We made extensive use of random numbers for generating values that had to be unique. Yes, there was a remote possibility that we might get a test case failure as a result of a random number collision, but from a practical perspective, this was a non-issue. 4. We structured our test suite to run sequentially, with the following initialization routines: o Generate unique “random” values and store in properties o Use REST calls to populate the data base with sufficient framework data to register a customer. Example, define the various valid address types such as home, business, other, etc. o Use REST calls to register a set of customers using characteristics pulled form test suite properties. These customers will not be modified during test execution and will be used exclusively for “get” and “search” customer related testing. o Define a series of test cases that register customers for subsequent record updating. These can be called from other test cases as needed. Note that we encountered problems exporting and importing test cases that used the Goto Test
step (they reference a GUID not the name of the test case), so we chose to call the test cases from a groovy script…
def testCase = testRunner.testCase.testSuite.testCases["Another TestCase"]def properties = new com.eviware.soapui.support.types.StringToObjectMap() def async = false testCase.run( properties, async )
5. Once the basic plumbing was in place we needed to grind out hundreds of test cases that when completed represented thousands of REST API calls and many thousands of test assertions. 6. Perhaps a note on assertions is warranted. There are a multitude of ways to validate that you get the expected values in a response using SoapUI. We utilized the following methods… o Contains assertion – On the surface this is a simple text comparison, but we also used regular expressions to accept multiple values such as (?s).*ALREADY.*|.*SUCCESS.* as well as property expansions. o Script Assertion – This was our most common assertion tool, which allowed to use the getXmlHolder object provided by SoapUI to easily query and assert conditions within a groovy script, such as..
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )def holder = groovyUtils.getXmlHolder(messageExchange.responseContent) def xPath = "//statusLabel[../SystemLabel='" + \ context.testCase.testSuite.properties["SystemLabel"].value.split( ",")[1] \ + "' and ../statusLabel='SUCCESS']" assert holder[xPath] == "SUCCESS" o
xPath Match – This was nice tool in that I could use an xPath statement to grab a parent node, then compare expected results using both wild cards and property expansions child elements. The only gotcha that I found was that if I wasn’t very careful with my xPath statement and multiple parent blocks were returned, xPath Match would compare the first instance, which wasn’t always what I wanted. But with simpler scenarios, this was definitively a very quick and dirty way to check a bunch of response values.
As most of our assertion testing involved comparing large sets of known values stored in test suite properties to API responses, we wrote some rather elaborate helper routines that simplified the assertion process. We did run into one obstacle in that we couldn’t find a clean way to “include” these routines in the assertions. The Pro version has a common library capability, but we didn’t want to lock ourselves into that. The open source version does allow a Jar file to be included in the the soapUI\bin\ext folder, which would have addressed the issue, but we wanted the test suite to run on a generic install of SoapUI. So, the utility routines were pasted into assertions throughout the suite (yuck). Just as the test suite was being finalized we discovered that there was a bug in the helper routines (not zero stuffing MD5 hash values) and were faced with editing 147 instances of the helper routines in the test suite (nightmare!). Fortunately, we were able to pull the entire SoapUI project into XMLSpy and globally edit the routines. This notion of editing the project from outside SoapUI might open up some interesting productivity opportunities.
We are now moving on to using SoapUI to load test this implementation and I will share our experiences with this effort in a week or so.
XML, SOAP and REST
XML (Extensible Markup Language) uses descriptive text-based elements (tags) and attributes to provide context for the content contained in an XML document. In effect, XML is as a language that allows you to define markup languages. An XML schema or DTD (Document Definition Type) provides a set of rules for element and attribute names allowed, attribute value data types, and allowable child elements. XHTML is an example of XML being used to define an application-specific markup language. You will most likely encounter XML when modifying web page configuration files or viewing the exchange of information in web services requests and responses. For more information, see Wikipedia or W3.org. Web Services are defined as software systems that enable machine-to-machine interactions over networks. Typically, a client makes a web service request to a server, and the resulting information is returned to the client in an XML format. Note that the “client” can be one server talking to another server. Two predominant web services frameworks, SOAP and REST, are used in web site development. SOAP (Simple Object Access Protocol) and REST (Representational State Transfer) provide mechanisms for requesting information from endpoints (SOAP) or from resources (REST). Perhaps the best way to think of these technologies is as a method of making a remote procedure calls against a well-defined API. SOAP has a more formal definition mechanism called WSDL (Web Services Definition Language) and is a bit more complex to implement. REST uses the standard HTTP request and response mechanism, simplifying implementation and providing for a looser coupling of the client and server. Note that REST also supports the transfer of non-XML messages such as JSON (JavaScript Object Notation). For more about SOAP, see Wikipedia or W3.org; for more about REST, go to Wikipedia or A Brief Introduction to REST on InfoQ. Testing a web services API implementation requires the same discipline and methodologies as with any core technology test effort. Formal test specifications, test cases with well-defined assertions, and a robust test execution platform are all part of a successful web services test effort.
Most major web properties, such as Yahoo, Google, YouTube, and Flickr, provide a web services API so that web site developers can pull information from their sites. In fact, a class of web sites exist called mashups, which pull data from a variety of sources using web services and combine the data in innovative ways. Validating a mashup or any web site that pulls data from external sources requires careful testing to ensure that the client can handle all possible variations of the received data.
Baseline Tests – These tests exercise each API method in isolation. o Boundary conditions o Repeating elements o Combinations of parameter values o Default value assumptions (i.e., optional parameters) o Data types and sizes o Correct return tags and values Task-Oriented Tests – Multiple API calls are made replicating common tasks and realworld scenarios documented in the Use Cases. Characteristics of these API tests are as follows: o Dependencies between API calls o Calling order o Repetitive transactions o State transitions o Propagation of data to external systems Forced Error Tests - These tests contain typical error scenarios, such as missing required elements, empty content, and content exceeding maximum limits, across a representative sampling of the API methods. Test assertions include: o Correct error messages o Fallback behavior o Transaction rollback behavior
Transferring values from the request to the response:
This is straightforward scripting done at the MockResponse level; the script uses properties of the mockRequest object to get hold of the incoming request, extracts the desired values via XPath and then writes the created result to a requestContext property:
def temp="IBM"; def groovyUtils=new com.eviware.soapui.support.GroovyUtils(context) def xml=new XmlSlurper().parseText(mockRequest.requestContent) xml.breadthFirst().each{ def v=it.toString() if(it.name()=="name"){ temp=it.text(); log.info("============================================="+it.text()); } } context.session=temp+' Munna' mockOperation.setDefaultResponse("R1");
a.
Created mock service as ContextResponse with script. Here will have single response, based on request will return response. From request will take name as a message will add with"Munna" string.
b.
This is the response
c. Start the server for contextService and execute request. Find the output below:
How to mock Service using SOAPUI??
Read the response from a database:
This one requires a bit more work as we need to set up and close a database connection which we can use in our scripts. This is best done in the MockService start script as follows; import groovy.sql.Sql //def mockService=mockRunner.mockService def sql=Sql.newInstance("HostName","username","password","databaseDriver"); context.dbConnection=sql def row=sql.firstRow("select * from table")
log.info "=================="+row.get("column1"); context .responseMessage = row.get("column2"); mockOperation.setDefaultResponse("R1"); if(context.dbConnection!=null){ log.info "Closing the connection"; context.dbConnection.close() }
Aim of Script: Here response is whatever we are getting data from DB. If we want to specifically want specify any condition based on request we can add like where condition with request element value. ** Connecting data base and selecting first row. From that row, based on column name we can specially get the value and set to the context.
a.
Created mock service for database example. Find the URL details for Service:
b.
This is the response for the request
c.
Now time to execute the request. Find the request execution
LOGGERS: Groovy log:Whatever loggers we are using in script, it will display in Groovy log window.
soapUI log:soapUI log for process.
http log: Here headers details and message details will display.
Error log: while running application, if we may face any problems we can verify errors in error log.
Memory log: memory details
Jetty log: whenever will start the mock service, we can identify event in jetty log.
How to mock Service using SOAPUI??
Selecting a response based on the request
To mock services based on request we can give static response. Here whatever request will pass will get same response from SOAP UI of mock Service. Here I want to show that how we can create dynamic responses using SOAP UI.
1. Selecting a response based on the request
This script is specified at the MockOperation level and uses the script code to extract the input values from the incoming request. Based on some validations it returns the name of the MockResponse to return to the client. The script is as follows:
Aim of Script:In request we have name property based name property we are returning different responses like R1, R2 and R3. * If name is "AA" then response R1, * Else if name is "BB" then response R2, * Else response is R3. def temp="XX"; def groovyUtils=new com.eviware.soapui.support.GroovyUtils(context) def xml=new XmlSlurper().parseText(mockRequest.requestContent) xml.breadthFirst().each{ def v=it.toString() log.info("==============="+it.name()+"==================="+it.text()); if(it.name()=="name"){ temp=it.text(); log.info("===================matching tag=========================="+it.text()); } } //log.info("*temp**"+temp); if(temp=='AA'){ log.info("if R1"); mockOperation.setDefaultResponse("R1"); }else if(temp=='BB'){ log.info("else if Response 1"); mockOperation.setDefaultResponse("R2"); }else{ log.info("else Response 1"); mockOperation.setDefaultResponse("R3"); }
a.
Create SOAP Project in SOAPUI
b.
Import wsdl into the project
c.
Here we can see operation is "doAction". Right click on request and create mock service.
d.
Mock service name is given as MultipleResponses. Our aim to create multiple responses for single request.� Click on operation and create responses like below.
e.
Already I have created two responses, creating one more response (R3) for operation.
f.
Responses R1, R2 and R3 are ready
g.
Now step is to write script for to handle request. Click on operation and change dispatch to SCRIPT.
h.
Copy that script whatever I have provided before creating SOAP project.
i.
Change the target URL for request to MultipleResponse
j.
Provide URL
k.
Start the mock server for "MultipleResponses".
l.
This is my R1 response
m. Execute request
Streamlined service simulation
MockServices in soapUI gives you the unique ability to mimic Web services and create/run Functional and Load Tests against them even before they are implemented. Better yet, this allows you to eliminate the expense of building full-scale replicas of your production systems, as well as provide your customers access to your services without having to wait for them to be complete or available. In soapUI, you can create standards-compliant Mocks with virtually no effort on your part : just select a WSDL from your desired location and soapUI automatically generates the MockService and its methods for you. Then populate it with pre-defined responses for requests, customize responses any way you like, and define different responses for a given operation. Use the advanced scripting features to simulate any kind of desired behavior : fixed responses, random errors, dynamic results, and much more.
1) MockService Scripting Overview
A quick recap of the MockService model in soapUI: A MockService contains an arbitrary number of MockOperations, which each mock some WSDL operation in the containing project. These MockOperations can mock operations in different WSDL services, so a single MockService can mock any number of WSDLs to any extent. Each MockOperation contains an arbitrary number of MockResponse messages, i.e. messages that are returned to the client when the MockOperation is invoked. Exactly which message to return is decided by the MockOperation dispatch method, for which several are available; sequence, random, script, etc. For the MockService itself there are number of scripting events available:
Start and Stop scripts:these are useful for opening or closing shared objects (for example collections or database connections) which can be made accessible to scripts "further down" in the mock object hierarchy.
OnRequest script:this is the main handler for simulating non-SOAP behavior (REST, etc); it is called before soapUI does any internal dispatching and has the possibility to return any kind of response to the client (thus bypassing the whole soapUI dispatch mechanism)
AfterRequest script:primarily intended for custom logging and reporting functionality
2) Mock Handler Objects
A number of objects are commonly available in most scripts; here comes a quick overview with links to their corresponding javadoc:
context:used for storing MockService-wide objects, for example database connections, security tokens, etc
requestContext:used for storing Mock Request-wide objects, for example dynamic content to be
inserted into the response message
mockRequest:an object corresponding to the actual request made to the MockService. Through this object you can get hold of the incoming message, its headers, etc. Also the underlying HttpRequest and HttpResponse objects are available for "low-level" processing
mockResult:an object encapsulating the result of the dispatched request, this is available in the MockService.afterRequest script.
mockResponse/mockOperation:objects available at the corresponding levels for accessing the underlying configuration and functionality
mockRunner:the object corresponding to the execution of the MockService; this gives you access to previous mock results, etc
A company that decides to implement a Web API must make two critical decisions. First, what protocol will they use to exchange information between the client and the server. Rest and Soap are the most popular, but there are many other options including developing a JavaScript library that hides the web services interaction from the client programmer (this is what Google Maps does). The second decision is to decide what data format to use when returning information to the client. The most popular data formats are XML and JSON, but there are many other formats supported by big players such as YouTube, which supports the Atom data format. The following table provides a view of the protocols and formats supported by the 10 most popular Web APIs (data obtained from www.ProgrammableWeb.com). Protocols Google Maps Flickr YouTube Amazon eCommerce Twitter eBay Microsoft Virtual Earth Del.icio.us Google Search Yahoo Maps JavaScript REST, SOAP, XML-RPC GData, Atom REST, SOAP REST REST, SOAP JavaScript REST SOAP REST, JavaScript, Flash Data Formats XML, VML, JSON, KML XML, JSON, PHP GData, RSS XML XML, JSON, RSS, Atom XML KML, GeoRSS XML XML XML
As part of a customer contract, over the past 6 weeks QualityLogic’s web services testing team utilized SoapUI to develop an automated functional conformance test for a REST API that provides account management services to a variety of related web sites. The 100 resource paths in the API all used http POST requests with an XML fragment contained in the body of the http message conveying the desired actions and/or response. One particular challenge with this project was that we were sharing the test environment with other groups and could make no assumptions about the availability static values in the back-end database.
Before jumping into how we approached this project, let me make a few general observations about SoapUI.
It is an amazingly high quality open source test tool. The use of groovy scripts, xPath, and dynamic property expansion created a very robust development environment. Although not being Java programmers, it was initially a bit confusing to sort out where Groovy stopped and Java began. A good book, Groovy in Action, solved that problem. We found test case development a bit more tedious than expected with a test cases consisting of a dozen steps and within each step perhaps a dozen assertions. It was just a lot of detail, with lots of mouse clicking to drill into things. We wrote some helper routines to consolidate a lot of the assertion checks which helped to some extent. We wound up buying the Pro version of SoapUI as we needed some technical support and this turned out to be a great decision. The support was excellent, we were more productive using the Pro version, and the code remained compatible with the open source version of SoapUI as long as we stayed away from certain Pro features. The SoapUI documentation was quite well written, but initially it was a challenge to sort out which things applied specifically to REST based testing.
The project proceeded as follows… 1. We created a SoapUI project, a rest service within the project, defined each of the 100 resource paths within the service, and created an empty test suite. 2. We decided to use test suite properties as a set of global variables. Some properties were used to store dynamic data that was stored and retrieved by various test steps, while other properties contained static content that represented various possible values for customer records. Referencing these properties in a variety of ways throughout the test suite was simple. A few examples… o Simple Property Expansion:
${#TestSuite#password} o
Dynamic Property Expansion where multiple comma separated values are stored in one property:
${=context.testCase.testSuite.properties["accountNumber"].value.s plit(",")[1]}
3. We made extensive use of random numbers for generating values that had to be unique. Yes, there was a remote possibility that we might get a test case failure as a result of a random number collision, but from a practical perspective, this was a non-issue. 4. We structured our test suite to run sequentially, with the following initialization routines: o Generate unique “random” values and store in properties o Use REST calls to populate the data base with sufficient framework data to register a customer. Example, define the various valid address types such as home, business, other, etc. o Use REST calls to register a set of customers using characteristics pulled form test suite properties. These customers will not be modified during test execution and will be used exclusively for “get” and “search” customer related testing. o Define a series of test cases that register customers for subsequent record updating. These can be called from other test cases as needed. Note that we encountered problems exporting and importing test cases that used the Goto Test
step (they reference a GUID not the name of the test case), so we chose to call the test cases from a groovy script…
def testCase = testRunner.testCase.testSuite.testCases["Another TestCase"]def properties = new com.eviware.soapui.support.types.StringToObjectMap() def async = false testCase.run( properties, async )
5. Once the basic plumbing was in place we needed to grind out hundreds of test cases that when completed represented thousands of REST API calls and many thousands of test assertions. 6. Perhaps a note on assertions is warranted. There are a multitude of ways to validate that you get the expected values in a response using SoapUI. We utilized the following methods… o Contains assertion – On the surface this is a simple text comparison, but we also used regular expressions to accept multiple values such as (?s).*ALREADY.*|.*SUCCESS.* as well as property expansions. o Script Assertion – This was our most common assertion tool, which allowed to use the getXmlHolder object provided by SoapUI to easily query and assert conditions within a groovy script, such as..
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )def holder = groovyUtils.getXmlHolder(messageExchange.responseContent) def xPath = "//statusLabel[../SystemLabel='" + \ context.testCase.testSuite.properties["SystemLabel"].value.split( ",")[1] \ + "' and ../statusLabel='SUCCESS']" assert holder[xPath] == "SUCCESS" o
xPath Match – This was nice tool in that I could use an xPath statement to grab a parent node, then compare expected results using both wild cards and property expansions child elements. The only gotcha that I found was that if I wasn’t very careful with my xPath statement and multiple parent blocks were returned, xPath Match would compare the first instance, which wasn’t always what I wanted. But with simpler scenarios, this was definitively a very quick and dirty way to check a bunch of response values.
As most of our assertion testing involved comparing large sets of known values stored in test suite properties to API responses, we wrote some rather elaborate helper routines that simplified the assertion process. We did run into one obstacle in that we couldn’t find a clean way to “include” these routines in the assertions. The Pro version has a common library capability, but we didn’t want to lock ourselves into that. The open source version does allow a Jar file to be included in the the soapUI\bin\ext folder, which would have addressed the issue, but we wanted the test suite to run on a generic install of SoapUI. So, the utility routines were pasted into assertions throughout the suite (yuck). Just as the test suite was being finalized we discovered that there was a bug in the helper routines (not zero stuffing MD5 hash values) and were faced with editing 147 instances of the helper routines in the test suite (nightmare!). Fortunately, we were able to pull the entire SoapUI project into XMLSpy and globally edit the routines. This notion of editing the project from outside SoapUI might open up some interesting productivity opportunities.
We are now moving on to using SoapUI to load test this implementation and I will share our experiences with this effort in a week or so.
XML, SOAP and REST
XML (Extensible Markup Language) uses descriptive text-based elements (tags) and attributes to provide context for the content contained in an XML document. In effect, XML is as a language that allows you to define markup languages. An XML schema or DTD (Document Definition Type) provides a set of rules for element and attribute names allowed, attribute value data types, and allowable child elements. XHTML is an example of XML being used to define an application-specific markup language. You will most likely encounter XML when modifying web page configuration files or viewing the exchange of information in web services requests and responses. For more information, see Wikipedia or W3.org. Web Services are defined as software systems that enable machine-to-machine interactions over networks. Typically, a client makes a web service request to a server, and the resulting information is returned to the client in an XML format. Note that the “client” can be one server talking to another server. Two predominant web services frameworks, SOAP and REST, are used in web site development. SOAP (Simple Object Access Protocol) and REST (Representational State Transfer) provide mechanisms for requesting information from endpoints (SOAP) or from resources (REST). Perhaps the best way to think of these technologies is as a method of making a remote procedure calls against a well-defined API. SOAP has a more formal definition mechanism called WSDL (Web Services Definition Language) and is a bit more complex to implement. REST uses the standard HTTP request and response mechanism, simplifying implementation and providing for a looser coupling of the client and server. Note that REST also supports the transfer of non-XML messages such as JSON (JavaScript Object Notation). For more about SOAP, see Wikipedia or W3.org; for more about REST, go to Wikipedia or A Brief Introduction to REST on InfoQ. Testing a web services API implementation requires the same discipline and methodologies as with any core technology test effort. Formal test specifications, test cases with well-defined assertions, and a robust test execution platform are all part of a successful web services test effort.
Most major web properties, such as Yahoo, Google, YouTube, and Flickr, provide a web services API so that web site developers can pull information from their sites. In fact, a class of web sites exist called mashups, which pull data from a variety of sources using web services and combine the data in innovative ways. Validating a mashup or any web site that pulls data from external sources requires careful testing to ensure that the client can handle all possible variations of the received data.
Baseline Tests – These tests exercise each API method in isolation. o Boundary conditions o Repeating elements o Combinations of parameter values o Default value assumptions (i.e., optional parameters) o Data types and sizes o Correct return tags and values Task-Oriented Tests – Multiple API calls are made replicating common tasks and realworld scenarios documented in the Use Cases. Characteristics of these API tests are as follows: o Dependencies between API calls o Calling order o Repetitive transactions o State transitions o Propagation of data to external systems Forced Error Tests - These tests contain typical error scenarios, such as missing required elements, empty content, and content exceeding maximum limits, across a representative sampling of the API methods. Test assertions include: o Correct error messages o Fallback behavior o Transaction rollback behavior
