automation, qa, web services, rest, soap, soapui

API Testing tips and tools

Many times is difficult to start on API Testing. Here are some tips and recommended tools to start.

Technology behind API. Here you need to  know what technology is around the API that you need to test. it could be REST, SOAP XML-RPC, etc. This will help us to select the correct tools to start the test and develop in the future a framework if it is required.

API Reference. first you  need to know the API, Services, methods, datatypes,etc. without this reference will be difficult start to test an API. so first ask for API reference or documentation.  You need to know what parameters receive each method and what is the expected result of each one of them. also the exceptions and errors.

Manual or Automated test.  We need to know if the testing over our API will be manual or automated. If the testing will be manual then we don’t require so much. but if testing will be automated we need more research about the tools and the best way to automate the API.

Scope of testing. You need to define the scope of the test, how many services and methods will be tested and what type of testing we will do (Acceptance, Negative)

Some Tools used in API testing

There are many tools that can be used in API testing according to tester preferences here are some of them (mostly used on manual testing)

  • Command Line tools (CLI). 
    • curl. If you like the command line (as me) this tool is very helpful combined with other add-ons in python mainly. you can test REST specially using this tool
  • UI Tools. These provides an GUI interface to work with the API, most knowing below:
    • SOAPUI. This is powerful tool that can be used to test REST, SOAP and XML-RPC, its main disadvantage is that consume a lot quantity of RAM
  • Addons or Extensions tools. most used in a browser. If you like to work on browser this tools are for you.

Frameworks to test API

If you decide to automate the API Testing you need to have a framework. Here are 3 cases:

  1. Develop your own framework (here you need to select the technologies that fit best with your project, select the correct language and see if it provides the interfaces to interact  with API services like REST, SOAP, etc)
  2. Use existent frameworks (Select the correct framework and to know if it provides all the necessary for your project)
  3. Mix 2 above, you can have an existent framework and extend it according to your requirements.

Here a list of some API test frameworks

  • Django. framework developed in python
  • Frisby. framework to test REST API developed in node.js and jasmine
  • SOAPUI. is also considered a framework and can be used easily in API testing automated.

 

References

http://www.soapui.org/

http://curl.haxx.se/

http://www.django-rest-framework.org/api-guide/testing/

http://frisbyjs.com/

 

Advertisements

Introduction.

This is a little example about how we can create an automation framework over a REST API using SOAPUI, ant and jenkins. The main idea is to show how SOAPUI works with REST services and how we can implement this tool to make an stable framework integrating with jenkins.

First at all we need to know some concepts.

REST.

REST (REpresentational State Transfer) is an architectural style, and an approach to communications that is often used in the development of web services. The use of REST is often preferred over the more heavyweight SOAP(Simple Object Access Protocol) style because REST does not leverage as much bandwidth, which makes it a better fit for use over the Internet.

SOAPUI.

SOAPUI is a tool that help us to test web services specially SOAP but also works with REST.

API.

First of all we need an API over which we will test, we can develop a small API, but is better to use a one that already exists. We can find several public APIs in the web, for this example we wil use the lastfm  API (a music site) http://www.last.fm/api . This  API provides both  REST and XML-RPC formats. We will use REST.

Create the project.

First thing is create the project opening SOAPUI and go to menu and select:  New REST Project   (CTRL+A+N)

Selection_266

Then we have a dialog requesting the URI in this case: http://ws.audioscrobbler.com/2.0/

Selection_267

as a result we have the project created as image below.

Selection_268

 

 

We can see a default request created but without parameters, if we execute this request we will have a response with errors as image below, this occurs generally due to malformed URI.

Selection_019

Call to methods

Now we going to do some examples of calls to methods using URIs with the right parameters. Right Click on the service and select New Resource

Tooltip_022

after Select New Resource from menu a dialog will be displayed, we need to enter the URI for this we will use the URI below:

http://ws.audioscrobbler.com/2.0/?method=artist.getsimilar&artist=cher&api_key=************&format=json

Selection_023

This is a REST request get the similar artists to Cher. once we click on OK  button SOAPUI creates the request with the parameters as the image below

Selection_018

Now we can execute the method clicking on green arrow and we will see the response en the right side of editor as image below

Selection_024

 

Second example is a method called gettoptracks this get the top tracks from an artists we going to use the URI below:

http://ws.audioscrobbler.com/2.0/?method=artist.gettoptracks&artist=thecure&api_key=************&format=json

Selection_017

 

and the results should be something similar to the image below

Selection_025

Suites and Test Cases.

The request generally is used for test the methods, but these request should be included in suites as test cases and those will be executed in our automation project.

We can create a test case clicking on the plus icon next to Method drop-down

Selection_026

Once created the test case and test suite a new request editor is displayed with the same parameters but additionaly with assertion option

Selection_027

In this case we will add a simple assertion: Valid HTTP Status code

Selection_028

 

as this is a little example the value for status code will be 200

 

Selection_029

 

once the assertion has been created we can execute the request and if all is ok the result should be passed

Selection_030

TestRunner

SOAPUI includes a tool (testunner.sh/bat)to execute the suites and test cases and generate outputs(reports) You can open the testrunner dialog in project, right click and select Launch TestRunner, the image below should be displayed.

Selection_031

 

We need to configure TestRunner path in Basic tab and the outputs in Reports tab, the next step click on Launch button and the tests will be executed by testrunner, here is very important to take note about the command execution as show the image below, this command will be used after by ant in jenkins integration

Selection_033

once all test were executed the image below should be displayed

Selection_032

Integration with Jenkins.

In order to integrate our project with jenkins, we need an xml file (build.xml) that will be used in an ant call, below is the build.xml file example. You can notice that the exec tag is the command line that testrunner executed in SOAPUI.

<project name="Soapui Nightly Build" default="testreport" basedir=".">
 <property environment="env" />
 <property name="label" value="${env.JOB_NAME}-${env.BUILD_NUMBER}" />
 <mkdir dir="reports/REST_${env.BUILD_NUMBER}" />
 <target name="soapui">
 <exec dir=".">
 <arg line="sh /opt/SoapUI-5.0.0/bin/testrunner.sh -e http://ws.audioscrobbler.com -r -a -j -f reports/REST_${env.BUILD_NUMBER} ~/soapui/REST-Project-1-soapui-project.xml
 </exec>
 </target>
 <target name="testreport" depends="soapui">
 <junitreport todir="reports/REST_${env.BUILD_NUMBER}">
 <fileset dir="reports/REST_${env.BUILD_NUMBER}">
 <include name="TEST-*.xml" /> </fileset>
 <report todir="reports/html" styledir="/usr/share/ant/etc" format="noframes"> </report>
 </junitreport>
 </target>
</project>

Job Configuration.

Now that we have the build file we can create a job in jenkins, providing the Name and Description for the project like below image:

Selection_013

 

then we need to setup the build.xml file for this we click on  Add Build Step and Select Invoke ant option

Selection_014

Reports.

Now we going to add the reports for this the option Add post-build action need to be added. in this case we will choose 3:

Archive the artifacts. to see the detailed executions in txt format

Publish HTML reports. to see the global report in a HTML format

Publish xUnit test result report. to see the junit results

Selection_015

 

Selection_016

Once created and configured the job the last step is run the job and verify the results (html, txt).

Once executed we will have the results like below image

Selection_034

Below is the HTML report that shows the results of the testing

Selection_035

and the txt file that  shows the report of an individual test

Selection_036

References.

http://www.last.fm/api

http://www.soapui.org/test-automation/running-functional-tests.html

http://jenkins-ci.org/

 

Thanks to everyone who will read, reviewed it and recommended it. I will appreciate your support.