Online Based Application Testing on a Cloud - A Survey

On-Demand Web based Application Testing on a Cloud - A Survey

PROF.V.JANANI.M.E. (Ph.D)
ASSISTANT PROFESSOR
DEPARTMENT OF CSE,
ADHIYAMAAN COLLEGE OF ENGINEEERING,
HOSUR, TAMILNADU.
[email protected]

Dr.K.KRISHNAMOORTHY.M.E.Ph.D
PROFESSOR & HEAD
DEPARTMENT OF CSE
SUDHARSAN COLLEGE OF ENGINEERING
PUDUKKOTTAI,TAMILNADU.
[email protected]

Abstract
Testing a new application/software often requires
costly server, storage and network devices only for a
limited time. For instance, web applications need to
be tested for multiple operating systems, browsers
and versions, different range of hardware and huge
number of concurrent users. It is more complex and
consumes huge capital, and resources in conventional
approach of in-house testing environments. To
address these issues with one click is cloud testing
i.e. software testing in cloud. It is a model of software
testing used to test an application provided as a
service to customers across the internet. It promotes
by enabling organizations and individuals to provide
and acquire testing services on-demand. Migrating to
the cloud has its own set of challenges and risks such
as data integrity, security, privacy, business
acceptability, etc. cloud enables hosting applications
on the cloud is all at one end, but companies also
need to understand the risks associated with it and
adopt proper mitigation plans, along with the core
which is testing. They need to adopt an end-to-end
testing approach starting from requirements to
deployment, because each stage has different testing
requirements. At the first level system integration
tests and user acceptance testing need to be carried
out to ensure the developed cloud meets functional
and business user needs. It reduces costs related to
installing and maintaining testing environments and
also introduces a new market where the provider and
consumers can reach skilled test engineers on-
demand.

Keywords
Cloud Testing, Load Testing, Performance Testing

Introduction
Software testing is performed to verify that the
completed software package functions according to
the expectations defined by the requirements
specifications. Testing requires expensive dedicated
infrastructure and resources that were only used
periodically which scrutinizes the application's
performance, reliability, speed, security and
functionality. Since, business applications are
growing in complexity, it is difficult for
organizations to build and maintain in-house testing
facilities that imitate real-time environments.
In very simple definition provided by Wikipedia,
Cloud testing is a form of software testing in which
web applications use cloud computing environments
(a "cloud") to simulate real-world user traffic. Cloud
computing is an internet-based computing, whereby
shared resources, software, and information are
provided to computers and other devices on demand
like the electricity grid. The cloud is often
characterized by self-service interface that let
customers acquire resources when needed and as long
as they are needed. Cloud is also the concept behind
an approach to build IT services that takes advantage
of growing power of servers and virtualization
technologies.
Cloud testing is also known as web testing wherein
testing is done using resources, machines or servers
from the cloud infrastructure. Besides, the entire
testing environment can be obtained from the cloud
on-demand at a cost that is practical and reasonable
due to the pay-for-use nature of cloud computing and
with a lead-time that is near impossible within a
company’s own data center. Cloud testing has
traditionally been used to refer to load and
performance testing of Websites. However with
increasing maturity of technology, all kinds of
enterprise software can be tested for functional and
performance issues before going in for full fledged
enterprise deployment.
Response Time

Load Testing of Web Application


A web application’s quality of service is measured by
its response time, throughput and availability.
According to Daniel A. Menasce’s QoS factor,
addresses that load testing is measured by
availability and response time.

Availability

Availability is the ability of configuration (or) IT
service to perform its agreed function when required.
Availability is usually calculated in percentage. This
calculation is often based on agree service time and
downtime. It is best practice to calculate availability
using measurements of the business output of the IT
service.



( )





Availability measures the percentage of time
customers can access web-based application.
Availability goals typically vary according to the
application type. Critical applications such as online
brokerage, often have more stringent requirements
than other applications. However, even in industries
availability is less critical, it is always crucial to a
company’s bottom line. In addition to application
type, availability requirements can vary according to
the time of day or for special events. For example,
during high market volatility, online brokerage sites
must be available as close to 100 percent of the time
as possible. Likewise, online ticket brokers must
exhibit high availability when tickets for a concert or
sporting event go on sale. Unfortunately, at just these
times, Websites are subjected to “flash crowds” that
stretch resources to their limits, sometimes leading to
rejected user requests and decreased availability.
Finally, Web or e-commerce site availability also
varies for different customers: ISPs and networks
provide different levels of service to customers
depending on their geographic location.







Response time (or) Responsiveness is simply how
quickly an interactive system responds to user input.
According to web, definitions of response time may
vary. Some researchers define response time from the
moment the user submits the request for a page to the
moment where the page begins to sender. Others
extend the final moment until the page is fully
rendered in the browser.
With web-based environments, it can be measured
end-to-end response time to determine how
customers perceive such things as page download and
keyword search times. Also, when defining end-to-
end response time
1
, it must distinguish between the
download time for the base HTML page and that for
other page components, such as images and ad
banners.
Campbell and Alsted described an equation that can
be used to characterize and measure web application
responsiveness,


(


) (
()

)



1. Payload-The total size in bytes sent to the
browser including the page and its resource
files.
2. Bandwidth-From client to server, the
minimal bandwidth in bps across all network
links.
3. AppTurns-The number of components
(images, scripts, css,flash,etc) needed for the
page to render.
4. RTT(Round- Trip Time)-The amount of
time (in ms) it takes to communicate from
client to server and back again.
5. Concurrency-The number of simultaneous
requests a browser will make for resources.
6. Server Compute time(C
s
)-The time it takes
for the server to parse the request,run
application code, fetch data, and compose a
response.
7. Client Compute time(C
c
)-The time it takes
for the browser to render HTML,execute
scripts,implement style sheets etc.





The load-testing process



Figure 1.The Load Testing Process

The Figure 1 shows load testing p r o c e s s
t h a t measures the websites QoS performance
based on actual customer behavior. When
customers access your site, a script recorder uses
their request to create interaction scripts. A load
generator then replays the scripts possibly
modified by test parameters against the website.

The load generator mimics browser behavior: It
continuously submits request to the Website, waits
for a period of time after the site sends a reply to
the request (the think time), and then submits a new
request. The load generator can emulate thousands
of concurrent users to test Website scalability. Each
emulated browser is called a virtual user, which is
a key load-testing concept. A load test is valid only
if virtual users behavior has characteristics similar
to those of actual users. You must therefore
ensure that your virtual user

■ follow patterns similar to real users
■ Use realistic think times, and
■ React like frustrated users abandoning a web
session if response time is excessive.

Failure to mimic real user behavior can generate
totally inconsistent results. Because customers who
abandon a session use fewer site resources than
those who complete it. For example, planning your
infrastructure capacity assuming that all started
sessions will be completed can lead you to over-
provision the site. Also, if you fail to consider
session abandonment, you cannot accurately
quantify important business metrics

such as:

■ Revenue throughput, which measures the
amount of money a Website generates per unit
time and for example, dollars per second.
■ Potential loss revenue throughput, which is the
amount of money in customers’ shopping carts
that was not converted into sales per unit time
due to session abandonment.

During the time a Website is subjected to the
load generated by virtual users. Then we measure
its performance and obtain metrics such as
response time and throughput f or each load
intensity value t h a t is based on the number of
virtual users.

Performance testing in cloud

Performance testing
2
, monitoring, tuning of cloud are
the key buzzwords nowadays. Performance refers to
information regarding your applications response
time, throughput and resource utilization levels.
Stakeholders are interested in application response
time. But when you add a large complex network like
cloud network response time, virtualization skew is
responsible for the total response time.
SaaS leaders like omniture, monster.com, salary.com
and webex have known for years that performance
problems can impact customer experience and have a
huge impact on revenues, renewals, brand perception
and overall user satisfaction. Companies like
salesforce.com and amazon have elevated their
performance as a point of differentiation and publish
regular reports on their performance.
Aberdeen’s June 2008 report, Application
Performance Management: The Lifecycle Approach
Brings IT and Business Together showed that issues
with application performance could impact corporate
revenues by upto 9%. The research also showed that
only 42% of organizations were satisfied with the
performance of business-critical applications.
LoadStorm: Load testing allows web developers to
know how their applications respond under heavy
volumes of HTTP traffic. The performance of your
website is critical to your revenue, and web
performance tuning can increase your profit by
10%.LoadStorm puts massive cloud resources in the
hands of web developers to enable them to improve
the performance of their web applications
CloudTest by SOASTA: provides performance,
functional web user interface and Asynchronous
JavaScript and XML (Ajax) testing as an appliance or
on-demand service. SOASTA’s CloudTest for
Amazon EC2 starts at $4 per instance per hour and its
Global CloudTest starts at $1,000 instance per hour.
VMLogix and Skytap offer entire scalable test labs
with management software in the cloud that can cost
several thousand dollars per month on average.
KeyNote
Pylot - Web Load Testing from Amazon Elastic
Compute Cloud (EC2): does the job well for simple
web performance and load tests. Pylot generates
concurrent load (HTTP Requests), verifies server
responses and produces reports with metrics. Test
suites are executed and monitored from a GUI or
shell/console.

HP LoadRunner
Grinder In The Cloud – A cloud based load testing
environment, open source based load test approach
that uses cloud computing facilities offered by
Amazon web services. This leverages the well known
Grinder Load Test framework by putting it in the
cloud easy to use with virtually unlimited firepower
at a competitive price
Xceptance LoadTest - Load testing and regression
tool from Xceptance Software Technologies.inc for
web and Java and other app load testing, includes
recording capabilities. XLT Cloud Service available,
tests implemented as JUnit 4 test cases. For web-
based tests, the framework provides a (headless)
browser that can emulate Internet Explorer or
Firefoxbehavior.

Load-Intelligence(Can Use JMETER in Cloud) -
Affordable load-testing “Software as a Service” from
Cloud-Intelligence. software and unlimited hardware
are all included. JMeter users can execute their test-
scripts in an unlimited, pre-configured and distributed
environment. Neither setup nor installations are
required. Immediate access to JMeter logs, reports,
test script, CSV files and more.
IntroSpectrum- A hosted performance monitoring
system which uses real browsers to provide the most
accurate view of web site performance hosted in the
cloud, so its platform/OS agnostic and works with
any public website or web application.
Techout - Measure and optimize speed and
availability of critical online applications. Website
monitoring, business transactions monitoring,
REST/SOAP Web Services Monitoring, Cloud
Monitoring/Amazon Web Service monitoring are
more, also available as an iPhone application.


Cloud performance as measured at the point of
application use is the sum of network performance,
application performance, and cloud infrastructure
performance. The cloud provider can be accountable
for the last of these and not the first two, so it is
important to understand what both factors contribute
to overall performance. The cost savings offered by
cloud-computing services are too compelling to
ignore, particularly when a company factors in the
reduced headcount that becomes possible with
outsourced IT infrastructure
Realizing the potential for the improvement of
performance in the cloud , average response time,
availability, response time of various Cloud Providers
and Significant Performance differences across
browsers/devices, how to scale the storage and
Analysis of Data Using Distributed Data Grids. This
section is dedicated to the list of various tools
available in the market to perform testing in the
cloud.


Conclusion
Cloud Testing has gained considerable interest in
recent years, it is a new direction in information
technology for developing and delivering computing
applications and services. This study results the
practical needs and expectations in cloud testing, and
analyzed using applications suitable for online
software testing, readymade online performance
testing tools for any customers then the quality
checks for application tested in the cloud. This
literature review also gives good idea about online
testing solutions for E-business systems. There is a
great demand for tools and service in cloud testing
and many companies are working on it. They include
security testing, presence service testing, testing of
interoperability of cloud computing infrastructures
and cloud testing for mobile applications.











REFERENCES
[1] Daniel A.Menasce Load Testing of Web Sites in
Proceedings of IEEE Internet Computing July-
August 2002.
[2] Neha Thakur Performance testing in cloud: A
pragmatic approach White Paper Submitted for STC
2010.
[3] Neha Mehrotra Cloud-Testing Vs Testing a Cloud
10
th
Annual international software testing conference
2010.
[4] LEF Grant Briefing, H.Kothandaraman Testing
Applications n cloud Computer Science Corporation
2011.
[5] koray Incki , Ismail Ari, Hasan sozer A survey of
software testing in the cloud IEEE sixth international
conference in software security and reliability
companion 2012.
[6] Taking testing to the cloud cognizant reports
March 2011.
[7] Sergiy VILKOMIR Cloud testing: A state-of-the-
art review An international journal vol.28. no.2 2012,
213-222.
[8] Jerry Gao, Xiaoying Bai, and Wei-Tek Tsai
Cloud Testing-Issues, challenges, needs and practice
software engineering: An international journal
(SEIJ), VOL 1, NO1. September 2011.
[9] http://www.microsoft.com/windowsazure/
[10] http://research.microsoft.com/projects/pex/
[11] https://sites.google.com/site/asergrp/projects/cloud
[12] http://phluffyfotos.codeplex.com/
[13]http://martinfowler.com/articles/mocksArentStub
s.html
[14]Kunal Taneja, Yi Zhang, Tao Xie: MODA: auto
mated test generation for database applications via
mock objects. In Proceedings of ASE, pages 289‐292,
2010.

[15] Yatendra Singh Pundhir cloud computing
applications and their testing methodology Bookman
International Journal of Software Engineering, Vol. 2
No. 1, Jan-Feb-Mar 2013.
[16] Dr. Rahul Malhotra Prince Jain Testing
Techniques and its Challenges in a Cloud Computing
Environment The SIJ Transactions on Computer
Science Engineering & its Applications (CSEA), Vol.
1, No. 3, July-August 2013.