Selenium Web Application Testing
Content
Selenium
Web Application Testing Maik Röder [email protected] http://www.ingeniweb.com
What is Selenium?
A test tool for web applications Selenium tests run directly in the browser
– Just like real users do – Browser plays back testing scripts – Uses JavaScript
Wide array of browsers and platforms supported
– Internet Explorer
• Windows
– Mozilla and Firefox
• Windows • Mac • Linux
– Safari
• Mac OS X
Web application testing
Two categories of test tools Tools that simulate browsers
– – – – implement HTTP request/response protocol some parse HTML into DOM examples: mechanize, webunit, PBP, Twill, MaxQ So called "Web protocol drivers"
Tools that automate browsers
– drive real browsers via JavaScript or COM calls – examples: Selenium, Pamie (IE), Jssh (Mozilla), Watir (Ruby/IE) – So called "Web browser drivers"
Selenium Demo
"TestRunner" mode
Static HTML, JavaScript, CSS and Test Suite deployed alongside the Application under Test (AUT) on the web server
– Both the application and Selenium on the same (arbitrary) web server
HTML Test Tables
Tests written as HTML tables
– similar to FitNesse
Driven Mode
Browser under the control of a process on the same machine
• Drivers available for Java, .Net, Ruby or Python
Leveraging a unit testing framework for functional testing Test Suite coded in Python (or another language)
Selenium Engine
Browser-Bot JavaScript engine
– Embedded in the Browser – Browser can be minimized during the tests
Engine drives the Browser
– Reads test tables written in Selenese – Executes Selenese commands – Accesses and runs Application under Test (AUT) in separate frame – Interacts with page elements – Runs tests or test suites automatically or interactively step by step – Keeps result statistics
• Optionally POST results back to server
HTML Test Tables
Simple HTML Page
– HTML Table rows contain the documentation and the commands
First row for documentation:
– <tr><td colspan="3">My Test</td></tr>
Further rows for Selenese commands:
– <td>COMMAND</td><td>TARGET</td><td>VALUE</td>
Two types of commands:
– Actions – Checks
Selenese Actions
manipulate the state of the application click, clickAndWait
– works on any clickable item
select, selectAndWait
– <select> widget
type, typeAndWait
– <input> or <textarea>
open (Don't overuse, as users rarely type a url in the browser)
– URL
Actions stop current test on failure or error
Selenese Checks
verify that the state of the application conforms to what is expected verifyLocation, assertLocation
– Location of the current page, e.g. "/Plone"
verifySelected, assertSelected
– Label of the selected option of a drop-down
verifyTextPresent, assertTextPresent
– Text appears somewhere on the page
Checks usually come in pairs:
– assert* checks
• Stop the test on failure
– Verify* checks
• Record failure and continues
Element Locators
Element Locators tell Selenium which HTML element a command refers to Identifier: Find element based on the id or name attribute of the element.
– example: TheElementId, TheElementName
DOM: Find element with built-in DOM traversal syntact of HTML
– example: document.forms['myForm'].myDropdown
Xpath: Locate Element using a defined Xpath expression
– example: //table[@id='table1']//tr[4]/td[2]
Variables
Selenium handles simple variables Commands:
– storeValue – storeText
Variables can be used in the target or value column:
– ${variableName}
Extending Selenium
No need to change core Selenium user-extension.js
– File gets loaded by Selenium if it exists
user-extension.js.sample
– contains demo code of how to write your own
• Action • Check • Locator Strategy
Test Development #1
Testing plan
– Contains all the test cases that need to be run – Should describe
• methodology used for testing • desired results
Testing Plan for "Simple Plone Search"
– Basic check for Plone Search – Only one example test case:
• 1. The user enters "Welcome to Plone" in search field • 2. The user clicks on search • 3. The results page contains "Welcome to Plone" text
Test Development #2
Open web browser with application under test
– View Source – Use DOM Inspector
Test Development #3
How to locate the search text field?
• Either by id: searchGadget or by name: SearchableText
Test Development #4
How to identify the search button to be pushed?
• //input[@value='Search']
Test Development #5
We should end up on the search template
• assert the location is "/Plone/search"
Test Development #6
What text needs to be checked?
• verify the text "Welcome to Plone" is present
Test Development #7
Write Test
Test Development #8
Run the Test
Zelenium
Zelenium is a Selenium Zope product
– By Tres Seaver – create TTW Selenium test suites inside the ZMI
Quick Start
– Install Zelenium / ExternalEditor products – Add Zuite instance – Populate with File instances
• content type: text/html • contain the test table table
– Point at filesystem using property
Resources:
– http://www.zope.org/Members/tseaver/Zelenium
PloneSelenium #1
Ingeniweb product Lets you define tests with a simple Python Script The goal is to move it into Plone
– Plone Roadmap #100 – Plone Sprint after EuroPython
Not a reimplementation of Zelenium
– Export to Zelenium to run static test suites
Not a reimplementation of Selenium for Plone by Jason Huggins
– PloneSelenium uses the Dispatcher and Interpreter
PloneSelenium #2
One Python Script per test
– Integrated development mode in a Plone configlet – Python programmers want the expressiveness of Python
• HTML table approach and Selenese command language limited
– Don't necessarily want to use Unit Testing framework to do some quick tests
Concentrates on one Test under development
– Tests are broken up into subtests
• SubTests have a title for better documentation • Restart subtests in development at a given point
– Import subtests from other test library by name – Exports to Zelenium once the test is stable
PloneSelenium #3
The “Simple Plone Search“ Test in Python Script
Plone Selenium #4
PloneSelenium #5
Install PloneSelenium from CVS
– http://ingeniweb.sourceforge.net/Products/PloneSeleni um/
Go to Selenium Configlet Add a functional test Edit the generated Python Script
– Created in portal_skins/custom – Contains the test boilerplate
Run the Test in development mode Register other tests in skin path
– have to be named "get" + mytestname + "ftests".py
Export Tests to Zelenium Test Suite
Selenium in Plone 2.2
Plone RoadMap #100:
– "Integrate Selenium for functional testing" – Planned for Plone 2.2
Plone Selenium Sprint
– Implement the initial testing plan – Two days after EuroPython 2005 – Sprint Wiki:
• http://wiki.python.org/moin/PloneSeleniumSprint
Resources
Selenium site
– http://selenium.thoughtworks.com
Zelenium product
– http://www.zope.org/Members/tseaver/Zelenium
PloneSelenium
– http://ingeniweb.sourceforge.net/Products/PloneSeleni um/
Credits
Some slides contain material from following presentation:
– "Agile Testing with Python Test Frameworks"
• Grig Gheorghiu • PyCon 2005, Washington D.C., March 22, 2005 • http://www.python.org/pycon/2005/papers/10/
I took some inspiration and wording from Tres Seaver's presentation:
– "Zelenium: Browser Testing for Zope"
• Tres Seaver • Fredericksburg Zope / Python Users' Group 2005/06/08 • http://palladion.com/home/tseaver/FXBGZPUGZelenium.sxi/file_view
Thanks for Listening!
Maik Röder
– [email protected]
Ingeniweb
– http://www.ingeniweb.com
Ingeniweb Support
– [email protected]
Web Application Testing Maik Röder [email protected] http://www.ingeniweb.com
What is Selenium?
A test tool for web applications Selenium tests run directly in the browser
– Just like real users do – Browser plays back testing scripts – Uses JavaScript
Wide array of browsers and platforms supported
– Internet Explorer
• Windows
– Mozilla and Firefox
• Windows • Mac • Linux
– Safari
• Mac OS X
Web application testing
Two categories of test tools Tools that simulate browsers
– – – – implement HTTP request/response protocol some parse HTML into DOM examples: mechanize, webunit, PBP, Twill, MaxQ So called "Web protocol drivers"
Tools that automate browsers
– drive real browsers via JavaScript or COM calls – examples: Selenium, Pamie (IE), Jssh (Mozilla), Watir (Ruby/IE) – So called "Web browser drivers"
Selenium Demo
"TestRunner" mode
Static HTML, JavaScript, CSS and Test Suite deployed alongside the Application under Test (AUT) on the web server
– Both the application and Selenium on the same (arbitrary) web server
HTML Test Tables
Tests written as HTML tables
– similar to FitNesse
Driven Mode
Browser under the control of a process on the same machine
• Drivers available for Java, .Net, Ruby or Python
Leveraging a unit testing framework for functional testing Test Suite coded in Python (or another language)
Selenium Engine
Browser-Bot JavaScript engine
– Embedded in the Browser – Browser can be minimized during the tests
Engine drives the Browser
– Reads test tables written in Selenese – Executes Selenese commands – Accesses and runs Application under Test (AUT) in separate frame – Interacts with page elements – Runs tests or test suites automatically or interactively step by step – Keeps result statistics
• Optionally POST results back to server
HTML Test Tables
Simple HTML Page
– HTML Table rows contain the documentation and the commands
First row for documentation:
– <tr><td colspan="3">My Test</td></tr>
Further rows for Selenese commands:
– <td>COMMAND</td><td>TARGET</td><td>VALUE</td>
Two types of commands:
– Actions – Checks
Selenese Actions
manipulate the state of the application click, clickAndWait
– works on any clickable item
select, selectAndWait
– <select> widget
type, typeAndWait
– <input> or <textarea>
open (Don't overuse, as users rarely type a url in the browser)
– URL
Actions stop current test on failure or error
Selenese Checks
verify that the state of the application conforms to what is expected verifyLocation, assertLocation
– Location of the current page, e.g. "/Plone"
verifySelected, assertSelected
– Label of the selected option of a drop-down
verifyTextPresent, assertTextPresent
– Text appears somewhere on the page
Checks usually come in pairs:
– assert* checks
• Stop the test on failure
– Verify* checks
• Record failure and continues
Element Locators
Element Locators tell Selenium which HTML element a command refers to Identifier: Find element based on the id or name attribute of the element.
– example: TheElementId, TheElementName
DOM: Find element with built-in DOM traversal syntact of HTML
– example: document.forms['myForm'].myDropdown
Xpath: Locate Element using a defined Xpath expression
– example: //table[@id='table1']//tr[4]/td[2]
Variables
Selenium handles simple variables Commands:
– storeValue – storeText
Variables can be used in the target or value column:
– ${variableName}
Extending Selenium
No need to change core Selenium user-extension.js
– File gets loaded by Selenium if it exists
user-extension.js.sample
– contains demo code of how to write your own
• Action • Check • Locator Strategy
Test Development #1
Testing plan
– Contains all the test cases that need to be run – Should describe
• methodology used for testing • desired results
Testing Plan for "Simple Plone Search"
– Basic check for Plone Search – Only one example test case:
• 1. The user enters "Welcome to Plone" in search field • 2. The user clicks on search • 3. The results page contains "Welcome to Plone" text
Test Development #2
Open web browser with application under test
– View Source – Use DOM Inspector
Test Development #3
How to locate the search text field?
• Either by id: searchGadget or by name: SearchableText
Test Development #4
How to identify the search button to be pushed?
• //input[@value='Search']
Test Development #5
We should end up on the search template
• assert the location is "/Plone/search"
Test Development #6
What text needs to be checked?
• verify the text "Welcome to Plone" is present
Test Development #7
Write Test
Test Development #8
Run the Test
Zelenium
Zelenium is a Selenium Zope product
– By Tres Seaver – create TTW Selenium test suites inside the ZMI
Quick Start
– Install Zelenium / ExternalEditor products – Add Zuite instance – Populate with File instances
• content type: text/html • contain the test table table
– Point at filesystem using property
Resources:
– http://www.zope.org/Members/tseaver/Zelenium
PloneSelenium #1
Ingeniweb product Lets you define tests with a simple Python Script The goal is to move it into Plone
– Plone Roadmap #100 – Plone Sprint after EuroPython
Not a reimplementation of Zelenium
– Export to Zelenium to run static test suites
Not a reimplementation of Selenium for Plone by Jason Huggins
– PloneSelenium uses the Dispatcher and Interpreter
PloneSelenium #2
One Python Script per test
– Integrated development mode in a Plone configlet – Python programmers want the expressiveness of Python
• HTML table approach and Selenese command language limited
– Don't necessarily want to use Unit Testing framework to do some quick tests
Concentrates on one Test under development
– Tests are broken up into subtests
• SubTests have a title for better documentation • Restart subtests in development at a given point
– Import subtests from other test library by name – Exports to Zelenium once the test is stable
PloneSelenium #3
The “Simple Plone Search“ Test in Python Script
Plone Selenium #4
PloneSelenium #5
Install PloneSelenium from CVS
– http://ingeniweb.sourceforge.net/Products/PloneSeleni um/
Go to Selenium Configlet Add a functional test Edit the generated Python Script
– Created in portal_skins/custom – Contains the test boilerplate
Run the Test in development mode Register other tests in skin path
– have to be named "get" + mytestname + "ftests".py
Export Tests to Zelenium Test Suite
Selenium in Plone 2.2
Plone RoadMap #100:
– "Integrate Selenium for functional testing" – Planned for Plone 2.2
Plone Selenium Sprint
– Implement the initial testing plan – Two days after EuroPython 2005 – Sprint Wiki:
• http://wiki.python.org/moin/PloneSeleniumSprint
Resources
Selenium site
– http://selenium.thoughtworks.com
Zelenium product
– http://www.zope.org/Members/tseaver/Zelenium
PloneSelenium
– http://ingeniweb.sourceforge.net/Products/PloneSeleni um/
Credits
Some slides contain material from following presentation:
– "Agile Testing with Python Test Frameworks"
• Grig Gheorghiu • PyCon 2005, Washington D.C., March 22, 2005 • http://www.python.org/pycon/2005/papers/10/
I took some inspiration and wording from Tres Seaver's presentation:
– "Zelenium: Browser Testing for Zope"
• Tres Seaver • Fredericksburg Zope / Python Users' Group 2005/06/08 • http://palladion.com/home/tseaver/FXBGZPUGZelenium.sxi/file_view
Thanks for Listening!
Maik Röder
– [email protected]
Ingeniweb
– http://www.ingeniweb.com
Ingeniweb Support
– [email protected]
