Check Swinburne's subscriptions for full text availabilitySearch Swinburne Research Bank
Please use this identifier to cite or link to this item: http://hdl.handle.net/1959.3/205849
- Title
- Automated functional testing of web search engines in the absence of an oracle
- Author(s)
- Zho, Zhi Quan Zhou; Tse, T. H.; Kuo, F.-C.; Chen, T. Y.
- Abstract
- A software product is functionally correct if it behaves according to the functional specification. Compared with other quality aspects of Web search engines such as performance and capacity, functional correctness is more fundamental but its verification suffers from the oracle problem: It is often difficult or impossible to decide whether outcomes of test case executions are correct. For example, how can testers decide whether the results returned by a Web search engine are actually correct and complete? In this paper, we present an approach to help alleviate the oracle problem in testing search engines. The testing process is fully automatic, including test case generation, execution, output collection, and verification. A tool has been developed to implement the approach and detected failures in popular search engines including Google, Yahoo!, and Microsoft's Live Search. The failures are illustrated using screenshots, and more experimental results are analyzed.
- Publication type
- Technical report
- Research centre
- Swinburne University of Technology
- Source
- CS technical reports
- Publication year
- 2007
- Keyword(s)
- Automated testing; Debugging; Experimentation; Internet searching; Metamorphic testing; Oracle problem; Reliability; Search engines; Software testing; Software verification; Testing tools; Verification
- Publisher
- Department of Computer Science, The University of Hong Kong
- Publisher URL
- http://www.cs.hku.hk/research/techreps/
- Copyright
- Copyright © 2007 The authors.
- Research Projects
-
Metamorphic testing: an in depth study to enhance the failure detection capability of software testing without an oracle, Australian Research Council grant number DP0771733
- Additional information
- Supported by Microsoft Research, the Australian Research Council, and the University of Wollongong.
