1. Formulate five criteria for the evaluation of search enginesTo effectively evaluate three different search engines from the perspective of an advanced web user, the following criteria were established:1. Relevance and accuracy of search results2.
Search speed3. Advanced search options4. Other services5. Site design and layoutMore information about these criteria and how they were used is available below.
2. Test three search engines against your criteriaFor this evaluation, the following three search engines were tested:1. AltaVista – http://www. altavista. com/2.
Google – http://www. google. com/3. DirectHit – http://www. directhit. com/3.
Explain your methodologyThe criteria outlined in Part 1 were reached by considering the things that would be important to an advanced user. Of course, factors like Relevance and Speed are critical to such a user (particularly in a work environment). The search options criteria is particularly important to advanced users who are after a specific page and do not want to waste time ‘wading’ through results from a ‘standard’ search. The other criteria are important to most users and mean that the search engines are more usable and useful. The search results from the three test search engines (Part 2) were evaluate according the criteria, explained it more detail below. Criteria 1 – Relevance and Accuracy of ResultsThis criterion was to evaluate the usefulness of a particular search engine according to the accuracy of its results.
I defined the relevance, according to personal standards, after browsing the pages in the results, for three separate search terms. The Relevance Percentage is calculated by counting the number of relevant pages that appear in the first 20 search ‘hits’ and then multiplying by 20 (for a percentage). Criteria 2 – Search SpeedThis criterion was to evaluate which of the test search engines returns results quicker. However, it is important to note that the speed test is both a test of the Internet connection to the remote machine, as well as the time taken by the search engine to perform the search.
Furthermore, tests of Internet speeds are very variable based on time of day, other internet traffic etc and are therefore of only limited accuracy. For this reason, no ‘hard numbers’ were calculated (ie timed searches). Instead, I rated each search engine based on 5 searches per engine at the same time, same day. The possible Speed Ratings are as follows: Excellent, Good, PoorCriteria 3 – Search OptionsThis criteria was to evaluate the search engines advanced search options. This is important for advanced web users because it allows them to track down specific pages, and weed out irrelevant pages from their searches, meaning they find the most relevant pages quickly. In evaluating this criteria, the three test search engines were tested to see if they had the following search options:Variable number of results per pageSearch for “phrases”Boolean Searches (AND, OR, NOT)Wildcard (*) searchesSearch for text in only title or bodySearch for pages in different languagesSearch for pages located in specific countriesSearch for pages under a certain domainSearch for pages containing links to a URL (eg link:http://www.
yahoo. com/)Search for pages LIKE a certain page (eg like:http://www. yahoo. com/)The number of search options each search engine had available was calculated and then recorded out of 10. Criteria 4 – Additional ServicesThis criteria was to evaluate the other services offered by the search engine to make web searching and browsing easier for advanced users. These features make the search engines more useful and more of a ‘one-stop destination’ for finding information, including news, sport results, TV guides, weather reports and other information in a ‘start page’ design to allow the user to easily access the information he needs quickly.
These services are particularly relevant to the advanced and regular web users because they use the web frequently, making such a page very convenient. In evaluating this criteria, the three test search engines were tested to see if they offered the following additional services/features:Online shopping guideSuggested searches. Searches based on what you have searched forOther searches – whitepages, yellowpages, maps/directions etcHand-picked categories or channels for web browsingPersonalisation of search results, layout, information etcThe number of additional services offered by each of the test search engines was calculated and then recorded out of a score of 5.Criteria 5 – Page Design and LayoutThis criteria was to evaluate the usefulness and usability of the page
