Boolean Searching in freeWAIS-sf

Table of Contents:-

1. Introduction and Credits
2. Summary
3. Upper/Lowercase for Search Terms & Boolean Operators
   1. Search terms
   2. Search terms and Boolean operators
4. Hierarchy of Boolean Operators
5. Truncation of Search Terms
6. Stemming

1. Introduction and Credits

This document is a free adaptation from four different documents prepared by:
Natalie Oakes Sturr
Systems Librarian, Penfield Library,
SUNY Oswego, Oswego, NY 13126
E-mail: sturr@oswego.edu

By navigating the Web, you can see these four original documents:

• Boolean Searching in WAIS Databases
  
• Hierarchy of Boolean Operators
  
• Truncation & Stemming
  
• Use of Upper or Lowercase for Search Terms and Boolean Operators
  

2. Summary

1. Search Terms: only lowercase terms search works consistently
2. Boolean Operators:
   • Both upper and lowercase operators work consistently
   • Evaluated in the following order:
     1. Operations in parentheses
     2. NOT
     3. AND
     4. OR
3. Truncation:
   • Works correctly and consistently when Porter stemming is turned off
   • When Porter stemming is turned on, search results can be misleading
4. Stemming: Porter stemming available as an option

3. Upper/Lowercase for Search Terms & Boolean Operators

a. Search terms

b. Search terms and Boolean operators

4. Hierarchy of Boolean Operators

In freeWAIS-sf, the hierarchy of Boolean operators is:

1. Operations in parentheses
2. NOT
3. AND
4. OR

NOT is evaluated before AND, which is evaluated before OR. Operations in parentheses are evaluated first of all.

Examples:

5. Truncation of Search Terms

Right-hand truncation, denoted with an asterisk (*), is listed as a feature of freeWAIS-sf.
This package also provides stemming, similar to automatic right-hand truncation, as an option (see later).

freeWAIS-sf, right-hand truncation produces correct results as long as stemming is NOT turned on.

• Play* retrieves documents containing the words play, plays, player, playful, and playground
• Plain* retrieves documents containing the words plain, plains, plainly, and plaintiff.
• Set*, retrieves documents containing the words set, settle, settled, settler and settlers.

6. Stemming

Porter stemming attempts to identify and index the word stem. If a word and its stem are different, only the word stem is indexed. Thus, it appears to the user that search terms are automatically truncated. For example, both physics and physical stem to physic.

Plural stemming attempts to identify and index the singular form of a search term. When searching either the singular or plural form of a term, both are retrieved:

Although Porter stemming is based on a computer algorithm, the English language is not! This can cause search statements to return unexpected results. For example, play is stemmed to plai. These inconsistencies are compounded when combined with right-hand truncation.

Data used to determine how truncation and stemming are implemented in freeWAIS-sf are presented below. They refer to freeWAIS-sf 1.1 that is the more recent version tested.
All searches were performed on a small database constructed for the purpose of testing truncation. The database has 10 records (one word per record) and was indexed using the paragraph format (-t para). waissearch was used to search the databases. The words listed are those retrieved with each search statement. The database contains the following words:

play, plays, player, playful, playground,
plain, plains, plainly, plaintiff, plait