Proximity Operations

A _proximity_ query searches for terms that are either near each other or occur in a specified order in a document rather than simply whether they occur in a document or not.

h3. Phrase Proximity Queries

Exact phrase matching is a powerful query tool, but frequently the phrasing used in relevant documents is not exactly the same. It is commonly the case that there are extra terms, or the terms may be in another order. In other cases, the phrase terms may be relatively near, with quite a few extra words between them. For example, the following two queries may return different results even though they are semantically equivalent:

{{"team development"}}

{{"development of teams"}}

The difference between the two is an extra word in the middle and a reversal of the two key terms.

We can write a single _phrase proximity query_ that will match both phrases:

{{"team development"~3}}

The tilde ("~") is used after a quoted phrase to indicate a phrase proximity search. It is followed by an integer (whole number) which is the maximum editing distance for phrases that will match the query phrase. The editing distance treats each term as a single unit and measures how many unit terms need to be moved to translate from one phrase to another. In this case, it takes one unit to move "team" to "of", a second unit to move it to "development", and a third unit to move it before "development".

To query for two terms that are within 50 words of each other:

{{"cat dog"~50}}

To query a person's name and allow for an optional middle initial:

{{"John Doe"~1}} matches {{"John Doe"}} and {{"John Q. Doe"}}

To query a person's name and allow for both first name first or last name first:

|| Advanced Operator || Sample Query || Matches ||
| {{NEAR}} | {{x near y}} | Documents containing "x" within 15 terms of "y", either before or after |
| {{BEFORE}} | {{x before y}} | Documents containing the term "x" no more than 15 terms before the term "y" |
| {{AFTER}} | {{x after y}} | Documents containing the term "x" no more than 15 terms before the term "y" |

{{x before:3 y}}

matches documents containing "x" no more than three terms before "y".

A distance of 0 (zero) means no intervening terms. For example,

{{x before:0 y}}

is the same as:

{{"x y"}}

which matches documents where the terms are adjacent and in that order.

{{x before y before z}}

matches documents containing "x" before "y" with no more than 15 intervening terms and followed by "z" with no more than 15 intervening terms after "y".

The distance limit can be controlled for each proximity operator, such as:

{{x before:10 y before:100 z}}

which requires that there be no more than 10 terms between "x" and "y", but "z" can be up to 100 terms after "y".

Any combination of any number of NEAR, BEFORE, and AFTER proximity operators can be composed into a sequence, such as

{{cat near dog before:50 fox after fish near:3 bat before zebra}}

{{(x before y) before z}}

In fact, the evaluation order does not matter in that example, which could also be written as:

{{x before (y before z)}}

But evaluation order does matter with:

{{x near:3 (y before:50 z)}}

where the intent is that "x" could be shortly before or after either end of the "y"/"z" sequence. But,

{{x near:3 y before:50 z}}

would evaluate as:

{{(x near:3 y) before:50 z}}

which would match "x" close to "y" but not close to "z".

Within parentheses used for operands of proximity operators, only the OR and proximity operators can be used. Other operators will be treated as if they were the OR operator.

{{"First step" before:200 "last step"}}

The terms in the quoted phrase must occur in order, with no intervening terms between the quoted terms.

{{"proposal development"~3 near:50 project}}

Would match the terms "proposal" and "development" (in that order) with no more than three intervening terms and occurring no more than 50 terms before or after "project".

{info:title=Note}
Unlike normal quoted proximity phrases, the phrase terms are expected to occur in order. So, this example will not match {{"development proposal...project"}}.

When several different terms are permitted at a position in a proximity sequence, the alternative terms can be specified using the OR operator and parentheses for either or both terms of the operator. For example,

{{(cd-rom or dvd) before:1 drive}}

would match documents with the term "drive" preceded by either "cd-rom" or "dvd" with at most one intervening term. Alternatives can also be used with composed proximity operators. For example,

{{(cd-rom or dvd) before:1 ((built-in or external) before:0 drive)}}

which requires "built-in" or "external" to immediately precede "drive", but an intervening term is permitted after "cd-rom" or "dvd".

{{pets before animal judgments before book}}

will match the same documents as:

{{pets before "animal judgments" before book}}

* Single term (but no wildcard or fuzzy term)
* Quoted phrase
* Parentheses enclosing:
** One or more proximity operators (evaluated left to right)
** Another term list
** List of term alternatives separated by OR operators. Each term alternative can be a full proximity expression, including nested parentheses.

For example,

{{red (light or sign) picture near street}}

would be equivalent to:

{{("red light picture" or "red sign picture") near street}}

which could also be written using nested term lists:

{{(red light picture or red sign picture) near street}}

which is also equivalent to:

{{((red light picture) or (red sign picture)) near street}}

{{title:x after (body:y near author:z)}}

is evaluated as:

{{title: x after (y near z)}}

A proximity query with no field or the DEFAULT field will query against all of the fields listed in the {{qf}} (query fields) request parameter. The proximity query will be evaluated against each field in turn and the results combined with the disjunction max query operation. But, that will still evaluate the full proximity query expression on only one field at a time.

{anchor:lucidparser.proximity.boolean}

h3. Boolean Operations on Proximity Expressions

Multiple proximity expressions, each with its own field, can be used within a single query simply by combining them with the AND, OR, or NOT boolean operators. The precedence of the boolean operators is such that entire proximity expressions will be evaluated before the surrounding boolean operators. So,

{{title: red before light or body: empty before tank}}

would evaluate as:

{{(title: red before light) or (body: empty before tank)}}

The AND operator can be used to require a set of proximity queries to be satisfied, such as:

{{(title: red before blue) and (body: night after day) and (town near city)}}

where "red" must occur before "blue" in the title field, "night" must occur after "day" in the body field, and "town" must occur near "city" in any field.