|Subject:||RE: A more difficult search engine|
Response to: RE: A more difficult search engine
Giff is probably writing for Google by now, but maybe this will help someone else. This is actually not as tough as it might appear. First, add an extra field to the pages table -call it page_text, and make it the text datatype so that it's not constrained size-wise. Next, in your spider, insert an extra line as follows:
This stores the entire page text, stripped of tags, in the table as a contiguous string.
This query searches the pages table for instances of the keyword phrase within the full text of the page.
Hope this helps!