Search Engine

This is a description of my search engine CGI program.

Overview

Douglas Thrift's Search Engine is an indexing search engine for use on small websites such as personal or small business sites. It is designed to be very similar to Google for end users and its output is customizable. For indexing, it supports both the Robots Exclusion Protocol and the Robots META Tag as specified at http://www.robotstxt.org/wc/exclusion.html.

The search engine is written in C++ and works on Windows, Linux, FreeBSD, and other UNIX systems.

Douglas Thrift's Search Engine is Open Source software according to the The Open Source Definition. Its License is identical to the Apache Software License except that it refers to me, Douglas Thrift, as the copyright holder.

This software is OSI Certified Open Source Software.
OSI Certified is a certification mark of the Open Source Initiative.

You are free to use to following image on pages produced by Douglas Thrift's Search Engine as well as pages where you have a Search form for Douglas Thrift's Search Engine:

If you would like to submit your own "Powered by Douglas Thrift's Search Engine" images email them to me as attachments, and if I approve of them I will add them to a "Powered by" images page on this website. I will not tolerate any inappropriate images, if you don't know what I mean by that don't waste my time.

If you would like to learn why I started this project and more about its history check out my Historical page.

This project is listed at freshmeat.net as Douglas Thrift's Search Engine. It is also listed in Open Directory - Computers: Open Source: Software: Internet: Web Applications and Zeal.com - United States - New - Computing - Computer Science - Programming - Open Source - Software - More Software.

Samples

Here are some examples of my Search Engine's output using the default templates:

Note: You can also view the template files used for my website's Search in search.

Download

Note: Douglas Thrift's Search Engine is copyrighted material and is released with a license agreement. This license is available in several formats here:

Douglas Thrift's Search Engine License (HTML)
Douglas Thrift's Search Engine License (Plain Text)

Note: Before downloading Douglas Thrift's Search Engine, I suggest that you read the ReadMe file; it is available in several formats here:

Douglas Thrift's Search Engine ReadMe (HTML)
Douglas Thrift's Search Engine ReadMe (Plain Text)

Here is my Search Engine for your downloading, compiling, and running pleasure:

Source Code Distribution 1.2	Size	Description
dtse-1.2-src.zip	91.1 kB	Source Code Distribution
dtse-1.2-src.tar.bz2	83.0 kB	Source Code Distribution
dtse-1.2-src.tar.gz	92.7 kB	Source Code Distribution
Binary Distribution 1.2	Size	Description
dtse-1.2-win32.exe	392.8 kB	Windows Installer
dtse-1.2-openssl-win32.exe	397.5 kB	Windows Installer OpenSSL

Note: The distribution files above and some other goodies are located in download.

Bugs

Although I believe I did a pretty good job of coding and getting bugs out of this software, there is always a possibility that there could be more. Since this is Open Source software, I have automatically submitted my work for peer review, and I appreciate help in making my Search Engine the best it could possible be. So without any further ado, you may use the form on my Bug Report page to submit bugs you have found.

Note: This form is for submitting actual bugs in my Search Engine, it is not to be used for technical support or help questions.

Mailing Lists

Note: I once had a mailing list for my Search Engine, but I never restarted it when I moved to a new server and I am not planning on restarting it. Anyway, I do have the archives available here.

File	Description
searchengine_computers.douglasthrift.net.mbox	Complete Archive Mailbox
2003-February.txt	February 2003 Archive
2003-April.txt	April 2003 Archive
2003-May.txt	May 2003 Archive
2003-June.txt	June 2003 Archive
2003-July.txt	July 2003 Archive
2003-September.txt	September 2003 Archive
2004-March.txt	March 2004 Archive
2004-April.txt	April 2004 Archive

Documentation

Here are some instructions for getting started using my Search Engine:

You should copy the files in the http directory to somewhere in your web server's document root.
The actual Search Engine program that handles both indexing and searching is Search which is located in the bin directory. If you want to index your site, which we will pretend is www.site.com, you will run it as follows; with the default settings you should run the program in the data directory:
[douglas@topsecret:data]$ Search index.xml -i http://www.site.com/ -d www.site.com
The index will be stored in index.xml and you should be able to search it. To test the index you can run the program again with different arguments and enter a search query.
[douglas@topsecret:data]$ Search index.xml douglas thrift
The output will be HTML so you might want to redirect it to a file that you can open with your browser.
You can get information about all the command line arguments for the Search program using the -help argument.
[douglas@topsecret:data]$ Search -help Usage: Search [index ...] [-P page] [-h header] [-b body] [-f footer] [-n notfound] [-p pages] [-i begin] [-d domain ...] [-r restriction ...] [-D] [-version] [-help] Options: index Index file to use (can only use one file for indexing) -P page Page of search to display (defaults to 1) -h header Header template to use (defaults to header.html) -b body Body template to use (defaults to body.html) -f footer Footer template to use (defaults to footer.html) -n notfound Not found template to use (defaults to notfound.html) -p pages Pages template to use (defaults to pages.html) -i begin URL to begin indexing (causes indexing rather than search) -d domain Domain to include in indexing -r restriction URL to restrict from indexing -D Display debug information -version Display version information and exit -license Display license information and exit -help Display this message and exit

There is currently no other documentation for my Search Engine, though I plan to write some when I have the time. For help using my Search Engine, I suggest you use the mailing lists mentioned above; I will use discussion in the mailing lists to help me gage what to put in the documentation and a FAQ page.