For data to be used, it must first be found. And if there is no proper categorization of data and files, it would be really difficult to work with them. Search Engines and Directories approach searching in two different ways. While Directories do the more logical process of classification and organization by putting a structure to data, Search Engines do the more realistic thing of assigning search parameters to data thereby helping in searching them.
Search Engines and Directories became a reality as soon as the internet crossed a certain popularity threshold. Internet was used by college students and professors, research scholars, libraries, scientists, governments, private individuals, and soon enough, by business and Big Business. This generated an awful lot of documents and files, multiple file formats and formatting, and of course, a colossal amount of data and information. Cause of the chaos was that no standards had been accepted, and there was no central authority to recommend or install systematic structure. Creating and publishing a document was so easy that no efforts were made to adhere to basic propriety or stick to the standards that had gained some acceptance. The result was universal chaos. Search engines and directories stepped into to alleviate the situation.
A Search Engine is any software that searches for files and documents meeting certain search criteria which are presented by the user. The search query, which contains the criteria, is used to tally with the database, and the tallied documents presented to the user. The Search Engine can be on an individual computer, a corporate network, on an individual site (the site-search) or on the Web, and in each case tries to help find documents against certain search criteria.
The Web Directory is an index of files on the Web where they are categorized according to their content or theme, making it more intuitive and easier to find them. The information revolution has ensured that many search engines and directories have become part and parcel of our lives. The very reliance on them requires of us that information generated be offered to them for indexation so that others may know about it. If the Search Engine or Directory does not know about any content, it effectively does not exist.