Searching the web for a special component.

The web has become huge. At any one time there are over one trillion accessible URLs and that number is growing rapidly. So how do we find that little bit of information we are hunting amongst all that that extraneous data?

Naturally we use our favourite search engine. Search engines use robots, little pieces of software, that trawl the internet 24/7 and so build up huge repositories of information about what can be found where. They find pages that have many other pages pointing to them through hyperlinks, and then they use hyperlinks they find on these pages to go to more pages, and so on and so forth.

What attracts them is unique information. Very many WebPages are simply copies of others and these are ignored. Even so the sizes of the search engine databases are growing at an enormous rate. The downside is that although search engines have these huge databases they are not very bright as nearly all data is non-contextual. We are simply doing word searches, and this can be extremely frustrating and time consuming.

Enter the Personal Web Agent. You can consider a web agent to be your invisible friend who works by your side and learns to understand you just as a true friend should. Your Personal Web Agent will understand your goals and your interests. It will work alongside you, retrieving information in an intelligent manner. It will adapt to your current interests and work load, analysing, understanding the nature and meaning of the information it retrieves, filtering it and presenting you with its results.

You can’t buy one yet, but there are groups of computer scientists all over the world that are working on these concepts in order to turn them into a reality. There are some prototypes that you can play with but for now they lack the intelligence that will be needed eventually. The new web structures along with developments in personal agent software will combine to make these advances possible and searching for that special bit of information or component will become a far less frustrating experience.