Universal Search
Computer Sciences and Information Technology
This group assignment aims to answer the following question “What are the challenges and limits of universal search for search engines?”
After having identified the key players in universal search, you will outline the main challenges of this evolution of search engines, by presenting the challenges, achievements, and limits of universal search.
Main Actors Of Universal Search
The primary actor in universal search is the end-user who at the time of looking for information, gets the exact word or information that is almost accurate to their keywords. In universal search, the user will require the information to be integrated with other media such as videos, images or maps displayed together with the organic results (Search Metrics, 2020). The content provided needs to be done precisely with the user’s intention. Notably, for this kind of information to be provided, three fundamental components on the search engine have to play their essential roles (Tutorials point, 2020). These include the web crawler, the databases and the search interfaces. The web crawler that is also called the spider or bots is the software actor that will traverse the web to gather information. The database is the storage unit for all the information on the web, and it constitutes massive web resources. Finally, the search interface is the component between the user and the database. The search interface aids the user to search through the database.
These three components are what will make the search engine to work accordingly. The search engines will then utilize the Boolean expression to and not for restricting and broadening the search results. To this effect, the three steps carried out by the search engine are. First, it looks for the keywords input in the index for the predefined database rather than directly going into the web searching for the keyword. The engine then utilizes software for searching for information within the database. This software is a web crawler. Finally, the search engine displays the relevant web pages after the web crawler finds them, the retrieved web pages will generally include the page titles, size of the text portion, several sentences in the web page among others.
The Challenges Of Its Evolution
Despite the successes achieved with the evolution into universal searches in search engines, the latter still face numerous challenges. One of them is the constant need to have the quality of these search results improved through a better alignment with the users’ needs, preferences and intentions to the content displayed in the web pages (Kim et al., 2008). The universal search must focus on decreasing the appearance of irrelevant search results that will continuously appear in the first pages forcing the users to scrutinize the content looking for what they exactly wanted. An example of irrelevant search results that are not in line with the needs and preferences of the user will be when one searches for the “Korean Consulate General office in Houston” then the results fail to include the homepage to this office. Rather one web pages and links to scholarship funds and training centres that contain the key terms searched.
In working towards improving the quality of the results from search engines, it is challenging to enhance ranking algorithms without having tests run for comparing the quality of the incoming ranking technique to the past online, to perform these comparisons and evaluations, especially with the use of human evaluators can be very work-intensive. There is also a risk of failing to reflect the needs of the users properly. To this effect, it is best when the end-users are involved in the evaluation activities as they understand their individual needs better. Nonetheless, the end-users are very reluctant in giving direct feedback. The web search engines can only collect the implicit user feedback through log data such as the position of clicks in the search or the time spent within one click. Still, this data is incomplete.
An example is when a user clicks on a search result, the engine will not be aware of the pages that the user visits until they return to the search engine. It is also challenging to determine whether the user clicking on that page will find the information relevant or useful, the incomplete nature of knowledge calls for the implementation of the experimental setup used in implicit user information. The process entails finding how the click-through and other information are to be collected together with the computation of metrics from the report.
The second challenge is spam, that is generally among the relevant search results. Some of the web authors will intentionally attempt to manipulate their placement in the search engine’s rankings resulting in spam. In the traditional information retrieval collections, the information rarely contains spam; hence the minimal research done on ensuring that search algorithms resist the spam techniques (Henzinger et al., 2002). Conversely, the web search engines have been in constant development and improvement of strategies that will detect and fight the spam. However, with these news developments of detecting, preventing and fighting, the malicious developers are already looking for new spam techniques to respond. Generally, the search engines will not publish their advanced anti-spam methods in avoiding the spammers from circumventing them. The historical trends indicate that the occurrence of spam will continue to increase in terms of utility and variety (Henzinger et al., 2002). This demonstrates that the developers have a huge challenge to beat about spam in search results.
Another challenge faced by the users has to do with abusive practices such as viruses, identity theft and break-ins (Abbate, 2017). The technical experts have tried to bring in solutions that would minimize these prevailing risks, including anti-virus systems, filters, secure online transactions, and better security systems. However, technological advancements limit the protection they can provide as the malware attackers develop their viruses and malware that beat the security systems. Also, some of the issues are too divisive to be handled through a technical solution as they need to satisfy a conflicting public opinion, especially when the problems go beyond national territories. Some state governments have put severe limits and closely monitor their citizens’ online activities (Abbate, 2017). While the human rights groups are constantly calling out the censorship and intimidating surveillance, these governments have asserted that they are focussed on protecting the citizens’ rights to public safety and morality. Some groups have indicated that the internet has become too open to content that is objectionable and illegal. While the filters and copyright protection devices provided ways to restrict the information flow, the devices themselves have their controversy. Political ramifications will also hinder particular transitions such as that of the Internet Protocol where UIPv4 was to be developed into IPv6 for a larger number of addresses, but disagreements would slow down its incorporation.
The evolution of the search engines will continue to present further technical issues, among other aspects. Uit is necessary that the developers engage in further research and development to see how they could bring in new systems that will cater to these challenges entirely.
References
Abbate, J. (2017). The internet: Global evolution and challenges. Retrieved from https://www.bbvaopenmind.com/en/articles/the-internet-global-evolution-and-challenges/
Henzinger, M. R., Motwani, R., & Silverstein, C. (2002, September). Challenges in web search engines. In ACM SIGIR Forum (Vol. 36, No. 2, pp. 11-22). New York, NY, USA: ACM.
Kim, W., Jeong, O. R., Ji, H., & Lee, S. (2008). On Web Search: Some Activities and Challenges. J. Object Technol., 7(3), 49-54.
Search Metrics. (2020, October 13). What is universal search – All info in the Searchmetrics glossary. Retrieved from https://www.searchmetrics.com/glossary/universal-search/
Tutorials point. (2020). Search engines. Retrieved from https://www.tutorialspoint.com/internet_technologies/search_engines.htm#
