Understanding Search Engines

Inhaltsverzeichnis

1. Frontmatter

2. 1. Introduction

   Dirk Lewandowski

   Abstract

   This chapter introduces the unique approach taken in this book, namely, integrating five perspectives on Web search and search engines that are usually dealt with separately. These five perspectives are the technical perspective, the user perspective, the Internet-based research perspective, the economic perspective, and the societal perspective. This chapter also introduces the critical role that Google plays in the search engine market and how this book will focus on Google as the most important search engine and look into what other players on the market can offer in terms of alternatives.

3. 2. Ways of Searching the Web

   Dirk Lewandowski

   Abstract

   This chapter describes the different ways users can obtain information from the Web, e.g., through Web directories or social media. It explains why search engines have become the almost exclusive means of accessing information from the Web. The discussion on different approaches for making Web content available provides background for subsequent chapters focusing on the strengths and limitations of search engines.

4. 3. How Search Engines Capture and Process Content from the Web

   Dirk Lewandowski

   Abstract

   This chapter describes the technical basis of search engines. This basis includes how the documents available on the Web are brought into the search engine and how they are made searchable, as well as how the link between a search query and the documents in the database is established. Details on the workings of the crawler, the indexer, and the searcher are given.

5. 4. User Interaction with Search Engines

   Dirk Lewandowski

   Abstract

   This chapter focuses on the user side and looks at what and how people search in search engines. Knowing user behavior helps us understand actual search engine users and transfer core elements of this behavior to other information systems and, if necessary, consider them when creating our own systems. On the one hand, search engines have shaped the behavior of their users through the search features they offer and the way they present results. But, on the other hand, as users, we also shape the structure of search engines, which adapt to our behavior. This background provides the basis for a thorough discussion of search engine rankings.

6. 5. Ranking Search Results

   Dirk Lewandowski

   Abstract

   Ranking is about putting the search results in a useful order so that the most relevant results are shown first and less relevant results further down the list. Results are sorted in descending order of relevance, i.e., the higher a result is in the list, the more relevant it is. Today’s search engines still display results in a list but enrich it with many results from other collections. Having results from different collections makes ranking more complex as there is no longer just one list that has to be ranked, but many results from various sources are compiled on the search engine result page. This chapter provides a detailed discussion of ranking approaches, dividing ranking into five areas: text matching, popularity, freshness, locality, and personalization.

7. 6. Vertical Search

   Dirk Lewandowski

   Abstract

   This chapter discusses the problems of general search engines and explains why vertical search engines, i.e., search engines that focus on a specific topic or file format, have been developed to make specific content easier to find. Furthermore, these vertical search engines are integrated into general Web search (through so-called universal search). This chapter deals with the structure and content of these vertical collections and their integration into general search engines. After explaining the generalities of such collection, it discusses for specific collections: news, scholarly content, images, and video.

8. 7. Search Result Presentation

   Dirk Lewandowski

   Abstract

   This chapter focuses on the presentation of search results. For some years now, the well-known search engines have deviated from the standard list form of search result presentation. Instead, they have established new forms of compiling search results with concepts such as Universal Search and Knowledge Graph. These changes made the result pages more attractive and increased the number of choices available. However, they also allow search engine providers to guide users’ attention in a targeted way, e.g., leading them to the provider’s offerings.

9. 8. The Search Engine Market

   Dirk Lewandowski

   Abstract

   Search engines are operated by commercial companies and must therefore not only be refinanced but also generate profits. This chapter explains which search engines play an important role commercially, how search engines earn money, and what partnerships exist among companies in the search engine market. The search engine market is dominated by Google, and other search engines only play a minor role. Many seem-to-be alternative search engines get their results from either Google or Bing and should, therefore, not be considered search engines but search portals. The information provided in this chapter provides background for discussing alternative search engines (Chap. 11) and the role of search engines (Chap. 15).

10. 9. Search Engine Optimization (SEO)

    Dirk Lewandowski

    Abstract

    This chapter deals with approaches how to make content available in the best possible way via search engines—i.e., how to exploit search engine optimizers. They use knowledge of the indexing and ranking procedures to make content easier to find and hence increase the number of visitors. Different techniques are used for this, ranging from simple text modification to complex procedures that consider the linking structure on the Web. The chapter provides an overview of search engine optimization techniques and discusses the implications of search engine optimization on result quality.

11. 10. Search Engine Advertising (SEA)

    Dirk Lewandowski

    Abstract

    This chapter describes in detail the advertising that is displayed in search engines. On the one hand, the ads shown on the result pages will be considered a type of search results, and on the other hand, the extent to which users can distinguish these ads from the actual search results will be addressed. As with search engine optimization, search engine advertising is a way to influence users in their knowledge acquisition processes. Therefore, this chapter focuses on how advertisements are generated, how their relevance is determined, and how users who may not distinguish ads from organic results might be influenced by certain content.

12. 11. Alternatives to Google

    Dirk Lewandowski

    Abstract

    This chapter focuses on the existing alternatives to Google. First, it shows different definitions of what makes a search engine an alternative search engine. Then, based on fundamental considerations and concrete search scenarios, it shows in which cases it is worth switching to a different search engine.

13. 12. Search Skills

    Dirk Lewandowski

    Abstract

    This chapter looks at search engines as tools for advanced Web-based research and provides an overview of essential search strategies and commands supported by the major search engines.

14. 13. Search Result Quality

    Dirk Lewandowski

    Abstract

    In this chapter, we examine the quality of search results from two different angles: On the one hand, it deals with how users can assess the quality of the individual results displayed by a search engine; on the other hand, it deals with research methods that can be used to systematically evaluate and compare the quality of the search results displayed by search engines.

15. 14. The Deep Web

    Dirk Lewandowski

    Abstract

    The part of the Web that cannot be or is not captured by search engines is referred to as the Deep Web. In this chapter, we discuss the role of the Deep Web, how its content can be accessed, and search engines’ approaches to integrating at least part of the Deep Web in their indexes.

16. 15. Search Engines Between Bias and Neutrality

    Dirk Lewandowski

    Abstract

    In this chapter, we focus on the societal role of search engines. What role do search engines play in knowledge acquisition today, and what role should they play? We start discussing this by providing background on search engine providers’ self-interests and then focus on search engine bias. Next, we categorize existing biases and show how they influence results. Based on this, we discuss how search engines could provide a fair result presentation and, more generally, how fair search could be made possible.

17. 16. The Future of Search

    Dirk Lewandowski

    Abstract

    This chapter considers future developments in search technology and the role that search engines will play economically and socially. It aims to look into the foreseeable future and focus on developments that are already emerging but have not yet come to fruition. We find that search engine technology will provide the basis for many applications, including personal assistants. However, we argue that the current market conditions hinder further technical development. A key to changing this situation that is not beneficial to users and content producers alike is not to regard search engines as neutral information brokers anymore.

18. Backmatter