Introduction to Search Engines – Part 1

Search engines have changed the way we work with the digital world completely. Digital world around us including the internet has evolved along with search engines almost simultaneously. Here is a primer on the working, technologies, statistics, history, tips for using, searching, etc. for Google.

How do they work?

Search engines work to search across billions of web pages, portals, documents, images, articles, news, journals, etc. to bring relevant and ranked results to your device. They do this by using few concepts like META Tags, Linking of web pages & content with each other, citations, source, web page credibility, social media relationships, configurations on websites like robots.txt / sitemap, etc.

Statistics

What you see as result on the first page of Google India is not the same in Google Europe for the same search word. Search results are customized and localized for each country, individual, session and even device at times. Currently as per publicly available statistics Google gets 40,000+ search requests per second which translates to few billion searches per day and would touch almost 1 Trillion searches approximately per year.

History of Google

Derived from word Googol and started at Stanford University. Initial algorithm was named BackRub.

Technology

Search engines have a spider / crawler which crawls across the internet to search and index / store the data in various databases. Ranking algorithm ranks all the content and refines it as per various parameters. Clustered servers across the world process the requests of search and provide content to devices. Lot of optimization based on new searches and corrections for old search terms / updating based on content changes happen regularly.

Tips & Tricks

Try: <vs.> <“EXACT SEARCH TERM”> <“SEARCH TERM -WITH SPECIFIC WORD EXCULED”> <“SITE: Search within a site”> <“DEFINE:”> <“SearchTerm1 * SearchTerm2 – Search everything including both terms and combinations with anything inbetween”>

Explore options of SETTINGS & TOOLS especially in IMAGE Search

Google Scholar

Search for scholarly papers, articles, theses, journals, books, reports, patents, etc.

Alternative search engines

Bing, DuckDuckGo, Yippy, Ask, DogPile, Yandex, etc.

If you want to learn more about the technologies used by search engines, try out:

Apache Solr

Apache Nutch

In Part – 2 we will discuss advanced topics like AdWords, AdSense, Google Analytics, Google Search Console, new implications of rankings based on https / SSL, details of Nutch & Solr, algorithms for search, other search engines like Yahoo, Bing, etc.