Software:Comparison of search engines

From HandWiki
Short description: None


Web search engines are listed in tables below for comparison purposes. The first table lists the company behind the engine, volume and ad support and identifies the nature of the software being used as free software or proprietary software. The second and third table lists internet privacy aspects along with other technical parameters, such as whether the engine provides personalization (alternatively viewed as a filter bubble).

Defunct or acquired search engines are not listed here.

Search crawlers

Current search engines with independent crawlers, as of December 2018.

Search engines employ automated bots, known as web crawlers or spiders, to discover and index content across the internet. The scope and frequency of these crawls vary significantly depending on the search engine's underlying architecture. Major index-based search engines utilize highly active crawlers (e.g. Googlebot for Google Search and Bingbot for Microsoft Bing) that continuously scan the web to build and maintain comprehensive, independent databases. In contrast, hybrid or privacy-focused search engines often license partner indexes for their primary results and deploy their own specialized crawlers on a more periodic, targeted basis. For instance, DuckDuckGo utilizes DuckDuckBot to periodically fetch specific data to populate features like instant answers, resulting in a significantly lighter server footprint than continuous global crawlers.[1]

Search engine Crawler name Founder(s) Company Launched Software distribution license Pages indexed Daily direct queries Results count Advertisements
Ahmia Juha Nurmi Juha Nurmi 2014 Proprietary
AOL William von Meister 1999 Proprietary
Baidu Baiduspider Robin Li Yanhong Baidu, Inc 2000 Proprietary
Brave Search Bravebot Brendan Eich and Brian Bondy Brave Software, Inc. 2021 Proprietary Unknown Unknown No Yes
CốcCốc Search coccocbot Nguyễn Thanh Bình, Lê Văn Thanh, and Nguyễn Đức Ngọc Cốc Cốc Ltd. 2013 Proprietary No Yes
DuckDuckGo DuckDuckBot Gabriel Weinberg Duck Duck Go, Inc. 2008 Mixed Unknown[2]
Ecosia Christian Kroll 2009 Proprietary No Yes
Exalead Exabot François Bourdoncle and Patrice Bertin Exalead 2000 Proprietary Unknown Unknown Yes No
Fireball Oli Kai Paulus, Helmut Hoffer von Ankershoffen, Nurhan Yildirim, and Benhui Chen[3] Fireball Labs GmbH 1996 Proprietary
Google Search Googlebot Larry Page, Sergey Brin & Scott Hassan Alphabet Inc. 1997 Proprietary hundreds of billions[4] 9.022 billion[5] Yes Yes
Kagi Vladimir Prelovac[6][7] Kagi Inc.[6] 2018[6] Proprietary No[6]
Kiddle 2014 Proprietary
KidRex 2008 Proprietary
Lycos Lycos_Spider Michael Loren Mauldin 1995 Proprietary
Microsoft Bing Bingbot Microsoft 1998/2009 Proprietary Unknown Unknown Yes Yes
Million Short 2012
Mojeek MojeekBot Marc Smith Mojeek 2004 Proprietary 5 billion[8] Unknown Yes Yes[9]
Naver Yeti Kim Sang-Hun Naver Corp. 1999 Proprietary Unknown Unknown No Yes
Petal PetalBot Huawei 2020 Proprietary Unknown Unknown Yes Yes
Qwant Qwantify Éric Léandri Qwant 2013 Proprietary 20 billion[10] 10 million[citation needed] No Yes
Seznam.cz SeznamBot Ivo Lukačovič 1996 Proprietary
Sogou Sogou Spider Tencent 2010 Proprietary Unknown Unknown Yes Yes
Startpage Surfboard Holding BV 1998
Swisscows Andreas Wiebe Swisscows 2014 Proprietary Unknown Unknown Yes Yes
WebCrawler Brian Pinkerton 1994 Proprietary
Yahoo! Search Yahoo! Slurp Jerry Yang & David Filo Yahoo! 1995 Proprietary Unknown Yes Yes
Yandex Search YandexBot Arkady Volozh Yandex 1997 Proprietary >2 billion[11] Unknown Yes Yes
Youdao YoudaoBot Feng Zhou 2007 Proprietary

Digital rights

Search engine Server's location(s) Dedicated servers Data center Cloud computing HTTPS available Tor gateway available Proxy gateway search links available
Ahmia Yes Yes
AOL Yes No
Baidu China Yes No Unknown
Blackle No No
Brave Search Yes Yes
CốcCốc Search Vietnam Yes No
DuckDuckGo[12] USA No Verizon Internet Services Amazon EC2 Yes Yes
Ecosia USA No Yes No
Exalead No No
Fireball Yes No
Google Search USA Yes Google data centers Yes No Unknown
Kagi Multiple[13] Google Cloud Platform Yes
Kiddle Yes No
KidRex Yes No
Lycos Yes No
Microsoft Bing USA / China Yes Yes No Unknown
Mojeek UK Yes Custodian Data Centres Yes No Unknown
Naver Yes No
Petal France Yes No Unknown
Qwant France Yes Yes Unknown Unknown
Seznam.cz Yes No
Sogou China Yes No Unknown
Startpage Netherlands Yes Yes
Swisscows Yes No
WebCrawler Yes No
Yahoo! Search USA Partial Yes[14] No Unknown
Yandex Search Russia Yes Yes[15] No Unknown
Youdao Yes No

Tracking and surveillance

Search engine HTTP tracking cookies Personalized results[lower-alpha 1][lower-alpha 2] IP address tracking[lower-alpha 3][lower-alpha 2] Information sharing[lower-alpha 2][clarification needed] Warrantless wiretapping of unencrypted backend traffic[lower-alpha 2]
Ahmia No
AOL Yes
Baidu Yes Unknown Unknown Unknown Unknown
Blackle No
Brave Search No
CốcCốc Search Yes
DuckDuckGo[12][16] No No No No [17] No |- Ecosia No[18] No No No Unknown
Exalead No
Fireball Yes
Google Search Yes Default[19] Yes[20] Yes[20] 2013 and prior[20][21]
Kiddle No
KidRex No
Lycos No
Microsoft Bing Yes Yes Yes[20] Yes[20] 2014 and prior[20][22][23]
Mojeek No No No No Unknown
Naver No
Petal Yes Unknown Unknown Unknown Unknown
Qwant No No No No No |- Seznam.cz No
Sogou Yes Unknown Unknown Unknown Unknown
Swisscows No
WebCrawler Yes
Yahoo! Search Yes Unknown Yes[20] Yes[20] 2014 and prior[20][24]
Yandex Search Yes Yes[25] Unknown Limited[26] Unknown
Youdao Yes
  1. ↑ The results of the search are arranged for the user in accordance to their interests as determined from previous search queries or other information available to the search engine.
  2. ↑ 2.0 2.1 2.2 2.3 Cannot be verified independently, as the information is handled by servers not accessible by the public.
  3. ↑ Tracking the user has to be conducted in order to provide personalized search results.

See also

References

  1. ↑ SpyFu Team (2023-01-15). "How Search Engine Crawlers Index Your Website". https://www.spyfu.com/blog/how-search-engine-crawlers-index-your-website/. 
  2. ↑ "Where do DuckDuckGo search results come from?". https://duckduckgo.com/duckduckgo-help-pages/results/sources/. 
  3. ↑ "Fireball (search engine)" (in en), Wikipedia, 2025-02-05, https://en.wikipedia.org/wiki/Fireball_(search_engine)?utm_source=askpandi&utm_medium=answer_engine, retrieved 2025-03-15 
  4. ↑ "How search works, organizing information". https://www.google.com/search/howsearchworks/how-search-works/organizing-information/. 
  5. ↑ "Google Annual Search Statistics". Statistic Brain. 23 April 2017. http://www.statisticbrain.com/Google-searches/. 
  6. ↑ 6.0 6.1 6.2 6.3 "About | Kagi's Docs" (in en-US). https://help.kagi.com/kagi/company/. 
  7. ↑ Koebler ·, Jason (2024-04-03). "Friendship Ended With GOOGLE Now KAGI Is My Best Friend" (in en). https://www.404media.co/friendship-ended-with-google-now-kagi-is-my-best-friend/. 
  8. ↑ "Mojeek Surpasses 5 Billion Pages". https://blog.mojeek.com/2022/03/five-billion-pages.html. 
  9. ↑ "Ads on Mojeek". https://www.mojeek.com/support/ads/. 
  10. ↑ Qwant (2018-11-20). "Web indexation: where does Qwant's independence stand?". https://medium.com/qwant-blog/web-indexation-where-does-qwants-independence-stand-8eab4f7856f8. 
  11. ↑ "SEC Filing 2011". Form 20-F. Our search index includes billions of webpages..: Yandex N.V.. 31 December 2011. p. 45. http://company.yandex.com/i/filings/YandexNV_12312011_20F.pdf. 
  12. ↑ 12.0 12.1 Holwerda, Thom (June 21, 2011), "DuckDuckGo: The Privacy-centric Alternative to Google", OSNews, http://www.osnews.com/story/24867/DuckDuckGo_The_Privacy-centric_Alternative_to_Google, retrieved March 30, 2012 
  13. ↑ "Search Speed | Kagi's Docs" (in en-US). https://help.kagi.com/kagi/search-details/search-speed.html. 
  14. ↑ Danny Sullivan (22 January 2014). "Yahoo Search Goes Secure". Search Engine Land. http://searchengineland.com/yahoo-search-goes-secure-182343. 
  15. ↑ "Yandex.Direct switches to HTTPS". Yandex. https://help.yandex.com/direct-news/n-2012-07-17.xml. 
  16. ↑ "DuckDuckGo Privacy". 2012-04-11. https://duckduckgo.com/privacy. 
  17. ↑ Weinberg, Gabriel (2010-08-10). "DuckDuckGo now operates a Tor exit enclave". http://www.gabrielweinberg.com/blog/2010/08/duckduckgo-now-operates-a-tor-exit-enclave.html. 
  18. ↑ "Learn more about our privacy policy and the data that we do collect". https://info.ecosia.org/privacy. 
  19. ↑ "Turn off search history personalization". https://support.google.com/accounts/answer/54048?hl=en. 
  20. ↑ 20.0 20.1 20.2 20.3 20.4 20.5 20.6 20.7 20.8 Johnson, Kevin; Martin, Scott; O'Donnell, Jayne; Winter, Michael (June 15, 2013). "Reports: NSA Siphons Data from 9 Major Net Firms". USA Today. https://www.usatoday.com/story/news/2013/06/06/nsa-surveillance-internet-companies/2398345/. 
  21. ↑ Gallagher, Sean (2013-11-06). "Googlers say "F*** you" to NSA, company encrypts internal network". Ars Technica. https://arstechnica.com/information-technology/2013/11/googlers-say-f-you-to-nsa-company-encrypts-internal-network/. 
  22. ↑ Danny Yadron (2013-12-05). "Microsoft Compares NSA to 'Advanced Persistent Threat' - Digits - WSJ". Blogs.wsj.com. https://blogs.wsj.com/digits/2013/12/05/microsoft-compares-nsa-to-advanced-persistent-threat/. 
  23. ↑ Tom Warren (2013-12-05). "Microsoft labels US government a 'persistent threat' in plan to cut off NSA spying". The Verge. https://www.theverge.com/2013/12/5/5177554/microsoft-plans-server-encryption-against-nsa-snooping. 
  24. ↑ Brandom, Russell (2013-11-18). "Yahoo plans to encrypt all internal data by early 2014 to keep the NSA out". The Verge. https://www.theverge.com/2013/11/18/5118150/yahoo-ceo-marissa-mayer-plans-to-encrypt-data-against-nsa-by-2014. 
  25. ↑ "Компания Яндекс — Персональный поиск". http://company.yandex.ru/technologies/personalised_search/index.xml. 
  26. ↑ "Privacy Policy – Legal Documents". Yandex.Company. 3.3.1.: LLC Yandex. 15 November 2011. http://legal.yandex.com/confidential/. 



Retrieved from "https://handwiki.org/wiki/index.php?title=Software:Comparison_of_search_engines&oldid=4399171"