Cloaking: This is the technique where web sites intercept each request, recognize the ones from search engine bots and serve up different content to the bots rather than serving up the same pages seen by the end user. It is obvious how this can be misused; you can load the web pages you serve up to the bots with misinformation to get a higher ranking. An action such as this can very well result in the search engines blocking your web site.
Link Farms: Being a part of link farms, where web masters agree to put links to each other’s sites on their pages to increase their ranking, is also a quick way to get blacklisted.
Monitor and Control bot Traffic: Modern search crawlers constantly update their caches, and hence, trawl through the World Wide Web at amazing regularity to detect changes. There is a good chance that these requests from bots might overwhelm the web site, and controlling this is of high importance. One solution is to have a mirror website running on the site dedicated to serving bots only. A load balancer can then be configured to redirect all requests from spider bots to that particular web server.
- The Ultimate Guide to Ecommerce Infographic
- 4 Tips to Prepare for What’s next in Tech
- Will M-commerce Overtake E-Commerce?
- The Payment Trends of 2014
- 2014 Digital Stats
- Digital Velocity 2014
- Email and Mobile Top Small Businesses’ 2014 Wishlists
- The Ecommerce Revolution in the Middle-East
- The Hottest Ecommerce Businesses to Start in 2014
- Holiday Survival Guide For Ecommerce