Are you scraping Google results for client rankings? Be careful!
 by Casey Markee

Are you scraping Google results for client rankings? Be careful!

  • For the last couple of months we have been using a self-written tool to scrape the Google database for client ranking results. We have been running the tool a couple of times a week but after a few hundred queries we get blocked. Can you provide some tips on how to get around this or maybe an alternative we can use?
Answer: If you are using a "few 100 queries" then you are probably using the tool too much. Google is clear that they DO NOT like automated querying of their database and have said as much in their Google Webmaster Guidelines:
"Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries absorbs resources and includes using any software (such as WebPosition Gold) to send automated queries to Google to determine how a website or webpage ranks in Google search results for various queries."

That being said this is going on and has been going on well before Google entered the scene. Nevertheless, those that do this (and do this effectively) run the queries across hundreds of different IP addresses. So if you can't do this at your end, then you need to consider using a tool or service to do it for you, and safely.

For example, our seoINsites.com Rank Monitor runs across dozens of different IP addresses. This guarantees no IP has an abnormal strain of automated queries to tip-off Google that the resulting queries are anything but natural. If you aren't a member of seoINsites.com, this is a great reason to sign-up.

Here's a VIP link to seoINsites.com that will allow to "kick the tires" and try all our grea...

Already a member? Sign in here

Read the rest of this article,
and get all this for only $1.

  • The Search Engine Strategies Updates for February 2017
  • Ultimate Guide to Avoiding Google Penalties
  • The Complete Site Audit Checklist
  • The Definitive Local Search Audit Checklist
  • 100's of Strategic SEO Articles and Q&As
  • The Professional Engine Master's Chart
  • The Internet Marketing Glossary
  • The Ultimate Directory Submission List
  • The Pro SEO's Local Search Directory List
  • 16 years of SEN Archives
  • PLUS, as a Full SEN Basic Member, you'll be eligible for hundreds of dollars of discounts on SEO courses ranging from beginner to master on a variety of topics including organic search, local search, and social networking.

  • Your $1 Trial is good for 7 days at which time your card will be charged $29/mo unless you cancel before March 3rd, 2017

 This form is encrypted for your security