Are you scraping Google results for client rankings? Be careful!
 by Casey Markee

Are you scraping Google results for client rankings? Be careful!

  • For the last couple of months we have been using a self-written tool to scrape the Google database for client ranking results. We have been running the tool a couple of times a week but after a few hundred queries we get blocked. Can you provide some tips on how to get around this or maybe an alternative we can use?
Answer: If you are using a "few 100 queries" then you are probably using the tool too much. Google is clear that they DO NOT like automated querying of their database and have said as much in their Google Webmaster Guidelines:
"Google's Terms of Service do not allow the sending of automated queries of any sort to our system without express permission in advance from Google. Sending automated queries absorbs resources and includes using any software (such as WebPosition Gold) to send automated queries to Google to determine how a website or webpage ranks in Google search results for various queries."

That being said this is going on and has been going on well before Google entered the scene. Nevertheless, those that do this (and do this effectively) run the queries across hundreds of different IP addresses. So if you can't do this at your end, then you need to consider using a tool or service to do it for you, and safely.

For example, our Rank Monitor runs across dozens of different IP addresses. This guarantees no IP has an abnormal strain of automated queries to tip-off Google that the resulting queries are anything but natural. If you aren't a member of, this is a great reason to sign-up.

Here's a VIP link to that will allow to "kick the tires" and try all our grea...

Already a member? Sign in here

Read the rest of this article,
and get all this for only $1.

  • The Search Engine Strategies Updates for March 2021
  • SEN's Monthly Marketing Podcast for Strategies on the Go
  • Ultimate Guide to Avoiding Google Penalties
  • The Complete Site Audit Checklist
  • The Definitive Local Search Audit Checklist
  • 100's of Strategic SEO Articles and Q&As
  • The Internet Marketing Glossary
  • The Pro SEO's Local Search Directory List
  • 2019 Unfair Advantage SEO Beginner's Book
  • Pro's Ultimate WordPress Plugin Guide
  • The Vault: SEN's Internet Marketing Research Libraries
  • PLUS, as a Full SEN Basic Member, you'll be eligible for hundreds of dollars of discounts on SEO courses ranging from beginner to master on a variety of topics including organic search, local search, and social networking.

  • Your $1 Trial is good for 7 days at which time your card will be charged $29/mo unless you cancel before March 13th, 2021

 This form is encrypted for your security

Related Articles & Guides