All About School - The Complete Education Forum and Classifieds

Members Login
Username 
 
Password 
    Remember Me  
 
All About School - Education Forum and Classifieds -> Education and Academics -> How to stop target sites from throwing CAPTCHAs during data collection?
Post Info TOPIC: How to stop target sites from throwing CAPTCHAs during data collection?
poretpo


Veteran Member

Status: Offline
Posts: 91
Date: September 14th
How to stop target sites from throwing CAPTCHAs during data collection?
Permalink   
 

Having your scripts get stopped every five minutes completely ruins the point of automating checks, especially when you have hundreds of product links to go through every morning.

__________________
likenerty


Senior Member

Status: Offline
Posts: 114
Date: September 14th
Permalink   
 

When websites see too many requests coming from the same server, they immediately flag your IP as a bot. Routing your requests through real household connections like https://www.simplynode.io/product/residential-proxies completely solves this. To the store’s server, it just looks like regular home internet users from different cities browsing products, so the requests go right through without triggering security checks or temporary bans.



__________________
poretpo


Veteran Member

Status: Offline
Posts: 91
Date: September 14th
Permalink   
 

 

Our team is trying to pull prices from a few online stores, but after hitting twenty pages, the site hits us with bot verification screens and blocks everything. What's the best way to keep requests going through smoothly?



__________________
Page 1 of 1  sorted by
 Add/remove tags to this thread
Quick Reply

Please log in to post quick replies.
All About School - Education Forum and Classifieds -> Education and Academics -> How to stop target sites from throwing CAPTCHAs during data collection?
Tweet this page Post to Digg Post to Del.icio.us


Create your own FREE Forum
Report Abuse 		Powered by ActiveBoard