https://blog.sksoumik.com/tags/web-scraping/ Recent content in web scraping on Sadman Kabir Soumik Hugo -- gohugo.io en Copyright © 2022, Sadman Kabir Soumik Wed, 09 Mar 2022 00:00:00 +0000 https://blog.sksoumik.com/software-engineering/rotate-ip-with-every-http-request-bypass-recaptcha-tor-proxy/ Wed, 09 Mar 2022 00:00:00 +0000 https://blog.sksoumik.com/software-engineering/rotate-ip-with-every-http-request-bypass-recaptcha-tor-proxy/ When we run web crawlers, sometimes we get blocked by the target site. Sometimes we get reCAPTCHA to solve, and crawling gets interrupted. We can rotate the IP address with each request to avoid these issues, which solves the IP blocking and reCAPTCHA issues. This blog is the written version of the video content I published on YouTube. If you prefer watching videos than reading blogs, then you can watch the video.