varelaz's Account

varelaz offline 5 years

Responsible Web Scraping: Gathering Data Ethically and Legally

0

I can suggest https://pypi.org/project/reppy/ for robots.txt parsing, it's way better than built in tool. Very good intension, good luck with sharing it.

Sign in

Forgot password?

Or use a social network account

 

By Signing In \ Signing Up, you agree to our privacy policy

Password recovery

You can also try to

Or use a social network account

 

By Signing In \ Signing Up, you agree to our privacy policy