A little like that j-thing, only in Go.
Go 10.6k 796
Polite, slim and concurrent web crawler.
Go 1.9k 193
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
Go 758 98
tiny Go library to normalize URLs
Go 403 58
Package rehttp implements a Go HTTP transport that handles retries.
Go 175 12
Proper URL escaping as per RFC3986
Go 23 14
Loading…
