A little like that j-thing, only in Go.
Go 12k 868
Polite, slim and concurrent web crawler.
Go 2k 194
A simple and flexible web crawler that follows the robots.txt policies and crawl delays.
Go 776 98
tiny Go library to normalize URLs
Go 440 60
Package rehttp implements a Go HTTP transport that handles retries.
Go 192 14
Proper URL escaping as per RFC3986
Go 25 15
Loading…