HTTrack
https://www.httrack.com/html/fcguide.html

advanced website cloner


Switches

-O output folder
-c4 number of multiple connections
-%c10 maximum number of connections per second
-A2000 maximum transfer rate in bytes/seconds
-m2000,5000 maximum file length for a non-html file, for html file (50000000 = 50mb)
-M10000 maximum mirror size which can be downloaded (1000000000 = 10Gb)
-r25 mirror depth
-%e1 external links depth
-H2 host is abandonned if: 0=never, 1=timeout, 2=slow, 3=timeout or slow
-i continue an interrupted mirror using the cache
-P proxy use (-P proxy:port or -P user:pass@proxy:port)
-GN pause transfer if N bytes reached, and wait until lock file is deleted (may be used with cron)
-RN number of retries, in case of timeout or non-fatal errors (*R1)
-n get non-html files near an html file
-t test all URLs (even forbidden ones)
-F user-agent field sent in HTTP headers (-F "user-agent name")


Examples

Before mirroring check target site MIME types and create mappings in .httrackrc or add --assume switch

--assume asp=text/html,php3=text/html,cgi=image/gif,zip=application/x-zip,mpg=application/x-mp3

Mirror given site, mirror all subdomains, output to given directory

httrack "http://www.hackthissite.org/" "+*.hackthissite.org/*" -O "/tmp/www.hackthissite.org"

Mirror given site and skip specific locations

httrack www.hackthissite.org -O /root/Desktop/httrack -c4 -%c10 -A20000000 -m50000000,1000000 -M1000000000 -r25 -R3 -H2 -n -t -www.hackthissite.org/forums/* -www.hackthissite.org/user/*