linkchecker for html pages with recursive checking, multithreaded, output can be colored or normal text, HTML, SQL, CSV or a sitemap graph in XML or GML format, additionally reports download time for HTML pages HTTP/1.1 and 1.0, HTTPS, FTP, mailto:, news:, nntp:, gopher, telnet and local file links are supported, restrict link checking with regular expression filters for URLs, proxy support, give username/password for HTTP and FTP authorization, robots.txt exclusion protocol support, i18n support, CGI web interface. The gui portion is no longer included as it is not maintained.

WWW: https://pypi.python.org/pypi/LinkChecker/
