Link Checker will spider website and check all anchor links.
The script recognizes different URLs that refer to the same location:
- http://www.domain.com/path/index.html
- http://www.domain.com/path/
- http://www.domain.com/path
- http://domain.com/path/index.html
and checks them only once
(it will optionally reduce such URLs to the same form by
auto-appending or stripping www. prefix from domain name
and auto-appending or stripping default filename index.* from URL).
