checker

overview

A broken link checker that crawls any local or live website and reports dead links, redirects, and errors. Works on localhost before you push and in CI/CD pipelines to block deploys with broken links.

challenge

The goal was to learn how web crawlers and link checkers work.

approach

I built a BFS crawler with concurrent HEAD/GET requests, robots.txt compliance, and binary asset detection, wrapped in a terminal interface with live output and JSON reporting.

outcome

The result is a fast, scriptable broken link checker that works on localhost before you push or as a CI/CD step to block deploys with dead links.