About SiteScope
SiteScope is a free technical SEO audit tool for small startups and student websites.
Why it exists
Paid crawlers are too expensive for many small teams, and most free tools check one page at a time. SiteScope audits a whole site and ranks the issues, so a developer knows what to fix first.
Who built it
SiteScope was built by Vikram Yadav (B.Tech CSE, Bennett University) as the CSET489 mini-project on search engine optimisation.
How it is built
Python and FastAPI with an asynchronous httpx crawler, BeautifulSoup for parsing, NetworkX for the internal link graph, SimHash for duplicate detection, Google PageSpeed Insights for Core Web Vitals and PostgreSQL for stored reports. It runs in Docker behind Nginx and Cloudflare.
The crawler
The SiteScope crawler identifies itself as SiteScopeBot/1.0, follows robots.txt, sends a few requests at a time with a delay between them, and only crawls the site you ask it to audit.