# Scrubnet

Practical tools and technical insight for people who build and optimise the web

[Explore Tools](https://scrubnet.org/tools.html) [Browse Articles](https://scrubnet.org/discoveries.html)

## Better ways to do technical work

Scrubnet is an independent toolkit and knowledge hub for technical SEO specialists and web developers. It brings together focused apps, practical articles and first-hand crawler data to make complex work easier to inspect, organise and understand.

- **Inspect:** investigate technical signals and diagnose website issues.
- **Organise:** turn complex information into useful, portable working knowledge.
- **Learn:** explore technical guidance, experiments and evidence from the live web.

## Featured Scrubnet tools

SEO Scrubbox and ScrubGraph support complementary parts of technical SEO and knowledge work, with a consistent local-first approach.

![SEO Scrubbox browser extension interface](https://scrubnet.org/seo-scrubbox.png)

Chrome extension

### SEO Scrubbox

Compare source and rendered SEO signals, test indexability, inspect bot access and run technical diagnostics directly in Chrome.

Free browser extension

[Explore SEO Scrubbox](https://scrubnet.org/seo-scrubbox.html)

![ScrubGraph visual knowledge tree interface](https://scrubnet.org/scrubgraph.png)

Knowledge app

### ScrubGraph

Build and navigate private visual knowledge trees with Markdown notes, flexible editing, search and portable branch or full-graph exports.

Private, local-first browser app

[Explore ScrubGraph](https://scrubnet.org/scrubgraph.html)

[**See the full tool collection →**](https://scrubnet.org/tools.html)

## Articles for technical practitioners

Scrubnet publishes practical articles about technical SEO, web development, AI search and crawler behaviour. Some explain better ways of working; others turn experiments and observed data into careful, useful analysis.

[**Browse all articles and research →**](https://scrubnet.org/discoveries.html)

## The crawler observatory

The observatory is Scrubnet’s live research layer. It records how recognised search and AI crawlers request authorised machine-readable feeds, providing public data for descriptive analysis of discovery, recrawling, freshness and format choices. Requests show access behaviour, not indexing, ranking or citation.

![Live Crawler Logs dashboard showing crawler request charts](https://scrubnet.org/crawler-log-dashboard.webp)

Live data

### Crawler request logs

Filter recent requests by crawler, path, format, response and date.

[Crawler request logs](https://scrubnet.org/dashboard.php)

![Freshness Observatory dashboard showing crawler request data by content age](https://scrubnet.org/freshness-observatory.webp)

Interactive analysis

### Freshness Observatory

Explore request timing, content age and crawler distribution visually.

[Freshness Observatory](https://scrubnet.org/freshness-lab.php)

Start here

### Observatory overview

Understand the research environment, its methods and how to interpret the data.

[Observatory overview](https://scrubnet.org/observatory.html)

Open infrastructure

### Public crawler feeds

Inspect the machine-readable feeds behind the observation environment.

[Public crawler feeds](https://scrubnet.org/llms.html)

## Practitioner-built and evidence-led

Scrubnet was founded by Charles Vayssié, a technical SEO specialist and web developer who leads strategy and delivery across complex e-commerce and international websites. His work spans migrations, rendering, site architecture, performance, structured data and international SEO, with technical findings translated into practical engineering roadmaps.

- **Independent:** Scrubnet is not tied to a search engine, AI platform or tool vendor.
- **Practical:** tools and guidance are created around real technical workflows.
- **Evidence-led:** observed behaviour is kept separate from hypotheses and marketing claims.
- **Open where possible:** crawler feeds, live logs and findings remain public wherever access and privacy allow.

[**Learn more about Scrubnet →**](https://scrubnet.org/about-us.html)

[![Member of Manchester Digital Badge](https://scrubnet.org/Member-MD-B&W.webp)](https://www.manchesterdigital.com/member/scrubnet-ltd)

## Take part

Add an authorised site to broaden the crawler dataset, suggest a technical topic, or collaborate with Scrubnet on an analysis, case study or useful way of working.

**Observatory participation is free:** add a website with up to 50,000 public URLs. There are no visibility or ranking guarantees.

[Collaborate](https://scrubnet.org/partnerships.html)

Want to contribute to the crawler research? [Add an authorised site](https://scrubnet.org/brands.html).

Or reach out at [contact@scrubnet.org](mailto:contact@scrubnet.org)