0
requests
0
unique resources
Coverage unavailable
Live Crawler Behaviour Logs
Freshness observatoryExplore requests to Scrubnet’s public feeds by path, crawler, response, format and date. The database and this view refresh every 10 minutes.
Interpretation: a request confirms feed access only; it does not prove indexing, ranking, model use or citation.
Dataset version loading…
Quick views:
| Time | Last modified | Content age | Method | Status | Bot | Verification | Format | Path | File size | Bytes sent | User Agent |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Loading data... | |||||||||||
Page 1 of 1
Requests by crawler over time
Top 10 crawlers are shown individually; any remaining crawlers are grouped as Other. Click a name to hide its line.
Loading chart...
Response outcomes over time
Content responses, revalidation, redirects and errors are shown separately.
Loading chart...
Requests by crawler, response, status, verification and method
| Crawler | Requests |
|---|---|
| Loading... | |
| Status | Requests |
|---|---|
| Loading... | |
| Verification | Requests |
|---|---|
| Loading... | |
| Method | Requests |
|---|---|
| Loading... | |
| Response outcome | Requests |
|---|---|
| Loading... | |
Requests by format
0
requests
0
unique resources
Coverage unavailable
0
requests
0
unique resources
Coverage unavailable
0
requests
0
unique resources
Coverage unavailable
Formats over time
Current filters apply. The chart includes HTML, TXT, JSON and Markdown feed requests; other paths are excluded. Click a format name to show or hide its line.
Loading chart...
Rolling 24 hours (UTC)
Path, crawler, method, format, response and verification filters apply; the selected date range does not.
Loading chart...
How to interpret this dataset
The default view includes verified crawler requests to Scrubnet feeds. Verification uses published provider IP ranges or forward-confirmed reverse DNS where available. “Content age” is the time between a resource’s recorded modification date and the request; unknown or future dates are excluded from age statistics and reported through coverage counters.
Request counts show access to Scrubnet, not indexing, ranking, training, retrieval or citation. Format totals are descriptive. Coverage denominators use the latest daily inventory snapshot and are unavailable when a path filter is active.