Credits
IPv6Matrix gives open access to all analysed crawler data via a RESTful API.
The API has been designed to be easy to understand and to return information with an appropriate level of detail, in a variety of common data formats.
Mythic Beasts
The servers for this service (both the webserver and crawler) are kindly provided by Mythic Beasts
Silvermouse
In 2021 James Lawrie from Silvermouse extracted the webserver, crawler, and history from two disk images provided by ISOC.
The crawler was written in Python2 and the webserver in NodeJS 0.10. A subcontractor (Robert Sandiford) updated the webserver to use a modern version of NodeJS.
James rewrote the Python application in Python3, and reconfigured it to store the data in SQLite as the crawl proceeded.
This allowed the crawler to only check each host once (many domains share MX, or CDN A/AAAA records), and also to resume in case of a crash.
The new version was released in December 2021.