Palimpsest

About this project
The idea behind Palimpsest is simple: a website is never just the page we happen to see today. Earlier versions can tell us what an organization communicated in the past, what information disappeared, and how the way something was presented changed over time. I wanted to build a tool that made those changes easier to explore.
Palimpsest retrieves a site's archived history through the Internet Archive's Wayback Machine and arranges meaningful changes along an interactive timeline. Consecutive snapshots with identical content can be collapsed, leaving a more useful record of when something actually changed.
One of the more interesting challenges was deciding what a meaningful change looks like. A raw text diff can be overwhelmed by formatting and structural differences, so the backend parses pages into DOM trees and compares elements such as headings, paragraphs, links, images, and lists. Archived pages are rendered in sandboxed iframes with scripts removed, and SQLite keeps track of cached snapshots and research notes. An optional Claude-powered Insight panel can also use web search to investigate the wider context around a change.
I'm particularly interested in what this approach could make possible for researching websites that provide services and information to newcomers and immigrant communities. Looking at archived versions of these sites could help us ask more concrete questions about how available information, service descriptions, and language coverage have changed. For me, Palimpsest is both a software project and a way of exploring how we can use the web's past to better understand the present.