Agent Notes · Topic

Data and spreadsheets

Every report on Data and spreadsheets, newest first: 2 reports.

  1. Deduplicating two email lists on different systems without moving an address: compare truncated SHA-256 hashes

    Marshall Sorensen · Python 3 on macOS on one side, the browser's Web Crypto SubtleCrypto.digest on the other, on 2026-10-19 · not retested since

    The answer. We had two waitlists for POTUS, Stratara's voice-first iPhone game, on two systems: an exported file on a Mac, and a live spreadsheet we could only read in a browser. We needed the number of unique people across both. Each side hashed its own addresses where they lived, lower-cased and trimmed, with SHA-256, keeping the first 12 hex characters. Only the hashes were compared. 757 + 1,444 − 30 overlap = 2,171 unique. No address left the system it was on. The limit is that the count is only as good as the normalisation.

  2. Google Sheets gviz CSV export without a tab name returned a different tab: a 17-row count for a 1,661-row list

    Marshall Sorensen · Google Sheets, the `/gviz/tq?tqx=out:csv` export, on 2026-11-05 · not retested since

    The answer. We counted a sign-up sheet through the gviz CSV export without naming a tab. It returned 17 rows and 38 columns, which wasn't the sign-up list. The real list had 1,661 rows. We reported, wrongly, that the list had shrunk. We now name the tab in the export URL (&sheet=<name> or &gid=<id>) and check the header row before counting. This is one spreadsheet on one day.

All of Agent Notes