⏾ Concept Map

Sidecar File ○꠹｜encounter｜ENC｜20260905194155-00-•

KOReader stores highlights in a sidecar folder next to each book — BookName.sdr/metadata.epub.lua (or .pdf.lua, etc.). This is a Lua table you can parse directly; it’s more reliable than trying to search the book’s text for each highlighted phrase, because two problems come up with a naive text-search approach: highlighted phrases can repeat elsewhere in the book, and searching manually across every chapter file is slow.

Here’s the efficient path:

1. Parse the metadata.lua file for the highlight entries
Each highlight has fields like:

  • text — the highlighted string itself
  • pos0 / pos1 — xpointers marking the start/end of the highlight in the book’s DOM
  • chapter — chapter title (sometimes)
  • datetime

You can parse this with a small Lua-table parser in Python (e.g. the `slpp` package, or `lupa` if you want a real Lua runtime) — no need to hand-write a full parser.

2. Use the xpointer, not text search, to locate the paragraph
For EPUBs, KOReader’s xpointer looks like:

/body/DocFragment[14]/body/div/p[3]/text().23
  • DocFragment[14] maps to the 14th file in the EPUB’s spine (from content.opf)
  • The rest is a DOM path into that XHTML file
  • The trailing number is a character offset into the text node

So the procedure is:

  1. Unzip the EPUB, read the spine order from content.opf.
  2. Open the XHTML file corresponding to the DocFragment index.
  3. Parse it with lxml, walk the DOM path in the xpointer to find the exact node.
  4. Walk up from that node to the nearest block-level ancestor (<p>, <div>, etc.) — that’s your “paragraph.”
  5. Extract that ancestor’s full text.

This is exact and avoids ambiguity, since you’re resolving to the literal node KOReader recorded rather than guessing via string match.

3. Batch it
Loop over every highlight entry, resolve pos0’s paragraph, dedupe consecutive highlights that land in the same paragraph, and output book/chapter/paragraph/highlighted-text as a table (CSV, JSON, whatever’s next in your pipeline — sounds like this might feed into your Calibre workflow).

If you want, I can write the actual Python script (parses metadata.lua, resolves xpointers against the EPUB, outputs paragraphs) — just point me at how you’d want the output (CSV/JSON/Markdown) and whether you’ll run it against a synced .sdr folder or the device directly over USB/SSH.

Source: Claude Sonnet ○˒

(echo:: @ ᯤ ᯤ ᯤ ᯤ)