Turning a TikTok data export into a permanent, searchable library
Solo Build · Personal Project · Self-Hosted · Docker · Aug 2024 - Present
Favorites on TikTok aren't really yours: the posts live on someone else's servers and vanish when a video is taken down or an account goes private. Over two years I built a self-hosted archiver that turns the official TikTok data export into a permanent local library, with every favorite downloaded, indexed, and searchable down to the words spoken inside the videos. I shipped v1.0.0 in August 2026, MIT licensed, and I use it daily on my own 11,000+ item archive.
- Photo slideshows rebuilt into MP4s with their original audio.
- Speech and on-screen text search with jump-to-timestamp results.
- Dead posts become position-stable placeholders; reruns never renumber.
- Survived an adversarial multi-round security audit.
- Multi-arch Docker image on GHCR; one compose command to deploy.
- Plex, Jellyfin, and Kodi sidecar export for the whole library.
The problem
I'd favorited over 11,000 videos, and they kept disappearing: deleted posts, private accounts, dead links. A saved list is just a bookmark to content someone else controls. I wanted a real copy on my own disk that nothing could take down.
An archive that size is its own engineering problem. The export data is messy, downloads fail mid-run, photo slideshows aren't even videos, and once everything is on disk you still have to find one clip out of 11,000. Each of those became a subsystem: a resumable download engine, a slideshow renderer, local speech and text search, and an interface fast enough to scroll the whole library.
What I built
- 1
A download engine that expects failure
Every favorite is fetched through a self-hosted Cobalt instance by a rate-limited worker pool with 429 backoff. Writes are atomic (.part, then rename), runs are resumable, and dead posts become position-stable placeholder markers instead of errors, so a rerun never renumbers or overwrites what's already archived.
- 2
Rebuilding photo slideshows into real videos
TikTok photo slideshows arrive as loose images plus an audio track. An ffmpeg/MoviePy pipeline reassembles them into MP4s with their original audio, so they play like any other video in the archive. No comparable tool does this.
- 3
Search inside the videos
Bundled whisper.cpp speech transcription and Tesseract OCR run fully local, in-container, indexing what's said and what's on screen. I can search 'parmesan' and jump to the exact second someone says it. whisper.cpp compiles from source in a multi-stage Dockerfile with SHA-256-pinned model downloads, adds 142 MiB to the image, and analysis is pausable and resumable.
- 4
An interface that keeps up with 11,000 items
A TikTok-style vertical feed and a virtualized, searchable gallery stay responsive across the whole archive. Fifteen UI tabs over roughly 90 API routes cover stats dashboards, creator and hashtag discovery, 'on this day' memories, private curation with stars, tags, and notes, duplicate detection, and similarity search built on a stdlib TF-IDF embedding.
- 5
A security model for a no-auth local app
A Host-header allowlist guards against DNS rebinding, a custom-header CSRF defense forces a CORS preflight on state-changing requests, and media serving walks directories with O_NOFOLLOW file descriptors to block path traversal. The only outbound calls are opt-in and off by default: Shazam song ID and Spotify playlist push. Zero telemetry.
- 6
Shipped like a product
v1.0.0 is a multi-arch Docker image (amd64 and arm64) published to GHCR through Actions, deployed with a single compose command, with install templates for Unraid, CasaOS, and Umbrel. Behind it sit 57 backend test files and 23 frontend behavior suites, all green in CI on every push, and a synthetic demo-data seeder that drives real app code paths for the screenshots and demo GIF.