Introducing SCM: AI‑powered search across all photos and video frames on macOS

What SCM Offers

SCM, short for Screen Memories, is a macOS application that lets users search every image and each frame of video using artificial intelligence.

The tool operates entirely on the local machine, requiring no user accounts, cloud services, or data uploads.

Search queries can be expressed in natural language, and a built‑in vision model interprets the description to locate matching media.

Video files are broken into individual scenes, enabling the user to jump directly to the exact moment that matches the query.

Optical character recognition (OCR) extracts visible text from images and video frames, supporting literal text matches in over 35 languages.

Speech transcription powered by OpenAI’s Whisper model allows exact spoken‑line searches within video audio.

Users may also enable optional large‑language‑model (LLM) assistance for more nuanced dialogue queries.

Search results appear as tiles that indicate why each item matched, whether through visual similarity, filename, OCR text, or dialogue.

Each tile includes a score breakdown tooltip, helping users understand the relevance of the match.

CJK (Chinese, Japanese, Korean) queries are handled as overlapping bigrams, so phrases like “台北車站” also match “台北” and “車站”.

How It Works Locally

All inference runs on the Mac, meaning the media never leaves the device and privacy is preserved.

The first time a model is used, its weights—about 435 MB for the default CLIP model—are downloaded once, after which the application works offline.

SCM watches designated folders, automatically imports new files, deduplicates content using hashes, and re‑embeds items when the model switches, all without blocking searches.

Search scoring combines cosine similarity of image embeddings with gated phrase and filename boosts, an honesty floor calibrated per model, and a near‑duplicate diversity filter.

When a video scene is matched, the tile shows a poster image with a timecode badge, and opening the video jumps straight to that moment.

A noise gate suppresses irrelevant scene matches, returning “no scene match” instead of flooding the user with results.

The application is built with Electron and React, packaged using electron‑builder, and runs on macOS 12 or later, including Apple Silicon.

Development uses Bun as both package manager and runtime, simplifying installation of Node modules.

Installing and Using the Tool

The simplest installation method is via Homebrew, where a dedicated tap handles the cask and clears macOS quarantine flags automatically.

Typical commands are “brew tap allenv0/scm”, “brew install --cask allenv0/scm/scm”, and “brew upgrade --cask allenv0/scm/scm” for updates.

Advanced users can trust only the cask instead of the entire tap with “brew trust --cask allenv0/scm/scm”.

Developers can run “bun install” to fetch dependencies, then “bun run dev” to build the renderer bundle and launch the Electron app.

For production builds, “bun run dist” creates signed installers, while “bun run dist:unsigned” skips code‑sign discovery.

The build process first compiles the React renderer with Vite, then packages the app with electron‑builder, placing the DMG and ZIP files in the “dist‑app” directory.

Once installed, users can save any query as a tab, toggle between screenshot and email tabs, and rely on automatic background re‑embedding when models are updated.

Overall, SCM provides a privacy‑first, AI‑enhanced way to locate visual memories without relying on external services.

Why This Matters

The most important implication is that users can now perform powerful, AI‑driven media searches while keeping all personal content strictly on their own Mac, eliminating privacy risks associated with cloud‑based indexing.