Pluto Photos includes an AI-powered contextual search engine that understands what's in your photos. Instead of searching by filename or tag, just describe what you're looking for in natural language.

How It Works

Automatic scanning — When you import photos, Pluto Photos runs two AI models locally on your machine:

SmolVLM2 generates a natural language caption for each photo (e.g., "a group of people sitting around a table with a birthday cake")
CLIP ViT-B/32 generates a semantic embedding vector that captures the visual meaning of the photo

Search by description — When you search, your query is converted to a CLIP embedding and compared against all photo embeddings using cosine similarity
Ranked results — Photos are ranked by how well they match your description, with the best matches shown first

What You Can Search For

Objects — "car", "bicycle", "laptop", "pizza"
Scenes — "beach at sunset", "snowy mountain", "city street at night"
Activities — "people playing soccer", "cooking in kitchen", "hiking trail"
Animals — "golden retriever", "cat sleeping", "birds in a tree"
Food — "sushi", "birthday cake", "coffee shop"
Abstract concepts — "happiness", "loneliness", "celebration"

Smart Search Features

Synonym expansion — Searching "kid" also matches "child", "toddler", "infant"
Stem matching — "swimming" matches "swim", "swimmer", "swims"
CLIP + text fallback — If the AI model can't load, search falls back to intelligent text matching against captions
Customizable synonyms — Edit the synonym dictionary to add domain-specific terms

100% Local Processing

This is what sets Pluto Photos apart. Google Photos, Apple Photos, and Amazon Photos all send your images to remote servers for AI analysis. Pluto Photos runs both models on your own machine — they are fetched once when you install and never leave your disk:

No internet connection required — works on an airplane, in a cabin, anywhere
No data ever sent to third parties — your photos are never uploaded for analysis
No API keys or accounts — the AI runs locally, no external services
Works in Docker — self-hosted server deployments get the same AI capabilities

Technical Details

Caption model: SmolVLM2-500M-Video-Instruct — a real vision-language model (SigLIP vision encoder + SmolLM2 decoder), quantized to ONNX
Search model: CLIP ViT-B/32 (512-dimensional embeddings)
Tokenizer: BPE tokenizer bundled with the app
Performance: measured at ~3.6 seconds per photo on a 6-core desktop CPU, for the caption and the search embedding together (no GPU required — the sessions run on CPU by design)
Model sizes: ~640 MB total — ~150 MB of CLIP and ~490 MB of SmolVLM2, fetched once when you install and cached on disk
Caption quality: the caption is CLIP-reranked against a tag-derived candidate and the better match wins, which catches the cases where the model invents a detail

Docker / Self-Hosted

AI contextual search works in Docker deployments too. The server automatically scans new photos in the background and builds the search index. Search via the Companion Browser just works — no extra configuration needed.

Part of Pluto Photos Pro

AI Contextual Search is available with a Pro license. Captions and search embeddings are stored in your local database and persist across restarts.

Search Your Photos by Describing Them.