SK Memory is the per-project search and indexing system in ThinkForge. It reads text-based files from the project folder, breaks them into searchable chunks, generates vector embeddings, and stores everything in a project-local database. This index powers document search, semantic similarity discovery, smart links, tag management, the Forge Graph visualization, and the MCP server's search tools.
SK Memory runs entirely locally. No content leaves the machine. Embeddings are generated by a model bundled with ThinkForge, and the database lives inside the project's .fire/ directory.
What Gets Indexed
SK Memory indexes text-based files with these extensions: .md, .txt, .rtf, .json, .html, .css, .js, .cs, .py, and .xml. Binary files — PDFs, images, video — are browsable in the folder tree but are not indexed for search.
The indexer scans up to five directory levels deep. Directories starting with . (including .fire itself), plus bin, obj, node_modules, and .git, are skipped.
Unity .meta sidecar files are excluded even though their contents are text. They stay on disk, because Unity needs them, but they never appear as SK Memory content in discovery, indexing, reads, or search.
Beyond disk files, SK Memory also stores content that has no file of its own:
- Conversation pairs from ThinkForge Chat
- Mind map content from nodes
- CSV data from table files
- Editor saves from the rich text or code editor
- MCP artifacts saved by AI clients through the MCP server
The Indexing Process
Indexing happens at several points.
Project open — When a project loads, ThinkForge initializes its index, starts watching the folder, and queues reconciliation in the background. Project tabs and Project Navigator can use the existing index immediately. Reconciliation compares files with the saved index, adds or updates known content, repairs a stored absolute path when the same project-relative path still owns the record, and restores missing chunks. It does not infer moves from matching content or delete a record merely because its file is absent.
If a complete scan cannot safely explain one or more missing records, reconciliation retains them and finishes in a degraded state. It also defers unmatched new files that might be offline renames. The status reports retained and deferred counts so you can resolve the file operation explicitly without losing document identity.
File saves — Saving a document writes the file and routes supported text through the index. Existing records are found by path and keep their identity.
File system watcher — ThinkForge watches the project folder for created, changed, renamed, and deleted files. Changes are queued with a short delay to prevent re-index storms during rapid saves. If the watcher reports that events may have been missed, a full reconciliation runs automatically.
Explicit indexing — Files can be indexed on demand through Chrome extension uploads, drag-and-drop, MCP artifact saves, and the MCP indexing tool. Selecting an unindexed text file in the active project's folder tree also indexes it before showing the document preview.
Reconciliation status counts
The SK Memory dashboard reports the index state as of its most recent refresh, not the number of files found by an in-progress reconciliation. On a new or reset index, the total can therefore climb through partial values while background work registers, chunks, and embeds the project files.
A manual Reconcile Project request made during startup joins the reconciliation already running rather than starting a second scan.
The displayed count can briefly lag the final state. Use the status and progress indicator to tell whether repair is still running. A failed operation produces an error result; an identity-ambiguous scan produces a successful but degraded result. If reconciliation has finished but the count still looks low, Refresh reloads the current statistics without re-running reconciliation.
Audit Duplicate Claims reports groups that share content and groups where more than one identity claims the same path. It never deletes or merges records.
Change detection
SK Memory stores a hash of each indexed document's content.
Reconciliation first compares file size and last-write time against what it recorded. Files whose metadata still matches are not opened or hashed at all. When metadata changed, SK Memory reads the content and compares hashes. Identical content updates the stored metadata without regenerating chunks.
Content hashes are change-detection data only. They do not determine identity and are not used to pair a missing path with a new one.
Chunking
Indexed content is split into searchable chunks. The strategy depends on the file type:
Markdown — Split on heading boundaries, falling back to paragraph breaks.
Code files (.cs, .js, .py, .java) — Split on class, method, and region boundaries, falling back to 50-line chunks.
Plain text — Split on paragraph breaks.
Everything else — Chunks of roughly 1,000 tokens with 100 tokens of overlap between them.
Each chunk stores its content, position, offsets, and token count.
Database
Each project's index is a single file at:
{projectRoot}/.fire/memory/semantic-memory.db
It holds documents with their full content and metadata, chunks, embeddings, tags at both document and chunk level, and named collections.
Because it lives inside the project folder, the index travels with a copied or backed-up project.
Document Model
Each indexed document is stored with:
- A permanent ID assigned when the record is first created, which never changes
- Absolute and relative file paths, which locate the content but do not determine identity
- Full extracted plain text content
- File type, MIME type, and file size
- Creation and modification timestamps
- A content hash for change detection
- Tags, both yours and system-generated
- Indexing state and timestamps
- Extended metadata such as a display title
Renames, folder moves, and reconciliation all update location fields while preserving the ID. This is why document links keep working after you rename or move a file.
Content-only records such as conversation memory and tool references use the same identity scheme but have no file path, so reconciliation does not treat them as missing files.
Project Scoping
SK Memory works on one project at a time by default, using the active project's index. Cross-project operations — MCP's search-all-projects, or reaching into an external project — scope to a specific project for that request without disturbing anything else in progress.
Relationship to Other Systems
SK Memory is the data foundation for several features:
- Search in the folder tree, Files, and MCP — every search mode reads from it
- Smart links in the Document Info Panel — semantic similarity between documents
- Forge Graph in Cluster View — embeddings drive spatial clustering
- Tags — stored alongside document records
- Overview — tag counts and document metadata
- Agent context — agent tools can search for relevant documents
- Chrome extension — imported content is indexed here
- MCP server — all MCP search and retrieval tools query it
See Search and Retrieval for how search works, and Embeddings for how vectors are generated.