[PR #7839] [MERGED] feat: support local ai embedding, local ai search, local document content search #8152

New issue

Closed

opened 2026-03-23 23:22:51 +00:00 by mirror · 0 comments

mirror commented

2026-03-23 23:22:51 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/AppFlowy-IO/AppFlowy/pull/7839
Author: @appflowy
Created: 4/27/2025
Status: ✅ Merged
Merged: 5/2/2025
Merged by: @appflowy

Base: main ← Head: local_embedding

📝 Commits (10+)

6e71dda chore: generate embeddings
f589d45 chore: save embedding
6d79c67 chore: vec sqlite
a641171 chore: clippy
fff9c77 chore: init vector
14d9830 chore: create vector db
6027a99 chore: periodically write embedding
3fbe711 chore: fix compile
f695137 chore: skip write
d6187ba chore: impl search

📊 Changes

139 files changed (+4998 additions, -4744 deletions)

View changed files

📝 .github/workflows/android_ci.yaml.bak (+1 -1)
📝 .github/workflows/flutter_ci.yaml (+1 -1)
📝 .github/workflows/ios_ci.yaml (+1 -1)
📝 .github/workflows/release.yml (+11 -11)
📝 .github/workflows/rust_ci.yaml (+1 -1)
📝 .github/workflows/rust_coverage.yml (+1 -1)
📝 frontend/Makefile.toml (+35 -34)
📝 frontend/appflowy_flutter/lib/workspace/application/command_palette/command_palette_bloc.dart (+1 -3)
📝 frontend/appflowy_flutter/lib/workspace/application/command_palette/search_service.dart (+4 -9)
📝 frontend/appflowy_flutter/macos/Podfile.lock (+23 -23)
📝 frontend/rust-lib/Cargo.lock (+385 -94)
📝 frontend/rust-lib/Cargo.toml (+13 -12)
📝 frontend/rust-lib/build-tool/flowy-ast/src/symbol.rs (+2 -2)
📝 frontend/rust-lib/collab-integrate/Cargo.toml (+4 -2)
📝 frontend/rust-lib/collab-integrate/src/collab_builder.rs (+18 -0)
➕ frontend/rust-lib/collab-integrate/src/instant_indexed_data_provider.rs (+226 -0)
📝 frontend/rust-lib/collab-integrate/src/lib.rs (+1 -1)
➖ frontend/rust-lib/collab-integrate/src/persistence/mod.rs (+0 -1)
📝 frontend/rust-lib/dart-ffi/src/lib.rs (+19 -7)
📝 frontend/rust-lib/event-integration-test/Cargo.toml (+2 -3)

...and 80 more files

📄 Description

• Remove the obsolete Supabase test that’s no longer used.
• Add support for generating and using local embeddings.
• Introduce local AI-powered search functionality.
• Enable full-text search over document content (previously we only indexed files that existed on disk).
• Refactor folder-view search so that each workspace maintains its own Tantivy index directory.
• Reindex a document automatically whenever its content hash changes.
• Add end-to-end integration tests to verify search functionality.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/AppFlowy-IO/AppFlowy/pull/7839 **Author:** [@appflowy](https://github.com/appflowy) **Created:** 4/27/2025 **Status:** ✅ Merged **Merged:** 5/2/2025 **Merged by:** [@appflowy](https://github.com/appflowy) **Base:** `main` ← **Head:** `local_embedding` --- ### 📝 Commits (10+) - [`6e71dda`](https://github.com/AppFlowy-IO/AppFlowy/commit/6e71dda399b57a6a33be79dc674e1e15c52f336a) chore: generate embeddings - [`f589d45`](https://github.com/AppFlowy-IO/AppFlowy/commit/f589d45974b1ffcc2adecb01169cf021c7f6b2bb) chore: save embedding - [`6d79c67`](https://github.com/AppFlowy-IO/AppFlowy/commit/6d79c67f4d5bc99f2842da8ea4bc898837e13593) chore: vec sqlite - [`a641171`](https://github.com/AppFlowy-IO/AppFlowy/commit/a6411712d933dd41ad695e0d6295e91b55df0f16) chore: clippy - [`fff9c77`](https://github.com/AppFlowy-IO/AppFlowy/commit/fff9c77a6bea74b1faabd533c2255b49e3b50c17) chore: init vector - [`14d9830`](https://github.com/AppFlowy-IO/AppFlowy/commit/14d983048b5c7929f91941d5f52aaf774787f054) chore: create vector db - [`6027a99`](https://github.com/AppFlowy-IO/AppFlowy/commit/6027a99df927e7eec301eeb67752d193db2c63fd) chore: periodically write embedding - [`3fbe711`](https://github.com/AppFlowy-IO/AppFlowy/commit/3fbe711d6b920ed3d32f9d460938095e69f8b36e) chore: fix compile - [`f695137`](https://github.com/AppFlowy-IO/AppFlowy/commit/f695137351265ee1438781d2338d6e84d684b01a) chore: skip write - [`d6187ba`](https://github.com/AppFlowy-IO/AppFlowy/commit/d6187ba1b8b802d4df08f7814ea3cd7e520fc283) chore: impl search ### 📊 Changes **139 files changed** (+4998 additions, -4744 deletions) <details> <summary>View changed files</summary> 📝 `.github/workflows/android_ci.yaml.bak` (+1 -1) 📝 `.github/workflows/flutter_ci.yaml` (+1 -1) 📝 `.github/workflows/ios_ci.yaml` (+1 -1) 📝 `.github/workflows/release.yml` (+11 -11) 📝 `.github/workflows/rust_ci.yaml` (+1 -1) 📝 `.github/workflows/rust_coverage.yml` (+1 -1) 📝 `frontend/Makefile.toml` (+35 -34) 📝 `frontend/appflowy_flutter/lib/workspace/application/command_palette/command_palette_bloc.dart` (+1 -3) 📝 `frontend/appflowy_flutter/lib/workspace/application/command_palette/search_service.dart` (+4 -9) 📝 `frontend/appflowy_flutter/macos/Podfile.lock` (+23 -23) 📝 `frontend/rust-lib/Cargo.lock` (+385 -94) 📝 `frontend/rust-lib/Cargo.toml` (+13 -12) 📝 `frontend/rust-lib/build-tool/flowy-ast/src/symbol.rs` (+2 -2) 📝 `frontend/rust-lib/collab-integrate/Cargo.toml` (+4 -2) 📝 `frontend/rust-lib/collab-integrate/src/collab_builder.rs` (+18 -0) ➕ `frontend/rust-lib/collab-integrate/src/instant_indexed_data_provider.rs` (+226 -0) 📝 `frontend/rust-lib/collab-integrate/src/lib.rs` (+1 -1) ➖ `frontend/rust-lib/collab-integrate/src/persistence/mod.rs` (+0 -1) 📝 `frontend/rust-lib/dart-ffi/src/lib.rs` (+19 -7) 📝 `frontend/rust-lib/event-integration-test/Cargo.toml` (+2 -3) _...and 80 more files_ </details> ### 📄 Description • Remove the obsolete Supabase test that’s no longer used. • Add support for generating and using local embeddings. • Introduce local AI-powered search functionality. • Enable full-text search over document content (previously we only indexed files that existed on disk). • Refactor folder-view search so that each workspace maintains its own Tantivy index directory. • Reindex a document automatically whenever its content hash changes. • Add end-to-end integration tests to verify search functionality. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>