AppFlowy-IO/AppFlowy

Fork 0

mirror of https://github.com/AppFlowy-IO/AppFlowy.git synced 2026-03-24 12:56:59 +00:00

[GH-ISSUE #2504] [Bug] Missing pages and documents #1079

New issue

Closed

opened 2026-03-23 20:45:14 +00:00 by mirror · 4 comments

mirror commented

2026-03-23 20:45:14 +00:00

Owner

Originally created by @izissise on GitHub (May 10, 2023).
Original GitHub issue: https://github.com/AppFlowy-IO/AppFlowy/issues/2504

Bug Description

I used Appflowy for 2 or 3 weeks without restarting the application.

Today I restarted it, and most of the pages are missing on the side panel, the pages that are here have the latest data.

I have a backup of the sqlite database before quitting AppFlowy, I can find text from missing pages when selecting document_rev_snapshot table .

So the data seems to be here, but isn't show, is there a way to have more logs on eventual error in the program?

How to Reproduce

Use AppFlowy for an extended period of time 2/3 weeks

Expected Behavior

Everything show up

Operating System

Linux

AppFlowy Version(s)

AppFlowy_x86_64-unknown-linux-gnu_ubuntu-18.04.tar.gz to 0.1.2 flathub commit 002cdf13925d5e54a5b1e52a92667449c751d5538e74dabd38d4436a8830f8b3

Screenshots

No response

Additional Context

It seems to me that the pages list has some kind deserialization error stopping the process midway leaving only pages before the one that crash

Originally created by @izissise on GitHub (May 10, 2023). Original GitHub issue: https://github.com/AppFlowy-IO/AppFlowy/issues/2504 ### Bug Description I used Appflowy for 2 or 3 weeks without restarting the application. Today I restarted it, and most of the pages are missing on the side panel, the pages that are here have the latest data. I have a backup of the sqlite database before quitting AppFlowy, I can find text from missing pages when selecting `document_rev_snapshot` table . So the data seems to be here, but isn't show, is there a way to have more logs on eventual error in the program? ### How to Reproduce Use AppFlowy for an extended period of time 2/3 weeks ### Expected Behavior Everything show up ### Operating System Linux ### AppFlowy Version(s) AppFlowy_x86_64-unknown-linux-gnu_ubuntu-18.04.tar.gz to 0.1.2 flathub commit 002cdf13925d5e54a5b1e52a92667449c751d5538e74dabd38d4436a8830f8b3 ### Screenshots _No response_ ### Additional Context It seems to me that the pages list has some kind deserialization error stopping the process midway leaving only pages before the one that crash

mirror closed this issue

2026-03-23 20:45:14 +00:00

mirror commented

2026-03-23 20:45:14 +00:00

Author

Owner

@Xazin commented on GitHub (May 10, 2023):

I'm not sure if you have personal/private things in your workspace, otherwise if you can share your workspace zipped, then maybe someone can try to debug it in a development environment.

If not, it might take longer time for discovery, since without a reproducible sample it's impossible for anyone else to debug it.

@Xazin commented on GitHub (May 10, 2023): I'm not sure if you have personal/private things in your workspace, otherwise if you can share your workspace zipped, then maybe someone can try to debug it in a development environment. If not, it might take longer time for discovery, since without a reproducible sample it's impossible for anyone else to debug it.

mirror commented

2026-03-23 20:45:15 +00:00

Author

Owner

@izissise commented on GitHub (May 10, 2023):

I think, I've found the issue, here is what I've done:

$ sqlite3 flowy-database.db 'select * from folder_rev_snapshot;' > dump.folder_rev_snapshot
# the file has 48 lines
$ for i in `seq 1 48`; do \
  echo $i; \
  sed -n "$i"p dump.folder_rev_snapshot | cut -d'|' -f6 | jq -r '.[0].insert' | jq --sort-keys -r .; \
done


...
17 -> ok
18 -> ok
19
parse error: Invalid numeric literal at line 1, column 2761
20
parse error: Invalid numeric literal at line 1, column 2967
21
parse error: Invalid numeric literal at line 1, column 2968
...

$ sed -n 19p dump.folder_rev_snapshot  | cut -d'|' -f6 | jq -r '.[0].insert' | head -c 2761; echo
...
"belongings":[],"modified_t0,"modified_time"


# indeed all subsequent entries has this weird modified_t0 in the entry
# after entry 20 there is also `modified_t82e`

Some weird corruption is happening, not sure how I can fix it

Line 25 in full as an example https://pastebin.com/U6P7Wujw

@izissise commented on GitHub (May 10, 2023): I think, I've found the issue, here is what I've done: ```bash $ sqlite3 flowy-database.db 'select * from folder_rev_snapshot;' > dump.folder_rev_snapshot # the file has 48 lines $ for i in `seq 1 48`; do \ echo $i; \ sed -n "$i"p dump.folder_rev_snapshot | cut -d'|' -f6 | jq -r '.[0].insert' | jq --sort-keys -r .; \ done ... 17 -> ok 18 -> ok 19 parse error: Invalid numeric literal at line 1, column 2761 20 parse error: Invalid numeric literal at line 1, column 2967 21 parse error: Invalid numeric literal at line 1, column 2968 ... $ sed -n 19p dump.folder_rev_snapshot | cut -d'|' -f6 | jq -r '.[0].insert' | head -c 2761; echo ... "belongings":[],"modified_t0,"modified_time" # indeed all subsequent entries has this weird modified_t0 in the entry # after entry 20 there is also `modified_t82e` ``` Some weird corruption is happening, not sure how I can fix it Line 25 in full as an example https://pastebin.com/U6P7Wujw

mirror commented

2026-03-23 20:45:15 +00:00

Author

Owner

@izissise commented on GitHub (May 10, 2023):

We some effort I've been able to recreate the last entry in folder_rev_snapshot by carefully recreating the json with the other entry, using sqlite3 cli to remove all others entry and update the last one with the fixed json. Hopefully this won't happen again

@izissise commented on GitHub (May 10, 2023): We some effort I've been able to recreate the last entry in `folder_rev_snapshot` by carefully recreating the json with the other entry, using sqlite3 cli to remove all others entry and update the last one with the fixed json. Hopefully this won't happen again

mirror commented

2026-03-23 20:45:15 +00:00

Author

Owner

@appflowy commented on GitHub (May 11, 2023):

@izissise Thanks for trying. We are currently refactoring the data structure in the development branch. We are replacing the SQLite database with a key/value database, which means that the document data structure will be completely different. We anticipate that after the refactoring is completed, AppFlowy will become more robust.

@appflowy commented on GitHub (May 11, 2023): @izissise Thanks for trying. We are currently refactoring the data structure in the development branch. We are replacing the SQLite database with a key/value database, which means that the document data structure will be completely different. We anticipate that after the refactoring is completed, AppFlowy will become more robust.