[PR #2635] [MERGED] [Fix] Integration test flakes. #5339

New issue

Closed

opened 2026-03-23 22:18:17 +00:00 by mirror · 0 comments

mirror commented

2026-03-23 22:18:17 +00:00

Owner

📋 Pull Request Information

Original PR: https://github.com/AppFlowy-IO/AppFlowy/pull/2635
Author: @a-wallen
Created: 5/26/2023
Status: ✅ Merged
Merged: 5/28/2023
Merged by: @a-wallen

Base: main ← Head: fix_integration_test_flakes

📝 Commits (4)

8d74b71 feat: add tearDown and tearDownAll to reset workspaces
e88595c fix: register tearDown and tearDownAll in empty_document_test
ed26054 feat: register tearDown and tearDownAll in board_test
6ce49e9 feat: register tearDown and tearDownAll in open_ai

📊 Changes

4 files changed (+28 additions, -1 deletions)

View changed files

📝 frontend/appflowy_flutter/integration_test/board_test.dart (+2 -0)
📝 frontend/appflowy_flutter/integration_test/empty_document_test.dart (+9 -1)
📝 frontend/appflowy_flutter/integration_test/open_ai_smart_menu_test.dart (+2 -0)
📝 frontend/appflowy_flutter/integration_test/util/data.dart (+15 -0)

📄 Description

Issue https://github.com/AppFlowy-IO/AppFlowy/issues/2346

Tests on a workspace with an empty document flakes. It succeeds in certain runs, but fails in others.

Here are the two exceptions that I caught for the flaky test.

1. Tests on a workspace with only an empty document /grid shortcut creates a new grid and view of the grid (failed)

  The following TestFailure was thrown running a test:
  Expected: at least 2 matching nodes in the widget tree
    Actual: _WidgetTypeFinder:<zero widgets with type "SelectionMenuItemWidget" (ignoring offstage
  widgets)>
     Which: means none were found but some were expected

-- and --

2. The following assertion was thrown while running async test code:
pumpAndSettle timed out

When the exception was thrown, this was the stack:
#0      WidgetTester.pumpAndSettle.<anonymous closure> (package:flutter_test/src/widget_tester.dart:651:11)
<asynchronous suspension>
<asynchronous suspension>

Different errors happen in CI, but they all have the exact root cause.

The first test run on this workspace creates a new view of a board. github.com/AppFlowy-IO/AppFlowy@18672b197c/frontend/appflowy_flutter/integration_test/empty_document_test.dart (L34)
Since the previous test didn't clean the workspace, the editor looks for the board created in the previous test.
The workspace can't find the board, so it returns a CircularProgressIndicator. (actively being worked on in https://github.com/AppFlowy-IO/AppFlowy/pull/2011/files).
The second test runs and calls pumpAndSettle after it requests focus from the editor.
github.com/AppFlowy-IO/AppFlowy@18672b197c/frontend/appflowy_flutter/integration_test/empty_document_test.dart (LL93C8-L93C8)
The call never settles because of the infinite animation for CircularProgressIndicator.

Solution: Remove the unzipped workspace before running the next test.

Since the editor reads from the file system, all documents must be cleaned between tests!

PR Checklist

My code adheres to the AppFlowy Style Guide
I've listed at least one issue that this PR fixes in the description above.
I've added a test(s) to validate changes in this PR, or this PR only contains semantic changes.
All existing tests are passing.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/AppFlowy-IO/AppFlowy/pull/2635 **Author:** [@a-wallen](https://github.com/a-wallen) **Created:** 5/26/2023 **Status:** ✅ Merged **Merged:** 5/28/2023 **Merged by:** [@a-wallen](https://github.com/a-wallen) **Base:** `main` ← **Head:** `fix_integration_test_flakes` --- ### 📝 Commits (4) - [`8d74b71`](https://github.com/AppFlowy-IO/AppFlowy/commit/8d74b71022997b099c26f29a10f668334e7107ef) feat: add tearDown and tearDownAll to reset workspaces - [`e88595c`](https://github.com/AppFlowy-IO/AppFlowy/commit/e88595cec350c1489c59018527cbe8215054798c) fix: register tearDown and tearDownAll in empty_document_test - [`ed26054`](https://github.com/AppFlowy-IO/AppFlowy/commit/ed260540920329c19a5e960e8380910bd4e0ef48) feat: register tearDown and tearDownAll in board_test - [`6ce49e9`](https://github.com/AppFlowy-IO/AppFlowy/commit/6ce49e9ba09d5cc66ca54b93663d98f4f77570c2) feat: register tearDown and tearDownAll in open_ai ### 📊 Changes **4 files changed** (+28 additions, -1 deletions) <details> <summary>View changed files</summary> 📝 `frontend/appflowy_flutter/integration_test/board_test.dart` (+2 -0) 📝 `frontend/appflowy_flutter/integration_test/empty_document_test.dart` (+9 -1) 📝 `frontend/appflowy_flutter/integration_test/open_ai_smart_menu_test.dart` (+2 -0) 📝 `frontend/appflowy_flutter/integration_test/util/data.dart` (+15 -0) </details> ### 📄 Description **Issue** https://github.com/AppFlowy-IO/AppFlowy/issues/2346 Tests on a workspace with an empty document flakes. It [succeeds in certain runs](https://github.com/AppFlowy-IO/AppFlowy/actions/runs/5086697721/jobs/9141396746#step:11:3083), but [fails in others](https://github.com/AppFlowy-IO/AppFlowy/actions/runs/5059606445/jobs/9081384873#step:11:3069). Here are the two exceptions that I caught for the flaky test. ``` 1. Tests on a workspace with only an empty document /grid shortcut creates a new grid and view of the grid (failed) The following TestFailure was thrown running a test: Expected: at least 2 matching nodes in the widget tree Actual: _WidgetTypeFinder:<zero widgets with type "SelectionMenuItemWidget" (ignoring offstage widgets)> Which: means none were found but some were expected -- and -- 2. The following assertion was thrown while running async test code: pumpAndSettle timed out When the exception was thrown, this was the stack: #0 WidgetTester.pumpAndSettle.<anonymous closure> (package:flutter_test/src/widget_tester.dart:651:11) <asynchronous suspension> <asynchronous suspension> ``` Different errors happen in CI, but they all have the exact root cause. 1. The first test run on this workspace creates a new view of a board. https://github.com/AppFlowy-IO/AppFlowy/blob/18672b197cf8e4ef79b8bfd7cbeae90bbc7f9da8/frontend/appflowy_flutter/integration_test/empty_document_test.dart#L34 2. Since the previous test didn't clean the workspace, the editor looks for the board created in the previous test. 3. The workspace can't find the board, so it returns a `CircularProgressIndicator.` (actively being worked on in https://github.com/AppFlowy-IO/AppFlowy/pull/2011/files). 4. The second test runs and calls `pumpAndSettle` after it requests focus from the editor. https://github.com/AppFlowy-IO/AppFlowy/blob/18672b197cf8e4ef79b8bfd7cbeae90bbc7f9da8/frontend/appflowy_flutter/integration_test/empty_document_test.dart#LL93C8-L93C8 5. The call never settles because of the infinite animation for `CircularProgressIndicator.` **Solution**: Remove the unzipped workspace before running the next test. Since the editor reads from the file system, all documents must be cleaned between tests! #### PR Checklist - [x] My code adheres to the [AppFlowy Style Guide](https://appflowy.gitbook.io/docs/essential-documentation/contribute-to-appflowy/software-contributions/submitting-code/style-guides) - [x] I've listed at least one issue that this PR fixes in the description above. - [x] I've added a test(s) to validate changes in this PR, or this PR only contains semantic changes. - [ ] All existing tests are passing. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>