Test fails around frameLoaded

The helm test fails repeatedly over the past few days with

AssertionError: Timed out retrying after 30000ms: Expected to find element: `[name="frameEditor"]`, but never found it.

added App::Nextcloud Bug CI Tests labels

Looking at a test recording, apparently the office document is created, but the frame with onlyoffice is not created in the browser for some reason. It waits for 30 seconds; I would assume that that is long enough to see at least the first signs of the onlyoffice frame being created, but maybe if onlyoffice is slow to respond it can take even longer?

I tried to recreate the failure manually. On my own dev instance I couldn't, it worked as the test expects. On main.ci.stackspin.net however, when I create a new document, it opens in a "pop-up" screen, a new tab (after I let firefox allow that). The test is not prepared for that I assume.

I don't understand where the difference in behaviour comes from:

• we haven't upgraded nextcloud in nearly three weeks;
• the behaviour on my dev instance, which I've set to follow main for the moment, is different from that of main.ci.stackspin.net.

I guess we could try to accommodate for both behaviours in the test, but I'd prefer to understand where the difference comes from.

Hm well there's a setting for that in the nextcloud onlyoffice integration app, but it's set to open in a new tab. I'm not sure why this worked before.

Oh maybe I do: there was this bug in the integration app which has the opposite effect, of always opening in the same tab. Maybe the test was developed while that bug was not yet fixed, so the test assumed the "same tab" behaviour. But now that the bug was fixed, office document open in a new tab again making the test fail.

The nasty thing is that we can't pin the version of the nextcloud apps, so this change was introduced "silently" without us changing either nextcloud or onlyoffice.

I'll try to change the test, or maybe change the setting so it opens in the same tab.

assigned to @arie

changed milestone to %1.0

added Doing label

created branch 1061-test-fails-around-frameloaded to address this issue

mentioned in merge request !228 (merged)

mentioned in commit 7d93484a

closed with merge request !228 (merged)