Skip to content

Enhancement: Make tables import asynchronous #1801

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Koc
Copy link
Contributor

@Koc Koc commented May 12, 2025

Right now module not ready to import large tables. This PR improves import flow by moving it into asynchronous job.

🗒️ TODO:

  • make import work for uploaded (not selected) files as well
  • revert changes to old endpoints, introduce new one instead
  • properly parse activity notification
  • fix pipelines
  • add new tests

🔍 Preview

nextcloud-tables-2025-05-26_18.14.20.mp4

@Koc Koc force-pushed the feature/asyncronous-import branch from 5cbf833 to 3c2422c Compare May 12, 2025 16:00
@blizzz
Copy link
Member

blizzz commented May 23, 2025

Please keep in mind, existing API shall remain stable, but may be flagged deprecated.

@juliusknorr juliusknorr added enhancement New feature or request 3. to review Waiting for reviews 2. developing Work in progress and removed 3. to review Waiting for reviews labels May 23, 2025
@Koc Koc force-pushed the feature/asyncronous-import branch 5 times, most recently from 9b0c3bd to 1dbdfe8 Compare May 26, 2025 15:47
@Koc Koc marked this pull request as ready for review May 26, 2025 15:47
@Koc Koc requested review from enjeck and blizzz as code owners May 26, 2025 15:47
@Koc
Copy link
Contributor Author

Koc commented May 26, 2025

@blizzz What exactly do you mean? Do not change already existent endpoint (keep it synchronous) but introduce v2/v3 for async tasks (even if this endpoints will be not in use on FE side)?

@blizzz
Copy link
Member

blizzz commented May 26, 2025

@blizzz What exactly do you mean? Do not change already existent endpoint (keep it synchronous) but introduce v2/v3 for async tasks (even if this endpoints will be not in use on FE side)?

Yes, exactly. The old ones can get the @deprecated annotation and point to the new method.

Copy link
Member

@blizzz blizzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

replied to questions

@Koc Koc force-pushed the feature/asyncronous-import branch 7 times, most recently from c0a8bee to 1c49d2c Compare June 9, 2025 14:49
@Koc Koc force-pushed the feature/asyncronous-import branch from 1c49d2c to 599fecd Compare June 9, 2025 14:52
@Koc
Copy link
Contributor Author

Koc commented Jun 9, 2025

@blizzz I've reverted all changes to old endpoints. So they are completely untouched. Now we have also scheduleImportIn(Table|View) + scheduleImportUploadIn(Table|View). FE uses new endpoints.

Also it is possible to import uploaded files (app data used under the hood)

Copy link
Member

@blizzz blizzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is shaping up nicely! It looks like a file was not committed. Did not test yet.

namespace OCA\Tables\Activity;

use OCA\Tables\AppInfo\Application;
use OCP\Activity\Exceptions\UnknownActivityException;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The underlying file does not seem to be committed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this exception should be located somewhere in the core. I've just stolen code to publish/parse activity from another module

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, indeed. But this is available only as of NC 30, on main we still support 29.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, how to deal with that? Can we just bump minimum NC supported version to 30? Like in other modules: Collectives, Forms, etc

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We weed need to branch-off, which of course is possible. We were aiming for September originally. Related changes (version bump and housekeeping) would happen in a separate PR. I'll get back about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2. developing Work in progress enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants