Migration from SS2/3 to 4 should update Folder records

[dnsl48]

@gheggie commented on Sat Mar 02 2019

When migrating from SS2/3 to 4, existing Folder records will remain in the File table only, assuming no changes to the Folder ORM. However, this doesn't take into account SS4's versioning for Folders, which should always have a corresponding File_Live table record (i.e. Folders are always considered to be published/Live).

The scenario I had: after a migration, a front-end FileField which uploads to the Uploads folder led to a duplicate Uploads folder record being created, because a Live version of the Uploads folder wasn't found. There were other duplicate Folder records created for other folders as well because of this.

https://gist.github.com/gheggie/b2cf93ba38b57b9eadce3e2f29ba8b3b is a task which fixed this issue. Although not perfect, I think some sort of migration task like this should be available by default.

---

@wernerkrauss commented on Tue Mar 12 2019

I can confirm this behaviour on a project I upgraded from SS3 to SS4: the folder "Uploads" wasn't published and recreated (with same ParentID), cause it only was in stage table.

Maybe it's a bug in Folder::find_or_make(), that it needs to look at the stage table inside a Versioned::withReadingMode() statement?

[sminnee]

This can be seen as a special case of "When Versioned is applied to an existing MyObject, the MyObject table should be populated from MyTable_Live and the MyObject_Versions table should have 1 version created for each record."

This, in turn is a special case of needing to have an initialising query when a new table is created as part of a migration of an existing database.

The other place where this has come up is creating, say, a Page table after there are a lot of SiteTree Class=Page records. This typically happens when you add a custom field to the Page for the first time.

This is the issue underlying a couple of long-standing bugs:

• [ORM] Integrity not enforced between base data table and subclass table silverstripe-framework#7277
• [ORM] Default values set in $db can not be relied upon for sub-class tables silverstripe-framework#4700

[sminnee]

FYI I've logged a ticket summarising the API change needed to fix this, and other, bugs. silverstripe/silverstripe-framework#8908

[dnsl48]

/cc @gheggie @wernerkrauss

[maxime-rainville]

Just clarifying my understanding of the problem here.

Folders are set to always stay in the draft table. If we have a front end form with a file field, when we try to save the file in a folder, we end up with duplicate folders, because queries are run with the versioned stage set to Live and can't find our existing folders.

Is that the problem?

[wernerkrauss]

@maxime-rainville afair that was what I encountered, yes.

[gheggie]

@maxime-rainville Yes, that was my experience. But the duplication isn't limited to front-end file uploads. Adding a file through the CMS file admin interface can also cause a duplicate folder.