Squash migrations and migrate db to utf8mb4

[arthurpar06]

This PR squashes all migrations into a single .sql file, enabling faster migration and a cleaner migrations folder. As we were performing some inserts into the migrations, I relocated them to seeds (base.yml).

This can be merged immediately, but it's advisable to ensure that we won't add new migrations until the release, as it would be preferable to have an empty migrations folder in the release.

Commands:

# For testing
php artisan migrate --database=testing # to have a sqlite db to dump
php artisan schema:dump --database=testing
# then rename testing-schema.sql to memory-schema.sql


# For mysql (use --prune to remove the migrations from the migrations folder)
php artisan schema:dump --prune

Closes #1731

[arthurpar06]

Okay, what's great about the PR regarding internal fields is that it allowed me to test what happens when we re-dump the db after adding a migration. Verdict: it works perfectly; Laravel keeps the old dump and adds the migrations present in the migrations folder to create a single dump.

[arthurpar06]

Important Notes Regarding This PR

Firstly, this PR squashes migrations. It is crucial to note that anyone who wants to upgrade to this version must first upgrade to the version immediately before this one and run all the migrations. This is because ALL migrations including those that were never run are now added to the dump, which is executed only when the database is empty. Therefore, it is VERY IMPORTANT that users upgrade to the previous version before upgrading to this one.

Additionally, this update changes the database charset to utf8mb4. As discussed with Nabeel, this will enable the addition of new features in the future, and it is better to make this change now so that all databases created from this release will not need to be converted later.
Before performing this update, it is VERY IMPORTANT to back up your database. Everything is automated and should go smoothly, but you can never be too careful.

I have tested with different databases containing varying amounts of data and did not encounter any issues with the migration. However, I would greatly appreciate feedback from @nabeelio, @FatihKoz and @ArthurHetem and anyone else who would like to test it. If you could test it locally with some of your databases and let me know if it works well, that would be great. Even though the change is fairly straightforward in theory.

Reference:

https://stackoverflow.com/questions/42704637/laravel-5-4-upgrade-converting-to-utf4mb-from-utf8

[FatihKoz]

How you will provide the previous version ? What will happen if people do NOT follow that logic ? What if a user is already using utf8mb4 ? Earliest i can test will be during weekend, I can replicate a live server locally which is on latest dev (Turksim, THYVA) and a server is behind like maybe 1-2 months (SagaVA, not sure when I last updated it) Sent from mobile device / Mobil cihaz ile gönderildi 10 Tem 2024 Çar 00:40 tarihinde Arthur Parienté ***@***.***> şunu yazdı:

…

Important Note Regarding This PR Firstly, this PR squashes migrations. It is crucial to note that anyone who wants to upgrade to this version must first upgrade to the version immediately before this one and run all the migrations. This is because *ALL* migrations including those that were never run are now added to the dump, which is executed only when the database is empty. Therefore, it is *VERY IMPORTANT* that users upgrade to the previous version before upgrading to this one. Additionally, this update changes the database charset to utf8mb4. As discussed with Nabeel, this will enable the addition of new features in the future, and it is better to make this change now so that all databases created from this release will not need to be converted later. Before performing this update, it is *VERY IMPORTANT* to back up your database. Everything is automated and should go smoothly, but you can never be too careful. I have tested with different databases containing varying amounts of data and did not encounter any issues with the migration. However, I would greatly appreciate feedback from @nabeelio <https://github.com/nabeelio> and @FatihKoz <https://github.com/FatihKoz> . If you could test it locally with some of your databases and let me know if it works well, that would be great. Even though the change is fairly straightforward in theory. Reference: https://stackoverflow.com/questions/42704637/laravel-5-4-upgrade-converting-to-utf4mb-from-utf8 — Reply to this email directly, view it on GitHub <#1804 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARXKVMJE4V5YG7QPHCH4AK3ZLRKD3AVCNFSM6AAAAABHS6R37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJYG43TMNBWGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[arthurpar06]

By previous version, I mean having run all the latest migrations that were merged into the dev branch before this PR. I was thinking about the zip on Discord. What I mean is that you shouldn't catch up on 15 versions at once by applying this PR, and the thing is, I don't have much of an alternative for that...

Our last migration was just last week... So basically, if they are on the current dev and move to this dev, there won't be any problems. If people don't do this, they will end up missing migrations...

I'd suggest maybe posting a message on Discord, clearly explaining that they need to upgrade to the latest in two steps if people are catching up from a long time ago.

Regarding what happens if the table is already in utf8mb4, there is a check at the beginning of the migration that performs the update. It will not attempt the update if it's already utf8mb4 :)

[ArthurHetem]

Everything worked fine here on my production data (in test env), there are no problems.

[FatihKoz]

I think you already know that the zip on discord is always the latest dev, even if you click a release 1 year before, you will get today's latest dev. This is why I asked that question. Consider a seperate interim build before this, save it somewhere different and publish its info in documents. This can be a solution maybe. Hope this helps Sent from mobile device / Mobil cihaz ile gönderildi 10 Tem 2024 Çar 02:47 tarihinde Arthur Hetem ***@***.***> şunu yazdı:

…

Everything worked fine here on my production data (in test env), there are no problems. — Reply to this email directly, view it on GitHub <#1804 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ARXKVMKMIKW3WNGUVNWKXSDZLRZARAVCNFSM6AAAAABHS6R37KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMJZGAZDOMJSHE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[arthurpar06]

So after discussing it with @nabeelio, the final solution we decided on is to keep the migrations in addition to the squash for now (which solves the issue of catching up many updates at once), and we will remove them at the beginning of version 7.1 (to allow compatibility with Laravel 11).