Quick fix? Improve query performance on larger datasets #9806
-
We have: And right now, we have slow performance on some simple queries, especially if we're trying to query against a field value. For instance, finding an asset: Resulting query:
It takes well over 4 seconds on a 2016 macbook pro. I'm not an SQL expert by any means, but i've been playing around for a bit trying to make some improvements Since we're only trying to get assets in this query, i've made the following changes (see bold):
Total query time: 1.6s So this already slices the time in half, can we possibly implement this if there are no adverse effects? Since we're running an asset query here, I don't see a problem with limiting elements that are joined by it's type. But other than this, it seems to be a tad innefficient. If we'd looked in the content table first for this value and just got elements by their ids we'd be well under 50ms here |
Beta Was this translation helpful? Give feedback.
Replies: 6 comments 11 replies
-
First, try running How is your query performance affected if you add the SELECT `elements`.`id`, `elements`.`fieldLayoutId`, `elements`.`uid`, `elements`.`enabled`, `elements`.`archived`, `elements`.`dateCreated`, `elements`.`dateUpdated`, `elements_sites`.`id` AS `siteSettingsId`, `elements_sites`.`slug`, `elements_sites`.`siteId`, `elements_sites`.`uri`, `elements_sites`.`enabled` AS `enabledForSite`, `elements`.`canonicalId`, `elements`.`dateLastMerged`, `assets`.`volumeId`, `assets`.`folderId`, `assets`.`filename`, `assets`.`kind`, `assets`.`width`, `assets`.`height`, `assets`.`size`, `assets`.`focalPoint`, `assets`.`keptFile`, `assets`.`dateModified`, `volumeFolders`.`path` AS `folderPath`, `assets`.`uploaderId`, `content`.`id` AS `contentId`, `content`.`title`, `content`.`field_alt_sxjohgdo`, `content`.`field_answers_tygamntk`, `content`.`field_body`, `content`.`field_completedRegistration_qchdvvhs`, `content`.`field_contentType_kyxwdomj`, `content`.`field_copyright_gfwfzhzb`, `content`.`field_country_iecvbgwi`, `content`.`field_darkMode_cxldzoux`, `content`.`field_description_swgutvjh`, `content`.`field_disabled_vzjyvmsv`, `content`.`field_embed_tgmajaqm`, `content`.`field_embedHtml_ymaynfgz`, `content`.`field_endDate_omlfmxgk`, `content`.`field_hidden_abkccahm`, `content`.`field_intro_nvfphrbd`, `content`.`field_rel_rqmyprus`, `content`.`field_location_rqxobnaa`, `content`.`field_newsletter_laxveajq`, `content`.`field_notifications_nnocfrbz`, `content`.`field_originalId_spvnsofp`, `content`.`field_entryDate_ryofrltd`, `content`.`field_raffles_alqbxcrw`, `content`.`field_seo_bdrmifmv`, `content`.`field_startDate_iruwhvkq`, `content`.`field_heading_fqcanuuv`, `content`.`field_topbarNotice_ieivkxiu`
FROM (SELECT `elements`.`id` AS `elementsId`, `elements_sites`.`id` AS `elementsSitesId`, `content`.`id` AS `contentId`
FROM `elements` `elements`
INNER JOIN `assets` `assets` ON `assets`.`id` = `elements`.`id`
INNER JOIN `volumefolders` `volumeFolders` ON `volumeFolders`.`id` = `assets`.`folderId`
INNER JOIN `elements_sites` `elements_sites` ON `elements_sites`.`elementId` = `elements`.`id`
INNER JOIN `content` `content` ON (`content`.`elementId` = `elements`.`id`) AND (`content`.`siteId` = `elements_sites`.`siteId`)
WHERE (`elements`.`type` = 'craft\\elements\\Asset') AND (`elements_sites`.`siteId`=1) AND (`content`.`field_originalId_spvnsofp`=7199178) AND (`elements`.`archived`=FALSE) AND (`elements`.`dateDeleted` IS NULL) AND (`elements`.`draftId` IS NULL) AND (`elements`.`revisionId` IS NULL)
ORDER BY `elements`.`dateCreated` DESC
LIMIT 1) `subquery`
INNER JOIN `assets` `assets` ON `assets`.`id` = `subquery`.`elementsId`
INNER JOIN `volumefolders` `volumeFolders` ON `volumeFolders`.`id` = `assets`.`folderId`
INNER JOIN `elements` `elements` ON `elements`.`id` = `subquery`.`elementsId`
INNER JOIN `elements_sites` `elements_sites` ON `elements_sites`.`id` = `subquery`.`elementsSitesId`
INNER JOIN `content` `content` ON `content`.`id` = `subquery`.`contentId`
ORDER BY `elements`.`dateCreated` DESC
I’m not sure I’m following you here. As in it would be faster if we joined the |
Beta Was this translation helpful? Give feedback.
-
@brandonkelly Your query, with index and column type int
Time: 14 ms The same query with no index and column-type reverted to textTime: 2.6 seconds The original query with no index, column type text and no
|
Beta Was this translation helpful? Give feedback.
-
@brandonkelly to give another example. I'm trying to get the latest asset here, in twig: Resulting sql:
It takes 3.7 seconds just to fetch this one row from the database. It seems that it's merging all rows, and joining all tables, and then after doing so it plucks the latest row. ` |
Beta Was this translation helpful? Give feedback.
-
For reference, our the tables sizes: |
Beta Was this translation helpful? Give feedback.
-
Just released Craft 3.7.12 with the added |
Beta Was this translation helpful? Give feedback.
-
Hi @brandonkelly with the 3.7.12 release, we have a little problem on our end related to this discussion. For one of our site, we have created a custom element type that extend the Anyway, we use this new ExampleElement and ExampleQuery to have custom element index page, custom validation rules/scenarios and alter the query with beforePrepare and afterPrepare methods. With the new I have try to put Thanks for your help :) |
Beta Was this translation helpful? Give feedback.
Just released Craft 3.7.12 with the added
elements.type
condition. Thanks for the suggestion!