Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(sql): use org and course_key filters to use primary key indexes #957

Merged
merged 4 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,13 @@ with
recent_activity as (
select course_key, COUNT(DISTINCT actor_id) as active_last_7_days
from {{ ASPECTS_XAPI_DATABASE }}.navigation_events
where emission_time >= NOW() - INTERVAL 7 DAY
where
emission_time >= NOW() - INTERVAL 7 DAY
{% include 'openedx-assets/queries/common_filters.sql' %}
group by course_key
)

select fss.*, COALESCE(ra.active_last_7_days, 0) as active_within_last_7_days
from {{ DBT_PROFILE_TARGET_DATABASE }}.fact_student_status fss
left join recent_activity ra on fss.course_key = ra.course_key
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,6 @@ with
select org, course_key, learners.actor_id as actor_id
from {{ DBT_PROFILE_TARGET_DATABASE }}.fact_student_status learners
join page_visits using (org, course_key, actor_id)
where approving_state = 'failed' and enrollment_status = 'registered'
where
approving_state = 'failed' and enrollment_status = 'registered'
{% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
) as at_risk_learners using (org, course_key, actor_id)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -29,3 +29,4 @@ where
approving_state = 'failed'
and enrollment_status = 'registered'
and page_visits.last_visited < subtractDays(now(), 7)
{% include 'openedx-assets/queries/common_filters.sql' %}

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ left join
{{ ASPECTS_EVENT_SINK_DATABASE }}.course_names cn
on fes.org = cn.org
and fes.course_key = cn.course_key
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
) as at_risk_learners using (org, course_key, actor_id)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
) as at_risk_learners using (org, course_key, actor_id)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
{% include 'openedx-assets/queries/fact_problem_engagement.sql' %}
with
fact_problem_engagement as (
{% include 'openedx-assets/queries/fact_problem_engagement.sql' %}
)
select fact_problem_engagement.*
from fact_problem_engagement pe
join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@ join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
) as at_risk_learners using (org, course_key, actor_id)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
) as at_risk_learners using (org, course_key, actor_id)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
) as at_risk_learners using (org, course_key, actor_id)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ join
(
{% include 'openedx-assets/queries/at_risk_learner_filter.sql' %}
) as at_risk_learners using (org, course_key, actor_id)
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}

This file was deleted.

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -66,3 +66,4 @@ left join
on fss.org = let.org
and fss.course_key = let.course_key
and fss.actor_id = let.actor_id
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Original file line number Diff line number Diff line change
Expand Up @@ -50,3 +50,4 @@ join
left outer join
{{ DBT_PROFILE_TARGET_DATABASE }}.dim_user_pii users
on toUUID(pv.actor_id) = users.external_user_id
where 1 = 1 {% include 'openedx-assets/queries/common_filters.sql' %}
Loading