django-shouty-orm

Author:	Keryn Knight
Version:	0.1.1

Rationale

I want to use MyModel.objects.only() and .defer() because that's the correct thing to do, even if it's not the default thing Django does. But using .only() and .defer() in Django is an absolute footgun because any attempt to subsequently access the missing fields will ... do another query

Similarly, I don't want Django to silently allow me to do N+1 queries for related managers/querysets. But it does, so there's another footgun.

This module then, is my terrible attempt to fix the footguns automatically, by forcing them to raise exceptions where possible, rather than do the query. This flies in the face of some other proposed solutions over the years on the mailing list, such as automatically doing prefetch_related or select_related.

I think/hope that the package pairs well with django-shouty-templates to try and surface some of the small pains I've had over the years.

What it does

All of the following examples should raise an exception because they pose a probable additional +1 (or more) queries.

Accessing fields intentionally not selected:

>>> u = User.objects.only('pk').get(pk=1)
>>> u.username
MissingLocalField("Access to username [...]")
>>> u = User.objects.defer('username').get(pk=1)
>>> u.email
>>> u.username
MissingLocalField("Access to username [...]")

Access to relationships that have not been selected:

>>> le = LogEntry.objects.get(pk=1)
>>> le.action_flag
>>> le.user.pk
MissingRelationField("Access to user [...]")

Access to reverse relationships that have not been selected:

>>> u = User.objects.only('pk').get(pk=1)
>>> u.logentry_set.all()
MissingReverseRelationField("Access to logentry_set [...]")

Pretty much all relationship access (normal or reverse, OneToOne or ForeignKey or ManyToMany) should be blocked unless select_related or prefetch_related were used to include them.

Setup

Add shoutyorm or shoutyorm.Shout to your settings.INSTALLED_APPS

I'd certainly suggest that you should only enable it when DEBUG is True or during your test suite.

Dependencies

• Django 2.2+ (obviously)
• wrapt 1.11+ (for proxying managers/querysets transparently)

Optional configuration

• settings.SHOUTY_LOCAL_FIELDS may be True|False

  Accessing fields which have been deferred via .only() and .defer() at the QuerySet level will error loudly.

• settings.SHOUTY_RELATION_FIELDS may be True|False

  Accessing OneToOnes which have not been .select_related() at the QuerySet level will error loudly. Accessing local foreignkeys which have not been prefetch_related() or select_related() at the queryset level will error loudly.

• settings.SHOUTY_RELATION_REVERSE_FIELDS may be True|False

  Accessing foreignkeys from the "other" side (that is, via the reverse relation manager) which have not been .prefetch_related() at the QuerySet level will error loudly.

Tests

Just run python3 -m shoutyorm and hope for the best. I usually do.

Alternatives

A similar similar approach is taken by django-seal but without the onus/burden of subclassing from specific models. I've not looked at the implementation details of how seal works, but I expect I could've saved myself quite a lot of headache by seeing what steps it takes in what circumstances, rather than constantly hitting breakpoints and inspecting state.

A novel idea is presented in django-eraserhead of specifically calling out when you might be able to use defer() and only() to reduce your selections, but introducing those optimisations still poses a danger of regression without a test suite and this module.

Having started writing this list of alternatives, I am reminded of nplusone and it turns out that has Django support and a setting for raising exceptions... So all of this patch may be moot, because I expect that covers a lot more? Again I've not looked at their implementation but I'm sure it's miles better than this abomination.

The license

It's FreeBSD. There's should be a LICENSE file in the root of the repository, and in any archives.