🚧 Warning, this Chapter is freshly updated for Django 5 + Python 3.13.
The code listings should have valid syntax, and I’ve been through and sense-checked the chapter text, but a few things might still be off! So let me know what you think of the chapter, via [email protected]
Now that we have a functional authentication system, we want to use it to identify users, and be able to show them all the lists they have created.
To do that, we’re going to have to write FTs that have a logged-in user. Rather than making each test go through the (time-consuming) login email dance, we want to be able to skip that part.
This is about separation of concerns. Functional tests aren’t like unit tests, in that they don’t usually have a single assertion. But, conceptually, they should be testing a single thing. There’s no need for every single FT to test the login/logout mechanisms. If we can figure out a way to "cheat" and skip that part, we’ll spend less time waiting for duplicated test paths.
|
Tip
|
Don’t overdo de-duplication in FTs. One of the benefits of an FT is that it can catch strange and unpredictable interactions between different parts of your application. |
It’s quite common for a user to return to a site and still have a cookie, which means they are "pre-authenticated", so this isn’t an unrealistic cheat at all. Here’s how you can set it up:
from django.conf import settings
from django.contrib.auth import BACKEND_SESSION_KEY, SESSION_KEY, get_user_model
from django.contrib.sessions.backends.db import SessionStore
from .base import FunctionalTest
User = get_user_model()
class MyListsTest(FunctionalTest):
def create_pre_authenticated_session(self, email):
user = User.objects.create(email=email)
session = SessionStore()
session[SESSION_KEY] = user.pk # (1)
session[BACKEND_SESSION_KEY] = settings.AUTHENTICATION_BACKENDS[0]
session.save()
## to set a cookie we need to first visit the domain.
## 404 pages load the quickest!
self.browser.get(self.live_server_url + "/404_no_such_url/")
self.browser.add_cookie(
dict(
name=settings.SESSION_COOKIE_NAME,
value=session.session_key, # (2)
path="/",
)
)-
We create a session object in the database. The session key is the primary key of the user object (which is actually the user’s email address).
-
We then add a cookie to the browser that matches the session on the server—on our next visit to the site, the server should recognise us as a logged-in user.
Note that, as it is, this will only work because we’re using LiveServerTestCase,
so the User and Session objects we create will end up in the same database
as the test server.
Later we’ll need to modify it so that it works
against the database on the staging server too.
Being an attempt to explain sessions, cookies, and authentication in Django.
Because HTTP is stateless, servers need a way of recognising different clients with every single request. IP addresses can be shared, so the usual solution is to give each client a unique session ID, which it will store in a cookie and submit with every request. The server will store that ID somewhere (by default, in the database), and then it can recognise each request that comes in as being from a particular client.
If you log in to the site using the dev server,
you can actually take a look at your session ID by hand if you like.
It’s stored under the key sessionid by default.
See Examining the session cookie in the Debug toolbar.
These session cookies are set for all visitors to a Django site, whether they’re logged in or not.
When we want to recognise a client as being a logged-in and authenticated user, again, rather than asking the client to send their username and password with every single request, the server can actually just mark that client’s session as being an authenticated session, and associate it with a user ID in its database.
A session is a dictionary-like data structure,
and the user ID is stored under the key given by django.contrib.auth.SESSION_KEY.
You can check this out in a ./manage.py shell if you like:
$ python src/manage.py shell
[...]
In [1]: from django.contrib.sessions.models import Session
# substitute your session id from your browser cookie here
In [2]: session = Session.objects.get(
session_key="8u0pygdy9blo696g3n4o078ygt6l8y0y"
)
In [3]: print(session.get_decoded())
{'_auth_user_id': '[email protected]', '_auth_user_backend':
'accounts.authentication.PasswordlessAuthenticationBackend'}
You can also store any other information you like on a user’s session,
as a way of temporarily keeping track of some state.
This works for non–logged-in users too.
Just use request.session inside any view, and it works as a dict.
There’s more information in the
Django docs on sessions.
To check that create_pre_authenticated_session() system works,
it would be good to use some of the code from our previous test.
Let’s make a couple of functions
called wait_to_be_logged_in and wait_to_be_logged_out.
To access them from a different test,
we’ll need to pull them up into FunctionalTest.
We’ll also tweak them slightly so that they can take an arbitrary email address
as a parameter:
class FunctionalTest(StaticLiveServerTestCase):
[...]
def wait_to_be_logged_in(self, email):
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "#id_logout"),
)
navbar = self.browser.find_element(By.CSS_SELECTOR, ".navbar")
self.assertIn(email, navbar.text)
def wait_to_be_logged_out(self, email):
self.wait_for(
lambda: self.browser.find_element(By.CSS_SELECTOR, "input[name=email]")
)
navbar = self.browser.find_element(By.CSS_SELECTOR, ".navbar")
self.assertNotIn(email, navbar.text)Hm, that’s not bad, but I’m not quite happy
with the amount of duplication of wait_for stuff in here.
Let’s make a note to come back to it,
and get these helpers working.
-
'Clean up wait_for stuff in base.py'
First we use them in 'test_login.py':
def test_login_using_magic_link(self):
[...]
# she is logged in!
self.wait_to_be_logged_in(email=TEST_EMAIL)
# Now she logs out
self.browser.find_element(By.CSS_SELECTOR, "#id_logout").click()
# She is logged out
self.wait_to_be_logged_out(email=TEST_EMAIL)Just to make sure we haven’t broken anything, we rerun the login test:
$ python src/manage.py test functional_tests.test_login [...] OK
And now we can write a placeholder for the "My Lists" test, to see if our pre-authenticated session creator really does work:
def test_logged_in_users_lists_are_saved_as_my_lists(self):
email = "[email protected]"
self.browser.get(self.live_server_url)
self.wait_to_be_logged_out(email)
# Edith is a logged-in user
self.create_pre_authenticated_session(email)
self.browser.get(self.live_server_url)
self.wait_to_be_logged_in(email)That gets us:
$ python src/manage.py test functional_tests.test_my_lists [...] OK
That’s a good place for a commit:
$ git add src/functional_tests $ git commit -m "test_my_lists: precreate sessions, move login checks into base"
When we pre-populate the database with test data,
as we’ve done here with the User object and its associated Session object,
what we’re doing is setting up what’s called a "test fixture".
Django comes with built-in support for saving database objects as JSON
(using the manage.py dumpdata),
and automatically loading them in your test runs
using the fixtures class attribute on TestCase.
You’ll find people out there saying don’t use JSON fixtures, and I tend to agree. They’re a nightmare to maintain when your model changes. Plus it’s difficult for the reader to tell which of the many attribute values specified in the JSON are critical for the behaviour under test, and which are just filler. Finally, even if tests start out sharing fixtures, sooner or later one test will want slightly different versions of the data, and you end up copying the whole thing around to keep them isolated, and again it’s hard to tell what’s relevant to the test and what is just happenstance.
It’s usually much more straightforward to just load the data directly using the Django ORM.
|
Tip
|
Once you have more than a handful of fields on a model,
and/or several related models,
you’ll want to factor out some nice helper methods with descriptive names
to build out your data.
A lot of people also like
factory_boy,
but I think the most important thing is the descriptive names.
|
We’ve used decorators a few times in our code so far, but it’s time to learn how they actually work by making one of our own.
First, let’s imagine how we might want our decorator to work.
It would be nice to be able to replace all the custom wait/retry/timeout logic
in wait_for_row_in_list_table() and the inline self.wait_fors()
in the wait_to_be_logged_in/out.
Something like this would look lovely:
@wait
def wait_for_row_in_list_table(self, row_text):
rows = self.browser.find_elements(By.CSS_SELECTOR, "#id_list_table tr")
self.assertIn(row_text, [row.text for row in rows])
@wait
def wait_to_be_logged_in(self, email):
self.browser.find_element(By.CSS_SELECTOR, "#id_logout")
navbar = self.browser.find_element(By.CSS_SELECTOR, ".navbar")
self.assertIn(email, navbar.text)
@wait
def wait_to_be_logged_out(self, email):
self.browser.find_element(By.CSS_SELECTOR, "input[name=email]")
navbar = self.browser.find_element(By.CSS_SELECTOR, ".navbar")
self.assertNotIn(email, navbar.text)Are you ready to dive in?
Although decorators are quite difficult to wrap your head around
(I know it took me a long time before I was comfortable with them,
and I still have to think about them quite carefully whenever I make one),
the nice thing is that we’ve already dipped our toes into functional programming
in our self.wait_for() helper function.
That’s a function that takes another function as an argument,
and a decorator is the same.
The difference is that the decorator doesn’t actually execute any code itself—it
returns a modified version of the function that it was given.
Our decorator wants to return a new function which will keep retrying the function being decorated, catching our usual exceptions, until a timeout occurs. Here’s a first cut:
def wait(fn): #(1)
def modified_fn(): #(3)
start_time = time.time()
while True: #(4)
try:
return fn() #(5)
except (AssertionError, WebDriverException) as e: #(4)
if time.time() - start_time > MAX_WAIT:
raise e
time.sleep(0.5)
return modified_fn #(2)-
A decorator is a way of modifying a function; it takes a function as an argument…
-
…and returns another function as the modified (or "decorated") version.
-
Here’s where we define our modified function.
-
And here’s our familiar loop, which will keep going, catching the usual exceptions, until our timeout expires.
-
And as always, we call our function and return immediately if there are no exceptions.
That’s almost right, but not quite; try running it?
$ python src/manage.py test functional_tests.test_my_lists
[...]
self.wait_to_be_logged_out(email)
TypeError: wait.<locals>.modified_fn() takes 0 positional arguments but 2 were
given
Unlike in self.wait_for, the decorator is being applied to functions
that have arguments:
@wait
def wait_to_be_logged_in(self, email):
self.browser.find_element(By.CSS_SELECTOR, "#id_logout")
[...]wait_to_be_logged_in takes self and email as positional arguments.
But when it’s decorated, it’s replaced with modified_fn,
which current takes no arguments.
How do we magically make it so our modified_fn can handle the same arguments
as whatever fn the decorator gets given has?
The answer is a bit of Python magic,
args and *kwargs, more formally known as
"variadic arguments",
apparently (I only just learned that):
def wait(fn):
def modified_fn(*args, **kwargs): #(1)
start_time = time.time()
while True:
try:
return fn(*args, **kwargs) #(2)
except (AssertionError, WebDriverException) as e:
if time.time() - start_time > MAX_WAIT:
raise e
time.sleep(0.5)
return modified_fn-
Using
argsand*kwargs, we specify thatmodified_fn()may take any arbitrary positional and keyword arguments. -
As we’ve captured them in the function definition, we make sure to pass those same arguments to
fn()when we actually call it.
One of the fun things this can be used for is to make a decorator that changes the arguments of a function. But we won’t get into that now. The main thing is that our decorator now works!
$ python src/manage.py test functional_tests.test_my_lists [...] OK
And do you know what’s truly satisfying?
We can use our wait decorator for our self.wait_for helper as well!
Like this:
@wait
def wait_for(self, fn):
return fn()Lovely! Now all our wait/retry logic is encapsulated in a single place,
and we have a nice easy way of applying those waits,
either inline in our FTs using self.wait_for(),
or on any helper function using the @wait decorator.
Let’s just check all the FTs still pass of course:
Ran 8 tests in 19.379s OK
And we’re good to cross of that scratchpad item:
-
'Clean up wait_for stuff in base.py'
In the next chapter we’ll try to deploy our code to staging, and use the pre-authenticated session fixtures on the server. As we’ll see it’ll help us catch a little bug or two!
- Decorators are nice
-
Decorators can be a great way of abstracting out different levels of concerns. They let us write our test assertions without having to think about waits at the same time.
- De-duplicate your FTs, with caution
-
Every single FT doesn’t need to test every single part of your application. In our case, we wanted to avoid going through the full login process for every FT that needs an authenticated user, so we used a test fixture to "cheat" and skip that part. You might find other things you want to skip in your FTs. A word of caution, however: functional tests are there to catch unpredictable interactions between different parts of your application, so be wary of pushing de-duplication to the extreme.
- Test fixtures
-
Test fixtures refers to test data that needs to be set up as a precondition before a test is run—often this means populating the database with some information, but as we’ve seen (with browser cookies), it can involve other types of preconditions.
- Avoid JSON fixtures
-
Django makes it easy to save and restore data from the database in JSON format (and others) using the
dumpdataandloaddatamanagement commands. I would tend to recommend against them, as they are painful to manage when your database schema changes. Use the ORM, with some nicely named helper functions instead.