Should IntegrationTestCase use self.client.get()?

[adam-thomas]

We have the option to use self.client.get() instead of view classes.

Pros:

• Much more integration-y.
• Handles rendering on our behalf.
• Less faffing with dealing with functional vs. class-based views.
• Easier to move around between pages within a test.

Cons:

• Less unit-test-y.
• We have URL tests etc. elsewhere, so we may not really need to use client.get().
• Slower.
• Requires URLs - more dependency, less DRYing.
• Works less well with HTTPRequest objects.
• Works differently to how our tests normally do.

@incuna/backend opinions?

[meshy]

Should IntegrationTestCase... ...Much more integration-y.

I rest my case. :P

[mlen108]

Slower.

Much slower.

We should always aim for fast tests execution.

More args to come... eventually..

[meshy]

Slower

Yes, it's slower, but we need a test that goes through the url resolving/middleware etc. The majority of tests should be normal unit tests, but to test that things have been wired up correctly, we need a true integration test. Remember that this class was intended to replace the more trivial selenium tests. They are slow, but to truly replace them, we should be doing end-to-end tests.

[adam-thomas]

As an example, I've just written this for another project, testing that a form (built in crispy_forms) renders a given number of templates. I've modified this example for anonymity; the numbers are random.

class TestFormPerformance(IntegrationTestCase):
    """
    A test to keep an eye on the number of templates rendered while accessing
    this page.
    """
    def setUp(self):
        password = 'secret'
        self.user = UserFactory.build()
        self.user.set_password(password)
        self.user.save()

        self.client = Client()
        self.client.login(username=self.user.email, password=password)

    def test_template_performance(self):
        """Assert the numbers of various templates that are rendered."""
        url = reverse('form_page')
        response = self.client.get(url)
        self.assertEqual(response.status_code, 200)

        templates = response.templates
        self.assertEqual(len(templates), 4242)        

        # Create a dict of {template_name: number_of_instances_rendered}.
        template_names = Counter([t.name for t in templates])

        # Create another dict of the values we expect in template_names.
        # This is not a comprehensive listing, but contains all the templates
        # whose numbers we care about.
        expected_template_counts = {
            'custom_fieldset.html': 42,
            'uni_form/layout/div.html': 42,
            'bootstrap/field.html': 42,
            # ...
        }

        for name, count in expected_template_counts.items():
            with self.subTest(name=name, count=count):
                self.assertEqual(template_names[name], count)

It would be good to roll not only the client setup (not necessarily in setUp, but perhaps in a helper method) but also some elements of the counting assertion into incuna-test-utils.

Asserting that dictionaries correlate using subTest like that seems useful, and we can possibly build other test methods on top of it (for instance, one that takes the equivalent of template_names as a list, and does the Counter-ing for you).

[LilyFoote]

@adam-incuna Looks like it could be worthwhile.

[adam-thomas]

👍

[adam-thomas]

Idea (@Ian-Foote you'll like this): A @client decorator that sets up a test client within a test method. Logging in would probably be done with a helper method so you could pass it a user, although there'd be no harm in adding an @client_authenticated decorator that logged in with a default user or something (in the same way as create_request). Thoughts anyone?

[LilyFoote]

Could be shiny. There may be some hidden pitfalls though.

[adam-thomas]

Yes. I'll probably find out what they are by just writing it. :P Let me know if you think of anything!