Mocking the Django "order_by" QuerySet Method

This week at work, I took on the task of testing order_by("?") in our Django codebase. This line says that we want our QuerySet randomly ordered. In testing, "random" translates to "mocking." In my brain, "mocking" translates to "I have carefully avoided having to mock anything since I learned it. How the hell do I do this?"

After several days and plenty of research, I landed on a solution that seems to work! There is very possibly a better way to do this, but I'm pleased with what I've got so far.

Understanding the Code We're Testing

I've been working on improving the test coverage of our codebase for a few weeks. It was already at around 98%, but we had a few sections that were not yet tested.

In one of the files I was working on, we had the single line qs = order_by("?") that was not yet tested.

Let's zoom out a bit more to see the entire function (or at least, a general representation. Unfortunately, my work doesn't have a software program that deals with foods):

class FoodListView(LoginRequiredMixin):
    template_name = "foods/food_list.html"
    model = Food

    @property
    def meal(self):
        return self.request.meal

    def get_queryset(self, queryset=None):  
        qs = (
            Food
            .objects
            .filter(meal=self.meal)) 
        if "scramble" in self.request.GET: 
            qs = qs.order_by("?")
        else:
            qs = qs.order_by(
                "name" 
            )
        return (
            qs
            .select_related('chef', 'nutrition')

We have have a Django view here that will show us a list of foods. I only want logged in users to be able to access this, so I've added the LoginRequiredMixin. I've also got my template to provide the HTML and my model this view references.

I want to filter on meal. For instance, I'll only show breakfast foods. For this, I have the meal method as a property which returns whatever meal is on the request.

get_queryset is a Django function that lets us define the particular QuerySet we want to show in our view if we want something other than all records in the model our view references.

In this case, I only want foods from the meal in my request. Then I have two options. Generally I will just show the foods in alphabetical order. But if I click the "Scramble" button on my page, I send a GET request with "scramble" and I'll put the foods in random order.

In the Django ORM, order_by is synonymous with ORDER BY in SQL. Calling it with "?" gives us random ordering. The way Django accomplishes this is interesting if you want to check it out.

Testing Random Ordering in Django

My goal was to test that, when the "Scramble" button was clicked, this view would return a list of foods not in alphabetical order. But the random ordering means that I can't test for a particular output since it will change every time. This is where mocking comes in.

I tried a variety of things here. My first thought was to test the entire view, mocking ordering within get_queryset, and checking that the HTML response contained all my foods in a given order. Because "scramble" comes from the request, this seemed like the easiest way to trigger the random ordering in my conditional.

I was unsuccessful with this, so I decided to test just get_queryset to see if it was returning the right QuerySet in the order I expected. This removed the complexity of having to deal with the view and the template, but added the challenge of having to create the request.

Add a Request Attribute to a Django View Test

To define the request, I relied on this StackOverflow post. With the Django ReqeustFactory, I could write this:

            # make a test request with admin user and meal
            test_request = RequestFactory().get("/foods/-admin/?scramble") 
            test_request.user = self.admin
            test_request.meal= self.food.meal

            # call the view with the test request
            view = FoodListView() 
            view.request = test_request

Mocking Django QuerySet Methods

I'll spare you the trial and error and just post the final result.

from unittest.mock import patch
from django.test import TestCase, RequestFactory
from foods.factories import FoodFactory
from foods.models import Food
from foods.views import FoodListView

class FoodListViewTestCase(TestCase):
def test_scramble_food_list(self): 
        """Test get_query_set button shuffles students."""
        FoodFactory( 
            id=100, 
            name="Bacon", 
        )
        FoodFactory( 
            id=101, 
            name="Waffles",
        )

        # order foods by id 
        foods = ( 
            Food
            .objects
            .filter(meal=self.food.meal.id) 
            .order_by("id")
        )

        with patch('foods.models.Food.objects') as mock_query: 
           
            # make a test request with admin user and meal
            test_request = RequestFactory().get("/foods/-admin/?scramble") 
            test_request.user = self.admin
            test_request.meal= self.food.meal
            # call the view with the test request
            view = FoodListView() 
            view.request = test_request

            # mock Queryset methods
            (mock_query
             .filter.return_value
             .order_by.return_value
             .select_related.return_value
             ) = foods
            
            results = view.get_queryset() 
            mock_query.filter().order_by.assert_called_once_with("?")
            self.assertEqual(results, foods)

Step by step, here's what we are doing in this test:

Import all the pieces we'll need for this test. I've got "patch" from the unittest library for mocking. I've also got a factory that creates my food items, assigning them to the meal "Breakfast" in the process.
In my test, I create a couple food items. I run a query to get the foods back in the order I expect, and set that resulting QuerySet equal to foods.
I then use patch to mock the Food items.
I make a request and add it to the view as explained above.
I mock the three QuerySet methods used in get_queryset - filter, order_by, and select_related. Then I set the return values of each of those chained calls equal to foods. The return_value is key here so that you are reassigning the actual result.
I call get_queryset and get the results from it.
get_queryset runs, but whenever the QuerySet methods are used, their results are replaced with my mock instead.
I check that order_by("?") ran once. I got the chain mock_query.filter().order_by from the call list. Since order_by is second in my function, this is what worked to make that assertion true.
I also test that the result of get_queryset with mocking is the same as the list of foods I expect.

The piece that gave me the most trouble (and I'm still not sure I understand it) is the idea that you should mock where the method is called, not where it is defined. In my mind, this meant I would mock order_by in get_queryset or in the view.

But that wasn't working. It wasn't until I mocked order_by in my model, so "foods.models.Food.objects", that I got some traction with my mocking. My best guess is that even though we are calling order_by here, the QuerySet methods are always called on the model.

Resources for Testing Django QuerySet Methods

I read through many Stack Overflow posts to get here, and some might be helpful for more examples and explanations:

Also special shout out to this post. It didn't help me at all, but the line "It's quite easy to mock all methods involved" in regards to the QuerySet methods was humbling.

Should You Mock Django QuerySet Methods?

The general idea with testing, and by extension mocking, is that you don't test your framework. The framework is already tested. Instead, you test the code that you write.

order_by is a built in method for Django QuerySets. I can trust that it will return my ordered QuerySet.

There is perhaps more value in testing that order_by is called with "?". That is reliant on the conditional that we wrote, but it is small win in this case.

Did this add much value to our codebase? No. I added 20 lines of testing to check that one (mostly just for fun) feature was working. It took me 4 days to figure out how to do it. If I just wanted coverage, there were easier ways to do it.

But it the words of my supervisor, "it will be a good learning experience."

And it was. I went really in depth on mocking, the Django ORM, and Django views. I also learned not to fear mocking, but instead regard it with a grudging respect.

I'm still waiting on feedback on my PR so I may soon learn about an entirely better way to do that. In the meantime, if you have any thoughts on making this more straightforward, let me know!

{ mocking the django "order_by" queryset method }