simon

How to manipulate the ManyToMany hidden table in Django#

Created: May 28, 2021 6:18 PM
Tags: ORM, Python

Today I encountered a question that had some discussions online, but no one proposed a solution. After researching it myself, I found some valuable information worth documenting. I will explain it in detail below.

Issue#

I want to modify the default user group table in Django, which can be accessed through the User table.

from django.contrib.auth.models import User

u1 = User.objects.get(id=1)
u1.groups  # This is the intermediate table for the auth user group
# You can add a group using the following method
u1.groups.add()

The current requirement is to directly manipulate this table to facilitate batch data insertion.

Background#

First, we need to check the structure of this table in the database. Although we can infer logically that it is a table with foreign key fields for both the Group table and the User table (based on Django's automatic creation of ManyToMany rules).

mysql> desc auth_user_groups;
+----------+---------+------+-----+---------+----------------+
| Field    | Type    | Null | Key | Default | Extra          |
+----------+---------+------+-----+---------+----------------+
| id       | int(11) | NO   | PRI | NULL    | auto_increment |
| user_id  | int(11) | NO   | MUL | NULL    |                |
| group_id | int(11) | NO   | MUL | NULL    |                |
+----------+---------+------+-----+---------+----------------+

As expected, we need to manually define a model to point to this table.

Solutions#

1. Custom model

from django.db import models as db
from django.contrib.auth.models import Group, User

class UserAuthGroup(db.Model):

    user = db.ForeignKey(User, on_delete=db.CASCADE)
    group = db.ForeignKey(Group, on_delete=db.CASCADE)

    class Meta:
        db_table = "auth_user_groups"  # Specify the table name

Now, we can manipulate the data in this table by opening the Python Console.

However, when we run the server, another error occurs.

After studying the source code, it turns out that when running the server, a system check is performed, which includes an all model check. One of the indicators is fields.E340, which represents:

The intermediate table of the field conflicts with the table name/

...

It can be seen that Django does not want us to directly manipulate the intermediate table, as it can easily lead to data inconsistency. However, sometimes we need to implement some hacky functionality, so we have to do it reluctantly.

2. Resolving startup errors

The solution is to comment out this check and add the following content to the settings.py file.

Copy

SILENCED_SYSTEM_CHECKS = ["fields.E340"]  # You can add multiple

Now, everything is working fine when running the server.

References#

1. http://c.biancheng.net/view/8001.html
2. https://www.cnblogs.com/fengbo123/p/10837696.html
3. https://www.pythonanywhere.com/forums/topic/14430/
4. https://docs.djangoproject.com/en/3.2/ref/models/options/