In this article we are going to show you exact steps that will help you to migrate existing models primary key from an integer pk to the uuid. For this purpose we will create a data migration, fill it in and migrate data between models.
- Create an empty Django migration for our purposes. [1]
- Fill model with a data migration. [2]
- How to move data between Django models. [3]
- Move data between models for specific objects. [3.1]
- Migrate model pk from an integer id to the uuid. [4]
Create an empty Django migration for our purposes
Django framework allows us to create not only automatically generated migration files but empty, where we can provide different manipulations ourselves. Here are some of them: moving data, rewriting fields, prepopulating tables, etc. To achieve this goal, we must create an empty migration using a built-in command manage.py makemigrations appname --empty, where "appname" is your target application name, and as a result, you will get a new migration file in your app's migrations folder which contains the next code:
from django.db import migrations
class Migration(migrations.Migration):
dependencies = [
('appname', '0001_initial'),
]
operations = []
Here we have the Migration class with dependencies and operations lists. Further work will be done in this file, where we will define a function above the Migration class and call it in its options.
Of course, after modifying our file we must apply it with the command manage.py migrate
Fill model with a data migration
Sometimes we need to prepopulate a database table with some data. For example, we have select options that are common for several projects, so we can reuse the model and migration file there. Down below, we have an example with prepopulating DB with lead sources:
from django.db import migrations
def fill_sources(apps, schema_editor):
Source = apps.get_model('appname', 'Source')
source_list = [
'Call', 'Email', 'Website', 'Social network', 'Existing Customer',
'Partner', 'Public Relations', 'Campaign', 'Other'
]
for source_name in source_list:
Source.objects.create(source=source_name)
class Migration(migrations.Migration):
dependencies = [
('appname', '0001_initial'),
]
operations = [
migrations.RunPython(fill_sources),
]
First of all, we define a function.
def fill_sources(apps, schema_editor):
You can choose the name yourself, but it should reflect the action performed by the function.
Then we need to get our model using the method apps.get_model()
Source = apps.get_model('appname', 'Source')
The next step is moving through source_list with cycle and creating a model object for each item.
source_list = [
'Call', 'Email', 'Website', 'Social network', 'Existing Customer',
'Partner', 'Public Relations', 'Campaign', 'Other'
]
for source_name in source_list:
Source.objects.create(source=source_name)
Finally, we have to call this script in our migration using built-in migration method migrations.RunPython() passing the name of our function there.
operations = [
migrations.RunPython(fill_sources),
]
The database table will be populated with the data once you run migrate command next time.
How to move data between Django models
Move data between models can be used for populating similar fields, copying data to another table for any weird reasons.
For these purposes, we use a similar function to the above, but with additional get_model method like this:
def move_data(apps, schema_editor):
Model1 = apps.get_model('appname', 'Model1')
Model2 = apps.get_model('appname', 'Model2')
for m1_object in Model1.objects.all():
Model2.objects.create(m2_field_name=m1_object.m1_field_name)
Here we get two models, and going through the objects of the first, we perform create action for the second one by filling in the appropriate fields.
Move data between models for specific objects
Also, we could have situations where we need to populate the specific object. So, we just have to add filtering for queryset.
def move_data(apps, schema_editor):
Model1 = apps.get_model('appname', 'Model1')
Model2 = apps.get_model('appname', 'Model2')
for m1_obj in Model1.objects.all():
m2_obj = Model2.objects.filter(m2_field_name=m1_obj.m1_fiels_name).first() # new
# we check the availability of the object to avoid an error
if m2_obj: # new
m2_obj.m2_field = m1_obj.m1_field # new
m2_obj.save(update_fields=['m2_field']) # new
Change model pk from an integer id to uuid
At the beginning of the project, we can define the uuid field and set it as a pk, also we can add this field, define as a pk and populate it any time we want, but only if we don't have models related to this. So, what do we have to do in the case of ongoing project?
Changing default integer ids to uuid without losing relations is not as easy as we would want it to be. First of all, we have to know that if we change field id with new data it creates a new object instead of populating the old object's field, but other fields will be copied.
First of all, we have to define the id field in our model.
id = models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name='ID'
)
We can't just change the field type to UUIDField and populate it with uuid because of different types of data. But we can change it to CharField, so we do it.
id = models.CharField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name='ID',
max_length=36
)
And now it`s possible to fill it with uuid. We create a function like in the previous example, getting a model and moving through its objects.
def fill_uuid(apps, schema_editor):
Model = apps.get_model('appname', 'Model')
for obj in Model.objects.all():
old_obj_id = obj.id
# it creates new object with the same data as a previous one
# but id field is now a uuid string
obj.id = uuid.uuid4()
obj.save()
# So, if we do not need to store objects with old integer ids we have to delete them.
Model.objects.filter(id=old_obj_id).first().delete()
Here we store old object's id old_obj_id = obj.id to delete it later. Then, assign uuid4 from the built-in uuid package to the object's id field obj.id = uuid.uuid4(). The .save() method must be called. And now we can filter and delete old object Model.objects.filter(id=old_obj_id).first().delete().
Change model pk to uuid with saving relations for the related model
When we have to modify a model which has a related model we must save its relations and override ForeignKey field to a new object.
models_dict = { # new
'appname': # new
{'Model2' : 'fk_field_name'}, # new
} # new
def fill_uuid(apps, schema_editor):
Model1 = apps.get_model('appname', 'Model')
for obj in Model1.objects.all():
old_obj_id = obj.id
# it creates new object as in the previous example
obj.id = uuid.uuid4()
obj.save()
# So, if we do not need to store objects with old integer ids we have to delete them.
Model1.objects.filter(id=old_obj_id).first().delete()
for app, subdict in models_dict.items(): # new
for model, field in subdict.items(): # new
Model = apps.get_model(app, model) # new
model_obj = Model.objects.filter(**{field: old_obj_id}).first() # new
setattr(model_obj, field, obj) # new
model_obj.save(update_fields=[field]) # new
In the models_dictvariable we store all models in any app with specific field names that store a relation to our target model and rewrite them at one time.
But it won`t work if the related model's field has parameter on_delete=CASCADE because in this case, we delete the object and the related object at the same time. So, we need to perform delete at the very end.
models_dict = {
'appname':
{'Model2' : 'fk_field_name'},
}
def fill_uuid(apps, schema_editor):
Model1 = apps.get_model('appname', 'Model')
for obj in Model1.objects.all():
old_obj_id = obj.id
# it creates new object
obj.id = uuid.uuid4()
obj.save()
for app, subdict in models_dict.items():
for model, field in subdict.items():
Model = apps.get_model(app, model)
model_obj = Model.objects.filter(**{field: old_obj_id}).first()
setattr(model_obj, field, obj)
model_obj.save(update_fields=[field])
Model1.objects.filter(id=old_obj_id).first().delete() # new
Change model pk to uuid for a model with ManyToMany field
As we know Django ManyToManyField creates a third table, so we cant rewrite ForeignKey fields in related objects like in the previous examples. For this purpose, we need to change the relations in that third table. The most frequent case is the m2m to the User model, so we will use it in our example.
So, first of all, we define a new model with two ForeignKey related to our models.
class Model1User(models.Model):
model1 = models.ForeignKey(Model1, on_delete=models.CASCADE,)
user = models.ForeignKey(User, on_delete=models.CASCADE,)
class Meta:
db_table = 'appname_modle1_m2mfieldname'
and add parameter through='appname.Model1User' to the ManyToManyField field. This will define a 'through' table explicitly in our code so we can manipulate it.
Then create a migration file like this:
from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('appname', '0002_auto_20220819_1111'),
]
operations = [
migrations.SeparateDatabaseAndState(state_operations=[
migrations.CreateModel(
name='Model1User',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('model1', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='leads.lead')),
('user', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to=settings.AUTH_USER_MODEL)),
],
options={
'db_table': 'appname_modle1_m2mfieldname',
},
),
migrations.AlterField(
model_name='model1',
name='assigned_to',
field=models.ManyToManyField(related_name='model1_relations', through='appname.Model1User', to=settings.AUTH_USER_MODEL),
),
])
]
Pay attention that after manage.py makemigrations we need to cover our migration options in migrations.SeparateDatabaseAndState(state_operations=[]) before migrate.
After that, we can define and change the type id field in our Model1 like in step 3 and create our next migration where we fill id with uuid and rewrite dependencies in the third model to the new objects.
def fill_uuid(apps, schema_editor):
Model1 = apps.get_model('appname', 'Model1')
Model1User = apps.get_model('appname', 'Model1User')
for obj in Model1.objects.all():
model1user = Model1User.objects.filter(model1=obj.id).first()
old_obj_id = obj.id
obj.id = uuid.uuid4()
obj.save()
model1user.model1_id = obj.id
model1user.save(update_fields=['model1'])
Model1.objects.filter(id=old_obj_id).delete()
Change model pk to uuid for a model with OneToOneField field
If we want to migrate the model with OneToOneField relations we can appear with an error, because, as we remember, rewriting the id field creates a new object, and there can not be two objects related to the same object through OneToOneField.
For this purpose, we have to store related object in a variable, and then, when we create a new object with uuid and put the object into the proper field.
def fill_uuid(apps, schema_editor):
Model1 = apps.get_model('appname', 'Model1')
for obj in Model1.objects.all():
# store related object in a variable
o2o_obj = obj.o2o_field
# create a dictionary from the old object
new_instance = model_to_dict(obj)
obj.delete()
# change id from int to uuid
new_instance['id'] = uuid.uuid4()
# assign a value to the field
new_instance['o2o_field'] = o2o_obj
Model1.objects.create(**new_instance)
Finally, after all these manipulations, we can change the field`s type to UUIDField like this:
id = models.UUIDField(
auto_created=True,
primary_key=True,
editable=False,
default=uuid.uuid4,
verbose_name='ID'
)
As we can see, with the right approach and skill in working with querysets, almost any problem related to databases can be solved. Such custom migrations can make the process of populating tables easier or solve imperfections in the initial design of models.
