Room Migration

Introduction

Hello, I'm Hasegawa from KINTO Technologies. I usually work as an Android engineer, developing an application called "my route by KINTO." In this article, I will talk about my experiences with database migration while developing the Android version of my route by KINTO.

Overview

Room is an official library in Android that facilitates easy local data persistence. Storing data on a device has significant advantages from a user's perspective, including the ability to use apps offline. On the other hand, from a developer's perspective, there are a few tasks that need to be done. One of them is migration. Although Room officially supports automated database migration, there are cases where updates involving complex database changes need to be handled manually. This article will cover simple automated migration to complex manual migration, along with several use cases.

What Happens If a Migration Is Not Done Correctly?

Have you ever thought about what happens if you don't migrate data correctly? There are two main patterns, determined by the level of support within apps.

App crashes
Data is lost

You may have experienced apps crashing if you use Room. The following errors occur depending on the case:

When the database version has been updated, but the appropriate migration path has not been provided

A migration from 1 to 2 was required but not found. Please provide the necessary Migration path

When the schema has been updated, but the database version has not been updated

Room cannot verify the data integrity. Looks like you've changed schema but forgot to update the version number.

When manual migration is not working properly

Migration didn't properly handle: FooEntity().

Basically, all of these can occur in the development environment, so I don’t think it will be that much of a problem. However, it should be noted that if the fallback~ described below is used to cover up migration failures, it may be very difficult to notice, and in some cases it may occur only in the production environment.

What about "data loss"? Well, Room can call fallbackToDestructiveMigration() when you create the database object. This is a function that permanently deletes data if migration fails and allows apps to start normally. I'm not sure if this is to address the errors mentioned above or to avoid the time-consuming process of database migration, but I have seen it used occasionally. If this is done, data loss will occur in the event of a migration failure which is difficult to detect. Therefore, it is best to strive for successful migrations.

Four Migration Scenarios

Here are four examples of schema updates that may occur in the course of app development.

1. New Table Addition

Since adding a new table does not affect existing data, it can be automatically migrated. For example, if you have an entity named FooClass in DB version 1 and you add an entity named BarClass in DB version 2, you can simply pass autoMigrations with AutoMigration(from = 1, to = 2) as follows.

@Database(
    entities = [
        HogeClass::class,
        HugaClass::class, // Add
    ],
    version = 2, // 1 -> 2
    autoMigrations = [
        AutoMigration(from = 1, to = 2)
    ]
)
abstract class AppDatabase : RoomDatabase() {}

2. Delete or Rename Tables, Delete or Rename Columns

Automated migration is possible for deletion and renaming, but you need to define AutoMigrationSpec. As an example of a column name change that is most likely to occur, suppose a column name of the entity User is changed to firstName.

@Entity
data class User(
    @PrimaryKey
    val id: Int,
    // val name: String, // old
    val firstName: String,  // new
    val age: Int,
)

First, define a class that implemented AutoMigrationSpec. Then, annotate it with @RenameColumn and give the necessary information for the column to be changed as an argument. Pass the created class to the corresponding version of AutoMigration and pass it to autoMigrations.

@RenameColumn(
    tableName = "User",
    fromColumnName = "name",
    toColumnName = "firstName"
)
class NameToFirstnameAutoMigrationSpec : AutoMigrationSpec

@Database(
    entities = [
        User::class,
        Person::class
    ],
    version = 2,
    autoMigrations = [
        AutoMigration(from = 1, to = 2, NameToFirstnameAutoMigrationSpec::class),
    ]
)
abstract class AppDatabase : RoomDatabase() {}

Room provides additional annotations, including @DeleteTable, @RenameTable, and @DeleteColumn, which facilitate the easy handling of deletions and name changes.

3. Add a Column

Personally, I think the addition of column is most likely to occur. Let's say that for the entity User, a column for height height is added.

@Entity
data class User(
    @PrimaryKey
    val id: Int,
    val name: String,
    val age: Int,
    val height: Int, // new
)

Adding columns requires manual migration. The reason is to tell Room the default value for height. Simply create an object that inherits from migraton as follows and pass it to addMigration() when creating the database object. Write the required SQL statements in database.execSQL.

val MIGRATION_1_2 = object : Migration(1, 2) {
    override fun migrate(database: SupportSQLiteDatabase) {
        database.execSQL(
            "ALTER TABLE User ADD COLUMN height Integer NOT NULL DEFAULT 0"
        )
    }
}

val db = Room.databaseBuilder(
  context,
  AppDatabase::class.java, "database-name"
)
  .addMigrations(MIGRATION_1_2)
  .build()

4. Add a Primary Key

In my app experience, there were cases where a primary key was added. This is the case when the primary key that was assumed when the table was created is not sufficient to maintain uniqueness, and other columns are added to the primary key. For example, suppose that in the User table, id was the primary key until now, but name is also the primary key and becomes a composite primary key.

// DB version 1
@Entity
data class User(
    @PrimaryKey
    val id: Int,
    val name: String,
    val age: Int,
)

// DB version 2
@Entity(
    primaryKeys = ["id", "name"]
)
data class User(
    val id: Int,
    val name: String,
    val age: Int,
)

In this case, not limited to Android, the common method is to create a new table. The following SQL statement creates a table named UserNew with a new primary key condition and copies the information from the User table. Then delete the existing User table and rename the UserNew table to User.

val migration_1_2 = object : Migration(1, 2) {
    override fun migrate(database: SupportSQLiteDatabase) {
        database.execSQL("CREATE TABLE IF NOT EXISTS UserNew (`id` Integer NOT NULL, `name` TEXT NOT NULL, `age` Integer NOT NULL, PRIMARY KEY(`id`, `name`))")
        database.execSQL("INSERT INTO UserNew (`id`, `name`, `age`) SELECT `id`, `name`, `age` FROM User")
        database.execSQL("DROP TABLE User")
        database.execSQL("ALTER TABLE UserNew RENAME TO User")
    }
}

Let's Check If The Migration Works Correctly!

There are many more complex cases in addition to the migration examples above. Even in the app I am involved in, there have been changes to tables where foreign keys are involved. In such a case, the only way is to write SQL statements, but you want to make sure that the SQL is really working correctly. For this purpose, Room provides a way to test migrations.

The following test code can be used to test whether migration is working properly. In order to test, the schema for each database version needs to be exported beforehand. See Export schemas for more information. Even if you did not export the schema of the old database version, it is recommended that you identify the past version from git tags, etc. and export the schema.

The point is to refer to the same values as the migration to be performed in the production code and the test code, as in the variable defined in the list manualMigrations. This way, even if you added migration5_6 in the production code, you can rest assured that the test code will automatically verify it.


// production code
val manualMigrations = listOf(
    migration1To2,
    migration2To3,
    // 3->4automated migration
    migration4To5,
)

// test code
@RunWith(AndroidJUnit4::class)
class MigrationTest {
    private val TEST_DB = "migration-test"

    @get:Rule
    val helper: MigrationTestHelper = MigrationTestHelper(
        InstrumentationRegistry.getInstrumentation(),
        AppDatabase::class.java,
    )

    @Test
    @Throws(IOException::class)
    fun migrateAll() {
        helper.createDatabase(TEST_DB, 1).apply {
            close()
        }

        Room.databaseBuilder(
            InstrumentationRegistry.getInstrumentation().targetContext,
            AppDatabase::class.java,
            TEST_DB
        ).addMigrations(*ALL_MIGRATIONS).build().apply {
            openHelper.writableDatabase.close()
        }

    }
}

Summary

Today I talked a bit about Room migration with a few use cases. I'd like to avoid manual migrations as much as possible, but I believe the key to achieving that is ensuring the entire team is involved in table design. Also, remember to export the schema for each database version. Otherwise, it would be a bit difficult for future developers to go back, export the schema using git, etc., and verify it.
Thank you for reading.