migration-helper

Convex Migration Helper

Safely migrate Convex schemas and data when making breaking changes.

When to Use

Adding new required fields to existing tables
Changing field types or structure
Splitting or merging tables
Renaming fields
Migrating from nested to relational data

Migration Principles

No Automatic Migrations: Convex doesn't automatically migrate data
Additive Changes are Safe: Adding optional fields or new tables is safe
Breaking Changes Need Code: Required fields, type changes need migration code
Zero-Downtime: Write migrations to keep app running during migration

Safe Changes (No Migration Needed)

Adding Optional Field

// Before users: defineTable({ name: v.string(), })

// After - Safe! New field is optional users: defineTable({ name: v.string(), bio: v.optional(v.string()), })

Adding New Table

// Safe to add completely new tables posts: defineTable({ userId: v.id("users"), title: v.string(), }).index("by_user", ["userId"])

Adding Index

// Safe to add indexes at any time users: defineTable({ name: v.string(), email: v.string(), }) .index("by_email", ["email"]) // New index

Breaking Changes (Migration Required)

Adding Required Field

Problem: Existing documents won't have the new field.

Solution: Add as optional first, backfill data, then make required.

// Step 1: Add as optional users: defineTable({ name: v.string(), email: v.optional(v.string()), // Start optional })

// Step 2: Create migration import { internalMutation } from "./_generated/server"; import { v } from "convex/values";

export const backfillEmails = internalMutation({ args: {}, handler: async (ctx) => { const users = await ctx.db.query("users").collect();

for (const user of users) {
  if (!user.email) {
    await ctx.db.patch(user._id, {
      email: `user-${user._id}@example.com`, // Default value
    });
  }
}

}, });

// Step 3: Run migration via dashboard or CLI // npx convex run migrations:backfillEmails

// Step 4: Make field required (after all data migrated) users: defineTable({ name: v.string(), email: v.string(), // Now required })

Changing Field Type

Example: Change tags: v.array(v.string()) to separate table

// Step 1: Create new structure (additive) tags: defineTable({ name: v.string(), }).index("by_name", ["name"]),

postTags: defineTable({ postId: v.id("posts"), tagId: v.id("tags"), }) .index("by_post", ["postId"]) .index("by_tag", ["tagId"]),

// Keep old field as optional during migration posts: defineTable({ title: v.string(), tags: v.optional(v.array(v.string())), // Keep temporarily })

// Step 2: Write migration export const migrateTags = internalMutation({ args: { batchSize: v.optional(v.number()) }, handler: async (ctx, args) => { const batchSize = args.batchSize ?? 100;

const posts = await ctx.db
  .query("posts")
  .filter(q => q.neq(q.field("tags"), undefined))
  .take(batchSize);

for (const post of posts) {
  if (!post.tags || post.tags.length === 0) {
    await ctx.db.patch(post._id, { tags: undefined });
    continue;
  }

  // Create tags and relationships
  for (const tagName of post.tags) {
    // Get or create tag
    let tag = await ctx.db
      .query("tags")
      .withIndex("by_name", q => q.eq("name", tagName))
      .unique();

    if (!tag) {
      const tagId = await ctx.db.insert("tags", { name: tagName });
      tag = { _id: tagId, name: tagName };
    }

    // Create relationship
    const existing = await ctx.db
      .query("postTags")
      .withIndex("by_post", q => q.eq("postId", post._id))
      .filter(q => q.eq(q.field("tagId"), tag._id))
      .unique();

    if (!existing) {
      await ctx.db.insert("postTags", {
        postId: post._id,
        tagId: tag._id,
      });
    }
  }

  // Remove old field
  await ctx.db.patch(post._id, { tags: undefined });
}

return { migrated: posts.length };

}, });

// Step 3: Run in batches via cron or manually // Run multiple times until all migrated

// Step 4: Remove old field from schema posts: defineTable({ title: v.string(), // tags field removed })

Renaming Field

// Step 1: Add new field (optional) users: defineTable({ name: v.string(), displayName: v.optional(v.string()), // New name })

// Step 2: Copy data export const renameField = internalMutation({ handler: async (ctx) => { const users = await ctx.db.query("users").collect();

for (const user of users) {
  await ctx.db.patch(user._id, {
    displayName: user.name,
  });
}

}, });

// Step 3: Update schema (remove old field) users: defineTable({ displayName: v.string(), })

// Step 4: Update all code to use new field name

Migration Patterns

Batch Processing

For large tables, process in batches:

export const migrateBatch = internalMutation({ args: { cursor: v.optional(v.string()), batchSize: v.number(), }, handler: async (ctx, args) => { const batchSize = args.batchSize; let query = ctx.db.query("largeTable");

// Use cursor for pagination if needed
const items = await query.take(batchSize);

for (const item of items) {
  await ctx.db.patch(item._id, {
    // migration logic
  });
}

return {
  processed: items.length,
  hasMore: items.length === batchSize,
};

}, });

Scheduled Migration

Use cron jobs for gradual migration:

// convex/crons.ts import { cronJobs } from "convex/server"; import { internal } from "./_generated/api";

const crons = cronJobs();

crons.interval( "migrate-batch", { minutes: 5 }, // Every 5 minutes internal.migrations.migrateBatch, { batchSize: 100 } );

export default crons;

Dual-Write Pattern

For zero-downtime migrations:

// Write to both old and new structure during transition export const createPost = mutation({ args: { title: v.string(), tags: v.array(v.string()) }, handler: async (ctx, args) => { const user = await getCurrentUser(ctx);

// Create post
const postId = await ctx.db.insert("posts", {
  userId: user._id,
  title: args.title,
  // Keep writing old field during migration
  tags: args.tags,
});

// ALSO write to new structure
for (const tagName of args.tags) {
  let tag = await ctx.db
    .query("tags")
    .withIndex("by_name", q => q.eq("name", tagName))
    .unique();

  if (!tag) {
    const tagId = await ctx.db.insert("tags", { name: tagName });
    tag = { _id: tagId };
  }

  await ctx.db.insert("postTags", {
    postId,
    tagId: tag._id,
  });
}

return postId;

}, });

// After migration complete, remove old writes

Testing Migrations

Verify Migration Success

export const verifyMigration = query({ args: {}, handler: async (ctx) => { const total = (await ctx.db.query("users").collect()).length; const migrated = (await ctx.db .query("users") .filter(q => q.neq(q.field("newField"), undefined)) .collect() ).length;

return {
  total,
  migrated,
  remaining: total - migrated,
  percentComplete: (migrated / total) * 100,
};

}, });

Migration Checklist

Identify breaking change
Add new structure as optional/additive
Write migration function (internal mutation)
Test migration on sample data
Run migration in batches if large dataset
Verify migration completed (all records updated)
Update application code to use new structure
Deploy new code
Remove old fields from schema
Clean up migration code

Common Pitfalls

Don't make field required immediately: Always add as optional first
Don't migrate in a single transaction: Batch large migrations
Don't forget to update queries: Update all code using old field
Don't delete old field too soon: Wait until all data migrated
Test thoroughly: Verify migration on dev environment first

Example: Complete Migration Flow

// 1. Current schema export default defineSchema({ users: defineTable({ name: v.string(), }), });

// 2. Add optional field export default defineSchema({ users: defineTable({ name: v.string(), role: v.optional(v.union( v.literal("user"), v.literal("admin") )), }), });

// 3. Migration function export const addDefaultRoles = internalMutation({ handler: async (ctx) => { const users = await ctx.db.query("users").collect(); for (const user of users) { if (!user.role) { await ctx.db.patch(user._id, { role: "user" }); } } }, });

// 4. Run migration: npx convex run migrations:addDefaultRoles

// 5. Verify: Check all users have role

// 6. Make required export default defineSchema({ users: defineTable({ name: v.string(), role: v.union( v.literal("user"), v.literal("admin") ), }), });

migration-helper

Safety Notice

Copy this and send it to your AI assistant to learn

Source Transparency

Related Skills

convex-helpers-guide

function-creator

components-guide

schema-builder