FireCrawl Migration Deep Dive

Overview

Comprehensive guide for migrating to or from FireCrawl, or major version upgrades.

Prerequisites

• Current system documentation

• FireCrawl SDK installed

• Feature flag infrastructure

• Rollback strategy tested

Migration Types

Type Complexity Duration Risk

Fresh install Low Days Low

From competitor Medium Weeks Medium

Major version Medium Weeks Medium

Full replatform High Months High

Pre-Migration Assessment

Step 1: Current State Analysis

set -euo pipefail

Document current implementation

find . -name ".ts" -o -name ".py" | xargs grep -l "firecrawl" > firecrawl-files.txt

Count integration points

wc -l firecrawl-files.txt

Identify dependencies

npm list | grep firecrawl pip freeze | grep firecrawl

Step 2: Data Inventory

interface MigrationInventory { dataTypes: string[]; recordCounts: Record<string, number>; dependencies: string[]; integrationPoints: string[]; customizations: string[]; }

async function assessFireCrawlMigration(): Promise<MigrationInventory> { return { dataTypes: await getDataTypes(), recordCounts: await getRecordCounts(), dependencies: await analyzeDependencies(), integrationPoints: await findIntegrationPoints(), customizations: await documentCustomizations(), }; }

Migration Strategy: Strangler Fig Pattern

Phase 1: Parallel Run ┌─────────────┐ ┌─────────────┐ │ Old │ │ New │ │ System │ ──▶ │ FireCrawl │ │ (100%) │ │ (0%) │ └─────────────┘ └─────────────┘

Phase 2: Gradual Shift ┌─────────────┐ ┌─────────────┐ │ Old │ │ New │ │ (50%) │ ──▶ │ (50%) │ └─────────────┘ └─────────────┘

Phase 3: Complete ┌─────────────┐ ┌─────────────┐ │ Old │ │ New │ │ (0%) │ ──▶ │ (100%) │ └─────────────┘ └─────────────┘

Implementation Plan

Phase 1: Setup (Week 1-2)

set -euo pipefail

Install FireCrawl SDK

npm install @firecrawl/sdk

Configure credentials

cp .env.example .env.firecrawl

Edit with new credentials

Verify connectivity

node -e "require('@firecrawl/sdk').ping()"

Phase 2: Adapter Layer (Week 3-4)

// src/adapters/firecrawl.ts interface ServiceAdapter { create(data: CreateInput): Promise<Resource>; read(id: string): Promise<Resource>; update(id: string, data: UpdateInput): Promise<Resource>; delete(id: string): Promise<void>; }

class FireCrawlAdapter implements ServiceAdapter { async create(data: CreateInput): Promise<Resource> { const firecrawlData = this.transform(data); return firecrawlClient.create(firecrawlData); }

private transform(data: CreateInput): FireCrawlInput { // Map from old format to FireCrawl format } }

Phase 3: Data Migration (Week 5-6)

async function migrateFireCrawlData(): Promise<MigrationResult> { const batchSize = 100; let processed = 0; let errors: MigrationError[] = [];

for await (const batch of oldSystem.iterateBatches(batchSize)) { try { const transformed = batch.map(transform); await firecrawlClient.batchCreate(transformed); processed += batch.length; } catch (error) { errors.push({ batch, error }); }

// Progress update
console.log(`Migrated ${processed} records`);

}

return { processed, errors }; }

Phase 4: Traffic Shift (Week 7-8)

// Feature flag controlled traffic split function getServiceAdapter(): ServiceAdapter { const firecrawlPercentage = getFeatureFlag('firecrawl_migration_percentage');

if (Math.random() * 100 < firecrawlPercentage) { return new FireCrawlAdapter(); }

return new LegacyAdapter(); }

Rollback Plan

set -euo pipefail

Immediate rollback

kubectl set env deployment/app FIRECRAWL_ENABLED=false kubectl rollout restart deployment/app

Data rollback (if needed)

./scripts/restore-from-backup.sh --date YYYY-MM-DD

Verify rollback

curl https://app.yourcompany.com/health | jq '.services.firecrawl'

Post-Migration Validation

async function validateFireCrawlMigration(): Promise<ValidationReport> { const checks = [ { name: 'Data count match', fn: checkDataCounts }, { name: 'API functionality', fn: checkApiFunctionality }, { name: 'Performance baseline', fn: checkPerformance }, { name: 'Error rates', fn: checkErrorRates }, ];

const results = await Promise.all( checks.map(async c => ({ name: c.name, result: await c.fn() })) );

return { checks: results, passed: results.every(r => r.result.success) }; }

Instructions

Assess current configuration

Document existing implementation and data inventory.

Step 2: Build Adapter Layer

Create abstraction layer for gradual migration.

Step 3: Migrate Data

Run batch data migration with error handling.

Step 4: Shift Traffic

Gradually route traffic to new FireCrawl integration.

Output

• Migration assessment complete

• Adapter layer implemented

• Data migrated successfully

• Traffic fully shifted to FireCrawl

Error Handling

Issue Cause Solution

Data mismatch Transform errors Validate transform logic

Performance drop No caching Add caching layer

Rollback triggered Errors spiked Reduce traffic percentage

Validation failed Missing data Check batch processing

Examples

Quick Migration Status

const status = await validateFireCrawlMigration(); console.log(Migration ${status.passed ? 'PASSED' : 'FAILED'}); status.checks.forEach(c => console.log( ${c.name}: ${c.result.success}));

Resources

• Strangler Fig Pattern

• FireCrawl Migration Guide

Flagship+ Skills

For advanced troubleshooting, see firecrawl-advanced-troubleshooting .