Workflow Management Skill

This skill helps you work with QStash-based workflows in apps/api/src/lib/workflows/ .

When to Use This Skill

• Adding new scheduled workflows for data fetching

• Debugging workflow execution errors

• Modifying existing workflow schedules or logic

• Integrating new data sources into the update pipeline

• Adding new social media posting workflows

Workflow Architecture

The project uses QStash workflows with the following structure:

apps/api/src/lib/workflows/ ├── cars/ # Car registration data workflows │ └── update.ts # Scheduled car data updates ├── coe/ # COE bidding data workflows │ └── update.ts # Scheduled COE data updates └── social/ # Social media posting workflows ├── discord.ts ├── linkedin.ts ├── telegram.ts └── twitter.ts

Key Patterns

1. Workflow Definition

Workflows are defined using QStash SDK:

import { serve } from "@upstash/workflow";

export const POST = serve(async (context) => { // Step 1: Fetch data await context.run("fetch-data", async () => { // Fetching logic });

// Step 2: Process data const processed = await context.run("process-data", async () => { // Processing logic });

// Step 3: Store results await context.run("store-results", async () => { // Storage logic }); });

2. Scheduling Workflows

Workflows are triggered via cron schedules configured in:

• SST infrastructure (infra/ )

• QStash console

• Manual API calls to workflow endpoints

3. Error Handling

Always include comprehensive error handling:

await context.run("step-name", async () => { try { // Logic here } catch (error) { console.error("Step failed:", error); // Log to monitoring service throw error; // Re-throw for workflow retry } });

Common Tasks

Adding a New Workflow

• Create workflow file in appropriate directory

• Define workflow steps using context.run()

• Add route handler in apps/api/src/routes/

• Configure scheduling (if needed)

• Add tests for workflow logic

Debugging Workflow Failures

• Check QStash dashboard for execution logs

• Review CloudWatch logs for Lambda errors

• Verify environment variables are set correctly

• Test workflow locally using development server

• Check database connectivity and Redis availability

Modifying Existing Workflows

• Read existing workflow implementation

• Identify which step needs modification

• Update step logic while maintaining error handling

• Test changes locally

• Deploy and monitor execution

Environment Variables

Workflows typically need:

• DATABASE_URL

• PostgreSQL connection

• UPSTASH_REDIS_REST_URL / UPSTASH_REDIS_REST_TOKEN

• Redis

• QSTASH_TOKEN

• QStash authentication

• Service-specific tokens (Discord webhook, Twitter API, etc.)

Testing Workflows

Run workflow tests:

pnpm -F @sgcarstrends/api test -- src/lib/workflows

Test individual workflow locally:

Start dev server

pnpm dev

Trigger workflow via HTTP

curl -X POST http://localhost:3000/api/workflows/cars/update

References

• QStash Workflows: Check Context7 for Upstash QStash documentation

• Related files:

• apps/api/src/routes/workflows.ts

• Workflow route handlers

• apps/api/src/config/qstash.ts

• QStash configuration

• apps/api/CLAUDE.md

• API service documentation

Best Practices

• Idempotency: Ensure workflows can safely retry without duplicating data

• Step Granularity: Break workflows into small, focused steps

• Logging: Add comprehensive logging for debugging

• Timeouts: Configure appropriate timeouts for long-running operations

• Testing: Write unit tests for workflow logic

• Monitoring: Track workflow execution metrics