Making LLM Extraction Retries Idempotent in Node.js for Supplier Invoices

Why idempotency matters for LLM‑driven invoice extraction

When you hook a large language model into a Node.js webhook to pull line items from supplier PDFs, the first thing that goes wrong is usually a network hiccup or a timeout. If you simply retry the whole request you end up with duplicate Invoice rows, extra billing, and a mess of audit logs. The fix is to separate the attempt from the outcome and give every logical extraction a tenant‑scoped operationId. That key lives in the database row that represents the extraction job, not in the LLM response.

Designing a single source of truth for attempts

Think of the extraction pipeline as three independent services:

• Webhook that receives the PDF URL and creates an operation record.
• Worker (e.g., BullMQ) that calls the LLM and stores the raw JSON.
• Post‑processor that normalizes the JSON into InvoiceLine rows.

Each service writes to the same operations table, updating a status column (pending, success, failed) and appending a new entry to an attempts JSON array. Because the operationId is unique per tenant, any retry simply adds another attempt object, leaving the original Invoice rows untouched until a success status is recorded.

Implementing the idempotent workflow in code

interface Operation {
  id: string; // tenant‑scoped UUID
  tenantId: string;
  pdfUrl: string;
  status: 'pending' | 'success' | 'failed';
  attempts: Attempt[];
  result?: ExtractedInvoice;
}

interface Attempt {
  timestamp: Date;
  llmResponse?: any;
  error?: string;
}

// webhook creates the operation
export async function receivePdf(req: Request, res: Response) {
  const { tenantId, pdfUrl } = req.body;
  const op = await db.operation.create({
    data: {
      id: generateScopedId(tenantId),
      tenantId,
      pdfUrl,
      status: 'pending',
      attempts: [],
    },
  });
  enqueueExtraction(op.id);
  res.json({ operationId: op.id });
}

// worker picks up the job and records an attempt
export async function runExtraction(opId: string) {
  const op = await db.operation.findUnique({ where: { id: opId } });
  const attempt: Attempt = { timestamp: new Date() };
  try {
    const raw = await callLlm(op.pdfUrl);
    attempt.llmResponse = raw;
    const invoice = parseInvoice(raw);
    await db.operation.update({
      where: { id: opId },
      data: {
        status: 'success',
        result: invoice,
        attempts: { push: attempt },
      },
    });
  } catch (e) {
    attempt.error = e instanceof Error ? e.message : String(e);
    await db.operation.update({
      where: { id: opId },
      data: { attempts: { push: attempt } },
    });
    // re‑queue if we want another try
    if (shouldRetry(attempt)) enqueueExtraction(opId);
  }
}

The key is the push on the attempts array – every retry is logged, but the result field is only written once when the status flips to success. Subsequent retries see the existing success and bail out early.

Observability: turning attempts into metrics

Because each attempt is a first‑class object, you can feed it to your logging pipeline without extra instrumentation. A simple query gives you per‑tenant success rates, average attempts per invoice, and failure reasons:

SELECT tenantId,
       COUNT(*) FILTER (WHERE status='success') AS succeeded,
       AVG(jsonb_array_length(attempts)) AS avgAttempts,
       jsonb_agg(DISTINCT attempts->>'error') AS errorTypes
FROM operations
GROUP BY tenantId;

These numbers are invaluable when you need to negotiate a per‑tenant LLM budget or decide whether to switch models.

Trade‑offs and when not to over‑engineer

Idempotent retries add a tiny amount of storage overhead (the attempts array) and a few extra DB writes. For low‑to‑moderate volume—say under a few thousand invoices a day—the cost is negligible. If you’re processing millions of PDFs, you might move attempts to a dedicated event store (Kafka, DynamoDB Streams) and keep the operations row lean.

Another pitfall is treating the LLM as a black box. Some providers already return an idempotency token; if you can pass that through, you can skip the DB‑level duplicate guard. But most public APIs don’t expose it, so the tenant‑scoped key remains the safest fallback.

Putting it into production

1. Add a unique index on (tenantId, id) to guarantee the key can’t collide.
2. Wrap the DB update in a transaction so the attempt log and status flip happen atomically.
3. Use exponential back‑off for retries and cap the max attempts to avoid runaway loops.
4. Emit a structured log line at the end of each attempt; tools like Vercel AI SDK can enrich it with request IDs for end‑to‑end tracing.

With these pieces in place you get a pipeline that is safe to retry, fully observable, and cheap to audit. The next time a supplier PDF fails because of a transient 502, you’ll know exactly how many times you tried, why it failed, and you’ll never create a duplicate invoice record again.