ReportBatchItemFailures: SQS partial batch failures in Lambda

Without a partial batch response, a single failed record makes Lambda return the entire SQS batch to the queue: nine successfully processed messages get redelivered along with the one that actually failed. The fix has two mandatory halves: set FunctionResponseTypes to ReportBatchItemFailures on the event source mapping, and return { batchItemFailures } from the handler listing only the records that failed. The response format fails closed, so its edge cases deserve as much attention as the happy path.

What happens by default when one record in a batch fails?

By default, the event source mapping treats the batch as a single unit: if the handler throws for any record, Lambda considers the whole invocation failed and leaves every message on the queue, including the ones already processed successfully. After the visibility timeout, all of them come back and your side effects run twice.

That is how duplicate emails, duplicate charges and duplicate rows are born. It also poisons the dead-letter queue: healthy messages accumulate receive counts every time they ride along with a poison record, and eventually cross maxReceiveCount for a failure that was never theirs. If you are still deciding whether a queue belongs in the design at all, start with our article on when SQS is the right tool; the rest of this one assumes the queue is already in place.

How does a partial batch response change deletion?

With ReportBatchItemFailures active, Lambda deletes every message of the batch except those listed in the response. Successful records leave the queue for good; only the reported identifiers become visible again after the visibility timeout, each with its own receive count incremented. The unit of failure shrinks from the batch to the record.

The response is a contract between your code and the event source mapping, documented in the Lambda guide on SQS error handling. Your handler never calls DeleteMessage itself: the mapping deletes on your behalf based on what you return. Report nothing and everything is deleted; report two identifiers and eight messages are deleted while two come back for retry.

How do you enable ReportBatchItemFailures?

Two changes, both required. First, declare ReportBatchItemFailures in the FunctionResponseTypes of the event source mapping. Second, return an object shaped { batchItemFailures: [{ itemIdentifier: messageId }] } from the handler. The setting without the return shape changes nothing; the return shape without the setting is silently ignored.

// CDK: enable partial batch responses on the mapping
import { SqsEventSource } from "aws-cdk-lib/aws-lambda-event-sources";

handler.addEventSource(
  new SqsEventSource(ordersQueue, {
    batchSize: 10,
    reportBatchItemFailures: true, // sets FunctionResponseTypes on the mapping
  })
);

In SAM or CloudFormation, the equivalent is FunctionResponseTypes: [ReportBatchItemFailures] on the event source resource. Whatever the tool, audit every SQS-triggered function: in our experience, this flag is the single most commonly missing line in otherwise solid serverless codebases.

What does a correct handler look like?

Wrap each record in its own try/catch and collect the messageId of failures. Never let an exception escape the handler: an uncaught error still fails the entire batch, exactly the behavior you are trying to escape. An empty batchItemFailures array means complete success, so returning it unconditionally at the end is correct.

import type { SQSHandler, SQSBatchItemFailure } from "aws-lambda";

export const handler: SQSHandler = async (event) => {
  const batchItemFailures: SQSBatchItemFailure[] = [];

  for (const record of event.Records) {
    try {
      await processOrder(JSON.parse(record.body));
    } catch (err) {
      // Log the messageId: it is the only key you get to trace the retry
      console.error("record failed", record.messageId, err);
      batchItemFailures.push({ itemIdentifier: record.messageId });
    }
  }

  // Empty array means full success: every message gets deleted
  return { batchItemFailures };
};

The loop stays sequential for clarity; with Promise.allSettled you can parallelize, as long as every rejection maps back to its messageId.

Which mistakes make the whole batch retry anyway?

The contract fails closed. A malformed response, an itemIdentifier matching no messageId in the batch, an empty string identifier, or an exception escaping the handler: each counts as a total failure, and Lambda returns every message to the queue. Failing closed is the right default, because the alternative would silently delete unprocessed messages.

The practical consequence: test the malformed path on purpose. One unit test should feed the handler a failing record and assert the exact JSON shape of the response; another should confirm that a typo like itemIdentifer gets caught by your types or your assertions. This contract is too often discovered in production, while investigating why retries multiplied instead of shrinking.

Why is idempotency still required?

Partial batch responses reduce duplicates, they do not eliminate them. Standard SQS delivery stays at-least-once: a function timeout after a side effect completed, a visibility timeout expiring mid-run, or a crash before the response is returned will all redeliver messages that were in fact processed.

Every side effect therefore needs an idempotency key, checked before the write. We covered the consumer-side deduplication reasoning in our EventBridge to SQS to Lambda article, and it applies unchanged here: the queue guarantees delivery, the consumer guarantees exactly-once effects. ReportBatchItemFailures narrows the duplication window; idempotency closes it.

What changes with FIFO queues?

On a FIFO queue, ordering within a message group must survive the retry. When a record fails, you must report it and every subsequent record of the batch as failed, even those that would have succeeded: deleting message four while message three returns to the queue would reorder the group.

for (const [i, record] of event.Records.entries()) {
  try {
    await process(record);
  } catch {
    // FIFO: fail this record and all the following ones to preserve order
    return {
      batchItemFailures: event.Records.slice(i).map((r) => ({
        itemIdentifier: r.messageId,
      })),
    };
  }
}
return { batchItemFailures: [] };

Records processed before the failure are deleted normally; the tail of the batch retries in order. This is also why we keep FIFO batch sizes small: the longer the tail, the more work a single failure discards.

How does this interact with maxReceiveCount and the DLQ?

Receive counts are tracked per message, so with partial batch responses only genuinely failing messages march toward maxReceiveCount and the dead-letter queue. The DLQ finally means what it should: messages that failed on their own merits, not bystanders swept up with a poison record.

Size maxReceiveCount to absorb a transient downstream outage (we rarely configure fewer than five attempts) and alarm on DLQ depth. For observability, emit one metric per invocation: the ratio of batchItemFailures length to batch size. A partial failure ratio climbing from near zero flags a degrading dependency long before the DLQ fills, and it distinguishes a single poison message (low, flat ratio) from a systemic outage (ratio near one).

Rollout checklist

• FunctionResponseTypes contains ReportBatchItemFailures on every SQS event source mapping.
• Per-record try/catch: no exception can escape the handler.
• Unit tests cover the malformed response and unknown identifier paths (expect a full batch retry).
• Every side effect carries an idempotency key.
• FIFO handlers fail every record from the first failure onward.
• maxReceiveCount sized for transient outages, alarm on DLQ depth.
• Partial failure ratio emitted as a metric and visible on a dashboard.