AWS Batch and Step Functions are not competitors on the same axis: one finds machines and runs jobs, the other decides what happens next and holds the state in between. The rule is short: if the hard part is getting CPU, GPU or hours for a single job, that is AWS Batch; if the hard part is the order, the retries and the branching around jobs, that is Step Functions. Batch queues containerized work onto EC2 or Fargate and sizes the fleet for you. Step Functions coordinates steps, catches failures and passes data along. Most real pipelines end up running both, as a state machine that submits a Batch job and waits for the verdict.
Are AWS Batch and Step Functions solving the same problem?
No, and the confusion is worth clearing before comparing features. AWS Batch is a job scheduler for containers: you submit a job, it lands in a job queue, and a compute environment provisions the EC2 instances or Fargate tasks needed to run it. Nothing in Batch knows about step two. Step Functions is the mirror image: it knows everything about step two, about what to do when step two fails, and about the payload moving between steps, but it runs no business compute of its own.
One vocabulary note, because search results conflate the two: a state machine is not an alternative to Step Functions. The state machine is the resource you define, written in Amazon States Language; Step Functions is the service that stores and runs it, and an execution is one run of it. Comparing a state machine with Step Functions is comparing a score with the orchestra that plays it.
What does AWS Batch give you that a Lambda function cannot?
Four things, and each one is a reason a workload leaves serverless compute behind.
- Time. A Batch job runs as long as the work takes. Lambda stops at 15 minutes, and a nightly aggregation or a training run does not negotiate with that ceiling.
- Size. Compute environments reach instance families Lambda has no equivalent for: many vCPUs, hundreds of gigabytes of memory, GPUs for training and inference.
- Your own image. A job is a container image you build, so native dependencies, CUDA drivers, ffmpeg builds and scientific Python or R stacks travel with it instead of being squeezed into a packaging format.
- Relationships between jobs. Array jobs fan one submission into thousands of indexed children, and job dependencies let a reduce step wait for every shard of a map step, including index to index dependencies between two arrays.
Batch also brings queueing semantics an event-driven design has to hand-build: several queues with different priorities in front of shared capacity, and fair share scheduling policies so one tenant dumping ten thousand jobs into the queue does not starve everyone else until morning.
When is Step Functions enough on its own?
More often than teams expect. Work that is described as "batch" is frequently a large number of small independent items, which is exactly what Distributed Map was built for. It reads items straight from an S3 prefix, a JSON or CSV object, or an S3 inventory, then runs child executions in parallel, up to ten thousand at a time. ItemBatcher groups items so each invocation does real work, ToleratedFailurePercentage stops one poison record from failing the whole run, and a ResultWriter aggregates outcomes back to S3.
If a single item fits inside a Lambda function, under the 15 minute ceiling and within the memory a function can be given, there is no cluster to justify: no job queue, no compute environment, no container image to rebuild on every dependency bump. We cover the orchestration side of that design in our Step Functions orchestration article, and the compute sizing question in Fargate vs Lambda.
Unsure whether your pipeline needs a cluster or just a better workflow? Describe your system: a one-page diagnosis within 48 hours.
Get my diagnosis →AWS Batch vs Step Functions vs Lambda alone: the comparison
Read this table by column rather than by row. Each column answers a different question about the same pipeline, which is why the winning scenarios barely overlap.
| Criterion | AWS Batch | Step Functions | Lambda alone |
|---|---|---|---|
| Unit of work | A containerized job submitted to a queue | An execution made of states | One function invocation |
| Duration limit | None imposed by the service; the job runs to completion | Up to one year for Standard, a few minutes for Express | 15 minutes, hard |
| State | None: data moves through S3, a database or job parameters | Held by the execution, within a payload size limit per state | Only what the caller passes in and the handler returns |
| Retries | Attempts per job, with evaluation on exit code | Per state, with backoff, jitter and typed Catch routes | Decided by the event source; the handler must be idempotent |
| Compute you manage | Compute environments: instance types, Spot, scaling bounds | None | None |
| When it wins | Long, heavy, GPU or dependency-rich jobs, with queue priorities | Sequencing, branching, human waits, fan-out over many items | Short items that fit the runtime and finish fast |
How do you submit and wait on a Batch job from a state machine?
Through the synchronous service integration. The state submits the job and stays open until Batch reports success or failure, so the workflow learns the outcome without a polling loop of its own. The Batch integration carries the job status back into the state machine, where retry policies and Catch routes apply exactly as they do anywhere else.
{
"Comment": "Submit a Batch array job and wait for it",
"StartAt": "RunTransform",
"States": {
"RunTransform": {
"Type": "Task",
"Resource": "arn:aws:states:::batch:submitJob.sync",
"Parameters": {
"JobName": "daily-transform",
"JobQueue": "arn:aws:batch:eu-west-1:123456789012:job-queue/etl",
"JobDefinition": "arn:aws:batch:eu-west-1:123456789012:job-definition/transform:7",
"ArrayProperties": { "Size": 120 },
"ContainerOverrides": {
"Environment": [
{ "Name": "INPUT_PREFIX", "Value.$": "$.inputPrefix" }
]
}
},
"Retry": [
{
"ErrorEquals": ["Batch.AWSBatchException"],
"IntervalSeconds": 30,
"MaxAttempts": 3,
"BackoffRate": 2
}
],
"Catch": [
{ "ErrorEquals": ["States.ALL"], "Next": "NotifyFailure" }
],
"Next": "PublishResults"
}
}
}Two habits keep this honest in production. Pass pointers, not payloads: an S3 key instead of the data, because the state payload has a hard size limit and a container reads from S3 far more comfortably than from a JSON blob. And make the job idempotent on its inputs, because a retried attempt restarts the container from the first line, not from where it stopped.
Fargate or EC2 compute environments, and where does Spot fit?
Fargate compute environments leave you no instances to patch, no scaling group to tune and one isolated task per job, at a higher price per vCPU and per gigabyte than the equivalent EC2 capacity, with a ceiling on vCPU and memory per job and no GPUs or multi-node parallel jobs. Take Fargate for short and medium jobs of ordinary size, especially when work arrives irregularly and a cluster would otherwise idle between runs.
EC2 compute environments answer the rest: GPU jobs, instance families with local NVMe, memory beyond what Fargate offers, and multi-node parallel jobs spread across several instances. They also open the door to Spot, and Batch is built around interruption, so a job that checkpoints or is safe to rerun buys the same work at a large discount off on-demand. A common layout keeps one small on-demand queue for the runs that must finish tonight and routes everything else to a Spot queue with a higher attempt count.
What do Batch and Step Functions cost?
Shapes, not figures. AWS Batch adds no service charge of its own: you pay for the EC2 instances or Fargate tasks the jobs run on, plus storage and data transfer, which is why idle capacity, oversized instances and lazy scale-down are where the money actually goes. The current rates live on the AWS Batch pricing page and the Fargate pricing page.
Step Functions bills on a different shape: Standard workflows per state transition, Express workflows on request count, duration and memory, as set out on the Step Functions pricing page. A workflow that submits one Batch job costs close to nothing, because a long synchronous wait is still a single transition. A Distributed Map over a million items is another animal: every child execution counts, so batching items with ItemBatcher is a cost decision as much as a throughput one.
Decision checklist
- Job longer than 15 minutes, GPU, large memory, or a container image full of native dependencies: AWS Batch.
- Many small independent items sitting in S3, each within the Lambda ceiling: Step Functions Distributed Map, no cluster.
- Several jobs with an order, conditions, approvals or failure branches: Step Functions, whatever runs the steps underneath.
- Both at once, which is the common case: a state machine calling the synchronous Batch integration.
- Queue priorities or fairness between tenants on shared capacity: Batch job queues with a fair share policy.
- Irregular arrivals and ordinary job sizes: Fargate. Heavy, long or interruptible work: EC2 with Spot and more attempts.
- Every time: idempotent jobs, S3 pointers instead of payloads, and an alarm on the failure path. More of these trade-offs are collected in our AWS decision guides.