Skip to main content

Support more than 100 items in `batchTrigger` at once

Feature Request

You can only provide 100 items in batchTrigger and batchTriggerAndWait at the moment. if you want to batch more items than this you need to call it multiple times, creating more than one batch.

We should add a way to create larger batches. It would involve multiple API calls and we'd probably want to make that explicit. For example, you would create a batch, add items to it and then finalize it.

Posted by Linear ·
Status: Completed4 comments

Comments4

  • cachho

    •

    Apr 20, 2025

    +1 - came to trigger because I was looking for a solution that could rely on to handle my clients requests, no matter how big and how complex they are.

    Now apparently complex isn’t an issue with trigger (no timeouts), but it doesn’t scale vertically due to the 500 item batch size limit.

    This means I have to implement chunking, which I have not completed yet because I’m stuck with a couple bugs in v4 (reported on github, might be unrelated), but anyways, I prefer not to mess with chunking. This reminds me too much of how it’s done now, where one endpoint calls another endpoint which calls another endpoint and streaming in the mix, to avoid timeouts.

    Because Matt asked on discord about the number I need, realistically I know my clients have up to 5000 items (all small tasks), so I would like to be able to have a batch size of 10000 to be on the safe side.

    Thanks

  • mattaitken

    Team•

    Dec 10, 2024

    We increased the batch limits and made other improvements to batches. Details here:

    https://trigger.dev/launchweek/0/batch-processing

    • mattaitken

      Team•

      Oct 21, 2024

      Unfortunately it’s more complicated than that. 100 items with large payloads can take a bit of time to process so supporting more than 100 needs a different approach.