First N of Total batchTriggerAndWait

Add the ability to complete a batch after N of the runs in the batch have completed, cancelling the incomplete runs. This would allow parallelizing multiple LLM calls and completing once one or more have completed, and stopping work on the others, to get back multiple results faster.

Upvoters
Status

In Review

Board

πŸ’‘ Feature Request

Date

8 months ago

Author

Eric Allam

Subscribe to post

Get notified by email when there are changes.