Shared storage between workers for large file processing

Add shared storage or memory between workers to enable parallel processing of large files without requiring each worker to download the entire file from object storage. This would significantly speed up workflows like video processing where a single large file needs to be split and processed across multiple workers.