Some powershell commands or cmdlets display progress. I am using Invoke-WebRequest in scripted pipeline. Prepending $ProgressPreference = 'SilentlyContinue' to my script block allows Invoke-WebRequest to perform downloads and uploads at expected speeds. Without it I was getting ~100x worse performance.
I don't understand this in detail. Please consider managing this ProgressPreference in the durable task powershell step.