ubuntuazure-devopssquid

Azure DevOps Self-Hosted Agent Fails Initialization Due to Very Slow Task Downloads

I’m trying to set up a self-hosted agent for Azure DevOps pipelines, but the agent never gets past the Initialization phase. It always hangs while downloading tasks.

Environment

The agent connects successfully and shows online, but every job fails during task download (for example, NodeTool, UsePythonVersion).

Symptoms

Error screenshot:

Error message

What I have already tried

I tested multiple network/VM configurations in VirtualBox:

The issue persists in all scenarios.

Log details (full exception)

[debug]System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer.
 ---> System.Net.Sockets.SocketException (104): Connection reset by peer
   at System.Net.Sockets.NetworkStream.ReadAsync(Memory`1 buffer, CancellationToken cancellationToken)
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Net.Sockets.SocketAsyncEventArgs.TransferCompletionCallbackCore(Int32 bytesTransferred, Memory`1 socketAddress, SocketFlags receivedFlags, SocketError socketError)
   at System.Net.Sockets.SocketAsyncEngine.System.Threading.IThreadPoolWorkItem.Execute()
   at System.Threading.ThreadPoolWorkQueue.Dispatch()
   at System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart()
--- End of stack trace from previous location ---

   --- End of inner exception stack trace ---
   at System.Net.Security.SslStream.EnsureFullTlsFrameAsync[TIOAdapter](CancellationToken cancellationToken, Int32 estimatedSize)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at System.Net.Security.SslStream.ReadAsyncInternal[TIOAdapter](Memory`1 buffer, CancellationToken cancellationToken)
   at System.Runtime.CompilerServices.PoolingAsyncValueTaskMethodBuilder`1.StateMachineBox`1.System.Threading.Tasks.Sources.IValueTaskSource<TResult>.GetResult(Int16 token)
   at System.Net.Http.HttpConnection.FillAsync(Boolean async)
   at System.Net.Http.HttpConnection.CopyToContentLengthAsync(Stream destination, Boolean async, UInt64 length, Int32 bufferSize, CancellationToken cancellationToken)
   at System.Net.Http.HttpConnection.ContentLengthReadStream.CompleteCopyToAsync(Task copyTask, CancellationToken cancellationToken)
   at Microsoft.VisualStudio.Services.Agent.Worker.TaskManager.DownloadAsync(IExecutionContext executionContext, TaskStepDefinitionReference task)
   at Microsoft.VisualStudio.Services.Agent.Worker.TaskManager.DownloadAsync(IExecutionContext executionContext, IEnumerable`1 steps)
   at Microsoft.VisualStudio.Services.Agent.Worker.JobExtension.InitializeJob(IExecutionContext jobContext, AgentJobRequestMessage message)

Question

Has anyone faced similar behavior where a self-hosted Azure DevOps agent becomes extremely slow only during Azure task downloads?
Is this related to VirtualBox networking, Brazil CDN edge issues, Azure throttling, or something else?

Any working setups, network adjustments, or insights on how to solve this are very welcome.

Solution

  • I solved the problem by switching the VirtualBox network adapter from Bridged Adapter to NAT.
    After that change, the Azure DevOps self-hosted agent immediately stopped hanging during task download and initialization, and all tasks started to download at normal speed.

    Before switching to NAT, the agent was stuck at “Downloading tasks…”, sometimes transferring only a few KB/s, and eventually failing with connection resets or partial downloads.
    Regular curl tests outside the agent worked fine, which made the issue confusing.

    Changing to NAT fixed everything instantly.

    This strongly suggests that the issue was caused by the VM’s network configuration in VirtualBox, not by Azure DevOps, the agent version, or internet connectivity. Bridged mode can expose MTU/fragmentation or TCP behavior from the host network directly to the VM, and Azure DevOps task downloads are sensitive to these conditions. NAT mode hides those details by handling packet translation inside VirtualBox, which results in stable connections for the agent.

    If anyone has deeper insight into why VirtualBox Bridged mode triggers this behavior (especially with Azure Pipelines agents, TLS, or large HTTP/2 transfers), additional contributions are welcome.