Why Are WebRequests Throttled? I Want More Throughput!

By default, a web browser – or in this case any .Net application that uses the System.Net.WebRequest class – will only allow up to 2 simultaneous threads at a time per host domain.

One way to lift this limit is to run Fiddler. It bypasses this limit and you can see much more throughput. I admit that this is a hack but it works when debugging.

When your code is running inside Windows Azure, Fiddler isn’t a possible hack, but wait there’s a better way!

Add the following configuration to your App.Config or Web.Config.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.net>
    <connectionManagement>
      <add address="*" maxconnection="1000" />
    </connectionManagement>
  </system.net>
</configuration>

This configuration will set a limit of 1000 connections for all addresses. After some tests with fiddler I got the same results with and without fiddler. 1000 requests to http://www.google.ca took 53.7946 seconds.

Stopwatch watch = new Stopwatch();
watch.Start();
var html = Enumerable.Range(0, 1000)
                        .AsParallel()
                        .Select(i => RestClient.Uri("http://www.google.ca")
                        .GetAsync((uri, code, arg3) =>
                        {
                            //Do nothing if error
                        })).ToArray();

Task.WaitAll(html);

watch.Stop();
Console.WriteLine(watch.Elapsed.TotalSeconds);

More details about the RestClient can be found in my post about Asynchronously Calling REST Services From a Windows Azure Role.

Executing the same test without the Connection Manager configuration takes 85.0352923 seconds. You can find more information about why this works in the following post about Greatly Increase the Performance of Azure Storage CloudBlobClient.

When making calls to Rest services, deactivating the NagleAlgorithm can reduce latency when the data transferred over the network is small.

Nagle’s algorithm, named after John Nagle, is a means of improving the efficiency of TCP/IP networks by reducing the number of packets that need to be sent over the network.

Applications that expect real time responses can react poorly with Nagle’s algorithm. Applications such as networked multiplayer video games expect that actions in the game are sent immediately, while the algorithm purposefully delays transmission, increasing bandwidth efficiency at the expense of latency. For this reason applications with low-bandwidth time-sensitive transmissions typically use TCP_NODELAY to bypass the Nagle delay.

You can deactivate the algorithm for a single endpoint as follows:

var servicePoint = ServicePointManager
                        .FindServicePoint(new Uri("http://www.google.ca"));
servicePoint.UseNagleAlgorithm = false;

you can also deactivate the algorithm globally for a process as follows:

ServicePointManager.UseNagleAlgorithm = false;

The ServicePointManager can also be used to augment the default connection limit.

ServicePointManager.DefaultConnectionLimit = 1000;

Furthermore, the Windows Azure Storage Team published Nagle’s Algorithm is Not Friendly towards Small Requests addressing this exact problem for the  Windows Azure Table Storage Service and the Windows Azure Queue Storage Service.

The following is a quote, from their blog post, provides more details:

Since Nagling is on by default, the way to turn this off is by resetting the flag in ServicePointManager. The ServicePointManager is a .NET class that allows you to manage ServicePoint where each ServicePoint provides HTTP connection management. ServicePointManager also allows you to control settings like maximum connections, Expect 100, and Nagle for all ServicePoint instances. Therefore, if you want to turn Nagle off for just tables or just queues or just blobs in your application, you need to turn it off for the specific ServicePoint object in the ServicePointManager.

Note, this has to be set for the role (process) before any calls to blob, table and queue are done for the setting to be applied.  In addition, this setting will only be applied to the process that executes it (it does not affect other processes) running inside the same VM instance.

Turning Nagle off should be considered for Table and Queue (and any protocol that deals with small sized messages). For large packet segments, Nagling does not have an impact since the segments will form a full packet and will not be withheld. But as always, we suggest that you test it for your data before turning Nagle off in production.