WCF response too long - authentication error

In the project that I currently work for, there is exposed a WCF service which returns an array of a business entity, let's call it Invoice :

Invoice[] GetInvoicesByTypeAndTime(InvoiceType invoiceType, byte startHour, byte? endHour);

The authentication mechanism used is Windows Authentication, the WCF service is hosted in a Web Application hosted on IIS 6.

At first when I used to get data more than 64kB a CommunicationException was thrown stating that "The maximum message size quota for incoming messages (65536) has been exceeded. To increase the quota, use the MaxReceivedMessageSize property on the appropriate binding element."

Fine, I just increased in App.config the values to 65536000 (I blatantly added three zeros at the end) for both maxReceivedMessageSize and maxBufferSize (the latter because it complained in an ArgumentException that "For TransferMode.Buffered, MaxReceivedMessageSize and MaxBufferSize must be the same value. Parameter name: bindingElement").

Now I could receive larger responses...

Until I hit another limit (I THINK) in that after 624 elements (approx. 2.2 MB) a strange exception is thrown :

System.ServiceModel.Security.MessageSecurityException: The HTTP request is unauthorized with client authentication scheme 'Negotiate'. The authentication header received from the server was 'Negotiate,NTLM'. ---> System.Net.WebException: The remote server returned an error: (401) Unauthorized.
   at System.Net.HttpWebRequest.GetResponse()
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   --- End of inner exception stack trace ---

Server stack trace:

   at System.ServiceModel.Channels.HttpChannelUtilities.ValidateAuthentication(HttpWebRequest request, HttpWebResponse response, WebException responseException, HttpChannelFactory factory)
   at System.ServiceModel.Channels.HttpChannelUtilities.ValidateRequestReplyResponse(HttpWebRequest request, HttpWebResponse response, HttpChannelFactory factory, WebException responseException)
   at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.WaitForReply(TimeSpan timeout)
   at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Dispatcher.RequestChannelBinder.Request(Message message, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
   at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs)
   at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
   at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)

Exception rethrown at [0]:

   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Test2.DBS.IDbService.GetInvoicesByTypeAndTime(InvoiceType invoiceType, Byte startHour, Nullable`1 endHour)
   at Test2.DBS.DbServiceClient.GetInvoicesByTypeAndTime(InvoiceType invoiceType, Byte startHour, Nullable`1 endHour) in D:\TEMP\Test2\Test2\Service References\DBS\Reference.cs:line 1445
   at Test2.Program.Main(String[] args) in D:\TEMP\Test2\Test2\Program.cs:line 19

Is there a limit on authenticated responses? Is there a limit from the ASP.NET settings?

I'm guessing that you're using Windows Authentication hence the 401 rather than a message explaining how you've blown your message limits. When you send through a Windows Authenticated request, WCF sends the SOAP request twice, once for it to fail and return an accepts header, and the second time to send it with the Windows Authentication headers.

However, from my testing, it looks like you still get the 401 if the message would have actually failed if it did get through.

To troubleshoot this, I had to put server trace logging in:

<system.diagnostics>
    <trace autoflush="true" />
    <sources>
      <source name="System.ServiceModel" switchValue="Critical, Error, Warning">
        <listeners>
          <add name="traceListener" type="System.Diagnostics.XmlWriterTraceListener" initializeData="C:\Logs\ServiceTrace.svclog"/>
        </listeners>
      </source>
    </sources>
  </system.diagnostics>

Then, I had to put in larger reader quotas, as above (but I used smaller values):

Then you generally have to put in a custom behaviour to increase the maximum number of items in an object graph:

<behaviors>
  <serviceBehaviors>
    <behavior name="MaximumItemsBehaviour">
      <dataContractSerializer maxItemsInObjectGraph="2147483647" />
      <!-- To avoid disclosing metadata information, set the value below to false and remove the metadata endpoint above before deployment -->
      <serviceMetadata httpsGetEnabled="true" httpGetEnabled="false" />
      <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
      <serviceDebug includeExceptionDetailInFaults="true" />
    </behavior>
  </serviceBehaviors>
</behaviors>

You'll need to add a "behaviourConfiguration" attribute to your "<system.serviceModel><services><service>" element with the value "MaximumItemsBehaviour".

Other advice I read, but didn't need myself was to add:

  <system.web>
    <compilation debug="true" targetFramework="4.0" />
    <httpRuntime maxRequestLength="2097151" />
  </system.web>

And:

  <system.webServer>
    <modules runAllManagedModulesForAllRequests="true" />
    <security>
      <requestFiltering>
        <requestLimits maxAllowedContentLength="209715200"/>
      </requestFiltering>
    </security>
  </system.webServer>