In Swift 3.0, I am currently trying to generate a large XML file that I want to send directly to a webserver via a HTTP POST request. Because this XML file can get very large, I do not want to store it entirely in memory, or first write it to disk and then read it again line-by-line when sending it to the server.
I have implemented the class that generates the XML file in such a way that it can write to an OutputStream
. This way, it doesn't matter whether that stream points to a file on disk, a Data
object in memory, or (hopefully) the body of an HTTP POST request.
After scouring the (somewhat scarce) Swift documentation for the URLSession
and Stream
classes and its accomplices, I settled on using a URLSession.uploadTask(withStreamedRequest)
task. This request requires an InputStream
to be delivered through one of the delegate methods:
urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void)
Within this callback, I bind an InputStream
and OutputStream
using Stream.getBoundStreams()
, after which I pass the OutputStream
to the class that generates the XML and return the InputStream
from the delegate method. The delegate method thus looks as follows:
func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void)
{
//Create the input and output stream and bind them, so that what the
//output stream writes ends up in the buffer of the input stream.
var input: InputStream? = nil
var output: OutputStream? = nil
let bufferSize: Int = 1024
Stream.getBoundStreams(withBufferSize: bufferSize, inputStream: &input, outputStream: &output)
//This part is not really important for you, it starts the generation of
//the XML, which is written directly to the output stream.
let converter = DatabaseConverterXml(prettyPrint: false)
let type = ConverterTypeSynchronization(progressAlert: nil)
type.convert(using: converter, writingTo: [Writable.Stream(output!)])
{
successfull in
print("Conversion Complete! Successfull: \(successfull)" )
}
//The input stream is then handed over via the
//completion handler of the delegate method.
completionHandler(input!)
}
Sometimes, the class generating the XML can take a little while before it writes the next line to the OutputStream
. If this happens for too long, the InputStream
may read so much that it actually clears its entire buffer. When this happens, somehow, the URLSession
framework (or perhaps the URLSessionUploadTask
itself), thinks the request is now finished and "submits" or "finalizes" it. This is a guess, however, as I am not sure of the inner workings of these classes (and the docs don't seem to help me much). This causes my webserver to receive an incomplete XML file and return a 500 Internal Server Error
.
Is there any way that I can stop the request from finalizing early? Preferably, I would like to "finalize" the input stream in the callback of the type.convert
call, as I know with certainty at that point that no more writes will occur (and the OutputStream
is in fact closed).
Is this the right way to approach the problem I am trying to solve? Is there perhaps any way I can directly interact with a stream that writes to the HTTP body? I feel very lost in this URLSession
framework and it has taken me a day and a half to get this far, so any advice is extremely appreciated. I'll buy anyone who is able to help me out with this a beer or two!
Thanks in advance for any help!
As @dgatwood pointed out, some of the variables are not retained properly. I've made the following changes to make sure that they do:
var mInput: InputStream? = nil
var mOutput: OutputStream? = nil
var mConverter: DatabaseConverterXml? = nil
var mType: ConverterTypeSynchronization? = nil
func urlSession(_ session: URLSession, task: URLSessionTask, needNewBodyStream completionHandler: @escaping (InputStream?) -> Void)
{
//Create the input and output stream and bind them, so that what the
//output stream writes ends up in the buffer of the input stream.
let bufferSize: Int = 1024
Stream.getBoundStreams(withBufferSize: bufferSize, inputStream: &mInput, outputStream: &mOutput)
//This part is not really important for you, it starts the generation of
//the XML, which is written directly to the output stream.
mConverter = DatabaseConverterXml(prettyPrint: false)
mType = ConverterTypeSynchronization(progressAlert: nil)
mType.convert(using: mConverter, writingTo: [Writable.Stream(mOutput!)])
{
successfull in
print("Conversion Complete! Successfull: \(successfull)" )
}
//The input stream is then handed over via the
//completion handler of the delegate method.
completionHandler(mInput!)
}
Short answer after a little bit of follow-up in chat:
These are, incidentally, pretty much the canonical things that folks do wrong when using stream-based networking APIs. I've made similar mistakes myself when working with related Foundation-level socket APIs.
IMO, it would make a lot more sense for the API to just buffer one object regardless of its length, and then send a space available message if it still had room in the socket's buffer. That wouldn't require any changes to existing clients, and would cause many fewer headaches... but I digress.