SSL bad write retry

I've just encountered some SSL "bad write retry" errors with ZServerSSL. Googling around, I see that somebody ran into this problem and asked about it on the Zope mailing list back in Feb 2003. I don't recall seeing that. (I'm subscribed to the Zope lists but I turned off delivery some years ago. Their traffic is simply too great.)

Anyways, based on my investigation, "bad write retry" arises from the way OpenSSL behaves in non-blocking mode; specifically, if a write cannot complete (because of SSL-layer protocol happenings) then the operation should be attempted again with the same data.

ZServerSSL is based on Medusa. In essence, ZServerSSL provides the SSL machinery; all else is Medusa's, including the HTTP implementation. Here's http_channel's 'send' method:

    def send (self, data):
        result = asynchat.async_chat.send (self, data)
        self.server.bytes_out.increment (len(data))
        return result

The operation is simple: Send some data and take note of the actual amount sent. For SSL, sometimes that number is zero. (Actually, M2Crypto.SSL returns -1 to mean "try again", but the Medusa-based https_server.py changes it to 0; I didn't bother to find out what the 'increment' method above does if it gets a negative argument.)

Based on black box observations of Medusa, it does not always invoke the 'send' method with the same amount of data. With TCP streams this is fine, but with OpenSSL in non-blocking mode and being unable to send a chunk of data earlier, Medusa's trying to send again with a bigger chunk results in "bad write retry".

I applied a simple fix: if the SSL write operation says, "try again", cache the chunk of data that wasn't sent. Subsequently Medusa may retry with a chunk that is the same size or bigger; if bigger, the new chunk includes the cached chunk, so simply make two writes: first for the cached chunk, then for the rest of the new chunk. So far this seems to have resolved my problem. I didn't cater for the case where Medusa retries with a smaller chunk. I haven't even looked at Medusa's code to see if it does that. Call it programmer's intuition. ;-)

I tested the above with Zope's ZMI over ZServerSSL and it was quite amusing to see overlapping and repeating chunks of the management screens appearing in the browser window as I fiddled with caching and replaying.