dimanche 19 juin 2016

GetQueuedCompletionStatus exiting prematurely


Let me give the general overview first. I'm recieving data thru three ports. I have a socket, a completion port and a worker thread for each. I call WSARecv and the worker thread process calls GetQueuedCompletionStatus followed by my parsing routine ReadMsgs. It sometimes happens that the buffer is unchanged when ReadMsgs is called and the buffer is updated while ReadMsgs is processing the buffer. The number of bytes processed, as returned by GetQueuedCompletionStatus is correct for the update when it occurs.

Does anyone know why this might occur and what I am doing wrong. Let me show you the code that seems most relevant. If you need to see more code, please be specific. My base socket class looks like this (I have omitted details that seem to me to be irrelevant. I have also omitted all error checking.)

class Socket_Base : public OVERLAPPED
{
public:
Socket_Base()
{
    // Initialize base OVERLAPPED object
    Internal = 0;
    InternalHigh = 0;
    Offset = 0;
    OffsetHigh = 0;
    hEvent = WSACreateEvent();

    // Initialize addr structure
    ZeroMemory( &addr, sizeof(struct sockaddr_in));

    // Create the completion port
    hCP = CreateIoCompletionPort( INVALID_HANDLE_VALUE, NULL, 0, 1);

    // Create the worker thread and bind it to the callback function and the completion port
    hThread = (HANDLE)_beginthreadex( NULL, 0, Callback_Socket, hCP, 0, NULL);

    // Create the socket
    Sock = WSASocket( AF_INET, SOCK_STREAM, IPPROTO_TCP, NULL, 0, WSA_FLAG_OVERLAPPED);

    // Bind the socket to the completion port
    CreateIoCompletionPort( (HANDLE)Sock, hCP, 0, 0);
}
void Connect() { WSAConnect( Sock, (SOCKADDR*)(&addr), sizeof(addr), NULL, NULL, NULL, NULL);}
void StartRecv()
{
    DWORD Flags = 0;
    DWORD numBytes = 0;
    if (WSARecv( Sock, &wsaBuf, 1, &numBytes, &Flags, (OVERLAPPED*)this, NULL) == 0) ReadMsgs( numBytes);
}
int ReadMsgs( int NumBytes);

protected:
    virtual ~Socket_Base() {}
    virtual void ProcessMsg() = 0;
    struct sockaddr_in addr;
    SOCKET Sock;
    HANDLE hCP;
    WSABUF wsaBuf;
    HANDLE hThread;
    char *readBuf;
    int bufsize;
};

Each port has its own derived socket class distinguised by port number and the virtual ProcessMsg function (which is called by ReadMsgs as each message is parsed). Here is one such class:

class Socket_Admin : public Socket_Base
{
public:
static const int bufcap = 1024;
Socket_Admin::Socket_Admin() : Socket_Base()
{
    // Buffer
    readBuf = new char[ bufcap];

    // The socket
    addr.sin_family = AF_INET;
    addr.sin_addr.s_addr = inet_addr("127.0.0.1");
    addr.sin_port = htons(9300);
    wsaBuf.buf = readBuf;
    wsaBuf.len = bufcap;
}
~Socket_Admin();
void ProcessMsg();
};

The worker thread process is

unsigned int Callback_Socket( void *lpParameter)
{
    HANDLE hCP = (HANDLE)lpParameter;
    DWORD NumBytes = 0;
    ULONGLONG CompletionKey;
    WSAOVERLAPPED *pOverlapped;
    while (GetQueuedCompletionStatus( hCP, &NumBytes, &CompletionKey, &pOverlapped, INFINITE) && CompletionKey == 0)
    {
        Socket_Base *pTCP = (Socket_Base*)pOverlapped;
        if (NumBytes > 0) pTCP->ReadMsgs( NumBytes);
        NumBytes = 0;
    }
    return 0;
}

There is one other thing I should explain. ReadMsgs does its parsing in place. The server delimits messages with a final line feed character and it delimits fields within a message with commas. ReadMsgs replaces commas and line feeds with null characters as it finds them and notes where each field begins in a separate array of pointers to locations in the buffer. Now when ReadMsgs gets to the end of the region of the buffer that was last filled it sometimes finds an incomplete message. This is copied to the beginning of the buffer, expecting the next read to complete the message, and wsaBuf is modified accordingly. Thus the end of ReadMsgs looks like this:

wsaBuf.buf = pchar;
wsaBuf.len = remsize;
StartRecv();

where pchar points to the character just beyond the partial message and remsize is the size of the remaining buffer.

I know what messages have been sent to my application from detailed server logs. The replacement of delimiters with null characters also makes it easy to see what part of the buffer has been processed. By saving the buffer to file and examining it, I can tell that it was updated after ReadMsgs was called. Also, by log messages not shown in the code above, I know that in these cases ReadMsgs was called by the worker thread. It doesnt happen every time, but it does happen.

If anyone can tell me what my mistake is, I would be grateful.


Aucun commentaire:

Enregistrer un commentaire