samedi 11 juin 2016

GetWindowLongPtr returns garbage


I am developing a Win32 dialog box wrapper.

.h file

class dlg {
    static INT_PTR DlgProcTmp(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp);
    INT_PTR DlgProc(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp);

    bool ismodal;
protected:
    HWND hwndDlg;
    int id;
public:
    virtual void oncreate(const widget &w) { }
    virtual void oncmd(const widget &w, int code, int idCntrl, HWND hwnd) { }
    virtual void onclose(const widget &w) { }
    dlg() { }
    dlg(int id);
    INT_PTR domodal(HWND hwndOwner = nullptr);
    widget domodeless(HWND hwndOwner = nullptr, int cmdshow = SW_SHOW);
};

.cpp file

INT_PTR dlg::DlgProcTmp(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
    dlg *This;

    if (wm == WM_INITDIALOG) {
        This = (dlg *) lp;
        setwinlong(hwnd, DWLP_USER, This);
        This->oncreate(widget(hwnd));

        return TRUE;
    }
    if ((This = getwinlong<dlg *>(hwnd, DWLP_USER)) != nullptr)
        return This->DlgProc(hwnd, wm, wp, lp);
    return FALSE;
}
INT_PTR dlg::DlgProc(HWND hwnd, UINT wm, WPARAM wp, LPARAM lp)
{
    this->hwndDlg = hwnd;
    switch (wm) {
        case WM_COMMAND:
            this->oncmd(widget(hwnd), HIWORD(wp), LOWORD(wp), (HWND) lp);
            if (this->ismodal)
                EndDialog(hwnd, LOWORD(wp));
            else
                DestroyWindow(hwnd);
            return TRUE;
        case WM_DESTROY:
            this->onclose(widget(hwnd));
            return TRUE;
    }
    return FALSE;
}

The dialog is created in domodal with a DialogBoxParam call. The last argument is the this pointer, which I then retrieve from the lparam of the WM_INITDIALOG message. To allow this to be used between different messages, I save this with the HWND in WM_INITDIALOG. However, whenever a message not WM_INITDIALOG arrives, and I get the this pointer with GetWindowLongPtr, it returns a garbage dlg class, whose vtable is corrupt. I use the vtable to call the correct handler function. As a result, my code crashes on the first line of the WM_COMMAND handler. Here is what the debugger shows as the value of this:

enter image description here

Why is GetWindowLongPtr returning garbage?

BTW, getwinlong is a wrapper for GetWindowLongPtr, and setwinlong is a wrapper for SetWindowLongPtr. Here are their implementations:

template <class TYPE> static TYPE getwinlong(HWND hwnd, int idx)
{
    return (TYPE) GetWindowLongPtr(hwnd, idx);
}
template <class TYPE> static void setwinlong(HWND hwnd, int idx, TYPE val)
{
    SetWindowLongPtr(hwnd, idx, (LONG_PTR) val);
}

I have seen the numerous posts about how GetWindowLongPtr will fail on Win64, if you cast to LONG instead of LONG_PTR, like https://blogs.msdn.microsoft.com/oldnewthing/20131226-00/?p=2263. I believe that my problem is different.

Edit: @andlabs wanted the code that creates the dialog:

INT_PTR dlg::domodal(HWND hwndOwner)
{
    this->ismodal = true;
    return DialogBoxParam(gethinst(hwndOwner), 
        MAKEINTRESOURCE(id), hwndOwner, DLGPROC(dlg::DlgProcTmp), 
        (LPARAM) this);
}

Aucun commentaire:

Enregistrer un commentaire