Hello Scene — Events, Organization, more drawing

There are some design principles I’m after with my little demo scene library. Staring at that picture is enough to make your eyes hurt, but we’ll explore when it’s time to call it quits on your own home brew drawing library, and rely on the professionals. We’re also going to explore the whole eventing model, because this is where a lot of fun can come into the picture.

What is eventing then? Mouse, keyboard, touch, pen, all those ways the user can give input to a program. At times, the thing I’m trying to explore is the eventing model itself, so I need some flexibility in how the various mouse and keyboard events are percolated through the system. I don’t want to be forced into a single model designated by the operating system, so I build up a structure that gives me that flexibility.

First things first though. On Windows, and any other system, I need to actually capture the mouse and keyboard stuff, typically decode it, and then deal with it in my world. That code looks like this in the appmain.cpp file.

/*
    Generic Windows message handler
    This is used as the function to associate with a window class
    when it is registered.
*/
LRESULT CALLBACK MsgHandler(HWND hWnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
    LRESULT res = 0;
 
    if ((msg >= WM_MOUSEFIRST) && (msg <= WM_MOUSELAST)) {
        // Handle all mouse messages
        HandleMouseMessage(hWnd, msg, wParam, lParam);
    }
    else if (msg == WM_INPUT) {
        res = HandleHIDMessage(hWnd, msg, wParam, lParam);
    }
    else if (msg == WM_DESTROY) {
        // By doing a PostQuitMessage(), a 
        // WM_QUIT message will eventually find its way into the
        // message queue.
        ::PostQuitMessage(0);
        return 0;
    }
    else if ((msg >= WM_KEYFIRST) && (msg <= WM_KEYLAST)) {
        // Handle all keyboard messages
        HandleKeyboardMessage(hWnd, msg, wParam, lParam);
    }
    else if ((msg >= MM_JOY1MOVE) && (msg <= MM_JOY2BUTTONUP)) 
    {
        // Legacy joystick messages
        HandleJoystickMessage(hWnd, msg, wParam, lParam);
    }
    else if (msg == WM_TOUCH) {
        // Handle touch specific messages
        //std::cout << "WM_TOUCH" << std::endl;
        HandleTouchMessage(hWnd, msg, wParam, lParam);
    }
    //else if (msg == WM_GESTURE) {
    // we will only receive WM_GESTURE if not receiving WM_TOUCH
    //}
    //else if ((msg >= WM_NCPOINTERUPDATE) && (msg <= WM_POINTERROUTEDRELEASED)) {
    //    HandlePointerMessage(hWnd, msg, wParam, lParam);
    //}
    else if (msg == WM_ERASEBKGND) {
        //loopCount = loopCount + 1;
        //printf("WM_ERASEBKGND: %d\n", loopCount);
        if (gPaintHandler != nullptr) {
            gPaintHandler(hWnd, msg, wParam, lParam);
        }
 
        // return non-zero indicating we dealt with erasing the background
        res = 1;
    }
    else if (msg == WM_PAINT) {
        if (gPaintHandler != nullptr) 
        {
                gPaintHandler(hWnd, msg, wParam, lParam);
        }
    }
    else if (msg == WM_WINDOWPOSCHANGING) {
        if (gPaintHandler != nullptr) 
        {
            gPaintHandler(hWnd, msg, wParam, lParam);
        }
    }
    else if (msg == WM_DROPFILES) {
        HandleFileDropMessage(hWnd, msg, wParam, lParam);
    }
    else {
        // Not a message we want to handle specifically
        res = ::DefWindowProcA(hWnd, msg, wParam, lParam);
    }
 
    return res;
}

Through the magic of the Windows API, this function ‘MsgHandler’ is going to be called every time there is a Windows Message of some sort. It is typical of all Windows applications, in one form or another. Windows messages are numerous, and very esoteric. There are a couple of parameters, and the values are typically packed in as bitfields of integers, or pointers to data structures that need to be further decoded. Plenty of opportunity to get things wrong.

What we do here is capture whole sets of messages, and hand them off to another function to be processed further. In the case of mouse messages, we have this little bit of code:

if ((msg >= WM_MOUSEFIRST) && (msg <= WM_MOUSELAST)) {
    // Handle all mouse messages
    HandleMouseMessage(hWnd, msg, wParam, lParam);
}

So, first design choice here, is delegation. We don’t know how any application is going to want to handle the mouse messages, so we’re just going to capture them, and send them somewhere. In this case, the HandleMouseMessage() function.

/*
    Turn Windows mouse messages into mouse events which can
    be dispatched by the application.
*/
LRESULT HandleMouseMessage(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{   
    LRESULT res = 0;
    MouseEvent e;
 
    e.x = GET_X_LPARAM(lParam);
    e.y = GET_Y_LPARAM(lParam);
 
    auto fwKeys = GET_KEYSTATE_WPARAM(wParam);
    e.control = (fwKeys & MK_CONTROL) != 0;
    e.shift = (fwKeys & MK_SHIFT) != 0;
 
    e.lbutton = (fwKeys & MK_LBUTTON) != 0;
    e.rbutton = (fwKeys & MK_RBUTTON) != 0;
    e.mbutton = (fwKeys & MK_MBUTTON) != 0;
    e.xbutton1 = (fwKeys & MK_XBUTTON1) != 0;
    e.xbutton2 = (fwKeys & MK_XBUTTON2) != 0;
    bool isPressed = e.lbutton || e.rbutton || e.mbutton;
 
    // Based on the kind of message, there might be further
    // information to be decoded
    // mostly we're interested in setting the activity kind
    switch(msg) {
        case WM_LBUTTONDBLCLK:
        case WM_MBUTTONDBLCLK:
        case WM_RBUTTONDBLCLK:
            break;
 
        case WM_MOUSEMOVE:
            e.activity = MOUSEMOVED;
            break;
 
        case WM_LBUTTONDOWN:
        case WM_RBUTTONDOWN:
        case WM_MBUTTONDOWN:
        case WM_XBUTTONDOWN:
            e.activity = MOUSEPRESSED;
            break;
        case WM_LBUTTONUP:
        case WM_RBUTTONUP:
        case WM_MBUTTONUP:
        case WM_XBUTTONUP:
            e.activity = MOUSERELEASED;
            break;
        case WM_MOUSEWHEEL:
            e.activity = MOUSEWHEEL;
            e.delta = GET_WHEEL_DELTA_WPARAM(wParam);
            break;
        case WM_MOUSEHWHEEL:
            e.activity = MOUSEHWHEEL;
            e.delta = GET_WHEEL_DELTA_WPARAM(wParam);
            break;
             
        break;
    }
 
    gMouseEventTopic.notify(e);
 
    return res;
}

Here, I do introduce a strong opinion. I create a specific data structure to represent a MouseEvent. I do this because I want to make sure to decode all the mouse event has to offer, and present it in a very straight forward data structure that applications can access easily. So, the design choice is to trade off some memory for the sake of ease of consumption. In the uievent.h file, are various data structures that represent the various event structures, for mouse, keyboard, joystick, touch, even file drops, and pointers in general. That’s not the only kinds of messages that can be decoded, but it’s the ones used most for user interaction.

// Basic type to encapsulate a mouse event
enum {
    // These are based on regular events
    MOUSEMOVED,
    MOUSEPRESSED,
    MOUSERELEASED,
    MOUSEWHEEL,         // A vertical wheel
    MOUSEHWHEEL,        // A horizontal wheel
 
    // These are based on application semantics
    MOUSECLICKED,
    MOUSEDRAGGED,
 
    MOUSEENTERED,
    MOUSEHOVER,         // like move, when we don't have focus
    MOUSELEFT           // exited boundary
};
 
struct MouseEvent {
    int id;
    int activity;
    int x;
    int y;
    int delta;
 
    // derived attributed
    bool control;
    bool shift;
    bool lbutton;
    bool rbutton;
    bool mbutton;
    bool xbutton1;
    bool xbutton2;
};

From a strict performance perspective, this data structure should be a “cache line size” amount of data ideally, so the processor cache will handle it most efficiently. But, that kind of optimization can be tackled later if this is really beneficial. Initially, I’m just concerned with properly decoding the information and presenting it in an easy manner.

At the very end of HandleMouseMessage(), we see this interesting call before the return

gMouseEventTopic.notify(e);

OK. This is where we depart from the norm of mouse handling and introduce a new concept, the publish/subscribe mechanism.

So far, we’ve got a tight coupling between the event coming in through MsgHandler, and being processed at HandleMouseMessage(). Within an application, the next logical step might be to have this explicitly call the mouse logic of the application. But, that’s not very flexible. What I’d really like to do is say “hay system, call this specific function I’m going to give you whenever a mouse event occurs”. But wait, didn’t we already do that with HandleMouseMessage()? Yes, in a sense, but that was primarily to turn the native system mouse message into something more palatable.

In general terms, I want to view the system as a publish/subscribe pattern. I want to look at the system as if it’s publishing various bits of information, and I want to ‘subscribe’ to various topics. What’s the difference? With the tightly couple function calling thing, one function calls another, calls another, etc. With pub/sub, the originator of an event doesn’t know who’s interested in it, it just knows that several subscribers have said “tell me when an event occurs”, and that’s it.

OK, so how does this work?

I need to tell the application runtime that I’m interested in receiving mouse events. I need to implement a function that has a certain interface to it, and ‘subscribe’ to the mouse events.

// This routine will be called whenever there
// is a mouse event in the application window
void handleMouseEvent(const MouseEventTopic& p, const MouseEvent& e)
{
    mouseX = e.x;
    mouseY = e.y;
 
    switch (e.activity)
    {
    case MOUSERELEASED:
        // change the color for the cursor
        cColor = randomColor();
        break;
    }
}
 
void onLoad()
{
    subscribe(handleMouseEvent);

That’s pretty much it. In the ‘onLoad()’ implementation, I call ‘subscribe()’, passing in a pointer to the function that will receive the mouse events when they occur. If you’re content with this, jump over the following section, and continue at Back To Sanity. Otherwise, buckle in for some in depth.

There are several subscribe() functions. Each one of them is a convenience for registering a function to be called in response to information being available for a specific topic. You can see these in apphost.h

APP_EXPORT void subscribe(SignalEventTopic::Subscriber s);
APP_EXPORT void subscribe(MouseEventTopic::Subscriber s);
APP_EXPORT void subscribe(KeyboardEventTopic::Subscriber s);
APP_EXPORT void subscribe(JoystickEventTopic::Subscriber s);
APP_EXPORT void subscribe(FileDropEventTopic::Subscriber s);
APP_EXPORT void subscribe(TouchEventTopic::Subscriber s);
APP_EXPORT void subscribe(PointerEventTopic::Subscriber s);

The construction ‘EventTopic::Subscriber’ is a manifestation of how these Topics are constructed. Let’s take a look at the Topic template to understand a little more deeply. The comments in the code below give a fair explanation of the Topic template. Essentially, you just want to have a way to identify a topic, and construct a function signature to match. The topic contains two functions of interest. ‘subscribe()’, allows you to register a function to be called when the topic wants to publish information, and the ‘notify()’ function, which is the way in which the information is actually published.

/*
    Publish/Subscribe is that typical pattern where a 
    publisher generates interesting data, and a subscriber
    consumes that data.
 
    The Topic class contains both the publish and subscribe
    interfaces.
 
 
    Whatever is responsible for indicating the thing happened
    will call the notify() function of the topic, and the
    subscribed function will be called.
 
    The Topic does not incoroporate any threading model
    A single topic is not a whole pub/sub system
    Multiple topics are meant to be managed together to create
    a pub/sub system.
 
    Doing it this way allows for different forms of composition and
    general topic management.
 
    T - The event payload, this is the type of data that will be
    sent when a subscriber is notified.
 
    The subscriber is a functor, that is, anything that has the 
    function signature.  It can be an object, or a function pointer,
    essentially anything that resolves as std::function<void>()
 
    This is a very nice pure type with no dependencies outside
    the standard template library
*/
template <typename T>
class Topic
{
public:
    // This is the form of subscriber
    using Subscriber = std::function<void(const Topic<T>& p, const T m)>;
 
private:
    std::deque<Subscriber> fSubscribers;
 
public:
    // Notify subscribers that an event has occured
    // Just do a simple round robin serial invocation
    void notify(const T m)
    {
        for (auto & it : fSubscribers) {
            it(*this, m);
        }
    }
 
    // Add a subscriber to the list of subscribers
    void subscribe(Subscriber s)
    {
        fSubscribers.push_back(s);
    }
};

So, it’s a template. Let’s look at some instantiations of the template that are made within the runtime.

// Within apphost.h
// Make Topic publishers available
using SignalEventTopic = Topic<intptr_t>;
 
using MouseEventTopic = Topic<MouseEvent&>;
using KeyboardEventTopic = Topic<KeyboardEvent&>;
using JoystickEventTopic = Topic<JoystickEvent&>;
using FileDropEventTopic = Topic<FileDropEvent&>;
using TouchEventTopic = Topic<TouchEvent&>;
using PointerEventTopic = Topic<PointerEvent&>;
 
 
// Within appmain.cpp
// Topics applications can subscribe to
SignalEventTopic gSignalEventTopic;
KeyboardEventTopic gKeyboardEventTopic;
MouseEventTopic gMouseEventTopic;
JoystickEventTopic gJoystickEventTopic;
FileDropEventTopic gFileDropEventTopic;
TouchEventTopic gTouchEventTopic;
PointerEventTopic gPointerEventTopic;

The application runtime, as we saw in the HandleMouseMessage() function, will then call the appropriate topic’s ‘notify()’ function, to let the subscribers know there’s some interesting information being published. Perhaps this function should be renamed to ‘publish()’.

And that’s it. All this pub/sub machinery makes it so that we can be more flexible about when and how we handle various events within the system. You can go further and create whatever other constructs you want from here. You can add queues, multiple threads, duplicates. You can decide you want to have two places react to mouse events, completely unbeknownst to each other.

Back to Sanity

Alright, let’s see how an application can actually use all this. This is the mousetrack application.

mousetrack

The app is simple. The read square follows the mouse around while it’s within the boundary of the window. All the lines from the top and bottom terminate at the point of the mouse as well.

In this case, we want to track where the mouse is, make note of that location, and use it in our drawing routines. In addition, of course, we want to draw the lines, circles, square, and background.

/*
    Demonstration of how to subscribe
    to keyboard and mouse events.
 
    Using encapsulated drawing and PixelArray
*/
#include "apphost.h"
#include "draw.h"
 
 
// Some easy pixel color values
#define black   PixelRGBA(0xff000000)
#define white   PixelRGBA(0xffffffff)
#define red     PixelRGBA(0xffff0000)
#define green   PixelRGBA(0xff00ff00)
#define blue    PixelRGBA(0xff0000ff)
#define yellow  PixelRGBA(0xffffff00)
 
 
// Some variables to track mouse and keyboard info
int mouseX = 0;
int mouseY = 0;
int keyCode = -1;
 
// global pixel array (gpa)
// The array of pixels we draw into
// This will just wrap what's already created
// for the canvas, for convenience
PixelArray gpa;
 
PixelPolygon gellipse1;
PixelPolygon gellipse2;
 
// For the application, we define the size of 
// the square we'll be drawing wherever the mouse is
constexpr size_t iconSize = 64;
constexpr size_t halfIconSize = 32;
 
// Define the initial color of the square we'll draw
// clicking on mouse, or space bar, will change color
PixelRGBA cColor(255, 0, 0);
 
// Simple routine to create a random color
PixelRGBA randomColor(uint32_t alpha = 255)
{
    return { 
        (uint32_t)maths::random(255), 
        (uint32_t)maths::random(255), 
        (uint32_t)maths::random(255), alpha };
    }
 
// This routine will be called whenever there
// is a mouse event in the application window
void handleMouseEvent(const MouseEventTopic& p, const MouseEvent& e)
{
    // Keep track of the current mouse location
    // Use this in the drawing routine
    mouseX = e.x;
    mouseY = e.y;
 
    switch (e.activity)
    {
    case MOUSERELEASED:
        // change the color for the cursor
        cColor = randomColor();
        break;
    }
}
 
// Draw some lines from the top and bottom edges of
// the canvas, converging on the 
// mouse location
void drawLines(PixelArray &pa)
{
    // Draw some lines from the edge to where
    // the mouse is
    for (size_t x = 0; x < pa.width; x += 4)
    {
        draw::copyLine(pa, x, 0, mouseX, mouseY, white);
    }
 
    for (size_t x = 0; x < pa.width; x += 16)
    {
        draw::copyLine(pa, x, pa.height-1, mouseX, mouseY, white, 1);
    }
 
}
 
// Simple routine to create an ellipse
// based on a polygon.  Very crude, but
// useful enough 
INLINE void createEllipse(PixelPolygon &poly, ptrdiff_t centerx, ptrdiff_t centery, ptrdiff_t xRadius, ptrdiff_t yRadius)
{
    static const int nverts = 72;
    int steps = nverts;
 
    ptrdiff_t awidth = xRadius * 2;
    ptrdiff_t aheight = yRadius * 2;
 
    for (size_t i = 0; i < steps; i++) {
        auto u = (double)i / steps;
        auto angle = u * (2 * maths::Pi);
 
        ptrdiff_t x = (int)std::floor((awidth / 2.0) * cos(angle));
        ptrdiff_t y = (int)std::floor((aheight / 2.0) * sin(angle));
        poly.addPoint(PixelCoord({ x + centerx, y + centery }));
    }
    poly.findTopmost();
}
 
// Each time through the main application 
// loop, do some drawing
void onLoop()
{
    // clear screen to black to start
    draw::copyAll(gpa, black);
 
    drawLines(gpa);
 
    // draw a rectangle wherever the mouse is
    draw::copyRectangle(gpa, 
        mouseX-halfIconSize, mouseY-halfIconSize, 
        iconSize, iconSize, 
        cColor);
 
    // Draw a couple of green ellipses
    draw::copyPolygon(gpa, gellipse1, green);
    draw::copyPolygon(gpa, gellipse2, green);
 
    // force the canvas to be drawn
    refreshScreen();
}
 
// Called as the application is starting up, and
// before the main loop has begun
void onLoad()
{
    setTitle("mousetrack");
 
    // initialize the pixel array
    gpa.init(canvasPixels, canvasWidth, canvasHeight, canvasBytesPerRow);
 
 
    createEllipse(gellipse1, 120, 120, 30, 30);
    createEllipse(gellipse2, (ptrdiff_t)gpa.width - 120, 120, 30, 30);
 
    // setup to receive mouse events
    subscribe(handleMouseEvent);
}

At the end of the ‘onLoad()’, we see the call to subscribe for mouse events. Within handleMouseEvent(), we simply keep track of the mouse location. Also, if the user clicks a button, we will change the color of the rectangle to be drawn.

Well, that’s pretty much it. We’ve wandered through the pub/sub mechanism for event dispatch, and looked specifically at how this applies to the mouse messages coming from the Windows operating system. The design principle here is to be loosely coupled, and allow the application developer to create the constructs that best suite their needs, without imposing too much of an opinion on how that must go.

I snuck in a bit more drawing. Now there are lines, in any direction, and thickness, as well as rudimentary polygons.

In the next installment, I’ll look a bit deeper into the drawing, and we’ll look at things like screen capture, and how to record our activities to turn into demo movies.