• Home
• Research
• Courses
• Publications
• People
• Affiliations
• Software

Large Display Framework: Tutorial 2

PART II: CREATING YOUR OWN STRATEGIES

The core of an application that uses the Large Display Framework lays in the set of strategies used by its components. The LDF Toolkit offers a small set of strategies based on some research applications developed at the UofC's iLab. Even though some of these strategies can be useful for a wide range of applications, it is very likely that the programs you will be creating will need some kind of functionality that is not yet offered. Therefore, it is crucial to know how to create your own strategies within this framework. This is the goal for the second part of this tutorial.

We will

1. define what strategies are,
2. present their class interface,
3. suggest some categories of strategies are suggested, and
4. give four examples in different categories

Definition

The strategies used in the Large Display Framework consist in an adaptation of the Strategy Pattern combined with the Decorator Pattern [Gamma et al., Design Patterns: Elements of Reusable Object-Oriented Software, 1995].

The Strategy Pattern provides a set of interchangeable algorithms that accomplish certain functionality. In the case of the components in the framework, the functionalities we want to delegate to the strategies consist in drawing, event handling, and buffer management. Therefore, a strategy for a visual component encapsulates algorithms for dealing with one or more of these functionalities into a single class.

The Decorator Pattern allows adding responsibilities to an object dynamically. This means that the functionalities of the visual components can be obtained from the combination of a set of "decorations" applied to it, instead of deriving them from a fixed class hierarchy, for example.

In this manner, we can define a strategy in the Large Display Framework as a combinatorial unit of functionality that can be applied to a visual component. In other words, the functionality (appearance and behavior) of a visual component is given by a combination of strategy objects and these functionalities might accumulate on top of each other (e.g., when drawing an image with one strategy and a border on top of it with another strategy).

Strategies in the Framework

The behavior and appearance of a component are defined by the combination of independent strategy objects associated to it. The role of the framework then is to provide the mechanism by which these strategies are put to work. This idea is illustrated in the processes for drawing components and handling events.

Drawing

Before we go into the specific strategies, it is important to understand how they are used by the framework. The Large Display Framework is meant to be used together with a rendering API, such as OpenGL or Direct3D. This API should provide a rendering loop that calls a drawing function at every frame. In this drawing function, the application submits requests to the framework to draw all its current components via the renderAll() method in an instance of a subclass of the LargeDisplayManager (for an explanation, please see Part I of this tutorial). What happens in the framework then, is that the manager object will call a drawing function in every component in the composite tree in a pre-order traversal. None of these components is aware how they should be drawn, and so they will forward the drawing command to its associated strategy objects. This is done by calling their draw() method, which is defined in the interface of the VisComponentStrategy class (the superclass of every strategy class). It is this method that defines how the component should be drawn. If more than one of the strategy objects associated with a component implements the draw() method, drawing will be accumulative (i.e., one strategy object will draw on top of what other objects have previously drawn).

Event Handling

TODO: Ambiquous what 'the component should handle it' means in comparison to the other options

When handling events, the application creates an instance of the LargeDisplayEvent class, initializes it, and sends it to the framework manager (examples of this process can be found in Part I of this tutorial). The event object can be sent to the Manager in four different ways: it can be sent to the component that is currently selected by the user who produced the event; it can be sent indicating the component that should handle it; it can be sent to all components simultaneously (broadcast); or it can be set to a subset of the components (multicast). Each component that receives the event will forward it to its associated strategy objects. This is done via the onEvent() method. The event is then handled independently by each of the strategy objects based on the type identifier set in the event object data. If more than one of the strategy objects handles the same type of event, special care should be taken to ensure that the responses to the event don't conflict.

The VisComponentStrategy Class Interface

The strategies are defined by the VisComponentStrategy class. To create your own strategy, you should extend it and implement one or more of its virtual methods. Each of these methods is described below.

This method is called by the associated component during initialization. It builds and initializes the i-buffers used by the strategy. Additional initialization might be added here, if deemed necessary.

This method is called by the associated component at every frame just prior to calling the draw() method. Therefore, any kind of action that is supposed to occur repeatedly before drawing takes place should be included here. An example is the position update done when a component is tossed.

This method is called by the associated component every time the component should be rendered (i.e., every frame). It is where the actual drawing commands should be placed. The list of currently selected component IDs is received as a parameter. This list can be used to check if the associated component is presently selected by any user and, if so, draw it differently.

This method is called by the associated component when this component is being drawn in the back-buffer for picking purposes. In this method, all that should be drawn is the component's shape filled with the color obtained from the component's ID (by calling the component getIdColor() method).

This method is called by the associated component whenever it receives an event from the framework manager. The event object is given as a parameter and holds the identifier of the type of event to be handled. This method implementation will then check what the event type is, and decide whether to handle it or not, and how.

This method is called when the associated component is resized. It has as arguments the new width and height of the component. This information is usually important when the strategy holds one or more active (or private) i-buffers, since usually, when the dimensions of the component change, the dimensions of its active buffers should change as well.

This method is called by the associated component whenever this component is processed from the manager's update list (the list of components that should have their composite tree structure updated by the manager). It usually happens after the component is "dropped" (or released) by a user. The most common use of this method is to trigger a state animation for the component in the case it was dropped inside a container that affects its state. The Boolean parameter indicates if the "dropping" caused a change of the component's parent. The method should return true only if it has triggered an animation. Please note that any event might add a component to the manager's update list and, by consequence, provoke this method to be called. One example of this case is when a tossed component stops moving.

This method is called by the associated component when the manager needs to update the state of this component based on reading one or more of its passive i-buffers. This is the case, for example, when state animations are triggered for this component. The list with the identifiers of the i-buffer types to be read is passed as parameter. All that the implementation of this method has to do is to go through the list of i-buffer types and read the proper passive i-buffer at the component's current position and update the corresponding state attributes.

This method is called by the associated component just before processing (when the process() method is called) and drawing (when the draw() method is called) the component. The implementation of this method should read all the passive i-buffers of this component at its current position and perform the proper updates to its state.

This method is called by the associated component at the same point as the readAllPassiveBuffers() method, except that it happens only when the component is being rendered for picking purposes. The implementation is similar as for the referred method with the difference that it might not be necessary to read all the passive i-buffers of a component for picking rendering (this would be the case of a color buffer, for example). The component should be properly updated with the information read from the selected buffers.

Types of Strategies

The methods defining the VisComponentStrategy interface allow for a great variety of strategy implementations. Based on the purpose of these implementations, we can organize strategies into five categories.

Behaviors: These strategies provide an interactive behavior to its associated component. Examples include translation, RNT (Rotation 'N Translation), and tossing.

Containers: These strategies have properties (i.e., active i-buffers) that affect the components that are inside them (their children in the composite tree). Examples of this type of strategy are friction surfaces and interface currents.

Observers: These strategies allow its associated component to respond to a certain type of i-buffer or container. Basically, they specify one or more passive i-buffers that the associated component should be aware of. Then, they constantly "observe" for the values of these passive i-buffers based on the component's position and perform the proper updates. Some examples are observers for scale, rotation, and interface current containers.

Drawers: These strategies define how the associated component should be drawn. They can be responsible for drawing either the core of the component or "decorations" on top of the previous drawings. Examples include images, borders, and resizing handles.

Widgets: These strategies provide its associated component with the capabilities of an interface widget. This allows a component to behave as a button or as a "garbage bin", for example.

Examples

Now that we know what strategies are, how they integrate with the framework, and what they offer to us, we can take a look at some examples and get ready to implement our own strategy classes.

Example 1: A Quad

Our first example is a drawer strategy that simply draws a quad filled with a color. We start by creating a class that extends the VisComponentStrategy.

Since all we want to do is to draw a quad using the component's dimensions, we are going to implement only the draw() and drawForPicking() methods of the VisComponentStrategy interface (the other methods will remain with their default empty implementation). Using OpenGL, the draw() method implementation looks like this:

This draws a red quad centered at (0, 0, 0) and with the same width and height as the associated component. Although this is a simple case, it illustrates some important elements. First, in a strategy class, we can directly access the associated component via the 'component' attribute (it is a pointer to a VisComponent object). We use this to get the current values for the width and height of the component. Second, we don't do anything regarding positioning or scaling or orienting what we are drawing. We simply consider our component to be centered at (0, 0, 0). This is only possible though, if we are creating our components using the VisComponentGL class (as in the example in Part I of this tutorial). The VisComponentGL class will do all the linear transformations (scaling, rotation, and translation) for a component once in the beginning of its rendering process. In this manner, these transformations don't need to be redone and undone for every drawer strategy associated with the component. This can provide a significant performance improvement for components that have multiple drawer strategies and are drawn many times per frame.

Let's say now that we want to draw a component in blue if it is selected. We can then use the list of selected component IDs that is passed as a parameter of the draw() method and check if the associated component is selected. This changes the previous code to this:

We added a simple check to see if the associated component's ID is in the list received in the parameter. If it is, we use blue to draw the component. Otherwise, we draw it in red, as before.

The drawForPicking() method for our quad is similar with the exception of the filling color. It can be implemented as follows:

The color used for picking is the one calculated from the ID of the component. It is obtained using the getIdColor() method.

And that's it. We have a strategy that will draw a component as a quad. Using it is identical to using any other strategy, as demonstrated in Part I of this tutorial.

Example 2: Translation

In our next example, we consider a behavior strategy. For simplicity sake, we will create a strategy that simply translates a component when it is pressed with a pointing device, dragged to another position, and released. The actual pointing device being used doesn't matter. The important element is that the application should be providing events of the type that are expected in this strategy.

The first step is to create a class that extends the VisComponentStrategy class. We can call it MyTranslationStrategy (to avoid using the same name as the TranslationStrategy given in the toolkit package). For this class, we only need to worry about the onEvent() method.

The onEvent() method can be given as this:

Let's look at this method implementation in more detail. The first thing we do is to check if the event object we get as a parameter is not null. If it is valid, we get the identifier of the event type. We also get a reference to the ConstantProvider singleton instance, since it is where the type identifiers are dynamically defined. Then, we start comparing the type of the event we are handling with the types that we care about. If it is a "PRESS" event, we verify if the event coordinates are inside the associated component bounds. If they are inside, we set the 'pressed' flag on and update the coordinates of the previous position with the current coordinates (these attributes should be properly declared in the class definition and initialized in its constructor). If the event is a "DRAG", we move the component's branch (i.e., the branch of the composite tree which this component is the root for) by the difference between the current and previous event coordinates and update the previous coordinates. Finally, if the event is a "RELEASE", we just set the 'pressed' flag off.

The source code for this strategy can be found in the 'TutorialCode' subdirectory in the 'LargeDisplayFramework' innernet directory.

Example 3: Scaling Container

Our next example shows how a container strategy is created. As in the other examples, we want to keep it simple, so we will implement a strategy that provides a container with the capability of linearly scaling the components that are placed inside it. For this, we will use an active buffer that we will refer to as the scale buffer.

The IBuffer template class provides the data structure for generic interaction buffers (for more on interaction buffers, see Isenberg et al., A Buffer Framework for Supporting Responsive Interaction in Information Visualization Interfaces, 2006). An instance of this class is encapsulated in the BufferProxy class together with the type of the buffer and a reference to its owner (i.e., the component that has it as an active buffer). Each component has a list for its active buffers and another for its passive buffers. These lists contain references to objects of the BufferProxy class.

Let's name our class as ScalingContainerStrategy. We can leave drawing to another strategy to be associated with this container (maybe a QuadStrategy, as above). Then, we have to implement only two methods: initProperties() and resize().

In the initProperties() method, we create an i-buffer for our scale buffer, set its type, and add it to the list of active i-buffers of this component:

We can see from this code sample that we first create an IBufferProxy object and set its type as a scale buffer. Then, we create the i-buffer object and set its ID with the same value used for the buffer type (we don't actually need buffer IDs for this simple application) and its dimensions (real and virtual – please see the documentation of the IBuffer class for more details) as the component's dimensions. We fill the buffer by calling the fillScaleBuffer() method, which is given below. Finally, we set this i-buffer to the IBufferProxy object we created before and add it to the list of active i-buffers of this component.

As filling an i-buffer can be a quite complex task, it is considered a good practice to have a separate method for filling our buffers. For our scale buffer, we can have a method like this:

This method fills the scale buffer so that all the components that read from it will be scale to half their size. In this case, our filling method is very straightforward, but, as components get more sophisticated, it can become much more complicated.

We also have to implement the resize() method, since we want to properly resize this buffer as the component is resized so that their dimensions remain consistent. The code for this is given below:

To resize the i-buffer, we first need to get a reference to it. This is done by retrieving the IBufferProxy object from the component's list of active i-buffers whose type matches the type of our scale buffer. If we get a valid reference, we reset the dimensions of the i-buffer with the new dimensions of the component. Then, we just need to fill the buffer again by calling the fillScaleBuffer() method. The source code for this strategy can be found in the 'TutorialCode' subdirectory in the 'LargeDisplayFramework' innernet directory.

Example 4: Scale Observer

Complementing the previous container strategy, we should provide a corresponding observer strategy, so that components can react to the container. Hence, we will create a ScaleObserverStrategy class that will set the previous scale buffer as a passive i-buffer of its associated component and will update the component's scale factor based on the values read from the i-buffer. For this class, we will implement the initProperties(), drop(), readPassiveBuffers(), readAllPassiveBuffers(), and readAllPickingPassiveBuffers() methods.

First, we have to set the scale buffer as a passive i-buffer of the component associated to this strategy. This is done in the initProperties() method:

This method creates an IBufferProxy object and set its type as the same as the type of the scale buffer we used for the ScalingContainerStrategy class. We don't need to specify a buffer reference (we set it as NULL), since the framework will update it with the reference to the proper i-buffer at runtime. Then, we add the IBufferProxy object to the list of passive i-buffers for this component.

The next method we need to implement is drop(). That's because we want to have a smooth animated transition between the original dimensions of the component and the dimensions resulting from the insertion into the container. For this, in the moment the component is "dropped" into the container, we have to trigger the state animation. We can do it like this:

We start by getting the parent of the component associated with this strategy. If it has a parent and the parent has at least one strategy associated with it, we proceed by checking if any of these strategies is an instance of the ScalingContainerStrategy class. In other words, we verify if the parent is a scaling container. Now, we check if during the "dropping" the parent of the component changed. If it did, we create a std::vector to hold the types of the i-buffers that should be read for the animation. In this case, that means only the scale buffer. Then, we start the animation by calling the animateAdding() method in the associated component. We pass as parameters the list of types of the i-buffers that should be read for the animation and the ORed identifiers of which state information should not be animated (in this case, the position and rotation angle of the component). After this, we set the 'added' flag on, indicating that an animation was triggered. The method ends by returning the value in the 'added' flag.

The three remaining methods, readPassiveBuffers(), readAllPassiveBuffers(), and readAllPickingPassiveBuffers(), are straightforward. They are given below:

All of these methods read the scale buffer via the readScale() method. For the readPassiveBuffers() method, a simple pass over the buffer types received in the parameter checking them against the scale buffer type is done for properly accomplishing what is expected from this method. Notice that the readAllPassiveBuffers() and readAllPickingPassiveBuffers() methods are identical since the dimensions of the component are important for both drawing and picking it.

Let's then see the readScale() method:

This method starts by getting the IBufferProxy object in the list of passive i-buffers of the associated component whose type matches the type of the scale buffer. Then, if this object is valid and has a valid i-buffer reference, we convert the coordinates of the position of the component (which are given in global coordinates) into the coordinate frame of the scale buffer (i.e., the coordinate frame of the container which owns the scale buffer as an active i-buffer). Then, we check if the result of this conversion is inside the scale buffer dimensions. If it is, we obtain the value at the calculate position and use it to set the component's scale factor.

Alternatively, we could have a shorter (but less elucidating) version of this method as follows:

In this version, we use the component's template method getPassiveBufferValue() to abbreviate the process of getting the passive i-buffer, converting the coordinates, and checking boundary conditions. This might not always be desired, therefore we support both ways in the framework.

The source code for this example can be found in the 'TutorialCode' subdirectory in the 'LargeDisplayFramework' innernet directory.

Conclusion of Part II

This concludes this Part II of our tutorial. By now, you should have a pretty good idea of how to create your own large display applications using our framework. If you have any questions, suggestions, critics, bugs, comments, or any other feedback, please feel free to drop me an e-mail (fabricio AT cpsc DOT ucalgary DOT ca).

Good luck!

Acknowledgements

Thanks to Julie Stromer for the excellent feedback on the early versions of this tutorial.

Fabricio Anastacio

Calgary, October 29, 2007
Last Review: November 16, 2007