Shadow Classes and the Sins of the Father
By Ray Quay

Learning Object Oriented Programming often seems more like a personal reevaluation than a science.  Clearly the concepts of OOP are a science but how deeply you in brace OOP is more often a matter of personality than an objective decision. Opps, no pun intended.  This is not because OOP is a fad, but rather becuase it is a new paradigm.  Replacing one paradigm for another is always a difficult personal task.  Paradigms are the filters we use in this high volume information age to weed out the wanted from the unwanted.  The deeper a paradigm is ingrained into our psyche, the better it works.  This can be beneficial because paradigms play a critical role in our thinking and productivity.  Without these powerful filters, most tasks would be impossible.  Imagine how hard it would be to write a program if every time you sat in front of the keyboard, you had to relearn where all the keys were located.  However, our paradigm while helping us filter unwanted information can occasionally filter out information that could lead to a solution to a particular problem.  This does not mean we want to abandon our paradigm, after all it works 99% of the time.  Rather we need to be wise enough to recognize this 1%, step around our paradigm and look at the problem from a different viewpoint.  This article is about just such a problem.  It is about inheritance and the sins of the father.

One of the basic concepts of OOP is inheritance.  To the OOP programmer this is OOP 101, so bear with me while I build my case.  Inheritance allows one to create a class that is a descendant of an existing class.  This new descendant class can inherit all the functionality of the ancestor class.  This inheritance occurs without writing any additional code, and will be provided to an instance of this descendant class by the compiler.  To this inherited functionality, new functionality can be added by adding new code.  If the behavior of a particular class' method is undesirable, then a descendant to the class is created and the method modified for this descendant.  If some error is found in an ancestor class, then the error can be corrected in this ancestors source and the correction via inheritance is available to all classes.

This concept of inheritance is central to the OOP paradigm, and provides part of the basis for OOP's power of reuse.  It also provides a different view of how we plan to solve a particular  problem.  Rather than seeing the solution as a linear landscape of program control or a wall of blocks one on top of the other, this view is more like a tree with branches leading to branches.  This is one aspect of the paradigm that is hard to grasp initially, but once embraced becomes a very strong filter.

In a perfect world, the concepts of inheritance would give the programmer a great deal of flexibility in modifying, extending, and building Class libraries   However, most of us do not work in a perfect world and there are occasions when inheritance can not solve our disatisfaction with the behavior of a Class.

Turbo Vision, which is an event driven application framework that comes with Turbo Pascal and Borland C++, can be used to demonstrate this problem.  Turbo Vision has a root class called TView (actually the root class is TObject but it is so basic we will ignore it here and only deal with TView's ancestry).   From this are several branches or descendant classes that provide much of Turbo Vision functionality, TProgram, TWindow, TFrame, TMenu, etc.  These are the objects used to build a user interface.  

For this example, lets assume you are developing an application for a client.  You have developed a prototype using Turbo Vision that is basically a user interface shell with most of the menus and some dialog boxes implemented.  You show it to the user, and together you begin identifying design changes.  While looking  at it the user decides that he has to have the top frame of all the windows and dialog boxes to be solid, so it looks more like Microsoft Windows.  What do you say? 

You could tell the client no, that can not be done.  He will likely not know whether it can or can not be done, and this is after all no simple request.  It is not just a matter of creating a descendant of the Twindow class and overriding the draw method.  The frame is actually a class of its own (there is that pun again) called TFrame.  You can not just create a new descendant of TFrame, because Twindow's initialization code creates an object of the TFrame class, and will not use your descendant class.  You could create a new class of Twindow that overrides the init method of Twindow to use your new TFrame class.  This would still not change the dialogs.  The TDialog class is a descendant of the Twindow Class, not your class.  You could create a new class descendant of TDialog and override its init class to use your new TFrame class.  We now have three class descendants required to modify the behavior of one class.  Of course, any other classes you created that use Twindow or TDialog will also have to be modified.  All very messy and time consuming.  Not a desirable option.  

You could modify the source code for Turbo Vision.  Just go into the TFrame method and modify the draw method.  Then recompile Turbo Vision.  That will not be to hard, but you will have to keep a separate copy of this Turbo Vision source and compiled OBJ or TPU files set aside for this project.  After all not everyone wants solid window tops.  If you are providing a library of routines for others to use, they will now have to use your TFrame.  If they have made modifications to the Turbo Vision RTL this will get messy very fast.  This could all become a big headache and not very desirable.

Just say no, that is simple.  Keep in mind, though, as soon as your client sees PC Tools or Norton Utilities for DOS he will see a solid bar at the top of the windows.  Then he is going to wonder why he hired an incompetent or lazy programmer.  That is not going to be very desirable and if your client happens to be your boss this may become more than undesirable. 

The problem is that we can not use inheritance in this case to get rid of the sins of the father (or mother, lets be fair in assigning blame).  If there was some way at run time to selectively change the behavior of a specific class, the problem would not be so hard.  Unfortunately, there appears to be no easy solution to this problem within the framework of the inheritance paradigm.  So as we must do with any problem where our paradigms restrict the ability to develop a solution, we must push the paradigm aside and look at something different.

A similar problem exists with IBM PC compatibles.  The PC compatible has conventional RAM addressed up to A000.  After that point, addressable space may point to a variety of things, video RAM, ROM, or nothing.  What is addressable beyond A000 will vary from system to system.  Those with a CGA have something from B800 to C000, but nothing from A000 to B800.  Machines with no other cards may have nothing from C000 to F000.  Those that have other cards may have small windows of ROM locations between C000 and E000.  Some machines with ultra compact BIOS ROM may have nothing from F800 to FF00.  As RAM cram became a problem on the PC, people begun to look for ways to put user accessible RAM at locations above A000.  Unfortunately, since addressing on 8088 technology was hardware controlled, hardware solutions were the only way to dynamically utilize space above A000.  Unfortunatley, this was not easily done with the confusion of boards that might be in any machine.  

Then along came the 386 and the concept of Shadow RAM.  With 386 addressing being dynamically under software control, thus giving dynamic control over what physical location software addresses were pointing.  This allowed the computer under software control to remap addresses that pointed to physically empty locations so they would point to locations that contained RAM.  The most popular thing to point to was more RAM.  This RAM was coined Shadow RAM.  Shadow RAM physically located somewhere else could be mapped into open addresses below DOS's 1 MB limit.  Thus came High RAM and the concept of loading programs dynamically into RAM locations above A000. 

This same concept can be used to solve our problem with inheritance.  If we slide aside our OOP paradigm we can look at the concept of classes and objects unobjectively (there is that pun again).  Though the OOP paradigm helps us to view objects and classes in an abstract non hardware specific manner, our code is still compiled into a specific and traditional machine language.  It is then executed as any other code, OOP or nonOPP would be executed.  At this level, the old paradigms of code development still apply and a solution to our problem exits. 

Fortunately, in most cases when we run in to this problem of inheritance we are working with virtual classes.  (If the problem is with a static method we are out of luck.)  Objects of a virtual class are more than just data.  Virtual objects also carry with them a reference to their code.  This reference is not an actual pointer to the code itself but a pointer to the Virtual Method Table for its class.  Every virtual class has a table in memory that contains references to the class' methods, one entry for each method.  Each entry is a pointer  to the code for each method.  The compiler when it encounters a method call, places in machine language the instructions to look up the methods' reference in this table and make an indirect call to the methods code using this reference.  At run time, when the method is executed, its reference is looked up in the method table and then called.  (A direct call would not allow polymorphism)  Here lays the answer to our problem.   If we can modify the Virtual Method Table of a class at run time, we can insert a reference to a new method, a shadow method.  At runtime, our new method will be called instead of the old.

This is quite easy to do.  First we create a new class, and shadow class.  This class is an ancestor of the class for the method we wish to modify.  We then redefine the method in question to change its behavior.  Here we use the paradigm of inheritance to help us out.  Since this class is a descendant of the original class, through inheritance it will have access to all the class' data and methods.  At runtime, we modify the table of methods for the original class by replacing the reference to the old method with a reference to our new method.   Now, any object that evokes this method, will then look up in the table of methods the reference to this method, which is now the new shadow method, and call it.  This method will function properly here because all references to class data will be into the object's data and  all references to other method's will be through the object's method table.

Using the technique of Shadow Classing to our original problem is fairly straight forward.  We create a new class as a descendant of Tframe.  We modify the draw method to draw the frame as we desire.  We then insert in our code before we begin our application a procedure to swap the address of this routine for the original TFrame.Method address.  No other changes are required.  Now every time any object makes a virtual call to the TFrame.Draw method, our new method will be executed, drawing a solid top.

A simple solution that requires one new class.  The behavior will be changed for all objects of this class (but will not change the method for descendant classes, which opens up other possibilties).  The modification will be unique to this project so there is no need to keep multiple version of TVision.  And you may keep your job. 

Shadow classing is not perfect and there are limitations and pitfalls to its use.  This concept will work only with languages that implement polymorphism through the use of method tables.  Shadow Class Methods can not be inherited (With Borland Pascal 7, Dynamic Method Tables were introduced which would allow shadow methods to be inherited.)  As mentioned earlier, some OOP languages, like Turbo Pascal, allow static classes and static methods in Virtual Classes.  With static methods an entry for the method is not inserted into a table, rather the compiler inserts an actual call to the method reference.  The prevents using shadow classing to solve our problem for static methods.  

Some languages have an optimization to insert actual calls to a method without a table lookup.  This provides faster code execution than a table lookup and a call.  In Turbo Pascal and Borland C++ (and early C++s) this can be accomplished by fully referencing a method call using the Class name.  For example, with Turbo Vision's TFrame class, you could call the draw method by using either an unreferenced call, such as DRAW, or a reference to an object such as MYOBJECT.DRAW or @SELF.DRAW.  In both of these cases the compiler inserts a table lookup and then a call.  However, if you make a call referenced by the Class' name, such as TFrame.draw, the compiler looks up the reference and inserts a direct call in the code.  This is because in the latter case, it is known that a polymorphic call will not be required.  In this case of a call referenced by the class name, if a shadow class method had been inserted in the table at run time, the original not the new method would be called. 

The tools required to implement  Shadow Classes for most OOP languages consists of a routine that provides access to a class' method table, a routine to reference a particular method's location in the method table, and a routine to replace a method's reference with another.

The implementation of Shadow Classes in Turbo Pascal version 5.5, 6,  is relatively simple and can be done with just 2 simple routines and less than 50 lines of code. (An implementation is possible for version 7, but must account for the new structure of the VMT and the complexity that dynamic methods add to the  class's method table.) The Turbo Pascal OOP language provides some basic support in dealing with a class Virtual Method Tables (VMT).  First, it is relatively easy to get a pointer to a virtual class VMT or a virtual object's VMT using the typeof() function.  The typeof() function when used with an object, or a class identifier will return a pointer to the objects or class VMT.  For example typeof(MyObject) will return a pointer to the VMT for the class that MYOBJECT belongs. When used with a class identifier, typeof() returnsa pointer to the VMT of the class data type. It is this pointer that will give us access into the Method Table.

Next we need a way to find a way to determine which slot the method we wish to replace is occupying in our table.  This is fairly easy.  Since each slot occupies a pointer to our method code, all we need to do is to search the table till we find a pointer that matches the pointer to method we wish to replace.  Versions 5.5 and 6 provide no means to tell how many entries are in a virtual method table, so we will set an arbitrary limit of 192.  This is how many entries some of Turbo Visions largest VMT tables have, which are some of the biggest I have seen, so this seems safe. 

Now all we need is an understanding of the VMT table structure and we are all set.  The VMT has a fairly simple structure that can be described with the following data structure:

 { Basic VMT structure }
  const
    MaxVMTPointers = 192;
  type
    PVMT = ^TVMT;
    TVMT = record
      Size,NegSize:word;
      Table:array[0..MaxVMTPointers] of pointer;
      end;

The first word entry in the VMT structure provides the size in bytes of the objects of this class.  This is NOT the size of the VMT but rather the size required to store all the data of the object and a pointer to the object's VMT.  This is used by the extended syntax of the new() procedure to allocated sufficient heap to hold an instance of the object.  This actual size value is not needed for our purposes except that the second word in the VMT structure is the negation of this size value.  This provides us a way to check if we have a pointer which actually points to a VMT.  The sum of these to words should be zero and if not, then it is not a valid VMT.  This is actually quite useful to us.  Later in our routine we are going to be change memory values, and we do not want to be working accidentally in some critical memory area.  The result will likely be a hard crash.  Since we are going to be using pointers to VMTs, this provides us some level of security that these pointers are correct.  The next part of the VMT is simply a table of 32 bit pointers, each pointing to the entry point of a method.

The technique of inserting our method is fairly simple.  If we know the entry point to the method we want to replace, we can simply search through the VMT's table of method entry points looking for one that matches our known entry point.  Once found, we simply replace it with the entry point of our new method.

Listing 1 is a Turbo Pascal unit that provides three simple functions that can be used to create shadow classes.  The first function VALIDVMT():BOOLEAN provides a check to see if our VMT is a valid VMT.  FINDMETHODSLOT():POINTER finds where our method is in the Class' VMT, and REPLACEMETHOD():BOOLEAN; provides the actual function that replaces the OLDMETHOD with the new shadow method.

Listing 2, 3, and 4 are Turbo Pascal examples of using these tools to create Shadow Classes.  Listing 2 shows a very simple example of replacing a Class method with another method and is provided to show implementation at it most basic level.  

Listing 3 is another simple but actually useful example for Turbo Vision users.  This is a good example because it demonstrates very clearly the utility of shadow classing.  This example fixes a "bug" in TV's TMenuBar class.  The bug occurs when you try to insert a menu into some view other than the Desktop.  The bug is actually caused by the TMenuBox class.

TMenuBar objects insert a TMenuBox object into the Owner of the TMenuBar object to display submenus.  This works fine when the TMenuBar object's owner is the desktop.  TMenuBox's palette is mapped to the desktop palette.  However, when a TMenuBar object is inserted into something other than the desktop, the palettes do not match. A descendant of TMenuBar can be created that uses a palette that matches where it is inserted, but TMenuBar will still insert a TMenuBox object with a palette matched to TDesktop not to the actual owner where the MenuBar is inserted.  This basically blocks any attempts by the user to control the palette of the submenus.

This behavior of the TMenuBar Class can be corrected by patching the GetPalette method of TMenuBox Class.  The unit NewMenu provided in Listing 3 is a shadow class for TMenuBox.  It modifies the behavior of the TMenuBox class' GetPalette Method.  The new shadow GetPalette method returns the Palette of the ParentMenu if there is one, and if not then it returns the normal Palette of TMenuBox.

Listing 4 provides a working example of this Shadow TMenuBar.

Though Shadow Classing does pose one solution to some "sins of the father" OOP problems, that should not be the conclusion of this article.  The conclusion should be that Shadow Classing solves an OOP related problem but stepping outside the OOP paradigm.  It does so very gently by still relying on OOP for its functionality but it still pushes beyond the veil of OOP.  There are other problems we will face as we implement the real world in the OOP paradigm whose solution will also lie beyond this veil.

Listing One

Listing Two

Listing Three

Listing Four