dlx.txt Proposed Expandable Dynamic Link Library specifications: Integrated Graphics 312 Nevada St. Northfield, MN 55057 hansonr@stolaf.edu THE BASIC IDEA -------------- Expandable DLL files (extension DLX) are dynamic link libraries that may be extended using a simple procedure at run-time. To be expandable, the library must consist of at least six sections: 1) a 64-byte (minimum) DOS executable header, 2) a 64-byte New Executable header, 3) a Resource Table containing pairs of entries: RT_GROUP and RT_IMAGE (one of each type, 12 bytes each, must be present for each icon), 4) a 12-byte Resident Name Table referencing the name "EXPNDABL", 5) a 2-byte empty Entry Table, and 6) a section for data referenced by the RT_GROUP and RT_IMAGE structures. All specifications for these sections are described below. The order is critical and must be as listed above. Any other information present (executable code, for example) must come PRIOR to the Resource Table and NOTHING except slack room ready for expansion (2x12 bytes per icon added) may appear between the Resource Table and the entry table. Basically, when an icon is added to the library, the Resource Table must expand by adding one new RT_GROUP name entry and one new RT_IMAGE group entry. The Resident Name Table is used as a reference to the end of the Resource Table; the Entry Table marks the end of the free space in the file. Thus, two pointers, one to the Resident Name Table, and one to the Entry Table, delineate exactly how much room is available for easy addition of icons. After that space is filled, some data pointed to by the Resource Table needs to be moved to the end of the file. what happens is that icons are added by: a) appending the ICON data to the file (800 bytes, see below); b) extending the Resource Table by 24 bytes, making room for one new RT_GROUP and one new RT_IMAGE structure; c) rewriting the Resident Name Table just after the newly extended Resource Table; and d) updating the New Executable Header Resident Name Table pointer. The RT_GROUP structures come first. They take the form of Name Entries, each of which is 12 bytes and are described in detail below. As icons are added, the Resident Name Table pushes toward the Entry Table. Inspection of the New Executable Header allows a determination of how much room is available. If not enough room is present, then the following steps are taken: a) the first icon (800 bytes total) after the Entry Table is popped to the end of the line; b) the Entry Table (0000h) is rewritten just before the SECOND (now first) icon; c) the New Executable Header is revised to indicate the new Entry Table offset; and d) the Resource Table is looked through, and the appropriate RT_GROUP and its corresponding RT_IMAGE offset references are revised to point to the new ICON data positions in the file; That's a bit of work, but really it goes very fast, and since each icon is 800 bytes, that allows for 33 more icon entries in the Resource Table (2x12 bytes each) before the next icon needs to be moved. IconMover does it slightly more efficiently, calculating for a collection of icons how much room is needed, and then doing it all at once prior to adding any icons. Moving 800 bytes to the end of a file on any 80x86 machine using GET and PUT takes essentially no time whatsoever. HOW ICON MOVER DOES IT ---------------------- The exact steps taken by Icon Mover are as follows: a) determine the icons to be appended and append them, being careful to align all data along 32-byte boundaries due the 32-byte blocking factor; b) concurrently write pairs of RT_GROUP and RT_IMAGE NameEntry structures to a temporary file. (12 bytes each); c) do the calculation and move whatever icon data need to be moved to the end of the file; d) look through the Resource Table for data entries in the region that was moved and revise those entries; e) pad out the Resource Table to make room for the new entries; f) add the new entries from the temporary file; and, finally, g) rewrite the Resident Name Table (12 bytes) and the Entry Table (2 bytes) and update the New Executable Header to indicate their new offsets. The key is to this arrangement is simply that the DLL file conform to the above specifications. Not all DLL files do, because there is no specific requirement for the ordering of structures in a DLL file. Some DLL files are considerably more sophisticated, of course, and must include several other structures. In principle, even these DLL files could be made expandable by careful rearranging of their component structures. Icon Mover does not attempt that, however, as I learned all this empirically with the aid of PC MAGAZINE's ICONJACK program for reference (6/27/95 issue). PROGRAMING DETAILS ------------------ HEADERS It turns out that the headers can be considerably simpler than generally described. No one really needs a DLL file to say, "This program requires Microsoft Windows," for example. Thus, much of the header information can be ignored (left 00h). As far as I can tell, only the following aspects of the headers are strictly required: offset(dec) contents description 0-63 minimal 64-byte DOS header 0-1 "MZ" DOS executable header signature 60-61 0040 pointer to new header (byte 64 in this case) 64-127 minimal 64-byte "New Executable" header 64-65 "NE" New Executable header signature 68-69 * Entry Table Absolute Offset 70-71 0002 Entry Table length (indicating just 2 bytes) 100-101 0040 Resource Table Relative Offset 102-103 * Resident Name Table Relative Offset 126-127 030A Windows Version expected where contents are in hex and * means an integer (2-byte) pointer to an address offest in the file (first byte is 0 in this system). Only the Entry Table offset and the Resident Name Table offset change as icons are added to the library. Resource Table The Resource Table consists of a 2-byte shift count followed by blocks of TYPEENTRY and NAMEENTRY entries, followed by an ENDOFTABLE mark: SHIFTCOUNT TYPEENTRY1 NAMEENTRY1a NAMEENTRY1b NAMEENTRY1c etc... TYPEENTRY2 NAMEENTRY2a NAMEENTRY2b NAMEENTRY2c etc... etc... ENDOFTABLE SHIFTCOUNT is an integer n, where 2^n is the "alignment factor" by which all offsets in the Resource Table must be multiplied to get the absolute file offset of an icon directory or image. In my implementation, this is 05h, indicating that all data are aligned along a 32-byte boundary. Thus, an offset of "10h" is really at file offset 16x32=512 (200h). ENDOFTABLE is a two-byte marker, 0000h. Each entry is a structure of 8 bytes (TYPEENTRY) or 12 bytes (NAMEENTRY): Type TYPEENTRY id As Integer RT_GROUP = 800Eh, RT_IMAGE = 8003h count As Integer how many name entries reserved As Long 000000000h End Type Type NAMEENTRY offset As Integer from beginning of file, NOT relative to the NEH, this is the true file offset divided by the alignment factor, 32 in my implementation. length As Integer 32 for an icon directory entry (RT_GROUP); 768 for a bitmap image entry (RT_IMAGE) Flags As Integer 1C30h for RT_GROUP; 1C10h for RT_IMAGE id As Integer 8000h + (index of this name entry, starting with 1) handle As Integer 0000h usage As Integer 0000h End Type In my version of the DLX file, this table starts at 80h (offset 128). It's pointer in the file at byte 100 (NEH offset + 36). In my DLX files, this reads "40h", reflecting that this pointer is relative to the beginning of the New Executable Header, 40h(40h + 40h = 80h). Any additional segments, data, or otherwise may appear in the DLX file between the New Executable Header and the Resource Table, however none are needed. In addition, the RT_GROUP and RT_IMAGE entries should be the last in the table if other entries exist. RESIDENT NAME TABLE The Resident Name Table is a single 12-byte entry indicating that this is an expandable DLL file: 0 08h length of name, eight bytes 1-8 "EXPNDABL" 9-11 00-00h end-of-name and end-of-table marks A pointer to this table must be in the New Executable Header at file byte 102 (NEH offset + 38). It, like the pointer to the Resource Table, is an offset relative to the beginning of the New Executable Header. In these DLX specifications, the Resident Name Table must IMMEDIATELY follow the Resource Table, and there must be only unused storage space after it. ENTRY TABLE There is no true entry point for a DLX file. This table is simply two bytes, 0000h, situated just prior to the icon resources themselves (the directories and images pointed to by the NAMETYPE entries of the Resource Table). Thus, the Entry Table simply marks the beginning of the data which may be pushed to the end of the file when more room for the Resource Table is needed. An integer pointer to this table must be present at byte 68 of the file (NEH offset + 4). This pointer is an ABSOLUTE offset, not relative to the New Executable Header. ICON DIRECTORY The RT_GROUP name entries point to individual Icon Directories, which themselves are 22 bytes, padded to 32 bytes by 00h to align them along a 32-byte boundary. These directories are essentially what is at the beginning of all ICO files: 0-1 0000 reserved 2-3 0001 type, must be 1 4-5 0001 count, must be 1 for DLX; could be more in other DLL files 6 20 width of image (32 bytes = 20h) 7 20 height of image 8 10 number of colors (repeated in bitmap data) 9 00 reserved 10-11 0001 number of planes (1; all icon bitmaps are single-plane) 12-13 0004 number of bits/pixel; this depends upon the number of colors, and is repeated in the bitmap data 14-17 00000280 size of the image (640 bytes) 18-21 000080nn POINTER TO RESOURCE RT_IMAGE ENTRY; nn = 01, 02, etc... 22-31 00-00 padding to fill 32-bit boundary. IMAGES Finally, the image data pointed to by the RT_IMAGE name entries must conform to standard 16-color bitmap structure, consisting of a 40-byte header, a 16-color color table (64 bytes), an XOR (color) bitmap (512 bytes), and an AND (monochrome) mask (128 bytes), for a total of 744 bytes, padded to 768 bytes (32 x 24) with an additional 24 bytes: 0-3 00000028 size of this header (40 bytes) 4-7 00000020 width of image (32 for icon) 8-11 00000040 height of image (64, because it includes an AND mask; for a BMP file, this is 00000020 12-13 0001 number of planes (must be 1) 14-15 0004 bits/pixel (1 for monochrome, 4 for 16-color, 8 for 256-color, 24 for 24-bit true color.) 16-19 00000000 compression type; 0 for icons 20-23 00000280 size of image, 280h for an icon; 200h for a 32x32 16-color bitmap (which lacks an AND mask) 24-27 00000000 designed X pixels per meter 28-31 00000000 designed Y pixels per meter 32-35 00000000 0 or actual length of table space in file 36-39 00000000 0 or number of colors important 40-103 whatever color table (64 bytes, four bytes per entry) 104-615 whatever XOR bitmap, starting with the last line of the table, indicating (usually) the 4-bit or 8-bit pointer index to the color table 616-743 whatever AND bitmask 744-767 00-00 padding The function of the AND mask is to indicate which pixels to blank prior to writing the icon, and which to leave unchanged. Generally a 0 in the AND mask means "put my color here" and a 1 in the mask means "leave this color alone." Generally a 0 in the AND mask corresponds to a pixel with color in the XOR bitmap; a 1 in the AND mask best correlates with a request for black, 000000h, that is, to whichever index of the color table hold the long hex value 00000000. Note that 744 + 22 (for the icon header) gives 766, the size of a standard icon file. Indeed, an ICO file is just the above 22-byte icon header (without the padding) followed immediately by the bitmap header, color table, XOR bitmap, and AND mask. In the case of ICO files, the image data pointer at bytes 18-21 reads "00000016", indicating that the image data follows the icon header immediately at file offset 22. Note that this AND mask, though just a string of bits 0001001000110100.... must be laid down carefully, considering the way that bits of numbers are laid down on a PC, rearranged in 16-bit chunks: binary: 0001 0010 0011 0100 hex: 1 2 3 4 is actually 3412h, not 1234h. Nonetheless, the AND mask DOES read left-to-right, with no rearranging. BMP files, on the other hand, contain only the bitmap header (40 bytes), color table (64 bytes), and the XOR table (512 bytes for 32x32). They have their own special 14-byte header as follows: 0-1 "BM" file type indicator 2-5 00000276 total file size (630 bytes for 32 x 32) 6-7 0000 reserved 8-9 0000 reserved 10-13 00000076 file offset to XOR image Thus, a BMP file corresponding to a 32x32 16-color icon is 14+40+64+512 = 630 bytes. Note that the AND mask is lost and must be reconstructed by the user if they are making an icon from a bitmap image. This is why in Icon Mover an option exists to make one color transparent. That color is set to black in the color table, and all pixels referencing it in the AND mask are set to 1. Without this option, all icons made from bitmaps would appear with a square background rather than the spiffy "cutout" image common to icons. SUMMARY ------- This DLX specification allows for icons to be stored in dynamic link libraries which can be expanded to fit, read by the Program Manager, and targeted by LOADICON or EXTRACTION functions within the Windows Application Programming Interface. They are really just simplified DLL files with a certain convenient structure. The essentials, once again, are as follows: 1) Resident Name Table must immediately follow Resource Table and it must contain only one entry, "EXPNDABL". 2) Resource Table entries RT_GROUP and RT_IMAGE must be in that order and be last in the table if others appear. 3) Entry Table must follow Resident Name Table, with space preferably allowed for expansion of at least a few icons. 4) Only resource data that can be moved (i.e. referenced only by the Resouce Table) should follow the Entry Table. All this really means is that if the library contains any other information other than resource data, that information must reside between the New Executable Header and the Resource Table. The 32-byte alignment factor means that offsets up to 32*2^16-32 = 2,097,120 may be referenced (about 2 Mb). At 824 bytes per icon (800 bytes of aligned data + 24 bytes of unaligned Resource Table space) less overhead for the various DLL tables (64+64+6+12+2=148 bytes), this allows for 2544 icons to be stored in a single DLX file. Perhaps there are slicker, already defined, ways of expanding a DLL file, but I don't know of any. Then again, I don't know A LOT! Give me a buzz (hansonr@stolaf.edu) if you know better how to do this.