CDB Copyright (C) 1991 by Daytris All rights reserved Revised: May 20th, 1991 INTRODUCTION Overview CDB is a sophisticated database toolkit for MS-DOS and UNIX developers. CDB includes the following features: - Quick data access through a sophisticated multi-key ISAM implementation. - Multiple data models. Both relational and network data models are implemented in CDB. The network model gives the developer the ability to create relationships between records without storing unique keys in those records. - Data Definition Language (DDL) for defining database layouts. The DDL is compiled into a binary format which is used by the database server as a roadmap. Using this concept, a developer can define and re-define a database with minimal effort and absolutely no code changes. The DDL is patterned after C for ease of programming. - Over 30 predefined database function calls for complete control of the database. - Portability. CDB is written entirely in C for portability and source code is included with your purchase. A version of CDB is also available for the MS-Windows platform. Contact the developers for more information about this product. - C++ compatibility. The library is callable from both C and C++. - Automatic re-use of deleted database space. There is no need for database reorganization. Deleted space is automatically reused by CDB. - Low overhead. The CDB database engine requires an extremely small amount of memory to operate. - Royalty-free distribution rights. Whether you have one customer or thousands, you pay for CDB just once for each environment that you are using. Obtaining CDB Refer to the document included in this release, ORDER.TXT, to order CDB. Your purchase will include CDB libraries, utilities, royalty- free use of library functions, full library and utility source code and make files. A printed manual will also be included. Future Enhancements Listed below are some of the enhancements planned for CDB. Any suggestions would also be greatly appreciated. - Multi-user database access. Networks and/or protocols to be supported are currently undefined. - Performance enhancements. Contacting the Developers CDB was developed solely by Daytris. If you have a question about the product or any suggestions please contact us at the phone number listed below. Or if you prefer, you can send us electronic mail on any of the information services listed. Daytris 81 Bright Street, Suite 1E Jersey City, NJ 07302 201-200-0018 CompuServe: 72500,1426 BIX: daytris GEnie: t.fearn GETTING STARTED Unpacking CDB is distributed in self-extracting ZIP files. When you first unpack your software, you may want to verify that you have a complete set. If you have the Test Drive version, you should have the files included in CDB.EXE. If you have purchased CDB, you should have the files included in both CDB.EXE and CDBSRC.EXE. The contents of each are listed below. CDB.EXE File Name Description --------- ----------- CDB.TXT This document. ORDER.TXT Order form for CDB. REGISTER.TXT Registration form for CDB. README.TXT Latest CDB info. UPDATES.TXT History of updates made to CDB. MSCCDBL.LIB CDB library. Microsoft C large model. TCCDBL.LIB CDB library. Turbo C large model. DBDLIST.EXE Database Definition (DBD) file display utility. DDLP.EXE Database Definition Language Parser. TEST.C TEST do nothing program. TEST.DDL TEST Data Definition Language File. DBMGR.H CDB header file. MSCTEST.MAK Make file for TEST (Microsoft C). Microsoft NMAKE and UNIX make compatible. TCTEST.MAK Make file for TEST (Turbo C). Microsoft NMAKE and UNIX make compatible. CDBSRC.EXE File Name Description --------- ----------- LICENSE.TXT Daytris software license agreement. MSCLIB.MAK Library make file (Microsoft C). Microsoft NMAKE and UNIX make compatible. TCLIB.MAK Library make file (Turbo C). Microsoft NMAKE and UNIX make compatible. DBADD.C Library source file. DBUPD.C . DBDEL.C . DBFIND.C . DBGET.C . DBCURR.C . DBFILE.C . DBMGR.C . DBPAGE.C . DBSLOT.C . DBTALK.C . DBFUNCS.C . DBMGR.H CDB header file. DBXTRN.H External definitions header file. STDINC.H Header file that includes other header files used in CDB. DDLP.MAK Make file for DDLP. Microsoft NMAKE and UNIX make compatible. MAIN.C DDLP source file. DDLP.C . PARSE.C . ERROR.C . DDLP.H DDLP header file. DBDLIST.MAK Make file for DBDLIST. Microsoft NMAKE and UNIX make compatible. DBDLIST.C DBDLIST source file. THE NETWORK DATABASE MODEL Introduction CDB provides both relational and network model features. The use of both models in a data design can greatly increase the performance of your database. For those of you who are already familiar with the relational database concepts, you will find the network model implementation very refreshing. For those of you who aren't familiar with the relational model, a very brief description of relational concepts follows. The Relational Model In a relational database, data is stored in a series of tables. Each table consists of a number of columns, which identify a particular type of data, and rows, which correspond to a particular record in the table. Individual records can be retrieved using the key fields defined for the table. If the developer has the desire to make an association between two tables, unique key fields must be defined in both records and unique data must be stored for retrieval to take place. What is the Network Model? This model allows you to define relationships between records through constructs called sets. A set defines a one-to-many relationship between two tables. In a relational model, records can only be related (connected) by storing unique keys in both tables. This method creates additional unwanted overhead. Duplicate data is stored in both records and duplicate indexes must be managed. Using the network model, records are connected by directly storing data pointers inside the record. Where the relational model requires multiple disk accesses to locate a related record, the network model allows the record to be located in a single disk access. Disk space is also saved when sets are used because no index is required. Another advantage of using the network model is the flexibility of owner/member relationships. A record may own multiple record types. A record may also be owned by multiple owner records. An example of this would be a 'client' record owning 'invoice' records and also owning 'address' records. In turn, an 'invoice' could also own 'address' records, perhaps a shipping and billing address. This kind of flexibility gives the developer the power to define complex data relationships with relative ease. THE DATA DEFINITION LANGUAGE Introduction The Data Definition Language is used for defining a database model. The language is basically a superset of C structure definitions. If you are familiar with defining C structures, the DDL should be very easy to pick up on. The DbOpen function call loads a binary image of a DDL file. The binary image is created by compiling the DDL file into a DBD (Database Definition) format. A DDL compiler is included with this release, DDLP.EXE (Data Definition Language Parser). An Example: /* sampledb.ddl */ prefix ABC; struct client { connect address key street; key long clientnbr; key char name[31]; char description[61]; double balance; }; struct address { char street[31]; char city[21]; char sate[3]; key char zip[11]; key char telephone[13]; char fax[13]; }; struct setup { key long nextclientnbr; }; Notice the close resemblence to C structure definitions. The only differences are the prefix, connect, and key words. prefix The prefix is used internally by the database library when a new database file must be created. In the SAMPLEDB.DDL shown above, the prefix is "ABC". By defining the prefix as "ABC", we are telling the library to use "ABC" as the first 3 characters of any file that is created for the SAMPLEDB database. The prefix can be from 1 to 4 characters in length. If a prefix is not defined, a default prefix, "TEST", is used. For more information about the CDB database file naming conventions, refer to the 'Database File Names' section in this manual. connect When using the connect keyword, you are taking advantage of the network database model implementation of CDB. Network model concepts can greatly increase the performance and efficiency of your database. The connect keyword defines a relationship between two records. struct client { connect address key street; . . }; In this example, we are defining a relationship between the client record and the address record. The client record will be an owner of the address record. The address record is a member of the client record. For now, ignore the 'key street' part of the connect phrase. By declaring this set relationship, we now have the capability to make connections between client and address records using the DbSet... function calls. A client may own 0, 1 or many address records. Without the network model concepts that we have just shown you, to make connections between two records would require the storage of a unique key in each individual record. void Function() { static CLIENT client = {1000L,"Daytris","A software company", 0.00}; static ADDRESS address = {"81 Bright Street, Suite 1E", "Jersey City","NJ","07302","201-200-0018",""}; DbRecordAdd( "client", &client); DbRecordAdd( "address", &address); DbSetAdd( "client", "address"); } The example above shows how to make a set connection between two records by using the DbSetAdd function. After the function call, the client "Daytris" is the owner of 1 address record. This address record can be retrieved using the DbSetGetFirst call: DbSetGetFirst( "client", "address", &address); Now lets add another address record to the set: void Function() { long key = 1000L; static ADDRESS address = {"30 Broad Street","New York","NY", "10015","212-555-1212","212-555-1212"}; /* Make client #1000 current */ DbRecordFindByKey( "client", "clientnbr", &key); /* Add another member */ DbRecordAdd( "address", &address); DbSetAdd( "client", "address"); } The client "Daytris" now owns 2 address records. We can use the DbSetGetFirst, DbSetGetLast, DbSetGetNext, or DbSetGetPrev to retrieve any of the address records in the set. Now lets take a look at how the sets are ordered. If we make a DbSetGetFirst call after adding the sets shown above, which address record would be returned? Lets return to our original DDL example: struct client { connect address key street; . . }; Member records can be ordered two ways. By the order in which they are added, or by a field in the member record. In our example, the set order is by the street field in the address record. Therefore, a DbSetGetFirst( "client", "address", ...) call would return the "30 Broad Street" address record. If the connect address phrase were defined without a key: struct client { connect address; . . }; the address members would be stored in the order that they were added. Therefore, a DbSetGetFirst( "client", "address", ...) call would return the "81 Bright Street, Suite 1E" address because this address was added first. A record can have more than one member. A record can also be owned by more than one owner. To illustrate this, lets take the SAMPLEDB.DDL and expand it to include invoicing capabilities. /* sampledb.ddl - with invoicing */ prefix ABC; struct client { connect address key street; connect invoice; key long clientnbr; key char name[31]; char description[61]; double balance; }; struct address { char street[31]; char city[21]; char state[3]; key char zip[11]; key char telephone[13]; char fax[13]; }; struct invoice { connect address; connect invoiceline; key long invoicenbr; long date; double totalprice; }; struct invoiceline { long quantity; char description[31]; double unitprice; double lineprice; }; struct setup { key long nextclientnbr; long nextinvoicenbr; }; In this example, a client record can own multiple address records and multiple invoice records. This makes sense because a client could have more than one address, i.e. a shipping and billing address. The client could also have more than one invoice if more than one order is placed. Also in this example, an owner/member relationship exists between the invoice and address records. If our invoice has both 'ship to' and 'bill to' addresses, the shipping address could be stored as the first member in the set and the billing address could be stored as the next member. A variable number of line items could exist on an invoice. This is the reason for the invoiceline record and its relationship with the invoice. An invoice record will own its invoice lines. The following example illustrates how all data pertaining to a specific invoice might be retrieved. Note: it is suggested that the CDB return values be taken more seriously than illustrated below. typedef struct invoice INVOICE; typedef struct invoiceline INVOICELINE typedef struct client CLIENT; typedef struct address ADDRESS; void Function() { long key = 2000L; UINT status; INVOICE invoice; INVOICELINE invoiceline; CLIENT client; ADDRESS shipaddress; ADDRESS billaddress; /* Get invoice #2000 */ DbRecordGetByKey( "invoice", "invoicenbr", &invoice, &key); /* Get the client that owns the invoice */ DbSetGetOwner( "client", "invoice", &client); /* Get the 'ship to' and 'bill to' addresses */ DbSetGetFirst( "invoice", "address", &shipaddress); DbSetGetNext( "invoice", "address", &billaddress); /* Retrieve all invoice lines (assuming at least 1 line) */ status = DbSetGetFirst( "invoice", "invoiceline", &invoiceline); while( status != E_NONEXT && status != E_NOTFOUND) { /* Store the line */ /* Get the next line */ status = DbSetGetNext( "invoice", "invoiceline", &invoiceline); } } This example illustrates some of the capabilities that you have with set relationships. The possibilities are endless. key The key word is used for defining key fields in records and key fields to be used in set relationships. See the connect section directly preceding this section for more details about the key fields in set relationships. Key fields are stored in ascending order in slots on pages in a key file. The key file is made up of a series of linked pages. struct client { connect address key street; key long clientnbr; char name[31]; char description[61]; double balance; }; This DDL structure definition contains only one key field, "clientnbr". Therefore all pages in the corresponding key file will contain will contain slots of sorted client numbers. struct client { connect address key street; key long clientnbr; key char name[31]; char description[61]; double balance; }; The DDL structure definition now contains two key fields, "clientnbr" and "name". Therefore two types of key pages will exist in the key file for this record type. Some pages will contain slots of sorted client numbers and other pages will contain slots of sorted client names. The data stored on the key file pages is directly related to the number of key fields defined in the DDL file. To maximize the efficiency of your database, it is suggested that you use as few key fields as possible. The maximum number of key fields allowed in a record is defined as MAXKEY in DBMGR.H. It is currently set to 8. See 'Modifying the Database Internals' section for more information about MAXKEY. If a structure is defined without a key field, the only way to access a record of this type is with a set relationship. The structure defined without a key field must be a member of another record. struct client { connect address key street; key long clientnbr; key char name[31]; char description[61]; double balance; }; struct address { char street[31]; char city[21]; char state[3]; char zip[11]; char telephone[13]; char fax[13]; }; In this example, the address record contains no key fields. Therefore, the address record cannot be accessed using any DbRecord... function calls because these functions require a key field as a parameter. However, the address is a member of the client record. Therefore, it could be accessed with the DbSetGet... function calls, provided a relationship exists. DDL Limitations The Data Definition Language does not currently support the definition of structures or unions defined from within a structure. Example: struct client { struct address addr; key long clientnbr; key char name[31]; char description[61]; double balance; }; These deficiencies will be supported in a later release of CDB. A way to get around this problem for now is to allocate enough space as a char field for the structure or union that could not be included. Example (assuming the address structure length is 92 bytes): struct client { char addr[92]; key long clientnbr; key char name[31]; char description[61]; double balance; }; After DDLP compilation, modify the C header file output by DDLP to include the proper structure definition. DATABASE CURRENCY What is Currency? Currency refers to the record position in a database key file. It is very similiar to the file pointer in an open file. For example, when you first open a file using the C run-time library "open" function, the file pointer points to the first byte in the file (it could point to the last byte depending on how its opened). After the file is open, you can seek to different positions in the file and read or write data. The file pointer position is kept internally by the operating system. You could think of this position as the current position or "currency". In CDB, the concepts are very similar. Each record structure defined in a DDL will have an associated currency table when this database is opened. An Example: /* sampledb.ddl */ prefix ABC; struct client { connect address key street; key long clientnbr; key char name[31]; char description[61]; double balance; }; struct address { char street[31]; char city[21]; char state[3]; key char zip[11]; key char telephone[13]; char fax[13]; }; struct setup { key long nextclientnbr; }; When this database is opened using DbOpen, 3 currency tables will be initialized to zero. One for each record type: 'client', 'address', and 'setup'. The currency table contains the following format: struct currency_index { struct { UINT page; UINT slot; } keydba[MAXKEY]; /* Array of key dba's */ ULONG datadba; /* Data database address */ }; keydba The currency table consists of two parts; a key currency (keydba) and a data record currency (datadba). A keydba exists for each key defined in the record table. In the example defined above, the 'client' record would use the first two keydba structures in the currency_index table for key currency storage. Records that do not have any keys defined would not make use of the keydba part of the currency_index. Lets say that we have three 'client' records in our database. The contents of each are as follows: Record 1: 1000L,"Daytris","Software Development",0.00 Record 2: 1001L,"Microsoft","Software Development",10000.00 Record 3: 1002L,"CompuServe","Computer Services",100.00 When the database is opened, the currency_index for the 'client' record, as well as all other records, is null. In other words, "the client record does not have currency". If we were to issue a: DbRecordFindNext( "client", "clientnbr"); at this time, an E_NONEXT return value would result. There is no next record to find! However, if we were to issue a: DbRecordFindFirst( "client", "clientnbr"); the return value would be 0 indicating a successful call. After this call, the keydba structure within the currency_index for the 'client' record would contain the appropriate page and slot number of the first record for the "clientnbr" index. In this case, the keydba[0] structure within the 'client' currency_index would point to client number 1000L, Daytris. If we were to now issue a: DbRecrordFindNext( "client", "clientnbr"); the keydba[0] structure within the 'client' currency_index would point to the next client sorted by "lClientNbr". In our example, it would point to 1001L, Microsoft. Keep in mind that we are not retrieving any records, we are only setting currency for the 'client' record type. If we would want to retrieve a record, we would use the DbRecordGet... function calls. Using the DbRecordFind... function calls we can essentially "seek" to positions within the database based on any index field within a record type. datadba The datadba field in the currency_index is used for "set" currency. When we issue a DbSetFind.. function call, the datadba is used to locate the current set record. The datadba field contains the actual slot number of the current record. "Next" and "previous" set pointers are stored at the beginning of each data slot in a data file. Differences Between Find and Get Function Calls The DbRecordFind... and DbSetFind... function calls only set currency for a specific record type. They do not retrieve records. You may want to think of the Find function calls as performing the same task as the C run-time lseek function. Essentially, we are seeking to a position in the database. If you wish to retrieve a record, use the DbRecordGet... or the DbSetGet... function calls. Note: The DbRecordGet... and DbSetGet... function calls call their Find counterparts first, and then retrieve the current record. For example, the DbRecordGetFirst function will perform a DbRecordFindFirst and then a DbRecordGetCurrent function call. Storing Currency Tables You can retrieve a copy of the current currency_index for each record defined in the DDL. Why would you want to do this? Lets suppose that you have a database that contains hundreds of 'client' records. Your application must be able to display these 'client' records in a small window, but you don't have enough memory to keep all of the 'client' records resident. Or it may be a waste of memory to do so. This is where storing currency tables becomes necessary. As previously explained, each record type defined in a DDL has an associated currency table. The contents of a currency table can be retrieved or updated at any time. Therefore, in the example explained above, we could retrieve a window of 'client' records along with their associated currency_index tables. Example: UINT GetWindowOfClients( BOOL bFirstTime) { register short i; UINT status; struct currency_index currency; for( i=0 ; i