Title: RECIO DESIGN AND DEVELOPMENT NOTES Copyright: (C) 1994-1995, William Pierpoint Version: 2.13 Date: September 4, 1995 1.0 DATA STRUCTURES 1.1 REC structure for each record stream * defined in recio.h. * 3 (!dos) or 4 (dos) static RECs for recin, recout, etc (included in ROPEN_MAX count). * allocate dynamic array of RECs dimensioned to ROPEN_MAX-NREC in ropen(). * Each REC has two associated buffers: 1) record string buffer containing current record; allocate when first record read; reallocate if record becomes larger. 2) field string buffer containing current field; allocate when first field read; reallocate if field becomes larger. * deallocate dynamic RECs and associated buffers in rclose() and rcloseall() if all record streams closed; deallocate associated buffers for recin with an exit function registered with atexit(). 1.2 REC r_flags assignments Bit Description ----- ------------------------------------------------------------- 0 If clear, colno start at 0; if set, colno start at 1 1 if clear, read mode; if set, write/append mode 2 if clear, current field not quoted by r_txtch; if set, current field quoted by r_txtch. 3-6 Reserved for future use. 7 If clear, EOF not reached; if set, EOF reached 8-11 If clear, no error; else rerror number 12-15 if clear, no warning; else rwarning number 1.3 Accessing REC Members and Associated Buffers How do I * access the name of the record stream? rnames() * access the current context number? rcxtno() * access the current record number? rrecno() * access the current field number? rfldno() * access the current column number? rcolno() * access the record string buffer? rrecs() * access the field string buffer? rflds() * determine the number of fields in a record? rnumfld() * determine if column numbers start at 0 or 1? rbegcolno() * determine if the current field was quoted? ristxtfld() * determine if there are more records left? reof() * determine if there is an error on the stream? rerror() * determine if there is a warning on the stream? rwarning() * access the error message for the stream? rerrstr() * force an error on a record stream? rseterr() * clear an error on a record stream? rclearerr() * scan the record buffer more than once? rresetrec() * increase the size of the record string buffer? rsetrecsiz() * increase the size of the field string buffer? rsetfldsiz() * replace the data in the field string buffer? rsetfldstr() * replace the data in the record string buffer? rsetrecstr() * get/set position of a field in a record? rgetfldpos()/rsetfldpos() * set the field delimiter character? rsetfldch() * set the text delimiter character? rsettxtch() * set the time format string? rsettmfmt() * set the context number? rsetcxtno() * set column numbering to start at 0 or 1? rsetbegcolno() * dynamically copy a string? scpys() * dynamically concatenate a string? scats() * trim white space from the ends of a string? strims() 2.0 CODE STRUCTURES 2.1 RECIO Structure Chart ΙΝΝΝΝΝΝΝΝΝ» ΙΝΝΝΝΝΝΝΝΝ» Ί rstr.c Ί Ί recio.c Ί ΘΝΝΝΝΡΝΝΝΝΌ ΘΝΡΝΝΡΝΝΡΝΌ ³ ³ ³ ³ ³ ΙΝΝΝΝΝΝΝΝ» input ³ ³ ³ output ΙΝΝΝΝΝΝΝΝ» ΐΔΔΔΔΆ rget.c ΗΔΔΔΔΔΔΔΔΔΔΔΩ ³ ΐΔΔΔΔΔΔΔΔΔΔΆ rput.c Ί char ΘΝΡΝΝΝΝΡΝΌ column ³ char ΘΝΡΝΝΝΝΡΝΌ column delimited ³ ³ delimited ³ delimited ³ ³ delimited ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» ³ ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» Ί rgetf.c ΗΔ΄ ΓΔΆ rcgetf.c Ί ³ Ί rputf.c ΗΔ΄ ΓΔΆ rcputf.c Ί ΘΝΝΝΝΝΝΝΝΝΌ ³ ³ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ΘΝΝΝΝΝΝΝΝΝΌ ³ ³ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ³ ³ ³ ³ ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» ³ ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» Ί rgets.c ΗΔ΄ ΓΔΆ rcgets.c Ί ³ Ί rputs.c ΗΔ΄ ΓΔΆ rcputs.c Ί ΘΝΝΝΝΝΝΝΝΝΌ ³ ³ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ΘΝΝΝΝΝΝΝΝΝΌ ³ ³ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ³ ³ ³ ³ ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» ³ ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» Ί rgett.c ΗΔ΄ ΓΔΆ rcgett.c Ί ³ Ί rputt.c ΗΔ΄ ΓΔΆ rcputt.c Ί ΘΝΝΝΝΝΝΝΝΝΌ ³ ³ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ΘΝΝΝΝΝΝΝΝΝΌ ³ ³ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ³ ³ ³ ³ ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» ³ ΙΝΝΝΝΝΝΝΝΝ» ³ ³ ΙΝΝΝΝΝΝΝΝΝΝ» Ί rbget.c ΗΔΩ ΐΔΆ rcbget.c Ί ³ Ί rbput.c ΗΔΩ ΐΔΆ rcbput.c Ί ΘΝΝΝΝΝΝΝΝΝΌ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ΘΝΝΝΝΝΝΝΝΝΌ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ΙΝΝΝΝΝΝΝΝΝΝ» ³ ΙΝΝΝΝΝΝΝΝΝ» ΙΝΝΝΝΝΝΝΝΝΝ» Ί rwarn.c ΗΔΔΔΑΔΔΔΆ rerr.c ΗΔΔΔΔΔΔΔΔΆ rfix.c Ί ΘΝΝΝΝΡΝΝΝΝΝΌ ΘΝΝΝΝΡΝΝΝΝΌ ΘΝΝΝΝΝΝΝΝΝΝΌ ³ ³ ΙΝΝΝΝΟΝΝΝΝΝ» ΙΝΝΝΝΟΝΝΝΝ» Ί rwmsg.c Ί Ί remsg.c Ί ΘΝΝΝΝΝΝΝΝΝΝΌ ΘΝΝΝΝΝΝΝΝΝΌ 2.2 Callback Error Functions The callback error function technique used by the RECIO library can be applied within your programs to other abstract data types. For abstract data types within the standard C library, you will need to write wrapper functions. ΙΝΝΝΝΝΝΝΝΝ» Ί errnofn Ί ΘΝΝΝΝΡΝΝΝΝΌ ³ ΪΔΔΔΔΔΔΔΔΔΔΔΔΔΔΕΔΔΔΔΔΔΔΔΔΔΔΔΔΔΒΔΔΔΔΔ etc ³ ³ ³ ΙΝΝΝΝΟΝΝΝΝ» ΙΝΝΝΝΟΝΝΝΝ» ΙΝΝΝΝΟΝΝΝΝ» Ί rerrfn Ί Ί ferrfn Ί Ί merrfn Ί ΘΝΝΝΝΝΝΝΝΝΌ ΘΝΝΝΝΝΝΝΝΝΌ ΘΝΝΝΝΝΝΝΝΝΌ REC FILE MEMORY 2.2.1 Errno Callback Error Function Skeleton [errnofn()] switch on error number [errno] case out of memory [ENOMEM] case out of range [ERANGE] case out of record or file pointers [EMFILE] case permission denied [EACCES] case invalid argument [EINVAL] default [anything else] endcase 2.2.2 RECIO Callback Error Function Skeleton [rerrfn(rp)] if valid record pointer [risvalid(rp)] if past end of file [reof(rp)] (if reof test removed, past EOF will else [error number set] become R_EMISDAT or R_WEMPSTR) switch on error number [rerror(rp)] case read data errors [R_ERANGE || R_EINVDAT || R_EMISDAT] case write data errors [R_ENOPUT] switch on context number [rcxtno(rp)] case RECIN switch field number [rfldno(rp)] case 1 (first field read) case 2 (second field read) ... endcase ... default [missing or unknown context number] endcase case out of memory [R_ENOMEM] case fatal errors [R_EINVAL || R_EINVMOD] default [anything else] endcase endif else [invalid record pointer] call error function that handles errno errors [errnofn()] endif 2.3 RECIO Callback Warning Function Skeleton [rwarnfn(rp)] if valid record pointer [risvalid(rp)] switch on warning number [rwarning(rp)] case read data warnings [R_WEMPSTR || R_WTMFMT] case write data warnings [R_WWIDTH] case code runtime warnings [R_WNOREG] default [anything else] endcase endif 2.4 RECIO Field Parsing Algorithm The RECIO library parses a field from an input record according to regular expressions that are specified below using PERL programming language notation (see the references in section 4.0). Four cases must be considered that depend on whether the field and text delimiters are set to whitespace or to something else. To illustrate the cases below, that something else is the comma character for the field delimiter and the double quote character for the text delimiter. case 1: fldch = ' '; txtch = ' '; \s*(\S*)(\s*(\s|$)) case 2: fldch = ','; txtch = ' '; ([^,]*)(,|$) case 3: fldch = ' '; txtch = '"'; A: \s*"(([^"]|"[^"]*"|"\S)*"?)("\s*(\s|$)) B: \s*"(([^"]|"\S)*"?)("\s*(\s|$)) notes: (1) use case 3 when first nonwhitespace character is txtch; otherwise use case 1. (2) use case 3A when txtch is the double quote character; use the form of case 3B for any other nonwhitespace text delimiter. case 4: fldch = ','; txtch = '"'; A: \s*"(([^"]|"[^"]*"|"\s*[^,])*"?)("\s*(,|$)) B: \s*"(([^"]|"\s*[^,])*"?)("\s*(,|$)) notes: (1) use case 4 when first nonwhitespace character is txtch; otherwise use case 2. (2) use case 4A when txtch is the double quote character; use the form of case 4B for any other nonwhitespace text delimiter. (Example: "use case "4A", okay?", is one field using case 4A, but would break after the first comma if using case 4B. Case 4A is limited to double quote characters since other delimiters might not be used in matching pairs. For instance, the single quote character is also used as an apostrophe.) When the parsing algorithm is finished, the field buffer contains the equivalent of the PERL variable $1. The field parsing code can be found in the _rfldlen and _rfldstr functions in rget.c. 3.0 DEVELOPMENT NOTES 3.1 fgets (Microsoft C 5.1) Previous notes of mine indicate that Microsoft's fgets function does not work correctly when it reads a line of text that consists of only a newline. However this can be worked around by first setting the string buffer to an empty string. If you plan on retaining the newline, you will need to test this further. The fgets function is used twice in the rgetrec function. If porting to Microsoft C, you may need to implement this fix in rget.c: *rrecs(rp) = '\0'; /* just prior to the first fgets (added v1.20) */ *str = '\0'; /* just prior to the second fgets */ 3.2 fopen (Borland C 3.1) fopen() calls __openfp() calls open(). Borland's "Library Reference" documents error numbers for open(), but not for fopen(). These error numbers are ENOENT, EMFILE, EACCES, and EINVACC. Because ropen() screens the access code, the EINVACC error should not occur from the recio library. 3.3 strtol & strtoul (Borland C 3.1) These functions stop consuming input once they overflow, setting ERANGE. Hence endptr can point into the middle of a sequence of valid characters having the expected form as given in ANSI X3.159-1989, Sections 4.10.1.5 and 4.10.1.6. IMHO this characteristic is not in conformance with the ANSI standard as endptr should only point to the first unrecognized character or to the terminating null character. Borland's strtod does not have this problem. Note that ANSI X3.159-1989 Section 4.10.1.6 allows strtoul (unsigned long) to have an optional negative sign. A negative unsigned long?! Borland 3.1 strtoul converts a negative long to an unsigned number without error. But I prefer to trap any negative numbers input to unsigned fields. So str2ul is a wrapper function for strtoul that first tests for a negative number and if one is found, flags the data as invalid and returns zero. The test suite includes -0 as a data value. The strtol function traps this as an ERANGE error and returns the overflow limit. The rfixi and rfixl functions substitute zero. 3.4 strtod (Borland C 3.1) Unlike the strtol and strtoul functions, strtod does not clear errno first. Starting with recio vers. 2.02, errno is cleared before any conversion where recio code checks the value of errno after the conversion. It is also cleared when a record stream is successfully opened. 3.5 mktime (Borland C 3.1) The Borland C 3.1 mktime function has valid range from 1 Jan 1970 00:00:00 to 19 Jan 2038 03:14:07. Actual upper limit can vary by a few hours depending on the timezone setting. For EST, upper limit is 18 Jan 2038 22:14:07. When the recio library reads a 2-digit year, it first represents it in a struct tm from 1970 to 2069. The mktime function is then used to convert to a time_t. Hence, when compiled with the Borland C 3.1 compiler, the recio library routines fail for any dates beyond 18 or 19 Jan 2038, or before 1 Jan 1970. The minimum and maximum values of time_t are controlled by the recio.h symbolic constants TIME_T_MIN and TIME_T_MAX. These are used only by the time functions in rfix.c. Borland C 3.1 represents time_t using a long. Thus TIME_T_MIN is set to 0 and TIME_T_MAX is set to LONG_MAX. LONG_MAX is defined in your compiler's header file limits.h. You should find your compiler's representation of time_t in your compiler's header file time.h. 3.6 ltoa and ultoa (ANSI-C) Your ANSI-C compiler might lack the ltoa and ultoa functions as these are missing from ANSI X3.159-1989. These functions convert a long and an unsigned long to a string. The recio library uses these functions. If they are missing from your compiler, you can write a simple version of these functions by using the sprintf function. The sprintf function limits the radix (base) to 8, 10, and 16, but this will probably be adequate for most applications. The prototypes for recio.h are: extern char *ltoa(long l, char *str, int base); extern char *ultoa(unsigned long ul, char *str, int base); The logical place for the ltoa and ultoa functions is in rput.c next to the dtoa function. The code is as follows: /****************************************************************************/ char * /* return pointer to string */ ltoa( /* convert long number to string */ long l, /* number to convert */ char *str, /* string buffer to use */ int base) /* radix of the number */ /****************************************************************************/ { switch (base) { case 8: sprintf(str, "%lo", (unsigned long) l); break; case 10: sprintf(str, "%ld", l); break; case 16: sprintf(str, "%lx", (unsigned long) l); break; default: /* error */ *str = '\0'; rseterr(NULL, EINVAL); } return str; } /****************************************************************************/ char * /* return pointer to string */ ultoa( /* convert unsigned long number to string */ unsigned long ul, /* number to convert */ char *str, /* string buffer to use */ int base) /* radix of the number */ /****************************************************************************/ { switch (base) { case 8: sprintf(str, "%lo", ul); break; case 10: sprintf(str, "%lu", ul); break; case 16: sprintf(str, "%lx", ul); break; default: /* error */ *str = '\0'; rseterr(NULL, EINVAL); } return str; } In recio.h, the value of NSBUFSIZ is used to dimension the internal string buffer that the recio library uses when it calls the ltoa, ultoa, and dtoa functions. This value must be large enough to hold every possible number (integral and floating point) without overflowing the string buffer. In determining the value, keep in mind the following points: * Integral numbers contain more digits as the radix gets smaller. * Reserve space for the exponent of a floating point number, including the 'E'. * Reserve space for possible sign(s). Floating point numbers can contain two signs, e.g. -1.11E-01. * Reserve space for the terminating null. * Valuable information on the representation of numbers is contained in your compiler's header files float.h and limits.h. 4.0 REFERENCES Wall, L. and Schwartz, R.L. Programming Perl. O'Reilly & Associates, Sebastopol, CA, 1991, p. 103 ff. Schreiner, A.T. and Friedman, Jr., H.G. Introduction to Compiler Construction with UNIX. Prentice-Hall, Englewood Cliffs, NJ, 1985, p. 25.