FWKCS(TM) Contents_Signature System, Ver. 1.20, 1993 May 28. (C)Copyright Frederick W. Kantor 1989-1993. All rights reserved. Summary: The FWKCS(TM) Contents_Signature System provides a powerful platform supporting contents_signature functions. Among other applications, FWKCS is used on giant bulletin boards for protecting against duplicate files and against accidentally uploaded commercial files. The FWKCS system has been shown, experimentally, to provide statistical resolution typically more than one thousand times as good as the best that is mathematically possible for the 32_bit CRC to do. The new low pricing, the automated installation procedures ranging from simple to complex, and the extensive on_line support provided by FWKCS on your computer, bring the FWKCS system within reach of almost anyone with a computer that can run DOS or a DOS session. Important: FWKCS(TM) is provided on a two_step test_before_you_decide basis. See "Notes", below. Your use of this software is at solely your own risk: please have proper backups of your files. New or changed in FWKCS(TM) Version 1.20: Pricing: ~~~~~~~~ Very low price, as the regular pricing. Example: 10 node system - Step 1 price: $20.00; Step 2 price: $2.00 (plus all applicable taxes) For more information, see REGISTER.DOC and REGISTER.FRM. Bulletin Board Systems: ~~~~~~~~~~~~~~~~~~~~~~~ 1. Added more nearly generic support for interfacing with various bulletin board systems, using BBSTART.BAT to configure a node's environment, and SCANFILE.BAT to test files. Instructions are included for editing these two programs. As shipped, they are configured for use with Mustang Software's Wildcat. 2. FWKCS can now recognize three environment variables for setting node ID: FWKNODE=ccc (any 3 DOS filename characters); PCBNODE=nnn ; WCNODEID=nnn. 3. Added support of different filedate formats, besides the U.S. standard. New Client/Host Configuration: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ On a network, each node can call on a "client" program to evaluate new uploaded files. The client, FWKCSC.COM, can fit in ca. 32 K of memory. The client then calls an FWKCS "host" to do the CPU_intensive work. Any additional supporting programs called by the host, such as Phil Katz's PKZIP/PKUNZIP for file integrity testing etc., and McAfee's SCAN for virus testing, do not use the client's memory. The host sends back a reply to the client, and the client uses this to handle the node's request. This process is fast. It uses a dynamic queue, which can be served by one or more processors running FWKCS as host. When running as host in various multitasking time_slice environments, FWKCS checks its queue for work; if the queue is empty, FWKCS releases the rest of its time_slice for other use. The process is essentially transparent to the end user. This client/host configuration can routinely let any BBS node on a network, including the slowest nodes, handle an end user's uploaded file by calling on the speed of the fastest processor(s) on the network. (client/host configuration is enabled by a code key, reserved for registered FWKCS) New or Changed in Installation Procedure: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Added QIKSTART, to set up the FWKCS system quickly without searching your file collection; this provides a convenient way to see the programs and literature, and to see the structure of the working system. 2. Modified the REPLACE procedure, so that systems using QBASIC can use less free memory for running the automatic upgrade to the new version of FWKCS. 3. FWKCS.EXE and FWKCSC.COM search the environment for node identification. They can recognize FWKNODE=ccc (where "ccc" is any 3 DOS filename characters), PCBNODE=nnn, and WCNODEID=nnn, in that order of priority. When available, node ID is usually used as part of the filename for temporary files, except for reply filenames when a client is sending a batch upload to a host and will not be waiting for the replies. Changes in FWKCS.EXE: ~~~~~~~~~~~~~~~~~~~~~ Compatibility with PKZIP/PKUNZIP v2.04g: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Revised FWKCS.EXE to treat the AV stamp in compliance with Phil Katz's PKZIP(R) version 2.04g. 2. If FWKCS finds than a file appears to be only one part of a multidisk disk_spanning zipfile, FWKCS returns exit errorlevel 26 2.1. Under Auxiliary Function 3 or 4, if not s, it changes the extension to ".PSZ" (Part of disk_Spanning Zipfile); 2.2. Under Auxiliary Function 5 or 6, 2.2.1. prepares a message to the user, 2.2.2. notes the file as "Span Zip" in the comments column of the FWKCS upload log (\CSA\FWKUPLD.LOG). 2.2.3. if not s, and if exist target for failed zipfiles, then FWKCS puts that uploaded file there (if you used the automatic installation, this would be in the \CSA\FZT (Failed Zipfile Test) subdirectory). 3. FWKCS /1b(1) d:filename.ext outfile new /1b option for a multidisk disk_spanning backup zipfile: /1b delivers the contents_signatures in the same order that the files are listed in the zipfile; if the central directory is split onto more than one disk, this will require that the last disk be reinsterted (it asks for the disks in a specific order). /1b1 delivers the contents_signatures by reading the last disk first, and then asks you for one or more other disks in a specified order; this is less work for the user, and is intended for use when the list of contents_signatures will be sorted. /1bd redates zipfile entry on last disk to latest date of any file found inside multidisk zipfile. 4. FWKCS d:\path\filename.ext /a2(b)(d) new b option for a multidisk disk_spanning backup zipfile; maximum number of files is 16383. /a2b finds matching contents_signatures, prepares a MATCHING.RPT, optionally displays results, and optionally keeps or deletes the MATCHING.RPT. If a multidisk zipfile contains at least 1 new entry, /a2b normally accessions the collection into the contents_signature data base. d reDate zipfile; when used with option b, will redate the part of the zipfile which appears on the last disk, to the latest file date FWKCS saw inside the multidisk zipfile (e.g., it does not look inside a zip_in_zip to see if the date on the zipfile has been deliberately set to appear earlier than the files inside). 5. added options z3(~3+-)=; and z43+-); , to send control characters to suppress selected PKZIP/PKUNZIP options, in case of incompatibility between PKZIP/PKUNZIP and your system. Other Changes in FWKCS.EXE: ~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following additional changes were made in FWKCS.EXE: 1. FWKCS now supports operation in a client/host system configuration. New Auxiliary Function 8 supports operation as client; new code supports operation as host. The host code is re_entrant; when running as host, FWKCS.EXE can stay in memory watching for new work, swapping itself out of memory when space is needed for programs it calls. Operation as an FWKCS host (including in a BBS environment) is called using DO_HOSTQ.BAT (see below). Clients running in other operating systems than DOS may access functions from FWKCS running as a host in DOS, provided that they can properly handle the file access procedures. (for more information, see FWKCS120.REF and SETUP.120) 2. For use under Auxiliary Function 5, added a family of options for split testing (a subset of the new functions for host mode), reserved for registered FWKCS; and the following new options fully enabled in the unregistered version: &a,acdmnopw; - if ATTACH, keep selected UPLOAD options. If you wish to drop certain tests or functions, omit the corresponding letter from the ,...; part of the &a,...; option: a - age test c - zipfile comment d - catching duplicate or redundant files l - log ATTACHed files (see L1,atlog; re keeping separate log) m - deleting all matches n - novelty test o - oust: deleting files from inside zipfiles p - purge non_AV files from AV_zipfiles w - wipe (w_flag) selected files from non_AV zipfiles (for use in processing files uploaded as attachments to messages, e.g., under PCBoard 15) &d,DMY; - set date format dd-mm-yy, any order of D,M,Y. Default is mm-dd-yy. The output format uses each two_character day, month, and year entry in the same order as the DMY letters appear in this option; for example, for dd-yy-mm, use &d,DYM; (upper or lower case OK). &fN - use description from FILE_ID.DIZ or DESC.SDI. If FILE_ID.DIZ or DESC_SDI is found, extract the files; if the contents_signature for FILE_ID.DIZ is not rejected, FILE_ID.DIZ has higher priority than DESC.SDI; maximum of ten lines output for text description, not counting any lines added by FWKCS. &lN - data line, Files: NNN firstdate...lastdate New: NNN , added after text description; suboptions to use or omit the "New: NNN" entry, and as to whether or not the line is suppressed in PCBoard graphics mode; if only one file in zipfile, then only one date is shown. &mNN - max. allowed uncompressed length, 0-99999999 Kilobytes. This is based on the apparent total uncompressed length of files in a zipfile. E.g., it can help in identifying "zip bombs" (small, contrived zipfiles containing large file(s) which, if unzipped, could get in the way of other operations); it catches them without unzipping them. ( &m can also be used under Auxiliary Functions 3 and 4) kl,d:\path; where to keep zipfiles which fail the &m length test. Default setting provided in macro [x] during automatic installation is &m6000 for 6000 K ("6000" is decimal, "K" = 1024 decimal); corresponding target directory for rejected files is \CSA\ZXL. Corresponding new "comment" entry for FWKUPLD.LOG is "UnzL>Lim"; zcs is given (total uncompressed filelength (hexadecimal) can be read from the zcs). Corresponding new exit errorlevel is 29. FWKCS can accept corresponding file materials provided by you to send along with, or instead of, its own report. l1,atlog; - separate log for files attached to messages. If the second variable on the command line (%2) is "ATTACH" (case insensitive), and option &a,acdlmnopw; includes "L" (upper or lower case OK), then this option L1,d:\path\atlog; can be used to designate a separate log for recording what FWKCS did re files attached to messages (rather than using the same log as for "UPLOAD"). The format is the same as for the upload log; files received as ATTACHed are labeled "Attached" in the comment column. 3. FWKCS /1c can now accept and process a command list which includes d:\path\filename.ext which contain gaps, such as the OS/2 2.00 system/hidden/read_only "EA DATA. SF" and the system/hidden "WP ROOT. SF" found by FWKDG on a full system scan (to access such files, boot directly from DOS on your hard drive or boot from a floppy); to terminate filename at first gap, use c0 (see below). This includes the ability to make contents_signatures for files in subdirectories when there is a gap in the subdirectory name. For example, in a DOS session under OS/2 2.00, to make cs's for files in your c:\hoo ha subdirectory on your FAT formatted drive (and remove spurious tails from .GIFs), do echo c:\hoo ha\*.*|fwkcs /1cfjpxz >cslist Default for /1 option c is to permit filenames with gap(s). If you wish to use only the first non_gap string on a text line as the designated (d:\path\)filename.ext, use /1 option c0 . If you use /1 option e , then c0 is implied. 4. The FWKCS /a7.8 text search functions can now accept quotation marks as part of the search string. To include a '"' as part of a search string, use a pair '""' to represent one '"' . (Here, the ASCII 39 character ' is used as a string delimiter to mark the string made of ASCII 34 " characters.) grammar: the first '"' marks the start of the search string and is not included in the search string, every subsequent pair '""' is included as one '"' in the search string, the first '"' which is not part of such a pair marks the end of the search string and is not included in the search string. For example, FWKCS infile.txt /a7.8f"""Oh?""" would search the file INFILE.TXT for each line containing the five_byte string '"Oh?"' and send the result to the screen (via the redirectable standard output). For the 3_byte string '"""', use /a7.8f"""""""" 5. You can now use FWKCS macros to store explicitly quoted case_ sensitive strings, which can include gaps and can include paired '""'. To enter a quote_delimited macro on the command line, the left '"' must be the first non_blank character after FWKCS on the command line. For example, you can make a macro which can be called as part of a text search. This command FWKCS "Case_sensitive string, with "" for search" /ds would include an exact copy of the complete quote_delimited string, including the quotes, in macro [s]. To see the entry, type FWKCS /d! . The following line (flush left) would then appear as part of the display: S. = "Case_sensitive string, with "" for search" Detection of the left quotation mark suppresses interpretation of square brackets [], and thus suppresses incorporation of a file by reference. A quote_delimited string can be entered from a file incorporated into the command line using [+filename], but the quotation marks must appear in the file. 6. revised code for 'find' command, to prevent repeated attempts to open a file when operator gives a nonexisting path for the output file. 7. revised the waiting for file_swap under option /a7.5a, so that it quits if the target file is updated during the wait. 8. split testing, client/host, and automatic text interpretation functions are enabled using a code key, reserved for registered FWKCS(TM). 9. Many modifications for operation as host, suppressing options which depend on "pipelining" or on direct interaction with the user; these restrictions apply only in host mode, not for direct use of FWKCS. 10. Modified Auxiliary Function 1, to accept path for DIRGUIDE.TXT on the command line; required for AuxF 1 in host/client configuration. 11. Modified Auxiliary Functions 3 and 4, to optionally send output to a file designated after the options list; permits capture of a3 and a4 output for client when running in client/host configuration. 12. Added release of unused part of time_slice when waiting for work in various multitasking time_slice environments; because of the range of methods which have been used in the past, also added ability to suppress that function via an environment variable or a command_line option. To suppress, type "SET FWKKTS=." (without the quotes; any permitted character can be used: the program looks for whether the variable is there or not). 13. Added a simple but highly flexible grammar for use in passing commands, citations of macros, filenames, etc. between client and host; this grammar can also be used to load sets of complex, non_contiguous commands into macros. 14. Enlarged the ability to customize messages to BBS users, to convey a second set of customized messages in cases where the file was ATTACHed to a message rather than UPLOADed in its own standing to the bulletin board (supports this new option in PCBoard 15). 15. Improved support of remote Lookup, including the ability to process a single lookup made remotely from a wildcard input of up to 512 non_zip files. New: FWKCSC.COM: ~~~~~~~~~~~~~~~~~ 1. FWKCSC.COM is specially designed to run as a client, with one or more FWKCS hosts. It is < 16 KB long, and can fit in ca. 32 KB of memory. It releases remaining part of time_slice when waiting in various multitasking time_slice environments; that function can be turned off by setting the environment variable "FWKKTS=." (without the quotes; any permitted character OK) or using command_line option k . To set up for BBS file testing in the client/host configuration, see also BBCLIENT.BAT, below. (for more information, see FWKCS120.REF and SETUP.120) Changes in FWKDG.COM: ~~~~~~~~~~~~~~~~~~~~~ 1. Added option p to Pause for approximately the time specified in tNNNN.NN, i.e., circa 0 - 9999.99 seconds. 2. Matched FWKDG version number to FWKCS version number. Changes in FWKHI.COM: ~~~~~~~~~~~~~~~~~~~~~ 1. Added release of unused part of time_slice when waiting for input in multitasking time_slice environment; because of the range of methods which have been used in the past, also added ability to use an environment variable to suppress that function. To suppress, type "SET FWKKTS=." (without the quotes; any permitted character OK). 2. Matched FWKHI version number to FWKCS version number. Changes in FWKCSS.COM and FWKCST.COM: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Matched FWKCSS and FWKCST version numbers to FWKCS version number. New or Changed in .BAT and .BAS programs: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. New: BBCLIENT.BAT, to set up for screening BBS uploads using the FWKCS system in the client/host configuration. 2. New: BBSTART.BAT, to set up environment variables and work area, and call a bulletin board system program with use of FWKCS. Instructions are included for editing this program. (As shipped, it is configured for use with Wildcat.) 3. New: DO_HOSTQ.BAT, for running one or more processors as FWKCS host(s). 4. New: FWKCHTST.BAT, for testing client/host configuration, primarily Auxiliary Functions 4 and 5 (related to testing files uploaded to an Electronic Bulletin Board System); requires a multitasking or multiprocessor (network) environment, registered FWKCS. 5. New: SCANFILE.BAT, for generic use. It calls FWKCS directly (including split_test), or in client/host configuration, to process uploads, etc. Instructions are included for customizing this program. It uses environment variables set by BBSTART.BAT (above). (As shipped, it is configured for use with Wildcat.) 6. changed: GET_DFLT.BAT, for convenience in editing the default entries and macros of both FWKCS.EXE and FWKCSC.COM, including the more flexible grammar introduced to support the new client/host configuration. 7. changes in DO_BLOG.BAT: 7.1 added new option to allow the program to continue looking for new work, repeatedly checking with optional pause between checks on a network; this monitors for work listed in \CSA\BLOG, and when that is used up (or not yet present), the screen displays a shifting pattern so that it's easy to see that it's working. 7.2 modified call to FWKCS using option x0, so that the text created by FWKCS when reporting FAIL is in plain ASCII (i.e., it does not contain any special PCBoard @codes). 7.3 added a dividing line between reports appended to the local file FAIL2.LST. 8. enclosed ^ with quotation marks in SPLITEST.BAT and in FWKDG120.BAT, for compatibility with "4DOS". 9. changed: SPLITEST.BAT, corrected the explanation of how to add a second_step zipfile comment. 10. changed: FWKCTEST.BAT, added Zipfile eXcessive uncompressed Length (ZXL) test and subdirectory. 11. changed: FWKCLEAR.BAT, to clean up after both the revised FWKCTEST.BAT and the new FWKCHTST.BAT. 12. many interlocked .BAT programs were changed to support or reflect the expanded FWKCS capabilities, including CSAM, CSAMACS, CSM, DELCSA, SETFWKCS, all the installation programs, etc. Notes: ~~~~~~ 1. FWKCS(TM) is provided on a two step test_before_you_decide basis. At each step, you may test the software before you decide whether to pay; if this software is used in a public, commercial, school, institutional, or governmental environment, then the test period for each step, respectively, is restricted to no more than 30 days. Except for Split Testing, the first step contains all the functions previously available in unregistered FWKCS (some have been enhanced), together with some new features. If you are pleased with what you have in step 1, then you should register as described in REGISTER.DOC, using the form REGISTER.FRM. The second step is available only for registered FWKCS, and uses a key. The second step contains client/host operation, split testing, and procedures for interpreting file descriptions so powerful that they warranted writing the assembly_language code for the FWKCS sorting functions. In the second step, test the additional features appropriately. If you are pleased with what you find in the second step, you should make the further registration specified. In both steps, test before you make your decision. In both steps, the program is not free -- if you do not register, you should discontinue using that functionality. 2. The remote lookup functions are available in a relatively small kit, FWKLU120.ZIP, released 1993 May 28. If you run a BBS, you may wish to get the FWKLU120.ZIP kit for your users, especially if your BBS is a "feeder BBS" and many of your users are other BBS's. The kit comes with instructions, and contains a short bulletin, FWKLU120.BLT, suitable for posting.