FWKCS(TM) Contents_Signature System, Ver. 2.00, 1995 Jan 16. (C)Copyright Frederick W. Kantor 1989, 1995. All rights reserved. What's new in FWKCS version 2.00: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ FWKCS is the premier system for automatically recognizing duplicate files and duplicate zipfiles, independent of filename. It is used on major electronic bulletin board systems; quality control is backed by more than 3,000,000 node_hours on giant systems, through which have passed copies of a large fraction of all the different shareware zipfile products that our civilization has seen. This new version of FWKCS contains many improvements, not only in new capabilities, but also in ease of use. Of special importance are the new features to help you protect your system from becoming involved in software piracy. The new release of FWKCS is version 2.00, beginning this new series. It issued 1995 Jan 16, in FWKCS200.ZIP. The 1.nn series has been discontinued. For ease in updating, the anti-piracy resource material for use with FWKCS is being distributed in a companion series, FWKCXnnn.ZIP; that series started with FWKCX001.ZIP, issued 1995 Jan 16 (note: that series number is not tied to the FWKCS version number). This resource material allows you to use powerful FWKCS features for the automatic recognition and automatic blocking of files, independent of filename. For where to get the most recent release, see "Note 2:" near the end of README.TXT. The executable code needed, is provided in this package (FWKCS200.ZIP). See especially the new XCLEANUP.BAT, below, and FLAG_REV.BAT (both are automatically installed in your \CS directory; for on_line help in your \CS directory, while in that directory do CSM ). ------------------------ For current users, below is a summary of what is new or changed in FWKCS(TM) Version 2.00. The new series includes a program (REPLACE.BAT) which lets you replace your existing, working version of FWKCS, Ver. 1.12 or later, while keeping your working CS lists, logs, special messages, and configuration. There are new functions, and changes, in FWKCS.EXE, FWKDG.COM, FWKHI.COM, DSA.COM, and many .BAT programs. And, there are new programs: FWKEM.COM, FWKFT.COM, FWKLM.COM, FWKM.COM, FWKQA.COM, CRLF0.COM, and XCLEANUP.BAT. Selected items are listed below. Changes in FWKCS.EXE: ~~~~~~~~~~~~~~~~~~~~~ 1. New functions: /c0 - convenience tool, for appending to a potentially busy file. format: FWKCS (newstuff) /TnnnC0 OUTFILE Waits up to Tnnn seconds for write access to OUTFILE, and append NEWSTUFF to end of OUTFILE. If NEWSTUFF is not listed on command line, "standard input" is used; for example, this command can be used at the end of a pipelined command line. If OUTFILE does not exist, it is created. /c1 - supports using FWKDG and FWKCS together to select files from a large system on the basis of file sizes, in one pass. If nothing found, output file is deleted. Can be used in a pipeline. Format: FWKCS ((d:\path\)infile1) /tNNNc1 (d:\path\)infile2 ((d:\path\)outfile) Both inputs must be sorted ASCII, ascending order. INFILE2 must be specified on the command_line. If a line from input 1 (e.g., INFILE1) has the same first 8 characters as a line from INFILE2, then the next character location in the line from input 1 is skipped, and the rest of that line is sent to the output. If OUTFILE is not specified, then FWKCS uses the redirectable "standard output". If OUTFILE is specified and does not exist, it is created. If it turns out that there is nothing found to put in OUTFILE, it is deleted. E.g., a .BAT program can test for existence of OUTFILE, and branch accordingly. For example, the following pipelined command line can be used to list in FILELIST the d:\path\filename.ext for all the files on drives C: - Z: whose lengths match any 8_place hexadecimal length specified in sorted list HEXLENS, leading zeros blanked: fwkdg /s | fwkcs /s | fwkcs /c1 hexlens filelist To use decimal lengths, change fwkdg /s to fwkdg /d . /c2 - selected cross match of left 16 characters. Format: FWKCS ((d:\path\)infile1) /tNNNc2 (d:\path\)infile2 ((d:\path\)outfile) Both inputs must be sorted ASCII, ascending order. INFILE2 must be specified on the command_line. If a line from input 1 (e.g., INFILE1) has the following properties, it is sent to the output: the same first 16 characters as a line from INFILE2 a blank space in column 17 (decimal, counting first as 1) a character A...Z in column 44 a ":" in column 45 For example, this supports high speed cross comparison between a long list of contents signatures and a large CSLIST.SRT data base, selecting for output only those cs_matching lines which do not have a Column_17 flag and which do have a drive designation. As with /c1, if an output file named on the command line does not receive any output, it is deleted. /c3,nnn - cross match of left n characters, n < 510. Format: FWKCS ((d:\path\)infile1) /tNNNc3,nnn (d:\path\)infile2 ((d:\path\)outfile) This is a simple cross comparison between the specified left part of lines from two files. It carries the network protection and speed of the FWKCS kernel. Both inputs must be sorted ASCII, ascending order. INFILE2 must be specified on the command_line. If the left nnn bytes of a line from input 1 (e.g., INFILE1) match the left nnn bytes of a line from INFILE2, it is sent to the output. Can be pipelined. As with /c1, if an output file named on the command line does not receive any output, it is deleted. 2. New option: under "get" command (/g), option b ("bare"), to deliver the lines found in the data base whose contents_signatures match that of the input, without any copy of the input line(s), dashed lines, etc. If an output file named on the command line does not receive any output, it is deleted, and the exit errorlevel is set to zero. The exit errorlevel indicates the results of the search, as with the usual /f or /g command; this includes setting exit errorlevel = 6 if an exclusion flag is found. 3. Changed: increased the unregistered trial period from 30 days to 45 days, and added an explanation displayed on the screen above the unregistered program's identification: "This is an unregistered copy. After evaluation day 45, if you do not register, a delay of about 1 second will be added for each additional day. The suggested test time is up to 30 days, to allow time for your registration to normally reach you by mail before those delays start. You are now at Evaluation Day 0. To register, see instructions in D:\PATH\REGISTER.FRM." The Evaluation Day is updated, counting the first day of use as day 0, and D:\PATH gives the drive and path for the registration form. Registration suppresses that message and those potential delays. The registration form, REGISTER.FRM, is a plain ASCII text file which can be copied to a standard printer. Also, when in the \CSA directory, you can run CSAM to normally view and print REGISTER.FRM; look in CSAM's literature menu. 4. Changed Auxiliary Function 5 option i (remote inquiry): added the ability to process remote inquiries based on captured images from PKZIP -v. (See also LOOKUPZ.BAT, below.) 5. Changed default exit errorlevel for option /mn to 0. 6. Corrected a bug which entered with version 1.24, in which merging a sorted file across a NUL and specifying /mp (not /mpb, and not /ms) could miss first line. 7. Revised Auxiliary Functions 7.0, 7.1, and 7.2: if one or two output files is specified on the command line, then if there is no output for that file, respectively, the file is deleted. Existence of the file(s) can be used as flag to control branching in .BAT programs. 8. Revised network delay settings permitted per /a2m, /a3m, and /a4m, to t60; and by /a5m, to t500. (for unsupervised machine_interface use) 9. Consolidated external help files into FWKCS200.H00. 10. Increased the default reserved amount of top part of low memory during sorting from 40960 to 65536 bytes, for compatibility with DOS 6.22 11. Various other changes. Changes in FWKDG.COM: ~~~~~~~~~~~~~~~~~~~~~ 1. Added options S and D, for making a file list including the filesize in an 8_place fixed field at left, with leading zeros blanked: S - hexadecimal fileSize: hhhhhhhh d:\path\filename.ext D - Decimal filesize : nnnnnnnn d:\path\filename.ext The hexadecimal filesize can accomodate the current maximum filelength supported by DOS, 2^32-1 bytes. FWKDG can provide such a list for a selected drive or range of drives on a computer or network in one pass. Fixed field format allows sorting by size, e.g., using FWKCS.EXE /s . 2. Added /ver, to return version number as exit errorlevel (without decimal point). 3. Revised error_reporting code. 4. Various minor changes. Changes in FWKCSC.COM: ~~~~~~~~~~~~~~~~~~~~~~ 1. Revised error_reporting code. 2. Various minor changes. Changes in FWKHI.COM: ~~~~~~~~~~~~~~~~~~~~~ 1. Added command line options: Format: FWKHI (/option) "option": * - this help screen. new 0...9 A...Z - set range, case insensitive; default: F. new YN - only y or n (case insensitive), returns 0 if n, 1 if y. new . - allow ASCII 0...255. new ver - set exit errorlevel per version number sans ".". Timesharing is handled as before: under various operating systems, FWKHI truncates time slice while waiting, unless it finds environment variable FWKKTS . As before, FWKHI waits for a keyboard input, and returns the (interpreted) value via its exit errorlevel; used in .BAT programs. (many FWKCS .BAT programs have benefited from this revision of FWKHI) 2. Changed output so that the help screen is sent to the "standard output" ("handle 1"), which is redirectable, rather than to the "standard error output" ("handle 2"). (e.g., for convenience in documentation) 3. Various minor changes. Changes in DSA.COM: ~~~~~~~~~~~~~~~~~~~ 1. Revised code. 2. Added /ver, to return version number as exit errorlevel (without decimal point). 3. Various minor changes. Changes in FWKCSS.COM and FWKCST.COM: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. Added /ver, to return version number as exit errorlevel (without decimal point). 2. Various minor changes. New: CRLF0.COM ("FWKCRLF0(TM)"): ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Purpose: to format (as for sorting with FWKCS) a file which has blocks of text, where only the first line of each block of text is flush left and all the rest are indented by at least one blank space (e.g., PCBoard text directories with multi_line file descriptions have this structure). This small utility reads from the redirectable "standard input" and writes to the redirectable "standard output". CRLF0 /i looks for places where the byte (character) immediately after a carriage_return__line_feed is greater than " " (ASCII 32 decimal), and inserts a 0_byte into the output stream; if the input contains a CRLF followed immediately by hexadecimal 01a (Ctrl-Z), CRLF0 /i replaces the 01a with 00. If the input ends with a CRLF (hexadecimal 0d,0a), it adds a 0_byte to the end. CRLF0 /s strips 0_byte immediately following CRLF: 0d,0a,00 --> 0d,0a. For example, to sort a file made of blocks of text, each with only the leading line flush left, and each block (including its last CRLF) not more than 1999 bytes long, CRLF0 /i < infile | FWKCS /sm2000t13,10,0 | CRLF0 /s > outfile This reformatting before and after sorting lets you use FWKCS's ability to support 520 sorting keys, and to sort such text_block files with total length limited by the operating system or available disk space, e.g., for DOS, up to nearly 4 Gigabytes. For help, type CRLF0 /* ; use /r to redirect this help screen to "standard output" (e.g., CRLF0 /r > HELPSCR to send the help screen to a file named HELPSCR). /ver returns version number as exit errorlevel (without decimal point). New: FWKEM.COM: ~~~~~~~~~~~~~~~ Purpose: to report re file extension (.EXT) match via exit errorlevel. Format: FWKEM %1 ex1 (ex2 (ex3 (...))) /ver - set errorlevel per version number sans ".". exit errorlevel: 0 - file does not have .EXT extension. 1...62 - matched ex1...e62. 98 - file has extension, did not find match. 99 - help message. version as above. For example, this can be used with properly labeled compressed libraries, to control branching in a .BAT program for testing uploaded files; e.g., to add new library compression routines. After the files have been decompressed, FWKCS can process the uncompressed files. For help, type FWKEM /* . New: FWKM.COM: ~~~~~~~~~~~~~~ Purpose: "move", to support new FWKCS .BAT features under older operating systems. Format: FWKM (/ver) ((\)path\)name1 ((\)path\)name2 /ver - set exit errorlevel per version number sans ".", quit. exit errorlevel: 0 - no error. 99 - help message. version as above. else DOS error + 100 decimal. The source and target locations must be on the same drive; can also be used to rename or move a subdirectory (and move all the subdirectories under it). For help, type FWKM /* . New: FWKQA.COM: ~~~~~~~~~~~~~~~ Purpose: to quickly append a short file to the end of a long file. Format: FWKQA (/option) (d:\path\)target (d:\path\)source "option": * - this help screen. b - Binary append; default: if last byte of target is hexadecimal 01a and source is not empty, then overwrite that 1 byte and copy source exactly. ver - set exit errorlevel per version number sans "." Note that the target file is on the left. The source and target files must exist; FWKQA does not create a missing target file. For help, type FWKQA /* . New: FWKFT.COM: ~~~~~~~~~~~~~~~ Purpose: to copy up to specified number of bytes or lines, counting from tail, of redirectable "standard input" to redirectable "standard output"; if linecount, counts lines which end with CRLF (hexadecimal 0d,0a); set errorlevel = least integer >= (log2(filelength+1)). Format 1: FWKFT /option < infile (>(>) outfile) 2: FWKFT /LL (d:\path\)filename.ext (does not open file) "option": (using only one; upper or lower case OK) * - this help screen; set errorlevel = 99 decimal. . - copy all of INFILE. Bn - copy n Bytes; Ln - copy n Lines; where n = 0 to 99999999 decimal. LL - format 2, only report re Log2(Len+1). r - Redirect this help screen to "standard output". ver - set errorlevel per version number sans ".". re system error: exit errorlevel = DOS error + 100 decimal. For help, type FWKFT /* . In manual use, FWKFT can be convenient for inspecting the end of a large file, such as CSLIST.SRT. E.g., FWKFT /L10 < CSLIST.SRT sends the last 10 lines of CSLIST.SRT to the screen, in a fraction of a second. In automatic operations, FWKFT can be used in a .BAT program to control branching to a different path of execution depending on the size of a specified file. New: FWKLW.COM: ~~~~~~~~~~~~~~~ Purpose: to return an exit errorlevel depending on the number of lines in, or on the width of (longest line in), a redirected input from a file or a "pipeline". Format: FWKLW /option < ((d:\path\)filename) "option": (using only one; upper or lower case OK) L - Linecount. W - Width (longest line). ver - set exit errorlevel per version number sans "." exit errorlevel: 0...98 - value for linecount or width. 99 - help message. version as above. else DOS error + 100 decimal. For help, type FWKLW /* . For example, in processing large batches of compressed libraries, FWKLW can be used in a .BAT program for automatically screening FILE_ID.DIZ description files; those which are too long or too wide can be set aside or rejected. New or changed .BAS and .BAT programs: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 1. New: XCLEANUP.BAT, for use with X_ADD.LST, to find and delete commercial software whose contents_signatures are listed in X_ADD.LST. If it finds any to delete, XCLEANUP makes a special XPURGE.BAT, and then asks if you wish to stop or continue. You can inspect and edit XPURGE.BAT before you run it. 2. New: LOOKUPZ.BAT, for use in remote lookup using PKZIP -v image -- this can be run without requiring any local execution of the FWKCS kernel. A small lookup package based on LOOKUPZ (without Rcrosref) is separately available for distribution (see FWKLZ200.ZIP, below). 3. Revised FLAG_REV.BAT, to use new functions provided by the FWKCS kernel, FWKHI, and FWKFT. FLAG_REV.BAT provides for the addition or removal of large batches of x_flagged or w_flagged contents_signature entries in the main data base. It is especially convenient for adding resource material provided in X_ADD.LST re software piracy. 4. Revised the form of PCBTEST.BAT created by FWKCS during installation. In case of partial system failure (e.g., network problems), it attempts to set an uploaded file aside in \CSA\UNC (the place for unchecked files), and creates a short PCBFAIL.TXT reporting system problems and apologizing to the user. 5. Revised S_REVCSL.BAT, so that it can use external lists drives.lst, include.lst, and exclude.lst to input, repectively, the list of drives to search, the list of file extensions or other text strings to search for in selecting d:\path\filename.ext line items to include, and the list of file extensions or other strings to exclude (all fully duplicate line entries are reduced to single lines before the data base is built). Because of the improvements in S_REVCSL.BAT, if you are installing FWKCS on a complicated BBS, it is suggested that you use QIKSTART to install the system, change to your \CS directory, and use S_REVCSL.BAT to build the date base. S_REVCSL is the most flexible of the .BAT procedures for building the data base; if you use it to build the original data base, you can use the same routines for system maintenance later. 6. Revised most of the .BAT program's user interface, to take advantage of the revised version of FWKHI (see above). A Y/N option is used in many .BAT programs, rather than using the DOS "prompt" with "Ctrl+C, y" combination. Unlike the DOS "prompt" inputs, the "y" and "n" for FWKHI can be keyed ahead. 7. Modified the replacement procedure, for putting in a new version of FWKCS over an older working version, so that the prior versions of S_CSNEW.BAT, S_NEWCSL.BAT, and S_REVCSL.BAT are respectively renamed from .BAT to .OLD. If you have customized any of them, your revised version is preserved for reference use when you transfer your modifications to the new versions. Important: do not try to run your .OLD copies of S_CSNEW.BAT, S_NEWCSL.BAT, and S_REVCSL.BAT,. 8. Modified various .BAT programs which can create a version of PCBTEST.BAT for use in client/host configuration, to accomodate a change in CDC's PCBoard 15.2 in which a non_file label "TEST" appears in .BAT call position %3, where otherwise only a filename appeared before. 9. Modified various .BATs, replacing the overwriting of target files in the same directory with the deletion of target files and renaming of source files. 10. Increased the allowed network delay times in various .BAT programs: YT.BAT, ACCESN.BAT, ACCZIP.BAT, and PCBTEST.BAT (all of which are generated from FWKCS.EXE code; see FWKCS.EXE item 6, above); and DO_BLOG.BAT, DO_HOSTQ.BAT, and BBCLIENT.BAT. PCBTEST.BAT is now set to allow 500 seconds, the others now all allow 60 seconds, of network delay (this can be increased further by editing the .BAT programs). 11. Corrected DOZIZ0.BAT line 98, changing "E(2)," to "E2,". 12. Revised on line help menus CSM.BAT (serves \CS directory) and CSAM.BAT (serves \CSA directory and system), to include new programs, and to use FWKHI's increased flexibility in handling user input. 13. Increased the time allowed for network delays in various .BAT programs. 14. Improved user interface in installation programs. 15. Numerous small changes in .BAT and .BAS programs. Changes in docs: ~~~~~~~~~~~~~~~~ 1. Simple, short, clear, user-friendly registration form. This registration file, REGISTER.FRM, is sensed by FWKCS to let you run all the features before your registration key arrives. Because it serves as a temporary key, be careful not to change the original REGISTER.FRM file. You can print it out and fill in by hand the few items for registering, or you can make a copy of REGISTER.FRM and use a word processor on the copy (be sure to sign it). 2. Numerous changes in documentation to add new material; and a few errata corrections. Note: ~~~~~ The remote lookup functions, including Rcrosref, are available in a relatively small kit, FWKLU200.ZIP, released 1995 Jan 16. Most of the remote lookup functions (but without Rcrosref), are available in a special, even smaller kit, FWKLZ200.ZIP, releaased 1995 Jan 16. FWKLZ200.ZIP does not require registration. If you run a BBS, you may wish to get the FWKLU200.ZIP kit or FWKLZ200.ZIP for your users, especially if your BBS is a "feeder BBS" and many of your users are other BBS's. The kit comes with instructions, and contains a short bulletin, FWKLU200.BLT, suitable for posting.