Kill QWK Dupes Version 1.03 Kill QWK Dupes Version 1.03 (C) Copyright 1991, 1992 by Dataware, Inc. All rights reserved. (C) Copyright 1991, 1992 by Dataware, Inc. All rights reserved. Table of Contents Table of Contents System Description 1 System Requirements 1 Technical Support 1 Registration 2 Software License 2 How to use Kill QWK Dupes - Overview 3 How to use Kill QWK Dupes - The command line 3 Command Line Options 5 -Log: 5 -CF: 6 -DN: 6 -MC: 7 -MD: 7 -NC: 7 -Com: 7 -FSL 7 -NoFSL 8 -NDX 8 -NoNDX 8 -NoKill 8 -Kill 9 Use of the KILLQDUP environment variable 10 Appendix A: How Kill QWK Dupes operates 11 Appendix B: How to install Kill QWK Dupes into the MarkMail door 12 Appendix C: How to install Kill QWK Dupes into the Qmail door 14 Appendix D: How to install Kill QWK Dupes into the RoseMail door 16 Appendix E: How to install Kill QWK Dupes into the Tomcat! door 18 Appendix F: How to install Kill QWK Dupes into the Tomcat! door 20 Appendix G: Multi-node considerations 22 Appendix H: Errorlevels returned by Kill QWK Dupes 22 History 23 i System Description System Description 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Ever download a QWK mail packet from your favorite BBS, only to find an overwhelming number of duplicate messages? Or import a netmail packet into your BBS and discover 1/10-1/2 of the messages are duplicates re-exported by some other node in the network (maybe you've accidentally done this yourself)? Are you getting tired of users uploading old REP packets more than once causing duplicate messages to be stored in your message base? Then Kill QWK Dupes is for YOU! Kill QWK Dupes will remove duplicate messages from your QWK and/or REP packets. It accomplishes this by maintaining its own independent CRC table of past messages (including those from previous packets), and removes messages from the current packet that match the CRC of a message stored in the table. It will even rebuild the *.NDX files such that a QWK packet continues to be usable by the QWK compatible offline readers (this feature can even be used to generate *.NDX files for a QWK packet that doesn't have them, or has had them corrupted somehow). System Requirements System Requirements 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Kill QWK Dupes will run on any IBM PC or compatible with DOS 2.0 or higher and at least 96k of free memory. Kill QWK Dupes is very light on code, but heavy on data space (depending on the size of the CRC table you specify). Technical Support Technical Support 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 If you have any questions or problems regarding Kill QWK Dupes or any other Dataware product, please call our office for assistance. Our office hours are Monday through Friday, 9:30am-5:30pm Pacific time. You can also leave a message or pick up the latest versions on our 24- hour BBS named "The Real Batchin' Board". You may also mail your comments and questions to the Dataware office. The Dataware address and phone numbers are as follows: Dataware, Inc. Suite 2571 1420 NW Gilman Boulevard Issaquah, Wa. 98027-5399 Voice (9:30am to 5:30pm PT): (206) 391-9333 The Real Batchin' Board BBS (available 24 hours): (206) 391-2330 1 Registration Registration 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 If you find that Kill QWK Dupes is useful to you, you can become a registered user. As a registered user, you will receive a diskette with the latest registered version on it. The 45 day expiration code present in the unregistered version is disabled in the registered version. You will also automatically receive, when available, a free upgrade to the next version of Kill QWK dupes. You will also receive the password necessary to download all future registered version from the support BBS. The current registration fees are as follows: Kill Qwk Dupes Version on 5 1/4" Diskette $29 Kill Qwk Dupes Version on 3 1/2" Diskette $31 Laser Printed Manual $ 5 Ups Ground $ 3 Ups Second Day Air $ 6 Ups Next Day Air $12 All Over-seas Shipping $ 6 Washington residents add 8.2% sales tax to the total Software License Software License 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 You are free to use (for a 30 day evaluation period), copy and distribute Kill QWK Dupes IF: IF - No fee is charged for its use, copying or distribution. - It is not modified in ANY way. Dataware hereby disclaims all warranties relating to this software, whether express or implied, including without limitation any implied warranties of merchantability or fitness for a particular purpose. Dataware will not be liable for any special, incidental, consequential, indirect or similar damages due to loss of data or any other reason, even if Dataware or an agent of Dataware has been advised of the possibility of such damages. In no event shall Dataware's liability for any damages ever exceed the price paid for the license to use software, regardless of the form of the claim. The person using the software bears all risk as to the quality and performance of the software. Sysops who run more then one BBS (as determined by the name they go by) are required to register one copy for each separate and distinct BBS name in which Kill QWK Dupes is being run on as part of a QWK compatible mail door. Multi-node sysops of the same distinct BBS name need register only one copy for all nodes regardless of how many different machines and how many different QWK compatible mail doors Kill QWK Dupes may be running on. Users need register only copy for use with all BBS's they may call. Network connected sysops need register only one copy for use with all networks they are connected to. 2 How to use Kill QWK Dupes - Overview How to use Kill QWK Dupes - Overview 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Kill QWK Dupes can be used in a number of environments. As a node in a network, you can use it to scan your incoming QWK packets and/or your outgoing REP packets. As a sysop of a BBS that has a QWK compatible mail door, you can use it to scan the REP packets uploaded by your users. NOTE: DO NOT use it to scan the QWK packets downloaded by the users unless the -NoKill parameter is used (described below), because all subsequent packets after the first one would appear as duplicates to the program and automatically removed from the users packet. As a user of an offline reader, you can use it to scan the QWK packets you receive from your favorite BBS and/or the REP packets you're uploading to the BBS. Kill QWK Dupes does not automatically extract the files out of the QWK and REP packets it works on. Nor does it automatically merge them back in after processing. That was deliberately left out of the program due to the myriad of compression programs being used today with QWK/REP packets. In addition, this makes it a perfect fit in for scanning a freshly extracted .REP packet. We also wanted to stay away from a configuration file for this program, so you'll find everything is command line and environment variable driven. In addition, it is intended to be run in a batch file, therefore it does no prompting for information. If it doesn't have enough information, or runs into an error while running, it simply reports the error and exits. Kill QWK Dupes works on the MESSAGES.DAT file after having been extracted from the QWK file. It also works on the BBS_ID.MSG file after having been extracted from the BBS_ID.REP file. If you ask it to rebuild the *.NDX files contained in QWK packets (so that your offline reader can continue to read the packet), it also requires the CONTROL.DAT file be extracted from the QWK file. It uses information contained in this file to properly rebuild the *.NDX files. How to use Kill QWK Dupes - The command line How to use Kill QWK Dupes - The command line 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Kill QWK Dupes is easy to use and fast. You tell Kill QWK Dupes what to do by what you enter on the command line at the DOS prompt or via an environment variable (explained below). All command line options that Kill QWK Dupes recognizes, along with examples, are explained in the following sections. The Kill QWK Dupes command line structure is as follows: KILLQDUP [d:][path]qwkfile [options] [d:][path]qwkfile = The name and path location of the QWK file to be scanned. If you're scanning a QWK packet, the filename should be MESSAGES.DAT. If you're scanning a REP packet, the filename should be BBS_ID.MSG (where BBS_ID is the 1-8 character ID used by your BBS). NOTE: Parameters listed in [] are optional. options = All the various Kill QWK Dupes options that you can use. 3 A simple example is as follows: C:\QPACKETS> KILLQDUP messages.dat Kill QWK Dupes v1.00. (C) Copyright 1991 by Dataware, Inc. UNREGISTERED version. This version will cease to function 45 days after activation. After the normal 30 day evaluation period, you will have 15 days in which to obtain your registered version before the software automatically ceases to function. After 45 days, you will have to delete your CRC file (default filename MSGSDUPE.CRC) to re-enable the software. QWK file to be processed : MESSAGES.DAT CRC tracking file : MSGSDUPE.CRC Number of CRC's to track : 32768 Number of days to track : 24855 Use FOSSIL driver : No Rebuild .NDX files : No Remove duplicate messages : Yes This software will cease to function on Friday September 06,1991 @10:56PM Total messages checked : 97 Total duplicates purged : 1 Total processing time : 2 seconds When Kill QWK Dupes starts up, it begins by displaying the copyright message and some status information about what it's going to do. It reports the file name it will process, the logfile (if any) it will write its output status to, the Comm. port it will talk to, whether or not it will use a FOSSIL driver for accessing the Comm. port, the name of the CRC file it will use for checking the CRC's, whether or not the *.NDX files will be built, the number of CRC's it will maintain before throwing away the oldest ones, and finally the number of days CRC's will be kept in the table before being thrown away. Additionally, if you have the unregistered version, it will also report exactly when your 45 day evaluation period will end. Kill QWK Dupes then starts its processing by making a .BAK copy of the original QWK file. You will find a MESSAGES.BAK file, or a BBS_ID.BAK file after it's done, which is an exact copy of the original input file. This is just a precaution in case something goes seriously wrong while processing. Next, Kill QWK Dupes looks at the file to see if it has been previously scanned before (every file is marked with the date it was last scanned, and the location it stopped scanning). If it has been previously scanned, Kill QWK Dupes will automatically advance to the point in the file where it last stopped and resume scanning. This feature is most useful to netmail sysops scanning their .REP packets prior to uploading to their HUB. If a .REP packet is created by an EXPORT function and scanned, and then sometime later more messages appended to it (such as a transfer with your HUB failed), scanning the packet again will not result in removing all the older messages as duplicates. 4 During its processing, Kill QWK Dupes will display a percent completion status every second. This indicator tells you what percent of the input file has been scanned. When it reaches 100%, processing is complete and a status summary is displayed. The status summary tells you how many messages in the input file were scanned, how many duplicates were removed from the input file and how many seconds it took to run. 3 Command Line Options Command Line Options 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 There are 12 command line options you may give to Kill QWK Dupes. They options are used to select a Logfile, the CRC file, the size of the CRC file, the Comm. port to use, whether or not to use a FOSSIL driver, whether or not to rebuild the *.NDX files, whether or not to actually remove duplicate messages. Each of them will be explained in detail in the following sections -Log:[d:][path]filename -Log:[d:][path]filename 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option causes the status summary that's displayed on the screen to also be written to a text file. This is very useful for unattended operation, such as automated netmail runs, or REP packet uploads into a mail door. Five pieces of information are written to the log file every time Kill QWK Dupes is run. First the date & time Kill QWK Dupes was run is recorded. Next the name of the mail packet file processed is recorded. Next, the count of the number of messages scanned is recorded. Following that the number of duplicate messages removed is recorded. And lastly, the total number of seconds used in processing is recorded. A logfile will look like the following: ***************************************** Kill QWK Dupes run on 08/07/91 at 21:25 QWK file MESSAGES.DAT processed Total messages checked : 804 Total duplicates purged : 5 Total processing time : 42 seconds ***************************************** Kill QWK Dupes run on 08/07/91 at 21:27 QWK file MESSAGES.DAT processed Total messages checked : 882 Total duplicates purged : 1 Total processing time : 52 seconds ***************************************** Kill QWK Dupes run on 08/07/91 at 21:28 QWK file MESSAGES.DAT processed Total messages checked : 1014 Total duplicates purged : 1 Total processing time : 50 seconds ***************************************** Kill QWK Dupes run on 08/07/91 at 21:31 QWK file MESSAGES.DAT processed 5 Total messages checked : 97 Total duplicates purged : 0 Total processing time : 3 seconds -CF:[d:][path]filename -CF:[d:][path]filename 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option is useful in many circumstances. As a user of multiple BBS's, you should keep separate CRC files, one for each BBS you use. If you try and use the same CRC file for two or more BBS's, you run the risk of similar messages between the two BBS's being invalidly removed from the QWK packets. In addition, if you send the same message to two or more BBS's, that message will probably be removed from the REP packet as it is scanned. As a node calling two or more HUB's for netmail, you should keep separate CRC files for the same reason as a user of two or more BBS's. As a sysop of a BBS that has a QWK compatible mail door and is also a node in a network, you should use different CRC files for the REP packet uploads and the netmail scans. If you try and use the same packet, the messages uploaded by your users will be seen as duplicates by Kill QWK Dupes and removed when it scans your netmail REP packet. As a sysop of a multi-node BBS having a QWK compatible mail door, you should use this option to make each node point to the same CRC file. Kill QWK Dupes is fully SHARE compatible allowing multiple nodes to access the same file without problems. And by using this option, you can prevent users from uploading the same message more than once by using more then one node. It is recommended however for BBS sysops to use the same CRC file for multiple mail doors. This way, no matter which mail door is used, Kill QWK Dupes will remove the duplicate messages without error. -DC:nnnn -DC:nnnn 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Instead of throwing away duplicate messages into the "bit bucket", you can optionally have them moved to a special conference for your review. By specifying this option, you are telling Kill QWK Dupes which conference to move the duplicate messages into. Without this option, all duplicate messages will simply be removed. -DN:name -DN:name 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Kill QWK Dupes recognizes the "Add/Drop/Reset" mail door commands that may be present in .REP packets. It does this solely based upon the name in the "TO" field of the message. If this field contains either QMAIL, MARKMAIL, ROSEMAIL, TOMCAT, QWIKER, or PCRELAY, it decides that the message is a special message to the mail door software and will not count it as a duplicate. 6 However, if your mail door is not one of the 5 listed above, Kill QWK Dupes may be fooled into believing the "Add/Drop/Reset" commands addressed to your mail door are duplicates and remove them from the packet by mistake. To prevent this from occurring, you can use this option to specify what name your mail door goes by when receiving this type of message. Please refer to the documentation and/or author of your mail door package for the specific name that needs to be configured in this parameter. -MC:32-32756, default is 32756 -MC:32-32756, default is 32756 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option is used to control the size of the CRC table. Each entry in the CRC table uses 6 bytes of disk space and memory to store it. 32756 entries requires a total of 196,536 bytes of disk space, and memory to store. You may want to limit the number of CRC entries to conserve disk space and/or control the amount of time Kill QWK Dupes requires to scan the entire table looking for a duplicate. Also, if you're running short on memory, this parameter can be used to reduce the amount of memory Kill QWK Dupes requires to operate. -MD:1-24855, default is 24855 -MD:1-24855, default is 24855 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This is another method of controlling the size of the CRC table. The difference is that this method controls the age of each CRC entry. CRC entries older then the specified number of days will automatically be purged from the table. This allows things like the same "For Sale add" to be re-posted every few days. -NC:1-32767, default is 3 -NC:1-32767, default is 3 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option controls the maximum number of different conferences the same message may appear. Useful to control the "letter bomb" where a user enters the same message in all conferences he has access to. You may wish to change the default to permit messages such as "For Sale" to be posted in more, or less, then a maximum of 3 different conferences. -Com:1-4 or NNNN where NNNN is the ports hexadecimal address -Com:1-4 or NNNN where NNNN is the ports hexadecimal address 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option causes all text displayed on the screen to also be sent out to the specified comm. port number. It's only useful to BBS sysops using Kill QWK Dupes to eliminate duplicates as part of their REP packet upload process. If -FSL is also specified, this number can be 0-7. Kill QWK Dupes will use the installed FOSSIL driver for your system instead of driving the comm. port directly. 7 -FSL -FSL 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option is only useful to BBS sysops that use a FOSSIL driver for their system. If you use a FOSSIL driver, you will need to include this option. It changes the meaning of the -Com: parameter. Instead of a physical comm. port number, it specifies the FOSSIL port number to use. Be sure you have specified things correctly when using either of these two options. Kill QWK Dupes does no error checking to see if the either the physical comm. port exists, or if the FOSSIL driver is loaded. -NoFSL -NoFSL 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option will override the -FSL option. Useful if you have included -FSL in your KILLQDUP environment variable and don't want to use the FOSSIL driver on a particular scan. -NDX -NDX 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option is will be used mostly by users wishing to scan QWK packets before reading them with your favorite QWK compatible offline reader. It can also be used to generate *.NDX files for a QWK packet that does not contain them, such as a netmail packet a node receives from its mail HUB. Prior to using this option, you must have extracted your CONTROL.DAT file along with the MESSAGES.DAT file from the QWK packet and placed both files in the same sub-directory. Kill QWK Dupes will put the *.NDX files in this sub-directory also. After running Kill QWK Dupes, you must re-pack the QWK packet with the new MESSAGES.DAT file, and the new *.NDX file. The CONTROL.DAT file is only used by Kill QWK Dupes for information necessary to correctly rebuild the *.NDX files, and therefore remains unchanged. -NoNDX -NoNDX 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option will override the -NDX option. Useful if you have included -NDX in your KILLQDUP environment variable and don't want the *.NDX files built on a particular scan. -NoKill -NoKill 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option is useful if you want to just scan the packet without actually removing any duplicate messages found in the input file. This option runs a lot faster because Kill QWK Dupes is not actually rebuilding the QWK file. All it does is scan the input file. NOTE: The CRC file is not updated when this option is used. Thus allowing you to do a quick scan check without effecting any of the data files. 8 -Kill -Kill 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This option will override the -NoKill option. Useful if you have included -NoKill in your KILLQDUP environment variable and do want duplicate messages removed on a particular scan. 9 Use of the KILLQDUP environment variable Use of the KILLQDUP environment variable 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Any of the 12 command line options may also be specified as the default operating mode by setting the environment variable "KILLQDUP". If Kill QWK Dupes finds this environment variable set, it will parse it's setting just like it does the command line. For example: SET KILLQDUP=-Log:D:\ALLOGS\DUPES.LOG -CF:NODE.CRC will cause Kill QWK Dupes to always write its status output to the file "D:\ALLOGS\DUPES.LOG" and use the file "NODE.CRC" as the CRC table in the default mode. In the event there are conflicting options specified, those entered on the command line will override those set via the environment variable. 10 Appendix A: How Kill QWK Dupes operates (aka what is a duplicate Appendix A: How Kill QWK Dupes operates (aka what is a duplicate message?) message?) 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Kill QWK Dupes works by maintaining its own table of message CRC's. For every message scanned, the CRC16 of the message body up to but not including taglines (if present) is calculated. Next the CRC16 of the message headers TO field is calculated. All other portions of the message header are ignored as unimportant information for the purpose of duplicate message detection. The CRC table is then scanned for a matching pair of CRC values that also matches the conference number the message is stored in. If no match is found in that conference, the new pair is added to the table for that conference number. If the addition of the CRC pair causes the CRC table to be filled, the 32 oldest CRC pairs are thrown away, making room for 32 more CRC pairs. If on the other hand a CRC pair match is found in 4 or more different conferences, the excess messages will also be removed as duplicate messages. This prevents the "letter bomb", a case where one user posts the same message to all conferences he has access to. If however a 3 way match (CRC16 of the message body, and CRC16 of the TO field, and the conference number) is found, the message is removed from the input file and the number of duplicates found counter is incremented by one. Kill QWK Dupes also maintains a table of date & time stamp table pointers so that it knows the exact age of each CRC pair contained in the table. When it finishes scanning, Kill QWK Dupes will trim the CRC table size to the maximum number of CRC entries you have specified, and also remove CRC pairs that are older then the date you have specified. Unless the -NoKill option is specified, Kill QWK Dupes will also modify the first 128 bytes of the output file (a header block) with a signature and ending pointer such that if the file is added onto (a node does an additional export appending to an existing .REP packet for example) Kill QWK Dupes will not re-scan the entire packet, but resume its scan at the point where the new messages were added. This prevents running Kill QWK Dupes on the same packet more then once and accidentally removing the already scanned messages believing they are duplicates. Each CRC pair entry in the table uses 6 bytes. Each date & time stamp table pointer uses 6 bytes. There's also 6 bytes of overhead in the CRC file when stored on disk. 11 Appendix B: How to install Kill QWK Dupes into the MarkMail door Appendix B: How to install Kill QWK Dupes into the MarkMail door 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This appendix is added with kudos to Mike King of IDC BBS for figuring out how to install Kill QWK Dupes in the MarkMail door. In order to make Kill QWK Dupes work with the MarkMail door, it must intercept MarkMail at a point where it's about to process the .REP packet. You do NOT want to intercept and scan the .QWK packets MarkMail sends because Kill QWK Dupes would see all messages sent to the various users as duplicates and remove them accordingly. The best place to intercept the .REP packet processing is where MarkMail is shelling out to a decompression program in order to decompress the .REP packet. What you do is to insert a batch file of your own that will decompress the .REP packet, scan for duplicates and leave the files just where MarkMail expects to find them. The following 5 steps (which assume you already have MarkMail working) outline the procedure on how to accomplish this: 1) Start by running the MarkMail System Manager program "MMSM": 旼컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴커 � MarkMail System Manager � � Copyright 1989, 1990, 1991 By Mark Turner � � � � 靈컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴캠 � � � Configuration Menu � � � 픔컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴캘 � � � � � � � A General Configuration � � � � B Compression Methods � � 읕컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴켸 2) Select option B to configure Compression methods: 旼컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴커 � �袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴� � � � Compression Methods � � � � � � � � � � � � � � � � � � Method #1 Name : Default (Default) � � � Compress Cmd Line : Pkzip � � � Un-Compress Cmd Line : Pkunzip � 읕컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴켸 3) On the third line substitute "MMKILLQ" in place of "Pkunzip". This is the name of the batch file that will do the decompressing and scanning of the .REP packets. You can use a different name if you wish. Leave all other items alone. 12 旼컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴커 � �袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴� � � � Compression Methods � � � � � � � � � � � � � � � � � � Method #1 Name : Default (Default) � � � Compress Cmd Line : Pkzip � � � Un-Compress Cmd Line : MmKillq � 읕컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴켸 4) After entering the change, press the Esc key twice to exit MMSM and save the change. 5) Next the batch file "MMKILLQ.BAT" must be created. Its contents should resemble the following (a sample MMKILLQ.BAT file has been included in the package for you to follow and modify as required): SET KILLQDUP = -MC:8000 -MD:14 -Com:1 PKUNZIP %1 KILLQ IDCBBS.MSG -CF:C:\QM4\IDCBBS.CRC -Log:C:\QM4\KILLQDUP.LOG DEL IDCBBS.BAK SET KILLQDUP = The first line sets the KILLQDUP environment variable such that the following KILLQ line doesn't become excessively long. In this case the environment variable is telling Kill QWK Dupes to maintain a maximum of 8000 entries in the CRC table, to only track the last 14 days worth of CRC's and to talk to the user via Comm. Port 1. The next line does the actual decompression of the .REP packet. Next is the KILLQ line which does the actual scanning of file containing the messages from the user. This line is also telling Kill QWK Dupes to use the filename "IDCBBS.CRC" as the CRC file and to use the file "KILLQDUP.LOG" to write/append its processing results to. Be sure to replace "IDCBBS" with the actual BBS_ID that you go by. The next line simply erases the IDCBBS.BAK file that Kill QWK Dupes creates during its processing. Again, be sure to replace "IDCBBS" with the actual BBS_ID that you go by. The last line clears out the KILLQDUP environment variable. It is likely that you will have to modify the MMKILLQ.BAT file to fit your setup. At a minimum, the path to be used for the log file will have to be changed. The Comm. Port assignment may or may not need to be changed (or may be removed altogether if you wish). All the other parameters may be changed as well to best fit your personal preferences. 13 Appendix C: How to install Kill QWK Dupes into the Qmail door Appendix C: How to install Kill QWK Dupes into the Qmail door 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This appendix is added with kudos to Mike King of IDC BBS for figuring out how to install Kill QWK Dupes in the MarkMail door. In order to make Kill QWK Dupes work with the Qmail door, it must intercept Qmail at a point where it's about to process the .REP packet. You do NOT want to intercept and scan the .QWK packets Qmail sends because Kill QWK Dupes would see all messages sent to the various users as duplicates and remove them accordingly. The best place to intercept the .REP packet processing is where Qmail is shelling out to a decompression program in order to decompress the .REP packet. What you do is to insert a batch file of your own that will decompress the .REP packet, scan for duplicates and leave the files just where Qmail expects to find them. The following 5 steps (which assume you already have Qmail working) outline the procedure on how to accomplish this: 1) Start by running the Qmail Setup Utility program "QMSETUP": 旼컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴커 � �袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴� � � � Qmail Door 4.00 Setup Utility � � � Copyright 1990 by Sparkware � � � � � � 靈컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴� � � � � Main Menu � � � 봬컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴� � � � � � � � � A Paths & Filenames � � � � B Transfer Protocols � � � � C Archiver Definitions � 읕컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴켸 2) Select option C for Archiver Definitions: 旼컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴커 � �袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴� � � � Archiver Definitions � � � � � � � � � Archiver #1 : PkZip � � � Pack command : PKZIP -A � � � Unpack command : PKUNZIP -O � 읕컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴켸 3) On the third line substitute "QMKILLQ" in place of "PKUNZIP -O". This is the name of the batch file that will do the decompressing and scanning of the .REP packets. You can use a different name if you wish. Leave all other items alone. 14 旼컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴커 � �袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴� � � � Archiver Definitions � � � � � � � � � Archiver #1 : PkZip � � � Pack command : PKZIP -A � � � Unpack command : QMKILLQ � 읕컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴켸 4) After entering the change, press the Esc key twice to exit QMSETUP and save the change. 5) Next the batch file "QMKILLQ.BAT" must be created. Its contents should resemble the following (a sample QMKILLQ.BAT file has been included in the package for you to follow and modify as required): SET KILLQDUP = -MC:8000 -MD:14 -Com:1 PKUNZIP %1 %2 %3 KILLQ %2IDCBBS.MSG -CF:C:\QM4\IDCBBS.CRC -Log:C:\QM4\KILLQDUP.LOG DEL %2IDCBBS.BAK SET KILLQDUP = The first line sets the KILLQDUP environment variable such that the following KILLQ line doesn't become excessively long. In this case the environment variable is telling Kill QWK Dupes to maintain a maximum of 8000 entries in the CRC table, to only track the last 14 days worth of CRC's and to talk to the user via Comm. Port 1. The next line does the actual decompression of the .REP packet. Next is the KILLQ line which does the actual scanning of file containing the messages from the user. This line is also telling Kill QWK Dupes to use the filename "IDCBBS.CRC" as the CRC file and to use the file "KILLQDUP.LOG" to write/append its processing results to. Be sure to replace "IDCBBS" with the actual BBS_ID that you go by. The next line simply erases the IDCBBS.BAK file that Kill QWK Dupes creates during its processing. Again, be sure to replace "IDCBBS" with the actual BBS_ID that you go by. The last line clears out the KILLQDUP environment variable. It is likely that you will have to modify the QMKILLQ.BAT file to fit your setup. At a minimum, the path to be used for the log file will have to be changed. The Comm. Port assignment may or may not need to be changed (or may be removed altogether if you wish). All the other parameters may be changed as well to best fit your personal preferences. 15 Appendix D: How to install Kill QWK Dupes into the RoseMail door Appendix D: How to install Kill QWK Dupes into the RoseMail door 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This appendix is added with kudos to Peter Wadsworth of Coconino BBS for figuring out how to install Kill QWK Dupes in the RoseMail door. In order to make Kill QWK Dupes work with the RoseMail door, it must intercept RoseMail at a point where it's about to process the .REP packet. You do NOT want to intercept and scan the .QWK packets RoseMail sends because Kill QWK Dupes would see all messages sent to the various users as duplicates and remove them accordingly. The best place to intercept the .REP packet processing is where RoseMail is shelling out to a decompression program in order to decompress the .REP packet. What you do is to insert a batch file of your own that will decompress the .REP packet, scan for duplicates and leave the files just where RoseMail expects to find them. The following 5 steps (which assume you already have RoseMail working) outline the procedure on how to accomplish this: 1) Start by running the RoseMail Manager program "RM" and go to the Packet decompression screen: �袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴 � RoseMail 1.52 System Manager (08-09-91 02:30) � Copyright (C) 1991 Rose Media Incorporated � � Packet decompression information � 튛etter ID Name Decompression program ==> � A 1A Phil Katz's PKARC PKARC -x � Z 50 Phil Katz's PKUNZIP PKUNZIP -o � 00 � 00 � 00 � 00 � � Press ESC (right mouse button) to return to main menu 勁袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴� 2) On the second line substitute "RMKILLQ" in place of "PKUNZIP -o". This is the name of the batch file that will do the decompressing and scanning of the .REP packets. You can use a different name if you wish. Leave all other items alone. 16 �袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴 � RoseMail 1.52 System Manager (08-09-91 02:30) � Copyright (C) 1991 Rose Media Incorporated � � Packet decompression information � 튛etter ID Name Decompression program ==> � A 1A Phil Katz's PKARC PKARC -x � Z 50 Phil Katz's PKUNZIP RMKILLQ � 00 � 00 � 00 � 00 � � Press ESC (right mouse button) to return to main menu 勁袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴袴� 3) After entering the change, exit RM to save the change. 4) Next the batch file "QMKILLQ.BAT" must be created. Its contents should resemble the following (a sample QMKILLQ.BAT file has been included in the package for you to follow and modify as required): SET KILLQDUP = -MC:8000 -MD:14 -Com:%DSZPORT% -Log:D:\KILLQDUP.LOG PKUNZIP %1 %2 IF "%2"=="COCONINO.MSG" KILLQ %1 -CF:F:\PCB\COCONINO.CRC DEL COCONINO.BAK SET KILLQDUP = The first line sets the KILLQDUP environment variable such that the following KILLQ line doesn't become excessively long. In this case the environment variable is telling Kill QWK Dupes to maintain a maximum of 8000 entries in the CRC table, to only track the last 14 days worth of CRC's and to talk to the user via the Comm. Port DSZ will use. The next line does the actual decompression of the .REP packet. Next is the KILLQ line which does the actual scanning of file containing the messages from the user. This line is also telling Kill QWK Dupes to use the filename "COCONINO.CRC" as the CRC file and to use the file "KILLQDUP.LOG" to write/append its processing results to. Be sure to replace "COCONINO" with the actual BBS_ID that you go by. The next line simply erases the COCONINO.BAK file that Kill QWK Dupes creates during its processing. Again, be sure to replace "COCONINO" with the actual BBS_ID that you go by. The last line clears out the KILLQDUP environment variable. It is likely that you will have to modify the RMKILLQ.BAT file to fit your setup. At a minimum, the path to be used for the log file will have to be changed. The Comm. Port assignment may or may not need to be changed (or may be removed altogether if you wish). All the other parameters may be changed as well to best fit your personal preferences. 17 Appendix E: How to install Kill QWK Dupes into the Tomcat! door Appendix E: How to install Kill QWK Dupes into the Tomcat! door 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This appendix is added with kudos to William Padilla of Panasia BBS for figuring out how to install Kill QWK Dupes in the Tomcat! door. In order to make Kill QWK Dupes work with the Tomcat! door, it must intercept Tomcat! at a point where it's about to process the .REP packet. You do NOT want to intercept and scan the .QWK packets Tomcat! sends because Kill QWK Dupes would see all messages sent to the various users as duplicates and remove them accordingly. The best place to intercept the .REP packet processing is where Tomcat! is shelling out to a decompression program in order to decompress the .REP packet. What you do is to insert a batch file of your own that will decompress the .REP packet, scan for duplicates and leave the files just where Tomcat! expects to find them. The following 5 steps (which assume you already have Tomcat! working) outline the procedure on how to accomplish this: 1) Bring up the Tomcat! configuration file into a text editor. ID = PANASIA CITY = Glendale, CA PHONE = 818-240-0415 WORK = C:\TOMCAT LOCAL = E:\DOWNLDS\ PACKER = Z , PKZIP/PKUNZIP , ZIP , PKZIP -A ! , PKUNZIP ! EXTERNAL = HELLO = PRELOG . . . 2) On the PACKER" line substitute "COMMAND /C TCKILLQ !" in place of "PKUNZIP !". This is the name of the batch file that will do the decompressing and scanning of the .REP packets. You can use a different name if you wish. Leave all other items alone. PACKER = Z , PKZIP/PKUNZIP , ZIP , PKZIP -A ! , COMMAND /C TCKILLQ ! 3) Save the configuration file and exit the text editor. 4) Next the batch file "TCKILLQ.BAT" must be created. Its contents should resemble the following (a sample TCKILLQ.BAT file has been included in the package for you to follow and modify as required): SET KILLQDUP = -MC:8000 -MD:14 -Com:1 PKUNZIP %1 KILLQ PANASIA.MSG -CF:D:\WC\KILLQDUP.CRC -LOG:D:\WC\KILLQDUP.LOG DEL PANASIA.BAK SET KILLQDUP = 18 The first line sets the KILLQDUP environment variable such that the following KILLQ line doesn't become excessively long. In this case the environment variable is telling Kill QWK Dupes to maintain a maximum of 8000 entries in the CRC table, to only track the last 14 days worth of CRC's and to talk to the user via Comm. Port 1. The next line does the actual decompression of the .REP packet. Next is the KILLQ line which does the actual scanning of file containing the messages from the user. This line is also telling Kill QWK Dupes to use the filename "PANASIA.CRC" as the CRC file and to use the file "KILLQDUP.LOG" to write/append its processing results to. Be sure to replace "PANASIA" with the actual BBS_ID that you go by. The next line simply erases the PANASIA.BAK file that Kill QWK Dupes creates during its processing. Again, be sure to replace "PANASIA" with the actual BBS_ID that you go by. The last line clears out the KILLQDUP environment variable. It is likely that you will have to modify the TCKILLQ.BAT file to fit your setup. At a minimum, the path to be used for the log file will have to be changed. The Comm. Port assignment may or may not need to be changed (or may be removed altogether if you wish). All the other parameters may be changed as well to best fit your personal preferences. 19 Appendix F: How to install Kill QWK Dupes into the Jetmail door Appendix F: How to install Kill QWK Dupes into the Jetmail door 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 This appendix is added with kudos to Ron Hossack of the Solid Rock BBS for figuring out how to install Kill QWK Dupes in the Jetmail door. In order to make Kill QWK Dupes work with the Jetmail door, it must intercept Jetmail at a point where it's about to process the .REP packet. You do NOT want to intercept and scan the .QWK packets Jetmail sends because Kill QWK Dupes would see all messages sent to the various users as duplicates and remove them accordingly. The best place to intercept the .REP packet processing is where Jetmail is shelling out to a decompression program in order to decompress the .REP packet. What you do is to insert a batch file of your own that will decompress the .REP packet, scan for duplicates and leave the files just where Jetmail expects to find them. The following 5 steps (which assume you already have Jetmail working) outline the procedure on how to accomplish this: 1) Bring up the Jetmail configuration file into a text editor. . . . PACK1=ZIP PACK1QWK=PKZIP -M PACK1REP=PKUNZIP -O PACK2=ARJ . . . 2) On the "PACK1REP" line substitute "COMMAND /C JMKILLQ" in place of "PKUNZIP -O". This is the name of the batch file that will do the decompressing and scanning of the .REP packets. You can use a different name if you wish. Leave all other items alone. PACK1REP=COMMAND /C JMKILLQ 3) Save the configuration file and exit the text editor. 4) Next the batch file "JMKILLQ.BAT" must be created. Its contents should resemble the following (a sample JMKILLQ.BAT file has been included in the package for you to follow and modify as required): SET KILLQDUP = -MC:8000 -MD:14 -Com:1 PKUNZIP -O %1 %2 KILLQ D:\TMP\ROCK.MSG -CF:C:\SF\JM\ROCK.CRC -Log:C:\SF\JM\KILLQDUP.LOG DEL D:\TMP\ROCK.BAK SET KILLQDUP = 20 The first line sets the KILLQDUP environment variable such that the following KILLQ line doesn't become excessively long. In this case the environment variable is telling Kill QWK Dupes to maintain a maximum of 8000 entries in the CRC table, to only track the last 14 days worth of CRC's and to talk to the user via Comm. Port 1. The next line does the actual decompression of the .REP packet. Next is the KILLQ line which does the actual scanning of file containing the messages from the user. This line is also telling Kill QWK Dupes to use the filename "ROCK.CRC" as the CRC file and to use the file "KILLQDUP.LOG" to write/append its processing results to. Be sure to replace "ROCK" with the actual BBS_ID that you go by. The next line simply erases the ROCK.BAK file that Kill QWK Dupes creates during its processing. Again, be sure to replace "ROCK" with the actual BBS_ID that you go by. The last line clears out the KILLQDUP environment variable. It is likely that you will have to modify the JMKILLQ.BAT file to fit your setup. At a minimum, the path to be used for the log file will have to be changed. The Comm. Port assignment may or may not need to be changed (or may be removed altogether if you wish). All the other parameters may be changed as well to best fit your personal preferences. 21 Appendix G: Multi-node considerations Appendix G: Multi-node considerations 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Kill QWK Dupes is fully network/DESQView and multi-node compatible. However there are some considerations that should be discussed as they may affect your setup. First, under multi-node operation, which Comm. Port to use is generally passed to the door program (such as Qmail and MarkMail) via some sort of PCBOARD.SYS/DOOR.SYS info. file. Since Kill QWK Dupes is unable to extract the Comm. Port information from your info. file, this information must be conveyed via some other means (or not specified at all such that Kill QWK Dupes doesn't attempt to talk to any Comm. Port). The best and easiest way to specify it is to have the KILLQDUP environment variable set permanently as part of the batch file that brings up the individual node. The Comm. Port to use would be specified in the environment variable, and the two "SET KILLQDUP" lines in the QMKILLQ.BAT and MMKILLQ.BAT files would have to be eliminated. This method allows a common batch file be run by the mail door to do the scanning, and still have the correct Comm. Port used by Kill QWK Dupes. Another method would be to setup each node such that a different batch file is run by each node when the mail door shells out to decompress the mail packet. Each batch file would have to correctly specify the Comm. Port for Kill QWK Dupes to use. Secondly file sharing/locking must be discussed. In order for a multi-node system to use and share the same CRC file among all the nodes, Kill QWK Dupes uses file sharing and locking. But in order for this feature to be active, you must be running DOS 3.0 or above, and have SHARE loaded. If you are not running multi-node, then DOS 2.x is all that's required to run Kill QWK Dupes since file sharing/locking is not required. Appendix H:Errorlevels returned by Kill QWK Dupes Appendix H:Errorlevels returned by Kill QWK Dupes 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 Kill QWK Dupes returns errorlevels to help make its use in a batch file easier. The following is a list of the errorlevels and their meanings: 0 - Normal completion. At least one duplicate message was found and removed (unless -NoKill was specified). 1 - Normal completion. No duplicates were found. 2 - Command line parameter error or no file specified to scan. 3 - Insufficient memory to allocate required data arrays. 4 - Unexpected system error occurred. 5 - Disk full error while writing to file. 22 History History 컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴컴 10/30/91 v1.00 - Initial public release. 11/05/91 v1.01 - Return errorlevels for better batch file use. 11/10/91 v1.02 - Calculate correct base comm. port address. - Display message status for each message scanned. - Handle case where -DN parameter is not specified. 01/09/92 v1.03 - Ignore extra spaces at end of lines when computing message CRC. - Added the -DC parameter. - Added PCRELAY as a recognized mail door name. 23