WSSoup Version 0.6a - 5 February 1995 (c) Copyright 1995 Magnus Cameron. All Rights Reserved magnus@ozemail.com.au http://www.ozemail.com.au/~magnus SUMMARY "A WinSock compliant Usenet news, NNTP to SOUP offline news and mail packer for internet users." DISCLAIMER Do not use this software for any purpose. Although all reasonable care has been taken to ensure that it performs as documented it has only been tested on a computer (yes, one!). There is a probability that use of this software may cause a loss of profit, life, property and/or other unpredicted events that may cause the user or others grief, pain, suffering, loss of income and other predictable and unpredictable effects. The author of this software advises you not to use this software for any purpose. In addition: 'THIS SOFTWARE IS PROVIDED "AS IS" AND WITHOUT ANY EXPRESS OF IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.' COPYRIGHT INFORMATION "The product", herein after referred to as "WSSoup" and "the software", includes all files supplied in the ".zip" archive file one of which is this document. This copyright information pertains to this version of the software known as "WSSoup 0.6a" and will also apply to other versions if not otherwise stated. Redistribution and use is permitted provided that the above copyright notice and this document are duplicated in all such forms and that any documentation, advertising materials, and other materials related to such distribution and use acknowledge that the software was developed by Magnus Cameron. In addition, if you wish to distribute this program with other samples of WinSock programs, or as a program on a CDROM or any other such distribution you must send the author a complementary copy of that distibution. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. Specifically, do not modify this program in any way and re-distribute it. WSSoup is copyrighted, shareware software. You are granted a license to use the software for a period of 30 days from first usage to evaluate the software to determine if you wish to continue using it. If you decide to continue using the software after this period, you must pay a license fee of $10US (or $10AUS in Australia) for each PC on which the software is used. In return, you will be given a license to use the product for at least the next two upgrades and substantial discounts on later versions and other products; in addition, the word "UNREGISTERED" will not appear on all news messages that you send. Please see the end of this document for registration payment details. A number of other items mentioned in this document or used by the software are copyrighted as follows: SOUP Simple Offline USENET Packet Format (SOUP) Version 1.2, Copyright (c) 1992-1993 Rhys Weatherley Paberboy Copyright (c) 1994 Michael Vartanian Windows Sockets Copyright 1992 by Martin Hall, Mark Towfiq Geoff Arnold, David Treadwell and Henry Sanders Portions of the Windows Sockets specification are derived from material which is Copyright (c) 1982-1986 by the Regents of the University of California. All rights are reserved. The Berkeley Software License Agreement specifies the terms and conditions for redistribution. Trumpet Newsreader, Trumpet Winsock (c) 1993-1994 Peter R. Tattum Chameleon Sampler (c) 1990-1993 Netmanage Inc. *If any other copyrighted or trademarked items appear in this document, a credit has not been left out intentionally. Please notify the author immediately of any such ommisions so that it can be rectified.* The following "Network Working Group" RFCs were referred to create this software: RFC 821 - Simple Mail Transfer Protocol RFC 822 - Standard for ARPA Internet Text Messages RFC 850 - Standard for Interchange of USENET Messages RFC 977 - Network News Transfer Protocol RFC 1460 - Post Office Protocol - Version 3 RFC 1651 - SMTP Service Extensions Thank you to those of you who have come before me and created the standards that made this project possible. QUICK START Copy all of the distribution files into an empty directory. Create three subdirectories, control, inbound and outbound. Edit the WSSOUP.INI file to match your directories and the other information required in the WSSOUP.INI file. Note that the WSSOUP.INI file must be located in the same directory as the program. Create a "COMMANDS" file to subscribe to the newsgroups you want to subscribe to and place it in the outbound directory. If your offline reader does not support a COMMANDS file, you must create a text file that has Line Feed only delimited records - Carriage Return/Line Feed records may not work. I use the "Programmer's File Editor (PFE)" (Copyright (c) Alan Phillips 1992 1993 1994) to create files in this format (also known as "Unix" format as opposed to "MSDOS" format files). Start your WINSOCK.DLL process up and connect to your internet service provider. Run "WSSOUP.EXE" and watch the modem lights flash :-) Basically, WSSoup firstly downloads any new news in the areas you are subscribed, uploads any replies, downloads any new news and mail, and sends any pending news posts and mail. See SOUP for full information on file formats used. You should read the screen for any error messages. These are written to DEBUG.TXT, INFO, and/or ERROR. A message will be displayed that "WSNews is complete" when the program has finished executing. If you have "AutoExit" set, the program will then exit. Otherwise, you will need to select "File/Exit" or double click on the system menu. The first time the program runs it may take some time as it downloads a list of all of the available newsgroups. Each subsequent time, it will add new newsgroups to the end of this list. INTRODUCTION I started using the Internet in November 1994. After using BBS's for a number of years I surprised to find a general lack of offline mail and news software that was "freely available". A look at the offline news conferences found enough information in the FAQs to inform me that there simply isn't enough software out there. "Hooray". A piece of software that I can write that nobody else has before me that I know of, and that others might just want to use. So here it is - WSSoup - Windows Sockets Simple Offline Usenet Packet WHAT DOES IT DO? Transmits news between a NNTP host and a PC, into SOUP format for reading by SOUP compliant news readers. Transfers news from the PC back to the NNTP host. Sends mail using SMTP from the PC to the Internet. Receives mail from the Internet to your PC using POP3. Maintains a list of all available news groups. Allows news groups to be subscribed and unsubscribed. Keeps track of what news has been received and endeavours not to download it again, using a "last message retrieved" pointer. If required, allows only news "headers" to be retrieved, and then later for the complete article contents to be retrieved, on a news group by news group basis. WSSoup complies with SOUP 1.2 "MINIMAL CONFORMANCE" as well as a number of additional features. WSSoup will be fully SOUP compliant in the near future. Can be used to update an existing inbound SOUP packet, so you don't have to move a packet to a different directory. Allows you to restrict the number of messages received for each group and to specify whether the latest or earliest messages will be retrieved. HOW DOES IT WORK, HOW DO I INSTALL IT? WSSoup is VERY boring as far as windows interfaces go. I considered giving it an interface but couldn't really think of anything interesting to do. Now that I have used the software I have some ideas for a better interface but I see no urgent requirement at this time. As for installation, time has prevented me from writing an installation and maintenance utility and this is high on the list of features for the near future. As an aside, as there is no user interaction required, the system can be configured to automatically pick up your mail and news without you having to be in attendance. OK. There are four directories required, as follows: INBOUND This is where mail you receive from the NNTP server will be placed, in SOUP format. This can be cleaned out at the beginning of each session if you so wish - so make sure you move it out of there after each session. Alternatively WSSoup can be configured to merely update the files in that directory with new news. It will append to existing files if the same areas receive new news. See the option "DeleteBeforeReceive". OUTBOUND This is where you place your "unzipped" or "uncompressed" SOUP mail packet for sending to the server. Normally, after sending, the files in this directory will be deleted to ensure that you don't send duplicate news and mail. This feature can be turned off if you wish. See the option "DeleteAfterSend". CONTROL This is where WSSoup stores important control files to keep track of what news groups you are subscribed to and what news you have already received. The files in this directory are guaranteed not to remain in their current format. Also, be very careful if you try to edit them. They may look like text files, but might be binary files! WSSoup This is the WSSoup directory where the software is installed, and also where the .INI file is stored. Yes, the .INI file is in the application directory. I have my reasons for doing this at present. A future version will explain the logic behind this, but lets just say for now it has something to do with network and multiusers configurations. If you are a Network Adminstrator and don't want to allow clients write access to "program" directories, you will have to put up with this for the moment. Create yourself four directories and copy all of the program files into the WSSoup directory. Then, edit the WSSOUP.INI file to reflect the directories you chose. See the comments in the .INI file for assistance. As this stuff is still changing as I write this, I decided to place comments in the .INI file until I have bedded down all of the features required. MAKE SURE YOU CHANGE ALL OF THE OTHER SETTINGS IN WSSOUP.INI BEFORE YOU CONTINUE. IF YOU DON"T KNOW WHAT AN OPTION DOES COMMENT IT OUT AND IF THE PROGRAM CRASHES WITH AN ERROR THAT THE SETTING IS REQUIRED THEN YOU WILL HAVE TO WORK OUT WHAT THE OPTION IS FOR. You should now create a "COMMANDS" file and place it in you OUTBOUND directory. You should include a SUBSCRIBE command to subscribe to a group. The distribution comes with an example COMMANDS file that you can place into your OUTBOUND directory. Remember that all files in the OUTBOUND directory are in UNIX format, so COMMANDS MUST have lines terminated with line feed characters only, not carriage return line feed that almost all DOS text editors create. Thats it. To test that all works, connect to your network and run the WSSoup.exe program. A boring window will explain what is happenning. If all works, when its finished you should have some files in the INBOUND directory. Start your SOUP offline mail reader. Now create a reply packet - subscribe to a newsgroup - copy it to the OUTBOUND directory and restart WSSoup. Your news replies should be uploaded to your news host. AVAILABILITY WSSoup will be released on a regular basis as bugs are rectified and new features are implemented and tested. Frequency will also be related to feedback received to the author. Lots of positive feedback will result in more frequent releases. Each version will have an expiry mechanism built in so that the program only lasts about two months. Once all "primary" features have been implemented and bugs fixed, this expiry mechanism will be removed. For this reason I would prefer that you not redistribute this program at this time until all of the major problems are ironed out. I maintain a WWW home page which lists the latest information on WSSoup and also allows you to download a PKZIP (version 2 format) "zip" file. This can be found at: http://www.ozemail.com.au/~magnus SOUP 1.2 COMPLIANCE STATEMENT Commands supported: subscribe, unsubscribe, list, catchup See WSSOUP.INI for supported file formats. TECHNICAL SUPPORT More documentation and program update will follow in a few weeks, depending the level and type of feedback I receive. If you want a feature, or have some questions, please email me directly - I will reply to all mail that I have time to. I will also monitor the alt.usenet.offline-reader newsgroup for a short period after initial release and continue to do so if it appears to be an appropriate place to do so. All technical support and enhancement suggestions inquires should be sent directly by email to the author at magnus@ozemail.com.au. If you have problems getting things going, double check your WSSoup.INI file. Make sure that you can get news using an online news reader such as Winvn or Trumpet Newsreader. I would appreciate knowing *any* problems that you have getting the thing going that you have solved and believe may be able to help others. DOCUMENTATION If you have read this much, then you know that I'm not really into documentation. If you would like to volunteer a better document you are welcome to. I will redistribute quality documentation with subsequent versions if anyone would care to send some to me. I reserve the right to edit and redistribute any documentation sent to me for this purpose, but will give all credit where it is due. FEATURES CURRENTLY BEING IMPLEMENTED - An installation and reconfiguration utility - Different timing values for different operations - Better date and time support for LIST - synchronise with time service. - Allow news format to be changed easier - Support for the following SOUP commands: version, date, hostname, software, sendme, mail, deletemail, mailindex, newsindex, supported cmd - A respectable user interface with progress indicators - Support for "enhanced" featues described in but not completely implemented by SOUP - eg, message area summaries switching on and off (this is currently implemented in WSSoup but there is no supported mechanism to turn it on and off!). - Maximum time online / maximum data to download setting - OS/2 version KNOWN PROBLEMS - A number of the file formats have not been tested properly as I don't have an offline reader capable of reading/generating them. - This software has only been tested with PaperBoy. I would like some feedback from users of other products. - If the program crashes, it may leave two temporary files in the inbound directory - it will delete them next time the program runs. RELEASE HISTORY 0.6a - 6 February 1995 A very quick fix to increase some network buffer sizes that *might* be causing a problem. 0.6 - 5 February 1995 0.5 didn't last long as I left a nasty bug in the news retrieval code that broke the program. Now its fixed and works better. Changed the way that LIST processing is done. A new LIST will only be retrieved if one doesn't exist in the CONTROL directory. This is automatically kept up to date on each connection. If you want a fresh list just delete list.dat in the CONTROL directory. The LIST command now does what was intended of it and uses the local copy of list.dat in response to the LIST command. 0.5 - 5 February 1995 The offset field in index files was not being set properly if the SOUP packet was being updated. This problem did not surface if SOUP packets were being created in an empty directory. Modified the information displayed when downloading news from groups to make it a little more interesting than a "." for every five messages retrieved. If no news was actually retrieved for a subscribed newsgroup, no MSG or IDX file would be created. However, an entry was written to the AREAS file. This caused PaperBoy to complain and then GPF. This has been fixed by removing the offending record from the AREAS file after news has been retrieved. Messages can now be as large as 512k - up from 64k. The deletion of inbound and outbound files now works as documented in the .INI file. SMTP and POP3 mail now appear to work. Only plain ascii mail is supported. High bit bytes will be converted to the '@' character. The "mode reader" and "slave" commands have been implemented if your NNTP server requires them. Network error messages are now expanded correctly. The catchup and unsubscribe commands now work correctly. If you only want to collect a certain number of messages from a group, a new option allows you to specify whether you want to receive the latest or earliest messages. The last message pointer is set to the last message retrieved so this can be used to perform "trickle" updating or a catchup, whichever you prefer. The sendme details in the 'C' and 'c' format indexes has been modified to now use the newsgroup name and article number, rather than the message id. When the full sendme command is implemented, this will make it easier to determine where to place the message. 0.4 - 4 February 1995 Completely rewritten news fetching algorithm - date/time based newnews commands replaced with article number tracking and reorg handling. Removed "retrieve no mail before date" option as not usable with new algorithm. Cleaned up logging destinations. Fixed regressed features. Documentation improvements and corrections. Advertised for Beta testers in alt.usenet.offline-reader Made available by WWW. 0.3 - 31 January 1995 First alpha release 0.2 Successfully posting news. 0.1 Succesfully receiving news. ABOUT THE DEVELOPMENT ENVIRONMENT I use a 486DX-33 with 8 meg and about 650meg of hard disk storage, Maestro 144FME modem, three CDROM drives, 4 port 16550 serial card. PPP using either Trumpet or Chameleon Sampler, Running OS/2 2.1, Borland C++ 4.5 and Watcom C/C++ 10.0. Although WSSoup is currently compiled for Windows, there is nothing stopping it being a native OS/2 app - as part of this project I have developed an object-oriented Socket module, called TWSocket. Some minor changes to this module would allow this program to be compiled under OS/2. If anyone would *really* like this feature, please attach this request to your registration cheque and consider it done ;-) About the only thing I am waiting for is the full version of OS/2 Warp to be released so that I can test the software before releasing it. REGISTRATION This software is being marketed as SHAREWARE. This software has cost me heaps of money to develop and test cause I pay for my Internet access by the hour and development software 'aint exactly cheap. However, this version of the software will expire on 30 March 1995, as it is a Beta version and I expect quite a number of fixes to be implemented between now and then. If you want to send money for registration, you are welcome to do so and your registration fee will be valid from the time the first non beta version is released. Registration costs $10US outside of Australia, or $10AUS within Australia. Payment can be made by cheque or money order. In return, you will receive a password that removes any "UNREGISTERED" notices. In addition, the program expiry date will not occur. You must send your Internet email address so that information can be sent to you. This must be the EXACT Internet email address that you use in the "From" line in the WSSOUP.INI file. Send money to: Magnus Cameron PO Box 363 KNOXFIELD MDA 3180 AUSTRALIA Thanks for your support.