P T E R M Paragon Terminal Application for Windows NT Version 04b.p10 1, January 1994 What's Going on Here? ===================== This is a beta of Paragon Terminal Application for Windows NT. What's so great about PTERM? Not much yet, but here is a list of features available so far: o Fully threaded, native Win32 application o Auto Zmodem download/upload with crash recovery and auto-renaming of incoming files which already exist. o Fast(er) ANSI (including color!) terminal emulation o Console based for speed o TELNET support The other valuable feature of PTERM is that as far as I know it is the only Zmodem capable native, multi-threaded Win32 terminal application available. It won't be for long, but for now... Some glaring omissions: o No Dialer!?!?! o No annoying "register me now, or die" messages and purposefully broken features. o Many other things which you will likely go insane without (but hang in there, it is my intention to evolve PTERM). *** Note: Please read the file RELNOTES.TXT before using PTerm. It contains important information on known bugs and limitations in this release of PTerm. About This Manual! ================== This is the sorriest excuse for manual yet -- but I am anxious to get this PTerm out to everyone, I'm not a tech-writer, and PTerm isn't all that compilcated. So if after reading this text you have the feeling that it was a poorly organized jumble of information, well, your feeling is correct. What I do hope this manual will do for you is to give you just enough information to get started using PTerm. If it doesn't, please email me and I will be happy to answer your questions (see my addresses below). Why Would You Do Such a Thing? ============================== Some months back, I was asked to beta test a C++ based, multi-platform communications library from a company called Lookout Mountain Software (William Herrera, owner/author, BBS 719-545-8572). The best method I could think of for testing this package was to use it as the core of a terminal application -- and PTERM was born. William has been extremely sensitive to the problems I found and enhancements I suggested, and should be commended for authoring such a stable and comprehensive communications library, which is available for DOS, Windows, OS/2 and Windows NT. All of the file transfer protocol code (Zmodem/Ymodem/Xmodem) is Williams, and he has done an excellent job implementing them (sealink and telink are also included in the communications library, but not in PTERM -- Xmodem and Ymodem will be available in the next release of PTERM). I Didn't Do It, I Swear! ======================== Every precaution has been taken to ensure the safe operation of PTERM in your Windows NT system. But, come'on, this copy of PTERM in beta, and I'm warning you of such, so obviously I cannot be held responsible for any damage done to your system (or psyche) by PTERM or any interaction involving PTERM. So all I can say is "I didn't do it!", so don't blame me! What Do I Need To Use PTerm? ============================ You need an Intel based (Intel Inside sticker optional) PC running Windows NT (March '93 build or later), with at least one serial port and a modem attached to that port. Actually, if you just want to use PTerm for TELNET you won't even need a serial port. However, I do not recommend using PTerm for any but the simplist of tasks while telneting. If someone would like to send me a MIPS R4400 or DEC Alpha AXP, I will be glad to get PTERM up on those platforms as well . Actually, I have had a number of requests for PTerm on the Alpha (surprisingly none for MIPS). My standard answer is that I don't have an Alpha machine and that since PTerm is based largely on a commercial library, I am not in a position to give the source to someone with an Alpha. This is too bad, but it will have to do for now... Alright, What Do You Want For This? =================================== The list of things I want would be much too large to include in this document, so you will have to settle for what I need. What I need, from you, is the following: o Bug reports (there should be plenty) o Suggestions (again, no shortage expected here either) So, please, please inundate me with this information, I promise I will consider anything submitted (but nothing not submitted). Whereami? ========= I can be reached via internet mail at: roncox@indirect.com rjc@infograph.com urjc!rjc@pcg.com FIDOnet people, try netmail to UUCP at 1:1/31, first line of the message body: To: roncox@indirect.com I plan on finding a reliable FIDOnet system here in town, and when I do I will post my FIDOnet address for netmail. You can reach me on Compuserve at: 71722,3175 For slowest response, choose snail-mail: Ron Cox ATTN: PTERM Paragon Consulting Group 4212 West Cactus STE 1110-229 Phoenix, AZ 85029 I'm Getting Sleeeepy... ======================= I know, I know -- I will make this easy on you (and me) and keep things real short. First, a couple of command line switches have been added to PTerm. They are: -s Go straight into the setup dialog. When PTerm is first run on your system, it will not find a PTERM.INI file to read configuration from. If you have a COM1, this is no problem since PTerm will default to COM1 and run, at which point you can press ALT-S to run through the setup (see below). However, if you have no COM1, PTerm will complain and then exit. Passing the '-s' option bypasses the check and lets you configure PTerm straight away. -ifile Since PTerm keeps its setup data in an ini file, I needed some method of allowing multiple copies of PTerm to run. PTerm normally defaults to looking for PTERM.INI in the \WINNT directory. By passing a '-ifile' switch, you can change which ini file PTerm will use. For instance, if PTERM.INI sets up PTerm for COM1, and you also want to run PTerm on COM2, you can copy PTERM.INI to PTERM2.INI and run PTerm as such: pterm -s -ipterm2.ini Make sure the -s is first on the command line! Now, go into the setup and change the comport to COM2, click Ok, then Save. Now you can use the above command line (removing the -s) to invoke PTerm on COM2. Note, the ini files MUST be in the \winnt directory, and you must pass only the name (not the path) with the '-i' switch. As is obvious from the discussion above, PTerm no longer uses environment variables for configuration! So you can remove them all from your PT.CMD (or master environment). PTerm configuration is now done through a set of dialog boxes. You get to the main setup menu by pressing ALT-S. From here you can choose to setup "Communications", "Modem Setup", "Macros", or "File Transfers". At the bottom of the main setup menu are three buttons: Save - Save any changes made to the setup. Apply - Apply the changes made but do not save them Done - Don't apply or save any changes made All of the options in these setup dialogs are self explanatory (I think), so I am not going to spend alot of time going over them. If you have questions, just drop me some email and I will be happy to help! The one area I would like to discuss is the "Macros". These are literal strings which PTerm sends out when the appropriate key is pressed (F1-F12) -- PTerm does NO fancy substitutions or expansions on these these strings except to append a carriage return to them before sending them out to the modem. The macro capability was added out of guilt for not adding a dialer, and as such is useful for defining upto 12 dial strings. Hitting ALT-C will show you the current settings as well as a list of key commands (this is the same information which appears when PTerm is first run). PTERM can be placed in (and executed from) the program manager, and even has its own embedded icon (just like a real Windows program!). When using a batch enabled file transfer protocol (Zmodem), the file selection dialog box which pops up for an upload will allow you to select multiple files from any one directory, using the standard Windows mouse/key combinations for multi-select lists (i.e. CTRL-CLICK adds a file to the list, and try SHIFT-CLICK to extend the list to the current point). For batch uploading, if PTERM finds a file called FILESTO.UPL in the current directory (the directory you were in when you started PTERM), it expects this file to contain a list of files (full paths) to upload, and will attempt to do so. If PTERM finds this file, it will go straight to uploading, bypassing the file selection dialog. I am sure I have forgotten a whole bunch of information here, but this should be enough to get you off the ground. Can I Run it Now? ================= Well... Ok... But, "Don't touch it, you'll break it..." (U.S. West T.V. ad) For those of you who have been spoiled all their lives by a dialer and do not know how to do it manually, if you have a modem which supports the Hayes command set (are there any which don't?) you can dial a number from PTERM like so: atdt555-5555 Tone dial 555-5555 atdp555-5555 Pulse dial 555-5555 (yuck!) On my USR Sportster, the command a/ executes the last command entered, and can be used to redial. Again, you can add upto 12 dial strings as macros if you like. PTerm now has text capture. It is toggled by pressing ALT-L. When activated, PTerm will bring up a standard Windows dialog box for getting a file name. Enter a valid file path and click Ok. If you are currently capturing text, the title bar of PTerm will have the word "Capturing" at the end of it. Press ALT-L to close the capture file. If you exit PTerm with an open capture file, PTerm will close it for you before terminating. PTerm pretty much stores everything except ANSI escape sequences in the capture file. Also, if you use a file which already exists, PTerm will append captured text to that file without disturbing what was already there. Some Closing Thoughts ===================== Its not much, but there it is. Please let me know of any problems or suggestions for enhancements you have. On my 386-33, running the NT GA (with USP1 installed), I have experienced excellent transfer speeds using the Zmodem in PTERM. On text files, at 14.4K with .v42bis (57.6K maximum), I have seen 3800+ cps. The same setup on compressed files yields 1650 cps and higher. The NT serial driver is absolutely flawless in its support of FIFO's. Make sure you have a serial card with a 16550AFN FIFO on it and you will get much better file transfer results! Another facet of the serial driver which operates well is RTS/CTS flow control. This is always turned on when PTERM runs, later it will be configurable. During heavy multi-tasking, RTS/CTS keeps the modem under control, holding it off when the serial buffer gets full. This has worked flawlessly. Ron Cox Paragon Consulting Group Special Thanks ============== To the following, I give special thanks: Dale Ross for being a sort of liason between Microsoft and myself to get some nasty bugs fixed. Not to mention the extensive testing he has done for me (also the other members of Team PTerm -- Bob Chronister, Michael Rod, Stephen Purpura, and Randell Kennedy)! Greg Kochaniak for his *extensive* help in getting PTERM's telnet support to be even as useful as it is! To ALL the other people (especially on internet) who emailed me with suggestions, and a couple even sent source code. Thanks! Technically Speaking ==================== Thought you were done, didn't you? Well, if you could care less about some of the technical details of PTERM, you are, else read on. As mentioned, PTERM makes full use of multi-threading. This stuff is a trip. It can make programming much more interesting (and in many cases greatly simplifies things!). There are 7 threads of execution, 6 secondary and the 1 main thread which all Win32 applications start with. The first two threads of interest are in Williams communications library. One is responsible for taking characters in from the serial port (actually, the serial driver) and placing them in a queue created by the library (the input queue) which is made visible to the user code. The second takes characters from another queue and writes them to the serial port (driver). The user code is responsible for placing characters to be sent out the port into this output queue, and does so through a set of 'Send' member functions. Also, both of Williams threads use overlapping (asynchronous) I/O, resulting in an even more efficient set up. Now for the threads I spawn. They are as follows: o Input thread Because I did not want to muck with the input queue being managed by the communications library, I created another layer. My input thread simply waits for characters to appear in the main input queue, and block copies them into a local queue used by the display thread (below). If there are no characters waiting in the main input queue, this thread sleeps until there is (in the discussions which follow, this thread will be referred to as 'my input thread', to differentiate it from the main input thread operating in the library). It may be that I can increase PTERM's efficiency slightly by removing this layer and having the display thread pull characters directly from the library managed input queue -- something I will consider. o Display thread This thread is solely responsible for removing characters from the local input queue (managed above) and deciding what to do with them. For instance, if the beginning of an ANSI escape sequence is detected this thread passes control to a function whose job it is to interpret the sequence (note: the ANSI code still executes within the display thread, no other thread has been created). One of the other jobs of the display thread is to optimize the output into the console window. This is done in a very simplistic manner. In the absence of ANSI escape sequences, the display thread will accumulate characters from the local input queue into a buffer -- up to 80. When the limit of 80 has been reached, or a special sequence is detected, the display thread blasts the buffer to the screen in a single call to WriteFile(). This is MUCH more efficient than calling something like putch() for each single character. However, at slow speeds (1200 and 2400 baud), it takes a noticeable amount of time to accumulate 80 characters, and this is the reason for the choppy display at these speeds. If you set the port to anything less than 9600 baud, PTERM reduces this 'blast' count to 8 characters, producing a smoother display. As if this wasn't enough, the display thread has one more job to perform. It watches for the tell-tale sequence of characters which signal the host is preparing for a Zmodem upload or download. If this sequence is found, it starts the appropriate code. If there are no characters in the local input queue, the display thread sleeps until there are. o User thread The user threads job is pretty easy. If the user hits a key, decide what to do with it. Typically the key will be sent right out the port. If it is a control key sequence (like ALT-X), then the user thread executes the code appropriate for that key. This thread blocks on the keyboard, and as such sleeps when there are no characters available. o Main thread The main thread is the first thread which executes (starting in main()). It initializes the port and other things, starts up the three threads above, and then goes to sleep waiting for all three of the threads above to terminate. At this point it wakes up and calls ExitProcess(), ending PTERM. Them's the threads, and a nice bunch of threads they are. However, if there is one thing I quickly learned from spawning threads all over the place, its that synchronization thingy. Here's the scenario: My input thread pulls characters from the main input queue to place in the local input queue. Fair enough. So, I start a Zmodem download. Boom! Of course, the Zmodem download code also wants to pull characters from the main input queue, so my thread fights with the Zmodem code. Remember, the Zmodem code is run as part of the display thread. So my input thread grabs some characters, then the Zmodem code [running in the display thread] grabs some characters, and so on. Zmodem will end up missing a bunch of characters it expected to see, and my input thread grab some characters from the transfer which the display thread will finally get and try to display. A mess... So, I need to find a way to synchronize the two threads. When Zmodem wants control of the main input queue, it needs to 'ask' for control from my input thread. Well, it really isn't as formal as all that. Just before my input thread grabs characters from the main input queue, it waits on a semaphore. Simplified, a semaphore is just an object which keeps count of how many threads have asked for control of it. When its count is 0, access is granted to the waiting thread. When a thread gets access to it, the semaphores count is incremented. Any other thread which waits on it goes to sleep until the semaphores count becomes 0 again. The count is decremented when a thread explicitly releases a semaphore, thus allowing another thread to gain access (this can get complicated when there are more than 2 threads). Ok, where were we? Oh, yes, my input thread waits on this semaphore gadget. Typically, it gets control instantly and drops into the main body of its code where it grabs characters from the main input queue and stuffs them in the local queue the display thread uses. Once it has transferred some characters, it releases the semaphore (and gives up the rest of its time slice). Its during the time between releasing the semaphore and waiting on it again that the file transfer code must act. The first line of code for a file transfer waits on the same semaphore. As soon as my input thread releases the semaphore, the file transfer code gets and holds access to it until the transfer is completed, thus preventing my input thread from mucking with the main input queue during the transfer. Whew! Being this is my first foray into threads, I am very interested to find out if my code breaks on a multi-processor machine. Does my code work now because it takes advantage of the synchronous behavior of a single processor? Interesting stuff, but I forgot to pick up my Sequent 16 processor monster-box at the grocery store the other day, so I suppose the answer will have to wait... I found one other interesting thing about Windows NT -- it seems to like to write data to the disk right away. Generally, this is a good thing, keeps your data safe. However, with Zmodem code writing 1K blocks to disk every .7 seconds or so (at 1650 cps), the disk access began to affect the performance of the download. The solution was easy, I just used a call to setvbuf() to create a memory buffer. The size I chose is 64K. So the system (C runtime) will accumulate 64K bytes from the fwrite()'s before writing to disk. Believe it or not, NT can actually write a 64K block to disk as quickly as a 1K block -- difference is, now Zmodem only hits the disk one a minute or so (at 1650 cps). Works great. Notice I said I made a 64K buffer. You 16 bit guys probably have (as I did) the signed integer maximum value memorized, 32K right? Well, under NT, a signed integer is 2^31, for a maximum [signed] value of 2 GB. So 64K was small compared to what I could make it. The file access areas of the transfer protocols are excellent candidates for overlapped I/O, probably doing away with the need for a write buffer. However, since Williams library is meant to be multi-platform, it is best to keep it as generic as possible. But, it is C++, and with proper inheritance the function responsible for writing data to disk could be overloaded (and William has isolated this operation for easy overloading!). A possible future enhancement to PTERM. One more technical note: PTERM does not change its base priority class, but it does manipulate the priority of several of its threads relative to the priority class it is started at. What? Ok, if you run PTERM from the command line like so: start pterm.exe then PTERM gets a base priority class of NORMAL. Using these commands: start /idle pterm.exe start /normal pterm.exe start /high pterm.exe start /realtime pterm.exe I strongly recommend leaving it at NORMAL priority (by using the /normal switch, or not giving a priority at all). You are, of course, free to experiment. Well, the rest of PTERM is plain and boring (as if the preceding was not!). So not much else to say. Any specific questions? Feel free to contact me!