Draft 112492 ÉÍÍÍÍÍÍÍÍÍÍÍÍÍÍ» º ÉÍÍÍÍÍÍÍÍ» º º º º º ÈÍͼ ÈÍͼ A product From International Communication Research ÉÍÍ» ÉÍÍ» º º º º º ÈÍÍÍÍÍÍÍͼ º ÈÍÍÍÍÍÍÍÍÍÍÍÍÍͼ ZyRion General Information Released By DreamScape "Another break in the wall of Communications" Copyright 1990,91,92 By International Communication Research Holland ZyRion Version 2.45.1.0 [ Level-1 ] PREFACE ±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±± ------- In early 1986, Telenet funded a project to develop an improved public domain application to application file transfer protocol. This protocol would alleviate the throughput problems network customers were experiencing with XMODEM and Kermit file transfers. In the beginning, we thought a few modifications to XMODEM would allow high performance over packet switched networks while preserving XMODEM's simplicity. The initial concept would add a block number to the ACK and NAK characters used by XMODEM. The resultant protocol would allow the sender to send more than one block before waiting for a response. But how to add the block number to XMODEM's ACK and NAK? WXMODEM, SEAlink, MEGAlink and some other protocols add binary byte(s) to indicate the block number. Pure binary was unsuitable for ZYRION because binary code combinations won't pass bidirectionally through some modems, networks and operating systems. Other operating systems may not be able to recognize something coming back unless a break signal or a system dependent code or sequence is present. By the time all this and other problems with the simple ACK/NAK sequences mentioned above were corrected, XMODEM's simple ACK and NACK characters had evolved into a real packet. The Frog was riveting. debugging The Source's SuperKermit protocol indicated a window size of about 1000 characters was needed at 1200 bps. High speed modems require a window of 20000 or more characters for full throughput. Much of the SuperKermit's inefficiency, complexity and debugging time centered around its ring buffering and window management. There had to be a better way to get the job done. A sore point with XMODEM and its progeny is error recovery. More to the point, how can the receiver determine whether the sender has responded, or is ready to respond, to a retransmission request? XMODEM attacks the problem by throwing away characters until a certain period of silence. Too short a time allows a spurious pause in output (network or timesharing congestion) to masquerade as error recovery. Too long a timeout devastates throughput, and allows a noisy line to lock up the protocol. SuperKermit solves the problem with a distinct start of packet character (SOH). ZMODEM and ZYRION use unique character sequences to delineate the start of frames. SEAlink and MEGAlink do not address this problem. A further error recovery problem arises in streaming protocols. How does the receiver know when (or if) the sender has recognized its error signal? Is the next packet the correct response to the error signal? Is it something left over "in the queue"? Or is this new subpacket one of many that will have to be discarded because the sender did not receive the error signal? How long should this continue before sending another error signal? How can the protocol prevent this from degenerating into an argument about mixed signals? SuperKermit uses selective retransmission, so it can accept any good packet it receives. Each time the SuperKermit receiver gets a data packet, it must decide which outstanding packet (if any) it "wants most" to receive, and asks for that one. In practice, complex software "hacks" are needed to attain acceptable robustness. For ZYRION, we decided to forgo the complexity of SuperKermit's packet assembly scheme and its associated buffer management logic and memory requirements. Another sore point with XMODEM and WXMODEM is the garbage added to files. This was acceptable with the old CP/M files which had no exact length, but not with newer systems such as PC-DOS and Unix. YMODEM uses file length information transmitted in the header block to trim the output file, but this causes data loss when transferring files that grow during a transfer. GENERAL ±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±± ------- All file transfer protocols make tradeoffs between throughput, reliability, universality, and complexity according to the technology and knowledge base available to their designers. In the design of ZYRION, three applications deserve special attention, these are the same as ZMODEM does. + Network applications with significant delays (relative to character transmission time) and low error rate + Timesharing and buffered modem applications with significant delays and throughput that is quickly degraded by reverse channel traffic. ZYRION's economy of reverse channel bandwidth allows modems that dynamically partition bandwidth between the two directions to operate at optimal speeds. Special ZYRION features allow simple, efficient implementation on a wide variety of timesharing hosts. + Traditional direct modem to modem communications with high error rate. ZYRION is not optimized for optimum throughput when error rate and delays are both high. This tradeoff markedly reduces code complexity and memory requirements. ZYRION generally provides faster error recovery than network compatible ZMODEM implementations. In the absence of network delays, rapid error recovery is possible, much faster than TMODEM, ZMODEM, MPT or YMODEM and XMODEM. File transfers begin immediately regardless of which program is started first. ZYRION ±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±± ------ In some cases, the file length may be unknown, as when data is obtained from a process. Variable length data subpackets solve both of these problems. Since some characters had to be escaped anyway, there wasn't any point wasting bytes to fill out a fixed packet length or to specify a variable packet length. In ZYRION, the length of data subpackets is denoted by ending each subpacket with an escape sequence similar to BISYNC and HDLC. The end result is a ZDFCX header containing a "frame type", four bytes of supervisory information, and its own DFC. Data frames consist of a header followed by 1 or more data subpackets. In the absence of transmission errors, an entire file can be sent in one data frame. Since the sending system may be sensitive to numerous control characters or strip parity in the reverse data path, all of the headers sent by the receiver are sent in hex. A common lower level routine receives all headers, allowing the main program logic to deal with headers and data subpackets as objects. With equivalent binary (efficient) and hex (application friendly) frames, the sending program can send an "invitation to receive" sequence to activate the receiver without crashing the remote application with unexpected control characters. The objects as above are fixed for 8 data bits, and NOT to its real variable bit coded matrix, each byte can overlap a characte. An Byte in normal transfer is one character [ all protocols ] handle it this way, ZYRION don't. ZyRion uses a fixed Matrix code table that reduce the overhead of extra bits Not compression or RLE, NO it use his own table for binary transfers. The bits are only shifted into the ringbuffer to get 8 bits as a simulated byte, REAL one byte can be 5 bits or 8 bits depended on the high or low DFC order control frame for the fixed matrix code table. Going "back to scratch" in the protocol design presents an opportunity to steal good ideas from many sources and to add a few new ones. From Kermit and UUCP comes the concept of an initial dialog to exchange system parameters. ZMODEM generalizes Compuserve B Protocol's host controlled transfers to single command AutoDownload and command downloading. A Security Challenge discourages password hackers and Trojan Horse authors from abusing ZMODEM's power. ZYRION has the advantage of this, we know the technology we need only to make it better, The security challenge of ZMODEM is used in ZYRION aswell but in another way, it will send a Signature that prevent MTE or other related infected remote or local ZYRION protocols. People try to design Virus scan technics to prevent infections, ZYRION will prevent the file to be saved to disk IF the Signature fails. Do NOT confuse this with checking a file while it will be transfered. Extensive testing has demonstrated ZYRION to be compatible with satellite links. THROUGHPUT ±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±± ---------- Between two single task PC-XT computers sending a program image on an in house Telenet link, SuperKermit provided 72 ch/sec throughput at 1200 baud. YMODEM-k yielded 85 chars/sec, and ZMODEM provided 113 chars/sec. XMODEM was not measured, but would have been much slower based on observed network propagation delays. Recent tests downloading large binary files to an IBM PC (4.7 mHz V20) running ZYRION 2.46.2.8 with table driven 64 bit DFC calculation yielded a throughput of 1936 cps on a 19200 bps direct connection. Tests with TELEBIT TrailBlazer modems have shown transfer rates approaching 1600 characters per second for long files. When files are compressed, effective transfer rates of 2200 characters per second are possible. ERROR RECOVERY IN TIMESYNC DFC BASED MODE ±±±±±±±±±±±±±±±±±±±±±±±±±± ----------------------------------------- Some tests of ZYRION protocol error recovery performance have been made. A PC-AT with SCO SYS V Xenix or DOS 5.0 was connected to a PC with DOS 3.1 either directly at 9600 bps or with unbuffered dial-up 1200 bps modems. The ZYRION software was configured to use 2048 byte data subpacket lengths above 2400 bps, 1024 otherwise. Because no time delays are necessary in normal file transfers, per file negotiations are much faster than with ZMODEM, the only observed delay being the time required by the program(s) to update matrix coded buffers. During a file transfer, a short line hit seen by the receiver usually induces a DFC error. The interrupt sequence is usually seen by the sender before the next data subpacket is completely sent, and the resultant loss of data throughput averages about half a data subpacket per line hit. At 1200 bps this is would be about .75 second lost per hit. At 10-5 error rate, this would degrade throughput by about 9 per cent. The throughput degradation increases with increasing channel delay, as more data subpackets in transit through the channel are discarded when an error is detected. A longer noise burst that affects both the receiver and the sender's reception of the interrupt sequence usually causes the sender to remain silent until the receiver times out in 6 seconds. If the round trip channel delay exceeds the receiver's 6 second timeout, recovery from this type of error may become difficult. Noise affecting only the sender is usually ignored, with one common exception. In summation, ZYRION performance in the presence of errors resembles that of ZMODEM or TMODEM, ZMAX e.c Short bursts cause minimal data retransmission. Long bursts (such as pulse dialing noises) often require a timeout error to restore the flow of data. Unlike traditional and many recently introduced protocols, ZYRION Signature Pulse safeguards all data and supervisory information with effective error detection. ZYRION rapidly transfers files, particularly with buffered (error correcting) modems, timesharing systems, satellite relays, and wide area packet switched networks. User Friendliness is an important ZYRION feature. ZYRION AutoDownload (Automatic file Download initiated without user intervention) greatly simplifies file transfers compared to the traditional protocols. ZMODEM also has this feature (it was the first one), and ZYRION folows this step. ZYRION provides advanced file management features including Crash Recovery, flexible control of selective file transfers, and security USER "coded in the key" verified sequence. ZYRION protocol features allow implementation on a wide variety of systems operating in a wide variety of environments. A choice of buffering and windowing modes allows ZYRION to operate on systems that cannot support other streaming protocols. Finely tuned control character escaping allows operation with real world networks without Kermit's high overhead. ZYRION and ZMODEM are the only high performance high reliability public protocols that does not require large buffer allocations for normal file transfers. In compare with ZMODEM, ZYRION uses another way to send his packets also the CRC method has been taken out of the loop, since DFC was a better - method to support the Bit Matrix Shift BMS[HI] and BMS[LO] sequences. This feature makes ZYRION one of the best protocols on the marked today. BETA TESTER! ±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±± ------------ IF you want to be a beta, please send your personal information to one of the ICR/Sites listed in "SITES.TXT"! FREE KEY FOR! ±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±±± REAL System Operators that can be found in the NODELIST! If you are the one, call also one of the systems listed in "SITES.TXT"! My Thanks to : ALL Systems listed in the file "SITES.TXT", Included : Jan Janmaat Dennis Hammerstein John Lots Herman ter Braake Reinier De Groot Rob V.D Hoeven Wim Meesters Truus Pisano Please IF you are not listed, excuse me but I have a busy live.