SEA Technical Memorandum #0304, High Volume GroupMail
Last updated: October 14, 1989
Copyright 1988,89 by System Enhancement Associates, Inc.



                           High Volume GroupMail


The basic GroupMail concept can suffer a performance degradation over a 
high-speed, high-volume link due to the repeated protocol negotiations 
caused by how GroupMail is packaged into multiple relatively small files.  
This can be alleviated by using the "StarLink Supergroup" mechanism.

A supergroup is an archive that contains one or more GroupMail archives.  
The supergroup archive must be unpacked before the individual GroupMail 
archives may be unpacked.  This is handled automatically in GROUP version 
2.16 when a supergroup is defined.

For example, consider a system that is going to receive the PBGROUPS 
supergroup from 520/583, where the PBGROUPS supergroup contains the BLATZ, 
SEATAC, and KITTEN groups.  This system would add the groups as follows:

    group add BLATZ Gzorniblatz ;520/583 /x
    group add SEATAC Technical help ;520/583 /x
    group add KITTEN Kitten sysops ;520/583 /x
    group add PBGROUPS Superstuff ;520/583 $

The three normal groups are added as usual, but the "/X" switch is used to 
tell GROUP that these groups should never be requested.  The PBGROUPS group 
is added as normal, but with the "$" flag to indicate that it is a 
supergroup.

When GROUP generates its update requests, it will generate a request for 
PBGROUPS, but not for the other three.  Then, when GROUP imports new 
GroupMail, it will extract the contents of any PBGROUPS archives it 
received before it looks for the three normal conferences.  



That handles the receiving end, but where do the supergroup archives come 
from in the first place?  They are generated "on the fly" by the STARLINK 
program, based on general delivery notes made by GROUP at the star system.

TM0302, "Undocumented Features of GROUP 2.16", describes the basic delivery 
mechanism using the DELIVER.GRP file.  The DELIVER.GRP file may mark 
selected linkages as being "StarLink delivery".  This is done by appending 
an asterisk to the link address.  For example, the following line in a 
DELIVER.GRP file: 

    BLATZ 520/542* 

would specify that node 520/542 should receive the BLATZ conference using 
StarLink.

This is then implemented by the STARLINK program, which is installed by a 
PROCESS statement in a SEAdog configuration file.  For example, it might be 
installed by inserting the following statement in your CONFIG.DOG file: 

    process mygroups.* starlink *a *n

where "mygroups.*" is the trigger for general delivery.  This must match 
whatever supergroup name people will be requesting from you.  Continuing 
our earlier example, if people will be getting the "PBGROUPS" supergroup 
from 520/583, then he must use a process trigger of "PBGROUPS.*".  In 
general, any given StarLink system should use the same supergroup name for 
all of his StarLink deliveries, and it should be different from that used 
by other StarLink systems.  

The rest of the PROCESS statement must be given exactly as shown, except 
that you can append a "/O" switch if you wish to use Overdrive.  


So where do the supergroup archives come from?  They are custom-built on 
the fly by STARLINK based on the general delivery notes left by GROUP.  The 
supergroup archives do not actually ever exist on the StarLink system, and 
hence they require no disk space.  The STARLINK program itself uses 
restartable SEAlink with overdrive available, and can hence deliver many 
small files in one shot with maximum utilization of the communications 
channel.  If the transfer is interrupted at any point for any reason, it 
may be resumed where it left off on a later call, provided only that the 
StarLink system has not added any GroupMail archives to the delivery list 
in the meanwhile.