BBBBBB EEEEEEE TTTTTTTT AAAA BB BB EE T TT T AA AA BBBBBB EEEEE TT AAAAAA BB BB EE TT AA AA BBBBBB EEEEEEE TT AA AA RRRRRR EEEEEE PPPPP OOOO RRRRRR TTTTTTTT RR RR EE PP PP OO OO RR RR T TT T RRRRRR EEEE PPPPP OO OO RRRRRR TT RR RR EE PP OO OO RR RR TT RR RR EEEEEE PP OOOO RR RR TT PPPPP RRRRR OOOO GGGGG RRRRR AAAA MMM MMM PP PP RR RR OO OO GG RR RR AA AA MM M MM PPPPP RRRRR OO OO GG GGG RRRRR AAAAAA MM M MM PP RR RR OO OO GG G RR RR AA AA MM M MM PP RR RR OOOO GGGGG RR RR AA AA MM M MM COPYRIGHT (c) 1992, MyLife Software - all rights reserved. ________ ____|__ | (R) --| | |------------------- | ----|-- | Association of | | |_| Shareware |__| o | Professionals -----| | |--------------------- |___|___| MEMBER Beta Test Program Table of Contents Introduction.......................................... 1 What's in the package................................. 2 Manual........................................... 3 Ideas............................................ 3 Software......................................... 3 System requirments............................... 3 What's a beta test program............................ 4 Design phase..................................... 5 Alpha testing.................................... 5 Beta testing..................................... 5 What beta testing is not......................... 6 Why is a beta test program important.................. 6 Less bugs........................................ 6 Quicker application turnaround................... 7 How will a beta test program help me.................. 7 Less bugs........................................ 7 More sets of eyes................................ 8 Developing a beta program............................. 8 Items needed..................................... 8 What to give to testers.......................... 9 Customizing...................................... 9 Making a beta program work........................... 10 Number of beta testers.......................... 10 Different pieces................................ 10 Share the effort................................ 11 Methods of implementation............................ 11 Same testers.................................... 12 Different testers............................... 12 Specific testers................................ 12 What's in it for the beta tester..................... 12 What is BETARPT.EXE.................................. 13 System requirements............................. 13 Why use BETARPT.EXE.................................. 15 How will BETARPT.EXE help me......................... 15 Summary.............................................. 15 Trademarks........................................... 15 INTRODUCTION Before we get started let me introduce myself. My name is John Michael Sanders. I write custom business software and shareware. I began my humble little business originally called, Energy Software, in 1989. I later dissolved that business and began MyLife software to better reflect the efforts I was putting forth in the business. I had hopes and dreams of being another Bill Gates. Well I must admit it hasn't turned out that way, but I haven't given up either. I started my business by writing small utility programs. My first major application was a program that computed monetary savings using various energy saving appliances and devices as apposed to conventional products. It was a good program, no sales, the company I wrote it for folded shortly there after. Several major projects later, I was faced with designing a 30,000 line database application that would access 21,000 files spread over 12 drives in 600 subdirectories in a network environment. Some application huh! Well, in 1991 the thought finally occurred to me how do I possible test, to the fullest extent possible the applications I have written. Therefore I wrote BETARPT. The use of BETARPT and an effective beta testing program reduced the time I could deliver an application to a client, or shareware market drastically. The quicker I could turnaround a software application that worked and was nearly bug free the more time I had to write other applications. The idea worked and I'm inching closer and closer to achieving financial independence. Look out, Bill. The following pages are intended to provide you, the software developer, the same flexibility I have gained. I use something the big guys; IBM, MicroSoft, Borland and others have been using all along, a beta test program. I will assume that you are not new to the computer industry and have some experience with computers and a general business sense about you. In addition, I hope you are generally knowledgable about computer bulletin boards, If not you'll have to get smart before proceeding. The big guys have known all along that they cannot detect all the possible bugs hidden within the applications they have written. Even with a good beta test program bugs in 1 software do get released by major companies, some are; MicroSoft-QuickBASIC v4.0, Central Point Software-PC Tools, v7.0, Ashton Tate-dBase v4.0 just to name a few. I won't focus on how to design your applications, market them, or dream up new ideas for you. The sole focus is to establish a method of allowing someone else, beta testers, to detect potential bugs in your software. I have also provided the basic tools for you to do this task. Two very fine Shareware applications that will give you much useful information on how to develop and market your Shareware applications are; a. The $hareware Marketing $ystem Seattle Scientific Photography Attn.: Jim Hood P.O. Box 1506 Mercer Island, WA 98040 (206) 236-0470 b. The Shareware Book Compass/New England Attn.: Bob Schenot P.O.Box 117 Portsmouth, NH 03802-0117 (603) 431-8030 These two applications would sure have made my life easier as a Shareware author if I knew they were available when I got started. A final free plug. I would suggest a considering a membership with the Association of Shareware Professionals. This association is dedicated to the growth of the shareware idea. The membership dues represent an investment, but; I believe the benefits far outweigh the cost. WHAT'S IN THE PACKAGE? The beta test package contains all the essential information needed to develop a successful beta test program within your organization. It is not designed to be totally inclusive. The reason behind this is each shareware developer operates differently. The scope of this application allows you the flexibility of building on a solid platform of proven performance. 2 I really encourage you to read, evaluate, and see if such an in-depth program will work for you. If you feel that your business is destined to grow slightly, or take off like wildfire and you use the ideas of this package then please register this application. Manual When we have received your registration fee we will immediately send you the registered beta test packet. The registered packet includes; a printed manual that is easy to read. The manual has been designed exclusively for the registered version. Ideas Included are ideas for the development of a somewhat aggressive beta test program. This program is the basic platform for you to modify to your special needs. The framework is there, and so is the ability to be successful. Software Enclosed in the Beta Test package are the following files; APPLICAT.TXT - Application for prospective beta testers to complete. (This form provides a notice the individual is interested in beta testing your applications.) BETARPT.EXE - Beta report generator. BETARPT.HLP - Help file for BETARPT.EXE. DESCRIPT.TXT - Description file of what the Beta Test Program is all about. LICENSE.TXT - BETARPT license agreement. README.TXT - README file for beta tester's. REGISTER.TXT - The beta test registration form. NONDIS.TXT - A generic nondisclosure statement that should be signed by prospective beta testers prior to testing software. This form should protect you from having you application stolen. In addition, if you are offering features not available in any other application this document can protect you. ** WHYBETA.TXT - This setup manual. 3 With registration you will be notified of product upgrades. This is important because the ever changing software market demands that you as a developer remain current to stay competitive. ** The nondisclosure statement is generic and should be reviewed by an attorney in your state to compensate for individual state laws. WHAT'S A BETA TEST PROGRAM? A beta test program is a program designed to remove any possible bugs from software prior to release to the public. This also includes finding and correcting discrepancies in the software's user manual or on-line documentation. In short as a developer you go through at least 10 phases in bringing your application from an idea to end product. Those phases, or stages, are; a. Idea b. Flow chart c. Rough working model d. Refinement of rough model e. Draft working model f. Refinement of draft model g. Near final working model h. Refinement of near final model i. Marketing of software application j. Release of software application You may not do all of them or not even in the order listed above. I haven't flow charted since college! The point is a serious application isn't developed over night. The above phases are broken into four distinct groups they are; 4 a. Design phase 1. Idea 2. Flow chart b. Alpha Test 1. Rough working model 2. Refinement of rough model c. Beta Test 1. Draft working model 2. Refinement of draft model 3. Near final working model 4. Refinement of near final model d. Marketing and Release 1. General marketing efforts. 2. Release of software application Design phase The design phase is the period that you use too mentally or on paper develop your application. It includes, but not limited; user interface, file I/O structures, user base of the software, and file design. Alpha testing Alpha testing is designed to make sure that all features of the product work with other hardware and software that you have available. All alpha testing should be done in-house by you or other individuals working directly on the application. Alpha testing is always done before the product goes to outside sources. In addition, all major bugs are resolved. The software documentation and user manual is in a draft stage. At this point the software should not have left the development site. Beta testing Once the in-house team has used the application sufficiently to ensure the application doesn't routinely lockup or blow off the software can be released for beta testing. The beta test version of the software may be mailed to beta testers or it may be made available to beta testers on a local bulletin board. 5 In the beta test phase the application is put through the paces of real world situations. Beta testers help by using the software in their day-to-day environments. The beta testers provide more variety of testing to catch cosmetic flaws, compatibility problems, and other peculiarities about the software than in-house testing could possibly detect. Beta testers use the software and report anything that is not described by the manual, or seems to function improperly. During this phase the beta tester is expected to report problems to you, you attempt to duplicate the problem and promptly correct the fault. Once several bugs have been detected by testers and fixed by the programmer(s) a beta test maintenance release should be issued to beta testers. Once a maintenance release is issued the beta testers should try to duplicate the errors they reported earlier to see if they were corrected in the maintenance release. What beta testing is not Beta testing should not be a phase for redesigning the software. The primary purpose of beta testing is to find problems the way the program currently stands. As with most software, there may be features that the beta tester will dislike. As a developer and marketer it is wise to listen to potential customer comments, however; software redesign should not occur. At this point the your software has gone through some very timely processes and redesign would be to time consuming. Beta tester comments can always be added in future software releases. WHY IS A BETA TEST PROGRAM IMPORTANT A beta testing program is important for two reasons; first it allows the developer to detect bugs early and it provides quicker turnaround of the application. Less bugs The primary reason for a beta test program is that bugs are detected before the software is released to the public. If you have not sufficiently tested your software and released your program potential customers will be finding the bugs and move on to a competitors product without the bugs. 6 In starting my fledgling software business, MyLife Software, I had a problem in effectively beta testing my software. First I tried to beta test my applications. If they were small I could do it just fine. Yet; larger applications cannot be tested correctly by just one person. I could not find all possible bugs that existed. Only through a well managed program can software bugs be removed, identified and fixed. Quicker application turnaround By using the beta tester to detect your bugs this provides a greater opportunity for you to turn your applications around quicker. As an example; consider 12 testers working one hour a day on your application. This is 12 man hours. If you had to test and debug your own software that is a day and a half down the tube. After one week of using a dedicated group of beta testers you will be able to detect more bugs than if you worked on an application yourself for a whole month. If only one individual debugs an application it defeats the purpose for testing. One user only uses the application a certain way. It would be very difficult for a single tester to alter there style of use. HOW WILL A BETA TEST PROGRAM HELP ME? Most shareware developers write software in hopes of making additional income. It is this driving force that should make you want to have a good beta test program. The cost of maintaining a beta test program is minimal in comparison to potential revenue lost. Less bugs By making use of an effective beta test program you can reduce the likelihood your target customer base will detect any bugs. Nothing will cause the loss of sales quicker than a nonfunctional or malfunctioning software application. I speak from experience on this matter. 7 More sets of eyes Prior to release it is best to have as many eyes looking at an application as possible. Those eyes can detect subtle differences if user interfaces, help screens, or other inconsistencies. I can't stress enough the importance of this area. Even a non-computer user looking at the various screens in an application may be able to provide very good advise. You would be surprised that even a child can provide useful input. DEVELOPING A BETA PROGRAM As in anything, developing a beta test program is more difficult than maintaining it. Some thought is required for development of a beta test program. Some large commercial software companies offer 800 numbers that access a dedicated beta test bulletin board and Federal Express all beta releases. Others will use a message base on CompuServe. Unless you are very wealthy I do not recommend this. An effective program can be set up with a minimal investment of money. Items needed Several things are required to make reporting procedures easier for your testers and distribution of beta test releases easier for you. The absolute minimum items are; a. A database program to maintain all valid beta testers. b. A bulletin board, or access to, to receive bug reports and distribute updated beta releases. c. On the bulletin board a message base is needed. In addition, a special user ID for each product tested is required to collect dedicated bug reports for review by you. Ensure that when the bulletin board is set up that you have a weekly bulletin that is available to beta testers that provides them the status of the test. This is important because it helps in maintaining the interest of the testers. 8 The above items are almost essential to get a beta test program off the ground. The database is required to maintain information about a potential tester likes and dislikes about different software applications. If would make no sense to send a beta tester a graphics application you have developed if they do not use graphics applications. You will find it is much more beneficial to select testers that have an interest in the application you are asking them to test. Your own bulletin board is a whole lot easier to maintain bug reports and distribute beta releases than using one belonging to someone else. However; if you cannot afford it here's a recommendation. Select a bulletin board in your area and ask the SYStems OPerator (Sysop) if you may use their board to run your beta test program from. Most will be more than will to help. What to give to testers When you have accepted an individual to become a beta tester the following files should be provided to the beta tester from the Beta Test package; BETARPT.EXE - Main report generator BETARPT.HLP - Beta report help README.TXT - Instructions for using BETARPT These are the minimum files that some be provided. You may decide to include additional files. This is your choice. Customizing The BETARPT.HLP and README.TXT files are in ASCII format. This was done intentionally to allow you, the developer, to customize the format as desired. I encourage you to do so. Altering the files will insert your companies favor and attitude. A note on the help screens. Each help screen is delimited by a ^ sign. If you look at the help file you will notice the carrot(^). Do not remove the ^ or the capitalized letters that follow it. Do not start a line with the ^ sign as the BETARPT program will interpret this as a new help topic. Each help screen is limited to 125 lines. I feel that you should be able to say what is required within this limit. 9 The README.TXT file will have to be minimally edited. If you do not a FAX then you'll need to remove the references to the FAX number. Also, you must insert your company name, address, and phone number. If there are any special procedures that you'll use on a regular basis this is the file to put them in. Think this out and be sure you say it clearly or you will get calls from your beta testers. MAKING A BETA PROGRAM WORK It takes work to maintain and operate an effective beta test program as I addressed earlier. If you are the chief cook and bottlewasher of your operation then set aside time solely to work with the program. I can't tell you what is an appropriate amount of time, but the bigger the application the more time you should spent in the maintenance of bug reports and answering beta tester messages. Number of beta testers This is a difficult topic to cover. The difficulty lies in the selecting of beta testers. For some of you 10 to 15 good beta tester should be sufficient. For others you might have to select 40 or more testers to test effectively your application. An absolute minimum of 10 beta testers should be used to evaluate accurately your application. I firmly believe more is better in the beta test arena. My bias comes because no two users ever use an application the same way. It only then seems logical more potential application flaws can be captured by more testers. I should add that this belief has not left me disappointed. Different pieces When I have specific custom applications tested I tend to break the major application into smaller pieces. An example is if I were working on a large database application I might send the report program to one set of testers and the screen generator to another set and so on. 10 I do this is done for three reasons; a. Many times the application has a very specific business function. The time it would take to get a beta tester fully knowledgable about the application may exceed the test period. b. The other big reason is I write a lot of proprietary software and I never allow never allow a tester to have the whole application. It is not to say that I don't trust the select group of testers that help me, I just never want to test or tempt them. c. Many of my business applications are very large and would exceed the capacity of the average testers harddrive. Spliting an application into pieces requires extra work you. It may include you creating special script files to be used. A script file is a dummy file that contains only the essential information to run the selected application. In the above example of the report generator you may develop a test database to give to the tester. The script files provided to the tester should be large enough to cover every aspect of the portion they are asked to test. If you fail to include an important section in your script file that portion will obviously not be tested. Share the effort Sometimes I like to share the effort. This means I ask a group of testers to focus on a certain aspect of an application. I personally dislike writing mouse interfaces. In knowing my dislikes, I ask a group to check out every screen in which a mouse appears or should appear. I may ask any group to test only user interface and yet another to test the help system. You could break your section into as narrow or wide section as you desire. METHODS OF IMPLEMENTATION Once you have got the BBS set up, a database on-line and are ready to go here's the next step. I would recommend that you ask local SYSOPs and users groups to pass around the files APPLICAT.TXT and NONDIS.TXT. These files contain the application that a prospective beta tester should fill out and the nondisclosure statement. 11 Same testers If a beta tester is helping you then by all means continue to use this tester. After a while these beta testers will become even quicker in submitting bug reports and prove to be a valuable asset. Sometimes, in exploring the flip side, you may have a tester with a very specific interest and can exploit the testers interest on the one application during each upgrade. Different testers You should constantly be seeking new beta testers. It is always better to have to many than not enough as I wrote earlier. Bringing in new testers in the middle of a beta test window should not occur. If you are a one person operation you'll most likely be to busy dealing with your present bunch of testers to bring new ones on board. Only between beta test windows should you review beta test applications and provide them access to the BBS or send them an acceptance letter. Specific testers Using the same testers repeatedly I have found not to be a wise idea. You must constantly bring new blood into your beta test program. The reason is beta testers learn your style and idiosyncracies. I'm not saying flush out all testers after every application test. Continually attempt to acquire new testers all the time. WHAT'S IN IT FOR THE BETA TESTER? Generally a person gets involved in beta testing applications because they have an interest in that particular type of application. They get to use the software before it is available to anyone else and can assist you in making it the best possible. Beta testers don't get paid for there services. They work for the love of using new applications. If fact, some testers even pay for the privilege of testing new software. During the beta test phase of OS/2 v2.0 selected persons paid $60.00 for the beta test disks and manuals. Hard to believe, isn't it! 12 Beta testers play with your application several hours a week and get to deal with the frustration of it not performing as expected, writing you reports, and gaining the satisfaction of their efforts being corrected in the final release. One benefit beta testers generally receive, is a registered copy of the software they have tested so faithfully. This is an important reward for their efforts. Sometimes it is not possible to provide them a registered copy of certain applications. As in the case of custom business software. However; a beta tester may be able to become a client in future software endeavors. Sometimes you as the software developer may offer a commercial software application as a prize to the beta tester reporting the most valid bugs. You could also have a random drawing for free software or hardware for any beta tester who has submitted a valid bug during the beta testing window. If you maintain your own BBS you could set up special message bases available only to valid beta testers. Another idea might be to raise there access time and file download limit. If you run a registered, pay, board you should give them free access. As you can see, the privileges are few and the expectations are high for beta testers. If you treat them well they will work hard for you and be a cornerstone of your business operation. WHAT IS BETARPT.EXE? BETARPT.EXE is a beta report generator used for creating bug reports. The program is designed to be used by your beta testers. The reports are to be used by you the software developer. This file, when used by the beta tester will generate a bug report that will allow you to isolate and fix a bug not detected during the alpha test phase. The BETARPT program is specifically tailored for developing bug reports. It only function is to do that, and it does it well. Yes, I have debugged it! I've used this program in one stage or another for over one year and it provides the optimum information required for removing annoying bugs and application flaws. 13 In order to allow this software to be customized with your business name do the following; a. Immediately create a working copy of the diskette. Use the DOS COPY command for this. b. load BETARPT with the command line \SC. EXAMPLE A:\> BETARPT \SC c. At this point enter your company name and press ENTER. d. Next, place your street address and press ENTER. e. Enter your City, State, and Zip. f. Finally, enter your Technical support phone number and press ENTER. g. If the information is correct press 'Y'. h. If you desire to save the information you have just entered then press 'Y' and the data will be saved to BETARPT.EXE. Your companies information will replace the information initially displayed on the main menu screen. The disk you just created will become your distribution diskette. Remember to distribute this diskette and not the clean diskette or you will have the reminder message displayed at the top of the screen. System Requirements The BETARPT.EXE program requires the minimum; a. Monitor (Color recommended) b. Single disk drive c. 1 floppy disk d. DOS 3.0 or above e. 8086 processor or above That is it! The entire program and the help file will fit on a single 360k floppy disk. 14 WHY USE BETARPT.EXE? The need is clear for the use of this application. By implementing BETARPT.EXE into your software testing and develop process you should be able to achieve new heights in developing applications. This is no high pressure software sales pitch. By using BETARPT.EXE or even you writing your own bug report generator, you could literally save hundreds of hours debugging your own software. HOW WILL BETARPT.EXE HELP ME? The BETARPT.EXE application will directly help you by standardizing all bug reports submitted by beta testers. With this standardization in mind it will make your life of correcting software bugs much, much easier. I can tell you that having a printed report of a bug is a whole lot easier to read and understand than a scribbled note submitted by a friend. Not to mention the reports can be saved to a file for later recall in developing software updates because I have provided you with a product recommendation area. SUMMARY In summary I have attempted to provide the information and software used to create a successful beta test program. Once again, this information is by no means complete. With the thousands of software companies in existence today each will operate differently, and so will you. Use this information as the bedrock for the development of your own program and you will not be disappointed. TRADEMARKS I have used a few product names and software manufacturers and wish to give them credit were credit is due; dBase IV is a trademark of Ashton-Tate, Borland International. PC Tools is a trademark of Central Point Software. QuickBASIC is a trademark of Microsoft Corporation. IBM and OS/2 is a trademark of International Business Machines Corporation. 15