Hamilton C shell(tm) User Guide and Reference Manual Release 2.2 October, 1993 Hamilton Laboratories, 13 Old Farm Road, Wayland, MA 01778-3117 Phone 508-358-5715 + FAX 508-358-1113 MCI Mail 389-0321 + Internet 3890321@mcimail.com BIX hamilton + CompuServe 70034,2025 + Telex 6503890321 Copyright (c) 1988 - 1993 by Hamilton Laboratories. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise without the prior written permission from Hamilton Laboratories. Printed in the United States of America. AT, PS/2 and OS/2 are registered trademarks of International Business Machines Corporation. Windows NT is a trademark of Microsoft Corporation. UNIX is a registered trademark of UNIX System Laboratories. Hamilton C shell is a trademark of Hamilton Laboratories. Table of Contents Preface .................................. v License Agreement....................... vii Introduction ............................. 1 Installation Guide ....................... 3 Installation on OS/2 3 Installation on Windows NT 15 Common Problems ......................... 18 Product Support ......................... 29 User Guide .............................. 32 The Utilities 32 I/O Redirection and Piping 55 The History Mechanism 66 Variables 70 Wildcarding 79 Editing 87 Quoting 92 Expressions 96 Aliases 105 Programming Constructs 108 Scheduling 130 Order of Evaluation 137 Customizing the Shell 140 Summary 153 Examples .............................. 155 Factor.csh 155 Whereis.csh 156 Samples Directory 157 Compatibility Guide ................... 162 Language Reference .................... 172 Basic Statements 172 Condition Testing 172 Iteration 175 Procedures 175 Aliases 176 Variable and Expression Manipulation 176 Local Variables 178 Function Keys 179 iii Miscellaneous Statements 181 Statement Relationships 183 I/O Redirection 183 Expression Operators 185 File System Tests 187 Special Devices 188 Wildcarding and Pattern Matching 190 Filename Completion 192 Command Line Editing 192 History Recall 196 Command Completion 196 Quoting 198 Escape Sequences 198 Variable Substitution 200 Substitution Modifiers 201 Pathname Editing 203 Predefined Variables .................. 205 Environmental Variables 205 Process-Wide Variables 210 Per-Thread Variables 212 Variables, Sorted by Name 218 Built-in Procedures ................... 230 Utilities ............................. 235 Popular Aliases ....................... 242 Help Information ...................... 248 Help for the shell 248 Help for the utilities 250 iv Preface Thank you for purchasing and using Hamilton C shell. Our goal and guarantee is your satisfaction. Hamilton C shell is an advanced command processing language for OS/2 and Windows NT. It's a professionally- oriented language for manipulating files, processes and threads and connections between these objects. As a language, it offers what we think of as the human characteristics of language: a useful vocabulary and grammar, a limitless freedom of expression and the ability to describe and relate events in time. Most important, it projects your influence into the future by allowing you to easily describe you want done even if what you want is quite complex and dependent on future events. Hamilton C shell is a full implementation of the C shell language popular on engineering workstations. It was created specifically for OS/2 protected mode and meticulously ported to Windows NT. Not one of the more than 105,000 lines of code in the current release was created on or ported from anything but OS/2 or NT. This product complies with accepted standards for the language and with the conventions of OS/2 and NT. Users with previous experience with the standard OS/2, NT or DOS command processors or the original Unix C shell should find enough reasonably familiar language constructs and features to make the product immediately productive. Douglas A. Hamilton Wayland, Massachusetts December 9, 1988 (Last revised October 4, 1993) v vi IMPORTANT -- READ CAREFULLY BEFORE OPENING. By opening this sealed package, you indicate your acceptance of the following Hamilton Laboratories License Agreement. Hamilton Laboratories License Agreement This is a legal agreement between you, the end user, and Hamilton Laboratories. By opening this sealed package, you are agreeing to be bound by the terms of this agreement. If you do not agree to the terms of this agreement, promptly return the unopened package and any accompanying items for a full refund. HAMILTON LABORATORIES SOFTWARE LICENSE 1. GRANT OF LICENSE. Hamilton Laboratories grants to you the right to use one copy of the enclosed Hamilton Laboratories software program (the ``SOFTWARE'') on a single terminal connected to a single computer (i.e., with a single CPU). You may not network the SOFTWARE or otherwise use it on more than one computer or computer terminal at the same time. 2. COPYRIGHT. The SOFTWARE is owned by Hamilton Laboratories or its suppliers and is protected by United States copyright laws and international treaty provisions. Therefore, you must treat the SOFTWARE like any other copyrighted material (e.g., a book or musical recording) except that you may either (a) make a reasonable number of copies of the SOFTWARE solely for backup purposes or (b) transfer the SOFTWARE to a single hard disk provided the original and any other copies are kept solely for backup or archival purposes. You may not copy the written materials accompanying the software. 3. OTHER RESTRICTIONS. You may not rent or lease the SOFTWARE, but you may transfer the SOFTWARE and accompanying written materials on a permanent basis provided you retain no copies and the recipient agrees to the terms of this Agreement. You may not reverse engineer, decompile or disassemble the SOFTWARE. If SOFTWARE is an update, any transfer must include the update and all prior versions. 4. DUAL MEDIA SOFTWARE. If this SOFTWARE package contains both 3 1/2'' and 5 1/4'' disks, you may use only the disks appropriate for your single-user computer. You may not use the other disks on another computer or loan, rent, lease, or transfer them to another user except as part of the permanent transfer (as provided above) of all SOFTWARE and written materials. LIMITED WARRANTY LIMITED WARRANTY. Hamilton Laboratories warrants that the SOFTWARE will perform substantially in accordance with the accompanying written materials for a period of 90 days from the date of purchase. Some states do not allow limitations on the duration of an implied warranty, so the above may not apply to you. CUSTOMER REMEDIES. Hamilton Laboratories' entire liability and your exclusive remedy shall be, at Hamilton Laboratories' option, either (a) return of the price paid or (b) repair or replacement of the SOFTWARE that does not meet this Limited Warranty and which is returned to Hamilton Laboratories with a copy of your receipt. During the first 90 days from the date of purchase, if you determine that the SOFTWARE is unsatisfactory in any way, you may return it with proof of purchase and a written description of why the SOFTWARE was unsatisfactory for a full refund. NO OTHER WARRANTIES. Hamilton Laboratories disclaims all other warranties, either express or implied, including, but not limited to implied warranties of merchantability and fitness for a particular purpose, with respect to the SOFTWARE and accompanying written materials. This limited warranty gives you specific legal rights. You may have others, which vary from state to state. NO LIABILITY FOR CONSEQUENTIAL DAMAGES. In no event shall Hamilton Laboratories or its suppliers be liable for any damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, loss of business information, or other pecuniary loss) arising out of the use of or inability to use this Hamilton Laboratories product, even if Hamilton Laboratories has been advised of the possibility of such damages. Because some states do not allow the exclusion or limitation of liability for consequential or incidental damages, the above limitation may not apply to you. This Agreement is governed by the laws of the State of Massachusetts. Should you have any questions concerning this Agreement, or if you wish to contact Hamilton Laboratories for any reason, please write: Hamilton Laboratories Customer Service, 13 Old Farm Road, Wayland, MA 01778-3117. Introduction Hamilton C shell(tm) Introduction Hamilton C shell is a language for interactively using OS/2 and Windows NT. Compared to the standard OS/2 and NT command processors, it provides a vocabulary and grammar that allows much more complex activities to be described. Some of its major innovations include + Command line editing of enormous statements with arrow keys and filename and command completion. + User-definable function keys. + Fully recursive grammar. Statements can be arbitrarily nested or piped without concern for statement length or other arbitrary restrictions. + Procedures and aliases. The vocabulary of the language is meant to be extensible by the user. + Variables, arrays and expressions. Integer and floating point arithmetic, pattern matching facilities and various file system tests and editing operators provide an expressive grammar. + Threads and processes. Child threads and processes can be spawned to run commands asynchronously or in the background. + Command substitution. The output of one command can be stuffed back on the command line as arguments to another. + History. Past commands can be recalled and edited. + Advanced filename wildcarding. This product complies fully with industry-accepted definitions for the C shell language. The user is not asked to learn yet another new proprietary language not available anywhere else. Instead, a tested, proven framework has been adapted with modern compiler technology for OS/2 and NT: Page 1 Introduction 1. A modern top-down parser is used for better language recognition and performance. 2. It's easier to use. The syntax and grammar has been made flexible and more consistent with other modern high level language conventions. 3. It knows about OS/2 AND NT: HPFS, long filenames, environmental variables, networks, international character sets, how to start PM applications and (under OS/2 2.x) about 32-bit and Virtual DOS machine (VDM) applications. 4. Threads are used extensively to achieve performance and functionality not possible in UNIX. 5. Feedback to the user, especially when reporting errors has been improved. Who is it Designed For? Most users of Hamilton C shell are relatively technically oriented computer users. Often, they're software developers. They have a business need for an OS/2 or an NT system. Peering over their shoulders, they typically have lots of windows open on the screen. Many of the windows are running copies of this shell. Some copies are transient, created to display with little snippets of information needed on the spur of the moment. Other copies of the shell would be used for more long-running projects: for example, getting a make working for a major application. A shell window is like any other application window but with a different paradigm. Instead of data, rows and columns of numbers or lines of text, the object being manipulated is the machine itself. A good shell tackles a different problem than icons and windows. Instead of the point-and-shoot immediacy of ``do this single thing now,'' a shell offers language and the ability to describe more customized or repetitive actions, e.g., identify a suitable set of files, perform some action against them and filter the results in some interesting way. Page 2 Installation Installation Guide This section outlines how to install the Hamilton C shell on your system. If you are installing the OS/2 version, follow the instructions beginning on this page. To install the Windows NT version of Hamilton C shell, please turn to page 15. If you encounter problems, consult the ``Common Problems'' section on page 18 or call us for technical support as described on page 29. Installation on OS/2 The first few steps, copying files from diskette to your hard disk and modifying your config.sys, are the same on all releases of OS/2. The remaining steps -- those associated with actually installing Hamilton C shell on your OS/2 desktop -- depend on which release of OS/2 you're running. We suggest a ``vanilla'' installation initially, but later you may want to customize it to your own tastes. For help with that, read the chapter on ``Customizing the Shell,'' beginnning on page 140. Once you've gained familiarity with both OS/2 and with the C shell, you may want to set up the C shell as the default command processor for OS/2, completely replacing cmd.exe as described on page 10. The advantage to be gained (except under the 6.167 Beta and LA builds of 2.0) is that the C shell will then be able to change its own title bar and icon when you run an external command. System Requirements Installation requires a 286-, 386- or 486-based AT+ or PS/2+ or compatible, running OS/2+ 1.1 (Presentation Manager) or Microsoft SDK 1.06 or later. Roughly 1.2 MB of disk space is used. Hamilton C shell and the utilities supplied with it fully support HPFS and long filenames when running under OS/2 1.2 or later. They will work properly in a Presentation Manager text window or full-screen and with networks such as LAN Manager or IBM LAN Server. If you're using OS/2 2.x, it knows how to run 32-bit applications and start up Page 3 Installation Multiple Virtual DOS machines and run Win3.x applications seamlessly on the desktop. The product is not copy protected. Basic Installation, Part I (All releases of OS/2) 1. Copy the executables in the bin directory into any desired directory on your search PATH, so long as it appears ahead of the directory containing the standard IBM/Microsoft more.com. (We supply a dramatically improved more.exe, which should take precedence.) If you're creating a new directory, remember to add it to your search PATH in config.sys and in the login.csh file you create next. 2. Edit config.sys, adding statements to define whatever directory you choose to designate as your HOME directory and to ensure you're configured for a sufficient number of threads. The significance of a home directory is principally that it will be convenient to specify pathnames relative to that directory. The default number of threads is too small if you expect to have lots of windows open. Also, be sure your PATH explicitly lists ``.'', the current directory. You may also want to include definitions for TABS and COLORS. more.exe and some of the other utilities look for TABS to see if you want them to display text with tabs expanded out to something other than the default of every 8 characters. By default, the C shell displays white characters on a black background. The COLORS variable lets you choose something different from this set: black, red, green, yellow, blue, magenta (or blue red), cyan (or blue green) and white. Foreground colors may also be bright, dim, blink or reverse. The keyword ``on'' introduces background colors. (Blink only causes true blinking full-screen; in a text window, it just makes the background brighter. Also, yellow is a true yellow only if it's bright. These are OS/2 limitations not related to the C shell.) For more information on setting screen colors, please refer to the customization chapter or to the colors.csh file in the samples directory. Here's an example of what you might add to config.sys: THREADS=255 SET HOME=D:\DOUG Page 4 Installation SET TABS=3 SET COLORS=WHITE ON BLUE (Please be sure your config.sys file contains only upper-case alphabetics, no lower-case, if you're using OS/2 1.1. Lower-case alphabetics were known to cause random OS/2 system failures in that release of OS/2. This was a known bug in the OS/2 kernel and was not application dependent.) 3. Copy the login.csh and startup.csh files into ``home'' directory. Unless you're convinced that you've set all your environmental variables in your config.sys (and that your PATH explicitly lists ``.'', the current directory), use the dumpenv utility to paste a series of setenv statements into the login.csh file to recreate the environment you've been using with cmd.exe: dumpenv >>login.csh (To see what dumpenv does, look at the source code in the samples directory or simply run dumpenv without redirecting the output.) The login.csh and startup.csh files can be edited with any ascii editor to customize the shell to your needs. The login.csh file has a lot of comments in it which can take the shell a second or more to read; you'll almost certainly want to delete some of them once you've read them so the shell will start up faster. Also, any setenv statements that just duplicate what's in your config.sys can be discarded. The remaining steps depend on which release of OS/2 you're running. Basic Installation, Part II (OS/2 1.1) 4. Add csh.exe with the title ``Hamilton C shell'' to the ``Start Programs'' menu. To do this, pull-down ``Program'' and select ``Add...'' from the menu bar. Fill in: Program title.... Hamilton C shell Path and file name ....as appropriate....\csh.exe Parameters.... -L Page 5 Installation The ``-L'' part tells csh.exe when it starts up that it's a ``login'' shell, which means it should look for a login.csh file. (Refer to page 248 for additional information on other options.) 5. You will likely want to create a second entry for running full-screen. It's more convenient if you're mostly working with applications that only run full- screen or if you want faster text display, especially scrolling. To do that, from the ``Start Programs'' menu, pull-down ``Program'' and select ``Copy...'' from the menu bar. In the Copy Programs popup, fill in the following and push the ``Copy'' button: Change Title to: Hamilton C shell -- Full Screen Back in the ``Start Programs'' window, select the new full screen title, pull-down ``Program'' and select ``Change...''. In the Change Program Information popup, push the ``Change'' button. This brings up the How to Run the Program popup; select ``Run the program full-screen'' and ``Enter''. 6. All the material in the samples directory is provided for its tutorial value; you may or may not wish to copy it onto your hard disk. 7. Reboot your system before starting Hamilton C shell for the first time. This causes the new statements in config.sys to take effect. Page 6 Installation Basic Installation, Part II (OS/2 1.2 or 1.3) 4. Add csh.exe with the title ``Hamilton C shell'' to the ``Group - Main'' menu. To do this, pull-down ``Program'' and select ``New...'' from the menu bar. Fill in: Program title: Hamilton C shell Path and file name: ....as appropriate....\csh.exe Parameters: -L The ``-L'' part tells csh.exe when it starts up that it's a ``login'' shell, which means it should look for a login.csh file. (Refer to page 248 for additional information on other options.) 5. You will likely want to create a second entry for running full-screen. It's more convenient if you're mostly working with applications that only run full- screen or if you want faster text display, especially scrolling. To do that, from the ``Group - Main'' menu, pull-down ``Program'' and select ``Copy...'' from the menu bar. In the Copy Programs popup, fill in the following and push the ``Copy'' button: Change Title to: Hamilton C shell -- Full Screen Back in the ``Group - Main'' window, select the new full screen title, pull-down ``Program'' and select ``Properties...'' . In the Properties popup, push the down arrow next to the ``Program Type:'' box and select ``OS/2 Full Screen'' on the list that will appear and then push the ``Change'' button. 6. All the material in the samples directory is provided for its tutorial value; you may or may not wish to copy it onto your hard disk. 7. Reboot your system before starting Hamilton C shell for the first time. This causes the new statements in config.sys to take effect. Page 7 Installation Basic Installation, Part II (OS/2 2.x) 4. Open the Templates folder and drag a program object to the desktop (or another folder) by pressing and holding the right mouse button as you drag. On the Program page of the ``Program - Settings'' window that will appear, fill in: Path and file name: ....as appropriate....\csh.exe Parameters: -L The ``-L'' part tells csh.exe when it starts up that it's a ``login'' shell, which means it should look for a login.csh file. (Refer to page 248 for additional information on other options.) 5. On the Window page of the Settings, you will probably want to set Minimized button behavior: Minimize window to desktop Object open behavior: Create new window Doing this will let you conveniently open up lots of copies of the C shell as needed. 6. On the General page of the Settings, set Title: Hamilton C shell Close the Settings by pressing Alt-F4. 7. You will likely want to create a second entry for running full-screen. It's more convenient if you're mostly working with applications that only run full- screen or if you want faster text display, especially scrolling. To do that, copy the C shell icon you just created by right-clicking on it and selecting ``Copy...'' and then choosing an appropriate destination folder (probably the desktop) for the copy. You can also copy the icon by pressing and holding the Ctrl key while dragging with the right mouse button. 8. Once you've made the copy, right-click on it and select ``Open'' and then ``Settings''. On the ``Session'' page, select ``OS/2 full screen''. Then go to the ``General'' page and type a new title: Title: Hamilton C shell Full Screen Page 8 Installation Close the Settings window for the copy by pressing Alt-F4. 9. All the material in the samples directory is provided for its tutorial value; you may or may not wish to copy it onto your hard disk. 10. Reboot your system before starting Hamilton C shell for the first time. This causes the new statements in config.sys to take effect. Page 9 Installation Installation as the Default Command Processor The C shell can also be installed as the default command processor OS/2 protected mode, meaning you specify it, not cmd.exe in your config.sys. The principal advantage is that when the when the C shell is run as the default command processor, PM allows the C shell to change its own title bar and, under OS/2 1.3 or 2.x (but not the 6.167 Beta or LA builds), its own icon to show what it's running. This can be quite helpful if you have lots of copies of the shell running minimized and would like to know what each one is doing. The disadvantage is that the installation is slightly messy and it does disable cmd.exe's ability to change its title bar and icon. For these reasons, most users will want to wait until they've gained some familiarity with the C shell and with OS/2 before installing it this way. To install the C shell as the default command processor, follow the instructions for the basic installation but then make these changes, as appropriate for your system: Default Command Processor Installation Procedure (OS/2 1.2 or 1.3) 1. Edit the PROTSHELL line in your config.sys, replacing the pathname and any parameters for cmd.exe (remembering what they were) with the pathname for the C shell and a -L (login) parameter. The resulting line should look something like this: PROTSHELL=C:\OS2\PMSHELL.EXE C:\OS2\OS2.INI C:\OS2\OS2SYS.INI C:\OS2\BIN\CSH.EXE -L 2. Change the pathname you specify for the C shell in Start Programs or Group-Main to * (an asterisk). Also, change the parameters line to be either blank (1.1 or 1.2) or (1.3): /K "%*" 3. Change the entries (probably named ``OS/2 Window'' or ``OS/2 Full Screen'') in Group-Main or Start Programs for cmd.exe to fill in the complete pathname for cmd.exe instead of an asterisk. Set the parameters to whatever you had specified following the pathname for cmd.exe (if anything) in your config.sys before changing it in step 1. 4. Change any entries in any of your program groups which invoke .cmd scripts to run them via cmd.exe Page 10 Installation explicitly. For example, if you had an entry that specified the program ``c:\myapp\foo.cmd'', change that to: Path and file name: c:\os2\cmd.exe Parameters: /C c:\myapp\foo.cmd ...any additional parameters... 5. Reboot. Page 11 Installation Default Command Processor Installation Procedure (OS/2 2.x) 1. Edit your config.sys to set OS2_SHELL to point to the C shell, specifying the -L (login) option, e.g., set OS2_SHELL=c:\hamilton\bin\csh.exe -L 2. Modify the Settings for the OS/2 Window and OS/2 Full Screen icons to show the full path for cmd.exe (e.g., ``c:\os2\cmd.exe'') rather than an asterisk on the Program page. 3. Modify the Settings for the Hamilton C shell icons to specify an asterisk pathname (meaning the default shell), deleting any mention of any startup parameters and explicitly specifying the C shell icon rather than the default icon: a. Right-click on the icon and open the Settings. b. On the Program page, set Path and file name: * Parameters: c. Select ``Find...'' next to the icon display. d. Select ``Locate'' on the Find screen. e. Select the ``Path'' page on the Locate Folder screen. f. Type the pathname of the directory containing the C shell's csh.ico icon file. (E.g., ``c:\hamilton\bin''.) g. Press the ``OK'' button on the Locate Folder screen. h. Type ``csh.ico'' in the Name field on the Find screen. i. Press the ``Find'' button. j. The Find Results screen should appear with the C shell icon highlighted. Press the ``OK'' button. k. Back in the General Settings screen, you should now see the C shell's icon. Press Alt-F4 to close the screen. Page 12 Installation 4. When you reboot, the C shell will be the default shell and it will appear with its correct icon both for starting and when you minimize it. Page 13 Installation Page 14 Installation Installation on Windows NT This section describes how to install the Windows NT version of Hamilton C shell. System Requirements: Installation requires a 386-, 486- or Pentium-based machine for the Intel x86 version, a MIPS R4000- or R4400-based machine for the MIPS version or a DEC Alpha AXP-based machine for the Alpha version of Hamilton C shell. The machine must be running the final release of Windows NT, build 511 (on Intel or MIPS) or 528 (Alpha) or later. Roughly 2.1 MB of disk space is used on an Intel machine, 3.5MB on a MIPS or 4.3MB on an Alpha. Basic Installation: 1. Copy the contents of the bin and samples directories onto your hard disk, putting them anywhere you like. (Notice that the bin directory is too big to fit on one diskette; you'll have to merge the two or more diskettes, depending on which system you have.) 2. Copy the login.csh and startup.csh files into any directory you care to designate as your ``home'' directory. The significance of a home directory is principally that it will be convenient to specify pathnames relative to this directory. 3. Edit the login.csh and startup.csh files, customizing them to meet your needs. The login.csh file has a lot of comments in it which can take the shell a second or more to read each time it starts up; you'll almost certainly want to delete some of these comments once you've read them so the shell will start up faster. 4. Edit the environment variables by opening the Control Panel and then, within that, opening the system icon. To define a variable through the Control Panel, type the variable name in the ``Variable:'' fill-in box, the value in the ``Value:'' box and click on the ``Set'' button. Page 15 Installation a. Create or edit your entry for the PATH variable, adding the full pathnames for the C shell's bin and samples directories to the list. b. Create an entry for the HOME environment variable, setting its value as the full pathname of the directory where you placed login.csh and startup.csh. c. You may also want to include definitions for TABS and COLORS. The shell and all the utilities look for TABS to see if you want them to display text with tabs expanded out to something other than the default of every 8 characters. By default, the C shell displays white characters on a black background. The COLORS variable lets you choose a combination from this set: black, red, green, yellow, blue, magenta (or blue red), cyan (or blue green) and white. Foreground collows may also be bright, dim, blink or reverse. The keyword ``on'' introduces background colors. (Blink only causes true blinking full-screen; in a text window, it just makes the background brighter. Also, yellow is a true yellow only if it's bright. These are system limitations not related to the C shell.) Other color settings you might want to specify now or at some later time through the Control Panel are MOREPROMPT, MOREFILLIN and MOREERROR (for customizing the more utility's command line) and DELETIONS and ADDITIONS (for customizing the diff utility). For more information on setting screen colors, please refer to the the colors.csh file in the samples directory or to the Customization chapter. Here's an example of the settings you might specify: HOME=d:\doug PATH=d:\hamilton\bin;d:\hamilton;samples COLORS=white on blue TABS=3 ADDITIONS=bright white on green DELETIONS=bright white on red MOREPROMPT=red on white MOREFILLIN=black MOREERROR=bright white on red Page 16 Installation 5. Add csh.exe with the title ``Hamilton C shell'' to the Program Manager. To do this, pull-down ``File'' and select ``New''. A pop-up will appear asking that you confirm this will be a new Program Item. On the next pop-up, fill in: Description: Hamilton C shell Command Line: ....as appropriate....\csh.exe -L The ``-L'' part tells csh.exe when it starts up that it's a ``login'' shell, which means it should look for a login.csh file. Page 17 Common Problems Common Problems When I try to start the C shell in a new window, it dies and goes away before I can read its messages. You've probably made an error on the ``Parameters'' line under OS/2 or in the ``Command Line'' under NT. Under NT, select the icon for Hamilton C shell and press Alt- Enter to examine the properties. Under OS/2, you can force the window will to stay up after the shell exits so you can read the message by following the instructions appropriate for your system: OS/2 1.1: Go to the ``How to Run the Program'' screen by clicking on the C shell entry in ``Start Programs'' and pulling down ``Program'' then selecting ``Change...''. Click on the check box beside ``Close the window...'' and press Enter. OS/2 1.2 or 1.3: Click on the C shell entry in ``Group - Main'', pulling down ``Program'' and selecting ``Properties''. Push the ``Options...'' button and click on the check box next to ``Close window on exit'', removing the X. OS/2 2.x: Right-click on the icon and select ``Open'' followed by ``Settings.'' On the Session page, click on the check box next to ``Close window on exit'', removing the check. The shell doesn't know how to run an external command. One of the environmental variables, particularly HOME, PATH or COMSPEC is probably set incorrectly. Typical symptoms are that the shell doesn't seem to know how to find an external command or that it doesn't know how to run a .cmd file, etc. Another variation might be that it runs the old IBM more.com rather than the new more.exe. If you experience symptoms like these, first check that these variables are set sensibly. The other common possibility under OS/2 1.x is that you're using a network and have execute, but not read access to the application you're trying to run. Due to a bug in the OS/2 1.x kernel, the C shell cannot use the Page 18 Common Problems kernel's DosQAppType function to determine whether the application should be started full-screen, in a text window or as a PM graphics application. Instead, the C shell is forced to read the application's .exe header itself; if it can't read it, it can't run it. The solution is to be sure you have read access. Page 19 Common Problems The shell won't run my new program. Path hashing can sometimes produce surprising results if you create a newer version in your current directory of a command that already exists in another of your path directories. The shell won't know you've done this; its hash will still only list the older version. To overcome this problem, use either the rehash or unhash commands. The shell won't execute commands in the current directory. Your should add the current directory to the list of directories in the PATH variable. cmd.exe always checks the current directory before looking in any of the PATH directories. Hamilton C shell does not make this assumption; if you want the current directory to be first one checked, you should specify it explicitly as ``.'' at the beginning of the list. For example: setenv PATH = '.;c:\os2;c:\os2\bin' The shell keeps running the old version my shell procedure. If you define a shell procedure with proc in a .csh script file and then execute the script, the procedure is compiled into an internal form that's much faster to run and it's kept around in memory to make it run the next time even faster. If you change the text in the script file but don't explicitly throw away the old definition using unproc, the C shell won't know it's supposed to recompile. The shell won't run any cmd.exe internal commands. Most probably, the shell is unable to find your startup.csh file when it starts up. This is the file that should hold the aliases the shell uses to intercept cmd.exe's built-in commands. Check to see that your HOME variable is set to the directory where you've placed startup.csh and that your startup.csh file isn't garbled. When I start an application from the C shell, it dies immediately. Page 20 Common Problems Under OS/2, if you find that an application dies immediately after starting, check that the .exe file is properly marked with its type, i.e., full-screen, PM text windowable or PM graphics. The shell tries to start up an application in accordance with the way it's marked; if it's marked wrong, the application just won't run. Even very recently, a number of PM applications including even e.exe, the System Editor, were being shipped unmarked, which by convention is supposed to mean full-screen. To look at or change how an application is marked, use the markexe.exe utility. (Type ``markexe -h'' for help.) Another possibility is that the application has a bug that makes it fail if the maximum file handle count it inherits from its parent process is greater than 20. This problem has been seen in some past releases of the Microsoft linker (discussed below) and of WordPerfect, for example. You can force the C shell not to bump the file limit when it starts up using the -Z option but this option only works from the Start Programs (1.1) or Group (1.2) menus, not from the command line. (A process always inherits its initial maximum file handle count from its parent; from there, a process can only raise its own limit, never lower it.) The Microsoft OS/2 linker fails under the C shell even though it works fine under cmd.exe. Microsoft has determined there was a bug in the version of the C library used to build the link.exe distributed with MS C 5.1. The linker can fail if it's run as a child of a process that has a maximum file handle count greater than 20; this is a problem because the C shell sets its maximum to 255. If you're encountering this problem, try patching your link.exe file with the patchlnk.exe utility. (Type ``patchlnk - h'' for help.) When I try to run Microsoft's make.exe in the background it hangs. This is a known problem under OS/2 with make and certain other applications that need to spawn child processes of their own. The OS/2 process initialization and completion logic requests a semaphore in KBDCALLS.DLL that's already owned by whatever process in that window is already sleeping in a KbdCharIn call. Until another keystroke is pressed, that semaphore is never released and the background processes are never allowed to cleanly exit. This problem has been fixed in OS/2 2.x and Page 21 Common Problems through CSD 5050 for OS/2 1.3 with a new KBDCALLS.DLL. That DLL for 1.3 is available on request from Hamilton Laboratories and can be downloaded from the listings area in the ``hamilton'' conference on BIX. copy or rename *.* doesn't work right. copy, xcopy, rename and del like to do their own wildcard expansion. To make them work sensibly, be sure your startup.csh file includes and that you use the aliases and procedure definitions we supply to intercept these commands to turn off shell wildcarding just long enough to run them. These definitions can also serve as a model if you discover other applications that must do their own wildcarding. For more information, refer to the discussion on page 23. The -! option doesn't work. The exclamation point is a special character for the shell. The shell lets you pick up text out of previous commands using history references that begin with ``!'' followed by a string that tells what text you're retrieving. To avoid having an exclamation confused as a history reference, be sure the exclamation is at the end of a word, so the next character is a space or a tab. grep '^foo' doesn't work. The circumflex has special meaning as the escape character to the C shell, even inside quotes. If you want to pass a literal ``^'' to grep (or anything else) from the command line, you must type ``^^'' unless the immediately preceding character was ``[''. When I list a directory over the network, not everything shows up. This is a known bug in the OS/2 networking code, not the C shell. The problem occurs if (1) the directory is read over a network, (2) directory entries are being read in blocks (for higher performance) rather than one-at-a- time and (3) the total number of characters in all the filenames in that directory happens to be just right. In all cases observed, adding or deleting any arbitrary entry in the directory makes the problem go away. The Page 22 Common Problems bug affects the C shell and its utilities because they use blocked reads; simpler programs like cmd.exe's DIR are unaffected because they read one entry at a time. The bug appears to have been introduced in IBM OS/2 EE CSD WR04098 and Microsoft Lan Manager 2.0, both issued around year-end, 1990. IBM has verified the problem and has developed a fix, which is now shipping as part of OS/2 EE 1.3. If you encounter the problem and you're an IBM customer, you should call 1-800-237-5511 or contact your local IBM representative and ask for a copy of the new netwksta.sys file being distributed as APAR IC02287. You can also download this file from the listings area of the ``hamilton'' vendor support conference on Bix or contact us directly and we'll mail you a copy. In the meantime, this release contains a work-around for disabling the block read feature. If you create an environmental variable, NETWORKBUG, and set it equal to 1, directory reads will be done only one-at-a-time, ensuring correct results at all times, albeit with some degradation in performance. You can do this either from the C shell: setenv NETWORKBUG = 1 or in your config.sys: SET NETWORKBUG=1 du, pwd and vol waste time sitting and spinning when they hit a removable drive that's empty. If you have a removable media device other than A: or B:, these utilities will normally try to report them. That's probably not you want, at least not usually; you can specify just the set of drives you do want reported using the DRIVEMASK environmental variable. Page 23 Common Problems cd /foo doesn't work. Hamilton C shell tries to serve users coming both UNIX and MS-DOS backgrounds. To do this, the C shell and all the utilities accept command line options to start with either ``-'' (UNIX-style) or ``/'' (DOS-style). It also recognizes filenames typed with either forward or backward slashes. But when you type ``cd /foo'', it guesses wrong and thinks you're trying to give it a command line option that it can't recognize. If this is really not what you intend, set the SWITCHCHARS environmental variable to just the specific characters you want recognized. E.g., you might include this in your config.sys to have only ``-'' recognized: set SWITCHCHARS=- I've just installed OS/2 1.2 and suddenly my environment variables don't work. The auto install program distributed with the fall, 1989 releases of OS/2 1.2 from Microsoft and IBM has a bug. It tries to automatically convert entries on the 1.1 Start Programs menu into corresponding entries on the new 1.2 Group Main menu. If the parameters line for starting a program has text on it (as the C shell's does), the entry is garbled even though it looks correct and causes a garbled environment to be passed to the shell. Editing the entry does not fix the problem. The only solution is to delete the entry and rekey it from scratch. I can't set my own screen colors. Yes, you can (finally, in this latest release.) But you cannot do it just by embedding ANSI escape sequences into your prompt since the C shell will immediately reset the colors back to what it thinks they should be. To set your own preferences for screen colors, you must use the COLORS environmental variable. See the chapter on customizing the shell or the colors.csh script in the samples directory for more information. The C shell's icon won't display in Group-Main. If you install the C shell as the default command processor by specifying it on the PROTSHELL line in Page 24 Common Problems config.sys and entering its path as ``*'' in Group-Main, you will see only the default OS/2 icon in Group-Main if you select View Icon. If you start, then minimize the C shell, it will have the correct icon, however. This has been reported to IBM. Their response is that, by design, when the path is an ``*,'' the Group code does not attempt to resolve the actual pathname (and whether there's any icon associated with it) until you actually click on the entry to start it. They agree this means you will not see the correct icon in the Group menu but claim this is what they intended and that it's not a bug. more crashes on the OS/2 2.0 Beta and LA Releases. The dynamic link library supporting 8514 displays in the beta and LA releases from IBM has a bug which causes some VIO applications, including more, to crash with a protection violation if they're run in a text window. They work fine full-screen. This problem has been fixed in the GA build. more hangs or exits prematurely on the OS/2 2.0 6.167 and LA releases. Under the 6.167 and LA releases, the 8514 display driver is completely unusable. It even has problems repainting the screen after a menu has been closed or displaying icons in the templates folder. It even causes more to hang the whole system if you have an 8514. But even using the VGA driver, random problems will be observed due, apparently, to bugs in the keyboard driver. Depending on what's fed to it through a pipe, more will occasionally prematurely exit after the first screenful. All these problems have been fixed in the GA release. The C shell can't change its title bar or icon under the OS/2 2.0 6.167 Beta and LA releases. This functionality was disabled in the 6.167 Beta and LA releases as part of the work to add the Workplace Shell. This problem has been fixed in the GA release. Alt-Enter doesn't work to grab commands from the history list under Windows NT and the OS/2 2.0 6.167 Beta Release. Page 25 Common Problems Under Windows NT and OS/2 2.0 6.167, Alt-Enter is gobbled up by the system as a keystroke combination used to signal that an application should be toggled back and forth between the desktop and a full-screen session. Under the these systens, you'll have to type Ctrl-Shift- Enter instead. The C shell (and lots of other applications) only have default icons under the OS/2 2.0 6.167 Beta and LA Releases. The Workplace Shell does not support .ico files. All icons for text applications must be stored in the extended attributes. The latest builds of the C shell have the icon both in the EA and in an .ico file but if you copied the C shell onto your disk with a utility (e.g.., something other than cp) that does not support EA's, that information probably got lost. To put an icon into the extended attributes, use the OS/2 1.3 File Manager, selecting the file, pulling down ``Properties'' and selecting ``Icon...''. I just installed the C shell as the PROTSHELL and now when I start Commmunications Manager, it dies immediately. Communications Manager is invoked via a .cmd script file. Follow the instructions in step 4 on page 10 to rewrite that entry to start that script explicitly via cmd.exe. I can't wildcard filenames with $, quoted or escaped characters in them. Yes, you can (finally, in this latest release.) To do so, just quote or escape the special characters. E.g., to get all the files that begin with $, you might type ^$* or '$'* . I can't run the C shell inside an Epsilon editor window. The Epsilon editor tries to run whatever command processor you use by creating a full-screen session and doing a KbdRegister to intercept the KbdStringIn API entry so that Epsilon can feed it whatever you type in Page 26 Common Problems the editor window. Output (stdout and stderr) from the child session is redirected over a pipe back to the editor. There are a couple problems in their approach: (1) They neglected to consider that not all applications use KbdStringIn; if stdin is attached to a keyboard, the C shell reads a keystroke at a time using KbdCharIn and those calls still end up tied to that full-screen session rather than being redirected. (If stdin is attached to anything else, it uses DosRead calls.) The authors of Epsilon really should have intercepted the whole set of Kbd calls, not just one of them. (2) Not all applications write their output to stdout or stderr; applications like more, that use Vio output, won't run properly. Their output appears in that full-screen session, not back in the editor window. Epsilon really should be doing a VioRegister to grab the Vio output also. We are working with Lugaru Software (the authors of Epsilon) on a solution that should be available shortly. A partial workaround is to tell Epsilon to use a separate program, which just reads input and pipes it to the C shell. Marty Klos at IBM has written a small C program to do that and placed it in the public domain. A copy is available on request from us or may be downloaded from the listings area of the ``hamilton'' vendor support conference on BIX. rm doesn't remove anything, it just puts everything in a hidden directory. You're using the notorious Microsoft rm command instead of the Hamilton rm. The Microsoft rm doesn't remove anything; it just puts things in a hidden system directory. Hamilton rm is actually in hrm.exe under Windows NT and should be aliased to rm in your startup.csh file. Fix that and then, to get rid of all those ``deleted'' directories: cd \; rm -x `ls -1ra +H | dim | grep 'deleted$'` Page 27 Common Problems Page 28 Support Product Support If you encounter problems or would like to make suggestions for a future revision, please contact us by any of the following or by regular mail; we promise a prompt response. Phone: 508-358-5715 FAX: 508-358-1113 MCI Mail: 389-0321 Telex: 6503890321 BIX: hamilton CompuServe: 70034,2025 Internet: 3890321@mcimail.com Also, on Bix, we have a vendor support conference. Do a ``join hamilton'' once you get on or follow the menus into the conference system. Bug Reports If you encounter what you believe to be a bug, please try to experiment to see what specific command or command sequence seems to be failing before calling. A problem that's easily reproducible is obviously easier to fix. Built in to Hamilton C shell are a number of consistency checks to trap bugs before they cause damage and to snapshot enough information to help us diagnose and repair the problem. If the shell is actually crashing, look to see if a new entry has been added to the error log, crash.csh, in your home directory; that information will be useful. When you call, we'll try to provide an immediate workaround if there is one. If the problem is serious but straight-forwardly correctable, we can generally offer an interim release at no charge to fix that specific problem. At the very least, we try to schedule it for an upcoming general release. Future Enhancements Work continues on additional features and enhancements. As they become available, we want you to have them. Page 29 Support Please return the registration form by mail or FAX. Without that, we often have no way of knowing who you are to send updates to. This is particularly true if your copy was purchased through your company's purchasing department or through a retail distributor. Also, we look forward to your feedback as we strive to improve the product. Page 30 Support Page 31 Getting Started User Guide Getting Started Starting Hamilton C shell is simple: select it from the Start Programs window or the Program Selector or type ``csh'' as a command to cmd.exe. After the initial greeting, you'll see the first prompt: (The underscore is meant to be the cursor.) Hamilton C shell(tm) Release 2.2 Copyright (c) 1988-1993 by Hamilton Laboratories. All rights reserved. 1 D% _ This tells you that it will remember what you type as command number 1 and that your current drive is D. The ``%'' is traditional; rather like the ``>'' for DOS. Naturally, you can change your prompt if you want, to be anything you like. For example, to get a prompt that looks like one you might get from cmd.exe+: 1 D% set prompt1 = '[$upper(cwd)] ' [D:\DOUG] _ This works by taking the value of the cwd (current working directory) variable, turning it to upper case using one of the built-in procedures and pasting left and right brackets around it. The value is recalculated each time a prompt is given, so it always displays an up-to- date value. (Lists of all the built-in variables and procedures are given in later sections.) To set it back: [D:\DOUG] set prompt1 = '$@ $CDISK% ' 3 D% _ ____________________ + We introduce this is as the first example with some trepidation: the prompt seems to be the first thing people want to change. But it can also be one of the more daunting projects if you're getting started. This example is offered more in the spirit of assurance that, with a little experience, the prompt can be set to anything you like. Page 32 Getting Started Basic Statements Generally speaking, whatever commands you might have typed into cmd.exe will still work here. Even an ``internal'' cmd.exe function like dir works: 3 D% dir The volume label in drive D is USER. Directory of D:\DOUG\SH\DOCS\SCRIPT\HELLO . 2-23-89 2:13p .. 2-23-89 2:13p HELLO C 72 2-23-89 12:56p HELLO EXE 7731 2-23-89 12:57p MEMOS 2-23-89 1:46p 5 File(s) 1581056 bytes free 4 D% _ If the command you type refers to a .cmd batch file or a cmd.exe internal function, Hamilton C shell passes it to a child process running cmd.exe for evaluation. (cmd.exe's built-in functions are intercepted with aliases defined in your startup.csh file.) Everything else is evaluated directly by Hamilton C shell. For example, if you type the name of an .exe file, the appropriate DosExecPgm( ) or DosStartSession( ) call to the OS/2 kernel or CreateProcess( ) call to the NT kernel to start that program will be done directly by Hamilton C shell. A bit-mapped hash mechanism is used so that when you type the name of a command, the shell zeroes right in on file you mean. It doesn't have to check every path directory for every possible extension. Naturally, if you type a command that doesn't exist, the shell complains: 4 D% zork csh: Couldn't find an executable file named 'zork'. By being more than merely a ``wrapper'' around an existing command processor, several advantages are created: (1) performance is understandably (and visibly!) much higher and (2) limitations on command line lengths, etc., become the relatively generous limits of OS/2 and NT, rather than the restrictive limits of cmd.exe. Customizing the Screen Colors Page 33 Getting Started The C shell's default screen colors are white characters on a black background. Highlighting and color are used to make some things (special files, etc.) stand out. All the use of color or highlighting is completely customizable. You can choose anything you like. The chapter on customization will go into this in detail, but for now, let's suppose we'd simply like to pick something a little easier on the eyes, like white characters on a blue background: 5 D% setenv COLORS = white on blue Command Line Editing With command line editing, you'll notice immediately how much easier it is do things quickly without a lot of retyping. As you try the examples, notice how the arrow, insert, delete, home, end and other keys can be used to recall previous commands or make changes anywhere on the line. Command line editing is like having a full-screen editor pasted onto the front end of the shell. Key bindings are intuitive and follow accepted conventions. You can create enormous commands that stretch over screen after screen and move around with the arrow keys, inserting or deleting anywhere. Watch changes ripple almost instantly down even an entire screenful of text. We think you'll find our command line editing superior to anything you've seen or used elsewhere. The basic key assignments are: Toggle between insert and overstrike modes. (The cursor is thicker when you're inserting.) Beginning/end of command line. One character left/right. Up/down one command in the history list. Pressing Ctrl with the arrow keys lets you move by words or lines. Pressing Alt instead does word or line deletion. (The convention we follow is that the Alt key is a little ``more powerful'' than the Ctrl key.) What you last deleted is kept in a scrap buffer and can be pasted back elsewhere. To paste something from Page 34 Getting Started the scrap buffer back into the command line, move the cursor to where you want it done and press: Ctrl- Paste one word at a time. Alt- Paste the whole thing. Command Completion In addition to backing up through your previous commands one at a time with ­ and ¯, you can also ask the shell to search back through any previous commands you've typed for the last command that either started with or contained the characters in the previous word. Ctrl- means ``look for a command that started with ...,'' and Alt- (again, a little ``stronger'') means ``look for a command that contained the string anywhere.'' (On NT, it's necessary to type Ctrl-Shift- because Alt- is grabbed by the system to mean switch to full-screen.) Repeatedly pressing these keys cycles up through all the matching commands you've previously typed. Command completion uses something called the history mechanism to recall commands you've previously typed. Later, we'll devote a whole chapter to some of the more advanced uses of history. Page 35 Getting Started Filename Completion Filename completion is another ``creature comfort:'' you type just a fragment of a filename and let the shell fill in the rest. The two variations are using the F key for basic filename completion or the D key if you want all the duplicates listed. Alt-F or Ctrl-F Filename completion. Look for a filename that starts with preceding characters. If it matches a single file, fill in the rest of the name. If more than one file matched, show the part that was the same for all, highlighted in green. (Bright red means there were no matches at all.) Alt-D or Ctrl-D Duplicate completions. Show any/all matching filenames, one after the other with spaces between. Filename completion is actually done with wildcarding. Unlike cmd.exe, Hamilton C shell does any wildcarding before excuting the command you type. It uses a powerful recursive pattern match algorithm that guarantees sensible matches even if you type a very complex pattern. Wildcarding is the subject of a whole chapter up ahead. The Tour Begins Here The following chapters introduce the various facilities Hamilton C shell provides, starting with some of its basic vocabulary: the simple utilities that come with it. Following discussion shifts to the intrinsic, internal functions provided by the shell itself: i/o redirection, pipes and command substitution; the history mechanism and wildcarding. Intermediate level discussion follows, describing expressions, variables and aliases and the editing and quoting facilities. The process and thread scheduling mechanism is described, outlining how an activity can be placed in the background. The tour will then cross the threshold from discussion of individual statements to discussion of structures of Page 36 Getting Started statements. Structures for iteration and condition- testing and procedural abstraction will be introduced. Finally, we'll wrap up with discussion of how to customize the shell together with a section detailing specific compatibility issues between the Hamilton and original Berkeley C shells. Page 37 Utilities The Utilities Hamilton C shell comes with a lot of utilities that form some of its vocabulary. They do small, but oft- needed functions, often in a novel, faster or more convenient way than you'd find in ``plain vanilla'' OS/2 or NT. This section provides a quick tour, outlining some of the capabilities and conventions. ls: List files ls is a somewhat nicer way to list a directory: 6 D% ls memos hello.c hello.exe sysstuff Subdirectories are highlighted (shown here in bold.) If a file or directory has the system bit set, it's still listed, displayed in green (shown here in italic.)+ Normally, ls lists everything in lower case for better readability. In long format: 7 D% ls -l D---- Feb 23 13:46 - memos ---A- Feb 23 12:56 72 hello.c ---A- Feb 23 12:57 7731 hello.exe -S-A- Feb 23 13:22 15 sysstuff Conventionally, ls lists things alphabetically, with directories ahead of files. There might be hidden files or directories, but to see them you have to ask: 8 D% ls +H memos hello.c hello.exe hiding sysstuff Conventions To find out how any of the utilities work, just use the -h option. For example, 9 D% ls -h ____________________ + All our examples will be given in terms of the default screen colors. But these are easily changed to your own preferences. See the chapter on customization or the colors.csh script file in the samples directory. Page 38 Utilities tells about options for more detailed listings, sorting the list by date or by size, selecting only certain types of files, etc. ls is a read-only activity; it never makes any changes to the file system. Lists are always sorted in memory; its speed and flexibility completely obsolete the old (and dangerous) ``directory sort'' utilities popular on DOS. The names of the utilities were chosen to be consistent with the names of similar functions on UNIX, where they provided much of the vocabularly of the original UNIX C shell. But changing the name of a utility is a simple matter: just rename the corresponding .exe file or, better still, create an alias (discussed later.) By convention, the utilities expect options to come ahead of any files you specify. Options are case- sensitive. We've tried to use mneumonic letters for options (e.g., h for help) and to use the same letter to mean the same thing across related utilities; achieving that is simply more feasible with 52, not just 26 characters to choose from. Our examples generally show options introduced with ``-'', but you could equally well follow the DOS-style convention of using ``/'' if you prefer. If indeed you want only ``-'' or only ``/'' interpreted as an option character, this can be set with the SWITCHCHARS environmental variable, which can be set either from the C shell or from your config.sys file on OS/2 or from the Control Panel on NT. Sadly, it won't have any effect on the standard OS/2 or NT commands like dir or xcopy or on applications you purchase elsewhere, but it will work on all the commands supplied with the C shell. For example, to have only ``-'' recognized as an option character, you might type this into the C shell: 10 D% setenv SWITCHCHARS = - or put this into config.sys (rebooting to make it take effect): set SWITCHCHARS=- You can type options in any order (except where one overrides another, in which case the last setting is used) and you group them together or type them separately as you choose. For example, ``ls -L -d -w'' is exactly the same as ``ls -dwL'' and produces a very long format (very detailed) list of the current directory, sorted by date (newest ones last), with sizes of any directories filled in by walking down through the directory tree, adding up all the sizes of all the files found there. Page 39 Utilities You can always unambiguously end the options with ``--'' in case you have a filename or an argument string that begins with one of option-introducing characters. Also, since the shell does the wildcard expansion, it's a bit more convenient and faster for the utilities to look for any options right at the beginning of what could be a very long list (up to 64 kilobytes) of filenames or other command-line text. We'll always follow the OS/2 and NT convention of using ``\'' in filenames in this book and we generally advise that you do too, not so much because the C shell cares but because so much other OS/2 and NT software does. To some fair degree, it's a case of ``when in Rome, doing as the Romans do.'' But if you really do prefer, you can generally use ``/'' with the C shell and all the utilities. Do remember, however, that if you type a filename starting with ``/'' to mean the root, you have to be careful that it can't be confused as the start of an option. (This is a good use for the ``--'' option or the SWITCHCHARS variable.) Page 40 Utilities echo echo is a little different than the vanilla OS/2 or NT echo. It does only one thing: it prints whatever arguments words you give it; there's no echo on or echo off-style status reporting function. But it does offer much finer control over what gets printed: you can write binary values, choose not to append a new line and write to stderr instead stdout. Here's an example where the ANSI escape sequences turning brightness on and off are embedded into a string being echoed. The ANSI escape character is octal 033; binary values or special characters like ``['' are introduced by the ``^'' shell escape. 11 D% echo Have a ^033^[1mnice^033^[0m day. Have a nice day. (Processing of the ^ escape sequences is done by the shell before any command ever sees it. As a result, you can use escape sequences to construct command line arguments for any command; this feature is introduced here only because it tends to be most often used with echo.) mv, cp and rm: Move, copy and remove The mv (move), cp (copy) and rm (remove) trio allows files and directories to be treated as simple objects. mv will move either files or directories treating them simply as objects, even across disk partitions. In this example, the two hello files are moved into a new directory, illustrating how mv understands that if there's a many-to-one relationship, the destination has to be a directory. 12 D% mv hello* hello 13 D% ls hello memos sysstuff 14 D% ls hello hello.c hello.exe Similarly, cp will copy a file or even an entire directory. The copies cp produces are always exact Page 41 Utilities logical copies, with correct timestamps+ and attribute bits and including any hidden or system files. 15 D% cp hello newhello 16 D% ls hello memos newhello sysstuff 17 D% ls -l hello ---A- Feb 23 12:56 72 hello.c ---A- Feb 23 12:57 7731 hello.exe 18 D% ls -l newhello ---A- Feb 23 12:56 72 hello.c ---A- Feb 23 12:57 7731 hello.exe cp does not consider it an error to copy over an existing file unless the file about to be overwritten has its read-only bit set. Finally, rm can be used to remove a file or even an entire directory. But it does insist that you tell it you really mean it if you ask to remove a directory that's not empty or anything that's marked with the system bit. 19 D% rm sysstuff rm: Can't remove system file 'systuff' without -S option. 20 D% rm -S sysstuff 21 D% ls hello memos newhello 22 D% rm newhello rm: Can't remove non-empty directory 'newhello' without -r option. 23 D% rm -r newhello 24 D% ls hello memos As you can see from these examples, the general style of the utilities is fairly terse. Like the proverbial Vermonter, they don't say anything unless they've got something to say. Even copying or removing a directory happens without fanfare as long as the appropriate ``yes, I really mean it'' options are supplied. more more is an especially fast browsing filter. There are two ways to use more. The first is in a pipeline, the ____________________ + Files only under OS/2 1.1. New directories always get the current timestamp unless you're running OS/2 1.2 or later. Page 42 Utilities way ``vanilla'' more might be used when you suspect the data may be longer than a screenful: 25 D% ls -l c:\os2\bin | more : : If the output turns out to be less than a screenful, it's as though you'd just typed the ls command by itself. In fact, there's not even a noticeable performance penalty. But if it's more than a screenful, more switches to an interactive mode where you can use the arrow keys, etc., to browse up and down through the listing. more can also be used for browsing a list of the files you give it on the command line: 26 D% more *.c more incorporates the Berkeley notion referred to, tongue-in-cheek, as ``more is less+'': it's a good paging filter that lets you go forwards and backwards. It also offers a number of different ways of looking at or searching the data including binary, as control characters, line-numbered, etc. Perhaps most important, it's fast. Part of more's speed comes from an internal cache of about 11K characters of text coupled to an indexing structure that it builds on the fly as it reads the input. When you move forward or backward within the cache, screen redraw rates are the limiting factor in performance. Outside of range of the cache, if the input is from a disk file, the indexing structure, technically an ISAM, tells more how to seek to the new location. There is also a ``huge'' version of more, called moreh, that was compiled in large model and while slightly slower, has the advantage of caching up to about 4M characters. moreh can be useful when speed is less important than being able to scroll all the way back through a large amount of text coming through a pipe. touch ____________________ + The story is now a part of computer folk history: at first, more only went forward. Then someone created a filter that went backwards, which he aptly named less. When later versions of Berkeley's more incorporated this feature, they were heralded by announcements that, finally, ``more was less.'' Page 43 Utilities touch lets you change the timestamps of individual files or directories+ or, using the -r (recursive) option, of everything in a whole directory tree. If the desired timestamp isn't given, touch uses the current time. If the filename doesn't exist, it's created as a zero-length file. 27 D% ls hello memos 28 D% touch zork 29 D% ls hello memos zork chmod chmod lets you set a file's attributes but leaves the timestamp alone. Here is an example, first setting the system bit (making it show up in green), then making it hidden: 30 D% chmod +S zork 31 D% ls hello memos zork 32 D% chmod +H zork 33 D% ls hello memos Of course, the file is still there and you can continue to manipulate its attributes: 34 D% ls -l zork -SHA- Feb 23 13:16 0 zork 35 D% ls +a . hello zork .. memos 36 D% chmod +R zork 37 D% ls -l zork -SHAR Feb 23 13:16 0 zork Many users will find that a file's system bit is more useful than they'd thought before. With chmod, it's easy to set or clear the bit and setting it doesn't make the file hidden. Quite the contrary, ls makes it stands out in green. Also, a file marked ``system'' is a little safer from accidental deletion or overwriting. These are often convenient characteristics to attach a few specific ____________________ + On an OS/2 1.1 system, the kernel allows you to change the timestamps only on files, not directories. touch'ing a directory does nothing unless you use the - r option to recursively touch the directory's contents. Page 44 Utilities files within a large directory. For example, the author tends to routinely mark make files within a C source code directory as ``system'' just so they'll stand out. du, vol and pwd du, vol and pwd provide quick snapshots of your disk partitions: du tells how much of the partition is used; vol displays the label; and pwd shows the current directory on each partition. 38 D% du c: 31.904 M Total = 29.465 M Used + 2.439 M ( 7.65%) Free root d: 23.920 M Total = 22.438 M Used + 1.482 M ( 6.20%) Free user e: 13.957 M Total = 8.520 M Used + 5.438 M (38.96%) Free misc 39 D% pwd c:\os2\include d:\doug\sh\docs e:\tmp 40 D% vol c: Jan 24 22:32:10 1988 root d: Nov 27 20:34:58 1988 user e: Jan 17 17:12:20 1988 misc A common convention observed by the utilities is that if one entry on a list is more current or special than the others, it's highlighted. du, vol and pwd each highlight the entry describing the current disk. For the benefit of those who have lots of partitions, some of which they don't want to bother listing all the time, du, vol and pwd look for a DRIVEMASK environmental variable which can be used to mask off just the drive you want. This is especially useful for excluding drives that take removable media; if they're empty, they can waste a lot of time trying to read a diskette that's not there. dirs, pushd, popd and rotd The shell provides a built-in mechanism for keeping several directories ``handy.'' This mechanism is the directory stack, which always contains a list of fully- qualified directory pathnames with the current directory at the top. You can display the list with the dirs command: Page 45 Utilities 41 D% dirs d:\doug\sh\docs Initially the list contains only your current directory. When you push a new directory on the stack with pushd, that becomes your new current disk and current directory. pushd also reports the resulting stack contents. 42 D% pushd c: c:\os2\include d:\doug\sh\docs 43 C% pushd e: e:\tmp c:\os2\include d:\doug\sh\docs Calling pushd without any arguments just swaps the top two directories: 44 E% pushd c:\os2\include e:\tmp d:\doug\sh\docs Popping elements off the stack is done with popd, which also reports the resulting stack. 45 C% popd e:\tmp d:\doug\sh\docs The stack can also be rotated with rotd. (We'll push another directory first so we can see that rotation is upward, with the top item going to the bottom of the stack.) 46 E% pushd \ e:\ e:\tmp d:\doug\sh\docs 47 E% rotd e:\tmp d:\doug\sh\docs e:\ You can pop multiple directory entries at once, but if you ask to pop more than exist, you'll get a message: 48 E% popd 5 csh: The built-in popd command can only accept a integer argument n, where n > 0 && n < number of elements on the directory stack. The default for n is 1. Page 46 Utilities 49 E% popd d:\doug\sh\docs e:\ fgrep and grep fgrep and grep are fast string search utilities. Their names and the regular expression syntax are traditional; it's an accepted standard and we've followed it. fgrep and grep are used to scan through long lists of files or filter data coming through a pipe for strings or patterns you specify. They'll quickly report all the matching lines. If you like, you can get more or less detail in the output, e.g., have line numbers shown or just get a total count of all the matches. fgrep and grep both have the ability to look for a large number of patterns in parallel (using the -s or -f options) with almost no discernable performance degredation. They're very fast. Both precompile and optimize their search patterns, use direct kernel api calls for all i/o and use a very high performance buffering structure to allow extremely fast scanning of large amounts of data. fgrep fgrep is the simpler and slightly faster of the two search utilities. It does a simple string compare between the string you're looking for and the characters on each line. If the search string is found anywhere on the line, it's a match. There are some options for ignoring differences in upper-/lower-case or in the amount of white space (spaces and tabs) between words but but mostly it's quite simple comparison. Here's an example of using fgrep to search a very simple personal phone directory where each record is just a line of text and we'll search it . (Later we'll learn how to package things like this up into aliases or shell procedures so you can call them with just a few keystrokes.) 50 D% fgrep -i doctor \phone Doctor James Gardner 508-999-0000 12 Canton St Doctor Karen Strickland 508-721-1223 N.E. Medical Offices Page 47 Utilities grep grep looks for special patterns called regular expressions, which are similar to (but slightly different from) filename wildcarding. The grammar is recursive, meaning a regular expression to be matched can be written, in turn, as a nested series of regular expressions: c Any ordinary character matches itself. \c Match the literal character c. ^ Beginning of line. $ End of line. . Match any single character. [...] Match any single character in the list. [^...] Match any single character not in the list. \n Match whatever literal text the n'th tagged \(...\) expression matched. r* Match zero or more occurrences of r. r1r2 Match expression r1 followed by r2. \(r\) Tagged regular expression. Match the pattern inside the \(...\), and remember the literal text that matched. At the lowest layer, you give a character or set of characters to be matched anchored, if you want, to match just the beginning or just the end of a line. At the next layer, the ``*'' character lets you match a variable number of repetitions of a pattern. When you type a regular expression on the command line, keep in mind: (1) Many of the characters have special meaning to the C shell and have to be inside quotes. (2) You have to type two ``^'s'' to get just one because ``^'' is the shell's literal escape character. (3) ``*'' is a postfix operator. It operates on the preceding regular expression; by itself, it is not a ``match zero or more characters'' wildcard character as you may be used to with filenames. Here's an example of searching through all the source code for a large application, looking for all occurrences of lines that begin with ``statement'' followed by a ``y'' somewhere on the line and showing the line numbers of any matches. (The -s option tells pushd and popd to work silently.) 51 D% pushd -s ~\sh 52 D% grep -n '^^statement.*y' *.c allocate.c:418:statement_obj *allocate_statement(size, type) 53 D% popd -s Page 48 Utilities sed sed is a stream editor. Just as you might think of using a regular editor to edit a file, deleting or inserting lines, doing search/replace operations, etc., sed lets you edit a stream of data: individual lines are read from stdin, edited according to the script you give and written to stdout. A very simple sort of script might be given right on the command line. Here's a simple search/replace: 54 D% echo hello world | sed s/world/everybody/ hello everybody sed uses the same regular expressions used by grep. It's possible to pick up pieces of the input as tagged expressions and move them around. In this example, the two strings on either side of the space are tagged, then swapped around. Quotes are used around the search/replace command so the C shell will treat it as one long literal string to be passed to sed. (Parentheses, spaces and asterisks otherwise have special meaning.) Notice how the ``*'' construct, meaning match zero or more occurrences actually matches as many repetitions as possible. 55 D% echo hello world | sed 's/\(.*\) \(.*\)/\2 \1/' world hello For more complex operations, sed offers a wide array of operators including even conditional branches and a hold buffer where a string can be saved temporarily from one line to the next. If your script is very long, the -f option lets you specify it in a file. diff diff is an extremely fast and flexible utility for quickly comparing ascii files, looking for differences. In the simplest form, you simply give it two filenames corresponding to the old and new versions and let it go to work, reporting sections that have been deleted or added in a traditional format. For example, as a software developer, I might use it to compare old and new versions of a C program: 56 D% diff archive\parse.c parse.c 1493 c 1493 < d->inline_cnt = src->inline_cnt++; --- > d->inline_cnt = ++src->inline_cnt; Page 49 Utilities Each change is reported in terms of the line number or range in the old version, whether it's an addition, change or deletion, the line numbers in the new version and then the affected lines from each file, separated by a line of ``---''. diff supports the traditional options for ignoring differences in upper-/lower-case or in the amount of white space on the line, for recursively comparing entire directory trees of files, etc. One of diff's most novel features is its ability with the -! option to generate a merged listing where text that's deleted is shown in red, new text is shown in green and the rest is displayed normally. This makes it extremely easy to view your changes in context. (To use this option, remember that ``!'' is a special character to the shell; type it at the end of the option list so there'll be a space following.) head and tail head and tail are used to display just the first or last few lines or characters of a file. Normally, they expand any tabs into spaces so you don't need to filter them through more. tail is particularly interesting. If all you want is the end of a very large file, tail doesn't waste time reading the whole file from start to finish. Instead, it jumps right to the end and reads it backwards! If the file is truly large (on the order of several megabytes) and all you want is a little bit off the end, this is the difference between chugging along for several seconds versus getting an almost instantaneous response. tail also has a -f follow option. What that means is that when it gets to the end of file, it enters an endless loop, sleeping for a second, then waking up to see if more has been added. This is particularly useful if, e.g., you have an operation, say a large make, active in one window with its output redirected to a file. From another window you can periodically check in on the progress by typing: 57 D% tail -f e:\tmp\make.log : ^C tail lets you watch lines get added without consuming much processor resource (since it sleeps in the kernel most of the time) so you can watch a background activity Page 50 Utilities progress without affecting its performance. After you've watched for a while, just type ^C to interrupt and get out. The interrupt only goes to the tail program; the application off in the background or in another window creating the file is not affected and will go on about its business until you come back once again to check on it. cut cut is a simple filter for selecting out just certain fields or character positions of each line of input. You choose what characters should be interpreted as the field delimiters and which fields should be copied to the output. For example, if you kept your phone book in \phone, you might strip off just the first word from each line to get everyone's first names: 58 D% cut -f1 -d' ' \phone Ed Helen Jack Vickie : The -f option means you want to count by fields, selecting the first field and that the delimiter is a space character. (Notice the quotes around the space.) Page 51 Utilities split split lets you break up a large file into smaller, fixed-size pieces counting either by lines or by characters. Each of the smaller files it creates are numbered, e.g., chunk.001, chunk.002, chunk.003, etc. One example of where you might use split might be if you had a very large file you wanted to transmit over a modem. If the line dropped suddenly, you wouldn't want to have to start all over on a 2M file. If you split it first into 200K chunks, you'd stand to lose a lot less. Another example might be if you had a truly enormous text file that was just too big to easily edit with your favorite editor. Splitting it up into chunks of only 10K lines each might be a solution. tabs tabs lets you expand or unexpand tab characters based on a set of tab settings you give it. Tab settings are religious. I like them every 3 spaces but you probably like something else. If you're composing something to be sent as email or posted on a bulletin board, it's probably nice to expand it out before you send it so everyone sees what you see. tr tr is a another simple filter for translating characters from input to output. For example, you could translate everything from lower to upper case by typing: 59 D% tr a-z A-Z hello world HELLO WORLD ^Z We typed the first hello world and tr has just echoed it in upper case. ^Z is the end-of-file character defined by OS/2 and NT. tr also has a number of options for squeezing out repeated sequences of the same character or editing out just certain characters and even for normalizing the text in a file, ensuring that every line ends with a carriage return/line feed combination. That's handy if you're importing a file from another operating system. Page 52 Utilities strings strings lets you simply list out all the ASCII strings in an otherwise binary file. Sometimes this can be useful for spelunking around through a file when you're really not sure at all just what's inside it. Various options are available to trimming the output so only strings of a minimum length, etc., will be shown. For example, 60 D% strings hello.exe !This program cannot be run in DOS mode. : Hello, world : Another example might be if you suspected an application was carrying a virus. Naturally, strings can't guarantee something's free of any virus, but on the other hand, if you scan it with strings and find something like this, obviously you should be careful: 61 D% strings a:suspect.exe : Aha! Gotcha! I just blew away your hard disk! dskread and dskwrite This pair of utilities can be used to quickly copy, format or mass duplicate diskettes in a single pass. Here's an example using dskread to read a whole diskette image onto your hard disk and then write it back out onto a new floppy with dskwrite. The dskwrite -a option means autoformat, i.e., if the new disk isn't already formatted, format each track as it's written. The -v option means read back and verify each write to be sure a good copy was made. 62 D% dskread a: >disk.image # Read the whole diskette 63 D% dskwrite -av a: