Linux

In UNIX, which is the parent operating system of Linux and the origin of many of the ideas and of the philosophy of the operating system, a variety of different shell programs are available. The most common on commercial versions of UNIX is probably the Korn shell, but there are many others. So why use a shell to program? Well, the shell leads a double life. Although it has superficial similarities to the Windows command prompt, it’s much more powerful, capable of running reasonably complex programs in its own right. You can not only execute commands and call Linux utilities; you can also write them. The shell uses an interpreted language, which generally makes debugging easier because you can execute single lines, and there’s no recompile time. However, this can make the shell unsuitable for time-critical or processor intensive tasks.

One reason to use the shell for programming is that you can program the shell quickly and simply. Also, a shell is always available even on the most basic Linux installation. So for simple prototyping you can find out if your idea works. The shell is also ideal for any small utilities that perform some relatively simple task where efficiency is less important than easy configuration, maintenance, and portability. You can use the shell to organize process control, so that commands run in a predetermined sequence dependent on the successful completion of each stage.

Linux Tags: programming, shell

The vast majority of Linux systems are reasonably well documented with respect to the system programming interfaces and standard utilities. This is true because, since the earliest UNIX systems, programmers have been encouraged to supply a manual page with their applications. These manual pages, which are sometimes provided in a printed form, are invariably available electronically.

The man command provides access to the online manual pages. The pages vary considerably in quality and detail. Some may simply refer the reader to other, more thorough documentation, while others give a complete list of all options and commands that a utility supports. In either case, the manual page is a good place to start.

The GNU software suite and some other free software use an online documentation system called info. You can browse full documentation online using a special program, info, or via the info command of the emacs editor. The benefit of the info system is that you can navigate the documentation using links and cross-references to jump directly to relevant sections. For the documentation author, the info system has the benefit that its files can be automatically generated from the same source as the printed, typeset documentation.

Linux Tags: documentation, help, online

One disadvantage of static libraries is that when we run many applications at the same time and they all use functions from the same library, we may end up with many copies of the same functions in memory and indeed many copies in the program files themselves. This can consume a large amount of valuable memory and disk space.

Many UNIX systems and Linux support shared libraries can overcome this disadvantage. A complete discussion of shared libraries and their implementation on different systems is beyond the scope of this book, so we’ll restrict ourselves to the visible implementation under Linux.

Shared libraries are stored in the same places as static libraries, but shared libraries have a different filename suffix. On a typical Linux system, the shared version of the standard math library is /usr/lib/libm.so.

When a program uses a shared library, it is linked in such a way that it doesn’t contain function code itself, but references to shared code that will be made available at run time. When the resulting program is loaded into memory to be executed, the function references are resolved and calls are made to the shared library, which will be loaded into memory if needed.

In this way, the system can arrange for a single copy of a shared library to be used by many applications at once and stored just once on the disk. An additional benefit is that the shared library can be updated independently of the applications that rely on it. Symbolic links from the /usr/lib/libm.so file to the actual library revision (/usr/lib/libm.so.N where N represents a major version number—6 at the time of writing) are used. When Linux starts an application, it can take into account the version of a library required by the application to prevent major new versions of a library from breaking older applications.

For Linux systems, the program (the dynamic loader) that takes care of loading shared libraries and resolving client program function references is called ld.so and may be made available as ldlinux.so.2 or ld-lsb.so.1. The additional locations searched for shared libraries are configured in the file /etc/ld.so.conf, which needs to be processed by ldconfig if changed (for example, if X11 shared libraries are added when the X Window System is installed).

We can see which shared libraries are required by a program by running the utility ldd. For example, if we try running it on our example application, we get the following:

$ ldd program
libc.so.6 => /lib/libc.so.6 (0×4002a000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0×40000000)

In this case, we see that the standard C library (libc) is shared (.so). Our program requires major Version 6. Other UNIX systems will make similar arrangements for access to shared libraries. Refer to your system documentation for details.

In many ways, shared libraries are similar to dynamic-link libraries used under Windows. The .so libraries correspond to .DLL files and are required at run time, while the .sa libraries are similar to .LIB files included in the program executable.

Linux Tags: libraries, shared

The simplest form of library is just a collection of object files kept together in a ready-to-use form. When a program needs to use a function stored in the library, it includes a header file that declares the function. The compiler and linker take care of combining the program code and the library into a single executable program. We must use the –l option to indicate which libraries other than the standard C runtime library are required.

Static libraries, also known as archives, conventionally have names that end with .a. Examples are /usr/lib/libc.a and /usr/X11/lib/libX11.a for the standard C library and the X11 library, respectively.

We can create and maintain our own static libraries very easily by using the ar (for archive) program and compiling functions separately with gcc -c. We should try to keep functions in separate source files as much as possible. If functions need access to common data, we can place them in the same source file and use static variables declared in that file.

Linux Tags: archive, libraries, static

Libraries are collections of precompiled functions that have been written to be reusable. Typically, they consist of sets of related functions to perform a common task. Examples include libraries of screen-handling functions (the curses and ncurses libraries) and database access routines (the dbm library).

Standard system libraries are usually stored in /lib and /usr/lib. The C compiler (or more exactly, the linker) needs to be told which libraries to search, as by default it searches only the standard C library. This is a remnant of the days when computers were slow and CPU cycles were expensive. It’s not enough to put a library in the standard directory and hope that the compiler will find it; libraries need to follow a very specific naming convention and need to be mentioned on the command line.

A library filename always starts with lib. Then follows the part indicating what library this is (like c for the C library, or m for the mathematical library). The last part of the name starts with a dot ., and specifies the type of the library:

• .a for traditional, static libraries
• .so for shared libraries (see the following)

The libraries usually exist in both static and shared formats, as a quick ls /usr/lib will show. We can instruct the compiler to search a library either by giving it the full path name or by using the -l flag. For example,

$ gcc -o fred fred.c /usr/lib/libm.a

tells the compiler to compile file fred.c, call the resulting program file fred, and search the mathematical library in addition to the standard C library to resolve references to functions. A similar result is achieved with the following command:

$ gcc -o fred fred.c -lm

The -lm (no space between the l and the m) is shorthand (shorthand is much valued in UNIX circles) for the library called libm.a in one of the standard library directories (in this case /usr/lib). An additional advantage of the -lm notation is that the compiler will automatically choose the shared library when it exists.

Although libraries are usually found in standard places in the same way as header files, we can add to the search directories by using the -L (uppercase letter) flag to the compiler. For example,

$ gcc -o x11fred -L/usr/openwin/lib x11fred.c -lX11

will compile and link a program called x11fred using the version of the library libX11 found in the /usr/openwin/lib directory.

Linux Tags: function, library, precompiled, reusable

For programming in C and other languages, we need header files to provide definitions of constants and declarations for system and library function calls. For C, these are almost always located in /usr/include and subdirectories thereof. You can normally find header files that depend on the particular incarnation of Linux that you are running in /usr/include/sys and /usr/include/linux.

Other programming systems will also have include files that are stored in directories that get searched automatically by the appropriate compiler. Examples include /usr/include/X11 for the X Window System and /usr/include/g++ for GNU C++.

We can use include files in subdirectories or nonstandard places by specifying the -I flag to the C compiler.
For example,

$ gcc -I/usr/openwin/include fred.c

will direct the compiler to look in the directory /usr/openwin/include, as well as the standard places, for header files included in the fred.c program. Refer to the manual page for the C compiler (man gcc) for more details.

It’s often convenient to use the grep command to search header files for particular definitions and function prototypes. Suppose we need to know the name of the #defines used for returning the exit status from a program. Simply change to the /usr/include directory and grep for a probable part of the name like this:

$ grep EXIT_ *.h
…
stdlib.h:#define EXIT_FAILURE 1 /* Failing exit status. */
stdlib.h:#define EXIT_SUCCESS 0 /* Successful exit status. */
…
$

Here grep searches all the files in the directory with a name ending in .h for the string EXIT_. In this example, it has found (among others) the definition we need in the file stdlib.h

Linux Tags: development, header

Applications are usually kept in directories reserved for them. Applications supplied by the system for general use, including program development, are found in /usr/bin. Applications added by system administrators for a specific host computer or local network are often found in /usr/local/bin or /opt.

Administrators favor /usr/local, as it keeps vendor-supplied files and later additions separate from the applications supplied by the system. Keeping /usr organized in this way may help when the time comes to upgrade the operating system, since only /usr/local need be preserved. We recommend that you compile your applications to run and access required files from the /usr/local hierarchy.

Additional features and programming systems may have their own directory structures and program directories. Chief among these is the X Window System, which is commonly installed in the /usr/X11 directory. Alternative locations include /usr/X11R6 for Revision 6, also used by the XFree86 variant for Intel processors distributed by the XFree consortium as well as many Linux distributions. Other UNIXlike systems may choose different locations, such as /usr/openwin for Sun’s Open Windows provided with Solaris.

The GNU compiler system’s driver program, gcc (which we used in our programming example earlier in the chapter), is typically located in /usr/bin or /usr/local/bin, but it will run various compilersupport applications from another location. This location is specified when you compile the compiler itself and varies with the host computer type. For Linux systems, this location might be a version-specific subdirectory of /usr/lib/gcc-lib/. On one of the author’s machines at the time of writing it is /usr/lib/gcc-lib/i486-suse-linux/3.3/. The separate passes of the GNU C/C++ compiler, and GNU-specific header files, are stored here.

Linux Tags: application, development, directory, location, reserved

On POSIX-compliant systems, the C compiler is called c89. Historically, the C compiler was simply called cc. Over the years, different vendors have sold UNIX-like systems with C compilers with different facilities and options, but often still called cc.

When the POSIX standard was prepared, it was impossible to define a standard cc command with which all these vendors would be compatible. Instead, the committee decided to create a new standard command for the C compiler, c89. When this command is present, it will always take the same options, independent of the machine.

On Linux systems that do try to implement the standards, you might find that any or all of the commands c89, cc, and gcc refer to the system C compiler, usually the GNU C compiler, or gcc. On UNIX systems, the C compiler is almost always called cc.

We’ll be using gcc because it’s provided with Linux distributions and because it supports the ANSI standard syntax for C. If you ever find yourself using a UNIX system without gcc, we recommend that you obtain and install it. You can find it at http://www.gnu.org. Wherever we use gcc in the book, simply substitute the relevant command on your system.

Linux Tags: C, c89, cc, compiler, gcc, gnu, posix

Linux applications are represented by two special types of files: executables and scripts. Executable files are programs that can be run directly by the computer; they correspond to Windows .exe files. Scripts are collections of instructions for another program, an interpreter, to follow. These correspond to Windows .bat or .cmd files, or interpreted BASIC programs.

Linux doesn’t require executables or scripts to have a specific filename or any extension whatsoever. File system attributes are used to indicate that a file is a program that may be run. In Linux, we can replace scripts with compiled programs (and vice versa) without affecting other programs or the people who call them. In fact, at the user level, there is essentially no difference between the two.

When we log in to a Linux system, we interact with a shell program (often bash) that runs programs in the same way that the Windows command prompt does. It finds the programs we ask for by name by searching for a file with the same name in a given set of directories. The directories to search are stored in a shell variable, PATH, in much the same way as with Windows. The search path (to which we can add) is configured by your system administrator and will usually contain some standard places where system programs are stored. These include:

• /bin: Binaries, programs used in booting the system
• /usr/bin: User binaries, standard programs available to users
• /usr/local/bin: Local binaries, programs specific to an installation

An administrator’s login, such as root, may use a PATH variable that includes directories where system administration programs are kept, such as /sbin and /usr/sbin.

Optional operating system components and third-party applications may be installed in subdirectories of /opt, and installation programs might add to your PATH variable by way of user install scripts.

Note that Linux, like UNIX, uses the colon (:) character to separate entries in the PATH variable, rather than the semicolon (;) that MS-DOS and Windows use. (UNIX chose : first, so ask Microsoft why Windows is different, not why UNIX is different!) Here’s a sample PATH variable:

/usr/local/bin:/bin:/usr/bin:.:/home/neil/bin:/usr/X11R6/bin

Here the PATH variable contains entries for the standard program locations, the current directory (.), a user’s home directory, and the X Window System.

Linux Tags: application, binaries, binary, execution, path, programming, script