This chapter describes the process of compiling and linking a program.

• Compiling Programs on HP-UX: An Example Provides an overview of compiling on HP-UX.
• Linking Programs on HP-UX Describes how ld creates an executable file from one or more object files.
• Linking with Libraries Describes conventions for using libraries with ld.
• Running the Program Describes the process of loading and binding programs at run time.

• Linker Thread-Safe Features Describes the thread-safe features.

To create an executable program, you compile a source file containing a main program. For example, to compile an ANSI C program named sumnum.c, shown below, use this command (-Aa says to compile in ANSI mode):

$ cc -Aa sumnum.c 

The compiler displays status, warning, and error messages to standard error output (stderr). If no errors occur, the compiler creates an executable file named a.out in the current working directory. If your PATH environment variable includes the current working directory, you can run a.out as follows:

$ a.out
Enter a number: 4
Sum 1 to 4: 10

The process is essentially the same for all HP-UX compilers. For instance, to compile and run a similar FORTRAN program named sumnum.f:

$ f90 sumnum.f      
//Compile and link sumnum.f. 
    
...                      //The compiler displays any messages here.
$ a.out 
                    
//Run the program.
   ... 
          
//Output from the program is displayed here.

Program source can also be divided among separate files. For example, sumnum.c could be divided into two files: main.c, containing the main program, and func.c, containing the function sum_n. The command for compiling the two together is:

$ cc -Aa main.c func.c
main.c:
func.c:

Notice that cc displays the name of each source file it compiles. This way, if errors occur, you know where they occur.

#include <stdio.h> 
             /* contains standard I/O defs */
int     
sum_n( int n ) 
          
/* sum numbers from n to 1 
   */
{
  int 
  
sum = 0; 
                
/* running total; initially 0 */
  
for (; n >= 1; n--) 
           
/* sum from n to 1 
           
*/
    
sum += n; 
                  
/* add n to sum 
              
*/
  
return sum; 
                  
/* return the value of sum 
   */
}
 
main()                          
/* begin main program 
        
*/
{
  int 
  n; 
                     
/* number to input from user 
 */
  
printf("Enter a number: "); 
  
/* prompt for number 
         
*/
  
scanf("%d", &n); 
             
/* read the number into n 
    */
  
printf("Sum 1 to %d: %d\\n", n, sum_n(n)); /* display the sum */
}

Generally speaking, the compiler reads one or more source files, one of which contains a main program, and outputs an executable a.out file, as shown in Figure 1: High-Level View of the Compiler .

Figure 1: High-level View of the Compiler

On the surface, it appears as though an HP-UX compiler generates an a.out file by itself. Actually, an HP-UX compiler is a driver that calls other commands to create the a.out file. The driver performs different tasks (or phases) for different languages, but two phases are common to all languages:

1. For each source file, the driver calls the language compiler to create an object file. (See also What is an Object File?.)

2. Then, the driver calls the HP-UX linker (ld) which builds an a.out file from the object files. This is known as the link-edit phase of compilation. (See also Compiler-Linker Interaction .)

Figure 2: Looking "inside" a Compiler summarizes how a compiler driver works.

Figure 2: Looking "Inside" a Compiler

The C, aC++, and Fortran90 compilers provide the -v (verbose) option to display the phases a compiler is performing. Compiling main.c and func.c with the -v option produced this output (\ at the end of a line indicates the line is continued to the next line).

An object file is basically a file containing machine language instructions and data in a form that the linker can use to create an executable program. Each routine or data item defined in an object file has a corresponding symbol name by which it is referenced. A symbol generated for a routine or data definition can be either a local definition or global definition. Any reference to a symbol outside the object file is known as an external reference.

To keep track of where all the symbols and external references occur, an object file has a symbol table. The linker uses the symbol tables of all input object files to match external references to global definitions.

A local definition is a definition of a routine or data that is accessible only within the object file in which it is defined. Such a definition cannot be accessed from another object file. Local definitions are used primarily by debuggers, such as adb. More important for this discussion are the global definitions and external references.

A global definition is a definition of a procedure, function, or data item that can be accessed by code in another object file. For example, the C compiler generates global definitions for all variable and function definitions that are not static. The FORTRAN compiler generates global definitions for subroutines and common blocks. In Pascal, global definitions are generated for external procedures, external variables, and global data areas for each module.

An external reference is an attempt by code in one object file to access a global definition in another object file. A compiler cannot resolve external references because it works on only one source file at a time. Therefore, the compiler simply places external references in an object file's symbol table; the matching of external references to global definitions is left to the linker or loader.

As described in Looking "inside" a Compiler , the compilers automatically call the linker to create an executable file. To see how the compilers call ld, run the compiler with the -v (verbose) option.

For example, compiling a C program to produce an IPF 32-bit share-bound application, produces the output below:

$ cc -v main.c func.c -lm
main.c:
 /opt/ansic/lbin/ecom -ia64abi all -architecture 32 -ext on -lang c \  
  
-exception off -sysdir /usr/include - inline_power 1 -link_type dynamic \
  
-fpeval float - tls_dyn on -target_os 11.23 --sys_include /usr/include \  
  
-D__hpux -D__unix -D__ia64=1 -D_BIG_ENDIAN=1 -D_ILP32 -D__HP_cc=60200 \
  
-D__STDC_EXT__ -D_HPUX_SOURCE -D_INCLUDE_LONGLONG -D_INLINE_ASM \  
  
-D_BIND_LIBCALLS -D_Math_errhandling=MATH_ERREXCEPT -D_FLT_EVAL_METHOD=O \  
  
-ucode hdriver=optlevel%1% -plusolistoption -O106const! -plusolistoption \
  
-O113moderate! -plusooption -Oq01,al,ag,cn,sz,ic,vo,Mf,Po,es,rs,Rf,Pr,sp,\
in,cl,om,vc,pi,fa,pe,rr,pa,pv,nf,cp,1x,Pg,ug,1u,lb,uj,dn,sg,pt,kt,em,np,ar,\
rp,dl,fs,bp,wp,pc,mp,1r,cx,cr,pi,so,Rc,fa,ft,fe,ap,st,lc,Bl,sr,ib,pl,sd,ll,\
 rl,dl,Lt,ol,fl,lm,ts,rd,dp,If! main.c
func.c:
 /opt/ansic/lbin/ecom -ia64abi all -architecture 32 -ext on -lang c \  
  
-exception off -sysdir /usr/include - inline_power 1 -link_type dynamic \
  
-fpeval float - tls_dyn on -target_os 11.23 --sys_include /usr/include \  
  
-D__hpux -D__unix -D__ia64=1 -D_BIG_ENDIAN=1 -D_ILP32 -D__HP_cc=60200 \
  
-D__STDC_EXT__ -D_HPUX_SOURCE -D_INCLUDE_LONGLONG -D_INLINE_ASM \  
  
-D_BIND_LIBCALLS -D_Math_errhandling=MATH_ERREXCEPT -D_FLT_EVAL_METHOD=O \  
  
-ucode hdriver=optlevel%1% -plusolistoption -O106const! -plusolistoption \
  
-O113moderate! -plusooption -Oq01,al,ag,cn,sz,ic,vo,Mf,Po,es,rs,Rf,Pr,sp,\
in,cl,om,vc,pi,fa,pe,rr,pa,pv,nf,cp,1x,Pg,ug,1u,lb,uj,dn,sg,pt,kt,em,np,ar,\
rp,dl,fs,bp,wp,pc,mp,1r,cx,cr,pi,so,Rc,fa,ft,fe,ap,st,lc,Bl,sr,ib,pl,sd,ll,\
 rl,dl,Lt,ol,fl,lm,ts,rd,dp,If! func.c
LPATH=/usr/lib/hpux32:/opt/langtools/lib/hpux32 
 /usr/ccs/bin/ld -o a.out -u__exit -umain main.o func.o -lm -lc 
 
removing /var/tmp/AAAa02486

This example shows that the cc driver calls the actual C compiler (/opt/ansic/lbin/ecom) for each source file. Then the driver calls the linker (/usr/ccs/bin/ld) on the object files created by the compiler (main.o and func.o).

The next-to-last line in the above example is the command line that the compiler used to invoke the linker, /usr/ccs/bin/ld. When building a share-bound executable, the startup functions are handled by the dynamic loader dld. By default, the dynamic loader is found in /usr/lib/hpux32. Thus, in most cases, the ld command does not include crtO.o. In the ld command line, ld combines the two object files created by the compiler (main.o and func.o). It also searches the libm (-lm) and libc (-lc) libraries.

On PA-RISC systems, the same compile command invokes the PA32 linker. The output is shown below.

$ cc -Aa -v main.c func.c -lm
cc: CCOPTS is not set.
main.c:
/opt/langtools/lbin/cpp.ansi main.c /var/tmp/ctmAAAa10102 \
  
-D__hp9000s700 -D__hp9000s800 -D__hppa -D__hpux \
  
-D__unix -D_PA_RISC1_1
cc: Entering Preprocessor.
/opt/ansic/lbin/ccom /var/tmp/ctmAAAa10102 main.o -O0 -Aa
func.c:
/opt/langtools/lbin/cpp.ansi func.c /var/tmp/ctmAAAa10102 \
  
-D__hp9000x700 -D__hp9000s800 -D__hppa -D__hpux \
  
-D__unix -D_PA_RISC1_1
cc: Entering Preprocessor.
/opt/ansic/lbin/ccom /var/tmp/ctmAAAa10102 func.o -O0 -Aa
cc: LPATH is /usr/lib/pa1.1:/usr/lib:/opt/langtools/lib:
/usr/ccs/bin/ld /opt/langtools/lib/crt0.o -u main main.o func.o -lm -lc
cc: Entering Link editor.

The next-to-last line in the above example is the command line the compiler used to invoke the PA32 mode linker, /usr/ccs/bin/ld. In this command, ld combines a startup file (crt0.o) and the two object files created by the compiler (main.o and func.o) Also, ld searches the libm and libc libraries.

For PA64 and IPF executables, the startup functions are handled by the dynamic loader. In most cases, the ld command line does not include crt0.o.

The HP-UX linker, ld, produces a single executable file from one or more input object files and libraries. In doing so, it matches external references to global definitions contained in other object files or libraries. It revises code and data to reflect new addresses. This process is known as relocation. If the input files contain debugger information, ld updates this information appropriately. The linker places the resulting executable code in a file named, by default, a.out.

In the C program example, (see Compiling Programs on HP-UX: An Example ) main.o contains an external reference to sum_n, which has a global definition in func.o. 

The linker (ld) matches the external reference to the global definition, allowing the main program code in a.out to access sum_n (see Figure 3: Matching the External Reference to sum_n ).

Figure 3: Matching the External Reference to sum_n

If the linker (ld) cannot match an external reference to a global definition, it displays a message to standard error output. If, for instance, you compile main.c without func.c, the linker (ld) cannot match the external reference to sum_n and displays this output:

$ cc -Aa main.c
ld: Unsatisfied symbol "func1" in file main.o
1 errors.

Notice that in the PA32 example in Compiler-Linker Interaction the first object file on the linker command line is /opt/langtools/lib/crt0.o, even though this file was not specified on the compiler command line. This file, known as a startup file, contains the program's entry point. The program's entry point is is the location at which the program starts running after HP-UX loads it into memory to begin execution. The startup code does such things as retrieving command line arguments into the program at run time, and activating the dynamic loader to load any required shared libraries. In the C language, it also calls the routine _start in libc which, in turn, calls the main program as a function.

The linker uses four startup files:

• 32-bit PA is /opt/langtools/lib/crt0.o

• 64-bit PA is /opt/langtools/lib/pa_64/crt0.o
  The linker uses this startup file when it is in compatibility mode (+compat) or it is in default standard mode (+std) with the -noshared option.

• 32-bit IPF is /opt/langtools/lib/hpux32/crt0.o
  The linker uses this startup file when it is in default standard mode (+std) with the -noshared option.

• 64-bit IPF is /opt/langtools/lib/hpux64/crt0.o
  The linker uses this startup file when it is in default standard mode (+std) with the -noshared option.

If the -p profiling option is specified on the compile line, the compilers link with -L /usr/ccs/lib/libp -lprof. If the -G profiling option is specified, the compilers link with /usr/ccs/lib/lip -lgprof.

PA-RISC ONLY: If the linker option -I is specified to create an executable file with profile-based optimization, in 32-bit mode icrt0.o is used, and in 64-bit mode the linker inserts /usr/ccs/lib/pa20_64/fdp_init.o. If the linker options -I and -b are specified to create a shared library with profile-based optimization, in 32-bit mode scrt0.o is used, and in 64-bit mode, the linker inserts /usr/ccs/lib/pa20_64/fdp_init-sl.o. In 64-bit mode, the linker uses the single 64-bit crt0.o to support these options.

For details on startup files, see crt0(3).

For archive-bound (using the -complete compiler option or the -noshared linker option) executables, the entry point is the location at which execution begins in the a.out file. The entry point is defined by the symbol $START$ in crt0.o. In share-bound executables, the entry point is defined by the symbol $START$ in the dynamic loader (dld.so).

The information contained in the resulting a.out file depends on which architecture the file was created on and what options were used to link the program. In any case, an executable a.out file contains information that HP-UX needs when loading and running the file. For example: Is it a shared executable? Does it reference shared libraries? Is it demand-loadable? Where do the text (code), data, and bss (uninitialized data) segments reside in the file? For details on the format of this file, see a.out(4).

In 32-bit mode, the linker records a magic number with each executable program that determines how the program should be loaded. There are three possible values for an executable file's magic number:

SHARE_MAGIC

The program's text (code) can be shared by processes; its data cannot be shared. The first process to run the program loads the entire program into virtual memory. If the program is already loaded by another process, then a process shares the program text with the other process.

DEMAND_MAGIC

As with SHARE_MAGIC the program's text is shareable but its data is not. However, the program's text is loaded only as needed - that is, only as the pages are accessed. This improves process startup time since the entire program does not need to be loaded; however, it can degrade performance throughout execution.

EXEC_MAGIC

Neither the program's text nor data is shareable. In other words, the program is an unshared executable. Usually, it is not desirable to create such unshared executables because they place greater demands on memory resources.

By default, the linker creates executables whose magic number is SHARE_MAGIC. Table 1: 32-bit Mode Magic Number Linker Options shows which linker option to use to specifically set the magic number.

Table 1: 32-bit Mode Magic Number Linker Options

An executable file's magic number can also be changed using the chatr command (see Changing a Program's Attributes with chatr(1) ). However, chatr can only toggle between SHARE_MAGIC and DEMAND_MAGIC; it cannot be used to change from or to EXEC_MAGIC. This is because the file format of SHARE_MAGIC and DEMAND_MAGIC is exactly the same, while EXEC_MAGIC files have a different format. For details on magic numbers, refer to magic(4).

In 64-bit mode, the linker sets the magic number to the predefined type for ELF object files (\177ELF). The value of the e_type field in the ELF object file header specifies how the file should be loaded.

If linker errors do not occur, the linker gives the a.out file read/write/execute permissions to all users (owner, group, and other). If errors occur, the linker gives read/write permissions to all users. Permissions are further modified if the umask is set (see umask(1)). For example, on a system with umask set to 022, a successful link produces an a.out file with read/write/execute permissions for the owner, and read/execute permissions for group and others:

$ umask
022
$ ls -l a.out
-rwxr-xr-x   
1 michael 
 users 
     
74440 Apr 
 
4 14:38 a.out

In addition to matching external references to global definitions in object files, the linker (ld) matches external references to global definitions in libraries. A library is a file containing object code for subroutines and data that can be used by other programs. For example, the standard C library, libc, contains object code for functions that can be used by C, C++, FORTRAN, and Pascal programs to do input, output, and other standard operations.

By convention, library names have the form:

libname. suffix

name

is a string of one or more characters that identifies the library.

suffix

is .a if the library is an archive library, .sl if the library is a PA-RISC shared library, or .so if the library is a shared library. (The suffix can be .sl for an IPF shared library. This naming convention is not recommended, but it is supported for backwards compatibility with PA-32 and PA-64.)

Typically, library names are referred to without the suffix. For instance, the standard C library is referred to as libc.

A compiler driver automatically specifies certain default libraries when it invokes ld. For example, cc automatically links in the standard library libc, as shown by the -lc option to ld in this example:

$ cc -Aa -v main.c func.c
    ...
/usr/ccs/bin/ld -o a.out -u__exit -umain main.o func.o -lc
cc: informational note 413: Entering Link editor.

Similarly, the Series 700/800 Fortran90 compiler automatically links with the libcl (C interface), libisamstub (ISAM file I/O), and libc libraries:

$ f90 -v sumnum.f
   ...
/usr/ccs/bin/ld -x /opt/langtools/lib/crt0.o \
 sumnum.o -lcl -lisamstub -lc

By default, ld searches for libraries in the directory /usr/lib/hpux32 for 32-bit executables and /usr/lib/hpux64 for 64-bit executables.

If the -p or -G compiler profiling option is specified on the command line, the compiler directs the linker to also search
/usr/lib/libp (for PA32)
/usr/lib/pa20__64/libp (for PA64)
/usr/lib/hpux32/libp (for IPF 32-bit)
/usr/lib/hpux64/libp (for IPF 64-bit)

The default order can be overridden with the LPATH environment variable, the -L linker option, specifies $ORIGIN in the library path, or the +origin option. These are described in Changing the Default Library Search Path with -L, LPATH, and $ORIGIN .

The linker searches libraries in the order in which they are specified on the command line - the link order. Link order is important in that a library containing an external reference to another library must precede the library containing the definition. This is why libc is typically the last library specified on the linker command line: because the other libraries preceding it in the link order often contain references to libc routines and so must precede it.

If multiple definitions of a symbol occur in the specified libraries, ld does not necessarily choose the first definition. It depends on whether the program is linked with archive libraries, shared libraries, or a combination of both. Depending on link order to resolve such library definition conflicts is risky because it relies on undocumented linker behavior that may change in future releases. (See also Caution When Mixing Shared and Archive Libraries .)

An executable file is created after the program has been compiled and linked. The next step is to run or load the program.

When you run an executable file created by ld, the program is loaded into memory by the HP-UX program loader, exec. This routine is actually a system call and can be called by other programs to load a new program into the current process space. The exec function performs many tasks; some of the more important ones are:

• Determining how to load the executable file by looking at its magic number. (See also The a.out File .)

• Determining where to begin execution of the program - that is, the entry point - For examples in share-bound executables in dld.so, in archive-bound executables in crt0.o. (See also The crt0.o Startup File.)

• When the program uses shared libraries, the crt0.o startup code invokes the dynamic loader, which in turn attaches any required shared libraries. If immediate binding was specified at link time, then the libraries are bound immediately. If deferred binding was specified, then libraries are bound as they are referenced. (See also What are Shared Libraries?.)

For details on exec, see the exec(2) page in the HP-UX Reference.

Since shared library routines and data are not actually contained in the a.out file, the dynamic loader must attach the routines and data to the program at run time. Attaching a shared library entails mapping the shared library code and data into the process's address space, relocating any pointers in the shared library data that depend on actual virtual addresses, allocating the bss segment, and binding routines and data in the shared library to the program.

The dynamic loader binds only those symbols that are reachable during the execution of the program. This is similar to how archive libraries are treated by the linker; namely, ld pulls in an object file from an archive library only if the object file is needed for program execution.

To accelerate program startup time, routines in a shared library are not bound until referenced. (Data items are always bound at program startup.) This deferred binding of shared library routines distributes the overhead of binding across the execution time of the program and is especially expedient for programs that contain many references that are not likely to be executed. In essence, deferred binding is similar to demand-loading.

The dynamic loader (dld) and its application interface library (libdl) are thread-safe.

Also, the linker toolset provides thread local storage support in:

• ld - the link editor

• crt0.o - the program startup file

Thread local storage (also called thread-specific data) is data specific to a thread. Each thread has its own copy of the data item.

Use of the __thread keyword in a shared library prevents that shared library from being dynamically loaded, that is, loaded by an explicit call to shl_load().

For More Information:

• See your HP compiler documentation to learn how to create thread local storage data items with the _thread compiler directive.

• See Programming with Threads on HP-UX for information on threads.

The 32-bit dynamic loader serializes all calls to dlopen, dlclose, shl_load, and shl_unload using a pthread mutex lock. A thread executing dlopen holds this lock, and another thread that wants to execute dlclose (for example), even on a different shared library, must wait for the dlopen in the first thread to release the lock after it is done.

Developers, who use these routines in multi-threaded applications (or shared libraries) and also use mutex locks for synchronizing threads should take note of this behavior to avoid deadlock situations. One example of such a situation is when a shared library initializer creates a new thread that calls dlopen - this would invariably lead to a deadlock in a multi-threaded application. A less trivial example could be as follows. Suppose thread A of an application calls dlopen to load a shared library, libA. Now this shared library has an initializer initA that wants to obtain a mutex lock M. But M is currently held by thread B of the application. Further, thread B wants to load another library, libB and then release the mutex lock M. Now thread B would call either dlopen or shl_load, either of which waits for the dld mutex lock to be released by the dlopen called by thread A. In turn, thread A waits for mutex lock M to be released before it can finish dlopen and release the dld mutex lock. This results in a deadlock.