Network Programming with Perl
Perl has been called the glue that holds the Internet together because it is an extremely powerful text processing and CGI programming language. Although Perl was designed in the beginning to be a text manipulation language, it has evolved into a potent multi-purpose programming language. One area in which Perl has shown its power is that of network programming.
Perl makes network programming easy by providing built-in functions that can be used to create low-level client/server programs from scratch. Also, many modules are freely available that make programming common networking tasks simple and quick. These tasks include pinging remote machines, TELNET and FTP sessions. This article presents examples of each of these types of network programs.
Client/server network programming requires a server running on one machine to serve one or more clients running on either the same machine or different machines. These different machines can be located anywhere on the network.
To create a server, simply perform the following steps using the built-in Perl function indicated:
Create a socket with socket.
Bind the socket to a port address with bind.
Listen to the socket at the port address with listen.
Accept client connections with accept.
Establishing a client is even easier:
Create a socket with socket.
Connect (the socket) to the remote machine with connect.
Several other required functions and variables are defined in the Socket.pm module. This module is probably already installed on your machine, but if not, it is available at the Comprehensive Perl Archive Network (CPAN), the official Perl source code repository (see Resources). To use this module in our programs, the following statement is required at the top of the program:
use Socket;
This statement will locate the file Socket.pm and import all of its exported functions and variables.
All examples in this article use modules that are available at no cost from CPAN.
Perl modules are usually self-documenting. If the author of the module follows the generally accepted rules of creating a Perl module, they will add Plain Old Documentation (POD) to the module's .pm file. One way to view the POD for the Socket module (assuming Perl and Socket.pm were installed correctly) is to execute the following at the shell:
perldoc Socket
This command displays Socket.pm's POD converted to a man page. The output is a relatively thorough discussion of the functions and variables defined in this module.
Another way to view the documentation is to convert the POD to text using:
pod2text \ /usr/lib/perl5/i686-linux/5.00404/Socket.pm | more
The program pod2text is included in the Perl distribution, as are the programs pod2html, pod2man, pod2usage and pod2latex.
Our first programming example is a simple server running on one machine that can service only one client program at a time connecting from the same or a different machine. Recall that the steps for creating a server were to create a socket, bind it to a port, listen at the port and accept client connections.
Listing 1, server1.pl, is the source code for this simple server. First, it is generally a good idea to compile using Perl's strict rules:
use strict;
This requires all variables to be declared with the my function before they are used. Using my may be inconvenient, but it can catch many common syntactically correct yet logically incorrect programming bugs.
The variable $port is assigned the first command-line argument or port 7890 as the default. When choosing a port for your server, pick one that is unused on your machine. Note that the only way to ensure you select a port that does not have a predefined use is to look at the appropriate RFC (see Resources).
Next, the socket is created using the socket function. A socket is like a file handle—it can be read from, written to or both. The function setsockopt is called to ensure that the port will be immediately reusable.
The sockaddr_in function obtains a port on the server. The argument INADDR_ANY chooses one of the server's virtual IP addresses. You could instead decide to bind only one of the virtual IP addresses by replacing INADDR_ANY with
inet_aton("192.168.1.1")
or
gethostbyname("server.onsight.com")
The bind function binds the socket to the port, i.e., plugs the
socket into that port. Then, the listen function causes the server
to begin listening at the port. The second argument to the listen
function is the maximum queue length or the maximum number of
pending client connections. The value SOMAXCONN
is the maximum queue length for the machine being used.
Once the server begins listening at the port, it can accept client connections using the accept function. When the client is accepted, a new socket is created named CLIENT which can be used like a file handle. Reading from the socket reads the client's output and printing to the socket sends data to the client.
To read from a file handle or socket in Perl, wrap it in angle brackets (<FH>). To write to it, use the print function:
print SOCKET;
The return value of the accept function is the Internet address of the client in a packed format. The function sockaddr_in takes that format and returns the client's port number and the client's numeric Internet address in a packed format. The packed numeric Internet address can be converted to a text string representing the numeric IP using inet_ntoa (numeric to ASCII). To convert the packed numeric address to a host name, the function gethostbyaddr is used.
Let's assume all of the servers referred to in this article are started on the machine named server.onsight.com. To start the server on this machine, execute:
[james@server networking]$ server1.pl SERVER started on port 7890
The server is now listening at port 7890 on server.onsight.com, waiting for clients to connect.
Listing 2, client1.pl, shows a simple client. The first command-line argument to this program is the host name to which it should connect, which defaults to server.onsight.com. The second command-line argument is the port number which defaults to 7890.
The host name and the port number are used to generate the port address using inet_aton (ASCII to numeric) and sockaddr_in. A socket is then created using socket and the client connects the socket to the port address using connect.
The while loop then reads the data the server sends to the client until the end-of-file is reached, printing this input to STDOUT. Then the socket is closed.
Let's assume all of the clients are started on the the machine named client.avue.com, although they could be executed from any machine on the network. To execute the client, type:
[james@client networking]$ client1.pl server.onsight.com Hello from the server: Tue Oct 27 09:48:40 1998
The following is the standard output from the server:
got a connection from: client.avue.com [192.168.1.2]
Creating sockets using the functions described above is good when you want to control how the socket is created, the protocol to be used, etc. But using the functions above is too hard; I prefer the easy way—IO::Socket.
The module IO::Socket provides an easy way to create sockets which can then be used like file handles. If you don't have it installed on your machine, it can be found on CPAN. To see this module's POD, type:
perldoc IO::Socket
Listing 3, serverIO.pl, is a simple server using IO::Socket. A new IO::Socket::INET object is created using the new method. Note that the arguments to the method include the host name, port number, protocol, queue length and an option indicating we want this port to be immediately reusable. The new method returns a socket that is assigned to $sock. This socket can be used like a file handle—we can either read the client output from it, or write to it by sending data to the client.
A client connection is accepted using the accept method. Note the accept method returns the client socket when evaluated in scalar context:
$new_sock = $sock->accept()
and returns the client's socket and the client's IP address when evaluated in list context:
($new_sock, $client_addr) = $sock->accept()The client address is computed and printed the same as in Listing 1, server1.pl. Then the socket associated with that client is read until end-of-file. The data read is printed to STDOUT. This example illustrates that the server can read from a client using < > around the socket variable.
Listing 4, clientIO.pl, is a simple client using IO::Socket. This time, a new object is created that connects to a host at a port using the TCP protocol. Ten strings are then printed to that server, then the socket is closed.
If the server in Listing 3, serverIO.pl, is executed and then the client Listing 4, clientIO.pl, connects, the output would be:
[james@server networking]$ serverIO.pl
got a connection from: client.avue.com [192.168.1.2] hello, world: 1 hello, world: 2 hello, world: 3 hello, world: 4 hello, world: 5 hello, world: 6 hello, world: 7 hello, world: 8 hello, world: 9 hello, world: 10
It is possible to create servers and clients that communicate with one another in both directions. For instance, the client may send information to the server, then the server may send information back to the client. Therefore, network programs can be written so that the server and client follow some predetermined protocol.
Listing 5, server2way.pl, shows how a simple server can be created to read a command from a client, then print out an appropriate response to the client. The module Sys::Hostname provides a function named hostname that returns the host name of the server. To insure output is seen as we print, IO buffering is turned off for the STDOUT file handle using the autoflush function. Then a while loop is executed that accepts connections. When a client connects, the server reads a line from the client, chopping off the newline character. Then a switch statement is executed. (The switch is cleverly disguised as a foreach loop, which happens to be one of my favorite ways of writing a switch.) Depending on the input entered by the client, the server outputs an appropriate response. All lines from the client are read until end-of-file.
Listing 6, client2way.pl, shows the companion client. A connection to the server is made, then the client prints a few commands to the server reads the response and prints the response to STDOUT.
The following is the output of the client code in Listing 6:
[james@client networking]$ client2way.pl server.onsight.com Hi server.onsight.com Tue Oct 27 15:36:19 1998 DEFAULT
If you want to write a client that accepts commands from STDIN and sends them to the server, the easiest solution is to write a client that forks a child. (A solution can be written using select that does not fork, but it is more complicated.) The client's parent process will read the commands from the user through STDIN and print them to the server. The client's child process will then read from the server and print the responses to STDOUT.
Listing 7, clientfork.pl, is an example of a client that forks.
To fork in Perl, call the cleverly named fork function. It returns undef if the fork fails. If it succeeds, it returns 0 to the child, non-zero (the child's pid) to the parent. In clientfork.pl, an if statement checks the value of $kid, the return value from the fork. If $kid is true (non-zero, the child's pid), parent executes reading from STDIN printing to the server. If $kid is false (zero), the child executes reading from the server printing to STDOUT.
The following is the example session executing the client code in Listing 7, clientfork.pl connecting to the code in Listing 5, server2way.pl:
[james@client networking]$ clientfork.pl server.onsight.com NAME server.onsight.com DATE Tue Oct 27 15:42:58 1998 HELP DEFAULT HELLO Hi
When the parent process is finished reading from STDIN, it executes the kill function to kill the child process. It is very important the parent reap its child so that the child does not outlive the parent and become a zombie.
Servers usually don't handle only one client at a time. One approach to a server that can handle more than one client is a server that forks a child process to handle each client connection. Listing 8, serverfork.pl, is an example of a forking server.
One way for the parent process to reap its children is to define a subroutine and assign a reference to that subroutine to $SIG{CHLD}. (The hash %SIG is Perl's way of handling signals.) In this example, a subroutine named REAP is defined and a reference to this subroutine is assigned to $SIG{CHLD}. When the parent receives the CHLD (child terminated) signal, the REAP subroutine will be invoked.
Within the while loop that accepts all the client connections, the server forks. If the fork returns true, the parent is running and it executes the next statement which immediately transfers control to the continue block, performs the housecleaning step of closing the child socket and waits for the next client to connect. If the fork returns undef, then the fork failed, so the server dies. If the fork returns neither true nor undef, then the child is running, so the parent socket is closed and the child reads from the client and processes the client. When the child is finished processing the client, the child exits and is reaped by the parent.
Perl version 5.005 supports thread programming. This means a threaded networking program can be created to be either a server or a client.
Listings 9, 10, and 11 are three different versions of a client that logs into several web servers and determines the type of server being used (Apache, Netscape, etc).
Listing 9, getservertype1.pl, shows a non-forking, non-threaded client. First, an array of hosts is created and initialized to a few web sites. The subroutine doit is defined to receive the web server name as an argument, open a client connection to that server at port 80 (the HTTP port), send the HTTP request and read each line of response. When the line starting Server: is read, it will extract the server name and store it in $1. Then the host name and web server name are printed. This subroutine is called for each host in the array of hosts.
Here is the output of getservertype1.pl:
processing www.ssc.com... www.ssc.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 processing www.linuxjournal.com... www.linuxjournal.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 processing www.perl.com... www.perl.com: Apache/1.2.6 mod_perl/1.11 processing www.perl.org... www.perl.org: Apache/1.2.5 processing www.nytimes.com... www.nytimes.com: Netscape-Enterprise/2.01 processing www.onsight.com... www.onsight.com: Netscape-Communications/1.12 processing www.avue.com... www.avue.com: Netscape-Communications/1.12
Note that the hosts are processed in the same order as stored in @hosts.
Listing 10, getservertype2.pl, is a forking version of getservertype1.pl. The forking occurs within the foreach loop. The fork is executed and if it returns true, the parent then executes the next statement to the next host name. If the fork returns undef, then the program dies. Otherwise, the child calls the doit function passing in the host, then exits. After the parent completes its work in the while loop, it waits for all child processes to finish, then exits.
Here is the output of getservertype2.pl:
processing www.ssc.com... processing www.linuxjournal.com... processing www.perl.com... processing www.perl.org... processing www.nytimes.com... processing www.onsight.com... processing www.avue.com... www.onsight.com: Netscape-Communications/1.12 www.nytimes.com: Netscape-Enterprise/2.01 www.avue.com: Netscape-Communications/1.12 www.linuxjournal.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.com: Apache/1.2.6 mod_perl/1.11 www.ssc.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.org: Apache/1.2.5 Parent exiting...
Note that the hosts are not printed in the order stored in @hosts. They are printed in the order processed, the slower hosts taking longer than the faster ones.
Listing 11, getservertype3.pl, is a threaded version. In the loop through the host names, a new Thread object is created. When creating the Thread, the new method is passed a reference to a subroutine that the thread will execute, as well as the arguments passed into that subroutine. The thread then executes its subroutine and when the subroutine returns, the thread is destroyed. Here is the output of getservertype3.pl:
processing www.ssc.com... processing www.linuxjournal.com... processing www.perl.com... processing www.perl.org... processing www.nytimes.com... processing www.onsight.com... processing www.avue.com... www.nytimes.com: Netscape-Enterprise/2.01 www.onsight.com: Netscape-Communications/1.12 www.avue.com: Netscape-Communications/1.12 www.linuxjournal.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.com: Apache/1.2.6 mod_perl/1.11 www.ssc.com: Stronghold/2.2 Apache/1.2.5 PHP/FI-2.0b12 www.perl.org: Apache/1.2.5
The Net::Ping module makes pinging hosts easy. Listing 12, ping.pl, is a program similar to a program on my server that pings my ISP to keep my connection alive. First, a new Net::Ping object is created. The protocol chosen is tcp (the choices are tcp, udp and icmp; the default is udp). The second argument is the timeout (two seconds). Then an infinite loop is executed, pinging the desired host. The ping() method returns true if the host responds, false otherwise, and an appropriate message is printed. Then the program sleeps ten seconds and pings again.
An example output of Listing 12, ping.pl, is:
Success: Wed Nov 4 14:47:58 1998 Success: Wed Nov 4 14:48:08 1998 Success: Wed Nov 4 14:48:18 1998 Success: Wed Nov 4 14:48:28 1998 Success: Wed Nov 4 14:48:38 1998 Success: Wed Nov 4 14:48:48 1998
The Net::Telnet module makes automating TELNET sessions easy. Listing 13, telnet.pl, is an example of connecting to a machine, sending a few system commands and printing the result.
First, a server and a user name are used. The user name defaults to the user running the script by assigning to $user the value $ENV{USER}. (The hash %ENV contains all of the environment variables the script inherits from the shell.)
Next, the password is requested, then read in. Note that turning off the stty echoing is done through a system call. It can also be done using the Term::ReadKey module.
Then, a Net::Telnet object is created. To log in to the server using this object, the login method is called. Several system commands are executed using the cmd method which returns the STDOUT of the system command which is then printed. Note that part of that output is the system prompt, which is printed along with the output of the command.
Also note that the code $tn->cmd('/usr/bin/who') is evaluated in list context and stored in @who, which is an array that contains all the lines of ouptut of that command, one line of output per array element.
After all of the system commands are executed, the TELNET session is closed.
Here is an example output of Listing 13, telnet.pl:
Enter password:
Hostname: server.onsight.com [james@server james] Here's who: james tty1 Oct 24 21:07 james ttyp1 Oct 27 20:59 (:0.0) james ttyp2 Oct 24 21:11 (:0.0) james ttyp6 Oct 28 07:16 (:0.0) james ttyp8 Oct 28 19:02 (:0.0) [james@server james] What is your command: date Thu Oct 29 14:39:57 EST 1998 [james@server james]
The Net::FTP module makes automating FTP sessions easy. Listing 14, ftp.pl, is an example of connecting and getting a file.
A Net::FTP object is created, the login is called to log in to the machine, the cwd changes the working directory and the get method gets the file. Then the session is terminated with quit.
There are methods to do many common FTP operations: put, binary, rename, delete, etc. To see a list of all the available methods, type:
perldoc Net::FTP
Here is an example output of Listing 14, ftp.pl:
[james@k2 networking]$ ftp.pl server.onsight.com Enter your password: Before ---------------------------------------- /bin/ls: *.gz: No such file or directory ---------------------------------------- After ---------------------------------------- perl5.005_51.tar.gz ----------------------------------------
Using both Net::Telnet and Net::FTP, a very simple script can be created that can archive a directory structure on a remote machine.
Listing 15, taritup.pl, is a Perl program that is similar to a program I use that logs in to my ISP and archives my web site.
The steps this program follows are:
Start a session on the remote machine with TELNET.
Go to the web page directory using cd.
Archive the directory using tar.
Start an FTP session to the remote machine.
Change to the directory containing the tar file.
Get the tar file.
Quit the FTP session.
Back in the TELNET session, delete the tar file on the remote machine.
Close the TELNET session.
This program outputs text to let the user know how the script is progressing.
Perl is a powerful, easy-to-use programming language. That power and ease of use includes network programming due to many built-in functions and modules. Best of all, it's free.
