Manipulating Disk Quotas (Perl for System Administration)

Manipulating Disk Quotas (Perl for System Administration) Book Home

2.5. Manipulating Disk Quotas

Perl scripts like our core-killers from the last section can offer a way to deal with junk files that cause unnecessary disk full situations. But even when run on a regular basis, they are still a reactive approach; the administrator deals with these files only after they've come into existence and cluttered the filesystem.

There's another, more proactive approach: filesystem quotas. Filesystem quotas, operating system permitting, allow you to constrain the amount of disk space a particular user can consume on a filesystem. Windows 2000 and all modern Unix variants offer quotas. NT4 requires a third-party product, and MacOS users are S.O.L. (Simply or Sore Out of Luck).

Though proactive, this approach is considerably more heavy-handed than cleanup scripts because it applies to all files, not just spurious ones like core dumps. Most system administrators find using a combination of the automated cleanup scripts and quotas to be the best strategy. The former helps prevent the latter from being necessary.

In this section, we'll deal with manipulating Unix quotas from Perl. Before we get to that subject, we should take a moment to understand how quotas are set and queried "by hand." To enable quotas on a filesystem, a Unix system administrator usually adds an entry to the filesystem mount table (e.g., /etc/fstab or /etc/vfstab) and then reboots the system or manually invokes the quota enable command (usually quotaon). Here's an example /etc/vfstab from a Solaris box:

#device            device              mount  FS    fsck    mount     mount
#to mount          to fsck             point  type  pass    at boot   options
/dev/dsk/c0t0d0s7 /dev/dsk/c0d0t0d0s7  /home  ufs   2       yes       rq

The rq option in the last column enables quotas on this filesystem. They are stored on a per-user basis. To view the quota entries for a user on all of the mounted filesystems that have quotas enabled, one can invoke the quota command like so:

$ quota -v sabrams

to produce output similar to this:

Disk quotas for sabrams (uid 670):
Filesystem   usage  quota    limit    timeleft  files  quota  limit timeleft
/home/users  228731 250000  253000                  0      0      0

For our next few examples, we're only interested in the first three columns of this output. The first number is the current amount of disk space being used by the user sabrams on the filesystem mounted at/home/users. The second is that user's "soft quota." The soft quota is the amount after which the OS begins complaining for a set period of time, but does not restrict space allocation. The final number is the "hard quota," the absolute upper bound for this user's space usage. If a program attempts to request more storage space on behalf of the user after this limit has been reached, the OS will deny this request and return an error message like disk quota exceeded.

If we wanted to change these quota limits by hand, we'd use the edquota command. edquota pops you into your editor of choice preloaded with a small temporary text file that contains the pertinent quota information. Setting the EDITOR environment variable in your shell specifies the editor. Here's an example buffer that shows a user's limits on each of the four quota-enabled filesystems. This user most likely has her home directory on /exprt/server2 since that's the only filesystem where she has quotas in place:

fs /exprt/server1 blocks (soft = 0, hard = 0) inodes (soft = 0, hard = 0)
fs /exprt/server2 blocks (soft = 250000, hard = 253000) inodes (soft = 0, hard = 0)
fs /exprt/server3 blocks (soft = 0, hard = 0) inodes (soft = 0, hard = 0)
fs /exprt/server4 blocks (soft = 0, hard = 0) inodes (soft = 0, hard = 0)

Using edquota by hand may be a comfy way to edit a single user's quota limits, but it is not a viable way to deal with tens, hundreds, or thousands of user accounts. One of Unix's flaws is its lack of command-line tools for editing quota entries. Most Unix variants have C library routines for this task, but no command- line tools that allow for higher-level scripting. True to the Perl motto "There's More Than One Way To Do It" (TMTOWTDI, pronounced "tim-toady"), we are going to look at two very different ways of setting quotas from Perl.

2.5.1. Editing Quotas with edquota Trickery

The first method involves a little trickery on our part. A moment ago we mentioned the process for manually setting a user's quota: edquota invokes an editor to allow a user to edit a small text file and then uses any changes to update the quota entries. There's nothing in this scenario mandating that an actual human has to type at a keyboard to make changes in the editor invoked by edquota. In fact, there's not even a constraint on which editor has to be used. All edquota needs is a program it can launch that will properly change a small text file. Any valid path (as specified in the EDITOR environment variable) to such a program will do. Why not point edquota at a Perl script? Let's look at just such a script for our next example.

Our example script will need to do double duty: first, it has to get some command-line arguments from the user, set EDITOR appropriately, and call edquota. edquota will then run another copy of our program to do the real work of editing this temporary file. Figure 2-1 shows a diagram of the action.

Figure 2.1. Changing quotas using a "sleight-of-hand" approach

The second copy must be told what to change by the initial program invocation. How it gets this information from the copy that called edquota is less straightforward than one might hope. The manual page for edquota says: "The editor invoked is vi(1) unless the EDITOR environment variable specifies otherwise." The idea of passing command-line arguments via EDITOR or another environment variable is a dicey prospect at best because we don't know how edquota will react. Instead, we'll have to rely on one of the other types of interprocess communication methods available from Perl. For instance, the two processes could:

Pass a temporary file between them
Create a named pipe and talk over that
Pass AppleEvents (under MacOS)
Use mutexes or mutually agreed upon registry keys (under NT/2000)
Rendezvous at network socket
Use a shared memory section

And so on. It's up to you as the programmer to choose the appropriate communication method, though often the data will dictate this for you. When looking at this data, you'll want to consider:

Direction of communication (one- or two-way?)
Frequency of communication (is this a single message or are there multiple chunks of information that need to be passed?)
Size of data (is it a 10MB file or 20 characters?)
Format of data (is it a binary file or just text characters, fixed width, or character separated?)

Finally, be conscious of how complicated you want to make your script.

In our case, we're going to choose a simple but powerful method to exchange information. Since the first process only has to provide the second one with a single set of change instructions (what quotas need to be changed and their new values), we're going to set up a standard Unix pipe between the two of them.[1] The first process will print a change request to its output and the copy spawned by edquota will read this info as its standard input.

[1]Actually, the pipe will be to the edquotaprogram, which is kind enough to hook up its input and output streams to the Perl script being spawned.

Let's write the program. The first thing the program has to do when it starts up is decide what role it's been asked to play. We can assume that the first invocation receives several command-line arguments (i.e., what to change) while the second, called by edquota, receives only one (i.e., the name of the temporary file). The program forces a set of command flags to be present if it is called with more than one argument, so we're pretty safe in using this assumption as the basis of our role selection. Here's the role selection code:

$edquota = "/usr/etc/edquota";     # edquota path
$autoedq = "/usr/adm/autoedquota"; # full path for this script

# are we the first or second invocation?

# if there is more than one argument, we're the first invocation
if ($#ARGV > 0) { 
    &ParseArgs;
    &CallEdquota;
}
# else - we're the second invocation and will have to perform the edits
else {
    &EdQuota(  );
}

Let's look at the code called by the first invocation to parse arguments and call edquota over a pipe:

sub ParseArgs{
    use Getopt::Std; # for switch processing
	
    # This sets $opt_u to the user ID, $opt_f to the filesystem name, 
    # $opt_s to the soft quota amount, and $opt_h to the hard quota 
    # amount
    getopt("u:f:s:h:"); # colon (:) means this flag takes an argument
    die "USAGE: $0 -u uid -f <fsystem> -s <softq> -h <hardq>\n"
      if (!$opt_u || !$opt_f || !$opt_s || !$opt_h);
}

sub CallEdquota{
    $ENV{"EDITOR"} = $autoedq; # set the EDITOR variable to point to us

    open(EPROCESS, "|$edquota $opt_u") or 
      die "Unable to start edquota:$!\n";

    # send the changes line to the second script invocation
    print EPROCESS "$opt_f|$opt_s|$opt_h\n";
	
    close(EPROCESS);
}

Here's the second part of the action:

sub EdQuota {
    $tfile = $ARGV[0]; # get the name of edquota's temp file
	
    open(TEMPFILE, $tfile) or
      die "Unable to open temp file $tfile:$!\n";
	
    # open a scratch file, could use IO::File  new_tmpfile( ) instead
    open(NEWTEMP, ">$tfile.$$") or 
      die "Unable to open scratch file $tfile.$$:$!\n";

    # receive line of input from first invocation and lop off the newline
    chomp($change = <STDIN>); 
    my($fs,$soft,$hard) = split(/\|/,$change); # parse the communique
	
    # read in a line from the temp file.  If it contains the
    # filesystem we wish to modify, change its values. Write the input
    # line (possibly changed) to the scratch file.
    while (<TEMPFILE>){
      if (/^fs $fs\s+/){
	     s/(soft\s*=\s*)\d+(, hard\s*=\s*)\d+/$1$soft$2$hard/;
	      print NEWTEMP;
	   }
    }
    close(TEMPFILE);
    close(NEWTEMP);
	
    # overwrite the temp file with our modified scratch file so 
    # edquota will get the changes
    rename("$tfile.$$",$tfile)
      or die "Unable to rename $tfile.$$ to $tfile:$!\n";
}

The above code is bare bones, but it still offers a way to make automated quota changes. If you've ever had to change many quotas by hand, this should be good news. Before putting something like this into production, considerable error checking and a mechanism that prevents multiple concurrent changes should be added. In any case, you may find this sort of sleight-of-hand technique useful in other situations besides quota manipulation.

2.5.2. Editing Quotas Using the Quota Module

Once upon a time, the previous method (or, to be honest, the previous hack) was the only way to automate quota changes unless you wanted to get into the gnarly business of hacking the C quota library routine calls into the Perl interpreter itself. Now that Perl's extension mechanism makes gluing library calls into Perl much easier, it was only an amount of time before someone produced a Quota module for Perl. Thanks to Tom Zoerner and some other porting help, setting quotas from Perl is now much more straightforward if this module supports your variant of Unix. If it doesn't, the previous method should work fine.

Here's some sample code that takes the same arguments as our last quota-editing example:

use Getopt::Std;
use Quota:;

getopt("u:f:s:h:"); 
die "USAGE: $0 -u uid -f <filesystem> -s <softquota> -h <hard quota>\n"
	 if (!$opt_u || !$opt_f || !$opt_s || !$opt_h);

$dev = Quota::getcarg($opt_f) or die "Unable to translate path $opt_f:$!\n";

($curblock,$soft,$hard,$curinode,$btimeout,$curinode,$isoft,$ihard,$itimeout)=
	 Quota::query($dev,$uid) or die "Unable to query quota for $uid:$!\n";

Quota::setqlim($dev,$opt_u,$opt_s,$opt_h,$isoft,$ihard,1) or 
	 die "Unable to set quotas:$!\n";

After we parse the arguments, there are three simple steps: first, we use Quota::getcarg( ) to get the correct device identifier to feed to the other quota routines. Next, we feed this identifier and the user ID to Quota::query( ) to get the current quota settings. We need these settings to avoid perturbing the quota limits we are not interested in changing (like number of files). Finally, we set the quota. That's all it takes, three lines of Perl code.

Remember, the Perl slogan TMTOWTDI means "there's more than one way to do it," not necessarily "several equally good ways."


2.4. Walking the Filesystem Using the File::Find Module		2.6. Querying Filesystem Usage