1134 lines
39 KiB
Plaintext
1134 lines
39 KiB
Plaintext
=head1 NAME
|
|
|
|
perlfaq5 - Files and Formats ($Revision: 1.34 $, $Date: 1999/01/08 05:46:13 $)
|
|
|
|
=head1 DESCRIPTION
|
|
|
|
This section deals with I/O and the "f" issues: filehandles, flushing,
|
|
formats, and footers.
|
|
|
|
=head2 How do I flush/unbuffer an output filehandle? Why must I do this?
|
|
|
|
The C standard I/O library (stdio) normally buffers characters sent to
|
|
devices. This is done for efficiency reasons, so that there isn't a
|
|
system call for each byte. Any time you use print() or write() in
|
|
Perl, you go though this buffering. syswrite() circumvents stdio and
|
|
buffering.
|
|
|
|
In most stdio implementations, the type of output buffering and the size of
|
|
the buffer varies according to the type of device. Disk files are block
|
|
buffered, often with a buffer size of more than 2k. Pipes and sockets
|
|
are often buffered with a buffer size between 1/2 and 2k. Serial devices
|
|
(e.g. modems, terminals) are normally line-buffered, and stdio sends
|
|
the entire line when it gets the newline.
|
|
|
|
Perl does not support truly unbuffered output (except insofar as you can
|
|
C<syswrite(OUT, $char, 1)>). What it does instead support is "command
|
|
buffering", in which a physical write is performed after every output
|
|
command. This isn't as hard on your system as unbuffering, but does
|
|
get the output where you want it when you want it.
|
|
|
|
If you expect characters to get to your device when you print them there,
|
|
you'll want to autoflush its handle.
|
|
Use select() and the C<$|> variable to control autoflushing
|
|
(see L<perlvar/$|> and L<perlfunc/select>):
|
|
|
|
$old_fh = select(OUTPUT_HANDLE);
|
|
$| = 1;
|
|
select($old_fh);
|
|
|
|
Or using the traditional idiom:
|
|
|
|
select((select(OUTPUT_HANDLE), $| = 1)[0]);
|
|
|
|
Or if don't mind slowly loading several thousand lines of module code
|
|
just because you're afraid of the C<$|> variable:
|
|
|
|
use FileHandle;
|
|
open(DEV, "+</dev/tty"); # ceci n'est pas une pipe
|
|
DEV->autoflush(1);
|
|
|
|
or the newer IO::* modules:
|
|
|
|
use IO::Handle;
|
|
open(DEV, ">/dev/printer"); # but is this?
|
|
DEV->autoflush(1);
|
|
|
|
or even this:
|
|
|
|
use IO::Socket; # this one is kinda a pipe?
|
|
$sock = IO::Socket::INET->new(PeerAddr => 'www.perl.com',
|
|
PeerPort => 'http(80)',
|
|
Proto => 'tcp');
|
|
die "$!" unless $sock;
|
|
|
|
$sock->autoflush();
|
|
print $sock "GET / HTTP/1.0" . "\015\012" x 2;
|
|
$document = join('', <$sock>);
|
|
print "DOC IS: $document\n";
|
|
|
|
Note the bizarrely hardcoded carriage return and newline in their octal
|
|
equivalents. This is the ONLY way (currently) to assure a proper flush
|
|
on all platforms, including Macintosh. That the way things work in
|
|
network programming: you really should specify the exact bit pattern
|
|
on the network line terminator. In practice, C<"\n\n"> often works,
|
|
but this is not portable.
|
|
|
|
See L<perlfaq9> for other examples of fetching URLs over the web.
|
|
|
|
=head2 How do I change one line in a file/delete a line in a file/insert a line in the middle of a file/append to the beginning of a file?
|
|
|
|
Those are operations of a text editor. Perl is not a text editor.
|
|
Perl is a programming language. You have to decompose the problem into
|
|
low-level calls to read, write, open, close, and seek.
|
|
|
|
Although humans have an easy time thinking of a text file as being a
|
|
sequence of lines that operates much like a stack of playing cards -- or
|
|
punch cards -- computers usually see the text file as a sequence of bytes.
|
|
In general, there's no direct way for Perl to seek to a particular line
|
|
of a file, insert text into a file, or remove text from a file.
|
|
|
|
(There are exceptions in special circumstances. You can add or remove at
|
|
the very end of the file. Another is replacing a sequence of bytes with
|
|
another sequence of the same length. Another is using the C<$DB_RECNO>
|
|
array bindings as documented in L<DB_File>. Yet another is manipulating
|
|
files with all lines the same length.)
|
|
|
|
The general solution is to create a temporary copy of the text file with
|
|
the changes you want, then copy that over the original. This assumes
|
|
no locking.
|
|
|
|
$old = $file;
|
|
$new = "$file.tmp.$$";
|
|
$bak = "$file.orig";
|
|
|
|
open(OLD, "< $old") or die "can't open $old: $!";
|
|
open(NEW, "> $new") or die "can't open $new: $!";
|
|
|
|
# Correct typos, preserving case
|
|
while (<OLD>) {
|
|
s/\b(p)earl\b/${1}erl/i;
|
|
(print NEW $_) or die "can't write to $new: $!";
|
|
}
|
|
|
|
close(OLD) or die "can't close $old: $!";
|
|
close(NEW) or die "can't close $new: $!";
|
|
|
|
rename($old, $bak) or die "can't rename $old to $bak: $!";
|
|
rename($new, $old) or die "can't rename $new to $old: $!";
|
|
|
|
Perl can do this sort of thing for you automatically with the C<-i>
|
|
command-line switch or the closely-related C<$^I> variable (see
|
|
L<perlrun> for more details). Note that
|
|
C<-i> may require a suffix on some non-Unix systems; see the
|
|
platform-specific documentation that came with your port.
|
|
|
|
# Renumber a series of tests from the command line
|
|
perl -pi -e 's/(^\s+test\s+)\d+/ $1 . ++$count /e' t/op/taint.t
|
|
|
|
# form a script
|
|
local($^I, @ARGV) = ('.orig', glob("*.c"));
|
|
while (<>) {
|
|
if ($. == 1) {
|
|
print "This line should appear at the top of each file\n";
|
|
}
|
|
s/\b(p)earl\b/${1}erl/i; # Correct typos, preserving case
|
|
print;
|
|
close ARGV if eof; # Reset $.
|
|
}
|
|
|
|
If you need to seek to an arbitrary line of a file that changes
|
|
infrequently, you could build up an index of byte positions of where
|
|
the line ends are in the file. If the file is large, an index of
|
|
every tenth or hundredth line end would allow you to seek and read
|
|
fairly efficiently. If the file is sorted, try the look.pl library
|
|
(part of the standard perl distribution).
|
|
|
|
In the unique case of deleting lines at the end of a file, you
|
|
can use tell() and truncate(). The following code snippet deletes
|
|
the last line of a file without making a copy or reading the
|
|
whole file into memory:
|
|
|
|
open (FH, "+< $file");
|
|
while ( <FH> ) { $addr = tell(FH) unless eof(FH) }
|
|
truncate(FH, $addr);
|
|
|
|
Error checking is left as an exercise for the reader.
|
|
|
|
=head2 How do I count the number of lines in a file?
|
|
|
|
One fairly efficient way is to count newlines in the file. The
|
|
following program uses a feature of tr///, as documented in L<perlop>.
|
|
If your text file doesn't end with a newline, then it's not really a
|
|
proper text file, so this may report one fewer line than you expect.
|
|
|
|
$lines = 0;
|
|
open(FILE, $filename) or die "Can't open `$filename': $!";
|
|
while (sysread FILE, $buffer, 4096) {
|
|
$lines += ($buffer =~ tr/\n//);
|
|
}
|
|
close FILE;
|
|
|
|
This assumes no funny games with newline translations.
|
|
|
|
=head2 How do I make a temporary file name?
|
|
|
|
Use the C<new_tmpfile> class method from the IO::File module to get a
|
|
filehandle opened for reading and writing. Use this if you don't
|
|
need to know the file's name.
|
|
|
|
use IO::File;
|
|
$fh = IO::File->new_tmpfile()
|
|
or die "Unable to make new temporary file: $!";
|
|
|
|
Or you can use the C<tmpnam> function from the POSIX module to get a
|
|
filename that you then open yourself. Use this if you do need to know
|
|
the file's name.
|
|
|
|
use Fcntl;
|
|
use POSIX qw(tmpnam);
|
|
|
|
# try new temporary filenames until we get one that didn't already
|
|
# exist; the check should be unnecessary, but you can't be too careful
|
|
do { $name = tmpnam() }
|
|
until sysopen(FH, $name, O_RDWR|O_CREAT|O_EXCL);
|
|
|
|
# install atexit-style handler so that when we exit or die,
|
|
# we automatically delete this temporary file
|
|
END { unlink($name) or die "Couldn't unlink $name : $!" }
|
|
|
|
# now go on to use the file ...
|
|
|
|
If you're committed to doing this by hand, use the process ID and/or
|
|
the current time-value. If you need to have many temporary files in
|
|
one process, use a counter:
|
|
|
|
BEGIN {
|
|
use Fcntl;
|
|
my $temp_dir = -d '/tmp' ? '/tmp' : $ENV{TMP} || $ENV{TEMP};
|
|
my $base_name = sprintf("%s/%d-%d-0000", $temp_dir, $$, time());
|
|
sub temp_file {
|
|
local *FH;
|
|
my $count = 0;
|
|
until (defined(fileno(FH)) || $count++ > 100) {
|
|
$base_name =~ s/-(\d+)$/"-" . (1 + $1)/e;
|
|
sysopen(FH, $base_name, O_WRONLY|O_EXCL|O_CREAT);
|
|
}
|
|
if (defined(fileno(FH))
|
|
return (*FH, $base_name);
|
|
} else {
|
|
return ();
|
|
}
|
|
}
|
|
}
|
|
|
|
=head2 How can I manipulate fixed-record-length files?
|
|
|
|
The most efficient way is using pack() and unpack(). This is faster than
|
|
using substr() when taking many, many strings. It is slower for just a few.
|
|
|
|
Here is a sample chunk of code to break up and put back together again
|
|
some fixed-format input lines, in this case from the output of a normal,
|
|
Berkeley-style ps:
|
|
|
|
# sample input line:
|
|
# 15158 p5 T 0:00 perl /home/tchrist/scripts/now-what
|
|
$PS_T = 'A6 A4 A7 A5 A*';
|
|
open(PS, "ps|");
|
|
print scalar <PS>;
|
|
while (<PS>) {
|
|
($pid, $tt, $stat, $time, $command) = unpack($PS_T, $_);
|
|
for $var (qw!pid tt stat time command!) {
|
|
print "$var: <$$var>\n";
|
|
}
|
|
print 'line=', pack($PS_T, $pid, $tt, $stat, $time, $command),
|
|
"\n";
|
|
}
|
|
|
|
We've used C<$$var> in a way that forbidden by C<use strict 'refs'>.
|
|
That is, we've promoted a string to a scalar variable reference using
|
|
symbolic references. This is ok in small programs, but doesn't scale
|
|
well. It also only works on global variables, not lexicals.
|
|
|
|
=head2 How can I make a filehandle local to a subroutine? How do I pass filehandles between subroutines? How do I make an array of filehandles?
|
|
|
|
The fastest, simplest, and most direct way is to localize the typeglob
|
|
of the filehandle in question:
|
|
|
|
local *TmpHandle;
|
|
|
|
Typeglobs are fast (especially compared with the alternatives) and
|
|
reasonably easy to use, but they also have one subtle drawback. If you
|
|
had, for example, a function named TmpHandle(), or a variable named
|
|
%TmpHandle, you just hid it from yourself.
|
|
|
|
sub findme {
|
|
local *HostFile;
|
|
open(HostFile, "</etc/hosts") or die "no /etc/hosts: $!";
|
|
local $_; # <- VERY IMPORTANT
|
|
while (<HostFile>) {
|
|
print if /\b127\.(0\.0\.)?1\b/;
|
|
}
|
|
# *HostFile automatically closes/disappears here
|
|
}
|
|
|
|
Here's how to use this in a loop to open and store a bunch of
|
|
filehandles. We'll use as values of the hash an ordered
|
|
pair to make it easy to sort the hash in insertion order.
|
|
|
|
@names = qw(motd termcap passwd hosts);
|
|
my $i = 0;
|
|
foreach $filename (@names) {
|
|
local *FH;
|
|
open(FH, "/etc/$filename") || die "$filename: $!";
|
|
$file{$filename} = [ $i++, *FH ];
|
|
}
|
|
|
|
# Using the filehandles in the array
|
|
foreach $name (sort { $file{$a}[0] <=> $file{$b}[0] } keys %file) {
|
|
my $fh = $file{$name}[1];
|
|
my $line = <$fh>;
|
|
print "$name $. $line";
|
|
}
|
|
|
|
For passing filehandles to functions, the easiest way is to
|
|
preface them with a star, as in func(*STDIN). See L<perlfaq7/"Passing
|
|
Filehandles"> for details.
|
|
|
|
If you want to create many anonymous handles, you should check out the
|
|
Symbol, FileHandle, or IO::Handle (etc.) modules. Here's the equivalent
|
|
code with Symbol::gensym, which is reasonably light-weight:
|
|
|
|
foreach $filename (@names) {
|
|
use Symbol;
|
|
my $fh = gensym();
|
|
open($fh, "/etc/$filename") || die "open /etc/$filename: $!";
|
|
$file{$filename} = [ $i++, $fh ];
|
|
}
|
|
|
|
Or here using the semi-object-oriented FileHandle module, which certainly
|
|
isn't light-weight:
|
|
|
|
use FileHandle;
|
|
|
|
foreach $filename (@names) {
|
|
my $fh = FileHandle->new("/etc/$filename") or die "$filename: $!";
|
|
$file{$filename} = [ $i++, $fh ];
|
|
}
|
|
|
|
Please understand that whether the filehandle happens to be a (probably
|
|
localized) typeglob or an anonymous handle from one of the modules,
|
|
in no way affects the bizarre rules for managing indirect handles.
|
|
See the next question.
|
|
|
|
=head2 How can I use a filehandle indirectly?
|
|
|
|
An indirect filehandle is using something other than a symbol
|
|
in a place that a filehandle is expected. Here are ways
|
|
to get those:
|
|
|
|
$fh = SOME_FH; # bareword is strict-subs hostile
|
|
$fh = "SOME_FH"; # strict-refs hostile; same package only
|
|
$fh = *SOME_FH; # typeglob
|
|
$fh = \*SOME_FH; # ref to typeglob (bless-able)
|
|
$fh = *SOME_FH{IO}; # blessed IO::Handle from *SOME_FH typeglob
|
|
|
|
Or to use the C<new> method from the FileHandle or IO modules to
|
|
create an anonymous filehandle, store that in a scalar variable,
|
|
and use it as though it were a normal filehandle.
|
|
|
|
use FileHandle;
|
|
$fh = FileHandle->new();
|
|
|
|
use IO::Handle; # 5.004 or higher
|
|
$fh = IO::Handle->new();
|
|
|
|
Then use any of those as you would a normal filehandle. Anywhere that
|
|
Perl is expecting a filehandle, an indirect filehandle may be used
|
|
instead. An indirect filehandle is just a scalar variable that contains
|
|
a filehandle. Functions like C<print>, C<open>, C<seek>, or
|
|
the C<E<lt>FHE<gt>> diamond operator will accept either a read filehandle
|
|
or a scalar variable containing one:
|
|
|
|
($ifh, $ofh, $efh) = (*STDIN, *STDOUT, *STDERR);
|
|
print $ofh "Type it: ";
|
|
$got = <$ifh>
|
|
print $efh "What was that: $got";
|
|
|
|
If you're passing a filehandle to a function, you can write
|
|
the function in two ways:
|
|
|
|
sub accept_fh {
|
|
my $fh = shift;
|
|
print $fh "Sending to indirect filehandle\n";
|
|
}
|
|
|
|
Or it can localize a typeglob and use the filehandle directly:
|
|
|
|
sub accept_fh {
|
|
local *FH = shift;
|
|
print FH "Sending to localized filehandle\n";
|
|
}
|
|
|
|
Both styles work with either objects or typeglobs of real filehandles.
|
|
(They might also work with strings under some circumstances, but this
|
|
is risky.)
|
|
|
|
accept_fh(*STDOUT);
|
|
accept_fh($handle);
|
|
|
|
In the examples above, we assigned the filehandle to a scalar variable
|
|
before using it. That is because only simple scalar variables,
|
|
not expressions or subscripts into hashes or arrays, can be used with
|
|
built-ins like C<print>, C<printf>, or the diamond operator. These are
|
|
illegal and won't even compile:
|
|
|
|
@fd = (*STDIN, *STDOUT, *STDERR);
|
|
print $fd[1] "Type it: "; # WRONG
|
|
$got = <$fd[0]> # WRONG
|
|
print $fd[2] "What was that: $got"; # WRONG
|
|
|
|
With C<print> and C<printf>, you get around this by using a block and
|
|
an expression where you would place the filehandle:
|
|
|
|
print { $fd[1] } "funny stuff\n";
|
|
printf { $fd[1] } "Pity the poor %x.\n", 3_735_928_559;
|
|
# Pity the poor deadbeef.
|
|
|
|
That block is a proper block like any other, so you can put more
|
|
complicated code there. This sends the message out to one of two places:
|
|
|
|
$ok = -x "/bin/cat";
|
|
print { $ok ? $fd[1] : $fd[2] } "cat stat $ok\n";
|
|
print { $fd[ 1+ ($ok || 0) ] } "cat stat $ok\n";
|
|
|
|
This approach of treating C<print> and C<printf> like object methods
|
|
calls doesn't work for the diamond operator. That's because it's a
|
|
real operator, not just a function with a comma-less argument. Assuming
|
|
you've been storing typeglobs in your structure as we did above, you
|
|
can use the built-in function named C<readline> to reads a record just
|
|
as C<E<lt>E<gt>> does. Given the initialization shown above for @fd, this
|
|
would work, but only because readline() require a typeglob. It doesn't
|
|
work with objects or strings, which might be a bug we haven't fixed yet.
|
|
|
|
$got = readline($fd[0]);
|
|
|
|
Let it be noted that the flakiness of indirect filehandles is not
|
|
related to whether they're strings, typeglobs, objects, or anything else.
|
|
It's the syntax of the fundamental operators. Playing the object
|
|
game doesn't help you at all here.
|
|
|
|
=head2 How can I set up a footer format to be used with write()?
|
|
|
|
There's no builtin way to do this, but L<perlform> has a couple of
|
|
techniques to make it possible for the intrepid hacker.
|
|
|
|
=head2 How can I write() into a string?
|
|
|
|
See L<perlform/"Accessing Formatting Internals"> for an swrite() function.
|
|
|
|
=head2 How can I output my numbers with commas added?
|
|
|
|
This one will do it for you:
|
|
|
|
sub commify {
|
|
local $_ = shift;
|
|
1 while s/^([-+]?\d+)(\d{3})/$1,$2/;
|
|
return $_;
|
|
}
|
|
|
|
$n = 23659019423.2331;
|
|
print "GOT: ", commify($n), "\n";
|
|
|
|
GOT: 23,659,019,423.2331
|
|
|
|
You can't just:
|
|
|
|
s/^([-+]?\d+)(\d{3})/$1,$2/g;
|
|
|
|
because you have to put the comma in and then recalculate your
|
|
position.
|
|
|
|
Alternatively, this commifies all numbers in a line regardless of
|
|
whether they have decimal portions, are preceded by + or -, or
|
|
whatever:
|
|
|
|
# from Andrew Johnson <ajohnson@gpu.srv.ualberta.ca>
|
|
sub commify {
|
|
my $input = shift;
|
|
$input = reverse $input;
|
|
$input =~ s<(\d\d\d)(?=\d)(?!\d*\.)><$1,>g;
|
|
return scalar reverse $input;
|
|
}
|
|
|
|
=head2 How can I translate tildes (~) in a filename?
|
|
|
|
Use the E<lt>E<gt> (glob()) operator, documented in L<perlfunc>. This
|
|
requires that you have a shell installed that groks tildes, meaning
|
|
csh or tcsh or (some versions of) ksh, and thus may have portability
|
|
problems. The Glob::KGlob module (available from CPAN) gives more
|
|
portable glob functionality.
|
|
|
|
Within Perl, you may use this directly:
|
|
|
|
$filename =~ s{
|
|
^ ~ # find a leading tilde
|
|
( # save this in $1
|
|
[^/] # a non-slash character
|
|
* # repeated 0 or more times (0 means me)
|
|
)
|
|
}{
|
|
$1
|
|
? (getpwnam($1))[7]
|
|
: ( $ENV{HOME} || $ENV{LOGDIR} )
|
|
}ex;
|
|
|
|
=head2 How come when I open a file read-write it wipes it out?
|
|
|
|
Because you're using something like this, which truncates the file and
|
|
I<then> gives you read-write access:
|
|
|
|
open(FH, "+> /path/name"); # WRONG (almost always)
|
|
|
|
Whoops. You should instead use this, which will fail if the file
|
|
doesn't exist. Using "E<gt>" always clobbers or creates.
|
|
Using "E<lt>" never does either. The "+" doesn't change this.
|
|
|
|
Here are examples of many kinds of file opens. Those using sysopen()
|
|
all assume
|
|
|
|
use Fcntl;
|
|
|
|
To open file for reading:
|
|
|
|
open(FH, "< $path") || die $!;
|
|
sysopen(FH, $path, O_RDONLY) || die $!;
|
|
|
|
To open file for writing, create new file if needed or else truncate old file:
|
|
|
|
open(FH, "> $path") || die $!;
|
|
sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT) || die $!;
|
|
sysopen(FH, $path, O_WRONLY|O_TRUNC|O_CREAT, 0666) || die $!;
|
|
|
|
To open file for writing, create new file, file must not exist:
|
|
|
|
sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT) || die $!;
|
|
sysopen(FH, $path, O_WRONLY|O_EXCL|O_CREAT, 0666) || die $!;
|
|
|
|
To open file for appending, create if necessary:
|
|
|
|
open(FH, ">> $path") || die $!;
|
|
sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT) || die $!;
|
|
sysopen(FH, $path, O_WRONLY|O_APPEND|O_CREAT, 0666) || die $!;
|
|
|
|
To open file for appending, file must exist:
|
|
|
|
sysopen(FH, $path, O_WRONLY|O_APPEND) || die $!;
|
|
|
|
To open file for update, file must exist:
|
|
|
|
open(FH, "+< $path") || die $!;
|
|
sysopen(FH, $path, O_RDWR) || die $!;
|
|
|
|
To open file for update, create file if necessary:
|
|
|
|
sysopen(FH, $path, O_RDWR|O_CREAT) || die $!;
|
|
sysopen(FH, $path, O_RDWR|O_CREAT, 0666) || die $!;
|
|
|
|
To open file for update, file must not exist:
|
|
|
|
sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT) || die $!;
|
|
sysopen(FH, $path, O_RDWR|O_EXCL|O_CREAT, 0666) || die $!;
|
|
|
|
To open a file without blocking, creating if necessary:
|
|
|
|
sysopen(FH, "/tmp/somefile", O_WRONLY|O_NDELAY|O_CREAT)
|
|
or die "can't open /tmp/somefile: $!":
|
|
|
|
Be warned that neither creation nor deletion of files is guaranteed to
|
|
be an atomic operation over NFS. That is, two processes might both
|
|
successful create or unlink the same file! Therefore O_EXCL
|
|
isn't so exclusive as you might wish.
|
|
|
|
See also the new L<perlopentut> if you have it (new for 5.006).
|
|
|
|
=head2 Why do I sometimes get an "Argument list too long" when I use E<lt>*E<gt>?
|
|
|
|
The C<E<lt>E<gt>> operator performs a globbing operation (see above).
|
|
By default glob() forks csh(1) to do the actual glob expansion, but
|
|
csh can't handle more than 127 items and so gives the error message
|
|
C<Argument list too long>. People who installed tcsh as csh won't
|
|
have this problem, but their users may be surprised by it.
|
|
|
|
To get around this, either do the glob yourself with readdir() and
|
|
patterns, or use a module like Glob::KGlob, one that doesn't use the
|
|
shell to do globbing. This is expected to be fixed soon.
|
|
|
|
=head2 Is there a leak/bug in glob()?
|
|
|
|
Due to the current implementation on some operating systems, when you
|
|
use the glob() function or its angle-bracket alias in a scalar
|
|
context, you may cause a leak and/or unpredictable behavior. It's
|
|
best therefore to use glob() only in list context.
|
|
|
|
=head2 How can I open a file with a leading "E<gt>" or trailing blanks?
|
|
|
|
Normally perl ignores trailing blanks in filenames, and interprets
|
|
certain leading characters (or a trailing "|") to mean something
|
|
special. To avoid this, you might want to use a routine like this.
|
|
It makes incomplete pathnames into explicit relative ones, and tacks a
|
|
trailing null byte on the name to make perl leave it alone:
|
|
|
|
sub safe_filename {
|
|
local $_ = shift;
|
|
s#^([^./])#./$1#;
|
|
$_ .= "\0";
|
|
return $_;
|
|
}
|
|
|
|
$badpath = "<<<something really wicked ";
|
|
$fn = safe_filename($badpath");
|
|
open(FH, "> $fn") or "couldn't open $badpath: $!";
|
|
|
|
This assumes that you are using POSIX (portable operating systems
|
|
interface) paths. If you are on a closed, non-portable, proprietary
|
|
system, you may have to adjust the C<"./"> above.
|
|
|
|
It would be a lot clearer to use sysopen(), though:
|
|
|
|
use Fcntl;
|
|
$badpath = "<<<something really wicked ";
|
|
open (FH, $badpath, O_WRONLY | O_CREAT | O_TRUNC)
|
|
or die "can't open $badpath: $!";
|
|
|
|
For more information, see also the new L<perlopentut> if you have it
|
|
(new for 5.006).
|
|
|
|
=head2 How can I reliably rename a file?
|
|
|
|
Well, usually you just use Perl's rename() function. But that may
|
|
not work everywhere, in particular, renaming files across file systems.
|
|
If your operating system supports a mv(1) program or its moral equivalent,
|
|
this works:
|
|
|
|
rename($old, $new) or system("mv", $old, $new);
|
|
|
|
It may be more compelling to use the File::Copy module instead. You
|
|
just copy to the new file to the new name (checking return values),
|
|
then delete the old one. This isn't really the same semantics as a
|
|
real rename(), though, which preserves metainformation like
|
|
permissions, timestamps, inode info, etc.
|
|
|
|
The newer version of File::Copy exports a move() function.
|
|
|
|
=head2 How can I lock a file?
|
|
|
|
Perl's builtin flock() function (see L<perlfunc> for details) will call
|
|
flock(2) if that exists, fcntl(2) if it doesn't (on perl version 5.004 and
|
|
later), and lockf(3) if neither of the two previous system calls exists.
|
|
On some systems, it may even use a different form of native locking.
|
|
Here are some gotchas with Perl's flock():
|
|
|
|
=over 4
|
|
|
|
=item 1
|
|
|
|
Produces a fatal error if none of the three system calls (or their
|
|
close equivalent) exists.
|
|
|
|
=item 2
|
|
|
|
lockf(3) does not provide shared locking, and requires that the
|
|
filehandle be open for writing (or appending, or read/writing).
|
|
|
|
=item 3
|
|
|
|
Some versions of flock() can't lock files over a network (e.g. on NFS
|
|
file systems), so you'd need to force the use of fcntl(2) when you
|
|
build Perl. See the flock entry of L<perlfunc>, and the F<INSTALL>
|
|
file in the source distribution for information on building Perl to do
|
|
this.
|
|
|
|
For more information on file locking, see also L<perlopentut/"File
|
|
Locking"> if you have it (new for 5.006).
|
|
|
|
=back
|
|
|
|
=head2 Why can't I just open(FH, ">file.lock")?
|
|
|
|
A common bit of code B<NOT TO USE> is this:
|
|
|
|
sleep(3) while -e "file.lock"; # PLEASE DO NOT USE
|
|
open(LCK, "> file.lock"); # THIS BROKEN CODE
|
|
|
|
This is a classic race condition: you take two steps to do something
|
|
which must be done in one. That's why computer hardware provides an
|
|
atomic test-and-set instruction. In theory, this "ought" to work:
|
|
|
|
sysopen(FH, "file.lock", O_WRONLY|O_EXCL|O_CREAT)
|
|
or die "can't open file.lock: $!":
|
|
|
|
except that lamentably, file creation (and deletion) is not atomic
|
|
over NFS, so this won't work (at least, not every time) over the net.
|
|
Various schemes involving link() have been suggested, but
|
|
these tend to involve busy-wait, which is also subdesirable.
|
|
|
|
=head2 I still don't get locking. I just want to increment the number in the file. How can I do this?
|
|
|
|
Didn't anyone ever tell you web-page hit counters were useless?
|
|
They don't count number of hits, they're a waste of time, and they serve
|
|
only to stroke the writer's vanity. Better to pick a random number.
|
|
It's more realistic.
|
|
|
|
Anyway, this is what you can do if you can't help yourself.
|
|
|
|
use Fcntl ':flock';
|
|
sysopen(FH, "numfile", O_RDWR|O_CREAT) or die "can't open numfile: $!";
|
|
flock(FH, LOCK_EX) or die "can't flock numfile: $!";
|
|
$num = <FH> || 0;
|
|
seek(FH, 0, 0) or die "can't rewind numfile: $!";
|
|
truncate(FH, 0) or die "can't truncate numfile: $!";
|
|
(print FH $num+1, "\n") or die "can't write numfile: $!";
|
|
# Perl as of 5.004 automatically flushes before unlocking
|
|
flock(FH, LOCK_UN) or die "can't flock numfile: $!";
|
|
close FH or die "can't close numfile: $!";
|
|
|
|
Here's a much better web-page hit counter:
|
|
|
|
$hits = int( (time() - 850_000_000) / rand(1_000) );
|
|
|
|
If the count doesn't impress your friends, then the code might. :-)
|
|
|
|
=head2 How do I randomly update a binary file?
|
|
|
|
If you're just trying to patch a binary, in many cases something as
|
|
simple as this works:
|
|
|
|
perl -i -pe 's{window manager}{window mangler}g' /usr/bin/emacs
|
|
|
|
However, if you have fixed sized records, then you might do something more
|
|
like this:
|
|
|
|
$RECSIZE = 220; # size of record, in bytes
|
|
$recno = 37; # which record to update
|
|
open(FH, "+<somewhere") || die "can't update somewhere: $!";
|
|
seek(FH, $recno * $RECSIZE, 0);
|
|
read(FH, $record, $RECSIZE) == $RECSIZE || die "can't read record $recno: $!";
|
|
# munge the record
|
|
seek(FH, -$RECSIZE, 1);
|
|
print FH $record;
|
|
close FH;
|
|
|
|
Locking and error checking are left as an exercise for the reader.
|
|
Don't forget them, or you'll be quite sorry.
|
|
|
|
=head2 How do I get a file's timestamp in perl?
|
|
|
|
If you want to retrieve the time at which the file was last read,
|
|
written, or had its meta-data (owner, etc) changed, you use the B<-M>,
|
|
B<-A>, or B<-C> filetest operations as documented in L<perlfunc>. These
|
|
retrieve the age of the file (measured against the start-time of your
|
|
program) in days as a floating point number. To retrieve the "raw"
|
|
time in seconds since the epoch, you would call the stat function,
|
|
then use localtime(), gmtime(), or POSIX::strftime() to convert this
|
|
into human-readable form.
|
|
|
|
Here's an example:
|
|
|
|
$write_secs = (stat($file))[9];
|
|
printf "file %s updated at %s\n", $file,
|
|
scalar localtime($write_secs);
|
|
|
|
If you prefer something more legible, use the File::stat module
|
|
(part of the standard distribution in version 5.004 and later):
|
|
|
|
# error checking left as an exercise for reader.
|
|
use File::stat;
|
|
use Time::localtime;
|
|
$date_string = ctime(stat($file)->mtime);
|
|
print "file $file updated at $date_string\n";
|
|
|
|
The POSIX::strftime() approach has the benefit of being,
|
|
in theory, independent of the current locale. See L<perllocale>
|
|
for details.
|
|
|
|
=head2 How do I set a file's timestamp in perl?
|
|
|
|
You use the utime() function documented in L<perlfunc/utime>.
|
|
By way of example, here's a little program that copies the
|
|
read and write times from its first argument to all the rest
|
|
of them.
|
|
|
|
if (@ARGV < 2) {
|
|
die "usage: cptimes timestamp_file other_files ...\n";
|
|
}
|
|
$timestamp = shift;
|
|
($atime, $mtime) = (stat($timestamp))[8,9];
|
|
utime $atime, $mtime, @ARGV;
|
|
|
|
Error checking is, as usual, left as an exercise for the reader.
|
|
|
|
Note that utime() currently doesn't work correctly with Win95/NT
|
|
ports. A bug has been reported. Check it carefully before using
|
|
it on those platforms.
|
|
|
|
=head2 How do I print to more than one file at once?
|
|
|
|
If you only have to do this once, you can do this:
|
|
|
|
for $fh (FH1, FH2, FH3) { print $fh "whatever\n" }
|
|
|
|
To connect up to one filehandle to several output filehandles, it's
|
|
easiest to use the tee(1) program if you have it, and let it take care
|
|
of the multiplexing:
|
|
|
|
open (FH, "| tee file1 file2 file3");
|
|
|
|
Or even:
|
|
|
|
# make STDOUT go to three files, plus original STDOUT
|
|
open (STDOUT, "| tee file1 file2 file3") or die "Teeing off: $!\n";
|
|
print "whatever\n" or die "Writing: $!\n";
|
|
close(STDOUT) or die "Closing: $!\n";
|
|
|
|
Otherwise you'll have to write your own multiplexing print
|
|
function -- or your own tee program -- or use Tom Christiansen's,
|
|
at http://www.perl.com/CPAN/authors/id/TOMC/scripts/tct.gz, which is
|
|
written in Perl and offers much greater functionality
|
|
than the stock version.
|
|
|
|
=head2 How can I read in a file by paragraphs?
|
|
|
|
Use the C<$/> variable (see L<perlvar> for details). You can either
|
|
set it to C<""> to eliminate empty paragraphs (C<"abc\n\n\n\ndef">,
|
|
for instance, gets treated as two paragraphs and not three), or
|
|
C<"\n\n"> to accept empty paragraphs.
|
|
|
|
Note that a blank line must have no blanks in it. Thus C<"fred\n
|
|
\nstuff\n\n"> is one paragraph, but C<"fred\n\nstuff\n\n"> is two.
|
|
|
|
=head2 How can I read a single character from a file? From the keyboard?
|
|
|
|
You can use the builtin C<getc()> function for most filehandles, but
|
|
it won't (easily) work on a terminal device. For STDIN, either use
|
|
the Term::ReadKey module from CPAN, or use the sample code in
|
|
L<perlfunc/getc>.
|
|
|
|
If your system supports the portable operating system programming
|
|
interface (POSIX), you can use the following code, which you'll note
|
|
turns off echo processing as well.
|
|
|
|
#!/usr/bin/perl -w
|
|
use strict;
|
|
$| = 1;
|
|
for (1..4) {
|
|
my $got;
|
|
print "gimme: ";
|
|
$got = getone();
|
|
print "--> $got\n";
|
|
}
|
|
exit;
|
|
|
|
BEGIN {
|
|
use POSIX qw(:termios_h);
|
|
|
|
my ($term, $oterm, $echo, $noecho, $fd_stdin);
|
|
|
|
$fd_stdin = fileno(STDIN);
|
|
|
|
$term = POSIX::Termios->new();
|
|
$term->getattr($fd_stdin);
|
|
$oterm = $term->getlflag();
|
|
|
|
$echo = ECHO | ECHOK | ICANON;
|
|
$noecho = $oterm & ~$echo;
|
|
|
|
sub cbreak {
|
|
$term->setlflag($noecho);
|
|
$term->setcc(VTIME, 1);
|
|
$term->setattr($fd_stdin, TCSANOW);
|
|
}
|
|
|
|
sub cooked {
|
|
$term->setlflag($oterm);
|
|
$term->setcc(VTIME, 0);
|
|
$term->setattr($fd_stdin, TCSANOW);
|
|
}
|
|
|
|
sub getone {
|
|
my $key = '';
|
|
cbreak();
|
|
sysread(STDIN, $key, 1);
|
|
cooked();
|
|
return $key;
|
|
}
|
|
|
|
}
|
|
|
|
END { cooked() }
|
|
|
|
The Term::ReadKey module from CPAN may be easier to use. Recent version
|
|
include also support for non-portable systems as well.
|
|
|
|
use Term::ReadKey;
|
|
open(TTY, "</dev/tty");
|
|
print "Gimme a char: ";
|
|
ReadMode "raw";
|
|
$key = ReadKey 0, *TTY;
|
|
ReadMode "normal";
|
|
printf "\nYou said %s, char number %03d\n",
|
|
$key, ord $key;
|
|
|
|
For legacy DOS systems, Dan Carson <dbc@tc.fluke.COM> reports the following:
|
|
|
|
To put the PC in "raw" mode, use ioctl with some magic numbers gleaned
|
|
from msdos.c (Perl source file) and Ralf Brown's interrupt list (comes
|
|
across the net every so often):
|
|
|
|
$old_ioctl = ioctl(STDIN,0,0); # Gets device info
|
|
$old_ioctl &= 0xff;
|
|
ioctl(STDIN,1,$old_ioctl | 32); # Writes it back, setting bit 5
|
|
|
|
Then to read a single character:
|
|
|
|
sysread(STDIN,$c,1); # Read a single character
|
|
|
|
And to put the PC back to "cooked" mode:
|
|
|
|
ioctl(STDIN,1,$old_ioctl); # Sets it back to cooked mode.
|
|
|
|
So now you have $c. If C<ord($c) == 0>, you have a two byte code, which
|
|
means you hit a special key. Read another byte with C<sysread(STDIN,$c,1)>,
|
|
and that value tells you what combination it was according to this
|
|
table:
|
|
|
|
# PC 2-byte keycodes = ^@ + the following:
|
|
|
|
# HEX KEYS
|
|
# --- ----
|
|
# 0F SHF TAB
|
|
# 10-19 ALT QWERTYUIOP
|
|
# 1E-26 ALT ASDFGHJKL
|
|
# 2C-32 ALT ZXCVBNM
|
|
# 3B-44 F1-F10
|
|
# 47-49 HOME,UP,PgUp
|
|
# 4B LEFT
|
|
# 4D RIGHT
|
|
# 4F-53 END,DOWN,PgDn,Ins,Del
|
|
# 54-5D SHF F1-F10
|
|
# 5E-67 CTR F1-F10
|
|
# 68-71 ALT F1-F10
|
|
# 73-77 CTR LEFT,RIGHT,END,PgDn,HOME
|
|
# 78-83 ALT 1234567890-=
|
|
# 84 CTR PgUp
|
|
|
|
This is all trial and error I did a long time ago, I hope I'm reading the
|
|
file that worked.
|
|
|
|
=head2 How can I tell whether there's a character waiting on a filehandle?
|
|
|
|
The very first thing you should do is look into getting the Term::ReadKey
|
|
extension from CPAN. As we mentioned earlier, it now even has limited
|
|
support for non-portable (read: not open systems, closed, proprietary,
|
|
not POSIX, not Unix, etc) systems.
|
|
|
|
You should also check out the Frequently Asked Questions list in
|
|
comp.unix.* for things like this: the answer is essentially the same.
|
|
It's very system dependent. Here's one solution that works on BSD
|
|
systems:
|
|
|
|
sub key_ready {
|
|
my($rin, $nfd);
|
|
vec($rin, fileno(STDIN), 1) = 1;
|
|
return $nfd = select($rin,undef,undef,0);
|
|
}
|
|
|
|
If you want to find out how many characters are waiting, there's
|
|
also the FIONREAD ioctl call to be looked at. The I<h2ph> tool that
|
|
comes with Perl tries to convert C include files to Perl code, which
|
|
can be C<require>d. FIONREAD ends up defined as a function in the
|
|
I<sys/ioctl.ph> file:
|
|
|
|
require 'sys/ioctl.ph';
|
|
|
|
$size = pack("L", 0);
|
|
ioctl(FH, FIONREAD(), $size) or die "Couldn't call ioctl: $!\n";
|
|
$size = unpack("L", $size);
|
|
|
|
If I<h2ph> wasn't installed or doesn't work for you, you can
|
|
I<grep> the include files by hand:
|
|
|
|
% grep FIONREAD /usr/include/*/*
|
|
/usr/include/asm/ioctls.h:#define FIONREAD 0x541B
|
|
|
|
Or write a small C program using the editor of champions:
|
|
|
|
% cat > fionread.c
|
|
#include <sys/ioctl.h>
|
|
main() {
|
|
printf("%#08x\n", FIONREAD);
|
|
}
|
|
^D
|
|
% cc -o fionread fionread.c
|
|
% ./fionread
|
|
0x4004667f
|
|
|
|
And then hard-code it, leaving porting as an exercise to your successor.
|
|
|
|
$FIONREAD = 0x4004667f; # XXX: opsys dependent
|
|
|
|
$size = pack("L", 0);
|
|
ioctl(FH, $FIONREAD, $size) or die "Couldn't call ioctl: $!\n";
|
|
$size = unpack("L", $size);
|
|
|
|
FIONREAD requires a filehandle connected to a stream, meaning sockets,
|
|
pipes, and tty devices work, but I<not> files.
|
|
|
|
=head2 How do I do a C<tail -f> in perl?
|
|
|
|
First try
|
|
|
|
seek(GWFILE, 0, 1);
|
|
|
|
The statement C<seek(GWFILE, 0, 1)> doesn't change the current position,
|
|
but it does clear the end-of-file condition on the handle, so that the
|
|
next <GWFILE> makes Perl try again to read something.
|
|
|
|
If that doesn't work (it relies on features of your stdio implementation),
|
|
then you need something more like this:
|
|
|
|
for (;;) {
|
|
for ($curpos = tell(GWFILE); <GWFILE>; $curpos = tell(GWFILE)) {
|
|
# search for some stuff and put it into files
|
|
}
|
|
# sleep for a while
|
|
seek(GWFILE, $curpos, 0); # seek to where we had been
|
|
}
|
|
|
|
If this still doesn't work, look into the POSIX module. POSIX defines
|
|
the clearerr() method, which can remove the end of file condition on a
|
|
filehandle. The method: read until end of file, clearerr(), read some
|
|
more. Lather, rinse, repeat.
|
|
|
|
There's also a File::Tail module from CPAN.
|
|
|
|
=head2 How do I dup() a filehandle in Perl?
|
|
|
|
If you check L<perlfunc/open>, you'll see that several of the ways
|
|
to call open() should do the trick. For example:
|
|
|
|
open(LOG, ">>/tmp/logfile");
|
|
open(STDERR, ">&LOG");
|
|
|
|
Or even with a literal numeric descriptor:
|
|
|
|
$fd = $ENV{MHCONTEXTFD};
|
|
open(MHCONTEXT, "<&=$fd"); # like fdopen(3S)
|
|
|
|
Note that "E<lt>&STDIN" makes a copy, but "E<lt>&=STDIN" make
|
|
an alias. That means if you close an aliased handle, all
|
|
aliases become inaccessible. This is not true with
|
|
a copied one.
|
|
|
|
Error checking, as always, has been left as an exercise for the reader.
|
|
|
|
=head2 How do I close a file descriptor by number?
|
|
|
|
This should rarely be necessary, as the Perl close() function is to be
|
|
used for things that Perl opened itself, even if it was a dup of a
|
|
numeric descriptor, as with MHCONTEXT above. But if you really have
|
|
to, you may be able to do this:
|
|
|
|
require 'sys/syscall.ph';
|
|
$rc = syscall(&SYS_close, $fd + 0); # must force numeric
|
|
die "can't sysclose $fd: $!" unless $rc == -1;
|
|
|
|
=head2 Why can't I use "C:\temp\foo" in DOS paths? What doesn't `C:\temp\foo.exe` work?
|
|
|
|
Whoops! You just put a tab and a formfeed into that filename!
|
|
Remember that within double quoted strings ("like\this"), the
|
|
backslash is an escape character. The full list of these is in
|
|
L<perlop/Quote and Quote-like Operators>. Unsurprisingly, you don't
|
|
have a file called "c:(tab)emp(formfeed)oo" or
|
|
"c:(tab)emp(formfeed)oo.exe" on your legacy DOS filesystem.
|
|
|
|
Either single-quote your strings, or (preferably) use forward slashes.
|
|
Since all DOS and Windows versions since something like MS-DOS 2.0 or so
|
|
have treated C</> and C<\> the same in a path, you might as well use the
|
|
one that doesn't clash with Perl -- or the POSIX shell, ANSI C and C++,
|
|
awk, Tcl, Java, or Python, just to mention a few. POSIX paths
|
|
are more portable, too.
|
|
|
|
=head2 Why doesn't glob("*.*") get all the files?
|
|
|
|
Because even on non-Unix ports, Perl's glob function follows standard
|
|
Unix globbing semantics. You'll need C<glob("*")> to get all (non-hidden)
|
|
files. This makes glob() portable even to legacy systems. Your
|
|
port may include proprietary globbing functions as well. Check its
|
|
documentation for details.
|
|
|
|
=head2 Why does Perl let me delete read-only files? Why does C<-i> clobber protected files? Isn't this a bug in Perl?
|
|
|
|
This is elaborately and painstakingly described in the "Far More Than
|
|
You Ever Wanted To Know" in
|
|
http://www.perl.com/CPAN/doc/FMTEYEWTK/file-dir-perms .
|
|
|
|
The executive summary: learn how your filesystem works. The
|
|
permissions on a file say what can happen to the data in that file.
|
|
The permissions on a directory say what can happen to the list of
|
|
files in that directory. If you delete a file, you're removing its
|
|
name from the directory (so the operation depends on the permissions
|
|
of the directory, not of the file). If you try to write to the file,
|
|
the permissions of the file govern whether you're allowed to.
|
|
|
|
=head2 How do I select a random line from a file?
|
|
|
|
Here's an algorithm from the Camel Book:
|
|
|
|
srand;
|
|
rand($.) < 1 && ($line = $_) while <>;
|
|
|
|
This has a significant advantage in space over reading the whole
|
|
file in. A simple proof by induction is available upon
|
|
request if you doubt its correctness.
|
|
|
|
=head2 Why do I get weird spaces when I print an array of lines?
|
|
|
|
Saying
|
|
|
|
print "@lines\n";
|
|
|
|
joins together the elements of C<@lines> with a space between them.
|
|
If C<@lines> were C<("little", "fluffy", "clouds")> then the above
|
|
statement would print:
|
|
|
|
little fluffy clouds
|
|
|
|
but if each element of C<@lines> was a line of text, ending a newline
|
|
character C<("little\n", "fluffy\n", "clouds\n")> then it would print:
|
|
|
|
little
|
|
fluffy
|
|
clouds
|
|
|
|
If your array contains lines, just print them:
|
|
|
|
print @lines;
|
|
|
|
=head1 AUTHOR AND COPYRIGHT
|
|
|
|
Copyright (c) 1997-1999 Tom Christiansen and Nathan Torkington.
|
|
All rights reserved.
|
|
|
|
When included as an integrated part of the Standard Distribution
|
|
of Perl or of its documentation (printed or otherwise), this work is
|
|
covered under Perl's Artistic Licence. For separate distributions of
|
|
all or part of this FAQ outside of that, see L<perlfaq>.
|
|
|
|
Irrespective of its distribution, all code examples here are public
|
|
domain. You are permitted and encouraged to use this code and any
|
|
derivatives thereof in your own programs for fun or for profit as you
|
|
see fit. A simple comment in the code giving credit to the FAQ would
|
|
be courteous but is not required.
|
|
|