d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

(Not tested yet. I may insist that ctm be invoked with absolute path. /phk)

Browse Source 
This patch fixes the concurrency problem, and adds a possibly useful -f switch
(which you can read about in the man page :-) ).  It also removes the absolute
path from the invocation of ctm.  I'll write a note about how to use a script
with sendmail and procmail or some such, and people can fix their PATH there.

BTW, this patch changes ctm_rmail.1, ctm_rmail.c and error.c in the ctm_rmail
directory.

Stephen.

Reviewed by:	phk
Submitted by:	Stephen McKay <syssgm@devetir.qld.gov.au>
This commit is contained in:
phk 1995-02-25 05:10:18 +00:00
parent b589193b1c
commit 36c6492739
3 changed files with 281 additions and 65 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

74
usr.sbin/ctm/ctm_rmail/ctm_rmail.1
View File

@ -5,9 +5,9 @@
.\"
.\" Author: Stephen McKay
.\"
.Dd January 15, 1995
.Os
.Dd January 24, 1995
.Dt CTM_MAIL 1
.Os
.Sh NAME
.Nm ctm_smail, ctm_rmail
.Nd send and receive
@ -21,7 +21,7 @@ deltas via mail
.Ar ctm-delta
.Ar mail-alias
.Nm ctm_rmail
.Op Fl D
.Op Fl Df
.Op Fl l Ar log
.Op Fl p Ar piecedir
.Op Fl d Ar deltadir
@ -37,14 +37,14 @@ and
are used to distribute changes to a source tree via email.
.Nm ctm_smail
is given a compressed
.Nm ctm
.Xr ctm
delta, and a mailing list to send it to. It splits the delta into manageable
pieces, encodes them as mail messages and sends them to the mailing list.
Each recipient uses
.Nm ctm_rmail
(either manually or automatically) to decode and reassemble the delta, and
optionally call
.Xr ctm 1
.Xr ctm
to apply it to the source tree.
At the moment,
only two source trees are distributed, and both by the same site. These are
@ -96,7 +96,7 @@ Collect pieces of deltas in this directory. Each piece corresponds to a
single mail message. Pieces are removed when complete deltas are built.
If this flag is not given, no input files will be read, but completed
deltas may still be applied with
.Xr ctm 1
.Xr ctm
if the
.Fl b
flag is given.
@ -120,11 +120,34 @@ file in
does not exist).
.It Fl D
Delete deltas after successful application by
.Xr ctm 1 .
.Xr ctm .
It is probably a good idea to avoid this flag (and keep all the deltas)
as one of the possible future enhancements to
.Xr ctm 1
.Xr ctm
is the ability to recover small groups of files from a full set of deltas.
.It Fl f
Fork and execute in the background while applying deltas with
.Xr ctm .
This is useful when automatically invoking
.Nm ctm_rmail
from
.Xr sendmail
because
.Xr ctm
can take a very long time to complete, causing other people's mail to
be delayed, and can in theory cause spurious
mail retransmission due to the remote
.Xr sendmail
timing out, or even termination of
.Nm ctm_rmail
by mail filters such as
.Xr "MH's"
.Xr slocal .
Don't worry about zillions of background
.Xr ctm
processes loading your machine, since locking is used to prevent more than one
.Xr ctm
invocation at a time.
.El
.Pp
The file arguments (or
@ -132,6 +155,15 @@ The file arguments (or
if there are none) are scanned for delta pieces. Multiple delta pieces
can be read from a single file, so an entire maildrop can be scanned
and processed with a single command.
.Pp
It is safe to invoke
.Nm ctm_rmail
multiple times concurrently (with different input files),
as might happen when
.Xr sendmail
.nh
is delivering mail asynchronously. This is because locking is used to
keep things orderly.
.Sh FILE FORMAT
Following are the important parts of an actual (very small) delta piece:
.Bd -literal
@ -174,7 +206,7 @@ You are then on your own!
To send delta 32 of
.Em src-cur
to a group of wonderful code hackers known to
.Xr sendmail 8
.Xr sendmail
as
.Em src-guys ,
limiting the mail size to roughly 60000 bytes, you could use:
@ -229,13 +261,15 @@ the window for mischief is quite small.
is careful to write only to the directories given to it (by not believing any
.Dq /
characters in the delta name), and the latest
.Nm ctm
disallows absolute pathnames in files it manipulates, so the worst you
.Xr ctm
disallows absolute pathnames and
.Dq \&\.\.
in files it manipulates, so the worst you
could lose are a few source tree files (recoverable from your deltas).
Since
.Nm ctm
.Xr ctm
requires that a
.Nm md5
.Xr md5
checksum match before it touches a file, only fellow
source recipients would be able to generate a fake delta, and they're such
nice folk that they wouldn't even think of it! :-)
@ -273,7 +307,7 @@ is expected to be called from a mail transfer program, and thus signals
failure only when the input mail message should be bounced (preferably into
your regular maildrop, not back to the sender). In short, failure to
apply a completed delta with
.Nm ctm
.Xr ctm
is not considered an error important enough to bounce the mail, and
.Nm ctm_rmail
returns an exit status of 0.
@ -293,10 +327,20 @@ ctm_rmail: src-cur.0250.gz 2/2 stored
ctm_rmail: src-cur.0250.gz complete
.Ed
.Pp
If any of the input files do not contain a valid delta piece,
.Nm ctm_rmail
will report:
.Bd -literal -offset indent
ctm_rmail: message contains no delta
.Ed
.sp \n(Ppu
and return an exit status of 1. You can use this to redirect wayward messages
back into your real mailbox if your mail filter goes wonky.
.Pp
These messages go to
.Em stderr
or to the log file. Messages from
.Nm ctm
.Xr ctm
turn up here too. Error messages should be self explanatory.
.\" The next request is for sections 2 and 3 error and signal handling only.
.\" .Sh ERRORS

245
usr.sbin/ctm/ctm_rmail/ctm_rmail.c
View File

@ -17,6 +17,8 @@
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <limits.h>
#include "error.h"
#include "options.h"
@ -30,7 +32,9 @@ int delete_after = 0; /* Delete deltas after ctm applies them. */
void apply_complete(void);
int read_piece(char *input_file);
int combine_if_complete(char *delta, int pce, int npieces);
int combine(char *delta, int npieces, char *dname, char *pname, char *tname);
int decode_line(char *line, char *out_buf);
int lock_file(char *name);
/*
* If given a '-p' flag, read encoded delta pieces from stdin or file
@ -51,11 +55,13 @@ main(int argc, char **argv)
{
char *log_file = NULL;
int status = 0;
int fork_ctm = 0;
err_prog_name(argv[0]);
OPTIONS("[-D] [-p piecedir] [-d deltadir] [-b basedir] [-l log] [file ...]")
OPTIONS("[-Df] [-p piecedir] [-d deltadir] [-b basedir] [-l log] [file ...]")
FLAG('D', delete_after)
FLAG('f', fork_ctm)
STRING('p', piece_dir)
STRING('d', delta_dir)
STRING('b', base_dir)
@ -71,6 +77,9 @@ main(int argc, char **argv)
if (log_file != NULL)
err_set_log(log_file);
/*
* Digest each file in turn, or just stdin if no files were given.
*/
if (argc <= 1)
{
if (piece_dir != NULL)
@ -82,8 +91,20 @@ main(int argc, char **argv)
status |= read_piece(*argv);
}
/*
* Maybe it's time to look for and apply completed deltas with ctm.
*
* Shall we report back to sendmail immediately, and let a child do
* the work? Sendmail will be waiting for us to complete, delaying
* other mail, and possibly some intermediate process (like MH slocal)
* will terminate us if we take too long!
*
* If fork() fails, it's unlikely we'll be able to run ctm, so give up.
* Also, the child exit status is unimportant.
*/
if (base_dir != NULL)
apply_complete();
if (!fork_ctm || fork() == 0)
apply_complete();
return status;
}
@ -109,23 +130,42 @@ void
apply_complete()
{
int i, dn;
int lfd;
FILE *fp, *ctm;
struct stat sb;
char class[20];
char delta[30];
char fname[1000];
char buf[2000];
char junk[2];
char here[1000];
char fname[PATH_MAX];
char here[PATH_MAX];
char buf[PATH_MAX*2];
/*
* Grab a lock on the ctm mutex file so that we can be sure we are
* working alone, not fighting another ctm_rmail!
*/
strcpy(fname, delta_dir);
strcat(fname, "/.mutex_apply");
if ((lfd = lock_file(fname)) < 0)
return;
/*
* Find out which delta ctm needs next.
*/
sprintf(fname, "%s/%s", base_dir, CTM_STATUS);
if ((fp = fopen(fname, "r")) == NULL)
{
close(lfd);
return;
}
i = fscanf(fp, "%s %d %c", class, &dn, junk);
fclose(fp);
if (i != 2)
{
close(lfd);
return;
}
/*
* We might need to convert the delta filename to an absolute pathname.
@ -151,14 +191,13 @@ apply_complete()
mk_delta_name(fname, delta);
if (stat(fname, &sb) < 0)
return;
break;
sprintf(buf, "(cd %s && /usr/sbin/ctm %s%s) 2>&1",
base_dir, here, fname);
sprintf(buf, "(cd %s && ctm %s%s) 2>&1", base_dir, here, fname);
if ((ctm = popen(buf, "r")) == NULL)
{
err("ctm failed to apply %s", delta);
return;
break;
}
while (fgets(buf, sizeof(buf), ctm) != NULL)
@ -172,7 +211,7 @@ apply_complete()
if (pclose(ctm) != 0)
{
err("ctm failed to apply %s", delta);
return;
break;
}
if (delete_after)
@ -180,6 +219,11 @@ apply_complete()
err("%s applied%s", delta, delete_after ? " and deleted" : "");
}
/*
* Closing the lock file clears the lock.
*/
close(lfd);
}
@ -204,6 +248,7 @@ read_piece(char *input_file)
int status = 0;
FILE *ifp, *ofp = 0;
int decoding = 0;
int got_one = 0;
int line_no = 0;
int i, n;
int pce, npieces;
@ -212,8 +257,8 @@ read_piece(char *input_file)
char out_buf[200];
char line[200];
char delta[30];
char pname[1000];
char tname[1000];
char pname[PATH_MAX];
char tname[PATH_MAX];
char junk[2];
ifp = stdin;
@ -241,8 +286,15 @@ read_piece(char *input_file)
while ((s = strchr(delta, '/')) != NULL)
*s = '_';
mk_piece_name(pname, delta, pce, npieces);
sprintf(tname,"%s.%d.tmp",pname,getpid());
got_one++;
strcpy(tname, piece_dir);
strcat(tname, "/p.XXXXXX");
if (mktemp(tname) == NULL)
{
err("*mktemp: '%s'", tname);
status++;
continue;
}
if ((ofp = fopen(tname, "w")) == NULL)
{
err("cannot open '%s' for writing", tname);
@ -282,9 +334,11 @@ read_piece(char *input_file)
continue;
}
mk_piece_name(pname, delta, pce, npieces);
if (rename(tname, pname) < 0)
{
err("error renaming %s to %s",tname,pname);
err("*rename: '%s' to '%s'", tname, pname);
err("%s %d/%d lost!", delta, pce, npieces);
unlink(tname);
status++;
continue;
@ -301,13 +355,13 @@ read_piece(char *input_file)
* Must be a line of encoded data. Decode it, sum it, and save it.
*/
n = decode_line(line, out_buf);
if (n < 0)
if (n <= 0)
{
err("line %d: illegal character: '%c'", line_no, line[-n]);
err("%s %d/%d discarded", delta, pce, npieces);
fclose(ofp);
unlink(pname);
unlink(tname);
status++;
decoding = 0;
@ -326,7 +380,7 @@ read_piece(char *input_file)
err("%s %d/%d discarded", delta, pce, npieces);
fclose(ofp);
unlink(pname);
unlink(tname);
status++;
}
@ -340,6 +394,12 @@ read_piece(char *input_file)
if (input_file != NULL)
fclose(ifp);
if (!got_one)
{
err("message contains no delta");
status++;
}
return (status != 0);
}
@ -351,32 +411,19 @@ read_piece(char *input_file)
int
combine_if_complete(char *delta, int pce, int npieces)
{
int i;
FILE *dfp, *pfp;
int c;
int i, e;
int lfd;
struct stat sb;
char pname[1000];
char dname[1000];
char pname[PATH_MAX];
char dname[PATH_MAX];
char tname[PATH_MAX];
/*
* All here?
*/
for (i = 1; i <= npieces; i++)
{
if (i == pce)
continue;
mk_piece_name(pname, delta, i, npieces);
if (stat(pname, &sb) < 0)
return 1;
}
mk_delta_name(dname, delta);
/*
* We can probably just rename() it in to place if it is a small delta.
* We can probably just rename() it into place if it is a small delta.
*/
if (npieces == 1)
{
mk_delta_name(dname, delta);
mk_piece_name(pname, delta, 1, 1);
if (rename(pname, dname) == 0)
{
@ -385,14 +432,70 @@ combine_if_complete(char *delta, int pce, int npieces)
}
}
if ((dfp = fopen(dname, "w")) == NULL)
/*
* Grab a lock on the reassembly mutex file so that we can be sure we are
* working alone, not fighting another ctm_rmail!
*/
strcpy(tname, delta_dir);
strcat(tname, "/.mutex_build");
if ((lfd = lock_file(tname)) < 0)
return 0;
/*
* Are all of the pieces present? Of course the current one is,
* unless all pieces are missing because another ctm_rmail has
* processed them already.
*/
for (i = 1; i <= npieces; i++)
{
err("cannot open '%s' for writing", dname);
if (i == pce)
continue;
mk_piece_name(pname, delta, i, npieces);
if (stat(pname, &sb) < 0)
{
close(lfd);
return 1;
}
}
/*
* Stick them together. Let combine() use our file name buffers, since
* we're such good buddies. :-)
*/
e = combine(delta, npieces, dname, pname, tname);
close(lfd);
return e;
}
/*
* Put the pieces together to form a delta.
* Returns 1 on success, and 0 on failure.
* Note: dname, pname, and tname are room for some file names that just
* happened to by lying around in the calling routine. Waste not, want not!
*/
int
combine(char *delta, int npieces, char *dname, char *pname, char *tname)
{
FILE *dfp, *pfp;
int i, n, e;
char buf[BUFSIZ];
strcpy(tname, delta_dir);
strcat(tname, "/d.XXXXXX");
if (mktemp(tname) == NULL)
{
err("*mktemp: '%s'", tname);
return 0;
}
if ((dfp = fopen(tname, "w")) == NULL)
{
err("cannot open '%s' for writing", tname);
return 0;
}
/*
* Ok, the hard way. Reconstruct the delta by reading each piece in order.
* Reconstruct the delta by reading each piece in order.
*/
for (i = 1; i <= npieces; i++)
{
@ -401,22 +504,38 @@ combine_if_complete(char *delta, int pce, int npieces)
{
err("cannot open '%s' for reading", pname);
fclose(dfp);
unlink(dname);
unlink(tname);
return 0;
}
while ((c = getc(pfp)) != EOF)
putc(c, dfp);
while ((n = fread(buf, sizeof(char), sizeof(buf), pfp)) != 0)
fwrite(buf, sizeof(char), n, dfp);
e = ferror(pfp);
fclose(pfp);
if (e)
{
err("error reading '%s'", pname);
fclose(dfp);
unlink(tname);
return 0;
}
}
fflush(dfp);
if (ferror(dfp))
e = ferror(dfp);
fclose(dfp);
if (e)
{
err("error writing '%s'", dname);
fclose(dfp);
unlink(dname);
err("error writing '%s'", tname);
unlink(tname);
return 0;
}
mk_delta_name(dname, delta);
if (rename(tname, dname) < 0)
{
err("*rename: '%s' to '%s'", tname, dname);
unlink(tname);
return 0;
}
fclose(dfp);
/*
* Throw the pieces away.
@ -424,7 +543,8 @@ combine_if_complete(char *delta, int pce, int npieces)
for (i = 1; i <= npieces; i++)
{
mk_piece_name(pname, delta, i, npieces);
unlink(pname);
if (unlink(pname) < 0)
err("*unlink: '%s'", pname);
}
err("%s complete", delta);
@ -500,3 +620,28 @@ decode_line(char *line, char *out_buf)
else
return -(ip - (unsigned char *)line);
}
/*
* Create and lock the given file.
*
* Clearing the lock is as simple as closing the file descriptor we return.
*/
int
lock_file(char *name)
{
int lfd;
if ((lfd = open(name, O_WRONLY|O_CREAT, 0600)) < 0)
{
err("*open: '%s'", name);
return -1;
}
if (flock(lfd, LOCK_EX) < 0)
{
close(lfd);
err("*flock: '%s'", name);
return -1;
}
return lfd;
}

27
usr.sbin/ctm/ctm_rmail/error.c
View File

@ -1,7 +1,22 @@
/*
* Routines for logging error messages or other informative messages.
*
* Log messages can easily contain the program name, a time stamp, system
* error messages, and arbitrary printf-style strings, and can be directed
* to stderr or a log file.
*
* Author: Stephen McKay
*
* NOTICE: This is free software. I hope you get some use from this program.
* In return you should think about all the nice people who give away software.
* Maybe you should write some free software too.
*/
#include <stdio.h>
#include <string.h>
#include <stdarg.h>
#include <time.h>
#include <errno.h>
#include "error.h"
static FILE *error_fp = NULL;
@ -38,6 +53,9 @@ err_prog_name(char *name)
/*
* Log an error.
*
* A leading '*' in the message format means we want the system errno
* decoded and appended.
*/
void
err(char *fmt, ...)
@ -46,6 +64,8 @@ err(char *fmt, ...)
time_t now;
struct tm *tm;
FILE *fp;
int x = errno;
int want_errno;
if ((fp = error_fp) == NULL)
{
@ -61,10 +81,17 @@ err(char *fmt, ...)
tm->tm_mon+1, tm->tm_mday, tm->tm_hour, tm->tm_min);
}
want_errno = 0;
if (*fmt == '*')
want_errno++, fmt++;
va_start(ap, fmt);
vfprintf(fp, fmt, ap);
va_end(ap);
if (want_errno)
fprintf(fp, ": %s", strerror(x));
fprintf(fp, "\n");
fflush(fp);
}