d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

This is the present state of CTM version 2. Please do not ask for

Browse Source 
subscriptions yet.  Wait for the announcement.

CTM is my humble attempt to get -current out to people beyond TCP/IP 
connections.  This is for people with dial-up connections and such.

CTM can make a delta from one version to another of a source-tree, in 
a efficient and verified way.  Even if there are binary files in the
tree.  It will even try to make the delta as small as possible.

It is OK with me if you yell "Bloating!" but I'll just forward your email
to some of the happy customers from CTM version 1, and let them tell you
what they think.

I will not put ctm into "make world" yet.  For now it is just the logical
way to get the sources out to people who helps me test this.

Poul-Henning
This commit is contained in:
Poul-Henning Kamp 1994-09-19 07:32:24 +00:00
parent 986c2270fe
commit ec5d3c392a
Notes: svn2git 2020-12-20 02:59:44 +00:00 
svn path=/cvs2svn/branches/phk/; revision=2886
16 changed files with 1304 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
usr.sbin/ctm/Makefile Normal file
View File

@ -0,0 +1,4 @@
SUBDIR= ctm ctm_scan
.include <bsd.subdir.mk>

97
usr.sbin/ctm/README Normal file
View File

@ -0,0 +1,97 @@
# ----------------------------------------------------------------------------
# "THE BEER-WARE LICENSE" (Revision 42):
# <phk@login.dknet.dk> wrote this file. As long as you retain this notice you
# can do whatever you want with this stuff. If we meet some day, and you think
# this stuff is worth it, you can buy me a beer in return. Poul-Henning Kamp
# ----------------------------------------------------------------------------
#
# $Id$
#
What will I not find in this file ?
-----------------------------------
Instructions on how to obtain FreeBSD via CTM.
Contact <phk@freefall.cdrom.com> for that.
What is CTM ?
-------------
CTM was originally "Cvs Through eMail", but has since changed scope to be
much more general.
CTM is now meant to be the definitive way to make and apply a delta between
two versions of a directory tree.
There are two parts to this, making the delta and applying it. These are two
entirely different things. CTM concentrates the computation-burden on the
generation og the deltas, as a delta very often is applied more times than
it is made. Second CTM tries to make the minimal size delta.
Why not use diff/patch ?
------------------------
Good question. Primarily because diff and patch doesn't do their job very
well. They don't deal with binary files (in this case files with '\0' or
'\0377' characters in them or files that doesn't end in '\n') which isn't
a big surprise: they were made to deal with text-files only. As a second
gripe, with patch you send the entire file to delete it. Not particular
efficient.
So what does CTM do exactly ?
-----------------------------
CTM will produce a file, (a delta) containing the instructions and data needed
to take another copy of the tree from the old to the new status. CTM means to
do this in the exact sense, and therefore the delta contains MD5 checksums to
verify that the tree it is applied to is indeed in the state CTM expects.
This means that if you have modified the tree locally, CTM might not be able
to upgrade your copy.
How do I make a CTM-delta ?
---------------------------
Don't. Send me email before you even try. This is yet not quite as trivial
as I would like. This is not to discourage you from using CTM, it is merely
to warn you that it is slightly tedious and takes much diskspace.
How do I apply a CTM-delta ?
----------------------------
You pass it to the 'ctm' command. You can pass a CTM-delta on stdin, or
you can give the filename as an argument. If you do the latter, you make
life a lot easier for your self, since the program can accept gzip'ed files
and since it will not have to make a temporary copy of your file. You can
specify multiple deltas at one time, they will be proccessed one at a time.
The ctm command runs in a number of passes. It will process the entire
input file in each pass, before commencing with the next pass.
Pass 1 will validate that the input file is OK. The syntax, the data and
the global MD5 checksum will be checked. If any of these fail, ctm will
never be able to do anything with the file, so it will simply reject it.
Pass 2 will validate that the directory tree is in the state expected by
the CTM-delta. This is done by looking for files and directories which
should/should not exists and by checking the MD5 checksums of files.
Pass 3 will actually apply the delta.
Should I delete the delta when I have applied it ?
--------------------------------------------------
No. You might want to selectively reconstruct a file latter on.
What features are are planned ?
-------------------------------
This list isn't exhaustive, and it isn't sorted in priority.
Reconstruct subset of tree.
Make tar-copy of things which will be affected.
Verify.
Internal editor instead of ed(1)
Support for mode
Support for uid/gid
Support for hardlinks
Support for symlinks
Isn't this a bit thin yet ?
---------------------------
Yes.
Can I say something ?
---------------------
Yes, email me: <phk@freefall.cdrom.com>
Poul-Henning

7
usr.sbin/ctm/ctm/Makefile Normal file
View File

@ -0,0 +1,7 @@
PROG= ctm_scan
NOTYET= ctm_ed.c
SRCS= ctm.c ctm_input.c ctm_pass1.c ctm_pass2.c ctm_pass3.c ctm_syntax.c
LDFLAGS+= -lmd
NOMAN= 1
.include <bsd.prog.mk>

171
usr.sbin/ctm/ctm/ctm.c Normal file
View File

@ -0,0 +1,171 @@
/* $Id$
*
* ----------------------------------------------------------------------------
* "THE BEER-WARE LICENSE" (Revision 42):
* <phk@login.dkuug.dk> wrote this file. As long as you retain this notice you
* can do whatever you want with this stuff. If we meet some day, and you think
* this stuff is worth it, you can buy me a beer in return. Poul-Henning Kamp
* ----------------------------------------------------------------------------
*
* $Id$
*
* This is the client program of 'CTM'. It will apply a CTM-patch to a
* collection of files.
*
* Options we'd like to see:
*
* -a Attempt best effort.
* -b <dir> Base-dir
* -B <file> Backup to tar-file.
* -c Check it out, "ma non troppo"
* -d <int> Debug TBD.
* -m <mail-addr> Email me instead.
* -p Less paranoid.
* -P Paranoid.
* -q Be quiet.
* -r <name> Reconstruct file.
* -R <file> Read list of files to reconstruct.
* -T <tmpdir>. Temporary files.
* -v Tell about each file.
*
* Exit-codes, bitmap, logical or of:
* 1 Couldn't do something we wanted to, not fatal.
* 2 Couldn't do something we wanted to, fatal.
* 4 Input file corrupt.
* 8 Cannot apply input file.
* 16 Corruption while applying input file.
*
*/
#define EXTERN /* */
#include "ctm.h"
extern int Proc(char *);
int
main(int argc, char **argv)
{
int stat=0;
int i,j,c;
extern int optopt,optind;
extern char * optarg;
Verbose = 1;
Paranoid = 1;
setbuf(stderr,0);
setbuf(stdout,0);
while((c=getopt(argc,argv,"ab:B:cd:m:pPqr:R:T:Vv")) != -1) {
switch (c) {
case 'p': Paranoid--; break; /* Less Paranoid */
case 'P': Paranoid++; break; /* More Paranoid */
case 'q': Verbose--; break; /* Quiet */
case 'v': Verbose++; break; /* Verbose */
case 'T': TmpDir = optarg; break;
case ':':
fprintf(stderr,"Option '%c' requires an argument.\n",optopt);
stat++;
break;
case '?':
fprintf(stderr,"Option '%c' not supported.\n",optopt);
stat++;
break;
default:
fprintf(stderr,"Option '%c' not yet implemented.\n",optopt);
break;
}
}
if(stat) {
fprintf(stderr,"%d errors during option processing\n",stat);
exit(1);
}
stat = 0;
argc -= optind;
argv += optind;
if(!argc)
stat |= Proc("-");
while(argc--)
stat |= Proc(*argv++);
return stat;
}
int
Proc(char *filename)
{
FILE *f;
int i;
char *p = strrchr(filename,'.');
if(!strcmp(filename,"-")) {
p = 0;
f = stdin;
} else if(!strcmp(p,".gz") || !strcmp(p,".Z")) {
p = Malloc(100);
strcpy(p,"gunzip < ");
strcat(p,filename);
f = popen(p,"r");
} else {
p = 0;
f = fopen(filename,"r");
}
if(!f) {
perror(filename);
return 1;
}
if(Verbose > 1)
fprintf(stderr,"Working on <%s>\n",filename);
if(FileName) Free(FileName);
FileName = String(filename);
/* If we cannot seek, we're doomed, so copy to a tmp-file in that case */
if(!p && -1 == fseek(f,0,SEEK_END)) {
char *fn = tempnam(NULL,"CMTclient");
FILE *f2 = fopen(fn,"w+");
int i;
if(!f2) {
perror(fn);
fclose(f);
return 2;
}
unlink(fn);
fprintf(stderr,"Writing tmp-file \"%s\"\n",fn);
while(EOF != (i=getc(f)))
putc(i,f2);
fclose(f);
f = f2;
}
if(!p)
rewind(f);
if((i=Pass1(f)))
return i;
if(!p) {
rewind(f);
} else {
pclose(f);
f = popen(p,"r");
}
if((i=Pass2(f)))
return i;
if(!p) {
rewind(f);
} else {
pclose(f);
f = popen(p,"r");
}
if((i=Pass3(f)))
return i;
if(!p) {
fclose(f);
} else {
pclose(f);
}
return 0;
}

101
usr.sbin/ctm/ctm/ctm.h Normal file
View File

@ -0,0 +1,101 @@
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <md5.h>
#include <ctype.h>
#include <string.h>
#include <malloc.h>
#include <errno.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/file.h>
/*
* We redefine the names to make it look nice...
*/
#define VERSION "2.0"
#define MAXSIZE (1024*1024*10)
/* The fields... */
#define CTM_F_MASK 0xff
#define CTM_F_Name 0x01
#define CTM_F_Uid 0x02
#define CTM_F_Gid 0x03
#define CTM_F_Mode 0x04
#define CTM_F_MD5 0x05
#define CTM_F_Count 0x06
#define CTM_F_Bytes 0x07
/* The qualifiers... */
#define CTM_Q_MASK 0xff00
#define CTM_Q_Name_File 0x0100
#define CTM_Q_Name_Dir 0x0200
#define CTM_Q_Name_New 0x0400
#define CTM_Q_MD5_After 0x0100
#define CTM_Q_MD5_Before 0x0200
#define CTM_Q_MD5_Chunk 0x0400
struct CTM_Syntax {
char *Key;
int *List;
};
extern struct CTM_Syntax Syntax[];
#define Malloc malloc
#define Free free
#ifndef EXTERN
# define EXTERN extern
#endif
EXTERN u_char *Version;
EXTERN u_char *Name;
EXTERN u_char *Nbr;
EXTERN u_char *TimeStamp;
EXTERN u_char *Prefix;
EXTERN u_char *FileName;
EXTERN u_char *BaseDir;
EXTERN u_char *TmpDir;
EXTERN int Verbose;
/*
* Paranoid -- Just in case they should be after us...
* 0 not at all.
* 1 normal.
* 2 somewhat.
* 3 you bet!.
*
* Verbose -- What to tell mom...
* 0 Nothing which wouldn't surprise.
* 1 Normal.
* 2 Show progress '.'.
* 3 Show progress names, and actions.
* 4 even more...
* and so on
*/
EXTERN int Paranoid;
char * String(char *s);
void Fatal_(int ln, char *fn, char *kind);
#define Fatal(foo) Fatal_(__LINE__,__FILE__,foo)
#define Assert() Fatal_(__LINE__,__FILE__,"Assert failed.")
#define WRONG {Assert(); return 1;}
u_char * Ffield(FILE *fd, MD5_CTX *ctx,u_char term);
int Fbytecnt(FILE *fd, MD5_CTX *ctx, u_char term);
u_char * Fdata(FILE *fd, int u_chars, MD5_CTX *ctx);
#define GETFIELD(p,q) if(!((p)=Ffield(fd,&ctx,(q)))) return 1
#define GETFIELDCOPY(p,q) if(!((p)=Ffield(fd,&ctx,(q)))) return 1; else p=String(p)
#define GETBYTECNT(p,q) if(0 >((p)= Fbytecnt(fd,&ctx,(q)))) return 1
#define GETDATA(p,q) if(!((p) = Fdata(fd,(q),&ctx))) return 1
int Pass1(FILE *fd);
int Pass2(FILE *fd);
int Pass3(FILE *fd);

80
usr.sbin/ctm/ctm/ctm_ed.c Normal file
View File

@ -0,0 +1,80 @@
int
ctm_edit(u_char *script, int length, char *filename, char *md5)
{
u_char *ep, cmd, c;
int ln, ln2, iln;
FILE *fi,*fo;
char buf[BUFSIZ];
fi = fopen(filename,"r");
if(!fi) {
/* XXX */
return 1;
}
strcpy(buf,filename);
strcat(buf,".ctm");
fo = fopen(filename,"w");
if(!fo) {
/* XXX */
return 1;
}
iln = 0;
for(ep=script;ep < script+length;) {
cmd = *ep++;
if(cmd != 'a' && cmd != 'd') ARGH
ln = 0;
while(isdigit(*ep)) {
ln *= 10;
ln += (*ep++ - '0');
}
if(*ep++ != ' ') BARF
ln2 = 0;
while(isdigit(*ep)) {
ln2 *= 10;
ln2 += (*ep++ - '0');
}
if(*ep++ != '\n') BARF
while(iln < ln) {
c = getf(fi);
putc(c,fo);
if(c == '/n')
iln++;
}
if(cmd == 'd') {
while(ln2) {
c = getf(fi);
if(c != '/n')
continue;
iln++;
ln2--;
}
continue;
}
if(cmd == 'a') {
while(ln2) {
c = *ep++;
putc(c,fo);
if(c != '/n')
continue;
ln2--;
}
continue;
}
ARGH
}
while(1) {
c = getf(fi);
if(c == EOF) break;
putc(c,fo);
}
fclose(fi);
fclose(fo);
if(strcmp(md5,MD5File(buf))) {
unlink(buf);
return 1; /*XXX*/
}
if(rename(buf,filename)) {
unlink(buf);
return 1; /*XXX*/
}
}

95
usr.sbin/ctm/ctm/ctm_input.c Normal file
View File

@ -0,0 +1,95 @@
#include "ctm.h"
/*---------------------------------------------------------------------------*/
char *
String(char *s)
{
char *p = malloc(strlen(s) + 1);
strcpy(p,s);
return p;
}
/*---------------------------------------------------------------------------*/
void
Fatal_(int ln, char *fn, char *kind)
{
if(Verbose > 2)
fprintf(stderr,"Fatal error. (%s:%d)\n",fn,ln);
fprintf(stderr,"%s Fatal error: %s\n",FileName, kind);
}
#define Fatal(foo) Fatal_(__LINE__,__FILE__,foo)
#define Assert() Fatal_(__LINE__,__FILE__,"Assert failed.")
/*---------------------------------------------------------------------------*/
/* get next field, check that the terminating whitespace is what we expect */
u_char *
Ffield(FILE *fd, MD5_CTX *ctx,u_char term)
{
static u_char buf[BUFSIZ];
int i,l;
for(l=0;;) {
if((i=getc(fd)) == EOF) {
Fatal("Truncated patch.");
return 0;
}
buf[l++] = i;
if(isspace(i))
break;
if(l >= sizeof buf) {
Fatal("Corrupt patch.");
printf("Token is too long.\n");
return 0;
}
}
buf[l] = '\0';
MD5Update(ctx,buf,l);
if(buf[l-1] != term) {
Fatal("Corrupt patch.");
fprintf(stderr,"Expected \"%s\" but didn't find it.\n",
term == '\n' ? "\\n" : " ");
return 0;
}
buf[--l] = '\0';
return buf;
}
int
Fbytecnt(FILE *fd, MD5_CTX *ctx, u_char term)
{
u_char *p,*q;
int u_chars;
p = Ffield(fd,ctx,term);
if(!p) return -1;
for(q=p;*q;q++)
if(!isdigit(*q)) {
Fatal("Bytecount contains non-digit.");
return -1;
}
u_chars=atoi(p);
if(u_chars > MAXSIZE) {
Fatal("Bytecount too large.");
return -1;
}
return u_chars;
}
u_char *
Fdata(FILE *fd, int u_chars, MD5_CTX *ctx)
{
u_char *p = Malloc(u_chars+1);
if(u_chars+1 != fread(p,1,u_chars+1,fd)) {
Fatal("Truncated patch.");
return 0;
}
MD5Update(ctx,p,u_chars+1);
if(p[u_chars] != '\n') {
if(Verbose > 3)
printf("FileData wasn't followed by a newline.\n");
Fatal("Corrupt patch.");
return 0;
}
p[u_chars] = '\0';
return p;
}

127
usr.sbin/ctm/ctm/ctm_pass1.c Normal file
View File

@ -0,0 +1,127 @@
#include "ctm.h"
/*---------------------------------------------------------------------------*/
/* Pass1 -- Validate the incomming CTM-file.
*/
int
Pass1(FILE *fd)
{
u_char *p,*q;
MD5_CTX ctx;
int i,j,sep,cnt;
u_char *md5=0,*trash=0;
struct CTM_Syntax *sp;
if(Verbose>3)
printf("Pass1 -- Checking integrity of incomming CTM-patch\n");
MD5Init (&ctx);
GETFIELD(p,' '); /* CTM_BEGIN */
if(strcmp(p,"CTM_BEGIN")) {
Fatal("Probably not a CTM-patch at all.");
if(Verbose>3)
fprintf(stderr,"Expected \"CTM_BEGIN\" got \"%s\".\n",p);
return 1;
}
GETFIELDCOPY(Version,' '); /* <Version> */
if(strcmp(Version,VERSION)) {
Fatal("CTM-patch is wrong version.");
if(Verbose>3)
fprintf(stderr,"Expected \"%s\" got \"%s\".\n",VERSION,p);
return 1;
}
GETFIELDCOPY(Name,' '); /* <Name> */
GETFIELDCOPY(Nbr,' '); /* <Nbr> */
GETFIELDCOPY(TimeStamp,' '); /* <TimeStamp> */
GETFIELDCOPY(Prefix,'\n'); /* <Prefix> */
for(;;) {
if(md5) {Free(md5), md5 = 0;}
if(trash) {Free(trash), trash = 0;}
cnt = -1;
GETFIELD(p,' '); /* CTM_something */
if (p[0] != 'C' || p[1] != 'T' || p[2] != 'M') {
Fatal("Expected CTM keyword.");
fprintf(stderr,"Got [%s]\n",p);
return 1;
}
if(!strcmp(p+3,"_END"))
break;
for(sp=Syntax;sp->Key;sp++)
if(!strcmp(p+3,sp->Key))
goto found;
Fatal("Expected CTM keyword.");
fprintf(stderr,"Got [%s]\n",p);
return 1;
found:
if(Verbose > 5)
fprintf(stderr,"%s ",sp->Key);
for(i=0;(j = sp->List[i]);i++) {
if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes)
sep = ' ';
else
sep = '\n';
if(Verbose > 5)
fprintf(stderr," %x(%d)",sp->List[i],sep);
switch (j & CTM_F_MASK) {
case CTM_F_Name: /* XXX check for garbage and .. */
case CTM_F_Uid: /* XXX check for garbage */
case CTM_F_Gid: /* XXX check for garbage */
case CTM_F_Mode: /* XXX check for garbage */
GETFIELD(p,sep);
break;
case CTM_F_MD5:
if(j & CTM_Q_MD5_Chunk)
GETFIELDCOPY(md5,sep); /* XXX check for garbage */
else
GETFIELD(p,sep); /* XXX check for garbage */
break;
case CTM_F_Count:
GETBYTECNT(cnt,sep); /* XXX check for garbage */
break;
case CTM_F_Bytes:
if(cnt < 0) WRONG
GETDATA(trash,cnt);
p = MD5Data(trash,cnt);
if(md5 && strcmp(md5,p)) {
Fatal("Internal MD5 failed.");
return 1;
default:
fprintf(stderr,"List = 0x%x\n",j);
Fatal("List had garbage.");
return 1;
}
}
}
if(Verbose > 5)
putc('\n',stderr);
continue;
}
q = MD5End (&ctx);
if(Verbose > 2)
printf("Expecting Global MD5 <%s>\n",q);
GETFIELD(p,'\n'); /* <MD5> */
if(Verbose > 2)
printf("Reference Global MD5 <%s>\n",p);
if(strcmp(q,p)) {
Fatal("MD5 sum doesn't match.");
fprintf(stderr,"\tI have:<%s>\n",q);
fprintf(stderr,"\tShould have been:<%s>\n",p);
return 1;
}
if (-1 != getc(fd)) {
Fatal("Trailing junk in CTM-file.");
return 1;
}
return 0;
}

116
usr.sbin/ctm/ctm/ctm_pass2.c Normal file
View File

@ -0,0 +1,116 @@
#include "ctm.h"
/*---------------------------------------------------------------------------*/
/* Pass2 -- Validate the incomming CTM-file.
*/
int
Pass2(FILE *fd)
{
u_char *p,*q;
MD5_CTX ctx;
int i,j,sep,cnt;
u_char *trash=0,*name=0;
struct CTM_Syntax *sp;
struct stat st;
if(Verbose>3)
printf("Pass2 -- Checking if CTM-patch will apply\n");
MD5Init (&ctx);
GETFIELD(p,' '); if(strcmp("CTM_BEGIN",p)) WRONG
GETFIELD(p,' '); if(strcmp(Version,p)) WRONG
GETFIELD(p,' '); if(strcmp(Name,p)) WRONG
/* XXX Lookup name in /etc/ctm,conf, read stuff */
GETFIELD(p,' '); if(strcmp(Nbr,p)) WRONG
/* XXX Verify that this is the next patch to apply */
GETFIELD(p,' '); if(strcmp(TimeStamp,p)) WRONG
GETFIELD(p,'\n'); if(strcmp(Prefix,p)) WRONG
/* XXX drop or use ? */
for(;;) {
if(trash) {Free(trash), trash = 0;}
if(name) {Free(name), name = 0;}
cnt = -1;
GETFIELD(p,' ');
if (p[0] != 'C' || p[1] != 'T' || p[2] != 'M') WRONG
if(!strcmp(p+3,"_END"))
break;
for(sp=Syntax;sp->Key;sp++)
if(!strcmp(p+3,sp->Key))
goto found;
WRONG
found:
for(i=0;(j = sp->List[i]);i++) {
if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes)
sep = ' ';
else
sep = '\n';
switch (j & CTM_F_MASK) {
case CTM_F_Name:
GETFIELDCOPY(name,sep);
/* XXX Check DR DM rec's for parent-dir */
if(j & CTM_Q_Name_New) {
/* XXX Check DR FR rec's for item */
if(-1 != stat(name,&st)) {
fprintf(stderr," %s: %s exists.\n",sp->Key,name);
}
break;
}
if(-1 == stat(name,&st)) {
fprintf(stderr," %s: %s doesn't exists.\n",
sp->Key,name);
break;
}
if (j & CTM_Q_Name_Dir) {
if((st.st_mode & S_IFMT) != S_IFDIR)
fprintf(stderr,
" %s: %s exist, but isn't dir.\n",
sp->Key,name);
break;
}
if (j & CTM_Q_Name_File) {
if((st.st_mode & S_IFMT) != S_IFREG)
fprintf(stderr,
" %s: %s exist, but isn't file.\n",
sp->Key,name);
break;
}
break;
case CTM_F_Uid:
case CTM_F_Gid:
case CTM_F_Mode:
GETFIELD(p,sep);
break;
case CTM_F_MD5:
if(!name) WRONG
GETFIELD(p,sep);
if((st.st_mode & S_IFMT) == S_IFREG) {
if(j & CTM_Q_MD5_Before && strcmp(MD5File(name),p)) {
fprintf(stderr," %s: %s md5 mismatch.\n",sp->Key,name);
}
}
break;
case CTM_F_Count:
GETBYTECNT(cnt,sep);
break;
case CTM_F_Bytes:
if(cnt < 0) WRONG
GETDATA(trash,cnt);
p = MD5Data(trash,cnt);
break;
default: WRONG
}
}
}
q = MD5End (&ctx);
GETFIELD(p,'\n'); /* <MD5> */
if(strcmp(q,p)) WRONG
if (-1 != getc(fd)) WRONG
return 0;
}

142
usr.sbin/ctm/ctm/ctm_pass3.c Normal file
View File

@ -0,0 +1,142 @@
#include "ctm.h"
/*---------------------------------------------------------------------------*/
/* Pass3 -- Validate the incomming CTM-file.
*/
int
Pass3(FILE *fd)
{
u_char *p,*q,buf[BUFSIZ];
MD5_CTX ctx;
int i,j,sep,cnt;
u_char *md5=0,*md5before=0,*trash=0,*name=0,*uid=0,*gid=0,*mode=0;
struct CTM_Syntax *sp;
FILE *ed=0;
struct stat st;
if(Verbose>3)
printf("Pass3 -- Applying the CTM-patch\n");
MD5Init (&ctx);
GETFIELD(p,' '); if(strcmp("CTM_BEGIN",p)) WRONG
GETFIELD(p,' '); if(strcmp(Version,p)) WRONG
GETFIELD(p,' '); if(strcmp(Name,p)) WRONG
GETFIELD(p,' '); if(strcmp(Nbr,p)) WRONG
GETFIELD(p,' '); if(strcmp(TimeStamp,p)) WRONG
GETFIELD(p,'\n'); if(strcmp(Prefix,p)) WRONG
for(;;) {
if(md5) {Free(md5), md5 = 0;}
if(uid) {Free(uid), uid = 0;}
if(gid) {Free(gid), gid = 0;}
if(mode) {Free(mode), mode = 0;}
if(md5before) {Free(md5before), md5before = 0;}
if(trash) {Free(trash), trash = 0;}
if(name) {Free(name), name = 0;}
cnt = -1;
GETFIELD(p,' ');
if (p[0] != 'C' || p[1] != 'T' || p[2] != 'M') WRONG
if(!strcmp(p+3,"_END"))
break;
for(sp=Syntax;sp->Key;sp++)
if(!strcmp(p+3,sp->Key))
goto found;
WRONG
found:
for(i=0;(j = sp->List[i]);i++) {
if (sp->List[i+1] && (sp->List[i+1] & CTM_F_MASK) != CTM_F_Bytes)
sep = ' ';
else
sep = '\n';
switch (j & CTM_F_MASK) {
case CTM_F_Name: GETFIELDCOPY(name,sep); break;
case CTM_F_Uid: GETFIELDCOPY(uid,sep); break;
case CTM_F_Gid: GETFIELDCOPY(gid,sep); break;
case CTM_F_Mode: GETFIELDCOPY(mode,sep); break;
case CTM_F_MD5:
if(j & CTM_Q_MD5_Before)
GETFIELDCOPY(md5before,sep);
else
GETFIELDCOPY(md5,sep);
break;
case CTM_F_Count: GETBYTECNT(cnt,sep); break;
case CTM_F_Bytes: GETDATA(trash,cnt); break;
default: WRONG
}
}
j = strlen(name)-1;
if(name[j] == '/') name[j] = '\0';
fprintf(stderr,"> %s %s\n",sp->Key,name);
if(!strcmp(sp->Key,"FM") || !strcmp(sp->Key, "FS")) {
i = open(name,O_WRONLY|O_CREAT|O_TRUNC,0644);
if(i < 0) {
perror(name);
continue;
}
if(cnt != write(i,trash,cnt)) {
perror(name);
continue;
}
close(i);
if(strcmp(md5,MD5File(name))) {
fprintf(stderr," %s %s MD5 didn't come out right\n",
sp->Key,name);
continue;
}
continue;
}
if(!strcmp(sp->Key,"FE")) {
ed = popen("ed","w");
if(!ed) {
WRONG
}
fprintf(ed,"e %s\n",name);
if(cnt != fwrite(trash,1,cnt,ed)) {
perror(name);
pclose(ed);
continue;
}
fprintf(ed,"w %s\n",name);
if(pclose(ed)) {
perror("ed");
continue;
}
if(strcmp(md5,MD5File(name))) {
fprintf(stderr," %s %s MD5 didn't come out right\n",
sp->Key,name);
continue;
}
continue;
}
if(!strcmp(sp->Key,"DM")) {
if(0 > mkdir(name,0755)) {
sprintf(buf,"mkdir -p %s",name);
system(buf);
}
if(0 > stat(name,&st) || ((st.st_mode & S_IFMT) != S_IFDIR)) {
fprintf(stderr,"<%s> mkdir failed\n",name);
exit(1);
}
continue;
}
if(!strcmp(sp->Key,"DR") || !strcmp(sp->Key,"FR")) {
if(0 > unlink(name)) {
sprintf(buf,"rm -rf %s",name);
system(buf);
}
continue;
}
WRONG
}
q = MD5End (&ctx);
GETFIELD(p,'\n');
if(strcmp(q,p)) WRONG
if (-1 != getc(fd)) WRONG
return 0;
}

53
usr.sbin/ctm/ctm/ctm_syntax.c Normal file
View File

@ -0,0 +1,53 @@
/*
* We redefine the names to make it look nice...
*/
#include "ctm.h"
/* The fields... */
#define Name CTM_F_Name
#define Uid CTM_F_Uid
#define Gid CTM_F_Gid
#define Mode CTM_F_Mode
#define MD5 CTM_F_MD5
#define Count CTM_F_Count
#define Bytes CTM_F_Bytes
/* The qualifiers... */
#define File CTM_Q_Name_File
#define Dir CTM_Q_Name_Dir
#define New CTM_Q_Name_New
#define After CTM_Q_MD5_After
#define Before CTM_Q_MD5_Before
#define Chunk CTM_Q_MD5_Chunk
static int ctmFM[] = /* File Make */
{ Name|File|New, Uid, Gid, Mode, MD5|After|Chunk, Count, Bytes,0 };
static int ctmFS[] = /* File Substitute */
{ Name|File, Uid, Gid, Mode, MD5|Before, MD5|After|Chunk, Count, Bytes,0 };
static int ctmFE[] = /* File Edit */
{ Name|File, Uid, Gid, Mode, MD5|Before, MD5|After, Count, Bytes,0 };
static int ctmFR[] = /* File Remove */
{ Name|File, MD5|Before, 0 };
static int ctmAS[] = /* Attribute Substitute */
{ Name, Uid, Gid, Mode, 0 };
static int ctmDM[] = /* Directory Make */
{ Name|Dir|New , Uid, Gid, Mode, 0 };
static int ctmDR[] = /* Directory Remove */
{ Name|Dir, 0 };
struct CTM_Syntax Syntax[] = {
{ "FM", ctmFM },
{ "FS", ctmFS },
{ "FE", ctmFE },
{ "FR", ctmFR },
{ "AS", ctmAS },
{ "DM", ctmDM },
{ "DR", ctmDR },
{ 0, 0} };

5
usr.sbin/ctm/ctm_scan/Makefile Normal file
View File

@ -0,0 +1,5 @@
PROG= ctm_scan
LDFLAGS+= -lmd
NOMAN= 1
.include <bsd.prog.mk>

141
usr.sbin/ctm/ctm_scan/ctm_scan.c Normal file
View File

@ -0,0 +1,141 @@
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/file.h>
#include <dirent.h>
#include <md5.h>
int barf[256];
int
pstrcmp(char **p, char **q)
{
return strcmp(*p,*q);
}
int
Do(char *path)
{
DIR *d;
struct dirent *de;
struct stat st;
int ret=0;
u_char buf[BUFSIZ];
u_char data[BUFSIZ],*q;
int bufp;
MD5_CTX ctx;
int fd,i,j,k,l,npde,nde=0;
char **pde;
npde = 1;
pde = malloc(sizeof *pde * (npde+1));
d = opendir(path);
if(!d) { perror(path); return 2; }
if(!strcmp(path,".")) {
*buf = 0;
} else {
strcpy(buf,path);
if(buf[strlen(buf)-1] != '/')
strcat(buf,"/");
}
bufp = strlen(buf);
while((de=readdir(d))) {
if(!strcmp(de->d_name,".")) continue;
if(!strcmp(de->d_name,"..")) continue;
if(nde >= npde) {
npde *= 2;
pde = realloc(pde,sizeof *pde * (npde+1));
}
strcpy(buf+bufp,de->d_name);
if(stat(buf,&st)) {
ret |= 1;
continue;
}
if((st.st_mode & S_IFMT) == S_IFDIR) {
strcat(buf,"/");
}
pde[nde] = malloc(strlen(buf+bufp)+1);
strcpy(pde[nde++],buf+bufp);
}
closedir(d);
if(!nde) return 0;
qsort(pde,nde,sizeof *pde,pstrcmp);
for(k=0;k<nde;k++) {
strcpy(buf+bufp,pde[k]);
free(pde[k]);
if(stat(buf,&st)) {
ret |= 1;
continue;
}
switch(st.st_mode & S_IFMT) {
case S_IFDIR:
i = printf("d %s %o %d %d - - -\n",
buf,st.st_mode & (~S_IFMT),st.st_uid,st.st_gid);
if(!i)
exit(-1);
ret |= Do(buf);
break;
case S_IFREG:
fd = open(buf,O_RDONLY);
if(fd < 0) {
ret |= 1;
continue;
}
MD5Init(&ctx);
l = j = 0;
while(0 < (i = read(fd,data,sizeof data))) {
l = (data[i-1] == '\n');
MD5Update(&ctx,data,i);
for(q=data;i && !j;i--)
if(barf[*q++])
j=1;
}
if(!l)
j=1;
close(fd);
i = printf("f %s %o %d %d %d %d %s\n",
buf,st.st_mode & (~S_IFMT),st.st_uid,st.st_gid,
j,st.st_size,MD5End(&ctx));
if(!i)
exit(-1);
break;
default:
fprintf(stderr,"%s: type 0%o\n",buf, st.st_mode & S_IFMT);
ret |= 4;
break;
}
}
free(pde);
return ret;
}
int
main(int argc, char **argv)
{
/*
* Initialize barf[], characters diff/patch will not appreciate.
*/
barf[0x00] = 1;
barf[0x7f] = 1;
barf[0x80] = 1;
barf[0xff] = 1;
/*
* First argument, if any, is where to do the work.
*/
if (argc > 1) {
if(chdir(argv[1])) {
perror(argv[1]);
return 2;
}
}
/*
* Scan the directories recursively.
*/
return Do(".");
}

9
usr.sbin/ctm/mkCTM/ctm_conf.cvs-cur Normal file
View File

@ -0,0 +1,9 @@
#!/usr/local/bin/tclsh
set CTMname cvs-cur
set CTMdest /u4/CTM
set CTMref /u1/CVS-FreeBSD
set CTMprefix .
set CTMcopy $CTMdest/$CTMname
set CTMtmp $CTMdest/_tmp_$CTMname
set CTMdate [exec date -u +%Y%m%d%H%M%SZ]

9
usr.sbin/ctm/mkCTM/ctm_conf.src-cur Normal file
View File

@ -0,0 +1,9 @@
#!/usr/local/bin/tclsh
set CTMname src-cur
set CTMdest /u1/CTM
set CTMref /u4/ftp/pub/FreeBSD/SRC-current/src
set CTMprefix .
set CTMcopy $CTMdest/$CTMname
set CTMtmp $CTMdest/_tmp_$CTMname
set CTMdate [exec date -u +%Y%m%d%H%M%SZ]

147
usr.sbin/ctm/mkCTM/mkCTM Normal file
View File

@ -0,0 +1,147 @@
#!/usr/local/bin/tclsh
source $argv
set tmp $CTMtmp
set dd $CTMdest
set d1 $CTMcopy
set d2 $CTMref
set foo $CTMdate
set foo $CTMprefix
set foo $CTMname
exec rm -f $tmp.*
set f1 [open "| ./ctm_scan $d1"]
set f2 [open "| ./ctm_scan $d2"]
set fo_del [open $tmp.del w]
set fo_rmdir [open $tmp.rmdir w]
set fo_mkdir [open $tmp.mkdir w]
set fo_files [open $tmp.files w]
set changes 0
####
# Find CTM#
for {set i 0} {1} {incr i} {
if {[file exists [format "%s/$CTMname.%04d" $dd $i]]} continue
if {[file exists [format "%s/$CTMname.%04d.gz" $dd $i]]} continue
break
}
set CTMnbr $i
puts "Doing CTMname $CTMname CTMnbr $CTMnbr CTMdate $CTMdate"
#####
# Type Name Mode User Group Barf Size Hash
proc CTMadd {t n m u g b s h} {
global fo_files fo_mkdir changes d2
puts stderr "A $b $t $n"
if {$t == "d"} {
puts $fo_mkdir "CTMDM $n $u $g $m"
incr changes
return
}
puts $fo_files "CTMFM $n $u $g $m $h $s"
flush $fo_files
exec cat $d2/$n >@ $fo_files
puts $fo_files ""
incr changes
return
}
proc CTMdel {t n m u g b s h} {
global fo_del fo_rmdir changes
puts stderr "D $b $t $n"
if {$t == "d"} {
puts $fo_rmdir "CTMDR $n"
incr changes
return
}
puts $fo_del "CTMFR $n $h"
incr changes
return
}
proc CTMchg {t1 n1 m1 u1 g1 b1 s1 h1 t2 n2 m2 u2 g2 b2 s2 h2} {
global fo_files d2 d1 changes
if {$t1 == "d" && $t2 == "d"} {
return
}
if {$t1 == "d" || $t2 == "d"} {
CTMdel $t1 $n1 $m1 $u1 $g1 $b1 $s1 $h1
CTMadd $t2 $n2 $m2 $u2 $g2 $b2 $s2 $h2
return
}
if {"x$h1" == "x$h2" && $s1 == $s2} {
return
puts stderr "M $b1$b2 $t1$t2 $n1"
puts $fo_files "CTMFA $n2 $u2 $g2 $m2 $h2"
incr changes
return
}
if {$b1 != "0" || $b2 != "0"} {
puts stderr "R $b1$b2 $t1$t2 $n1"
puts $fo_files "CTMFS $n2 $u2 $g2 $m2 $h1 $h2 $s2"
flush $fo_files
exec cat $d2/$n2 >@ $fo_files
puts $fo_files ""
incr changes
return
}
puts stderr "E $b1$b2 $t1$t2 $n1"
set i [catch "exec diff -e $d1/$n1 $d2/$n2 > tmp" j]
set s [file size tmp]
puts $fo_files "CTMFE $n1 $u2 $g2 $m2 $h1 $h2 $s"
flush $fo_files
exec cat tmp >@ $fo_files
puts $fo_files ""
incr changes
}
#####
set l1 ""
set l2 ""
while 1 {
if {$l1 == ""} {gets $f1 l1}
if {$l2 == ""} {gets $f2 l2}
if {$l1 == "" && $l2 == ""} break
set n1 [lindex $l1 1]
set n2 [lindex $l2 1]
if {$l1 == $l2} { set l1 "" ; set l2 "" ; continue }
if {$l1 == "" } { eval CTMadd $l2 ; set l2 "" ; continue }
if {$l2 == "" } { eval CTMdel $l1 ; set l1 "" ; continue }
if {$n1 < $n2 } { eval CTMdel $l1 ; set l1 "" ; continue }
if {$n1 > $n2 } { eval CTMadd $l2 ; set l2 "" ; continue }
if {$n1 == $n2} { eval CTMchg $l1 $l2 ; set l1 "" ; set l2 "" ; continue }
}
close $fo_del
close $fo_rmdir
close $fo_mkdir
close $fo_files
exec echo CTM_BEGIN 2.0 $CTMname $CTMnbr $CTMdate $CTMprefix > $tmp.begin
exec echo -n "CTM_END " >> $tmp.end
set m [exec cat $tmp.begin $tmp.del $tmp.rmdir $tmp.mkdir $tmp.files $tmp.end | md5]
exec echo "$m" >> $tmp.end
if {!$changes} {
puts "no changes"
exec sh -c "rm -f $tmp.*"
exit 0
}
set nm [format "%s/%s.%04d" $dd $CTMname $CTMnbr]
exec cat $tmp.begin $tmp.del $tmp.rmdir $tmp.mkdir $tmp.files $tmp.end | gzip -9 -v > ${nm}.gz 2>@ stdout
exec sh -c "rm -f $tmp.*"
exec sh -e -x -c "cd $CTMcopy ; /root/CTM/ctm -v -v -v ${nm}.gz" >&@ stdout