associated changes that had to happen to make this possible as well as
bugs fixed along the way.
Bring in required TLI library routines to support this.
Since we don't support TLI we've essentially copied what NetBSD
has done, adding a thin layer to emulate direct the TLI calls
into BSD socket calls.
This is mostly from Sun's tirpc release that was made in 1994,
however some fixes were backported from the 1999 release (supposedly
only made available after this porting effort was underway).
The submitter has agreed to continue on and bring us up to the
1999 release.
Several key features are introduced with this update:
Client calls are thread safe. (1999 code has server side thread
safe)
Updated, a more modern interface.
Many userland updates were done to bring the code up to par with
the recent RPC API.
There is an update to the pthreads library, a function
pthread_main_np() was added to emulate a function of Sun's threads
library.
While we're at it, bring in NetBSD's lockd, it's been far too
long of a wait.
New rpcbind(8) replaces portmap(8) (supporting communication over
an authenticated Unix-domain socket, and by default only allowing
set and unset requests over that channel). It's much more secure
than the old portmapper.
Umount(8), mountd(8), mount_nfs(8), nfsd(8) have also been upgraded
to support TI-RPC and to support IPV6.
Umount(8) is also fixed to unmount pathnames longer than 80 chars,
which are currently truncated by the Kernel statfs structure.
Submitted by: Martin Blapp <mb@imp.ch>
Manpage review: ru
Secure RPC implemented by: wpaul
Reviewed by: David Cross <dec@freebsd.org>, jkh <jkh@freebsd.org>
Approved by: jkh <jkh@freebsd.org>
Obtained from: Ian Dowse <iedowse@maths.tcd.ie>, David Cross <dec@freebsd.org>
We have been running this patch on a production NIS server for 2.5 weeks now.
Normally we would have ypserv die at least once a week, and often many times
a day.
This patch treats and error from select as zeroing out the FD_SET to indicate
that no fds are ready for reading. This is safe because the rpc code
always re-inits the FDSET before calling select.
adding (weak definitions to) stubs for some of the pthread
functions. If the threads library is linked in, the real
pthread functions will pulled in.
Use the following convention for system calls wrapped by the
threads library:
__sys_foo - actual system call
_foo - weak definition to __sys_foo
foo - weak definition to __sys_foo
Change all libc uses of system calls wrapped by the threads
library from foo to _foo. In order to define the prototypes
for _foo(), we introduce namespace.h and un-namespace.h
(suggested by bde). All files that need to reference these
system calls, should include namespace.h before any standard
includes, then include un-namespace.h after the standard
includes and before any local includes. <db.h> is an exception
and shouldn't be included in between namespace.h and
un-namespace.h namespace.h will define foo to _foo, and
un-namespace.h will undefine foo.
Try to eliminate some of the recursive calls to MT-safe
functions in libc/stdio in preparation for adding a mutex
to FILE. We have recursive mutexes, but would like to avoid
using them if possible.
Remove uneeded includes of <errno.h> from a few files.
Add $FreeBSD$ to a few files in order to pass commitprep.
Approved by: -arch
just use _foo() <-- foo(). In the case of a libpthread that doesn't do
call conversion (such as linuxthreads and our upcoming libpthread), this
is adequate. In the case of libc_r, we still need three names, which are
now _thread_sys_foo() <-- _foo() <-- foo().
Convert all internal libc usage of: aio_suspend(), close(), fsync(), msync(),
nanosleep(), open(), fcntl(), read(), and write() to _foo() instead of foo().
Remove all internal libc usage of: creat(), pause(), sleep(), system(),
tcdrain(), wait(), and waitpid().
Make thread cancellation fully POSIX-compliant.
Suggested by: deischen
Updated date. 1987 was a while ago.
Removed trailing comma in NAME section.
Uncapitalised Bindresvport and Bindresvport_sa in DESCRIPTION section.
Don't use .Nm there either.
Added bindresvport_sa() to the RETURN VALUES and ERROR sections.
-changed bindresvport2 to bindresvport_sa
-merged the man into bindresvport.3
All discussion between Jean-Luc Richier <Jean-Luc.Richier@imag.fr>,
Theo de Raadt <deraadt@cvs.openbsd.org>, itojun, is reflected to
this code. (Actually Theo de Raadt write the code simultaneously as the
discussion change.)
points. For library functions, the pattern is __sleep() <--
_libc_sleep() <-- sleep(). The arrows represent weak aliases. For
system calls, the pattern is _read() <-- _libc_read() <-- read().
is an application space macro and the applications are supposed to be free
to use it as they please (but cannot). This is consistant with the other
BSD's who made this change quite some time ago. More commits to come.
MAN8+= rstat_svc.8
The file it talks about doesn't exist on FreeBSD, so there's no point in
installing the manual page. There was already a comment to this effect in
this file, but the entry hadn't been commented out.
rstat.1 and rstat_svc.8 can probably actually be removed.
PR: docs/13767
Submitted by: Seth <seth@freebie.dp.ny.frb.org>
mode. This addresses a well-known race condition that can cause
servers to hang in accept(). The relevant case is when somebody
connects to the server and then immediately kills the connection
by sending a TCP reset. On the server this causes select to report
a ready condition on the socket, after which the accept call blocks
because there is no longer any pending connection to accept.
In -current there is already a work-around for this in the kernel.
It was merged into -stable some time ago, but then David Greenman
reverted it because it seemed to be causing a socket leak in some
cases. (See uipc_socket.c revision 1.51.2.3.) Hence this userland
fix is needed in -stable, and I plan to merge it into that branch
soon because it fixes a potential DoS attack. It may also be needed
in -current if the suspected socket leak turns out to be real. In
any case, after thinking it over I believe the fix belongs in
userland. An application shouldn't assume that a ready return from
select guarantees that the subsequent I/O operation cannot block.
A lot can happen between the select and the accept.
A similar fix should most likely be applied to the Unix domain
socket transport too.
Submitted by: peter
Reviewed by: jdp
It used to loop back up to the accept() call and block there,
shutting out all other transports until a new connection came in.
Now it returns instead after dropping the connection. That will
take it back to the select() loop where all transports can be
serviced. I intend to MFC this within a day or two since it
fixes a DoS vulnerability.
track.
The $Id$ line is normally at the bottom of the main comment block in the
man page, separated from the rest of the manpage by an empty comment,
like so;
.\" $Id$
.\"
If the immediately preceding comment is a @(#) format ID marker than the
the $Id$ will line up underneath it with no intervening blank lines.
Otherwise, an additional blank line is inserted.
Approved by: bde
uses readtcp() to gather data from the network; readtcp() uses select(),
with a timeout of 35 seconds. The problem with this is that if you
connect to a TCP server, send two bytes of data, then just pause, the
server will remain blocked in readtcp() for up to 35 seconds, which is
sort of a long time. If you keep doing this every 35 seconds, you can
keep the server occupied indefinitely.
To fix this, I modified readtcp() (and its cousin, readunix() in svc_unix.c)
to monitor all service transport handles instead of just the current socket.
This allows the server to keep handling new connections that arrive while
readtcp() is running. This prevents one client from potentially monopolizing
a server.
Also, while I was here, I fixed a bug in the timeout calculations. Someone
attempted to adjust the timeout so that if select() returned EINTR and the
loop was restarted, the timeout would be reduced so that rather than waiting
for another 35 seconds, you could never wait for more than 35 seconds total.
Unfortunately, the calculation was wrong, and the timeout could expire much
sooner than 35 seconds.
recently in BUGTRAQ. If a stream oriented transport fails to properly decode
an RPC message header structure where there should be one, it should mark
the stream as dead so that the connection will be dropped.
partway through its attempt to decode the result structure sent by
the server. If this happens, it can leave the result partially
populated with dynamically allocated memory. In this event, the
xdr_replymsg() failure is detected and RPC_CANTDECODERES is returned,
but the memory in the partially populated result struct is not
free()d.
The end result is that memory is leaked when an RPC_CANTDECODERES
error occurs. (This condition can occur if a CLIENT * handle is created
using clntudp_bufcreate() with a receive buffer size that is too small
to handle the result sent by the server.)
Fixed by setting reply_xdrs.x_op to XDR_FREE and calling
xdr_replymsg() again to free the memory if an RPC_CANTDECODERES error
is detected.
I suspect that the clnt_tcp.c, clnt_unix.c and clnt_raw.c modules
may ha a similar problem, but I haven't duplicated the condition with
those yet.
Found by: dbmalloc
to fail under certain circumstances.
1. In one spot, the ifr_flags member was being examined in the
wrong structure, thus it contained garbage. On a machine in which
only the loopback interface was up, this caused everything that
wanted to talk to the portmapper to fail -- a particular problem
with laptops, where the pccard ethernet interface is likely to come
up long after the attempt to start mountd, nfsd, amd, etc.
2. Compounding the above problem, get_myaddress() returned a
successful status even though it failed to find an address that it
considered good enough.
made to the RPC code some months ago. The value of __svc_fdsetsize is being
calculated incorrectly.
Logically, one would assume that __svc_fdsetsize is being used as a
substitute for FD_SETSIZE, with the difference being that __svc_fdsetsize
can be expanded on the fly to accomodate more descriptors if need be.
There are two problems: first, __svc_fdsetsize is not initialized to 0.
Second, __svc_fdsetsize is being calculated in svc.c:xprt_registere() as:
__svc_fdsetsize = howmany(sock+1, NFDBITS);
This is wrong. If we are adding a socket with index value 4 to the
descriptor set, then __svc_fdsetsize will be 1 (since fds_bits is
an unsigned long, it can support any descriptor from 0 to 31, so we
only need one of them). In order for this to make sense with the
rest of the code though, it should be:
__svc_fdsetsize = howmany(sock+1, NFDBITS) * NFDBITS;
Now if sock == 4, __svc_fdsetsize will be 32.
This bug causes 2 errors to occur. First, in xprt_register(), it
causes the __svc_fdset descriptor array to be freed and reallocated
unnecessarily. The code checks if it needs to expand the array using
the test: if (sock + 1 > __svc_fdsetsize). The very first time through,
__svc_fdsetsize is 0, which is fine: an array has to be allocated the
first time out. However __svc_fdsetsize is incorrectly set to 1, so
on the second time through, the test (sock + 1 > __svc_fdsetsize)
will still succeed, and the __svc_fdset array will be destroyed and
reallocated for no reason.
Second, the code in svc_run.c:svc_run() can become hopelessly confused.
The svc_run() routine malloc()s its own fd_set array using the value
of __svc_fdsetsize to decide how much memory to allocate. Once the
xprt_register() function expands the __svc_fdset array the first time,
the value for __svc_fdsetsize becomes 2, which is too small: the resulting
calculation causes the code to allocate an array that's only 32 bits wide
when it actually needs 64 bits. It also uses the valuse of __svc_fdsetsize
when copying the contents of the __svc_fdset array into the new array.
The end result is that all but the first 32 file descriptors get lost.
Note: from what I can tell, this bug originated in OpenBSD and was
brought over to us when the code was merged. The bug is still there
in the OpenBSD source.
Total nervous breakdown averted by: Electric Fence 2.0.5
undefined symbol referenced from libc. Without the stub, it is
impossible to execute any program using the shared library if
LD_BIND_NOW=1 is in the environment. The stub always returns
failure, but it can be overridden outside the library when necessary.
I don't know whether this is the "correct" fix, but it is intolerable
to have any undefined symbols referenced from libc.
The logic in get_myaddress() is broken: it always returns the loopback
address due to the following rule:
if ((ifreq.ifr_flags & IFF_UP) &&
ifr->ifr_addr.sa_family == AF_INET &&
(loopback == 1 && (ifreq.ifr_flags & IFF_LOOPBACK))) {
The idea is that we want to select the interface address only if it's
up and it's in the AF_INET family. If it turns uout we don't have
such an interface available, we make a second pass through the loop,
this time settling for the loopback interface. But the logic inadvertently
locks out all cases when loopback == 0, so nothing is ever selected until
the second pass (when loopback == 1).
This is changed to:
if (((ifreq.ifr_flags & IFF_UP) &&
ifr->ifr_addr.sa_family == AF_INET) ||
(loopback == 1 && (ifreq.ifr_flags & IFF_LOOPBACK))) {
which I think does the right thing.
This is yet another bogon I discovered during NIS+ testing; I need
get_myaddress() to work correctly so that the callback code in the
client library will work.
- bde's change to includes section in getrpcent.3
- Lost comment in svc_run.c (the code here was actually the same since
I had fixed the 'fds + 1' bug in my stuff at home before mailing
Peter about it, but I didn't notce that he'd made a change to the
comment right above the changed line).
Also pointed out by the ever vigilant: bde
This concludes tonight's entertainment. Once I'm sure I haven't destroyed
the world with all these changes, I'll import the utilities. Everything
should continue to work as before. If it doesn't let me know.
Special thanks to Mark Murray for running a test 'make world' for me to
shake out the bugs, which, hopefully, I have fixed.
(And there was much rejoicing.)
Note: you'll need to rinstalkl all your includes before compiling libc
the next time you update your sources in order for all this to work.
Reviewed by: Mark Murray
so that all these makefiles can be used to build libc_r too.
Added .if ${LIB} == "c" tests to restrict man page builds to libc
to avoid needlessly building them with libc_r too.
Split libc Makefile into Makefile and Makefile.inc to allow the
libc_r Makefile to include Makefile.inc too.
This will make a number of things easier in the future, as well as (finally!)
avoiding the Id-smashing problem which has plagued developers for so long.
Boy, I'm glad we're not using sup anymore. This update would have been
insane otherwise.
interfaces, until it's redone to use sysctl().
- bump the SIOCGIFCONF buffer size from 1K to 8K
- if we didn't find a suitable address, return a failure. Previously
if it didn't find anything it left the return address uninitialised.
Perhaps it would be better to return AF_INET/111/127.0.0.1 rather than
failing?
(There may be a behavior difference between the 2.1 and 2.2/3.0 kernels
in this area, it seemed to work for me but I have a horribly hacked
select() that might have a bug in the handling of this)
Submitted by: wpaul
Restore the clamp on the return value from rpc_dtablesize().. Some programs
(eg: ypserv) use this as an indication of how large svc_fdset is in their
hand-rolled svc_run() loops. The svc_fdset table is maintained by the
rpc library explicitly for compatability with such programs. (It uses
a different variable-sized bitmap itself internally)
- prototypes now in include files
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
Note: potential bug here, It looks like there could be a null pointer
dereference depending on what has already been called to initialise some
shared data.
- kill non-FD_SETSIZE code
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
Note, there was a nasty bug with our old code here. It would trash the
stack if a fd > 31 was passed in. It was using a "long" as though it
was an "fd_set", ie: it was assuming that a long was 256 bits wide. :-(
This has been lurking here for a while, since the FD_SETSIZE #ifdef's
were first implemented.
- fix timeout code
- better sequence number generation (for long running daemons)
- dont close an unopen socket
- use standard functions
- 64 bit type safe for wire protocols
- unlimited file descriptors
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
- ensure we're not spoofed/confused while trying to talk to the portmapper
- handle new get_myaddress failure cases
- prototype now in include file
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
- fix timeout code
- better "random" initial transaction id for long running daemons
- unlimited number of file descriptors to select().
- 64 bit type safe wire protocol
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
- typo (spelling police :-)
- dont die on select() that returns time remaining (on my systems)
- improve initial "random" sequence number, to make it harder to guess
in long running daemons.
- fix timeout code.
- unlimited number of fd's in select.
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
- Protect against select() that returns time remaining (on my systems).
- don't exit. It's bad form for libc to exit() or abort() instead of
returning an error.
- only use loopback addresses after checking the real interfaces.
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
- canonical function declaration
- use constants from includes, not magic numbers
- use standard functions
Obtained from: a diff of FreeBSD vs. OpenBSD/NetBSD rpc code.
Fixed a couple of nitpick warnings, plus one that slipped through the
net earlier.
This directory now compiles without any warnings with -Wall! (Until
the next gcc upgrade...)
1. Added missing function prototypes.
2. Added missing function return types.
3. Added missing function argument types.
4. Added missing headers for system function prototypes.
5. Corrected format specifier in printf().
6. Added extra parentheses around assignment used as truth value.
7. Added missing "default" cases in switch statements.
8. Added casts for function pointers.
9. Did *not* change int declarations of uid and gid to uid_t/gid_t
because I don't know if that would affect the protocol. Put in
explicit casts to int instead, to make things more obvious.
10. Moved declarations of variables that are only used if YP is
defined inside the '#ifdef YP' conditionals.
1. Added missing function prototypes.
2. Added missing function return types.
3. Added missing function argument types.
4. Added missing headers for system function prototypes.
5. Corrected casts in select() args.
6. Got rid of more "extern int errno" rubbish.
7. Added extra parentheses around assignment used as truth value.
8. Fixed bug in clnt_{tcp, udp}create() where pointers could be free'd
even if they hadn't been successfully malloc()'d.
1. Added missing function prototypes.
2. Added missing function return types.
3. Added missing function argument types.
4. Added missing headers for system function prototypes.
5. Got rid of "extern int errno" rubbish.
a machine with aliase ip addresses on the same subnet of an
interfaces' `real' ip addresses would generate <n> duplicate
broadcasts in clnt_broadcast().
Basically, this fix does a purge on the list of bradcast addresses.
but a commit mail got lost, it's the same as for this commit:
lib/libc/gen confstr.c crypt.c disklabel.c fstab.c getcap.c
getgrent.c getgrouplist.c getpass.c getpwent.c
initgroups.c nlist.c psignal.c pwcache.c setmode.c
sleep.c sysconf.c sysctl.c syslog.c usleep.c
lib/libc/locale none.c read_runemagi.c setlocale.c
lib/libc/net gethostbydns.c getnetbydns.c getnetbynis.c
lib/libc/nls msgcat.c
lib/libc/quad Makefile.inc
lib/libc/regex engine.c regcomp.c regerror.c
Minor cleanup, mostly unused vars and missing #includes.
Limit the number of quad functions we pull in for 'i386'.
I still belive the quad stuff should go back into gcc.
Add compile-time warnings about crypt functions.
Back out the 'help NIS rebind faster' hack. This change used a
connect()/send() pair rather than the original sendto() to allow
RPC to pass ICMP host unreachable and similar errors up to RPC
programs that use UDP. This is not a terrible thing by itself, but it can
cause trouble in environments with multi-homed hosts: if the portmapper
on the multi-homed machine sends a reply with a source address
that's different than the one associated with the connection by
connect(), the kernel will send a port unreachable message and
drop the reply. For the sake of compatibility with everybody else
on the planet, it's best to revert to the old behavior.
*long, heavy sigh*
select() returns EINVAL if you try to feed it a value of FD_SETSIZE greater
that 256. You can apparently adjust this by specifying a larger value of
FD_SETSIZE when configuring your kernel. However, if you set the maximum
number of open file descriptors per process to some value greater than
the FD_SETSIZE value that select() expects, many selects() within the RPC
library code will be botched because _rpc_dtablesize() will return
invalid numbers. This is to say that it will return the upper descriptor
table size limit which can be much higher than 256. Unless select() is
prepared to expect this 'unusually' high value, it will fail. (A good
example of this can be seen with NIS enabled: if you type 'unlimit' at
the shell prompt and then run any command that does NIS calls, you'll
be bombarded with errors from clnttcp_create().)
A temporary fix for this is to clamp the value returned by _rpc_dtablesize()
at FD_SETSIZE (as defined in <sys/types.h> (256)). I suppose the Right
Thing would be to provide some mechanism for select() to dynamically
adjust itself to handle FD_SETSIZE values larger than 256, but it's a
bit late in the game for that. Hopefully 256 file descriptors will be enough
to keep RPC happy for now.
Obtained from: Casper H. Dik (by vay of Usenet)
Small patch to help improve NIS rebinding times (among other things):
>From: casper@fwi.uva.nl (Casper H.S. Dik)
>Newsgroups: comp.sys.sun.misc,comp.sys.sun.admin
>Subject: FIX for slow rebinding of NIS.
>Summary: a small change in libc makes life with NIS a lot easier.
>Message-ID: <1992Jan17.173905.11727@fwi.uva.nl>
>Date: 17 Jan 92 17:39:05 GMT
>Sender: news@fwi.uva.nl
>Organization: FWI, University of Amsterdam
>Lines: 138
>Nntp-Posting-Host: halo.fwi.uva.nl
Have you been plagued by long waits when your NIS server is rebooted?
READ ON!
Sun has a patch, but the README says:
********************* WARNING ******************************
This is a new version of ypbind that never uses the NIS
binding file to cache the servers binding. This will have
the effect of fixing the current symptom. However, it might
degrade the overall performance of the system when the
server is available. This is most likely to happen on an
overloaded server, which will cause the network to produce
a broadcast storm.
*************************************************************
Therefor, I have produced another fix.
o What goes wrong.
When the NIS server is rebooted, ypserv will obtain different ports
to listen for RPC requests. All clients will continue to use the old
binding they obtained earlier. The NIS server will send ICMP dst unreachable
messages for the RPC requests that arrive at the old port. These ICMPs
are dropped on the floor and the client code will continue sending the
requests until the timer has expired. The small fix at the end of this
message will pick up these ICMP messages and deliver them to the RPC layer.
o Before and after.
I've tested this on some machines and this is the result:
(kill and restart ypserv on the server)
original% time ypmatch user passwd
user:....
0.040u 0.090s 2:35.64 0.0% 0+126k 0+0io 0pf+0w (155 seconds elapsed time)
fixedhost% time ypmatch user passwd
user:....
0.050u 0.050s 0:10.20 0.9% 0+136k 0+0io 0pf+0w (10 seconds elapsed time)
Rebinding is almost instantaneous.
o Other benefits.
RPC calls that use UDP as transport will no longer time out but
will abort much sooner. (E.g., the remote host is unreachable or
111/udp is filtered by an intermediate router)
command available yet.
Changed an entry in getprcent.3 from rpcinfo(8C) to rpcinfo(8).
Changed an entry in getrpcport.3 from 3R to 3.
Changed two entries in rpc.3 from 3N to 3.
