when I came up with this idea weren't strong enough to help me see it
through. If this was a self-contained application and I had complete
control over what data got sent through what socket and when, I might
be able to get everything to work right without blocking, but instead
I have RPC/XDR in between me and the socket layer, and they have their
own ideas about what to do.
Maybe one day I'll go totally mad and figure out the right way to do
this; in the meantime this mess goes on the back burner.
_without_ using fork().
The problem with YPPROC_ALL is that it transmits an entire map through
a TCP pipe as the result of a single RPC call. First of all, this requires
certain hackery in the XDR filter. Second, if the map being sent is
large, the server can end up spending lots of time in the XDR filter
sending to just the one client, while requests for other clients will
go unanswered.
My original solution for this was to fork() the request into a child
process which terminates after the map has been transmitted (or the
transfer is interrupted due to an error). This leaves the parent free
to handle other requests. But this solution is kind of lame: fork()
is relatively expensive, and we have to keep a cap on the number of
child processes to keep from swamping the system.
What we do now is grab control of the service transport handle and XDR
handle from the RPC library and send the records one at a time ourselves
instead of letting the RPC library do it. We send a record, then go
back to the svc_run() loop and select() on the socket. If select() says
we can still write data, we send the next record. Then we call
svc_getreqset() and handle other RPCs and loop around again. This way,
we can handle other RPCs between records.
We manage multiple YPPROC_ALL requests using a circular queue. When a
request is done, we dequeue it and destroy the handle. We also tag
each request with a ttl which is decremented whevever we run the queue
and a handle isn't serviced. This lets us nuke requests that have sat
idle for too long (if we didn't do this, we might run out of socket
descriptors.)
Now all I have to do is come up with an async resolver, and ypserv
won't need to fork() at all. :)
Note: these changes should not go into 2.2 unless they get a very
throrough shakedown before the final cutoff date.
Fix some comments to reflect reality (in some cases I made changes
to code but not to the comments).
Change some instances of 'inline' to '__inline' to pacify
gcc -ansi -pedantic.
Use rcsid strings more consistently.
Make 'oldaddr' static in yp_access().
Use strcpy()/strcat() in yp_open_db_cache() instead of snprintf().
(Seems to be a little faster this way.)
yp_dblookup.c:
- Implement database handle caching. What this means is that instead
of opening and closing map databases for each request, we open a
database and save the handle (and, if requested, the key index)
in an array. This saves a bit of overhead on things like repeated
YPPROC_NEXT calls, such as you'd get from getpwent(). Normally,
each YPPROC_NEXT would require open()ing the database, seeking
to the location supplied by the caller (which is time consuming with
hash databases as the R_CURSOR flag doesn't work), reading the
data, close()ing the database and then shipping the data off to
the caller. The system call overhead is prohibitive, especially
with very large maps. By caching the handle to an open database,
we elimitate at least the open()/close() system calls, as well
as the associated DB setup and tear-down operations, for a large
percentage of the time. This improves performance substantially at
the cost of consuming a little more memory than before.
Note that all the caching support is surrounded by #ifdef DB_CACHE
so that this same source module can still be used by other programs
that don't need it.
- Make yp_open_db() call yp_validdomain(). Doing it here saves cycles
when caching is enabled since a hit on the map cache list by
definition means that the domain being referenced is valid.
- Also make yp_open_db() check for exhaustion of file descriptors,
just in case.
yp_server.c:
- Reorganize things a little to take advantage of the database
handle caching. Add a call to yp_flush_all() in ypproc_clear_2_svc().
- Remove calls to yp_validdomain() from some of the service procedures.
yp_validdomain() is called inside yp_open_db() now, so procedures that
call into the database package don't need to use yp_validdomain()
themselves.
- Fix a bogosity in ypproc_maplist_2_svc(): don't summarily initiallize
the result.maps pointer to NULL. This causes yp_maplist_free()
to fail and leaks memory.
- Make ypproc_master_2_svc() copy the string it gets from the database
package into a private static buffer before trying to NUL terminate it.
This is necessary with the DB handle caching: stuffing a NUL into the
data returned by DB package will goof it up internally.
yp_main.c:
- Stuff for DB handle caching: call yp_init_dbs() to clear the
handle array and add call to yp_flush_all() to the SIGHUP
signal handler.
Makefile.yp:
- Reorganize to deal with database caching. yp_mkdb(8) can now be used
to send a YPPROC_CLEAR signal to ypserv(8). Call it after each map
is created to refresh ypserv's cache.
- Add support for mail.alias map.
Contributed by Mike Murphy (mrm@sceard.com).
- Make default location for the netgroups source file be /var/yp/netgroup
instead of /etc/netgroup.
mkaliases:
- New file: script to generate mail.alias map.
Contributed by Mike Murphy (mrm@sceard.com).
Makefile:
- Install Makefile.yp as /var/yp/Makefile.dist and link it to
/var/yp/Makefile only if /var/yp/Makefile doesn't already exist.
Suggested by Peter Wemm.
- Install new mkaliases script in /usr/libexec along with mknetid.
- Use somewhat saner approach to generating rpcgen-dependent files
as suggested by Garrett Wollman.
are currently implemented:
YPOLDPROC_NULL
YPOLDPROC_DOMAIN
YPOLDPROC_DOMAIN_NONACK
YPOLDPROC_FIRST
YPOLDPROC_NEXT
YPOLDPROC_MATCH
YPOLDPROC_POLL
These are all implemented as wrappers around their v2 counterparts.
The YPOLDPROC_PUSH, PULL and GET procedures are not implemented since
a) I couldn't figure out exactly what to have them do, and b) I
suspect they're used for doing map transfers between master and
slave servers, which we already do using the v2 protocol anyway.
This means we can server NIS v1 clients but can't be a master or
slave with NIS v1-only servers. I think I'll get over it. :)
The -k (sunos_4_kludge) flag and associated code has been removed
since it is no longer needed.
Also tweaked yp_access() to handle both sets of procedures and
updated the man page.
in the same was as the SunOS ypserv (same format, described in ypserv man
page). If the user wants tcpwrapper style access control, they can
recompile ypserv to use that instead. This way we get securenets without
having to ship libwrap.a and tcpd.h with core FreeBSD distribution.
If /var/yp/securenets doesn't exist, ypserv allows all connections.
- Add a ypxfr_callback() function that we can use to signal failure to
yppush(8) in the event that we can't fork()/exec() ypxfr(8). yppush
only checks the return status from YPPROC_XFR enough to determine
that the RPC succeded: it relies on its callback service to figure
out whether or not the transfer actually worked.
- Give yp_dblookup.c its own debug variable (ypdb_debug) so that DB
access debugging messages can be turned on or off independent of the
program's global debug messages.
- Have the Makefile rpcgen the ypushresp_xfr_1() client stub for us and
nuke the unneeded rule for yp_xdr.c that I left in by mistake (the XDR
filters live in libc now).
