95 lines
3.9 KiB
Plaintext
95 lines
3.9 KiB
Plaintext
|
TODO file for GNU ptx - last revised 05 November 1993.
|
|||
|
|
Copyright (C) 1992, 1993 Free Software Foundation, Inc.
|
|||
|
|
Francois Pinard <pinard@iro.umontreal.ca>, 1992.
|
|||
|
|
|
|||
|
|
The following are more or less in decreasing order of priority.
|
|||
|
|
|
|||
|
|
* Use rx instead of regex.
|
|||
|
|
|
|||
|
|
* Correct the infinite loop using -S '$' or -S '^'.
|
|||
|
|
|
|||
|
|
* Use mmap for swallowing files (maybe wrong when memory edited).
|
|||
|
|
|
|||
|
|
* Understand and mimic `-t' option, if I can.
|
|||
|
|
|
|||
|
|
* Sort keywords intelligently for Latin-1 code. See how to interface
|
|||
|
|
this character set with various output formats. Also, introduce
|
|||
|
|
options to inverse-sort and possibly to reverse-sort.
|
|||
|
|
|
|||
|
|
* Improve speed for Ignore and Only tables. Consider hashing instead
|
|||
|
|
of sorting. Consider playing with obstacks to digest them.
|
|||
|
|
|
|||
|
|
* Provide better handling of format effectors obtained from input, and
|
|||
|
|
also attempt white space compression on output which would still
|
|||
|
|
maximize full output width usage.
|
|||
|
|
|
|||
|
|
* See how TeX mode could be made more useful, and if a texinfo mode
|
|||
|
|
would mean something to someone.
|
|||
|
|
|
|||
|
|
* Provide multiple language support
|
|||
|
|
|
|||
|
|
Most of the boosting work should go along the line of fast recognition
|
|||
|
|
of multiple and complex boundaries, which define various `languages'.
|
|||
|
|
Each such language has its own rules for words, sentences, paragraphs,
|
|||
|
|
and reporting requests. This is less difficult than I first thought:
|
|||
|
|
|
|||
|
|
. Recognize language modifiers with each option. At least -b, -i, -o,
|
|||
|
|
-W, -S, and also new language switcher options, will have such
|
|||
|
|
modifiers. Modifiers on language switchers will allow or disallow
|
|||
|
|
language transitions.
|
|||
|
|
|
|||
|
|
. Complete the transformation of underlying variables into arrays in
|
|||
|
|
the code.
|
|||
|
|
|
|||
|
|
. Implement a heap of positions in the input file. There is one entry
|
|||
|
|
in the heap for each compiled regexp; it is initialized by a re_search
|
|||
|
|
after each regexp compile. Regexps reschedule themselves in the heap
|
|||
|
|
when their position passes while scanning input. In this way, looking
|
|||
|
|
simultaneously for a lot of regexps should not be too inefficient,
|
|||
|
|
once the scanning starts. If this works ok, maybe consider accepting
|
|||
|
|
regexps in Only and Ignore tables.
|
|||
|
|
|
|||
|
|
. Merge with language processing boundary processing options, really
|
|||
|
|
integrating -S processing as a special case. Maybe, implement several
|
|||
|
|
level of boundaries. See how to implement a stack of languages, for
|
|||
|
|
handling quotations. See if more sophisticated references could be
|
|||
|
|
handled as another special case of a language.
|
|||
|
|
|
|||
|
|
* Tackle other aspects, in a more long term view
|
|||
|
|
|
|||
|
|
. Add options for statistics, frequency lists, referencing, and all
|
|||
|
|
other prescreening tools and subsidiary tasks of concordance
|
|||
|
|
production.
|
|||
|
|
|
|||
|
|
. Develop an interactive mode. Even better, construct a GNU emacs
|
|||
|
|
interface. I'm looking at Gene Myers <gene@cs.arizona.edu> suffix
|
|||
|
|
arrays as a possible implementation along those ideas.
|
|||
|
|
|
|||
|
|
. Implement hooks so word classification and tagging should be merged
|
|||
|
|
in. See how to effectively hook in lemmatisation or other
|
|||
|
|
morphological features. It is far from being clear by now how to
|
|||
|
|
interface this correctly, so some experimentation is mandatory.
|
|||
|
|
|
|||
|
|
. Profile and speed up the whole thing.
|
|||
|
|
|
|||
|
|
. Make it work on small address space machines. Consider three levels
|
|||
|
|
of hugeness for files, and three corresponding algorithms to make
|
|||
|
|
optimal use of memory. The first case is when all the input files and
|
|||
|
|
all the word references fit in memory: this is the case currently
|
|||
|
|
implemented. The second case is when the files cannot fit all together
|
|||
|
|
in memory, but the word references do. The third case is when even
|
|||
|
|
the word references cannot fit in memory.
|
|||
|
|
|
|||
|
|
. There also are subsidiary developments for in-core incremental sort
|
|||
|
|
routines as well as for external sort packages. The need for more
|
|||
|
|
flexible sort packages comes partly from the fact that linguists use
|
|||
|
|
kinds of keys which compare in unusual and more sophisticated ways.
|
|||
|
|
GNU `sort' and `ptx' could evolve together.
|
|||
|
|
|
|||
|
|
|
|||
|
|
Local Variables:
|
|||
|
|
mode: outline
|
|||
|
|
outline-regexp: " *[-+*.] \\| "
|
|||
|
|
eval: (hide-body)
|
|||
|
|
End:
|