Package gnu.lists

Contains utility classes and interfaces for sequences (lists), arrays, and trees.

Features:

  • The classes and interfaces are compatible with the Java2 Collections framework, but do not require them. A configuration option allows you to specify whether Sequence extends java.util.List or not.
  • Positions in a Sequence and iterators are the same kind of object, and are represented using magic cookies. In most collections frameworks, when you define a new class that implements a sequence, you would typically define a new iterator class as well. You never do that with gnu.lists; instead you implement suitable methods in the sequence class itself. This approach may seem strange, but it has important performance advantages, especially in applications where you want to go up and down nested sequences (i.e. in trees, such as XML elements), or remember sets of positions (such as XML node-sets).
  • The classes are designed for efficiency, especially concentrating on minimizing object allocation.
  • The framework supports general multi-dimensional arrays.
  • The package is self-contained; it does not depend on classes or features not in JDK 1.1.
  • Various useful sequence implementations, including Lisp-style linked lists; arrays with various element types; adjustable arrays that use a "buffer-gap" to make insertions and deletions more efficient; and adjustable arrays that automatically update positions on insertions and deletions.
  • The TreeList class can compactly represent a nested heterogenous tree. One intended usage is as a very efficient representation for XML data.
  • The Consumer interface is an abstraction of "data sinks" - objects that can receive data and do something with them. It is similar to SAX's DocumentHandler but at a more abstract (and sometimes more efficient) level.

The iteration and position framework

A position points between two elements in its "containing" sequence, or it points before the first element (if any), or after the last. Sometimes, we loosely say that the position refers to or points to an element of the sequence; in that case we mean the position is immediately before the element.

An iterator is an object that has a current position, but that can be moved so it gets a new position. What needs to be emphasized is that all Sequence implementations. use the same SeqPosition class to implement positions and iteration. In other "collections frameworks" each sequence class has its corresponding iterator class, but in this framework you instead add the methods that handle iteration to the sequence class itself, extending the AbstractSequence class. A consequence of this is that if you have an unused SeqPosition object, you can use it to iterate over any Sequence you choose. This potentially saves memory allocation, which gains you the most when iterating over a nested sequence structure, like a tree of XML data.

We do this by requiring that any position be representable using a pair of an int and an Object. In other words, the state of an iterator is restricted to be an (int, Object) pair. Of course that is all you could need, since if you need more state you can always create a helper object, and store the extra state there. The assumption we make though is that for most Sequences, an (int, Object) would be enough, without creating any new objects (beyond, sometimes, the SeqPosition itself).

The int part of a position-state is normally named ipos, and the Object part of a position-state is named xpos. Neither of these values have any meaning in themselves, they only have meaning as interpreted by the specific Sequence. For example, ipos might be the offset of the position in the sequence, but it might also some "magic cookie". If a Sequence supports insertions and deletions, and you want positions to be automatically adjusted, then a position has to be a magic cookie rather than an offset. (A typical implementation is for such a position to be an index into a table of positions managed by the Sequence itself.) So a complete position is actually a (int, Object, AbstractSequence) triple, where the AbstractSequence component is the object that interprets the (int, Object) pair. Normally, the AbstractSequence part of a position triple is the Sequence "containing" the position, but it can be any AbstractSequence that implements the various methods that provide the semantics for the (int, Object) pair.

The PositionContainer interface is a more abstract version of SeqPosition, in that it can represents more than one position. For example, instead of an array of separately allocated SeqPosition objects, you might use some data structure that implements PositionContainer, which is abstractly a list of (int, Object, Sequence)-triples.

The consumer protocol

You typically use an iteration framework it process the elements of a sequence in such a way that the data consumer is active and in control, while the sequence itself (the data producer) is a passive object. The Consumer works the other way round: The data producer is active and in control, while the data consumer is passive, consuming elements when requested by the data producer. The Consumer is a more abstract variant of SAX's ContentHandler interface for processing "events"; however Consumer is intended for general value sequences, and is less tied to XML.

Iteration

int iter = 0; // standard start position
for (;;)
{
  iter = list.nextPos(iter);
  if (iter == 0)
    break;
  Object value = list.getPosPrevious(iter);
  ... use value ...;
}

Position values for buffer-based sequences

The position encoding for sequences implemented using an array is simple. Assume the next index (as returned by nextIndex) is i. If the position is a before/backwards position, then position value is (i<<1). If the position is an after/forwards position, then position value is ((i<<1))|1.

But what each each item in the buffer has variable size? One example is a UTF-8 string. Another example is a buffer of text where each "item" is a line. Then we have the choice: Should the position value encode the index (making nextIndex and get cheap), or should it encode the offset into the buffer (making sequential access cheap)? Since sequential access is preferred, we do the latter. Thus for a before/backwards position, if the buffer offset for item i is pi, then the position value is (pi << 1). However, there is a complication when moving forwards using a ListIterator since using set or remove affects the previous item. It may be much more expensive to calculate the buffer position of the previous item than the next item. (Given pi it may be cheap to calculate pi+1 but expensive to calculate pi-1.) Therefore, we suggest using (((pi-1+1)<<1))|1, where p-1 is defined as -1. The addition of 1 allows us to handle the case of i=0 without conflicting with the use of -1 to mean end-of-sequence. Plus it makes the previous case of a simple array a special case.