Class FSIndexRepositoryImpl

  • All Implemented Interfaces:
    FSIndexRepositoryMgr, FSIndexRepository, LowLevelIndexRepository

    public class FSIndexRepositoryImpl
    extends java.lang.Object
    implements FSIndexRepositoryMgr, LowLevelIndexRepository
    There is one instance of this class per CAS View. Some parts of the data here are shared between all views of a CAS. Many things refer to specific types, and their associated Java Cover classes. Java impl classes are always used for each type; If there is no JCas cover class defined for a type, then the most specific superclass which has a JCas defined class is used; this is the class TOP or one of its subclasses. Generic typing: User facing APIs can make use of the (JCas) Java cover types, for indexes and iterators over them The general generic type used is typically written here as T extends FeatureStructure, where FeatureStructure is the super interface of all JCas types. APIs having no reference to Java cover types (i.e., low level iterators) are not generic, unless they are needed to be to pass along the associated type to other APIs.
    • Field Detail

      • ITEM_REMOVED_FROM_INDEX

        public static final boolean ITEM_REMOVED_FROM_INDEX
        See Also:
        Constant Field Values
      • V2_ANNOTATION_COMPARE_TYPE_ORDER

        public static final boolean V2_ANNOTATION_COMPARE_TYPE_ORDER
        set next to true to debug issues with different treatment of no type priorities in v3
        See Also:
        Constant Field Values
      • DEFAULT_INDEX_SIZE

        public static final int DEFAULT_INDEX_SIZE
        The default size of an index.
        See Also:
        Constant Field Values
      • SKIP_BAG_INDEXES

        public static final boolean SKIP_BAG_INDEXES
        flag used when removing FSs due to corruption avoidance
        See Also:
        Constant Field Values
      • ALLOW_DUP_ADD_TO_INDEXES

        public static final java.lang.String ALLOW_DUP_ADD_TO_INDEXES
        Define this JVM property to allow adding the same identical FS to Set and Sorted indexes more than once.
        See Also:
        Constant Field Values
      • DISABLE_ENHANCED_WRONG_INDEX

        public static final java.lang.String DISABLE_ENHANCED_WRONG_INDEX
        See Also:
        Constant Field Values
      • IS_DISABLE_ENHANCED_WRONG_INDEX_CHECK

        private static final boolean IS_DISABLE_ENHANCED_WRONG_INDEX_CHECK
      • cas

        private final CASImpl cas
        Replicated per view
      • locked

        private boolean locked
      • indexArray

        final FSIndexRepositoryImpl.IndexesForType[] indexArray
        An array of information about defined indexes, one for each type in the type hierarchy. - includes for each type, an unordered list of FsIndex_iicps for that type, corresponding to the different index definitions over that type. The key is the typecode of the type.
      • name2indexMap

        final java.util.HashMap<java.lang.String,​FsIndex_iicp<TOP>> name2indexMap
        A map from names to FsIndex_iicps, which represent the index at the top-most type declared in the index specification. Different names may map to the same iicp. The keys are the same across all views, but the values are different, per view
      • indexUpdates

        private final java.util.List<TOP> indexUpdates
      • indexUpdateOperation

        private final java.util.BitSet indexUpdateOperation
      • logProcessed

        private boolean logProcessed
      • usedIndexes

        private final IntVector usedIndexes
      • isUsed

        private final java.util.BitSet isUsed
    • Constructor Detail

      • FSIndexRepositoryImpl

        private FSIndexRepositoryImpl()
        ----------------------- Support for flattened indexes -----------------
      • FSIndexRepositoryImpl

        FSIndexRepositoryImpl​(CASImpl cas)
        Constructor. Assumption: called first before next constructor call, with the base CAS view
        Parameters:
        cas -
      • FSIndexRepositoryImpl

        FSIndexRepositoryImpl​(CASImpl cas,
                              FSIndexRepositoryImpl baseIndexRepo)
        Constructor for additional views.
        Parameters:
        cas -
        baseIndexRepository -
    • Method Detail

      • init

        private void init()
        Initialize data. Common initialization called from the constructors.
      • createIndex

        <T extends FeatureStructure> void createIndex​(FSIndexRepositoryImpl baseIndexRepo,
                                                      java.lang.String key)
        create indexes in a view, by copying the baseCas's index repository's definitions Called when creating or refreshing (after deserializing) a view
        Parameters:
        baseIndexRepo - - where the base definitions are
        key - - the name of the index.
      • createIndexNoQuestionsAsked

        public <T extends FeatureStructure> boolean createIndexNoQuestionsAsked​(FSIndexComparator comp,
                                                                                java.lang.String label,
                                                                                int indexType)
        This is public only until the xml specifier format supports specifying index kinds (set, bag etc.).
        Type Parameters:
        T - -
        Parameters:
        comp - -
        label - -
        indexType - -
        Returns:
        -
      • removeIndex

        public void removeIndex​(java.lang.String label)
        just for testing purposes removes the named index Also removes indexes for all subtypes. - NOTE this might remove another index definition's index, if it matches Only valid if no add-to-index operations have happened.
        Parameters:
        label - the name of the index to remove
      • flush

        public void flush()
        Reset all indexes, in one view.
      • addNewIndex

        private <T extends TOPFsIndex_iicp<T> addNewIndex​(FSIndexComparatorImpl comparator,
                                                            int initialSize,
                                                            int indexType)
        This is where the actual index gets created.
      • isAnnotationIndex

        boolean isAnnotationIndex​(FSIndexComparator c,
                                  int indexKind)
      • addNewIndexCore

        <T extends TOPFsIndex_singletype<T> addNewIndexCore​(FSIndexComparatorImpl comparatorForIndexSpecs,
                                                              int initialSize,
                                                              int indexingStrategy)
        The routine which actually creates a new index, for a single type.
        Parameters:
        comparatorForIndexSpecs - -
        initialSize - -
        indexingStrategy - -
        Returns:
        -
      • addNewIndexRecursive

        private FsIndex_iicp<TOP> addNewIndexRecursive​(FSIndexComparator compForIndexSpecs,
                                                       int indexType)
        Top level call to add the indexes for a particular index definition
        Parameters:
        compForIndexSpecs -
        indexType -
        Returns:
        the iicp for the top new index
      • addNewIndexRec

        private FsIndex_iicp<TOP> addNewIndexRec​(FSIndexComparatorImpl comp4indexSpecs,
                                                 int indexType)
        Add an index for a type, and then (unless it's a DEFAULT_BAG_INDEX), call yourself recursively to add the indexes for all the directly subsumed subtypes.
        Parameters:
        comp4indexSpecs -
        indexType -
        Returns:
        the new iicp for the new index
      • getLabels

        public <T extends FeatureStructure> java.util.Iterator<java.lang.String> getLabels​(FSIndexComparator comp)
        Get the labels for a specific comparator.
        Type Parameters:
        T - type of Feature Structure
        Parameters:
        comp - The comparator.
        Returns:
        An iterator over the labels.
      • getIndex

        public <T extends FeatureStructureLowLevelIndex<T> getIndex​(java.lang.String label)
        Description copied from interface: FSIndexRepository
        Retrieve an index according to a label. Generics: T is the associated Java cover class for the top type defined for this index name.
        Specified by:
        getIndex in interface FSIndexRepository
        Type Parameters:
        T - the Java class associated with the top-most type of the index
        Parameters:
        label - The name of the index.
        Returns:
        The index with the name label, or null if no such index is defined.
        See Also:
        FSIndexRepository.getIndex(String)
      • removeAllExcludingSubtypes

        public void removeAllExcludingSubtypes​(Type type)
        Remove all instances of a particular type (but not its subtypes) from all indexes
        Specified by:
        removeAllExcludingSubtypes in interface FSIndexRepository
        Parameters:
        type - -
      • removeAllIncludingSubtypes

        public void removeAllIncludingSubtypes​(Type type)
        Remove all instances of a particular type (including its subtypes) from all indexes
        Specified by:
        removeAllIncludingSubtypes in interface FSIndexRepository
        Parameters:
        type - Type to remove (including all its subtypes) from this particular view.
      • walkIndexedFSs

        public void walkIndexedFSs​(java.util.function.Consumer<TOP> action)
        For this view, walk the indexed FSs in arbitrary order.
        Parameters:
        action - the action to do on each FS
      • walkSortedIndexedFSs

        public void walkSortedIndexedFSs​(java.util.function.Consumer<TOP> action)
        For this view, walk the indexed FSs, sorted by id (e.g. creation time)
        Parameters:
        action - -
      • getNonSetSingleIndexForType

        public FsIndex_singletype<TOP> getNonSetSingleIndexForType​(int typecode)
      • getNonSetSingleIndexForUsedType

        public FsIndex_singletype<TOP> getNonSetSingleIndexForUsedType​(int i)
      • addFS

        public void addFS​(int fsRef)
      • ll_addFS

        public void ll_addFS​(int fsRef)
        Description copied from interface: LowLevelIndexRepository
        Add a FS reference to all appropriate indexes in the repository.
        Specified by:
        ll_addFS in interface LowLevelIndexRepository
        Parameters:
        fsRef - The FS reference to be added to the repository. If fsRef is not a valid FS reference, the subsequent behavior of the system is undefined.
      • ll_removeFS

        public void ll_removeFS​(int fsRef)
        Description copied from interface: LowLevelIndexRepository
        Remove a FS reference from all indexes in the repository. Note that this only removes the reference from the index repository, it does not free memory on the heap.
        Specified by:
        ll_removeFS in interface LowLevelIndexRepository
        Parameters:
        fsRef - The FS reference to be removed from the indexes.
      • addFS

        public <T extends FeatureStructure> void addFS​(T fs)
        Description copied from interface: FSIndexRepository
        Add a feature structure to all appropriate indexes in the repository. If no indexes exist for the type of FS that you are adding, then a bag (unsorted) index will be automatically created.

        Important: after you have called addFS() on a FS, do not change the values of any features used for indexing. If you do, the index will become corrupted and may be unusable. If you need to change an index feature value, first call removeFS() on the FS, change the feature values, then call addFS() again.

        Specified by:
        addFS in interface FSIndexRepository
        Type Parameters:
        T - the generic type of the FeatureStructure
        Parameters:
        fs - The FS to be added.
        See Also:
        FSIndexRepository.addFS(org.apache.uima.cas.FeatureStructure)
      • removeFS

        public void removeFS​(int fsRef)
      • ll_getIndex

        public <T extends FeatureStructureLowLevelIndex<T> ll_getIndex​(java.lang.String indexName,
                                                                         int typeCode)
        Description copied from interface: LowLevelIndexRepository
        Get an index by a name and type. The type must be a subtype of the index's type. Note that there is no special checked version of this method, the type parameters are always checked.
        Specified by:
        ll_getIndex in interface LowLevelIndexRepository
        Type Parameters:
        T - type of Feature Structure
        Parameters:
        indexName - The name of the index.
        typeCode - The code of the desired subtype.
        Returns:
        The corresponding index, or null if no such index exists.
      • ll_addFS

        public final void ll_addFS​(int fsRef,
                                   boolean doChecks)
        Description copied from interface: LowLevelIndexRepository
        Add a FS reference to all appropriate indexes in the repository.
        Specified by:
        ll_addFS in interface LowLevelIndexRepository
        Parameters:
        fsRef - The FS reference to be added to the repository. If fsRef is not a valid FS reference, the subsequent behavior of the system is undefined.
        doChecks - Check if the FS reference argument is a valid reference.
      • addback

        public <T extends TOP> void addback​(T fs)
      • addFS_common

        private <T extends TOP> void addFS_common​(T fs,
                                                  boolean isAddback)
      • getAutoIndexNameForType

        private static final java.lang.String getAutoIndexNameForType​(Type type)
      • removeFS_ret

        boolean removeFS_ret​(TOP fs,
                             boolean skipBagIndexes)
        Common remove FS code; all remove operations call this, except bulk remove (flush and removeall...) Removes FS from all indexes in this view (except bag if skipBagIndexes is true)
        Parameters:
        fs - the fs to remove
        skipBagIndexes - set true by protect-indexes style of temporary removal
        Returns:
        true if it was removed
      • getAllIndexedFS

        public <T extends FeatureStructureLowLevelIterator<T> getAllIndexedFS​(Type type)
        Description copied from interface: FSIndexRepository
        Gets an iterator over all indexed (in this View) FeatureStructures of the specified Type (and any of its subtypes). The elements are returned in arbitrary order. Generics: T is the Java class for aType.
        Specified by:
        getAllIndexedFS in interface FSIndexRepository
        Type Parameters:
        T - The Java class associated with aType
        Parameters:
        type - The type
        Returns:
        An iterator that returns all indexed FeatureStructures of type aType and its subtypes, in no particular order.
      • getIndexedFSs

        public java.util.Collection<TOP> getIndexedFSs()
        Description copied from interface: FSIndexRepository
        Returns an unmodifiable collection of all of the FSs that are indexed in this view, in an arbitrary order. Subsequent modifications to the indexes do not affect this collection.
        Specified by:
        getIndexedFSs in interface FSIndexRepository
        Returns:
        an unmodifiable, unordered collection of all indexed (in this view) Feature Structures of the specified type (including subtypes)
      • getIndexedFSs

        public <T extends TOP> java.util.Collection<T> getIndexedFSs​(java.lang.Class<T> clazz)
        Description copied from interface: FSIndexRepository
        Returns an unmodifiable collection of all the FSs of this type and its subtypes, that are indexed in this view, in an arbitrary order. Subsequent modifications to the indexes do not affect this collection.
        Specified by:
        getIndexedFSs in interface FSIndexRepository
        Type Parameters:
        T - The Java class associated with type
        Parameters:
        clazz - The JCas class corresponding to the type
        Returns:
        an unmodifiable, unordered collection of all indexed (in this view) Feature Structures of the specified type (including subtypes)
      • getIndexedFSs

        public <T extends TOP> java.util.Collection<T> getIndexedFSs​(Type type)
        Description copied from interface: FSIndexRepository
        Returns an unmodifiable collection of all the FSs of this type and its subtypes, that are indexed in this view, in an arbitrary order. Subsequent modifications to the indexes do not affect this collection.
        Specified by:
        getIndexedFSs in interface FSIndexRepository
        Type Parameters:
        T - The Java class associated with type
        Parameters:
        type - the type of Feature Structures to include (including subtypes)
        Returns:
        an unmodifiable, unordered set of all indexed (in this view) Feature Structures of the specified type (including subtypes)
      • getCollectionFromCows

        private <T extends TOP> java.util.Collection<T> getCollectionFromCows​(java.util.ArrayList<CopyOnWriteIndexPart<T>> indexes)
      • streamNonEmptyIndexes

        public java.util.stream.Stream<FsIndex_singletype<TOP>> streamNonEmptyIndexes​(Type type)
        Stream instances of all of the non-empty indexes themselves
        Parameters:
        type - - the type to filter the indexes with
        Returns:
        all of the non-empty indexes, one for each sorted or default bag per type
      • streamNonEmptyIndexes

        public java.util.stream.Stream<FsIndex_singletype<TOP>> streamNonEmptyIndexes​(java.lang.Class<? extends TOP> clazz)
      • getAnnotationIndex

        <T extends AnnotationFSFsIndex_annotation<T> getAnnotationIndex​(TypeImpl ti)
        returns the annotation index for a type which is Annotation or a subtype of it. remembers answer in hashmap annotationIndexes, key = TypeImpl
        Parameters:
        typeCode -
        Returns:
        the index for that type
      • logIndexOperation

        private <T extends TOP> void logIndexOperation​(T fs,
                                                       boolean added)
      • processIndexUpdates

        private void processIndexUpdates()
        Go through the journal, and use those entries to update added, deleted, and reindexed lists in such a way as to guarantee: a FS is in only one of these lists, (or in none) For a journal "add-to-indexes" event: fs in "deleted": remove from "deleted", add to "reindexed" fs in "reindexed": do nothing fs in "added": do nothing fs not in any of these: add to "added" For a journal "remove-from-indexes" event: fs in "added": remove from "added" (don't add to "deleted") fs in "reindexed": remove from "reindexed" and add to "deleted") fs in "deleted": do nothing fs not in any of these: add to "deleted" The journal is cleared after processing.
      • getUpdatedFSs

        public java.util.Set<TOP> getUpdatedFSs​(java.util.Set<TOP> items)
      • getAddedFSs

        public java.util.Set<TOP> getAddedFSs()
      • getDeletedFSs

        public java.util.Set<TOP> getDeletedFSs()
      • getReindexedFSs

        public java.util.Set<TOP> getReindexedFSs()
      • isModified

        public boolean isModified()
      • toString

        public java.lang.String toString()
        Overrides:
        toString in class java.lang.Object
      • createAnnotationFsComparator

        private java.util.Comparator<TOP> createAnnotationFsComparator​(FSComparators withId,
                                                                       FSComparators withTypeOrder)
      • getAnnotationFsComparatorWithoutId

        public java.util.Comparator<TOP> getAnnotationFsComparatorWithoutId()
      • getAnnotationFsComparatorWithId

        java.util.Comparator<TOP> getAnnotationFsComparatorWithId()
      • createAnnotationFsComparator

        private java.util.Comparator<TOP> createAnnotationFsComparator()
      • createAnnotationFsComparatorWithId

        private java.util.Comparator<TOP> createAnnotationFsComparatorWithId()
      • getIndexBySpec

        public <T extends FeatureStructureFsIndex_iicp<T> getIndexBySpec​(int typeCode,
                                                                           int indexingStrategy,
                                                                           FSIndexComparatorImpl comp)
        Get the FsIndex_iicp for a given typeCode, indexingStrategy, and comparator (type ignored)
        Type Parameters:
        T - type of Feature Structure
        Parameters:
        typeCode - -
        indexingStrategy - -
        comp - -
        Returns:
        -
      • removeIndexBySpec

        private void removeIndexBySpec​(int typeCode,
                                       int indexingStrategy,
                                       FSIndexComparatorImpl comp)
      • getCasImpl

        public CASImpl getCasImpl()