Class FilteredNormalizer2
- Author:
- Markus W. Scherer
-
Nested Class Summary
Nested classes/interfaces inherited from class com.ibm.icu.text.Normalizer2
Normalizer2.Mode
-
Constructor Summary
ConstructorsConstructorDescriptionFilteredNormalizer2
(Normalizer2 n2, UnicodeSet filterSet) Constructs a filtered normalizer wrapping any Normalizer2 instance and a filter set. -
Method Summary
Modifier and TypeMethodDescriptionappend
(StringBuilder first, CharSequence second) Appends the second string to the first string (merging them at the boundary) and returns the first string.int
composePair
(int a, int b) Performs pairwise composition of a & b and returns the composite if there is one.int
getCombiningClass
(int c) Gets the combining class of c.getDecomposition
(int c) Gets the decomposition mapping of c.getRawDecomposition
(int c) Gets the raw decomposition mapping of c.boolean
hasBoundaryAfter
(int c) Tests if the character always has a normalization boundary after it, regardless of context.boolean
hasBoundaryBefore
(int c) Tests if the character always has a normalization boundary before it, regardless of context.boolean
isInert
(int c) Tests if the character is normalization-inert.boolean
Tests if the string is normalized.normalize
(CharSequence src, Appendable dest) Writes the normalized form of the source string to the destination Appendable and returns the destination Appendable.normalize
(CharSequence src, StringBuilder dest) Writes the normalized form of the source string to the destination string (replacing its contents) and returns the destination string.normalizeSecondAndAppend
(StringBuilder first, CharSequence second) Appends the normalized form of the second string to the first string (merging them at the boundary) and returns the first string.Tests if the string is normalized.int
Returns the end of the normalized substring of the input string.Methods inherited from class com.ibm.icu.text.Normalizer2
getInstance, getNFCInstance, getNFDInstance, getNFKCCasefoldInstance, getNFKCInstance, getNFKDInstance, normalize
-
Constructor Details
-
FilteredNormalizer2
Constructs a filtered normalizer wrapping any Normalizer2 instance and a filter set. Both are aliased and must not be modified or deleted while this object is used. The filter set should be frozen; otherwise the performance will suffer greatly.- Parameters:
n2
- wrapped Normalizer2 instancefilterSet
- UnicodeSet which determines the characters to be normalized
-
-
Method Details
-
normalize
Writes the normalized form of the source string to the destination string (replacing its contents) and returns the destination string. The source and destination strings must be different objects.- Specified by:
normalize
in classNormalizer2
- Parameters:
src
- source stringdest
- destination string; its contents is replaced with normalized src- Returns:
- dest
-
normalize
Writes the normalized form of the source string to the destination Appendable and returns the destination Appendable. The source and destination strings must be different objects.Any
IOException
is wrapped into aICUUncheckedIOException
.- Specified by:
normalize
in classNormalizer2
- Parameters:
src
- source stringdest
- destination Appendable; gets normalized src appended- Returns:
- dest
-
normalizeSecondAndAppend
Appends the normalized form of the second string to the first string (merging them at the boundary) and returns the first string. The result is normalized if the first string was normalized. The first and second strings must be different objects.- Specified by:
normalizeSecondAndAppend
in classNormalizer2
- Parameters:
first
- string, should be normalizedsecond
- string, will be normalized- Returns:
- first
-
append
Appends the second string to the first string (merging them at the boundary) and returns the first string. The result is normalized if both the strings were normalized. The first and second strings must be different objects.- Specified by:
append
in classNormalizer2
- Parameters:
first
- string, should be normalizedsecond
- string, should be normalized- Returns:
- first
-
getDecomposition
Gets the decomposition mapping of c. Roughly equivalent to normalizing the String form of c on a DECOMPOSE Normalizer2 instance, but much faster, and except that this function returns null if c does not have a decomposition mapping in this instance's data. This function is independent of the mode of the Normalizer2.- Specified by:
getDecomposition
in classNormalizer2
- Parameters:
c
- code point- Returns:
- c's decomposition mapping, if any; otherwise null
-
getRawDecomposition
Gets the raw decomposition mapping of c.This is similar to the getDecomposition() method but returns the raw decomposition mapping as specified in UnicodeData.txt or (for custom data) in the mapping files processed by the gennorm2 tool. By contrast, getDecomposition() returns the processed, recursively-decomposed version of this mapping.
When used on a standard NFKC Normalizer2 instance, getRawDecomposition() returns the Unicode Decomposition_Mapping (dm) property.
When used on a standard NFC Normalizer2 instance, it returns the Decomposition_Mapping only if the Decomposition_Type (dt) is Canonical (Can); in this case, the result contains either one or two code points (=1..4 Java chars).
This function is independent of the mode of the Normalizer2. The default implementation returns null.
- Overrides:
getRawDecomposition
in classNormalizer2
- Parameters:
c
- code point- Returns:
- c's raw decomposition mapping, if any; otherwise null
-
composePair
public int composePair(int a, int b) Performs pairwise composition of a & b and returns the composite if there is one.Returns a composite code point c only if c has a two-way mapping to a+b. In standard Unicode normalization, this means that c has a canonical decomposition to a+b and c does not have the Full_Composition_Exclusion property.
This function is independent of the mode of the Normalizer2. The default implementation returns a negative value.
- Overrides:
composePair
in classNormalizer2
- Parameters:
a
- A (normalization starter) code point.b
- Another code point.- Returns:
- The non-negative composite code point if there is one; otherwise a negative value.
-
getCombiningClass
public int getCombiningClass(int c) Gets the combining class of c. The default implementation returns 0 but all standard implementations return the Unicode Canonical_Combining_Class value.- Overrides:
getCombiningClass
in classNormalizer2
- Parameters:
c
- code point- Returns:
- c's combining class
-
isNormalized
Tests if the string is normalized. Internally, in cases where the quickCheck() method would return "maybe" (which is only possible for the two COMPOSE modes) this method resolves to "yes" or "no" to provide a definitive result, at the cost of doing more work in those cases.- Specified by:
isNormalized
in classNormalizer2
- Parameters:
s
- input string- Returns:
- true if s is normalized
-
quickCheck
Tests if the string is normalized. For the two COMPOSE modes, the result could be "maybe" in cases that would take a little more work to resolve definitively. Use spanQuickCheckYes() and normalizeSecondAndAppend() for a faster combination of quick check + normalization, to avoid re-checking the "yes" prefix.- Specified by:
quickCheck
in classNormalizer2
- Parameters:
s
- input string- Returns:
- the quick check result
-
spanQuickCheckYes
Returns the end of the normalized substring of the input string. In other words, withend=spanQuickCheckYes(s);
the substrings.subSequence(0, end)
will pass the quick check with a "yes" result.The returned end index is usually one or more characters before the "no" or "maybe" character: The end index is at a normalization boundary. (See the class documentation for more about normalization boundaries.)
When the goal is a normalized string and most input strings are expected to be normalized already, then call this method, and if it returns a prefix shorter than the input string, copy that prefix and use normalizeSecondAndAppend() for the remainder.
- Specified by:
spanQuickCheckYes
in classNormalizer2
- Parameters:
s
- input string- Returns:
- "yes" span end index
-
hasBoundaryBefore
public boolean hasBoundaryBefore(int c) Tests if the character always has a normalization boundary before it, regardless of context. If true, then the character does not normalization-interact with preceding characters. In other words, a string containing this character can be normalized by processing portions before this character and starting from this character independently. This is used for iterative normalization. See the class documentation for details.- Specified by:
hasBoundaryBefore
in classNormalizer2
- Parameters:
c
- character to test- Returns:
- true if c has a normalization boundary before it
-
hasBoundaryAfter
public boolean hasBoundaryAfter(int c) Tests if the character always has a normalization boundary after it, regardless of context. If true, then the character does not normalization-interact with following characters. In other words, a string containing this character can be normalized by processing portions up to this character and after this character independently. This is used for iterative normalization. See the class documentation for details.Note that this operation may be significantly slower than hasBoundaryBefore().
- Specified by:
hasBoundaryAfter
in classNormalizer2
- Parameters:
c
- character to test- Returns:
- true if c has a normalization boundary after it
-
isInert
public boolean isInert(int c) Tests if the character is normalization-inert. If true, then the character does not change, nor normalization-interact with preceding or following characters. In other words, a string containing this character can be normalized by processing portions before this character and after this character independently. This is used for iterative normalization. See the class documentation for details.Note that this operation may be significantly slower than hasBoundaryBefore().
- Specified by:
isInert
in classNormalizer2
- Parameters:
c
- character to test- Returns:
- true if c is normalization-inert
-