com.jivesoftware.util.search
Class CJKAnalyzer

java.lang.Object
  org.apache.lucene.analysis.Analyzer
      com.jivesoftware.util.search.CJKAnalyzer

public final class CJKAnalyzer
extends org.apache.lucene.analysis.Analyzer

An analyzer for Chinese, Japanese and Korean.

Nested Class Summary

class

CJKAnalyzer.CJKTokenizer CJKTokenizer was modified from StopTokenizer which does a decent job for most European languages and it performs other token method for double-byte Characters: the token will return at each two charactors with overlap match.

Field Summary

static java.lang.String[]

STOP_WORDS

Constructor Summary

CJKAnalyzer()
Builds an analyzer.

CJKAnalyzer(java.lang.String[] stopWords)
Builds an analyzer with the given stop words.

Method Summary

org.apache.lucene.analysis.TokenStream

tokenStream(java.lang.String fieldName, java.io.Reader reader)

Methods inherited from class org.apache.lucene.analysis.Analyzer

getPositionIncrementGap

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

STOP_WORDS

public static final java.lang.String[] STOP_WORDS

Constructor Detail

CJKAnalyzer

public CJKAnalyzer()

Builds an analyzer.

CJKAnalyzer

public CJKAnalyzer(java.lang.String[] stopWords)

Builds an analyzer with the given stop words.

Method Detail

tokenStream

public org.apache.lucene.analysis.TokenStream tokenStream(java.lang.String fieldName,
                                                          java.io.Reader reader)

Specified by:

tokenStream in class org.apache.lucene.analysis.Analyzer