Highlighting

The Unified Highlighter is the newest highlighter (as of Solr 6.4), which stands out as the most performant and accurate of the options. It can handle typical requirements and others possibly via plugins/extension. We recommend that you use this highlighter as a first choice.

The UH highlights a query very accurately and thus is true to what the underlying Lucene query actually matches. Other highlighters highlight terms more liberally (over-highlight). For esoteric/custom queries, this highlighter has a greater likelihood of supporting it than the others.

A strong benefit to this highlighter is that you can opt to configure Solr to put more information in the underlying index to speed up highlighting of large documents; multiple configurations are supported, even on a per-field basis. There is little or no such flexibility of offset sources for the other highlighters. More on this below.

There are some reasons not to choose this highlighter: Passage scoring does not consider boosts in the query. Some users want more/better passage breaking flexibility. The "alternate" fallback options are more primitive.

The Original Highlighter, sometimes called the "Standard Highlighter" or "Default Highlighter", is Lucene’s original highlighter – a venerable option with a high degree of customization options. Its query accuracy is good enough for most needs, although it’s not quite as good/perfect as the Unified Highlighter.

The Original Highlighter will normally analyze stored text on the fly in order to highlight. It will use full term vectors if available. If the text isn’t "stored" but is in doc values (docValues="true"), this highlighter can work with it.

Where this highlighter falls short is performance; it’s often twice as slow as the Unified Highlighter. And despite being the most customizable, it doesn’t have a BreakIterator based fragmenter (all the others do), which could pose a challenge for some languages.

The FastVector Highlighter requires full term vector options (termVectors, termPositions, and termOffsets) on the field, and is optimized with that in mind. It is nearly as configurable as the Original Highlighter with some variability.

This highlighter notably supports multi-colored highlighting such that different query words can be denoted in the fragment with different marking, usually expressed as an HTML tag with a unique color.

This highlighter’s query-representation is less advanced than the Original or Unified Highlighters: for example it will not work well with the surround parser, and there are multiple reported bugs pertaining to queries with stop-words.

By default, the Unified Highlighter will usually pick the right offset source (see above). However it may be ambiguous such as during a migration from one offset source to another that hasn’t completed.

The offset source can be explicitly configured to one of: ANALYSIS, POSTINGS, POSTINGS_WITH_TERM_VECTORS, or TERM_VECTORS.

This parameter influences where the first match (i.e., highlighted text) in a passage is positioned.

The default value of 0.33 means to align the match to the left third. A value of 0.0 means to align the match to the left, while 1.0 to align it to the right. This setting is a best-effort hint, as there are a variety of factors. When there’s lots of text to be highlighted, lowering this number can help performance a lot.

When true, the hl.fragsize parameter is treated as a (soft) minimum fragment size; provided there is enough text, the fragment is at least this size. When false, it’s an optimal target — the highlighter will on average produce highlights of this length. A false setting is slower, particularly when there’s lots of text and hl.bs.type=SENTENCE.

By default, each snippet is returned as a separate value (as is done with the other highlighters). Set this parameter to instead return one string with this text as the delimiter. Note: this is likely to be removed in the future.

If true, use the leading portion of the text as a snippet if a proper highlighted snippet can’t otherwise be generated.

Specifies BM25 term frequency normalization parameter 'k1'. For example, it can be set to 0 to rank passages solely based on the number of query terms that match.

Specifies BM25 length normalization parameter 'b'. For example, it can be set to "0" to ignore the length of passages entirely when ranking.

Specifies BM25 average passage length in characters.

Specifies the breakiterator language for dividing the document into passages.

Specifies the breakiterator country for dividing the document into passages.

Specifies the breakiterator variant for dividing the document into passages.

Specifies the breakiterator type for dividing the document into passages. Can be SEPARATOR, SENTENCE, WORD*, CHARACTER, LINE, or WHOLE. SEPARATOR is special value that splits text on a user-provided character in hl.bs.separator.

Indicates which character to break the text on. Use only if you have defined hl.bs.type=SEPARATOR.

This is useful when the text has already been manipulated in advance to have a special delineation character at desired highlight passage boundaries. This character will still appear in the text as the last character of a passage.

Tells the UH to use Lucene’s "Weight Matches" API instead of doing SpanQuery conversion. This is the most accurate highlighting mode reflecting the query. Furthermore, phrases will be highlighted as a whole instead of word by word. Currently, this setting slows down the unified highlighter a lot when many fields are highlighted.

If either hl.usePhraseHighlighter or hl.multiTermQuery are set to false, then this setting is effectively false no matter what you set it to.

Instructs Solr to collapse contiguous fragments into a single fragment. A value of true indicates contiguous fragments will be collapsed into single fragment.

Specifies the maximum number of entries in a multi-valued field to examine before stopping. This can potentially return zero results if the limit is reached before any matches are found.

If used with the maxMultiValuedToMatch, whichever limit is reached first will determine when to stop looking.

Specifies the maximum number of matches in a multi-valued field that are found before stopping.

If hl.maxMultiValuedToExamine is also defined, whichever limit is reached first will determine when to stop looking.

Specifies a field to be used as a backup default summary if Solr cannot generate a snippet (i.e., because no terms match).

Specifies the maximum number of characters of the field to return. Any value less than or equal to 0 means the field’s length is unlimited.

This parameter is only used in conjunction with the hl.alternateField parameter.

If set to true and hl.alternateFieldName is active, Solr will show the entire alternate field, with highlighting of occurrences. If hl.maxAlternateFieldLength=N is used, Solr returns max N characters surrounding the best matching fragment.

If set to false, or if there is no match in the alternate field either, the alternate field will be shown without highlighting.

Selects a formatter for the highlighted output. Currently the only legal value is simple, which surrounds a highlighted term with a customizable pre- and post-text snippet.

Specifies the text that should appear before (hl.simple.pre) and after (hl.simple.post) a highlighted term, when using the simple formatter. The default is <em> and </em>.

Specifies a text snippet generator for highlighted text. The standard fragmenter is gap, which creates fixed-sized fragments with gaps for multi-valued fields.

Another option is regex, which tries to create fragments that resemble a specified regular expression.

When using the regex fragmenter (hl.fragmenter=regex), this parameter defines the factor by which the fragmenter can stray from the ideal fragment size (given by hl.fragsize) to accommodate a regular expression.

For instance, a slop of 0.2 with hl.fragsize=100 should yield fragments between 80 and 120 characters in length. It is usually good to provide a slightly smaller hl.fragsize value when using the regex fragmenter.

Specifies the regular expression for fragmenting. This could be used to extract sentences.

Instructs Solr to analyze only this many characters from a field when using the regex fragmenter (after which, the fragmenter produces fixed-sized fragments).

Note, applying a complicated regex to a huge field is computationally expensive.

If true, multi-valued fields will return all values in the order they were saved in the index. If false, only values that match the highlight request will be returned.

When hl.usePhraseHighlighter is true and the indexed field has payloads but not term vectors (generally quite rare), the index’s payloads will be read into the highlighter’s memory index along with the postings.

If this may happen and you know you don’t need them for highlighting (i.e., your queries don’t filter by payload) then you can save a little memory by setting this to false.

The snippet fragmenting algorithm. The weighted fragListBuilder uses IDF-weights to order fragments.

Other options are single, which returns the entire field contents as one snippet, or simple. You can select a fragListBuilder with this parameter, or modify an existing implementation in solrconfig.xml to be the default by adding "default=true".

The fragments builder is responsible for formatting the fragments, which uses <em> and </em> markup by default (if hl.tag.pre and hl.tag.post are not defined).

Another pre-configured choice is colored, which is an example of how to use the fragments builder to insert HTML into the snippets for colored highlights if you choose. You can also implement your own if you’d like. You can select a fragments builder with this parameter, or modify an existing implementation in solrconfig.xml to be the default by adding "default=true".

The maximum number of phrases to analyze when searching for the highest-scoring phrase.

Text to use to separate one value from the next for a multi-valued field. The default is " " (a space).