<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Ravindra Harige</title>
    <description>The latest articles on DEV Community by Ravindra Harige (@rharige).</description>
    <link>https://dev.to/rharige</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F3870682%2F19155621-2d87-4624-9e13-1748ced863b1.jpeg</url>
      <title>DEV Community: Ravindra Harige</title>
      <link>https://dev.to/rharige</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/rharige"/>
    <language>en</language>
    <item>
      <title>Types of Search Queries: From Search Intent to Query Understanding</title>
      <dc:creator>Ravindra Harige</dc:creator>
      <pubDate>Tue, 15 Sep 2026 00:00:00 +0000</pubDate>
      <link>https://dev.to/rharige/types-of-search-queries-from-search-intent-to-query-understanding-38mm</link>
      <guid>https://dev.to/rharige/types-of-search-queries-from-search-intent-to-query-understanding-38mm</guid>
      <description>&lt;p&gt;When people talk about the types of search queries, they usually mean search intent: informational, navigational, commercial, and transactional. These categories describe what the searcher is trying to accomplish: learn something, find a particular destination, research options, or take an action.&lt;/p&gt;

&lt;p&gt;That classification is useful, especially for SEO. But it describes only one dimension of a search query: intent.&lt;/p&gt;

&lt;p&gt;Consider:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;nike waterproof running shoes under EUR 100&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;From an SEO perspective, this might be described as a commercial query. From an information-retrieval perspective, there is much more to understand. It targets a product category, contains attributes and a price constraint, expresses a comparison-oriented goal, and may require geographic, language, or contextual interpretation depending on the search system.&lt;/p&gt;

&lt;p&gt;Ask three search engineers what type of query this is, and you may get three different answers.&lt;/p&gt;

&lt;p&gt;One may call it a keyword query. Another may call it a category query with attributes and a price constraint. A third may care that it is standalone, English-language, and written by a person rather than generated by an application.&lt;/p&gt;

&lt;p&gt;All three can be right. They are describing different properties of the same request.&lt;/p&gt;

&lt;p&gt;This distinction matters because different search problems require different properties to be identified. A relevance engineer may distinguish exact lookup from exploratory search. A query-understanding system needs to detect ambiguity, entities, structure, and constraints. A conversational system may need prior turns to interpret what the user means. A production engineer may care about the computational work that the resulting request creates.&lt;/p&gt;

&lt;p&gt;So a search query does not necessarily have one type. It has a profile of characteristics.&lt;/p&gt;

&lt;p&gt;This makes the idea of query shape more precise: a query profile records the characteristics that matter for a particular search or engineering decision.&lt;/p&gt;

&lt;h2&gt;
  
  
  Nine dimensions of a query
&lt;/h2&gt;

&lt;p&gt;The information retrieval literature classifies queries by intent, ambiguity, target, session context, modality, language, and other properties. The map below brings those views together for practical system design. It is not a standard nine-part taxonomy, and the dimensions are not processing stages.&lt;/p&gt;

&lt;p&gt;Some distinctions are longstanding in information retrieval; others have become more operationally important as conversational, multimodal, and agent-generated search have expanded.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Dimension&lt;/th&gt;
&lt;th&gt;Engineering question&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Form&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;How is the request expressed?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Target and specificity&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Does the requester know the exact target, a class of targets, or only the general subject?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Internal structure&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Which entities, relationships, constraints, or subproblems appear in the request?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Specification and ambiguity&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;How much usable information does the request provide, and how many interpretations remain plausible?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Context dependence&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Can the request be understood on its own?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Intent and goal&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;What is the requester trying to accomplish?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Modality&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;What kind of input carries the request?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Language relation&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Which language does the query use, and how does it relate to the corpus?&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Origin and generation path&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Was the request written by a person, derived from a UI, or generated by an application or agent?&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Some characteristics are visible in the query. Others depend on the user, session, corpus, application, or task. Intent is often an inference, not an observed fact. A useful query profile preserves that uncertainty.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why these distinctions matter
&lt;/h2&gt;

&lt;p&gt;Query classification is useful when it changes a diagnosis, evaluation, or implementation decision.&lt;/p&gt;

&lt;h3&gt;
  
  
  Ambiguity can look like a ranking failure
&lt;/h3&gt;

&lt;p&gt;Suppose &lt;code&gt;jaguar&lt;/code&gt; returns wildlife documents when the user wanted the car brand.&lt;/p&gt;

&lt;p&gt;Boosting or reranking may not solve the problem. The system may have selected the wrong interpretation before ranking began. The same visible failure can require query clarification, entity interpretation, personalization, or ranking, depending on where it originated.&lt;/p&gt;

&lt;h3&gt;
  
  
  Context can change the evaluation task
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;show me cheaper ones&lt;/code&gt; makes little sense without the previous result set and the meaning of "cheaper" in that interaction.&lt;/p&gt;

&lt;p&gt;Putting the string into a relevance set without its preceding turn does more than add noise. It creates a different task. The evaluator is now testing a standalone query that no user actually issued.&lt;/p&gt;

&lt;h3&gt;
  
  
  Aggregate metrics can hide exact-query regressions
&lt;/h3&gt;

&lt;p&gt;A system can improve semantic matching for broad discovery queries while becoming worse at product codes, patent numbers, legal citations, or known-item requests.&lt;/p&gt;

&lt;p&gt;If those classes are averaged together, an apparently healthy aggregate result can conceal a serious product failure. Query profiles make it possible to evaluate the traffic segments that carry different success conditions.&lt;/p&gt;

&lt;h3&gt;
  
  
  Generated queries need their task context
&lt;/h3&gt;

&lt;p&gt;An agent-generated subquery may look like an ordinary text query. Its role can still be different: it may fill one evidence gap inside a longer research task. Treating every generated request as an independent human information need can distort evaluation and traffic analysis.&lt;/p&gt;

&lt;p&gt;The dimensions below provide vocabulary for locating these differences.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Form
&lt;/h2&gt;

&lt;p&gt;Form describes how the request is expressed.&lt;/p&gt;

&lt;p&gt;An &lt;strong&gt;identifier query&lt;/strong&gt; such as &lt;code&gt;AB-1234&lt;/code&gt;, &lt;code&gt;WO2026123456&lt;/code&gt;, or an ISBN may contain one token and still identify its target precisely. Product codes, patent numbers, error codes, ticket numbers, document IDs, and legal citations often need dedicated recognition rather than ordinary text matching.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;keyword query&lt;/strong&gt; such as &lt;code&gt;nike running shoes&lt;/code&gt; or &lt;code&gt;vector compression papers&lt;/code&gt; states the need telegraphically. Grammar and relationships are mostly implicit.&lt;/p&gt;

&lt;p&gt;Here, keyword describes the query's expression, not whether the system uses lexical, vector, or hybrid retrieval.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;structured or fielded query&lt;/strong&gt; makes part of the retrieval instruction explicit:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="s"&gt;author:smith&lt;/span&gt;
&lt;span class="s"&gt;brand:nike price:&amp;lt;100&lt;/span&gt;
&lt;span class="s"&gt;year:[2024 TO *]&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Boolean, phrase, proximity, exclusion, and wildcard syntax can encode much richer requests. These remain important in patent, legal, scientific, intelligence, and regulatory search. The open &lt;a href="https://nlp.stanford.edu/IR-book/html/htmledition/irbook.html" rel="noopener noreferrer"&gt;Introduction to Information Retrieval&lt;/a&gt; covers these established query constructs.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;natural-language query&lt;/strong&gt; states the need directly:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;comfortable waterproof shoes for walking all day
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Natural language can be standalone or dependent on earlier interaction. That dependence belongs to the context dimension, not the surface form alone.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Target and specificity
&lt;/h2&gt;

&lt;p&gt;Queries with the same form can express very different levels of target certainty.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;SKU-71829&lt;/code&gt; is an &lt;strong&gt;exact lookup&lt;/strong&gt; : the identifier names the target.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;the Brin and Page paper that introduced PageRank&lt;/code&gt; is a &lt;strong&gt;known-item query&lt;/strong&gt; : the requester seeks one particular document believed to exist. This usage follows the established &lt;a href="https://www.nist.gov/publications/trec-5-confusion-track-comparing-retrieval-methods-scanned-text" rel="noopener noreferrer"&gt;known-item retrieval task&lt;/a&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;running shoes&lt;/code&gt; names a &lt;strong&gt;category&lt;/strong&gt; with many acceptable results.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;red waterproof hiking shoes under EUR 150&lt;/code&gt; is &lt;strong&gt;attribute-constrained&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;methods for reducing hallucination in retrieval-augmented generation&lt;/code&gt; is &lt;strong&gt;conceptual&lt;/strong&gt; : the need is clear, but the best documents may use different vocabulary.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;good resources for learning information retrieval&lt;/code&gt; is &lt;strong&gt;exploratory&lt;/strong&gt; : search helps the requester discover the answer space.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These classes do not share one success condition. Missing the intended known item can make the search a complete failure. Exploratory search can return several different sets of useful results. Segmenting them can reveal regressions hidden by an aggregate metric.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. Internal structure
&lt;/h2&gt;

&lt;p&gt;Query length is a poor proxy for query complexity. &lt;code&gt;jaguar&lt;/code&gt; is syntactically simple and semantically difficult. &lt;code&gt;running shoes under EUR 100&lt;/code&gt; is short but combines a category with a numeric constraint.&lt;/p&gt;

&lt;p&gt;Common structural patterns include:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Atomic:&lt;/strong&gt; &lt;code&gt;jaguar&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Entity plus attribute:&lt;/strong&gt; &lt;code&gt;Nike Air Max red&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Constraint-bearing:&lt;/strong&gt; &lt;code&gt;running shoes under EUR 100&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Exclusion-bearing:&lt;/strong&gt; &lt;code&gt;running shoes without Gore-Tex&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Comparative:&lt;/strong&gt; &lt;code&gt;Pixel versus iPhone battery life&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Relational:&lt;/strong&gt; &lt;code&gt;papers citing ColBERT&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Temporal:&lt;/strong&gt; &lt;code&gt;changes since January 2025&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Spatial:&lt;/strong&gt; &lt;code&gt;hotels within 5 km of Schiphol&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Compositional:&lt;/strong&gt; &lt;code&gt;waterproof hiking shoes for wide feet under EUR 150&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Multi-part:&lt;/strong&gt; &lt;code&gt;What changed between the original contract and the amendment, and which termination clauses still apply?&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The last example contains several retrieval needs: find the correct versions, locate the clauses, compare them, and determine what remains effective. A RAG system or agent may decompose it into multiple searches, but the structure exists before that implementation choice.&lt;/p&gt;

&lt;p&gt;Constraints can also arrive outside the visible text. A user may type &lt;code&gt;running shoes&lt;/code&gt; and select a price facet in the interface. The semantic requirement resembles &lt;code&gt;running shoes under EUR 100&lt;/code&gt;, but the application constructs it differently. That difference helps locate whether a failure came from query understanding, UI state, filtering, or serving logic.&lt;/p&gt;

&lt;h2&gt;
  
  
  4. Specification and ambiguity
&lt;/h2&gt;

&lt;p&gt;Length does not tell us whether a query is well specified.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;SKU-72918&lt;/code&gt; is short and precise. &lt;code&gt;running shoes&lt;/code&gt; is &lt;strong&gt;underspecified&lt;/strong&gt; because use case, fit, size, terrain, and price remain open. &lt;code&gt;jaguar&lt;/code&gt; is &lt;strong&gt;ambiguous&lt;/strong&gt; because several distinct interpretations are plausible. &lt;code&gt;iphnoe case&lt;/code&gt; is &lt;strong&gt;noisy&lt;/strong&gt; : its likely meaning is specific despite the spelling error. &lt;code&gt;nice laptop&lt;/code&gt; is &lt;strong&gt;vague&lt;/strong&gt; because "nice" provides little operational guidance.&lt;/p&gt;

&lt;p&gt;Ambiguous queries are a distinct research problem. Song et al. study how to identify ambiguity and separate it from more specific queries in &lt;a href="https://doi.org/10.1016/j.ipm.2008.09.005" rel="noopener noreferrer"&gt;Identification of Ambiguous Queries in Web Search&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Long requests create a different challenge. They may contain hard constraints, use-case context, preferences, and explanatory language, all with different retrieval importance. More words can add information without removing uncertainty.&lt;/p&gt;

&lt;p&gt;Ambiguity, underspecification, noise, vagueness, and verbosity call for different responses. Combining them into one "difficult query" class removes information an engineer can act on.&lt;/p&gt;

&lt;h2&gt;
  
  
  5. Context dependence
&lt;/h2&gt;

&lt;p&gt;A &lt;strong&gt;standalone query&lt;/strong&gt; such as &lt;code&gt;waterproof hiking shoes&lt;/code&gt; carries most of its meaning in the expression itself.&lt;/p&gt;

&lt;p&gt;A &lt;strong&gt;session-dependent query&lt;/strong&gt; may rely on an earlier search:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;electric cars
used under 30k
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;a href="https://trec.nist.gov/data/session.html" rel="noopener noreferrer"&gt;TREC Session Track&lt;/a&gt; studied retrieval over sessions using earlier queries, results, clicks, and dwell time rather than treating every query in isolation.&lt;/p&gt;

&lt;p&gt;Conversational dependence is more explicit:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;what about the cheaper one?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;a href="https://pages.nist.gov/trec-browser/trec28/cast/overview/" rel="noopener noreferrer"&gt;TREC Conversational Assistance Track&lt;/a&gt; defines effective response selection as requiring the question's dialogue history.&lt;/p&gt;

&lt;p&gt;Other queries depend on user context (&lt;code&gt;my recent invoices&lt;/code&gt;), environmental context (&lt;code&gt;coffee near me&lt;/code&gt;, &lt;code&gt;what is open now?&lt;/code&gt;), or task state (&lt;code&gt;recall notices for the remaining SKUs&lt;/code&gt;). Location, time, identity, permissions, previous results, and workflow state can all be part of the effective query even when they are absent from the text.&lt;/p&gt;

&lt;p&gt;One query, several dimensions&lt;/p&gt;

&lt;p&gt;&lt;code&gt;show me cheaper ones&lt;/code&gt;&lt;/p&gt;

&lt;dl&gt;

&lt;dt&gt;Form&lt;/dt&gt;
&lt;dd&gt;Natural language&lt;/dd&gt;


&lt;dt&gt;Structure&lt;/dt&gt;
&lt;dd&gt;Comparative&lt;/dd&gt;


&lt;dt&gt;Context&lt;/dt&gt;
&lt;dd&gt;Conversational&lt;/dd&gt;


&lt;dt&gt;Intent&lt;/dt&gt;
&lt;dd&gt;Refinement&lt;/dd&gt;


&lt;dt&gt;Origin&lt;/dt&gt;
&lt;dd&gt;Human&lt;/dd&gt;

&lt;/dl&gt;

&lt;h2&gt;
  
  
  6. Intent and goal
&lt;/h2&gt;

&lt;p&gt;Intent describes what the requester is trying to accomplish. It is the best-known query taxonomy, but it is only one dimension.&lt;/p&gt;

&lt;p&gt;Broder's &lt;a href="https://research.google/pubs/a-taxonomy-of-web-search/" rel="noopener noreferrer"&gt;web-search taxonomy&lt;/a&gt; distinguishes informational, navigational, and transactional needs. &lt;a href="https://arxiv.org/abs/2205.00926" rel="noopener noreferrer"&gt;ORCAS-I&lt;/a&gt; further separates informational queries into factual and instrumental subclasses, with an abstain category for uncertain cases.&lt;/p&gt;

&lt;p&gt;Intent classes can also be domain-specific. Research on &lt;a href="https://doi.org/10.1145/3209978.3210152" rel="noopener noreferrer"&gt;e-commerce query taxonomies&lt;/a&gt; derives shopping-oriented classes from commercial search behavior. Enterprise, legal, support, and RAG systems may need different labels because they serve different decisions.&lt;/p&gt;

&lt;p&gt;Intent is usually inferred. The query &lt;code&gt;jaguar&lt;/code&gt; does not reveal whether the user wants information, a website, a product, an animal, a car, or something else. Context and behavior may change the estimate.&lt;/p&gt;

&lt;h2&gt;
  
  
  7. Modality
&lt;/h2&gt;

&lt;p&gt;Queries are not limited to text.&lt;/p&gt;

&lt;p&gt;An image may retrieve visually similar products. Speech may express the same need as typed text while introducing recognition errors and more conversational phrasing. Inputs can also combine modalities:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;[shoe image] + "something like this, but in black"
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Neither part contains the complete request alone.&lt;/p&gt;

&lt;p&gt;Query modality and result modality are separate properties. &lt;code&gt;red Ferrari&lt;/code&gt; used to retrieve images is a text query against an image corpus. A photograph used to retrieve similar images is an image query. A photograph plus a text constraint is multimodal.&lt;/p&gt;

&lt;p&gt;In &lt;a href="https://www.ijcai.org/proceedings/2022/759" rel="noopener noreferrer"&gt;cross-modal image-text retrieval&lt;/a&gt;, the query and retrieval collection belong to different modalities. Naming both sides avoids using "image search" for several different system shapes.&lt;/p&gt;

&lt;h2&gt;
  
  
  8. Language relation
&lt;/h2&gt;

&lt;p&gt;Language becomes a relational property when the query and corpus differ.&lt;/p&gt;

&lt;p&gt;A system may serve one language, accept several languages against a multilingual corpus, receive code-switched queries such as &lt;code&gt;beste noise cancelling headphones onder EUR 300&lt;/code&gt;, or handle transliteration and mixed scripts.&lt;/p&gt;

&lt;p&gt;In &lt;strong&gt;cross-language information retrieval&lt;/strong&gt; , the query and documents use different languages:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Dutch query -&amp;gt; English documents
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Kishida's &lt;a href="https://doi.org/10.1016/j.ipm.2004.06.007" rel="noopener noreferrer"&gt;review of cross-language information retrieval&lt;/a&gt; defines the problem in those terms and surveys the translation and matching choices it creates.&lt;/p&gt;

&lt;p&gt;Identifiers such as &lt;code&gt;WO2026123456&lt;/code&gt;, &lt;code&gt;BRCA1&lt;/code&gt;, and &lt;code&gt;ERR_CONNECTION_RESET&lt;/code&gt; may require little or no linguistic translation. The system still needs to preserve them accurately across tokenization, normalization, and matching.&lt;/p&gt;

&lt;h2&gt;
  
  
  9. Origin and generation path
&lt;/h2&gt;

&lt;p&gt;Origin records how a query or retrieval request was produced. It is provenance, not meaning.&lt;/p&gt;

&lt;p&gt;A request may be:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Human-authored&lt;/strong&gt; , through typed or spoken input.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;UI-derived&lt;/strong&gt; , when facets, map movement, sorting, or a saved view contribute structured parameters.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Application-generated&lt;/strong&gt; , for recommendations, related items, dashboards, monitoring, or enrichment jobs.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Agent-generated&lt;/strong&gt; , as one search inside a longer plan.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Consider a user asking:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;Which of our products had safety recalls in Europe this year?
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;An agent may identify products, search regulatory sources, inspect the evidence, and issue follow-up searches for missing jurisdictions or product variants. One intermediate query might be:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;EU safety recall notices for Model X in 2026
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Nothing in that string makes it intrinsically agentic. Its distinguishing property is its origin and role inside a larger task. A person and an agent can issue identical text while creating different context, evaluation, and traffic assumptions.&lt;/p&gt;

&lt;h2&gt;
  
  
  Build a query profile, not a single label
&lt;/h2&gt;

&lt;p&gt;A profile combines only the dimensions relevant to the engineering problem.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Query&lt;/th&gt;
&lt;th&gt;Useful profile&lt;/th&gt;
&lt;th&gt;Evaluation risk&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;AB-1234&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Identifier, exact lookup, standalone, language-light&lt;/td&gt;
&lt;td&gt;Aggregate relevance can hide lookup failure&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;jaguar&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Keyword, atomic, ambiguous, uncertain intent&lt;/td&gt;
&lt;td&gt;Judgments may mix incompatible interpretations&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;red hiking shoes under EUR 150&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Keyword, category, attribute-constrained, standalone&lt;/td&gt;
&lt;td&gt;Attribute extraction, filtering, and ranking failures can be conflated&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;what changed between the agreement and amendment?&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Natural language, known evidence set, comparative, temporal, multi-part&lt;/td&gt;
&lt;td&gt;Version selection and retrieval may be confused with synthesis&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;code&gt;EU safety recall notices for Model X in 2026&lt;/code&gt;&lt;/td&gt;
&lt;td&gt;Keyword, entity-specific, temporal, task-dependent, agent-generated&lt;/td&gt;
&lt;td&gt;A subquery may be evaluated without its parent task or evidence gap&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;Not every value is observable ground truth. Form and modality are often visible. Intent, target, and ambiguity may be hypotheses. Store confidence or an unknown value where certainty matters instead of forcing every query into a complete profile.&lt;/p&gt;

&lt;h2&gt;
  
  
  Query profile is not query workload
&lt;/h2&gt;

&lt;p&gt;A &lt;strong&gt;query&lt;/strong&gt; is the expression or input supplied by a person, application, or agent. A &lt;strong&gt;retrieval request&lt;/strong&gt; is broader. It may add filters, permissions, ranking configuration, candidate depth, sorting, grouping, timeout, and other serving parameters.&lt;/p&gt;

&lt;p&gt;The same logical query can therefore execute as:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;BM25
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;or:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;BM25 + filtered ANN
-&amp;gt; fusion
-&amp;gt; reranking
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Its semantic profile may be unchanged while its execution cost, candidate coverage, filtering behavior, and latency change substantially.&lt;/p&gt;

&lt;p&gt;A query profile describes the query's meaning, expression, context, and provenance. A query workload describes the distribution of retrieval requests and the work they create under production traffic.&lt;/p&gt;

&lt;p&gt;The distribution of query length, clause count, filters, candidate depth, QPS, concurrency, latency budgets, and requests per task helps define that workload. Corpus size alone does not.&lt;/p&gt;

&lt;p&gt;Keeping the concepts separate prevents two different questions from collapsing into one:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;What must the system understand about this request?&lt;/li&gt;
&lt;li&gt;What must the system do to serve it under production conditions?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Both feed into the wider &lt;a href="https://www.searchplex.net/retrieval-foundation" rel="noopener noreferrer"&gt;Retrieval Foundation&lt;/a&gt;, but they lead to different diagnostics and design decisions.&lt;/p&gt;

&lt;h2&gt;
  
  
  Use the dimensions that change a decision
&lt;/h2&gt;

&lt;p&gt;The map is not a requirement to build nine classifiers.&lt;/p&gt;

&lt;p&gt;Classify a dimension when it changes what the system should do or how it should be evaluated. Form can change parsing. Target specificity can change matching and success metrics. Structure can trigger filters or decomposition. Context can determine whether an evaluation example is valid. Intent can influence ranking and presentation. Modality and language determine available representations. Origin can affect provenance, traffic analysis, and task-level evaluation.&lt;/p&gt;

&lt;p&gt;Use the query profile to describe what the request means and what context it needs. Use the workload profile to describe what serving those requests makes the system do.&lt;/p&gt;

&lt;p&gt;That is enough precision to stop asking for one universal query type and start asking which characteristics matter for the decision at hand.&lt;/p&gt;

</description>
      <category>queryunderstanding</category>
      <category>informationretrieval</category>
      <category>searchevaluation</category>
      <category>searchworkload</category>
    </item>
    <item>
      <title>Why Search Breaks in Production</title>
      <dc:creator>Ravindra Harige</dc:creator>
      <pubDate>Thu, 09 Apr 2026 23:51:58 +0000</pubDate>
      <link>https://dev.to/rharige/why-search-breaks-in-production-4m5a</link>
      <guid>https://dev.to/rharige/why-search-breaks-in-production-4m5a</guid>
      <description>&lt;p&gt;Originally published on &lt;a href="https://www.searchplex.net/blog/what-makes-search-hard" rel="noopener noreferrer"&gt;Searchplex Blog&lt;/a&gt;&lt;/p&gt;




&lt;p&gt;Similar-looking search systems can behave very differently in production. Take two search systems. Both have an index of around 100k docs.&lt;/p&gt;

&lt;p&gt;One is a search system for construction and civil engineering documents. Queries look like &lt;code&gt;latest approved floor plan for basement mechanical room&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Another is an ecommerce search system. Queries look like &lt;code&gt;waterproof hiking shoes size 11 under 150, in stock, sorted by rating&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;Both support hybrid search, filters, reranking - so, same broad feature set. But they behave differently in the production. Document count is often considered an useful proxy for scale, but that explains only part of the system's behavior. The difference comes from the &lt;b&gt;interaction&lt;/b&gt; of the six factors in the search engine: query shape, document shape, retrieval scope, execution shape, operating pressure, and product contract.&lt;/p&gt;

&lt;h2&gt;
  
  
  Search systems are more than retrieval
&lt;/h2&gt;

&lt;p&gt;At the interface boundary, search can look simple:&lt;/p&gt;

&lt;p&gt;query -&amp;gt; index lookup -&amp;gt; ranked results&lt;/p&gt;

&lt;p&gt;Once the system carries real product responsibility, retrieval has to work together with filters, counts, sorting, paging, reranking, freshness, access control, and business rules.&lt;/p&gt;

&lt;p&gt;The hard part is in those interactions. Each mechanism can change the behavior of the others, while users experience the results as one system. &lt;/p&gt;

&lt;h2&gt;
  
  
  Query shape, document shape, and retrieval scope
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Query shape.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Queries do different jobs, and they vary on the levels of complexity:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;state-aware lookup, e.g. &lt;code&gt;latest approved floor plan for basement mechanical room&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;constraint-heavy retrieval, e.g. &lt;code&gt;waterproof hiking shoes size 11 under 100, in stock&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;known-item lookup, e.g. &lt;code&gt;Nike Pegasus 41&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;scoped similarity search, e.g. &lt;code&gt;find similar language across these custodians&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;comparison over a bounded set, e.g. &lt;code&gt;compare concrete curing requirements across these three revisions&lt;/code&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These are different retrieval job. They drive different execution shape, and different forms of hardware pressures.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Document shape.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Short product records, long contracts, revision-heavy drawing packages, email threads, multilingual content, and mixed-content PDFs do not create the same retrieval problem. Document shape affects how content is structured, segmented, stored, and matched against the user query.&lt;/p&gt;

&lt;p&gt;The retrieval unit (e.g. chunks, fields, etc.) often matters more than the document the document abstraction suggests. Teams often think they are debugging documents when they are really debugging retrieval units.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Retrieval scope.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
The system rarely searches everything it stores. It searches the live slice for this request: one project, one tenant, one matter, one seller slice, a full catalog, a portfolio-wide index.&lt;/p&gt;

&lt;p&gt;That scope defined the retrieval job. A narrow, well-structured scope can make a large stored collection cheap to search. A smaller collection with a broad or unstable live scope can be much harder to serve.&lt;/p&gt;

&lt;h2&gt;
  
  
  Execution shape
&lt;/h2&gt;

&lt;p&gt;Execution shape is how the system turns that retrieval job into results.&lt;/p&gt;

&lt;p&gt;Lexical, vector, and hybrid retrieval are only the headline labels. Production behavior usually depends on lower-level choices.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Approximation.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Exact search and ANN change latency, recall, memory use, filtering behavior, and update cost. They also change what kinds of surprises the system produces under load.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Candidate generation.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
A reranker only improves what it gets to see. If the right items never enter the candidate set, the reranker has nothing to rescue. Many ranking problems start as candidate-generation problems and only show up later as "relevance" complaints.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Filter timing.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Pre-search filtering defines eligibility before candidate generation. Post-search filtering trims what a broader search already surfaced. Those are not the same system. In vector and hybrid retrieval, that difference often decides recall, latency, and whether counts, hits, and ranking stay aligned.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Seams.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Responsibility is often split across retrieval, filtering, fusion, reranking, application logic, business rules, and sometimes a generative layer. Each boundary is another place where relevance shifts, latency accumulates, and debugging crosses ownership lines. When the same retrieval layer feeds RAG or multi-step agents, those shifts are carried forward into the answer layer.&lt;/p&gt;

&lt;h2&gt;
  
  
  Operating pressures and Product contract
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Hardware pressure.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
CPU, RAM, storage, cache behavior, and headroom decide how much inefficiency the system can absorb before users notice.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Update pressure.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Static corpora, daily batch workloads, and continuously updated indexes are different systems. Refresh behavior, merge behavior, stale state, and indexing cost start to matter as soon as retrieval structures have to stay current.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Traffic pressure.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
QPS and concurrency expose hidden cost fast. A system used by a few people can look fine with late filtering, over-retrieval, and expensive reranking. The same system under hundreds of concurrent users, while indexes are also updating, can become unstable quickly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Product contract.&lt;/strong&gt;&lt;br&gt;&lt;br&gt;
Low latency, stable paging, reliable counts, current state, explanation, auditability, and policy correctness are not the same kind of requirement, but they all define what "working" means. A system that is acceptable for internal search can be unusable for marketplace ranking or compliance review.&lt;/p&gt;

&lt;p&gt;This is why teams report opposite experiences with superficially similar systems. They operating under different pressures, and against different product contracts.&lt;/p&gt;

&lt;p&gt;&lt;a href="/blog/blog-table.png" class="article-body-image-wrapper"&gt;&lt;img src="/blog/blog-table.png" alt="Comparison of construction or civil document search versus ecommerce or marketplace search across corpus, features, users, query shape, document shape, retrieval scope, execution shape, pressure, and product contract."&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Where failures show up
&lt;/h2&gt;

&lt;p&gt;Most failures collapse into three buckets:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;recall failure&lt;/strong&gt;: the right thing never enters the candidate set&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;scope failure&lt;/strong&gt;: the system searches or returns the wrong slice&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;ranking failure&lt;/strong&gt;: the right candidates are present, but the ordering is weak or unstable&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;One more term belongs here:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;cracks&lt;/strong&gt;: seams that become visible under pressure&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Cracks show up when concurrent user queries are happening, simultaneously index is upating, counts diverge from matching hits, paging wobbles after reranking, lexical and vector paths disagree under filters, freshness diverges across paths, or one bad result requires tracing three different layers to explain. &lt;/p&gt;

&lt;h2&gt;
  
  
  Case in point: hybrid search
&lt;/h2&gt;

&lt;p&gt;Hybrid search exposes these problems quickly because it combines more than one retrieval behavior inside one product surface.&lt;/p&gt;

&lt;p&gt;The hybrid search is &lt;code&gt;lexical + vector&lt;/code&gt; with optionally, an &lt;code&gt;reranker&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;In production, the system has to present retrieval, filters, aggregations, sorting, paging, reranking, freshness, explanation, and business rules as one coherent result model.&lt;/p&gt;

&lt;p&gt;Each part pulls in a different directions. Filters want the right slice. Aggregations want counts over that slice. Sorting wants stable order. Paging wants page two to behave like page two. Reranking wants a rich candidate set. Vector retrieval wants semantic proximity. Lexical retrieval wants exact term sensitivity.&lt;/p&gt;

&lt;p&gt;This is where systems start to wobble in visible ways: page one looks plausible, counts drift, filters expose disagreements between retrieval paths - lexical and vectors, and latency climbs as more work gets pushed into reranking or application logic.&lt;/p&gt;

&lt;p&gt;In RAG systems, the hybrid retrieval inconsistencies propogate into context selection, grounding, and the final answer.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three systems with similar corpus size
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Construction and engineering search&lt;/strong&gt; is typically revision-heavy and approval-sensitive.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;latest approved floor plan for basement mechanical room&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;fire suppression specification for the east wing&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The dominant pressure is workflow-state correctness. What cracks first is usually scope, revision, or state coherence, especially around words like &lt;code&gt;latest&lt;/code&gt; and &lt;code&gt;approved&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Ecommerce and marketplace search&lt;/strong&gt; is typically filter-heavy, freshness-sensitive, and ranking-sensitive.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;waterproof hiking shoes size 11 under 150, in stock&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;espresso machine with grinder sorted by rating&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The dominant pressure is keeping filters, ranking, paging, and current state coherent under interactive load. What cracks first is usually paging stability, facet coherence, freshness correctness, or tail latency. The result often looks plausible on page one and inconsistent by page two.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Legal, litigation, and compliance retrieval&lt;/strong&gt; is typically metadata-heavy, scope-heavy, and explanation-sensitive.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;code&gt;emails discussing pricing strategy between January and March&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;&lt;code&gt;find similar language across these custodians&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The dominant pressure is defensibility under selective scope and permissions. What cracks first is usually scope correctness, family or thread coherence, or auditability. A system can look relevant and still be unusable if the scope is even slightly wrong.&lt;/p&gt;

&lt;p&gt;Same corpus size. Different retrieval job. Different execution shape. Different pressure. Different cracks.&lt;/p&gt;

&lt;h2&gt;
  
  
  The takeaway
&lt;/h2&gt;

&lt;p&gt;Search gets hard when multiple parts of the system have to agree on one result set under real operating pressure. Concurrent search queries, hardware limits, evolving indexes, latency and freshness requirements, LLM serving dependencies, etc. all strain that agreement. The failures become visible when the system stops behaving like one coherent system.&lt;/p&gt;

&lt;p&gt;Demos mislead for the same reason. Similar corpus sizes can produce very different systems, and the same architecture can feel acceptable in one product and broken in another.&lt;/p&gt;

&lt;p&gt;The same is true for production AI. When retrieval stops behaving like one coherent system, the answer layer inherits the failure.&lt;/p&gt;

</description>
      <category>ai</category>
      <category>elasticsearch</category>
      <category>vectordatabase</category>
      <category>rag</category>
    </item>
  </channel>
</rss>
