Interactive explainer

RAG foundation: retrieve source context with TF-IDF

A transparent retrieval experiment: enter a query, select sources and inspect conflicts. The output consists of document excerpts, not generated text.

Interactive

RAG foundation: retrieve source context with TF-IDF

The local search calculates TF-IDF similarities and displays retrieved documents as source context. No language model runs.

What is calculated

Five fictional documents describe purchasing, travel and IT incidents. Two purchasing rules contain different approval limits. The older version is explicitly archived. The query and each document are converted into weighted word vectors and compared using cosine similarity.

The calculator displays only matches with a positive retrieval score. It selects the highest-scoring sources up to the chosen limit. Weighting always uses all five documents; the filter changes only which sources are eligible. Identifier, date, status and text remain visible. Nothing is presented as a generated answer or as output from a language model.

Three experiments

  1. Find a contradiction: Search for “purchase approval”, allow five sources and include the archive. Compare the limits in P-2026 and P-2024.
  2. Consider validity: Switch to current sources only. The older rule is excluded. Check whether the remaining context actually answers your question.
  3. Missing evidence: Enter “quantumbanana”. With no matching words, no result appears. Then try “hotel” to retrieve a different part of the collection.

Reading the result

A retrieval score measures word similarity, not truth. The conflict notice compares predefined fact annotations; it does not understand the text. Synonyms and translations are not recognised either. You can expand the complete collection below the results to inspect what was available to search.

The article on Retrieval-augmented generation explains how retrieval, source checking and possible later generation should remain distinct steps.

From NyxAI

smartlytics is the learning hub of NyxAI GmbH.

Explore the methods we work with, explained openly, with sources and calculations you can reproduce.

Decision Core

NyxAI is developing a Decision Core for Decision Intelligence that combines optimisation, simulation, statistics and reinforcement learning: compare options, make uncertainty visible and learn from outcomes, with people in control.

AI workflows at NyxAI

DeepSquare

Explainable chess AI with human playing styles: NyxAI’s testing ground for understandable decisions under time pressure.