NTokenizers.Extensions.Spectre.Console

The NTokenizers.Extensions.Spectre.Console library provides advanced syntax highlighting using Spectre.Console for XML, JSON, Markup, TypeScript, JavaScript, C#, and SQL.

Example

This library builds on:

Spectre.Console for advanced console rendering
NTokenizers for modular, stream‑based tokenization

Together, they enable expressive syntax highlighting directly in the console.

Example Usage

await AnsiConsole.Console.WriteMarkupTextAsync(stream);

Especially suitable for parsing AI chat streams, NTokenizers excels at processing real-time tokenized data from AI models, enabling efficient handling of streaming responses and chat conversations without buffering entire responses.

Check out AI Example for an HowTo.

Token visualization demo

Link: Youtube

Ai demo

Link: Youtube

String output

var result = await AnsiConsole.Console.WriteMarkupTextAsync(stream);

In addition to writing to the console, the original input is returned for convenience.