Download
RU EN DE FR ES PT 中文

AI Chat — User Guide | YoBench

How to use the AI Chat module in YoBench: OpenAI/Claude/YandexGPT/local LLM providers, streaming, system prompt, RAG, tool use.

What the AI Chat module does

The module is a single place to talk to AI right inside YoBench. Connect OpenAI, Anthropic Claude, YandexGPT or any OpenAI-compatible local LLM (LM Studio, Ollama, vLLM, llama.cpp), run multiple parallel conversations, attach RAG contexts to them and let the model use a production toolkit — browser, terminal, file system, web search, and Word / Excel / PowerPoint / PDF generation — under your control with explicit permissions.

What you get:

  • Agent-based AI Core — the assistant runs as an agent with explicit permission dialogs, a unified execution timeline, and a structured chat logger that supports filtering and export.
  • Production tool suite — generate and edit Word, Excel, PowerPoint and PDF documents, extract images, drive a hardened cross-platform terminal and file system (path classification, symlink resolution, SSRF protection).
  • Pluggable web search — Brave, DuckDuckGo or Tavily as the search backend, switchable per workspace.
  • Unified voice experience — one microphone button plus two toggles. Streaming TTS by sentences, automatic language detection (en/ru/de/fr/pt/es), automatic mic restart for hands-free dialogue.
  • Cancellation as a first-class outcome — Cancelled by user is shown separately from errors in the timeline and logs, with proper cleanup of long-running tools.
  • Vision support — image-bearing tool results (screenshots, extracted images) are visible to the assistant on the next turn.
  • Multiple parallel conversations — each with its own provider, model, system prompt, RAG context.
  • Streaming responses — tokens arrive in chunks, not on completion.
  • RAG contexts — answers from your documents via hybrid search (vector + BM25).
  • Logs — every chat event (LLM requests, streaming, tool execution) is recorded separately with API-key masking.
  • Context compaction — old history is auto-summarized into context_summary so token limits aren't hit.

Supported providers

The codebase implements:

  • OpenAI — official API via OpenAIAdapter.
  • Anthropic Claude — via ClaudeAdapter.
  • YandexGPT — via YandexGPTAdapter.
  • OpenAI-compatible local LLMs — LM Studio, Ollama, vLLM, llama.cpp; routed through OpenAIAdapter.

Provider registry parameters:

  • API key — the key.
  • API URL — base URL (optional, falls back to provider default).
  • Model — model identifier.
  • Temperature — creativity (0–2, default 0.7).
  • Max tokens — output token cap (default 4096).
  • Frequency penalty — repetition penalty (0–2, default 0). For local LLMs 0.3–0.5 is recommended.
  • Presence penalty — new-topic penalty (0–2, default 0).
  • Proxy — optional, for outgoing requests.

Conversation parameters

When creating a conversation (chat_conversations) you set:

  • Title — display name; can be auto-generated from the first message.
  • Provider — which AI provider to use.
  • System prompt — instructions for the model.
  • Temperature — overrides the global value.
  • Tools enabled — toggle tool use.
  • Max auto-iterations — how many tool→result iterations are allowed (default 3).
  • Web search provider — separate provider for web search (when used).
  • Context — attached RAG context from the Context Manager.

Messages live in chat_messages (role: user / assistant / system, content).

Tool use

When Tools enabled is on, the model runs as an agent and can call tools registered in ToolRegistry. The codebase ships:

  • browser — actions navigate, click, fill, type, key, screenshot, wait_for_selector, wait_for_navigation, wait_for_url_change, evaluate, get_content, close. Backed by headless Chromium.
  • terminal — shell command execution on a hardened cross-platform shell.
  • file system — read / write / list with path classification (project / system / user) and symlink resolution.
  • web — web operations through pluggable search adapters (Brave, DuckDuckGo, Tavily) plus fetch.
  • documents — generate and edit Word (DOCX), Excel (XLSX), PowerPoint (PPTX) and PDF; extract images from documents.

Flow: the model picks tool__action → the ExecutionEngine runs it → the result is fed back into the context → another iteration may happen (up to max_auto_iterations).

Permissions

Sensitive actions trigger an explicit permission dialog before they run:

  • File-system writes outside the project scope.
  • Terminal commands that look destructive.
  • Outbound network calls to non-allow-listed domains.

You can Allow once, Allow for this conversation, or Deny. Denials surface as a clean cancelled outcome — the agent does not retry and the timeline records the decision.

Cancellation

Cancellation is a first-class outcome rather than an error:

  • Stop the run at any time → the scope key is cancelled, long-running tools (browser, terminal) are torn down.
  • The timeline shows cancelled distinct from error, with a reason string ("Cancelled by user", "Permission denied", etc.).
  • Logs preserve the partial state so you can pick up where you left off.

Vision and image-bearing results

Tools that return images (browser__screenshot, documents__extract_images) emit image-bearing tool results. On the next turn the assistant sees those images alongside the text, so vision-capable models can reason about screenshots and extracted figures without you re-attaching them.

Voice

The voice experience is unified — one microphone button plus two toggles for speak replies aloud and auto-dialog mode:

  • Streaming TTS by sentences — the model's response starts speaking as soon as the first sentence is ready, instead of waiting for completion.
  • Automatic language detection — en / ru / de / fr / pt / es. The voice picks up your language without manual selection.
  • Automatic mic restart — in auto-dialog mode the mic re-arms after the assistant finishes speaking, so you can keep the conversation going hands-free.
  • Offline by default — speech-to-text via Whisper, text-to-speech via Piper. No external services unless you wire one up explicitly.

RAG context

In conversation settings you can pick a context from the Context Manager. On the first message the module retrieves relevant chunks and injects them into the system prompt. Later turns skip retrieval — the context is already there.

Each session can also switch the retrieval mode:

  • Fast (default) — hybrid search (vector + keyword) returns results in a fraction of a second.
  • Accurate — adds a reranker step after the hybrid search. Slower, noticeably better on long or ambiguous queries.

Global settings

Under Settings → AI Chat:

  • chatDefaultProviderId — default provider for new conversations.
  • chatDefaultContextId — default RAG context.
  • chatDefaultSystemPrompt — default system prompt.
  • chatDefaultTemperature — default temperature (0.7).
  • chatDefaultToolsEnabled — turn tool use on by default (default on).
  • chatDefaultMaxAutoIterations — auto-iteration cap (default 3).
  • chatDefaultWebSearchProviderId — web search provider.
  • chatDefaultMaxTokens — output token cap (default 4096).
  • chatDefaultFrequencyPenalty / chatDefaultPresencePenalty — penalties (default 0).

Workflow

1. Wire up a provider

In Settings → Providers add a provider: type, API key, model, optional base URL and proxy.

2. Start a conversation

  1. Open AI Chat from the left sidebar.
  2. Click New conversation.
  3. Pick a provider (or use the default).
  4. Optionally set system prompt, temperature, RAG context.

3. Send messages

Type a request → response streams in. Toggle Tools enabled per conversation if needed.

4. Attach images and contexts

  • Images — drag-and-drop into the composer; base64 goes into the request (provider must support vision).
  • RAG context — picked in conversation settings.

5. Control tool use

Set Max auto-iterations in conversation settings. The model may call browser__navigate → terminal__run and so on. Inspect details in Logs.

6. Inspect logs

The Logs tab shows the event stream: LLM requests, streaming, tool execute, errors. Filter by level, category. Secrets (API keys, Bearer tokens) are masked automatically.

7. Manage conversations

  • Rename — auto-naming on the first message, manually editable.
  • Delete — removes the conversation and its messages.
  • Stop generation — chat:stop-generation aborts the current stream.

Next steps

  • Wire up Contexts & RAG for grounded answers.
  • Use the Web Parser to prepare data you then ask AI about.
  • For document writing/editing, the built-in AI assistant in Docs is faster.

Help and feedback

Want Gemini, Mistral support, or more tools? Contact us via the feedback form.