- Graphwise Platform Documentation
- GraphRAG
- GraphRAG Configuration Guide
- LLM Configuration
LLM Configuration
06/07/2026
GraphRAG version 1.2.0
All operational Large Language Model (LLM) processing steps across the platform - except for the specialized graph execution agent (See GraphDB Integration (MCP) section of this guide) - route variables through a unified LLM Call sub-workflow . This centralized workflow dynamically manages regional infrastructure fallbacks, handles prompt context token distribution, and enforces automated exception-capturing rules.
The orchestration pipeline distinguishes between global routing variables modified via the runtime configuration layer and internal node-level hyperparameters:
primaryLLMModelDefines the primary model engine used for non-MCP processing phases. Default: OpenAI.
secondaryLLMModelDefines the alternative fallback model endpoint, which activates automatically if your primary platform hosting region experiences an outage. For optimal architectural resilience, it is highly recommended to bind this to an isolated, alternative cloud region or distinct model provider framework.
systemMessageImplements targeted behavioral guidelines specific to each individual LLM checkpoint. This property is modified directly within the workflow interface by navigating to Options → Add Option → System Message.
Customizing this baseline configuration away from generic assistant defaults is highly recommended to achieve tight domain accuracy.
structuredOutputParserJsonSchemaInstructs the model execution thread to structure raw generated payloads according to a strictly defined JSON schema validation model.
maxNumberTokensCaps the maximum volume of completion tokens that a generated output response window is permitted to consume.
samplingTemperatureControls completion boundary randomness and linguistic variance. This setting is strictly standardized to 0 across all integrated pipeline workflows to ensure deterministic, reproducible enterprise text completions.
Note
Configuration Guardrail
To maintain cluster execution stability, only modify the primaryLLMModel and secondaryLLMModel fields within your centralized Configuration workflow script block. Do not modify internal node operational parameters or hyperparameters unless explicitly instructed by your system administrator.
LLM orchestration routines run sequentially at five critical validation checkpoints across the execution lifecycle:
Input Guardrails
Evaluates and filters incoming end-user queries to validate prompt safety, identify malicious injection payloads, and classify request intent prior to pipeline processing.
Short-Term Memory Compression
Triggers automated contextual compression and historical summarization workflows immediately when an active session's cumulative token footprint crosses the declared
shortMemoryMaxUncompressedSizeInTokensboundary. This configuration can be found in the Configuration Workflow in the Orchestrator engine.GraphDB MCP Agent
Operates independently to construct syntactically valid structured SPARQL graph query operations and translate complex semantic triple outputs into legible runtime contexts.
Final Generation
Aggregates and structures all retrieved hybrid context data packages, compiles the final grounded response payload for delivery to the client interface, and dynamically generates relevant follow-up exploration items.
Output Guardrails
Performs post-generation quality, security, compliance, and factual hallucination mitigation reviews before exposing the final text stream to the end user.
The primary and secondary LLM models (also called Main and Fallback) are used to guarantee failsafe if the primary one fails. Currently they are used in “LLM call” and “Step - GraphDB MCP - Async workflows”.
In both “LLM call” and “Step - GraphDB MCP - Async workflows” switch the connections from the AI agent to the Main and Fallback models. This achieves the desired results. The credentials used in the two models should also be switched to avoid confusing names - the Main model will use credentials named "Secondary LLM model" and the Fallback model will use credentials named “Primary LLM model”. So in the Credentials tab the names of the two credentials should also be accordingly switched.
The Structured Output Parser is used for serializing the LLM output into a desired json format using Json schema. This node is optional but recommended to ensure the LLM output is delivered in the requested format.
No changes are required when switching the primary and secondary models. Ensure to connect the Structured Output Parser to the AI agent node and either the primary or secondary LLM model node.
Note
Some models (usually older/deprecated) may fail to serialize the LLM output, in this case an upgrade to a newer model is highly recommended.