使用 Label Studio ReactCode 构建 Agentic Tracing 理赔评估标注界面：从 Trace JSON 到可导出的标注结果

使用 Label Studio ReactCode 构建 Agentic Tracing 理赔评估标注界面从 Trace JSON 到可导出的标注结果【免费下载链接】label-studioLabel Studio is a multi-type data labeling and annotation tool with standardized output format项目地址: https://gitcode.com/GitHub_Trending/la/label-studio本文档以 Label Studio 官方模板Agentic Tracing for Claims位于 docs/source/templates/react_claims.md为核心完整讲解如何用ReactCode可编程标签把 AI Agent 执行轨迹多智能体协作、工具调用、多步推理的层级 JSON 数据转换成交互式树形评估界面并在每个步骤上完成多维标注。读完你将掌握模板的六大核心能力、底层数据转换与标签本体论设计、SQL/图表富渲染原理、Claude 追溯对话集成以及从示例输入到标注导出的完整闭环。模板概览为什么需要为 Agent 执行轨迹做标注随着 Agent 工作流在生产环境落地评估其执行质量成为刚需。但 Agent 的中间过程是高度结构化的层级数据——一个根 Agent 会派生子 Agent子 Agent 会调用工具工具返回 SQL 查询结果、图表数据其间穿插着思考Thought与最终输出。传统标注界面无法直观展示这种嵌套结构。该模板创建了一套完整的评估标注界面将 trace JSON含嵌套children数组自动转换为步骤化格式渲染为可折叠的树形结构展示以下节点类型Agent steps—— 多智能体工作流中的单个智能体节点Tool calls—— 携带输入与输出的函数调用SQL responses—— 以格式化表格展示的数据库查询结果Thoughts and reasoning—— 内部推理步骤Outputs—— 最终结果与回复标注者随后可用自定义本体论single-select 下拉式单选维度与 textarea 自由文本对每一步骤的正确性、质量等维度打分所有标注以 Label Studio 标准 region 形式保存可复用平台的审核、导出与分析能力。前置条件ReactCode 标签与 Enterprise 限制!!! error Enterprise 该模板与ReactCode标签仅在Label Studio Enterprise中可用。ReactCode允许在 Label Studio 内嵌入自定义标注 UI同时仍将输出保存为标准 region/result详见 ReactCode 标签参考文档。它最关键的用法是自引用标签不使用控制标签时必须让toName指向nameReactCode namecustom toNamecustom data$treedata$tree意味着组件接收任务数据中的tree字段即整棵 trace 树对象。编写 React 代码时有几点硬性约束CDATA 包裹代码内含、、等字符时必须包在![CDATA[与]]之间避免 XML 解析错误无 JSX必须用React.createElement()不能写div函数式组件代码必须是接收 props 并返回 React 元素的函数props组件收到React含 hooks、addRegion、regions、data、viewState。标注数据的持久化依赖 Regions APIaddRegion(value)创建新 regionvalue 必须是可 JSON 序列化的载荷regions数组中的每个 region 提供value、id、update(value)、delete()。本模板正是围绕addRegion与region.update实现每步骤一条标注的读写闭环。配置详解核心 XML 与 Trace 数据转换模板的标注配置整体结构如下完整 3300 余行代码内嵌于ReactCode的 CDATA 中此处展示骨架View Style .lsf-main-view__annotation { padding: 0 !important; } .lsf-topbar { display: none; } /Style ReactCode styleheight: calc(100vh - 55px) namecustom toNamecustom data$tree ![CDATA[ function AgenticLabelingInterface({ React, addRegion, regions, data }) { // ... } ]] /ReactCode /ViewStyle中隐藏了 Label Studio 默认标注顶栏并去掉了 annotation 容器内边距让自定义界面占据全屏。核心的AgenticLabelingInterface组件内部第一件大事是数据格式转换。从 trace 树到步骤列表transformTraceToSteps组件通过data$tree直接拿到树对象用transformTraceToSteps(treeData)将其转换为界面可渲染的步骤结构。转换前先校验格式if (!treeData || !treeData.children) { return null; // 不是树格式则放弃走 fallback }随后从treeData.metadata中提取用户查询依次尝试claim_text、query、user_query都缺失则用No query provided并以metadata.timestamp为基准、按节点索引每秒递增生成步骤时间戳。节点类型映射是整个转换的关键mapNodeType把 trace 源节点type归一化为界面步骤类型源节点 type界面步骤类型语义agent/rootagent智能体节点text_output/outputoutput文本/最终输出tool_call/tooltool_call工具调用thought/thinkingthought推理步骤subagent/sub_agentsubagent子智能体user_messageuser_message用户消息assistant_responseassistant_response助手回复其他output按内容兜底transformNode递归处理每个节点生成层级 id子节点 id 为父id-索引如main-1、从node.name/author/label提取标签、从node.output/content/text提取正文。工具调用节点会做字段适配——trace 格式用name/args/response界面期望tool/input/outputif (stepType tool_call) { step.tool node.name || node.tool || unknown; step.input node.args || node.input || null; step.output node.response || node.output || node.result || null; step.response node.response || null; // 保留原始响应结构供富渲染 }若树有根节点treeData.name treeData.type root会额外包一个 id 为main的根步骤其正文为Processing pipeline for: 查询前 100 字符。已保存标注的恢复初始化时组件遍历regions从region.value.custominterface.step兼容region.value.step与region.step恢复历史对话步骤并按 id 末尾数字排序后拼接到转换结果之后同时从region.value.custominterface.labels恢复每个步骤已选的标签值实现下次打开继续标注。标签本体论面向步骤类型的多维标注模板的标注系统围绕LABELING_ONTOLOGY对象组织——每种步骤类型配一套维度dimensions每个维度要么是single-select单选分类要么是textarea自由文本。完整本体论如下步骤类型维度single-select 均为单选另有 notes 文本域_default兜底Correctness、NotesagentCorrectness、Delegation QualityAppropriate / Over-delegated / Under-delegated、NotesthoughtReasoning Correctness、CoherenceClear / Somewhat Clear / Confused、Completeness、Notesthought:PlanningPlan Correctness、Plan Feasibility、Plan EfficiencyOptimal / Acceptable / Suboptimal、Notestool_callTool Use Correctness、NecessityNecessary / Helpful / Unnecessary、Notestool:sql_queryQuery Correctness、Query Efficiency、SQL SecuritySecure / Potential Risk / Insecure、Notestool:chart_rendererRendering Correctness、Visualization Choice、NotessubagentCorrectness、Delegation Appropriateness、NotesoutputAnswer Correctness、Completeness、Clarity、Helpfulness、Notesuser_messageRelevance、Notesassistant_responseCorrectness、Helpfulness、NotesgetOntologyForStep按优先级解析某步骤应使用哪套本体先查tool:工具名如tool:sql_query再查步骤类型:标签如thought:Planning然后查纯步骤类型最后落到_default。每个步骤的标注状态由第一个单选维度推导getStepStatus检查该维度值含 Incorrect 判定为 incorrect含 Correct 且非 Partially/Incorrect 判定为 correct否则为 partial未标注则为 unlabeled——这一状态驱动了树节点的彩色状态徽标绿/黄/红、筛选器计数与进度条。标注写入通过handleLabelChange完成先在本地labels状态更新再查regions中是否已存在同stepId的 region——存在则existingRegion.update({ stepId, step, labels })不存在则addRegion({ stepId, labels })。这就是标签保存为 Label Studio region 以便导出分析的底层实现。工具输出富渲染SQL 表格、图表与 Pretty/Raw 切换工具调用节点展开后由ToolContent组件渲染这是模板最具特色的部分。它优先读取step.response保留下来的原始响应结构按以下顺序智能渲染SQL 查询响应含sql或sql_query字段时以深色代码块展示 SQL 原文直接表格响应根级含columnsrows时渲染主表命名多表tables对象中的每个表独立渲染标题、表头、行数据支持行内 SQL数据库多表查询结果分别展示图表plot_data或charts对象按chart_type默认 bar渲染横向条形图取前 8 项按百分比/计数归一化宽度汇总卡片summary对象中的数值字段格式化显示1 的小数转百分比、大数加千分位其余键值对排除已渲染字段后嵌套对象渲染为卡片、对象数组推断为表格、简单数组与标量直接展示元数据页脚execution_time_ms、inference_time_ms、model_id、row_count等指标以灰色小字收尾。表格单元格还有智能格式化0~1 的小数显示为百分比≥1000 的数字加千分位布尔值渲染为 ✓/✗null 渲染为 —。若响应没有可结构化渲染的内容则兜底以 JSON 展示。用户可在每步右上角用Pretty / Raw切换器查看渲染效果或原始 JSON。非工具类节点的正文由ContentRenderer处理它内置一个轻量Markdown 检测与渲染器命中标题、代码块、分隔线等强模式之一即判定为 Markdown弱模式加粗、斜体、列表、链接、行内代码、引用、表格需命中至少 2 个。支持 Formatted/Raw 切换并标注 auto-detected markdown。Claude 集成追溯对话与纠偏模板内置了与 Claude 的对话通道让标注者在评估过程中直接向 Agent提问或纠偏。核心配置在CONFIG对象中const CONFIG { enableFollowUpChat: true, enableProgressBar: true, claude: { enabled: true, apiKey: YOUR-API-KEY, // ⚠️ 必须替换为真实 API Key model: claude-sonnet-4-20250514, apiEndpoint: https://api.anthropic.com/v1/messages, apiVersion: 2023-06-01, maxTokens: 4096, } };注意原文档提示你需要在下述代码中填入你自己的 API Keyline 278即上表中的CONFIG.claude.apiKey占位符。发送前serializeStep将每个步骤按类型序列化为带缩进的文本[Agent: xxx]、[Thought]、[Tool Call - xxx]含 Input/Output JSON、[Sub-agent]、[Output]等serializeTrace在最前面拼上[User Query]得到完整 trace 上下文。sendToClaude使用原生fetch调用 Messages API请求体含model、max_tokens、system提示与messages请求头除Content-Type外还带x-api-key、anthropic-version、anthropic-dangerous-direct-browser-access: true浏览器直连所需。响应按content数组过滤type text块拼成文本非 2xx 时抛出从响应中提取的错误信息。请求通过AbortController支持取消。对话流程handleSendMessage非常巧妙将聊天记录也变成可标注的标注数据用户发送消息 → 以user_message类型步骤写入界面并立即addRegion({ stepId, step: userStep, labels: {} })持久化序列化完整 trace 用户消息发给 Claude返回的回复以assistant_response类型步骤追加到列表同样addRegion保存并自动选中支持编辑重发编辑user_message步骤后submitEditedMessage会删除该步骤之后的所有步骤及其 regionregionToDelete.deleteRegion()用新消息重新生成后续对话出错时展示ErrorAlert组件NO_API_KEY / WRONG_API_KEY / HTTP_ERROR / NETWORK_ERROR / ABORTED 等错误分类并提供 Retry重发上一条消息与 Dismiss。搜索、筛选、进度与快捷键界面顶栏提供Search steps... (/ )搜索框与FilterDropdown筛选器All / Unlabeled / Correct / Partially Correct / Incorrect各带计数徽标并有一键 Expand / Collapse 全部按钮。筛选后命中结果以FlatStepItem扁平渲染未筛选时以带树状连线的VirtualizedTreeNode递归渲染层级。侧栏底部有Evaluation Progress进度条已标注/总数百分比与按状态着色的步骤速览列表。全局快捷键如下快捷键功能↑/↓或J/K在步骤间导航自动展开并滚动到目标←/→或H/L折叠 / 展开当前步骤1–9为当前步骤选择第 N 个单选选项N聚焦备注Notes文本域/聚焦搜索框Esc清空搜索 / 关闭快捷键面板E全部展开C全部折叠?切换快捷键面板示例输入一份理赔处理管线的 trace JSON将下面的 JSON 保存为文件并导入到使用了上述配置的项目中即可看到树形界面数据来自原文档 Example input字段结构可直接复用{ data: { tree: { name: Claims Processing Pipeline, type: root, status: completed, metadata: { trace_id: trace_2025-12-18T09-12-06, timestamp: 2025-12-18T09:12:06.972741, claim_text: I found a small scratch on my passenger door this morning in the parking lot. Its about 6 inches long and just surface level - looks like someones shopping cart hit it. The paint is scraped but no dent. Car is a 2020 Honda Civic. }, children: [ { name: Intake Planner, type: agent, author: IntakePlannerAgent, output: Processing path: STANDARD. Estimated claim value: $950. This is a typical single-panel cosmetic damage claim., status: completed, children: [] }, { name: Evidence Collection, type: agent, author: EvidenceCollectionAgent, output: All photos are usable. Quality grades: B, B, A., status: completed, children: [ { id: adk-ed92bcf3-c1fd-4c69-9072-dc2534c7641a, name: image_quality_scorer, type: tool_call, args: { photo_ids: [photo_001, photo_002, photo_003] }, response: { sql: SELECT photo_id, analyze_quality(blob) as sharpness_score, lighting_score, resolution, quality_grade, usable FROM photos WHERE id IN (photo_001, photo_002, photo_003), rows: [ { photo_id: photo_001, resolution: 1920x1080, quality_grade: B, lighting_score: 0.75, sharpness_score: 0.8, usable: true }, { photo_id: photo_002, resolution: 1920x1080, quality_grade: B, lighting_score: 0.8, sharpness_score: 0.85, usable: true }, { photo_id: photo_003, resolution: 1920x1080, quality_grade: A, lighting_score: 0.85, sharpness_score: 0.9, usable: true } ], columns: [photo_id, sharpness_score, lighting_score, resolution, quality_grade, usable], row_count: 3, execution_time_ms: 245 } } ] }, { name: Damage Assessment, type: agent, author: DamageAssessmentAgent, output: Damage detected: Rear Bumper (dent 0.72 severity, paint scrape 0.45 severity), Trunk Lid (minor dent 0.25 severity). Overall severity: 0.65., status: completed, children: [ { id: adk-837c1124-d977-46fc-bbcb-adc541e8b73d, name: vision_damage_model, type: tool_call, args: { photo_ids: [photo_001, photo_002, photo_003], vehicle_type: 2020 Honda Civic }, response: { sql: SELECT panel, damage_type, severity, area_cm2, repair_method FROM damage_detections WHERE photo_id IN (photo_001, photo_002, photo_003), rows: [ { panel: rear_bumper, area_cm2: 450, severity: 0.72, damage_type: dent, repair_method: PDR repaint }, { panel: rear_bumper, area_cm2: 280, severity: 0.45, damage_type: paint_scrape, repair_method: sand and repaint }, { panel: trunk_lid, area_cm2: 120, severity: 0.25, damage_type: minor_dent, repair_method: PDR } ], columns: [panel, damage_type, severity, area_cm2, repair_method], summary: { confidence: 0.94, overall_severity_score: 0.65, total_loss_probability: 0.02 }, model_id: DamageNet-v3.2, row_count: 3, inference_time_ms: 1823 } } ] }, { name: Policy Coverage, type: agent, author: PolicyCoverageAgent, output: Policy POL-2024-78432: Comprehensive coverage enabled, limit $50,000, deductible $400., status: completed, children: [ { id: adk-944bb79c-3aa4-405c-a7c0-9f3121294f2d, name: policy_database, type: tool_call, args: { policy_id: POL-2024-78432 }, response: { sql: SELECT * FROM policies p JOIN coverage_details c ON p.id c.policy_id WHERE p.policy_number POL-2024-78432, tables: { policy: { rows: [ { policy_id: POL-2024-78432, holder_name: Laura Chen, policy_type: Comprehensive Auto, status: Active, loyalty_tier: Gold } ], columns: [policy_id, holder_name, policy_type, status, loyalty_tier], row_count: 1 }, coverage: { rows: [ { coverage_type: comprehensive, enabled: true, limit_amount: 50000, deductible: 400 }, { coverage_type: collision, enabled: true, limit_amount: 50000, deductible: 600 } ], columns: [coverage_type, enabled, limit_amount, deductible], row_count: 2 } }, execution_time_ms: 45 } } ] } ] } } }这份数据演示了模板覆盖的全部富渲染形态无子节点的普通 AgentIntake Planner、带工具调用的 AgentEvidence Collection →image_quality_scorer单表 SQL、带模型推理摘要的工具Damage Assessment →vision_damage_model含summary与model_id、以及多表联查Policy Coverage →policy_databasetables下含 policy 与 coverage 两张表。示例输出标注结果的导出格式导入上述任务并完成标注后导出的注解annotation遵循 Label Studio 标准格式。每个标注步骤对应一条type: reactcode的 result用户数据保存在value.reactcode中含stepId与labels[ { id: 243533361, annotations: [ { id: 83543096, completed_by: { id: 11551, email: heidihumansignal.com, first_name: Heidi, last_name: Opossum }, result: [ { id: 28yskHp2Ew, type: reactcode, value: { reactcode: { labels: { delegation: Appropriate }, stepId: main } }, origin: manual, to_name: custom, from_name: custom }, { id: PLUJjgUwJw, type: reactcode, value: { reactcode: { labels: { delegation: Appropriate }, stepId: main-0 } }, origin: manual, to_name: custom, from_name: custom }, { id: lJ8UN3Qsfp, type: reactcode, value: { reactcode: { labels: { correctness: Correct }, stepId: main-1 } }, origin: manual, to_name: custom, from_name: custom }, { id: z51Oi2o82C, type: reactcode, value: { reactcode: { labels: { notes: Looks complete to me, but should reference all images, delegation: Appropriate }, stepId: main-3 } }, origin: manual, to_name: custom, from_name: custom }, { id: gAuCT2miRs, type: reactcode, value: { reactcode: { labels: { correctness: Correct }, stepId: main-5 } }, origin: manual, to_name: custom, from_name: custom }, { id: pRiJwDAjt4, type: reactcode, value: { reactcode: { labels: { delegation: Appropriate, correctness: Correct }, stepId: main-5 } }, origin: manual, to_name: custom, from_name: custom } ], reviews: [], was_cancelled: false, ground_truth: false, created_at: 2026-01-13T18:31:35.384154Z, updated_at: 2026-01-13T18:31:35.384165Z, draft_created_at: 2026-01-13T18:30:57.550499Z, lead_time: 50.092, prediction: {}, result_count: 6, unique_id: 64a34a36-5a2c-476e-9623-4718e8ac42fe, import_id: null, last_action: submitted, bulk_created: false, task: 243533361, project: 222216, updated_by: 11551, parent_prediction: null, parent_annotation: null, last_created_by: 11551 } ] } ]从上例可以看出stepId与转换时生成的层级 id 一一对应main为根 Agentmain-0/main-1/main-3/main-5为各子步骤labels中即标注者选择的维度值——这份结构化数据可直接用于统计各 Agent/工具步骤的正确率分布或作为后续模型评估的 ground truth。使用步骤与注意事项部署步骤在 Label Studio Enterprise 中新建项目将上文配置详解中的 XML含 CDATA 内完整 React 代码见原模板 docs/source/templates/react_claims.md粘贴到项目的 Labeling Config将CONFIG.claude.apiKey占位符替换为真实 API Key若不需要对话功能可将CONFIG.claude.enabled置为false导入示例输入格式的 JSON 任务data.tree为含children数组的树结构开始标注展开树节点、切换 Pretty/Raw、在右侧维度面板打标、用快捷键加速、必要时通过对话向 Agent 提问。注意事项模板依赖data$tree直接接收树对象因此任务数据必须包含tree字段且根节点至少要有children数组格式不符会走 fallback 空结构并在控制台输出警告转换器对源字段做了多别名兼容name/tool、args/input、response/output/result、output/content/text可适配多数 Agent 框架的 trace 导出格式但节点type需能被mapNodeType识别见类型映射表Claude 集成走浏览器直连请求头含anthropic-dangerous-direct-browser-access: true请自行评估 API Key 暴露风险与跨域策略标注界面通过addRegion/region.update与 Label Studio region 系统双向同步切换任务或刷新后会自动从 regions 恢复已保存的标签与对话步骤本模板属于可编程接口能力可进一步阅读 ReactCode 标签文档 了解outputsschema、src外部应用模式等进阶用法或参考同类的 Spreadsheet、多通道音频等其他可编程模板将这套树形 trace 多维标注 LLM 对话纠偏的模式迁移到客服工单、代码审查、金融审批等更多 Agent 评估场景。【免费下载链接】label-studioLabel Studio is a multi-type data labeling and annotation tool with standardized output format项目地址: https://gitcode.com/GitHub_Trending/la/label-studio创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考