{
    "format": "skillpro/v1",
    "skill_id": "chatboxai-chatbox-scripts-session-rag-eval-skill-md",
    "name": "session-rag-eval",
    "version": "1.0.0",
    "description": "Run and debug Chatbox session attachment RAG model evaluation with synthetic and real long-file fixtures.",
    "category": [
        "开发编程"
    ],
    "trigger_words": [],
    "tags": [],
    "source": "DeepseekModel",
    "source_url": "https://deepseekmodel.com/skill?id=chatboxai-chatbox-scripts-session-rag-eval-skill-md",
    "exported_at": "2026-09-16T06:36:43+08:00",
    "system_prompt": "name session-rag-eval description Run and debug Chatbox session attachment RAG model evaluation with synthetic and real long-file fixtures. Session Attachment RAG Eval Use this skill when validating file question-answering behavior for large chat attachments. Goal Verify both sides of model behavior: The model calls query_session_attachment when the answer depends on an uploaded large file. The model avoids retrieval for clearly unrelated user requests. Prefer the Chatbox conversation-flow harness for product validation because it exercises the real renderer, config, license, local API, file upload, indexing, tool registration, and persisted messages. Fixtures Default fixture repo: ../../chatbox-session-rag-eval-fixtures Fixture types: synthetic long documents with hidden needle facts; Wikipedia-derived real long documents with source attribution; direct positive cases; implicit positive cases where the user does not explicitly ask to search the file; unrelated negative cases. Regenerate fixtures in the fixture repo: node scripts/generate-fixtures.mjs node scripts/fetch-real-fixtures.mjs Chatbox Flow Start the local API before running the harness. Then build with USE_LOCAL_API=true ; the renderer API origin is compiled into the bundle. USE_LOCAL_API= true node ./node_modules/electron-vite/bin/electron-vite.js build --mode development pnpm eval :session-rag:chatbox -- -- case long-citrine-threshold --keep-user-data The harness copies the real config.json into an isolated temporary userDataDir, injects a temporary default chat model if missing, and stores the session RAG sqlite DB at a separate temporary path. Known Failure Modes \"This attachment is too large...\" means session_attachment_embedding capability is false or unavailable for the active local API/license path. If requests go to production, rebuild with USE_LOCAL_API=true . If Electron hangs at startup, check stale CDP/Electron processes. The harness now allocates a free CDP port. If the assistant says the file is still indexing after DB says ready, inspect the persisted user message. The message file must have sessionAttachmentIndexStatus: \"ready\" before context building. Do not trust a log line that says the message was submitted unless the textarea cleared and the session contains the user message. Useful Cases pnpm eval :session-rag:chatbox -- -- case long-citrine-threshold pnpm eval :session-rag:chatbox -- -- case implicit-citrine-current-policy pnpm eval :session-rag:chatbox -- -- case real-wiki-apollo-implicit-landing-site pnpm eval :session-rag:chatbox -- -- case multi-turn-real-wiki-apollo-followup pnpm eval :session-rag:chatbox -- -- case unrelated-simple-math pnpm eval :session-rag:chatbox -- -- case real-wiki-unrelated-capital For fast model-behavior iteration without Electron: pnpm eval :session-rag -- --dry-run pnpm eval :session-rag -- -- case implicit-citrine-current-policy",
    "model_config": {
        "provider": "deepseek",
        "model": "deepseek-chat",
        "temperature": 0.7,
        "max_tokens": 4096,
        "top_p": 0.9
    },
    "examples": [
        {
            "input": "请用session-rag-eval帮我处理问题",
            "output": "好的，我是session-rag-eval。Run and debug Chatbox session attachment RAG model evaluation with synthetic and real long-file fixtures. 我会根据你的需求提供专业帮助。"
        },
        {
            "input": "介绍一下你的能力",
            "output": "我是session-rag-eval，专注于开发编程领域。Run and debug Chatbox session attachment RAG model evaluation with synthetic and real long-file fixtures."
        }
    ],
    "install_guide": {
        "coze": "在 Coze 平台创建 Bot -> 技能配置 -> 导入此 .skill 文件",
        "dify": "在 Dify 平台创建应用 -> 添加知识库 -> 导入此 .skill 配置",
        "claude": "将 system_prompt 字段内容复制到 Claude 自定义指令中",
        "custom": "将此 .skill 文件加载到你的 AI Agent 框架中，解析 system_prompt 和 model_config 即可使用"
    },
    "scripts": {
        "python": "# session-rag-eval - Python extension\n# Add custom Python logic here\ndef process(input_data):\n    return input_data\n",
        "javascript": "// session-rag-eval - JavaScript extension\n// Add custom JS logic here\nfunction process(inputData) {\n    return inputData;\n}\n"
    },
    "tools": {
        "mcp_servers": [],
        "api_endpoints": []
    },
    "dependencies": {
        "python": [],
        "node": []
    },
    "hooks": {
        "on_load": "echo \"Skill loaded: session-rag-eval\"",
        "on_call": "",
        "on_error": "echo \"Skill error: please check logs\""
    }
}