Merge branch 'Mai-with-u:dev' into dev

pull/1398/head^2
Lofu 2025-12-02 16:11:58 +08:00 committed by GitHub
commit 07bc84d21f
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
17 changed files with 881 additions and 818 deletions

View File

@ -46,7 +46,7 @@
## 🔥 更新和安装 ## 🔥 更新和安装
**最新版本: v0.11.5** ([更新日志](changelogs/changelog.md)) **最新版本: v0.11.6** ([更新日志](changelogs/changelog.md))
可前往 [Release](https://github.com/MaiM-with-u/MaiBot/releases/) 页面下载最新版本 可前往 [Release](https://github.com/MaiM-with-u/MaiBot/releases/) 页面下载最新版本
@ -71,16 +71,21 @@
**技术交流群:** **技术交流群:**
[麦麦脑电图](https://qm.qq.com/q/RzmCiRtHEW) | [麦麦脑电图](https://qm.qq.com/q/RzmCiRtHEW) |
[麦麦脑磁图](https://qm.qq.com/q/wlH5eT8OmQ) |
[麦麦大脑磁共振](https://qm.qq.com/q/VQ3XZrWgMs) | [麦麦大脑磁共振](https://qm.qq.com/q/VQ3XZrWgMs) |
[麦麦要当VTB](https://qm.qq.com/q/wGePTl1UyY) [麦麦要当VTB](https://qm.qq.com/q/wGePTl1UyY) |
为了维持技术交流和互帮互助的氛围,请不要在技术交流群讨论过多无关内容~
**聊天吹水群:** **聊天吹水群:**
- [麦麦之闲聊群](https://qm.qq.com/q/JxvHZnxyec) - [麦麦之闲聊群](https://qm.qq.com/q/JxvHZnxyec)
麦麦相关闲聊群
**插件开发/测试版讨论群:** **插件开发/测试版讨论群:**
- [插件开发群](https://qm.qq.com/q/1036092828) - [插件开发群](https://qm.qq.com/q/1036092828)
进阶内容,包括插件开发,测试版使用等等
## 📚 文档 ## 📚 文档
**部分内容可能更新不够及时,请注意版本对应** **部分内容可能更新不够及时,请注意版本对应**

View File

@ -1,17 +1,52 @@
# Changelog # Changelog
## [0.11.5] - 2025-11-26 ## [0.11.6] - 2025-12-2
### 主要功能更改 ### 🌟 重大更新
- 大幅提高记忆检索能力略微提高token消耗
- 重构历史消息概括器,更好的主题记忆
- 日志查看器性能革命性优化
- 支持可视化查看麦麦LPMM知识图谱
- 支持根据不同的模型提供商/模板/URL自动获取模型可以不用手动输入模型了
- 新增Baka引导系统使用React-JoyTour实现很棒的用户引导系统让Baka也能看懂
- 本地聊天室功能你可以直接在WebUI网页和麦麦聊天
- 使用cookie模式替换原有的LocalStorage Token存储可能需要重新手动输入一遍Token
- WebUI本地聊天室支持用户模拟和平台模拟的功能
- WebUI新增黑话管理 & 编辑界面
### 细节功能更改 ### 细节功能更改
- 可选记忆识别中是否启用jargon
- 解耦表情包识别和图片识别 - 解耦表情包识别和图片识别
- 修复部分破损json的解析问题 - 修复部分破损json的解析问题
- 黑话更高的提取效率,增加提取准确性 - 黑话更高的提取效率,增加提取准确性
- 升级jargon更快更精准 - 升级jargon更快更精准
- 新增Lpmm可视化 - 新增Lpmm可视化
### webui细节更新
- 修复侧边栏收起、UI及表格横向滚动等问题优化Toast动画
- 修复适配器配置、插件克隆、表情包注册等相关BUG
- 新增适配器/模型预设模式及模板自动填写URL和类型
- 支持模型任务列表拖拽排序
- 更新重启弹窗和首次引导内容
- 多处界面命名及标题优化,如模型配置相关菜单重命名和描述更新
- 修复聊天配置“提及回复”相关开关命名错误
- 调试配置新增“显示记忆/Planner/LPMM Prompt”选项
- 新增卡片尺寸、排序、字号、行间距等个性化功能
- 聊天ID及群聊选择优化显示可读名称
- 聊天编辑界面精简字段新增后端聊天列表API支持
- 默认行间距减小,显示更紧凑
- 修复页面滚动、表情包排序、发言频率为0等问题
- 新增React异常Traceback界面及模型列表搜索
- 更新WebUI Icon修复适配器docker路径等问题
- 插件配置可视化编辑,表单控件/元数据/布局类型扩展
- 新增插件API与开发文档
- 新增机器人状态卡片和快速操作按钮
- 调整饼图显示、颜色算法,修复部分统计及解析错误
- 新增缓存、WebSocket配置
- 表情包支持上传和缩略图
- 修复首页极端加载、重启后CtrlC失效、主程序配置移动端适配等问题
- 新增表达反思设置和WebUI聊天室“思考中”占位组件
- 细节如移除部分字段或UI控件、优化按钮/弹窗/编辑逻辑等
## [0.11.5] - 2025-11-21 ## [0.11.5] - 2025-11-21
### 🌟 重大更新 ### 🌟 重大更新
- WebUI 现支持手动重启麦麦,曲线救国版“热重载” - WebUI 现支持手动重启麦麦,曲线救国版“热重载”

View File

@ -1,5 +1,6 @@
from datetime import datetime from datetime import datetime
import time import time
import asyncio
from typing import Dict from typing import Dict
from src.chat.utils.chat_message_builder import ( from src.chat.utils.chat_message_builder import (
@ -46,6 +47,8 @@ class FrequencyControl:
self.frequency_model = LLMRequest( self.frequency_model = LLMRequest(
model_set=model_config.model_task_config.utils_small, request_type="frequency.adjust" model_set=model_config.model_task_config.utils_small, request_type="frequency.adjust"
) )
# 频率调整锁,防止并发执行
self._adjust_lock = asyncio.Lock()
def get_talk_frequency_adjust(self) -> float: def get_talk_frequency_adjust(self) -> float:
"""获取发言频率调整值""" """获取发言频率调整值"""
@ -56,68 +59,78 @@ class FrequencyControl:
self.talk_frequency_adjust = max(0.1, min(5.0, value)) self.talk_frequency_adjust = max(0.1, min(5.0, value))
async def trigger_frequency_adjust(self) -> None: async def trigger_frequency_adjust(self) -> None:
msg_list = get_raw_msg_by_timestamp_with_chat( # 使用异步锁防止并发执行
chat_id=self.chat_id, async with self._adjust_lock:
timestamp_start=self.last_frequency_adjust_time, # 在锁内检查,避免并发触发
timestamp_end=time.time(), current_time = time.time()
) previous_adjust_time = self.last_frequency_adjust_time
if time.time() - self.last_frequency_adjust_time < 160 or len(msg_list) <= 20: msg_list = get_raw_msg_by_timestamp_with_chat(
return
else:
new_msg_list = get_raw_msg_by_timestamp_with_chat(
chat_id=self.chat_id, chat_id=self.chat_id,
timestamp_start=self.last_frequency_adjust_time, timestamp_start=previous_adjust_time,
timestamp_end=time.time(), timestamp_end=current_time,
limit=20,
limit_mode="latest",
) )
message_str = build_readable_messages( if current_time - previous_adjust_time < 160 or len(msg_list) <= 20:
new_msg_list, return
replace_bot_name=True,
timestamp_mode="relative",
read_mark=0.0,
show_actions=False,
)
time_block = f"当前时间:{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}"
bot_name = global_config.bot.nickname
bot_nickname = (
f",也有人叫你{','.join(global_config.bot.alias_names)}" if global_config.bot.alias_names else ""
)
name_block = f"你的名字是{bot_name}{bot_nickname},请注意哪些是你自己的发言。"
prompt = await global_prompt_manager.format_prompt( # 立即更新调整时间,防止并发触发
"frequency_adjust_prompt", self.last_frequency_adjust_time = current_time
name_block=name_block,
time_block=time_block,
message_str=message_str,
)
response, (reasoning_content, _, _) = await self.frequency_model.generate_response_async(
prompt,
)
# logger.info(f"频率调整 prompt: {prompt}") try:
# logger.info(f"频率调整 response: {response}") new_msg_list = get_raw_msg_by_timestamp_with_chat(
chat_id=self.chat_id,
timestamp_start=previous_adjust_time,
timestamp_end=current_time,
limit=20,
limit_mode="latest",
)
if global_config.debug.show_prompt: message_str = build_readable_messages(
logger.info(f"频率调整 prompt: {prompt}") new_msg_list,
logger.info(f"频率调整 response: {response}") replace_bot_name=True,
logger.info(f"频率调整 reasoning_content: {reasoning_content}") timestamp_mode="relative",
read_mark=0.0,
show_actions=False,
)
time_block = f"当前时间:{datetime.now().strftime('%Y-%m-%d %H:%M:%S')}"
bot_name = global_config.bot.nickname
bot_nickname = (
f",也有人叫你{','.join(global_config.bot.alias_names)}" if global_config.bot.alias_names else ""
)
name_block = f"你的名字是{bot_name}{bot_nickname},请注意哪些是你自己的发言。"
final_value_by_api = frequency_api.get_current_talk_value(self.chat_id) prompt = await global_prompt_manager.format_prompt(
"frequency_adjust_prompt",
name_block=name_block,
time_block=time_block,
message_str=message_str,
)
response, (reasoning_content, _, _) = await self.frequency_model.generate_response_async(
prompt,
)
# LLM依然输出过多内容时取消本次调整。合法最多4个字但有的模型可能会输出一些markdown换行符等需要长度宽限 # logger.info(f"频率调整 prompt: {prompt}")
if len(response) < 20: # logger.info(f"频率调整 response: {response}")
if "过于频繁" in response:
logger.info(f"频率调整: 过于频繁,调整值到{final_value_by_api}") if global_config.debug.show_prompt:
self.talk_frequency_adjust = max(0.1, min(1.5, self.talk_frequency_adjust * 0.8)) logger.info(f"频率调整 prompt: {prompt}")
elif "过少" in response: logger.info(f"频率调整 response: {response}")
logger.info(f"频率调整: 过少,调整值到{final_value_by_api}") logger.info(f"频率调整 reasoning_content: {reasoning_content}")
self.talk_frequency_adjust = max(0.1, min(1.5, self.talk_frequency_adjust * 1.2))
self.last_frequency_adjust_time = time.time() final_value_by_api = frequency_api.get_current_talk_value(self.chat_id)
else:
logger.info("频率调整response不符合要求取消本次调整") # LLM依然输出过多内容时取消本次调整。合法最多4个字但有的模型可能会输出一些markdown换行符等需要长度宽限
if len(response) < 20:
if "过于频繁" in response:
logger.info(f"频率调整: 过于频繁,调整值到{final_value_by_api}")
self.talk_frequency_adjust = max(0.1, min(1.5, self.talk_frequency_adjust * 0.8))
elif "过少" in response:
logger.info(f"频率调整: 过少,调整值到{final_value_by_api}")
self.talk_frequency_adjust = max(0.1, min(1.5, self.talk_frequency_adjust * 1.2))
except Exception as e:
logger.error(f"频率调整失败: {e}")
# 即使失败也保持时间戳更新,避免频繁重试
class FrequencyControlManager: class FrequencyControlManager:

View File

@ -2,6 +2,7 @@ import time
import json import json
import os import os
import re import re
import asyncio
from typing import List, Optional, Tuple from typing import List, Optional, Tuple
import traceback import traceback
from src.common.logger import get_logger from src.common.logger import get_logger
@ -91,6 +92,9 @@ class ExpressionLearner:
# 维护每个chat的上次学习时间 # 维护每个chat的上次学习时间
self.last_learning_time: float = time.time() self.last_learning_time: float = time.time()
# 学习锁,防止并发执行学习任务
self._learning_lock = asyncio.Lock()
# 学习参数 # 学习参数
_, self.enable_learning, self.learning_intensity = global_config.expression.get_expression_config_for_chat( _, self.enable_learning, self.learning_intensity = global_config.expression.get_expression_config_for_chat(
self.chat_id self.chat_id
@ -139,32 +143,45 @@ class ExpressionLearner:
Returns: Returns:
bool: 是否成功触发学习 bool: 是否成功触发学习
""" """
if not self.should_trigger_learning(): # 使用异步锁防止并发执行
return async with self._learning_lock:
# 在锁内检查,避免并发触发
# 如果锁被持有,其他协程会等待,但等待期间条件可能已变化,所以需要再次检查
if not self.should_trigger_learning():
return
try: # 保存学习开始前的时间戳,用于获取消息范围
logger.info(f"在聊天流 {self.chat_name} 学习表达方式") learning_start_timestamp = time.time()
# 学习语言风格 previous_learning_time = self.last_learning_time
learnt_style = await self.learn_and_store(num=25)
# 立即更新学习时间,防止并发触发
self.last_learning_time = learning_start_timestamp
# 更新学习时间 try:
self.last_learning_time = time.time() logger.info(f"在聊天流 {self.chat_name} 学习表达方式")
# 学习语言风格,传递学习开始前的时间戳
learnt_style = await self.learn_and_store(num=25, timestamp_start=previous_learning_time)
if learnt_style: if learnt_style:
logger.info(f"聊天流 {self.chat_name} 表达学习完成") logger.info(f"聊天流 {self.chat_name} 表达学习完成")
else: else:
logger.warning(f"聊天流 {self.chat_name} 表达学习未获得有效结果") logger.warning(f"聊天流 {self.chat_name} 表达学习未获得有效结果")
except Exception as e: except Exception as e:
logger.error(f"为聊天流 {self.chat_name} 触发学习失败: {e}") logger.error(f"为聊天流 {self.chat_name} 触发学习失败: {e}")
traceback.print_exc() traceback.print_exc()
return # 即使失败也保持时间戳更新,避免频繁重试
return
async def learn_and_store(self, num: int = 10) -> List[Tuple[str, str, str]]: async def learn_and_store(self, num: int = 10, timestamp_start: Optional[float] = None) -> List[Tuple[str, str, str]]:
""" """
学习并存储表达方式 学习并存储表达方式
Args:
num: 学习数量
timestamp_start: 学习开始的时间戳如果为None则使用self.last_learning_time
""" """
learnt_expressions = await self.learn_expression(num) learnt_expressions = await self.learn_expression(num, timestamp_start=timestamp_start)
if learnt_expressions is None: if learnt_expressions is None:
logger.info("没有学习到表达风格") logger.info("没有学习到表达风格")
@ -374,18 +391,22 @@ class ExpressionLearner:
return matched_expressions return matched_expressions
async def learn_expression(self, num: int = 10) -> Optional[List[Tuple[str, str, str, str]]]: async def learn_expression(self, num: int = 10, timestamp_start: Optional[float] = None) -> Optional[List[Tuple[str, str, str, str]]]:
"""从指定聊天流学习表达方式 """从指定聊天流学习表达方式
Args: Args:
num: 学习数量 num: 学习数量
timestamp_start: 学习开始的时间戳如果为None则使用self.last_learning_time
""" """
current_time = time.time() current_time = time.time()
# 使用传入的时间戳如果没有则使用self.last_learning_time
start_timestamp = timestamp_start if timestamp_start is not None else self.last_learning_time
# 获取上次学习之后的消息 # 获取上次学习之后的消息
random_msg = get_raw_msg_by_timestamp_with_chat_inclusive( random_msg = get_raw_msg_by_timestamp_with_chat_inclusive(
chat_id=self.chat_id, chat_id=self.chat_id,
timestamp_start=self.last_learning_time, timestamp_start=start_timestamp,
timestamp_end=current_time, timestamp_end=current_time,
limit=num, limit=num,
) )

View File

@ -249,3 +249,112 @@ async def explain_jargon_in_context(chat_id: str, messages: List[Any], chat_cont
""" """
explainer = JargonExplainer(chat_id) explainer = JargonExplainer(chat_id)
return await explainer.explain_jargon(messages, chat_context) return await explainer.explain_jargon(messages, chat_context)
def match_jargon_from_text(chat_text: str, chat_id: str) -> List[str]:
"""直接在聊天文本中匹配已知的jargon返回出现过的黑话列表
Args:
chat_text: 要匹配的聊天文本
chat_id: 聊天ID
Returns:
List[str]: 匹配到的黑话列表
"""
if not chat_text or not chat_text.strip():
return []
query = Jargon.select().where((Jargon.meaning.is_null(False)) & (Jargon.meaning != ""))
if global_config.jargon.all_global:
query = query.where(Jargon.is_global)
query = query.order_by(Jargon.count.desc())
matched: Dict[str, None] = {}
for jargon in query:
content = (jargon.content or "").strip()
if not content:
continue
if not global_config.jargon.all_global and not jargon.is_global:
chat_id_list = parse_chat_id_list(jargon.chat_id)
if not chat_id_list_contains(chat_id_list, chat_id):
continue
pattern = re.escape(content)
if re.search(r"[\u4e00-\u9fff]", content):
search_pattern = pattern
else:
search_pattern = r"\b" + pattern + r"\b"
if re.search(search_pattern, chat_text, re.IGNORECASE):
matched[content] = None
logger.info(f"匹配到 {len(matched)} 个黑话")
return list(matched.keys())
async def retrieve_concepts_with_jargon(concepts: List[str], chat_id: str) -> str:
"""对概念列表进行jargon检索
Args:
concepts: 概念列表
chat_id: 聊天ID
Returns:
str: 检索结果字符串
"""
if not concepts:
return ""
results = []
exact_matches = [] # 收集所有精确匹配的概念
for concept in concepts:
concept = concept.strip()
if not concept:
continue
# 先尝试精确匹配
jargon_results = search_jargon(keyword=concept, chat_id=chat_id, limit=10, case_sensitive=False, fuzzy=False)
is_fuzzy_match = False
# 如果精确匹配未找到,尝试模糊搜索
if not jargon_results:
jargon_results = search_jargon(keyword=concept, chat_id=chat_id, limit=10, case_sensitive=False, fuzzy=True)
is_fuzzy_match = True
if jargon_results:
# 找到结果
if is_fuzzy_match:
# 模糊匹配
output_parts = [f"未精确匹配到'{concept}'"]
for result in jargon_results:
found_content = result.get("content", "").strip()
meaning = result.get("meaning", "").strip()
if found_content and meaning:
output_parts.append(f"找到 '{found_content}' 的含义为:{meaning}")
results.append("".join(output_parts))
logger.info(f"在jargon库中找到匹配模糊搜索: {concept},找到{len(jargon_results)}条结果")
else:
# 精确匹配
output_parts = []
for result in jargon_results:
meaning = result.get("meaning", "").strip()
if meaning:
output_parts.append(f"'{concept}' 为黑话或者网络简写,含义为:{meaning}")
results.append("".join(output_parts) if len(output_parts) > 1 else output_parts[0])
exact_matches.append(concept) # 收集精确匹配的概念,稍后统一打印
else:
# 未找到,不返回占位信息,只记录日志
logger.info(f"在jargon库中未找到匹配: {concept}")
# 合并所有精确匹配的日志
if exact_matches:
logger.info(f"找到黑话: {', '.join(exact_matches)},共找到{len(exact_matches)}条结果")
if results:
return "【概念检索结果】\n" + "\n".join(results) + "\n"
return ""

View File

@ -182,6 +182,9 @@ class JargonMiner:
self.stream_name = stream_name if stream_name else self.chat_id self.stream_name = stream_name if stream_name else self.chat_id
self.cache_limit = 100 self.cache_limit = 100
self.cache: OrderedDict[str, None] = OrderedDict() self.cache: OrderedDict[str, None] = OrderedDict()
# 黑话提取锁,防止并发执行
self._extraction_lock = asyncio.Lock()
def _add_to_cache(self, content: str) -> None: def _add_to_cache(self, content: str) -> None:
"""将提取到的黑话加入缓存保持LRU语义""" """将提取到的黑话加入缓存保持LRU语义"""
@ -436,261 +439,265 @@ class JargonMiner:
return bool(recent_messages and len(recent_messages) >= self.min_messages_for_learning) return bool(recent_messages and len(recent_messages) >= self.min_messages_for_learning)
async def run_once(self) -> None: async def run_once(self) -> None:
try: # 使用异步锁防止并发执行
if not self.should_trigger(): async with self._extraction_lock:
return
chat_stream = get_chat_manager().get_stream(self.chat_id)
if not chat_stream:
return
# 记录本次提取的时间窗口,避免重复提取
extraction_start_time = self.last_learning_time
extraction_end_time = time.time()
# 拉取学习窗口内的消息
messages = get_raw_msg_by_timestamp_with_chat_inclusive(
chat_id=self.chat_id,
timestamp_start=extraction_start_time,
timestamp_end=extraction_end_time,
limit=20,
)
if not messages:
return
# 按时间排序,确保编号与上下文一致
messages = sorted(messages, key=lambda msg: msg.time or 0)
chat_str, message_id_list = build_readable_messages_with_id(
messages=messages,
replace_bot_name=True,
timestamp_mode="relative",
truncate=False,
show_actions=False,
show_pic=True,
pic_single=True,
)
if not chat_str.strip():
return
msg_id_to_index: Dict[str, int] = {}
for idx, (msg_id, _msg) in enumerate(message_id_list or []):
if not msg_id:
continue
msg_id_to_index[msg_id] = idx
if not msg_id_to_index:
logger.warning("未能生成消息ID映射跳过本次提取")
return
prompt: str = await global_prompt_manager.format_prompt(
"extract_jargon_prompt",
bot_name=global_config.bot.nickname,
chat_str=chat_str,
)
response, _ = await self.llm.generate_response_async(prompt, temperature=0.2)
if not response:
return
if global_config.debug.show_jargon_prompt:
logger.info(f"jargon提取提示词: {prompt}")
logger.info(f"jargon提取结果: {response}")
# 解析为JSON
entries: List[dict] = []
try: try:
resp = response.strip() # 在锁内检查,避免并发触发
parsed = None if not self.should_trigger():
if resp.startswith("[") and resp.endswith("]"):
parsed = json.loads(resp)
else:
repaired = repair_json(resp)
if isinstance(repaired, str):
parsed = json.loads(repaired)
else:
parsed = repaired
if isinstance(parsed, dict):
parsed = [parsed]
if not isinstance(parsed, list):
return return
for item in parsed: chat_stream = get_chat_manager().get_stream(self.chat_id)
if not isinstance(item, dict): if not chat_stream:
continue return
content = str(item.get("content", "")).strip() # 记录本次提取的时间窗口,避免重复提取
msg_id_value = item.get("msg_id") extraction_start_time = self.last_learning_time
extraction_end_time = time.time()
if not content:
continue # 立即更新学习时间,防止并发触发
if contains_bot_self_name(content):
logger.info(f"解析阶段跳过包含机器人昵称/别名的词条: {content}")
continue
msg_id_str = str(msg_id_value or "").strip()
if not msg_id_str:
logger.warning(f"解析jargon失败msg_id缺失content={content}")
continue
msg_index = msg_id_to_index.get(msg_id_str)
if msg_index is None:
logger.warning(f"解析jargon失败msg_id未找到content={content}, msg_id={msg_id_str}")
continue
target_msg = messages[msg_index]
if is_bot_message(target_msg):
logger.info(f"解析阶段跳过引用机器人自身消息的词条: content={content}, msg_id={msg_id_str}")
continue
context_paragraph = build_context_paragraph(messages, msg_index)
if not context_paragraph:
logger.warning(f"解析jargon失败上下文为空content={content}, msg_id={msg_id_str}")
continue
entries.append({"content": content, "raw_content": [context_paragraph]})
cached_entries = self._collect_cached_entries(messages)
if cached_entries:
entries.extend(cached_entries)
except Exception as e:
logger.error(f"解析jargon JSON失败: {e}; 原始: {response}")
return
if not entries:
return
# 去重并合并raw_content按 content 聚合)
merged_entries: OrderedDict[str, Dict[str, List[str]]] = OrderedDict()
for entry in entries:
content_key = entry["content"]
raw_list = entry.get("raw_content", []) or []
if content_key in merged_entries:
merged_entries[content_key]["raw_content"].extend(raw_list)
else:
merged_entries[content_key] = {
"content": content_key,
"raw_content": list(raw_list),
}
uniq_entries = []
for merged_entry in merged_entries.values():
raw_content_list = merged_entry["raw_content"]
if raw_content_list:
merged_entry["raw_content"] = list(dict.fromkeys(raw_content_list))
uniq_entries.append(merged_entry)
saved = 0
updated = 0
for entry in uniq_entries:
content = entry["content"]
raw_content_list = entry["raw_content"] # 已经是列表
try:
# 查询所有content匹配的记录
query = Jargon.select().where(Jargon.content == content)
# 查找匹配的记录
matched_obj = None
for obj in query:
if global_config.jargon.all_global:
# 开启all_global所有content匹配的记录都可以
matched_obj = obj
break
else:
# 关闭all_global需要检查chat_id列表是否包含目标chat_id
chat_id_list = parse_chat_id_list(obj.chat_id)
if chat_id_list_contains(chat_id_list, self.chat_id):
matched_obj = obj
break
if matched_obj:
obj = matched_obj
try:
obj.count = (obj.count or 0) + 1
except Exception:
obj.count = 1
# 合并raw_content列表读取现有列表追加新值去重
existing_raw_content = []
if obj.raw_content:
try:
existing_raw_content = (
json.loads(obj.raw_content) if isinstance(obj.raw_content, str) else obj.raw_content
)
if not isinstance(existing_raw_content, list):
existing_raw_content = [existing_raw_content] if existing_raw_content else []
except (json.JSONDecodeError, TypeError):
existing_raw_content = [obj.raw_content] if obj.raw_content else []
# 合并并去重
merged_list = list(dict.fromkeys(existing_raw_content + raw_content_list))
obj.raw_content = json.dumps(merged_list, ensure_ascii=False)
# 更新chat_id列表增加当前chat_id的计数
chat_id_list = parse_chat_id_list(obj.chat_id)
updated_chat_id_list = update_chat_id_list(chat_id_list, self.chat_id, increment=1)
obj.chat_id = json.dumps(updated_chat_id_list, ensure_ascii=False)
# 开启all_global时确保记录标记为is_global=True
if global_config.jargon.all_global:
obj.is_global = True
# 关闭all_global时保持原有is_global不变不修改
obj.save()
# 检查是否需要推断(达到阈值且超过上次判定值)
if _should_infer_meaning(obj):
# 异步触发推断,不阻塞主流程
# 重新加载对象以确保数据最新
jargon_id = obj.id
asyncio.create_task(self._infer_meaning_by_id(jargon_id))
updated += 1
else:
# 没找到匹配记录,创建新记录
if global_config.jargon.all_global:
# 开启all_global新记录默认为is_global=True
is_global_new = True
else:
# 关闭all_global新记录is_global=False
is_global_new = False
# 使用新格式创建chat_id列表[[chat_id, count]]
chat_id_list = [[self.chat_id, 1]]
chat_id_json = json.dumps(chat_id_list, ensure_ascii=False)
Jargon.create(
content=content,
raw_content=json.dumps(raw_content_list, ensure_ascii=False),
chat_id=chat_id_json,
is_global=is_global_new,
count=1,
)
saved += 1
except Exception as e:
logger.error(f"保存jargon失败: chat_id={self.chat_id}, content={content}, err={e}")
continue
finally:
self._add_to_cache(content)
# 固定输出提取的jargon结果格式化为可读形式只要有提取结果就输出
if uniq_entries:
# 收集所有提取的jargon内容
jargon_list = [entry["content"] for entry in uniq_entries]
jargon_str = ",".join(jargon_list)
# 输出格式化的结果使用logger.info会自动应用jargon模块的颜色
logger.info(f"[{self.stream_name}]疑似黑话: {jargon_str}")
# 更新为本次提取的结束时间,确保不会重复提取相同的消息窗口
self.last_learning_time = extraction_end_time self.last_learning_time = extraction_end_time
if saved or updated: # 拉取学习窗口内的消息
logger.info(f"jargon写入: 新增 {saved} 条,更新 {updated}chat_id={self.chat_id}") messages = get_raw_msg_by_timestamp_with_chat_inclusive(
except Exception as e: chat_id=self.chat_id,
logger.error(f"JargonMiner 运行失败: {e}") timestamp_start=extraction_start_time,
timestamp_end=extraction_end_time,
limit=20,
)
if not messages:
return
# 按时间排序,确保编号与上下文一致
messages = sorted(messages, key=lambda msg: msg.time or 0)
chat_str, message_id_list = build_readable_messages_with_id(
messages=messages,
replace_bot_name=True,
timestamp_mode="relative",
truncate=False,
show_actions=False,
show_pic=True,
pic_single=True,
)
if not chat_str.strip():
return
msg_id_to_index: Dict[str, int] = {}
for idx, (msg_id, _msg) in enumerate(message_id_list or []):
if not msg_id:
continue
msg_id_to_index[msg_id] = idx
if not msg_id_to_index:
logger.warning("未能生成消息ID映射跳过本次提取")
return
prompt: str = await global_prompt_manager.format_prompt(
"extract_jargon_prompt",
bot_name=global_config.bot.nickname,
chat_str=chat_str,
)
response, _ = await self.llm.generate_response_async(prompt, temperature=0.2)
if not response:
return
if global_config.debug.show_jargon_prompt:
logger.info(f"jargon提取提示词: {prompt}")
logger.info(f"jargon提取结果: {response}")
# 解析为JSON
entries: List[dict] = []
try:
resp = response.strip()
parsed = None
if resp.startswith("[") and resp.endswith("]"):
parsed = json.loads(resp)
else:
repaired = repair_json(resp)
if isinstance(repaired, str):
parsed = json.loads(repaired)
else:
parsed = repaired
if isinstance(parsed, dict):
parsed = [parsed]
if not isinstance(parsed, list):
return
for item in parsed:
if not isinstance(item, dict):
continue
content = str(item.get("content", "")).strip()
msg_id_value = item.get("msg_id")
if not content:
continue
if contains_bot_self_name(content):
logger.info(f"解析阶段跳过包含机器人昵称/别名的词条: {content}")
continue
msg_id_str = str(msg_id_value or "").strip()
if not msg_id_str:
logger.warning(f"解析jargon失败msg_id缺失content={content}")
continue
msg_index = msg_id_to_index.get(msg_id_str)
if msg_index is None:
logger.warning(f"解析jargon失败msg_id未找到content={content}, msg_id={msg_id_str}")
continue
target_msg = messages[msg_index]
if is_bot_message(target_msg):
logger.info(f"解析阶段跳过引用机器人自身消息的词条: content={content}, msg_id={msg_id_str}")
continue
context_paragraph = build_context_paragraph(messages, msg_index)
if not context_paragraph:
logger.warning(f"解析jargon失败上下文为空content={content}, msg_id={msg_id_str}")
continue
entries.append({"content": content, "raw_content": [context_paragraph]})
cached_entries = self._collect_cached_entries(messages)
if cached_entries:
entries.extend(cached_entries)
except Exception as e:
logger.error(f"解析jargon JSON失败: {e}; 原始: {response}")
return
if not entries:
return
# 去重并合并raw_content按 content 聚合)
merged_entries: OrderedDict[str, Dict[str, List[str]]] = OrderedDict()
for entry in entries:
content_key = entry["content"]
raw_list = entry.get("raw_content", []) or []
if content_key in merged_entries:
merged_entries[content_key]["raw_content"].extend(raw_list)
else:
merged_entries[content_key] = {
"content": content_key,
"raw_content": list(raw_list),
}
uniq_entries = []
for merged_entry in merged_entries.values():
raw_content_list = merged_entry["raw_content"]
if raw_content_list:
merged_entry["raw_content"] = list(dict.fromkeys(raw_content_list))
uniq_entries.append(merged_entry)
saved = 0
updated = 0
for entry in uniq_entries:
content = entry["content"]
raw_content_list = entry["raw_content"] # 已经是列表
try:
# 查询所有content匹配的记录
query = Jargon.select().where(Jargon.content == content)
# 查找匹配的记录
matched_obj = None
for obj in query:
if global_config.jargon.all_global:
# 开启all_global所有content匹配的记录都可以
matched_obj = obj
break
else:
# 关闭all_global需要检查chat_id列表是否包含目标chat_id
chat_id_list = parse_chat_id_list(obj.chat_id)
if chat_id_list_contains(chat_id_list, self.chat_id):
matched_obj = obj
break
if matched_obj:
obj = matched_obj
try:
obj.count = (obj.count or 0) + 1
except Exception:
obj.count = 1
# 合并raw_content列表读取现有列表追加新值去重
existing_raw_content = []
if obj.raw_content:
try:
existing_raw_content = (
json.loads(obj.raw_content) if isinstance(obj.raw_content, str) else obj.raw_content
)
if not isinstance(existing_raw_content, list):
existing_raw_content = [existing_raw_content] if existing_raw_content else []
except (json.JSONDecodeError, TypeError):
existing_raw_content = [obj.raw_content] if obj.raw_content else []
# 合并并去重
merged_list = list(dict.fromkeys(existing_raw_content + raw_content_list))
obj.raw_content = json.dumps(merged_list, ensure_ascii=False)
# 更新chat_id列表增加当前chat_id的计数
chat_id_list = parse_chat_id_list(obj.chat_id)
updated_chat_id_list = update_chat_id_list(chat_id_list, self.chat_id, increment=1)
obj.chat_id = json.dumps(updated_chat_id_list, ensure_ascii=False)
# 开启all_global时确保记录标记为is_global=True
if global_config.jargon.all_global:
obj.is_global = True
# 关闭all_global时保持原有is_global不变不修改
obj.save()
# 检查是否需要推断(达到阈值且超过上次判定值)
if _should_infer_meaning(obj):
# 异步触发推断,不阻塞主流程
# 重新加载对象以确保数据最新
jargon_id = obj.id
asyncio.create_task(self._infer_meaning_by_id(jargon_id))
updated += 1
else:
# 没找到匹配记录,创建新记录
if global_config.jargon.all_global:
# 开启all_global新记录默认为is_global=True
is_global_new = True
else:
# 关闭all_global新记录is_global=False
is_global_new = False
# 使用新格式创建chat_id列表[[chat_id, count]]
chat_id_list = [[self.chat_id, 1]]
chat_id_json = json.dumps(chat_id_list, ensure_ascii=False)
Jargon.create(
content=content,
raw_content=json.dumps(raw_content_list, ensure_ascii=False),
chat_id=chat_id_json,
is_global=is_global_new,
count=1,
)
saved += 1
except Exception as e:
logger.error(f"保存jargon失败: chat_id={self.chat_id}, content={content}, err={e}")
continue
finally:
self._add_to_cache(content)
# 固定输出提取的jargon结果格式化为可读形式只要有提取结果就输出
if uniq_entries:
# 收集所有提取的jargon内容
jargon_list = [entry["content"] for entry in uniq_entries]
jargon_str = ",".join(jargon_list)
# 输出格式化的结果使用logger.info会自动应用jargon模块的颜色
logger.info(f"[{self.stream_name}]疑似黑话: {jargon_str}")
if saved or updated:
logger.info(f"jargon写入: 新增 {saved} 条,更新 {updated}chat_id={self.chat_id}")
except Exception as e:
logger.error(f"JargonMiner 运行失败: {e}")
# 即使失败也保持时间戳更新,避免频繁重试
class JargonMinerManager: class JargonMinerManager:

View File

@ -1,17 +1,16 @@
import time import time
import json import json
import re
import asyncio import asyncio
from typing import List, Dict, Any, Optional, Tuple, Set from typing import List, Dict, Any, Optional, Tuple, Set
from src.common.logger import get_logger from src.common.logger import get_logger
from src.config.config import global_config, model_config from src.config.config import global_config, model_config
from src.chat.utils.prompt_builder import Prompt, global_prompt_manager from src.chat.utils.prompt_builder import Prompt, global_prompt_manager
from src.plugin_system.apis import llm_api from src.plugin_system.apis import llm_api
from src.common.database.database_model import ThinkingBack, Jargon from src.common.database.database_model import ThinkingBack
from json_repair import repair_json
from src.memory_system.retrieval_tools import get_tool_registry, init_all_tools from src.memory_system.retrieval_tools import get_tool_registry, init_all_tools
from src.memory_system.memory_utils import parse_questions_json
from src.llm_models.payload_content.message import MessageBuilder, RoleType, Message from src.llm_models.payload_content.message import MessageBuilder, RoleType, Message
from src.jargon.jargon_utils import parse_chat_id_list, chat_id_list_contains from src.jargon.jargon_explainer import match_jargon_from_text, retrieve_concepts_with_jargon
logger = get_logger("memory_retrieval") logger = get_logger("memory_retrieval")
@ -101,10 +100,6 @@ def init_memory_retrieval_prompt():
Prompt( Prompt(
"""你的名字是{bot_name}。现在是{time_now} """你的名字是{bot_name}。现在是{time_now}
你正在参与聊天你需要搜集信息来回答问题帮助你参与聊天 你正在参与聊天你需要搜集信息来回答问题帮助你参与聊天
**重要限制**
- 思考要简短直接切入要点
当前需要解答的问题{question} 当前需要解答的问题{question}
已收集的信息 已收集的信息
{collected_info} {collected_info}
@ -113,11 +108,13 @@ def init_memory_retrieval_prompt():
- 如果涉及过往事件或者查询某个过去可能提到过的概念或者某段时间发生的事件可以使用聊天记录查询工具查询过往事件 - 如果涉及过往事件或者查询某个过去可能提到过的概念或者某段时间发生的事件可以使用聊天记录查询工具查询过往事件
- 如果涉及人物可以使用人物信息查询工具查询人物信息 - 如果涉及人物可以使用人物信息查询工具查询人物信息
- 如果没有可靠信息且查询时间充足或者不确定查询类别也可以使用lpmm知识库查询作为辅助信息 - 如果没有可靠信息且查询时间充足或者不确定查询类别也可以使用lpmm知识库查询作为辅助信息
- 如果信息不足需要使用tool说明需要查询什么并输出为纯文本说明然后调用相应工具查询可并行调用多个工具 - **如果信息不足需要使用tool说明需要查询什么并输出为纯文本说明然后调用相应工具查询可并行调用多个工具**
- **如果当前已收集的信息足够回答问题且能找到明确答案调用found_answer工具标记已找到答案**
**思考** **思考**
- 你可以对查询思路给出简短的思考 - 你可以对查询思路给出简短的思考思考要简短直接切入要点
- 你必须给出使用什么工具进行查询 - 如果信息不足你必须给出使用什么工具进行查询
- 如果信息足够你必须调用found_answer工具
""", """,
name="memory_retrieval_react_prompt_head", name="memory_retrieval_react_prompt_head",
) )
@ -149,198 +146,40 @@ def init_memory_retrieval_prompt():
) )
def _parse_react_response(response: str) -> Optional[Dict[str, Any]]:
"""解析ReAct Agent的响应
Args:
response: LLM返回的响应
Returns:
Dict[str, Any]: 解析后的动作信息如果解析失败返回None
格式: {"thought": str, "actions": List[Dict[str, Any]]}
每个action格式: {"action_type": str, "action_params": dict}
"""
try:
# 尝试提取JSON可能包含在```json代码块中
json_pattern = r"```json\s*(.*?)\s*```"
matches = re.findall(json_pattern, response, re.DOTALL)
if matches:
json_str = matches[0]
else:
# 尝试直接解析整个响应
json_str = response.strip()
# 修复可能的JSON错误
repaired_json = repair_json(json_str)
# 解析JSON
action_info = json.loads(repaired_json)
if not isinstance(action_info, dict):
logger.warning(f"解析的JSON不是对象格式: {action_info}")
return None
# 确保actions字段存在且为列表
if "actions" not in action_info:
logger.warning(f"响应中缺少actions字段: {action_info}")
return None
if not isinstance(action_info["actions"], list):
logger.warning(f"actions字段不是数组格式: {action_info['actions']}")
return None
# 确保actions不为空
if len(action_info["actions"]) == 0:
logger.warning("actions数组为空")
return None
return action_info
except Exception as e:
logger.error(f"解析ReAct响应失败: {e}, 响应内容: {response[:200]}...")
return None
async def _retrieve_concepts_with_jargon(concepts: List[str], chat_id: str) -> str: def _log_conversation_messages(
"""对概念列表进行jargon检索 conversation_messages: List[Message],
head_prompt: Optional[str] = None,
Args: final_status: Optional[str] = None,
concepts: 概念列表 ) -> None:
chat_id: 聊天ID
Returns:
str: 检索结果字符串
"""
if not concepts:
return ""
from src.jargon.jargon_miner import search_jargon
results = []
exact_matches = [] # 收集所有精确匹配的概念
for concept in concepts:
concept = concept.strip()
if not concept:
continue
# 先尝试精确匹配
jargon_results = search_jargon(keyword=concept, chat_id=chat_id, limit=10, case_sensitive=False, fuzzy=False)
is_fuzzy_match = False
# 如果精确匹配未找到,尝试模糊搜索
if not jargon_results:
jargon_results = search_jargon(keyword=concept, chat_id=chat_id, limit=10, case_sensitive=False, fuzzy=True)
is_fuzzy_match = True
if jargon_results:
# 找到结果
if is_fuzzy_match:
# 模糊匹配
output_parts = [f"未精确匹配到'{concept}'"]
for result in jargon_results:
found_content = result.get("content", "").strip()
meaning = result.get("meaning", "").strip()
if found_content and meaning:
output_parts.append(f"找到 '{found_content}' 的含义为:{meaning}")
results.append("".join(output_parts))
logger.info(f"在jargon库中找到匹配模糊搜索: {concept},找到{len(jargon_results)}条结果")
else:
# 精确匹配
output_parts = []
for result in jargon_results:
meaning = result.get("meaning", "").strip()
if meaning:
output_parts.append(f"'{concept}' 为黑话或者网络简写,含义为:{meaning}")
results.append("".join(output_parts) if len(output_parts) > 1 else output_parts[0])
exact_matches.append(concept) # 收集精确匹配的概念,稍后统一打印
else:
# 未找到,不返回占位信息,只记录日志
logger.info(f"在jargon库中未找到匹配: {concept}")
# 合并所有精确匹配的日志
if exact_matches:
logger.info(f"找到黑话: {', '.join(exact_matches)},共找到{len(exact_matches)}条结果")
if results:
return "【概念检索结果】\n" + "\n".join(results) + "\n"
return ""
def _match_jargon_from_text(chat_text: str, chat_id: str) -> List[str]:
"""直接在聊天文本中匹配已知的jargon返回出现过的黑话列表"""
# print(chat_text)
if not chat_text or not chat_text.strip():
return []
start_time = time.time()
query = Jargon.select().where((Jargon.meaning.is_null(False)) & (Jargon.meaning != ""))
if global_config.jargon.all_global:
query = query.where(Jargon.is_global)
query = query.order_by(Jargon.count.desc())
query_time = time.time()
matched: Dict[str, None] = {}
for jargon in query:
content = (jargon.content or "").strip()
if not content:
continue
if not global_config.jargon.all_global and not jargon.is_global:
chat_id_list = parse_chat_id_list(jargon.chat_id)
if not chat_id_list_contains(chat_id_list, chat_id):
continue
pattern = re.escape(content)
if re.search(r"[\u4e00-\u9fff]", content):
search_pattern = pattern
else:
search_pattern = r"\b" + pattern + r"\b"
if re.search(search_pattern, chat_text, re.IGNORECASE):
matched[content] = None
# end_time = time.time()
logger.info(
# f"记忆检索黑话匹配: 查询耗时 {(query_time - start_time):.3f}s, "
# f"匹配耗时 {(end_time - query_time):.3f}s, 总耗时 {(end_time - start_time):.3f}s, "
f"匹配到 {len(matched)} 个黑话"
)
return list(matched.keys())
def _log_conversation_messages(conversation_messages: List[Message], head_prompt: Optional[str] = None) -> None:
"""输出对话消息列表的日志 """输出对话消息列表的日志
Args: Args:
conversation_messages: 对话消息列表 conversation_messages: 对话消息列表
head_prompt: 第一条系统消息head_prompt的内容可选 head_prompt: 第一条系统消息head_prompt的内容可选
final_status: 最终结果状态描述例如找到答案/未找到答案可选
""" """
if not global_config.debug.show_memory_prompt: if not global_config.debug.show_memory_prompt:
return return
log_lines = [] log_lines: List[str] = []
# 如果有head_prompt先添加为第一条消息 # 如果有head_prompt先添加为第一条消息
if head_prompt: if head_prompt:
msg_info = "\n[消息 1] 角色: System 内容类型: 文本\n========================================" msg_info = "========================================\n[消息 1] 角色: System 内容类型: 文本\n-----------------------------"
msg_info += f"\n{head_prompt}" msg_info += f"\n{head_prompt}"
log_lines.append(msg_info) log_lines.append(msg_info)
start_idx = 2 start_idx = 2
else: else:
start_idx = 1 start_idx = 1
if not conversation_messages and not head_prompt: if not conversation_messages and not head_prompt:
return return
for idx, msg in enumerate(conversation_messages, start_idx): for idx, msg in enumerate(conversation_messages, start_idx):
role_name = msg.role.value if hasattr(msg.role, "value") else str(msg.role) role_name = msg.role.value if hasattr(msg.role, "value") else str(msg.role)
# 处理内容 - 显示完整内容,不截断 # 处理内容 - 显示完整内容,不截断
if isinstance(msg.content, str): if isinstance(msg.content, str):
full_content = msg.content full_content = msg.content
@ -353,25 +192,28 @@ def _log_conversation_messages(conversation_messages: List[Message], head_prompt
else: else:
full_content = str(msg.content) full_content = str(msg.content)
content_type = "未知" content_type = "未知"
# 构建单条消息的日志信息 # 构建单条消息的日志信息
msg_info = f"\n[消息 {idx}] 角色: {role_name} 内容类型: {content_type}\n========================================" msg_info = f"\n========================================\n[消息 {idx}] 角色: {role_name} 内容类型: {content_type}\n-----------------------------"
if full_content: if full_content:
msg_info += f"\n{full_content}" msg_info += f"\n{full_content}"
if msg.tool_calls: if msg.tool_calls:
msg_info += f"\n 工具调用: {len(msg.tool_calls)}" msg_info += f"\n 工具调用: {len(msg.tool_calls)}"
for tool_call in msg.tool_calls: for tool_call in msg.tool_calls:
msg_info += f"\n - {tool_call}" msg_info += f"\n - {tool_call}"
if msg.tool_call_id: if msg.tool_call_id:
msg_info += f"\n 工具调用ID: {msg.tool_call_id}" msg_info += f"\n 工具调用ID: {msg.tool_call_id}"
log_lines.append(msg_info) log_lines.append(msg_info)
total_count = len(conversation_messages) + (1 if head_prompt else 0) total_count = len(conversation_messages) + (1 if head_prompt else 0)
logger.info(f"消息列表 (共{total_count}条):{''.join(log_lines)}") log_text = f"消息列表 (共{total_count}条):{''.join(log_lines)}"
if final_status:
log_text += f"\n\n[最终结果] {final_status}"
logger.info(log_text)
async def _react_agent_solve_question( async def _react_agent_solve_question(
@ -407,7 +249,7 @@ async def _react_agent_solve_question(
thinking_steps = [] thinking_steps = []
is_timeout = False is_timeout = False
conversation_messages: List[Message] = [] conversation_messages: List[Message] = []
last_head_prompt: Optional[str] = None # 保存最后一次使用的head_prompt first_head_prompt: Optional[str] = None # 保存第一次使用的head_prompt用于日志显示
for iteration in range(max_iterations): for iteration in range(max_iterations):
# 检查超时 # 检查超时
@ -430,40 +272,6 @@ async def _react_agent_solve_question(
remaining_iterations = max_iterations - current_iteration remaining_iterations = max_iterations - current_iteration
is_final_iteration = current_iteration >= max_iterations is_final_iteration = current_iteration >= max_iterations
# 每次迭代开始时,先评估当前信息是否足够回答问题
evaluation_prompt = await global_prompt_manager.format_prompt(
"memory_retrieval_react_final_prompt",
bot_name=bot_name,
time_now=time_now,
question=question,
collected_info=collected_info if collected_info else "暂无信息",
current_iteration=current_iteration,
remaining_iterations=remaining_iterations,
max_iterations=max_iterations,
)
# if global_config.debug.show_memory_prompt:
# logger.info(f"ReAct Agent 第 {iteration + 1} 次迭代 评估Prompt: {evaluation_prompt}")
eval_success, eval_response, eval_reasoning_content, eval_model_name, eval_tool_calls = await llm_api.generate_with_model_with_tools(
evaluation_prompt,
model_config=model_config.model_task_config.tool_use,
tool_options=[], # 评估阶段不提供工具
request_type="memory.react.eval",
)
if not eval_success:
logger.error(f"ReAct Agent 第 {iteration + 1} 次迭代 评估阶段 LLM调用失败: {eval_response}")
# 评估失败,如果还有剩余迭代次数,尝试继续查询
if not is_final_iteration:
continue
else:
break
logger.info(
f"ReAct Agent 第 {iteration + 1} 次迭代 评估响应: {eval_response}"
)
# 提取函数调用中参数的值,支持单引号和双引号 # 提取函数调用中参数的值,支持单引号和双引号
def extract_quoted_content(text, func_name, param_name): def extract_quoted_content(text, func_name, param_name):
"""从文本中提取函数调用中参数的值,支持单引号和双引号 """从文本中提取函数调用中参数的值,支持单引号和双引号
@ -522,88 +330,132 @@ async def _react_agent_solve_question(
return None return None
# 从评估响应中提取found_answer或not_enough_info # 如果是最后一次迭代使用final_prompt进行总结
found_answer_content = None if is_final_iteration:
not_enough_info_reason = None evaluation_prompt = await global_prompt_manager.format_prompt(
"memory_retrieval_react_final_prompt",
bot_name=bot_name,
time_now=time_now,
question=question,
collected_info=collected_info if collected_info else "暂无信息",
current_iteration=current_iteration,
remaining_iterations=remaining_iterations,
max_iterations=max_iterations,
)
if eval_response: if global_config.debug.show_memory_prompt:
found_answer_content = extract_quoted_content(eval_response, "found_answer", "answer") logger.info(f"ReAct Agent 最终评估Prompt: {evaluation_prompt}")
if not found_answer_content:
not_enough_info_reason = extract_quoted_content(eval_response, "not_enough_info", "reason")
# 如果找到答案,直接返回 eval_success, eval_response, eval_reasoning_content, eval_model_name, eval_tool_calls = await llm_api.generate_with_model_with_tools(
if found_answer_content: evaluation_prompt,
eval_step = { model_config=model_config.model_task_config.tool_use,
"iteration": iteration + 1, tool_options=[], # 最终评估阶段不提供工具
"thought": f"[评估] {eval_response}", request_type="memory.react.final",
"actions": [{"action_type": "found_answer", "action_params": {"answer": found_answer_content}}], )
"observations": ["评估阶段检测到found_answer"]
}
thinking_steps.append(eval_step)
logger.info(f"ReAct Agent 第 {iteration + 1} 次迭代 评估阶段找到关于问题{question}的答案: {found_answer_content}")
# React完成时输出消息列表
_log_conversation_messages(conversation_messages, last_head_prompt)
return True, found_answer_content, thinking_steps, False
# 如果评估为not_enough_info且是最终迭代返回not_enough_info if not eval_success:
if not_enough_info_reason: logger.error(f"ReAct Agent 第 {iteration + 1} 次迭代 最终评估阶段 LLM调用失败: {eval_response}")
if is_final_iteration: _log_conversation_messages(
conversation_messages,
head_prompt=first_head_prompt,
final_status="未找到答案最终评估阶段LLM调用失败",
)
return False, "最终评估阶段LLM调用失败", thinking_steps, False
logger.info(
f"ReAct Agent 第 {iteration + 1} 次迭代 最终评估响应: {eval_response}"
)
# 从最终评估响应中提取found_answer或not_enough_info
found_answer_content = None
not_enough_info_reason = None
if eval_response:
found_answer_content = extract_quoted_content(eval_response, "found_answer", "answer")
if not found_answer_content:
not_enough_info_reason = extract_quoted_content(eval_response, "not_enough_info", "reason")
# 如果找到答案,返回
if found_answer_content:
eval_step = { eval_step = {
"iteration": iteration + 1, "iteration": iteration + 1,
"thought": f"[评估] {eval_response}", "thought": f"[最终评估] {eval_response}",
"actions": [{"action_type": "found_answer", "action_params": {"answer": found_answer_content}}],
"observations": ["最终评估阶段检测到found_answer"]
}
thinking_steps.append(eval_step)
logger.info(f"ReAct Agent 第 {iteration + 1} 次迭代 最终评估阶段找到关于问题{question}的答案: {found_answer_content}")
_log_conversation_messages(
conversation_messages,
head_prompt=first_head_prompt,
final_status=f"找到答案:{found_answer_content}",
)
return True, found_answer_content, thinking_steps, False
# 如果评估为not_enough_info返回空字符串不返回任何信息
if not_enough_info_reason:
eval_step = {
"iteration": iteration + 1,
"thought": f"[最终评估] {eval_response}",
"actions": [{"action_type": "not_enough_info", "action_params": {"reason": not_enough_info_reason}}], "actions": [{"action_type": "not_enough_info", "action_params": {"reason": not_enough_info_reason}}],
"observations": ["评估阶段检测到not_enough_info"] "observations": ["最终评估阶段检测到not_enough_info"]
} }
thinking_steps.append(eval_step) thinking_steps.append(eval_step)
logger.info( logger.info(
f"ReAct Agent 第 {iteration + 1} 次迭代 评估阶段判断信息不足: {not_enough_info_reason}" f"ReAct Agent 第 {iteration + 1} 次迭代 最终评估阶段判断信息不足: {not_enough_info_reason}"
) )
# React完成时输出消息列表 _log_conversation_messages(
_log_conversation_messages(conversation_messages, last_head_prompt) conversation_messages,
head_prompt=first_head_prompt,
return False, not_enough_info_reason, thinking_steps, False final_status=f"未找到答案:{not_enough_info_reason}",
else:
# 非最终迭代,信息不足,继续搜集信息
logger.info(
f"ReAct Agent 第 {iteration + 1} 次迭代 评估阶段判断信息不足: {not_enough_info_reason},继续查询"
) )
return False, "", thinking_steps, False
# 如果是最终迭代但没有明确判断视为not_enough_info # 如果没有明确判断视为not_enough_info返回空字符串不返回任何信息
if is_final_iteration:
eval_step = { eval_step = {
"iteration": iteration + 1, "iteration": iteration + 1,
"thought": f"[评估] {eval_response}", "thought": f"[最终评估] {eval_response}",
"actions": [{"action_type": "not_enough_info", "action_params": {"reason": "已到达最后一次迭代,无法找到答案"}}], "actions": [{"action_type": "not_enough_info", "action_params": {"reason": "已到达最后一次迭代,无法找到答案"}}],
"observations": ["已到达最后一次迭代,无法找到答案"] "observations": ["已到达最后一次迭代,无法找到答案"]
} }
thinking_steps.append(eval_step) thinking_steps.append(eval_step)
logger.info(f"ReAct Agent 第 {iteration + 1} 次迭代 已到达最后一次迭代,无法找到答案") logger.info(f"ReAct Agent 第 {iteration + 1} 次迭代 已到达最后一次迭代,无法找到答案")
# React完成时输出消息列表 _log_conversation_messages(
_log_conversation_messages(conversation_messages, last_head_prompt) conversation_messages,
head_prompt=first_head_prompt,
final_status="未找到答案:已到达最后一次迭代,无法找到答案",
)
return False, "已到达最后一次迭代,无法找到答案", thinking_steps, False return False, "", thinking_steps, False
# 非最终迭代且信息不足使用head_prompt决定调用哪些工具 # 前n-1次迭代使用head_prompt决定调用哪些工具包含found_answer工具
tool_definitions = tool_registry.get_tool_definitions() tool_definitions = tool_registry.get_tool_definitions()
logger.info( logger.info(
f"ReAct Agent 第 {iteration + 1} 次迭代,问题: {question}|可用工具数量: {len(tool_definitions)}" f"ReAct Agent 第 {iteration + 1} 次迭代,问题: {question}|可用工具数量: {len(tool_definitions)}"
) )
head_prompt = await global_prompt_manager.format_prompt( # head_prompt应该只构建一次使用初始的collected_info后续迭代都复用同一个
"memory_retrieval_react_prompt_head", if first_head_prompt is None:
bot_name=bot_name, # 第一次构建使用初始的collected_info即initial_info
time_now=time_now, initial_collected_info = initial_info if initial_info else ""
question=question, first_head_prompt = await global_prompt_manager.format_prompt(
collected_info=collected_info if collected_info else "", "memory_retrieval_react_prompt_head",
current_iteration=current_iteration, bot_name=bot_name,
remaining_iterations=remaining_iterations, time_now=time_now,
max_iterations=max_iterations, question=question,
) collected_info=initial_collected_info,
last_head_prompt = head_prompt # 保存最后一次使用的head_prompt current_iteration=current_iteration,
remaining_iterations=remaining_iterations,
max_iterations=max_iterations,
)
# 后续迭代都复用第一次构建的head_prompt
head_prompt = first_head_prompt
def message_factory( def message_factory(
_client, _client,
@ -635,7 +487,7 @@ async def _react_agent_solve_question(
request_type="memory.react", request_type="memory.react",
) )
logger.info( logger.debug(
f"ReAct Agent 第 {iteration + 1} 次迭代 模型: {model_name} ,调用工具数量: {len(tool_calls) if tool_calls else 0} ,调用工具响应: {response}" f"ReAct Agent 第 {iteration + 1} 次迭代 模型: {model_name} ,调用工具数量: {len(tool_calls) if tool_calls else 0} ,调用工具响应: {response}"
) )
@ -643,8 +495,7 @@ async def _react_agent_solve_question(
logger.error(f"ReAct Agent LLM调用失败: {response}") logger.error(f"ReAct Agent LLM调用失败: {response}")
break break
# 注意这里不检查found_answer或not_enough_info这些只在评估阶段memory_retrieval_react_final_prompt检查 # 注意这里会检查found_answer工具调用如果检测到found_answer工具会直接返回答案
# memory_retrieval_react_prompt_head只用于决定调用哪些工具来搜集信息
assistant_message: Optional[Message] = None assistant_message: Optional[Message] = None
if tool_calls: if tool_calls:
@ -686,16 +537,42 @@ async def _react_agent_solve_question(
continue continue
# 处理工具调用 # 处理工具调用
# 首先检查是否有found_answer工具调用如果有则立即返回不再处理其他工具
found_answer_from_tool = None
for tool_call in tool_calls:
tool_name = tool_call.func_name
tool_args = tool_call.args or {}
if tool_name == "found_answer":
found_answer_from_tool = tool_args.get("answer", "")
if found_answer_from_tool:
step["actions"].append({"action_type": "found_answer", "action_params": {"answer": found_answer_from_tool}})
step["observations"] = ["检测到found_answer工具调用"]
thinking_steps.append(step)
logger.debug(f"ReAct Agent 第 {iteration + 1} 次迭代 通过found_answer工具找到关于问题{question}的答案: {found_answer_from_tool}")
_log_conversation_messages(
conversation_messages,
head_prompt=first_head_prompt,
final_status=f"找到答案:{found_answer_from_tool}",
)
return True, found_answer_from_tool, thinking_steps, False
# 如果没有found_answer工具调用或者found_answer工具调用没有答案继续处理其他工具
tool_tasks = [] tool_tasks = []
for i, tool_call in enumerate(tool_calls): for i, tool_call in enumerate(tool_calls):
tool_name = tool_call.func_name tool_name = tool_call.func_name
tool_args = tool_call.args or {} tool_args = tool_call.args or {}
logger.info( logger.debug(
f"ReAct Agent 第 {iteration + 1} 次迭代 工具调用 {i + 1}/{len(tool_calls)}: {tool_name}({tool_args})" f"ReAct Agent 第 {iteration + 1} 次迭代 工具调用 {i + 1}/{len(tool_calls)}: {tool_name}({tool_args})"
) )
# 跳过found_answer工具调用已经在上面处理过了
if tool_name == "found_answer":
continue
# 普通工具调用 # 普通工具调用
tool = tool_registry.get_tool(tool_name) tool = tool_registry.get_tool(tool_name)
if tool: if tool:
@ -740,15 +617,10 @@ async def _react_agent_solve_question(
step["observations"].append(observation_text) step["observations"].append(observation_text)
collected_info += f"\n{observation_text}\n" collected_info += f"\n{observation_text}\n"
if stripped_observation: if stripped_observation:
tool_builder = MessageBuilder() # 检查工具输出中是否有新的jargon如果有则追加到工具结果中
tool_builder.set_role(RoleType.Tool)
tool_builder.add_text_content(observation_text)
tool_builder.add_tool_call(tool_call_item.call_id)
conversation_messages.append(tool_builder.build())
if enable_jargon_detection: if enable_jargon_detection:
jargon_concepts = _match_jargon_from_text(stripped_observation, chat_id) jargon_concepts = match_jargon_from_text(stripped_observation, chat_id)
if jargon_concepts: if jargon_concepts:
jargon_info = ""
new_concepts = [] new_concepts = []
for concept in jargon_concepts: for concept in jargon_concepts:
normalized_concept = concept.strip() normalized_concept = concept.strip()
@ -756,10 +628,18 @@ async def _react_agent_solve_question(
new_concepts.append(normalized_concept) new_concepts.append(normalized_concept)
seen_jargon_concepts.add(normalized_concept) seen_jargon_concepts.add(normalized_concept)
if new_concepts: if new_concepts:
jargon_info = await _retrieve_concepts_with_jargon(new_concepts, chat_id) jargon_info = await retrieve_concepts_with_jargon(new_concepts, chat_id)
if jargon_info: if jargon_info:
collected_info += f"\n{jargon_info}\n" # 将jargon查询结果追加到工具结果中
logger.info(f"工具输出触发黑话解析: {new_concepts}") observation_text += f"\n\n{jargon_info}"
collected_info += f"\n{jargon_info}\n"
logger.info(f"工具输出触发黑话解析: {new_concepts}")
tool_builder = MessageBuilder()
tool_builder.set_role(RoleType.Tool)
tool_builder.add_text_content(observation_text)
tool_builder.add_tool_call(tool_call_item.call_id)
conversation_messages.append(tool_builder.build())
thinking_steps.append(step) thinking_steps.append(step)
@ -776,13 +656,18 @@ async def _react_agent_solve_question(
logger.warning("ReAct Agent达到最大迭代次数直接视为not_enough_info") logger.warning("ReAct Agent达到最大迭代次数直接视为not_enough_info")
# React完成时输出消息列表 # React完成时输出消息列表
_log_conversation_messages(conversation_messages, last_head_prompt) timeout_reason = "超时" if is_timeout else "达到最大迭代次数"
_log_conversation_messages(
conversation_messages,
head_prompt=first_head_prompt,
final_status=f"未找到答案:{timeout_reason}",
)
return False, "未找到相关信息", thinking_steps, is_timeout return False, "", thinking_steps, is_timeout
def _get_recent_query_history(chat_id: str, time_window_seconds: float = 300.0) -> str: def _get_recent_query_history(chat_id: str, time_window_seconds: float = 600.0) -> str:
"""获取最近一段时间内的查询历史 """获取最近一段时间内的查询历史(用于避免重复查询)
Args: Args:
chat_id: 聊天ID chat_id: 聊天ID
@ -832,6 +717,49 @@ def _get_recent_query_history(chat_id: str, time_window_seconds: float = 300.0)
return "" return ""
def _get_recent_found_answers(chat_id: str, time_window_seconds: float = 600.0) -> List[str]:
"""获取最近一段时间内已找到答案的查询记录(用于返回给 replyer
Args:
chat_id: 聊天ID
time_window_seconds: 时间窗口默认10分钟
Returns:
List[str]: 格式化的答案列表每个元素格式为 "问题xxx\n答案xxx"
"""
try:
current_time = time.time()
start_time = current_time - time_window_seconds
# 查询最近时间窗口内已找到答案的记录,按更新时间倒序
records = (
ThinkingBack.select()
.where(
(ThinkingBack.chat_id == chat_id)
& (ThinkingBack.update_time >= start_time)
& (ThinkingBack.found_answer == 1)
& (ThinkingBack.answer.is_null(False))
& (ThinkingBack.answer != "")
)
.order_by(ThinkingBack.update_time.desc())
.limit(3) # 最多返回5条最近的记录
)
if not records.exists():
return []
found_answers = []
for record in records:
if record.answer:
found_answers.append(f"问题:{record.question}\n答案:{record.answer}")
return found_answers
except Exception as e:
logger.error(f"获取最近已找到答案的记录失败: {e}")
return []
def _store_thinking_back( def _store_thinking_back(
chat_id: str, question: str, context: str, found_answer: bool, answer: str, thinking_steps: List[Dict[str, Any]] chat_id: str, question: str, context: str, found_answer: bool, answer: str, thinking_steps: List[Dict[str, Any]]
) -> None: ) -> None:
@ -969,8 +897,8 @@ async def build_memory_retrieval_prompt(
bot_name = global_config.bot.nickname bot_name = global_config.bot.nickname
chat_id = chat_stream.stream_id chat_id = chat_stream.stream_id
# 获取最近查询历史最近1小时内的查询) # 获取最近查询历史最近10分钟内的查询用于避免重复查询)
recent_query_history = _get_recent_query_history(chat_id, time_window_seconds=300.0) recent_query_history = _get_recent_query_history(chat_id, time_window_seconds=600.0)
if not recent_query_history: if not recent_query_history:
recent_query_history = "最近没有查询记录。" recent_query_history = "最近没有查询记录。"
@ -1000,7 +928,7 @@ async def build_memory_retrieval_prompt(
return "" return ""
# 解析概念列表和问题列表 # 解析概念列表和问题列表
_, questions = _parse_questions_json(response) _, questions = parse_questions_json(response)
if questions: if questions:
logger.info(f"解析到 {len(questions)} 个问题: {questions}") logger.info(f"解析到 {len(questions)} 个问题: {questions}")
@ -1009,7 +937,7 @@ async def build_memory_retrieval_prompt(
if enable_jargon_detection: if enable_jargon_detection:
# 使用匹配逻辑自动识别聊天中的黑话概念 # 使用匹配逻辑自动识别聊天中的黑话概念
concepts = _match_jargon_from_text(message, chat_id) concepts = match_jargon_from_text(message, chat_id)
if concepts: if concepts:
logger.info(f"黑话匹配命中 {len(concepts)} 个概念: {concepts}") logger.info(f"黑话匹配命中 {len(concepts)} 个概念: {concepts}")
else: else:
@ -1020,12 +948,12 @@ async def build_memory_retrieval_prompt(
# 对匹配到的概念进行jargon检索作为初始信息 # 对匹配到的概念进行jargon检索作为初始信息
initial_info = "" initial_info = ""
if enable_jargon_detection and concepts: if enable_jargon_detection and concepts:
concept_info = await _retrieve_concepts_with_jargon(concepts, chat_id) concept_info = await retrieve_concepts_with_jargon(concepts, chat_id)
if concept_info: if concept_info:
initial_info += concept_info initial_info += concept_info
logger.info(f"概念检索完成,结果: {concept_info}") logger.debug(f"概念检索完成,结果: {concept_info}")
else: else:
logger.info("概念检索未找到任何结果") logger.debug("概念检索未找到任何结果")
if not questions: if not questions:
logger.debug("模型认为不需要检索记忆或解析失败,不返回任何查询结果") logger.debug("模型认为不需要检索记忆或解析失败,不返回任何查询结果")
@ -1060,67 +988,47 @@ async def build_memory_retrieval_prompt(
elif result is not None: elif result is not None:
question_results.append(result) question_results.append(result)
# 获取最近10分钟内已找到答案的缓存记录
cached_answers = _get_recent_found_answers(chat_id, time_window_seconds=600.0)
# 合并当前查询结果和缓存答案(去重:如果当前查询的问题在缓存中已存在,优先使用当前结果)
all_results = []
# 先添加当前查询的结果
current_questions = set()
for result in question_results:
# 提取问题(格式为 "问题xxx\n答案xxx"
if result.startswith("问题:"):
question_end = result.find("\n答案:")
if question_end != -1:
current_questions.add(result[4:question_end])
all_results.append(result)
# 添加缓存答案(排除当前查询中已存在的问题)
for cached_answer in cached_answers:
if cached_answer.startswith("问题:"):
question_end = cached_answer.find("\n答案:")
if question_end != -1:
cached_question = cached_answer[4:question_end]
if cached_question not in current_questions:
all_results.append(cached_answer)
end_time = time.time() end_time = time.time()
if question_results: if all_results:
retrieved_memory = "\n\n".join(question_results) retrieved_memory = "\n\n".join(all_results)
logger.info(f"记忆检索成功,耗时: {(end_time - start_time):.3f}秒,包含 {len(question_results)} 条记忆") current_count = len(question_results)
cached_count = len(all_results) - current_count
logger.info(
f"记忆检索成功,耗时: {(end_time - start_time):.3f}秒,"
f"当前查询 {current_count} 条记忆,缓存 {cached_count} 条记忆,共 {len(all_results)} 条记忆"
)
return f"你回忆起了以下信息:\n{retrieved_memory}\n如果与回复内容相关,可以参考这些回忆的信息。\n" return f"你回忆起了以下信息:\n{retrieved_memory}\n如果与回复内容相关,可以参考这些回忆的信息。\n"
else: else:
logger.debug("所有问题均未找到答案") logger.debug("所有问题均未找到答案,且无缓存答案")
return "" return ""
except Exception as e: except Exception as e:
logger.error(f"记忆检索时发生异常: {str(e)}") logger.error(f"记忆检索时发生异常: {str(e)}")
return "" return ""
def _parse_questions_json(response: str) -> Tuple[List[str], List[str]]:
"""解析问题JSON返回概念列表和问题列表
Args:
response: LLM返回的响应
Returns:
Tuple[List[str], List[str]]: (概念列表, 问题列表)
"""
try:
# 尝试提取JSON可能包含在```json代码块中
json_pattern = r"```json\s*(.*?)\s*```"
matches = re.findall(json_pattern, response, re.DOTALL)
if matches:
json_str = matches[0]
else:
# 尝试直接解析整个响应
json_str = response.strip()
# 修复可能的JSON错误
repaired_json = repair_json(json_str)
# 解析JSON
parsed = json.loads(repaired_json)
# 只支持新格式包含concepts和questions的对象
if not isinstance(parsed, dict):
logger.warning(f"解析的JSON不是对象格式: {parsed}")
return [], []
concepts_raw = parsed.get("concepts", [])
questions_raw = parsed.get("questions", [])
# 确保是列表
if not isinstance(concepts_raw, list):
concepts_raw = []
if not isinstance(questions_raw, list):
questions_raw = []
# 确保所有元素都是字符串
concepts = [c for c in concepts_raw if isinstance(c, str) and c.strip()]
questions = [q for q in questions_raw if isinstance(q, str) and q.strip()]
return concepts, questions
except Exception as e:
logger.error(f"解析问题JSON失败: {e}, 响应内容: {response[:200]}...")
return [], []

View File

@ -8,7 +8,8 @@ import json
import re import re
from datetime import datetime from datetime import datetime
from typing import Tuple from typing import Tuple
from difflib import SequenceMatcher from typing import List
from json_repair import repair_json
from src.common.logger import get_logger from src.common.logger import get_logger
@ -16,101 +17,56 @@ from src.common.logger import get_logger
logger = get_logger("memory_utils") logger = get_logger("memory_utils")
def parse_md_json(json_text: str) -> list[str]:
"""从Markdown格式的内容中提取JSON对象和推理内容"""
json_objects = []
reasoning_content = ""
# 使用正则表达式查找```json包裹的JSON内容 def parse_questions_json(response: str) -> Tuple[List[str], List[str]]:
json_pattern = r"```json\s*(.*?)\s*```" """解析问题JSON返回概念列表和问题列表
matches = re.findall(json_pattern, json_text, re.DOTALL)
# 提取JSON之前的内容作为推理文本
if matches:
# 找到第一个```json的位置
first_json_pos = json_text.find("```json")
if first_json_pos > 0:
reasoning_content = json_text[:first_json_pos].strip()
# 清理推理内容中的注释标记
reasoning_content = re.sub(r"^//\s*", "", reasoning_content, flags=re.MULTILINE)
reasoning_content = reasoning_content.strip()
for match in matches:
try:
# 清理可能的注释和格式问题
json_str = re.sub(r"//.*?\n", "\n", match) # 移除单行注释
json_str = re.sub(r"/\*.*?\*/", "", json_str, flags=re.DOTALL) # 移除多行注释
if json_str := json_str.strip():
json_obj = json.loads(json_str)
if isinstance(json_obj, dict):
json_objects.append(json_obj)
elif isinstance(json_obj, list):
for item in json_obj:
if isinstance(item, dict):
json_objects.append(item)
except Exception as e:
logger.warning(f"解析JSON块失败: {e}, 块内容: {match[:100]}...")
continue
return json_objects, reasoning_content
def calculate_similarity(text1: str, text2: str) -> float:
"""
计算两个文本的相似度
Args: Args:
text1: 第一个文本 response: LLM返回的响应
text2: 第二个文本
Returns: Returns:
float: 相似度分数 (0-1) Tuple[List[str], List[str]]: (概念列表, 问题列表)
""" """
try: try:
# 预处理文本 # 尝试提取JSON可能包含在```json代码块中
text1 = preprocess_text(text1) json_pattern = r"```json\s*(.*?)\s*```"
text2 = preprocess_text(text2) matches = re.findall(json_pattern, response, re.DOTALL)
# 使用SequenceMatcher计算相似度 if matches:
similarity = SequenceMatcher(None, text1, text2).ratio() json_str = matches[0]
else:
# 尝试直接解析整个响应
json_str = response.strip()
# 如果其中一个文本包含另一个,提高相似度 # 修复可能的JSON错误
if text1 in text2 or text2 in text1: repaired_json = repair_json(json_str)
similarity = max(similarity, 0.8)
return similarity # 解析JSON
parsed = json.loads(repaired_json)
# 只支持新格式包含concepts和questions的对象
if not isinstance(parsed, dict):
logger.warning(f"解析的JSON不是对象格式: {parsed}")
return [], []
concepts_raw = parsed.get("concepts", [])
questions_raw = parsed.get("questions", [])
# 确保是列表
if not isinstance(concepts_raw, list):
concepts_raw = []
if not isinstance(questions_raw, list):
questions_raw = []
# 确保所有元素都是字符串
concepts = [c for c in concepts_raw if isinstance(c, str) and c.strip()]
questions = [q for q in questions_raw if isinstance(q, str) and q.strip()]
return concepts, questions
except Exception as e: except Exception as e:
logger.error(f"计算相似度时出错: {e}") logger.error(f"解析问题JSON失败: {e}, 响应内容: {response[:200]}...")
return 0.0 return [], []
def preprocess_text(text: str) -> str:
"""
预处理文本提高匹配准确性
Args:
text: 原始文本
Returns:
str: 预处理后的文本
"""
try:
# 转换为小写
text = text.lower()
# 移除标点符号和特殊字符
text = re.sub(r"[^\w\s]", "", text)
# 移除多余空格
text = re.sub(r"\s+", " ", text).strip()
return text
except Exception as e:
logger.error(f"预处理文本时出错: {e}")
return text
def parse_datetime_to_timestamp(value: str) -> float: def parse_datetime_to_timestamp(value: str) -> float:
""" """
@ -140,29 +96,3 @@ def parse_datetime_to_timestamp(value: str) -> float:
except Exception as e: except Exception as e:
last_err = e last_err = e
raise ValueError(f"无法解析时间: {value} ({last_err})") raise ValueError(f"无法解析时间: {value} ({last_err})")
def parse_time_range(time_range: str) -> Tuple[float, float]:
"""
解析时间范围字符串返回开始和结束时间戳
Args:
time_range: 时间范围字符串格式"YYYY-MM-DD HH:MM:SS - YYYY-MM-DD HH:MM:SS"
Returns:
Tuple[float, float]: (开始时间戳, 结束时间戳)
"""
if " - " not in time_range:
raise ValueError(f"时间范围格式错误,应为 '开始时间 - 结束时间': {time_range}")
parts = time_range.split(" - ", 1)
if len(parts) != 2:
raise ValueError(f"时间范围格式错误: {time_range}")
start_str = parts[0].strip()
end_str = parts[1].strip()
start_timestamp = parse_datetime_to_timestamp(start_str)
end_timestamp = parse_datetime_to_timestamp(end_str)
return start_timestamp, end_timestamp

View File

@ -14,6 +14,7 @@ from .tool_registry import (
from .query_chat_history import register_tool as register_query_chat_history from .query_chat_history import register_tool as register_query_chat_history
from .query_lpmm_knowledge import register_tool as register_lpmm_knowledge from .query_lpmm_knowledge import register_tool as register_lpmm_knowledge
from .query_person_info import register_tool as register_query_person_info from .query_person_info import register_tool as register_query_person_info
from .found_answer import register_tool as register_found_answer
from src.config.config import global_config from src.config.config import global_config
@ -21,6 +22,7 @@ def init_all_tools():
"""初始化并注册所有记忆检索工具""" """初始化并注册所有记忆检索工具"""
register_query_chat_history() register_query_chat_history()
register_query_person_info() register_query_person_info()
register_found_answer() # 注册found_answer工具
if global_config.lpmm_knowledge.lpmm_mode == "agent": if global_config.lpmm_knowledge.lpmm_mode == "agent":
register_lpmm_knowledge() register_lpmm_knowledge()

View File

@ -0,0 +1,40 @@
"""
found_answer工具 - 用于在记忆检索过程中标记找到答案
"""
from src.common.logger import get_logger
from .tool_registry import register_memory_retrieval_tool
logger = get_logger("memory_retrieval_tools")
async def found_answer(answer: str) -> str:
"""标记已找到问题的答案
Args:
answer: 找到的答案内容
Returns:
str: 确认信息
"""
# 这个工具主要用于标记,实际答案会通过返回值传递
logger.info(f"找到答案: {answer}")
return f"已确认找到答案: {answer}"
def register_tool():
"""注册found_answer工具"""
register_memory_retrieval_tool(
name="found_answer",
description="当你在已收集的信息中找到了问题的明确答案时,调用此工具标记已找到答案。只有在检索到明确、具体的答案时才使用此工具,不要编造信息。",
parameters=[
{
"name": "answer",
"type": "string",
"description": "找到的答案内容,必须基于已收集的信息,不要编造",
"required": True,
},
],
execute_func=found_answer,
)

View File

@ -275,10 +275,6 @@ async def get_chat_history_detail(chat_id: str, memory_ids: str) -> str:
except (json.JSONDecodeError, TypeError, ValueError): except (json.JSONDecodeError, TypeError, ValueError):
pass pass
# 添加原文内容
if record.original_text:
result_parts.append(f"原文内容:\n{record.original_text}")
results.append("\n".join(result_parts)) results.append("\n".join(result_parts))
if not results: if not results:
@ -318,7 +314,7 @@ def register_tool():
# 注册工具2获取记忆详情 # 注册工具2获取记忆详情
register_memory_retrieval_tool( register_memory_retrieval_tool(
name="get_chat_history_detail", name="get_chat_history_detail",
description="根据记忆ID展示某条或某几条记忆的具体内容。包括主题、时间、参与人、关键词、概括、关键信息点和原文内容等详细信息。需要先使用search_chat_history工具获取记忆ID。", description="根据记忆ID展示某条或某几条记忆的具体内容。包括主题、时间、参与人、关键词、概括和关键信息点等详细信息。需要先使用search_chat_history工具获取记忆ID。",
parameters=[ parameters=[
{ {
"name": "memory_ids", "name": "memory_ids",

View File

@ -5,7 +5,6 @@ from pydantic import BaseModel
from typing import Optional, List, Dict from typing import Optional, List, Dict
from src.common.logger import get_logger from src.common.logger import get_logger
from src.common.database.database_model import Expression, ChatStreams from src.common.database.database_model import Expression, ChatStreams
from .token_manager import get_token_manager
from .auth import verify_auth_token_from_cookie_or_header from .auth import verify_auth_token_from_cookie_or_header
import time import time

View File

@ -1,7 +1,7 @@
"""黑话(俚语)管理路由""" """黑话(俚语)管理路由"""
import json import json
from typing import Optional, List from typing import Optional, List, Annotated
from fastapi import APIRouter, HTTPException, Query from fastapi import APIRouter, HTTPException, Query
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
from peewee import fn from peewee import fn
@ -331,19 +331,19 @@ async def get_jargon_stats():
total = Jargon.select().count() total = Jargon.select().count()
# 已确认是黑话的数量 # 已确认是黑话的数量
confirmed_jargon = Jargon.select().where(Jargon.is_jargon == True).count() confirmed_jargon = Jargon.select().where(Jargon.is_jargon).count()
# 已确认不是黑话的数量 # 已确认不是黑话的数量
confirmed_not_jargon = Jargon.select().where(Jargon.is_jargon == False).count() confirmed_not_jargon = Jargon.select().where(~Jargon.is_jargon).count()
# 未判定的数量 # 未判定的数量
pending = Jargon.select().where(Jargon.is_jargon.is_null()).count() pending = Jargon.select().where(Jargon.is_jargon.is_null()).count()
# 全局黑话数量 # 全局黑话数量
global_count = Jargon.select().where(Jargon.is_global == True).count() global_count = Jargon.select().where(Jargon.is_global).count()
# 已完成推断的数量 # 已完成推断的数量
complete_count = Jargon.select().where(Jargon.is_complete == True).count() complete_count = Jargon.select().where(Jargon.is_complete).count()
# 关联的聊天数量 # 关联的聊天数量
chat_count = ( chat_count = (
@ -519,8 +519,8 @@ async def batch_delete_jargons(request: BatchDeleteRequest):
@router.post("/batch/set-jargon", response_model=JargonUpdateResponse) @router.post("/batch/set-jargon", response_model=JargonUpdateResponse)
async def batch_set_jargon_status( async def batch_set_jargon_status(
ids: List[int] = Query(..., description="黑话ID列表"), ids: Annotated[List[int], Query(description="黑话ID列表")],
is_jargon: bool = Query(..., description="是否是黑话"), is_jargon: Annotated[bool, Query(description="是否是黑话")],
): ):
"""批量设置黑话状态""" """批量设置黑话状态"""
try: try:

View File

@ -5,7 +5,6 @@ from pydantic import BaseModel
from typing import Optional, List, Dict from typing import Optional, List, Dict
from src.common.logger import get_logger from src.common.logger import get_logger
from src.common.database.database_model import PersonInfo from src.common.database.database_model import PersonInfo
from .token_manager import get_token_manager
from .auth import verify_auth_token_from_cookie_or_header from .auth import verify_auth_token_from_cookie_or_header
import json import json
import time import time

View File

@ -5,7 +5,6 @@
""" """
import os import os
import sys
import time import time
from datetime import datetime from datetime import datetime
from fastapi import APIRouter, HTTPException from fastapi import APIRouter, HTTPException

File diff suppressed because one or more lines are too long

View File

@ -7,7 +7,7 @@
<link rel="icon" type="image/x-icon" href="/maimai.ico" /> <link rel="icon" type="image/x-icon" href="/maimai.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" /> <meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>MaiBot Dashboard</title> <title>MaiBot Dashboard</title>
<script type="module" crossorigin src="/assets/index-DuFwC87p.js"></script> <script type="module" crossorigin src="/assets/index-DJb_iiTR.js"></script>
<link rel="modulepreload" crossorigin href="/assets/react-vendor-Dtc2IqVY.js"> <link rel="modulepreload" crossorigin href="/assets/react-vendor-Dtc2IqVY.js">
<link rel="modulepreload" crossorigin href="/assets/router-CWhjJi2n.js"> <link rel="modulepreload" crossorigin href="/assets/router-CWhjJi2n.js">
<link rel="modulepreload" crossorigin href="/assets/utils-CCeOswSm.js"> <link rel="modulepreload" crossorigin href="/assets/utils-CCeOswSm.js">