JOJO
478d24c4dd
本次变更是对版本控制功能的大范围重构，核心目标：
- 默认使用浅备份（只追踪 write_file/edit_file 实际修改的文件），避免大工作区首次初始化耗时十几秒；
- 保留可选的完全备份（git write-tree），不污染 git log；
- 回溯时统一选择范围（仅对话 / 对话+工作区）和模式（覆盖 / 复制）；
- 修复复制对话、侧边栏同步、新建对话延迟、diff 统计与渲染等连锁问题。
后端改动：
- modules/shallow_versioning.py（新增）
- 实现 ShallowVersioningManager：track_edit、make_snapshot、rewind、list_snapshots。
- 基于 difflib.SequenceMatcher 计算真实 insertions/deletions，replace 场景正确显示 +N/-M。
- diff 详情使用结构化 add/remove/context 行，不再解析 unified_diff 文本。
- modules/versioning_manager.py
- 完全备份从 git commit 改为 git write-tree，字段 commit/parent_commit 改为 tree_hash/parent_tree_hash。
- 恢复改用 read-tree + checkout-index；create_checkpoint 支持 diff_summary 参数。
- get_checkpoint_detail 保留 API 字段名 last_commit/restored_commit 以兼容前端。
- core/main_terminal_parts/tools_execution.py
- write_file/edit_file 在执行前调用浅备份 track_edit，保留编辑前状态。
- server/chat_flow_task_main.py
- 消息轮次结束时根据 backup_mode 选择完全备份 create_checkpoint 或浅备份 make_snapshot。
- 浅备份模式下同时写入 ConversationVersioningManager 检查点行，让前端能统一展示。
- server/conversation.py
- restore/detail/list API 支持 shallow 模式；list/detail 对浅备份实时重新计算统计。
- 创建对话时读取 personalization 的 versioning_backup_mode，shallow 模式固定 tracking_mode 为 conversation_only。
- core/web_terminal.py
- _ensure_conversation_versioning_enabled 读取默认备份模式并写入 versioning meta。
- modules/file_manager/crud_mixin.py / replace_mixin.py
- write_file / edit_file 返回 original_file / new_file / replacement_details（真实旧行/新行）。
- modules/personalization_manager.py
- 新增 versioning_enabled_by_default、versioning_backup_mode（shallow/full）。
- utils/perf_log.py（新增）
- 创建新对话全链路性能日志，用于定位延迟瓶颈。
- utils/conversation_manager/{index,list_search,crud}_mixin.py
- 加性能日志；index_mixin 主动检查新增对话文件，修复侧边栏不同步。
前端改动：
- static/src/components/overlay/VersioningDialog.vue
- 去掉顶部管理范围下拉；底部新增回溯范围与回溯模式选择。
- diff 详情改为 div.diff-line 网格布局（marker + content），支持横向滚动，状态显示缩写 A/M/D。
- 修复 pre 标签空白黑行、容器被挤窄导致折行等问题。
- static/src/app/methods/versioning.ts
- 移除 mismatch 弹窗逻辑；支持 copy 模式；selectVersioningCheckpoint 增强错误捕获。
- static/src/components/personalization/PersonalizationDrawer.vue
- 「工作区与权限」新增版本控制默认开关与备份方式选择。
- static/src/stores/personalization.ts
- 新增 versioning_enabled_by_default、versioning_backup_mode 字段。
- static/src/components/chat/actions/ToolAction.vue / toolRenderers.ts
- edit_file / write_file 工具展示基于 result.details / result.new_file 的真实替换结果。
- static/src/app/state.ts / watchers.ts / App.vue
- 新增 versioningInitializingBackupToastId，完全备份初始化时显示 toast。
- static/src/app/methods/conversation/action.ts / message/send.ts
- host + 完全备份 + 默认开启时，创建新对话/从 /new 发送首条消息显示「正在初始化备份」toast。
验证：
- /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9 -m unittest test.test_server_refactor_smoke 通过。
- npm run build 通过。
- 宿主机与 docker 模式新建对话、编辑文件、回溯、复制对话均验证通过。
387 lines
15 KiB
Python
387 lines
15 KiB
Python
# utils/conversation_manager.py - 对话持久化管理器（集成Token统计）
|
||
|
||
import json
|
||
import os
|
||
import time
|
||
import tempfile
|
||
import threading
|
||
from datetime import datetime
|
||
from pathlib import Path
|
||
from typing import Dict, List, Optional, Any
|
||
from dataclasses import dataclass
|
||
import time
|
||
try:
|
||
from config import DATA_DIR, HOST_WORKSPACES_FILE
|
||
except ImportError:
|
||
import sys
|
||
from pathlib import Path
|
||
project_root = Path(__file__).resolve().parents[1]
|
||
if str(project_root) not in sys.path:
|
||
sys.path.insert(0, str(project_root))
|
||
from config import DATA_DIR, HOST_WORKSPACES_FILE
|
||
|
||
try:
|
||
from utils.perf_log import perf_log
|
||
except Exception:
|
||
perf_log = None
|
||
|
||
@dataclass
|
||
class ConversationMetadata:
|
||
"""对话元数据"""
|
||
id: str
|
||
title: str
|
||
created_at: str
|
||
updated_at: str
|
||
project_path: Optional[str]
|
||
project_relative_path: Optional[str]
|
||
thinking_mode: bool
|
||
total_messages: int
|
||
total_tools: int
|
||
run_mode: str = "fast"
|
||
model_key: Optional[str] = None
|
||
has_images: bool = False
|
||
has_videos: bool = False
|
||
status: str = "active" # active, archived, error
|
||
|
||
|
||
class ListSearchMixin:
|
||
"""ConversationManager list search mixin 能力 mixin。"""
|
||
|
||
def get_conversation_list(self, limit: int = 50, offset: int = 0, non_empty: bool = False) -> Dict:
|
||
"""
|
||
获取对话列表
|
||
|
||
Args:
|
||
limit: 限制数量
|
||
offset: 偏移量
|
||
non_empty: 仅返回有内容（total_messages > 0）的对话，分页基于过滤后的结果
|
||
|
||
Returns:
|
||
Dict: 包含对话列表和统计信息
|
||
"""
|
||
t0 = time.perf_counter()
|
||
if perf_log:
|
||
perf_log("get_conversation_list enter", extra={"limit": limit, "offset": offset, "non_empty": non_empty})
|
||
try:
|
||
if non_empty:
|
||
# 过滤模式：全量加载后剔除空对话，再在过滤结果上分页，
|
||
# 保证 total / has_more 与"有内容对话"的真实数量一致。
|
||
index = self._ensure_index_covering(limit=10000, offset=0)
|
||
sorted_conversations = sorted(
|
||
index.items(),
|
||
key=lambda x: x[1].get("updated_at") or "",
|
||
reverse=True
|
||
)
|
||
sorted_conversations = [
|
||
item for item in sorted_conversations
|
||
if (item[1].get("total_messages", 0) or 0) > 0
|
||
]
|
||
total = len(sorted_conversations)
|
||
conversations = sorted_conversations[offset:offset + limit]
|
||
else:
|
||
# 总对话数按文件数统计，防止初始索引截断导致"没有更多"按钮消失
|
||
total_files = len(self._iter_conversation_files(sort_by_mtime=False))
|
||
|
||
index = self._ensure_index_covering(limit=limit, offset=offset)
|
||
|
||
# 按更新时间倒序排列（处理 None 值）
|
||
sorted_conversations = sorted(
|
||
index.items(),
|
||
key=lambda x: x[1].get("updated_at") or "",
|
||
reverse=True
|
||
)
|
||
|
||
# 分页
|
||
total = max(len(sorted_conversations), total_files)
|
||
conversations = sorted_conversations[offset:offset+limit]
|
||
|
||
# 格式化结果
|
||
result = []
|
||
for conv_id, metadata in conversations:
|
||
result.append({
|
||
"id": conv_id,
|
||
"title": metadata.get("title", "未命名对话"),
|
||
"created_at": metadata.get("created_at"),
|
||
"updated_at": metadata.get("updated_at"),
|
||
"project_path": metadata.get("project_path"),
|
||
"project_relative_path": metadata.get("project_relative_path"),
|
||
"thinking_mode": metadata.get("thinking_mode", False),
|
||
"total_messages": metadata.get("total_messages", 0),
|
||
"total_tools": metadata.get("total_tools", 0),
|
||
"status": metadata.get("status", "active")
|
||
})
|
||
|
||
elapsed_ms = (time.perf_counter() - t0) * 1000
|
||
if perf_log:
|
||
perf_log("get_conversation_list done", elapsed_ms=elapsed_ms, extra={"result_count": len(result), "total": total})
|
||
return {
|
||
"conversations": result,
|
||
"total": total,
|
||
"limit": limit,
|
||
"offset": offset,
|
||
"has_more": offset + limit < total
|
||
}
|
||
except Exception as e:
|
||
elapsed_ms = (time.perf_counter() - t0) * 1000
|
||
if perf_log:
|
||
perf_log("get_conversation_list error", elapsed_ms=elapsed_ms, extra={"error": str(e)})
|
||
print(f"⌘ 获取对话列表失败: {e}")
|
||
return {
|
||
"conversations": [],
|
||
"total": 0,
|
||
"limit": limit,
|
||
"offset": offset,
|
||
"has_more": False
|
||
}
|
||
|
||
def get_recent_conversation_summaries(
|
||
self,
|
||
limit: int = 10,
|
||
*,
|
||
exclude_conversation_id: Optional[str] = None,
|
||
first_message_max_chars: int = 100,
|
||
) -> List[Dict]:
|
||
"""返回当前工作区最近对话摘要：标题 + 首条用户消息。"""
|
||
try:
|
||
target_limit = max(1, int(limit))
|
||
listing = self.get_conversation_list(limit=10000, offset=0)
|
||
summaries: List[Dict] = []
|
||
for item in listing.get("conversations") or []:
|
||
conv_id = item.get("id")
|
||
if not conv_id or conv_id == exclude_conversation_id:
|
||
continue
|
||
data = self.load_conversation(conv_id) or {}
|
||
first_user_message = self._extract_first_user_message(
|
||
data.get("messages") or [],
|
||
max_chars=max(1, int(first_message_max_chars or 100)),
|
||
)
|
||
if not first_user_message:
|
||
continue
|
||
summaries.append({
|
||
"id": conv_id,
|
||
"title": item.get("title") or data.get("title") or "未命名对话",
|
||
"created_at": item.get("created_at") or data.get("created_at"),
|
||
"updated_at": item.get("updated_at") or data.get("updated_at"),
|
||
"total_messages": item.get("total_messages") or (data.get("metadata") or {}).get("total_messages", 0),
|
||
"total_tools": item.get("total_tools") or (data.get("metadata") or {}).get("total_tools", 0),
|
||
"first_user_message": first_user_message,
|
||
})
|
||
if len(summaries) >= target_limit:
|
||
break
|
||
return summaries
|
||
except Exception as e:
|
||
print(f"⌘ 获取最近对话摘要失败: {e}")
|
||
return []
|
||
|
||
def search_conversation_summaries(
|
||
self,
|
||
query: str = "",
|
||
*,
|
||
keywords: Optional[List[str]] = None,
|
||
start_date: Optional[str] = None,
|
||
end_date: Optional[str] = None,
|
||
limit: int = 10,
|
||
first_message_max_chars: int = 100,
|
||
exclude_conversation_id: Optional[str] = None,
|
||
) -> List[Dict]:
|
||
"""在当前工作区内搜索标题 + 首条用户消息，可按创建日期过滤。"""
|
||
try:
|
||
raw_keywords = keywords if isinstance(keywords, list) else []
|
||
normalized_keywords = [str(item).strip().lower() for item in raw_keywords if str(item or "").strip()]
|
||
fallback_query = (query or "").strip().lower()
|
||
if fallback_query:
|
||
normalized_keywords.append(fallback_query)
|
||
normalized_keywords = normalized_keywords[:3]
|
||
start_key = (start_date or "").strip()
|
||
end_key = (end_date or "").strip()
|
||
listing = self.get_conversation_list(limit=10000, offset=0)
|
||
results: List[Dict] = []
|
||
for item in listing.get("conversations") or []:
|
||
conv_id = item.get("id")
|
||
if not conv_id or conv_id == exclude_conversation_id:
|
||
continue
|
||
created_at = str(item.get("created_at") or "")
|
||
date_key = created_at[:10]
|
||
if start_key and date_key and date_key < start_key:
|
||
continue
|
||
if end_key and date_key and date_key > end_key:
|
||
continue
|
||
data = self.load_conversation(conv_id) or {}
|
||
first_message = self._extract_first_user_message(
|
||
data.get("messages") or [],
|
||
max_chars=max(1, int(first_message_max_chars or 100)),
|
||
)
|
||
# 空对话（没有任何用户文本）不应出现在搜索/列表结果中，
|
||
# 否则无关键词 list 最近对话时会出现大量“新对话”。
|
||
if not first_message:
|
||
continue
|
||
title = item.get("title") or data.get("title") or "未命名对话"
|
||
haystack = f"{title} {first_message}".lower()
|
||
if normalized_keywords and not any(keyword in haystack for keyword in normalized_keywords):
|
||
continue
|
||
results.append({
|
||
"id": conv_id,
|
||
"title": title,
|
||
"first_user_message": first_message,
|
||
"created_at": item.get("created_at") or data.get("created_at"),
|
||
"updated_at": item.get("updated_at") or data.get("updated_at"),
|
||
"total_messages": item.get("total_messages") or (data.get("metadata") or {}).get("total_messages", 0),
|
||
"total_tools": item.get("total_tools") or (data.get("metadata") or {}).get("total_tools", 0),
|
||
})
|
||
if len(results) >= max(1, int(limit or 10)):
|
||
break
|
||
return results
|
||
except Exception as e:
|
||
print(f"⌘ 搜索对话摘要失败: {e}")
|
||
return []
|
||
|
||
def delete_conversation(self, conversation_id: str) -> bool:
|
||
"""
|
||
删除对话
|
||
|
||
Args:
|
||
conversation_id: 对话ID
|
||
|
||
Returns:
|
||
bool: 删除是否成功
|
||
"""
|
||
try:
|
||
# 删除对话文件
|
||
file_path = self._get_conversation_file_path(conversation_id)
|
||
if file_path.exists():
|
||
file_path.unlink()
|
||
|
||
# 从索引中删除
|
||
index = self._load_index()
|
||
if conversation_id in index:
|
||
del index[conversation_id]
|
||
self._save_index(index)
|
||
|
||
# 如果删除的是当前对话，清除当前对话ID
|
||
if self.current_conversation_id == conversation_id:
|
||
self.current_conversation_id = None
|
||
|
||
print(f"🗑️ 已删除对话: {conversation_id}")
|
||
return True
|
||
except Exception as e:
|
||
print(f"⌘ 删除对话失败 {conversation_id}: {e}")
|
||
return False
|
||
|
||
def archive_conversation(self, conversation_id: str) -> bool:
|
||
"""
|
||
归档对话（标记为已归档，不删除）
|
||
|
||
Args:
|
||
conversation_id: 对话ID
|
||
|
||
Returns:
|
||
bool: 归档是否成功
|
||
"""
|
||
try:
|
||
# 更新对话状态
|
||
conversation_data = self.load_conversation(conversation_id)
|
||
if not conversation_data:
|
||
return False
|
||
|
||
conversation_data["metadata"]["status"] = "archived"
|
||
conversation_data["updated_at"] = datetime.now().isoformat()
|
||
|
||
# 保存更新
|
||
self._save_conversation_file(conversation_id, conversation_data)
|
||
self._update_index(conversation_id, conversation_data)
|
||
|
||
print(f"📦 已归档对话: {conversation_id}")
|
||
return True
|
||
except Exception as e:
|
||
print(f"⌘ 归档对话失败 {conversation_id}: {e}")
|
||
return False
|
||
|
||
def search_conversations(self, query: str, limit: int = 20) -> List[Dict]:
|
||
"""
|
||
搜索对话
|
||
|
||
Args:
|
||
query: 搜索关键词
|
||
limit: 限制数量
|
||
|
||
Returns:
|
||
List[Dict]: 匹配的对话列表
|
||
"""
|
||
try:
|
||
index = self._load_index()
|
||
results = []
|
||
|
||
query_lower = query.lower()
|
||
|
||
for conv_id, metadata in index.items():
|
||
# 搜索标题
|
||
title = metadata.get("title", "").lower()
|
||
if query_lower in title:
|
||
score = 100 # 标题匹配权重最高
|
||
results.append((score, {
|
||
"id": conv_id,
|
||
"title": metadata.get("title"),
|
||
"created_at": metadata.get("created_at"),
|
||
"updated_at": metadata.get("updated_at"),
|
||
"project_path": metadata.get("project_path"),
|
||
"match_type": "title"
|
||
}))
|
||
continue
|
||
|
||
# 搜索项目路径
|
||
project_path = metadata.get("project_path", "").lower()
|
||
if query_lower in project_path:
|
||
results.append((50, {
|
||
"id": conv_id,
|
||
"title": metadata.get("title"),
|
||
"created_at": metadata.get("created_at"),
|
||
"updated_at": metadata.get("updated_at"),
|
||
"project_path": metadata.get("project_path"),
|
||
"match_type": "project_path"
|
||
}))
|
||
|
||
# 按分数排序
|
||
results.sort(key=lambda x: x[0], reverse=True)
|
||
|
||
# 返回前N个结果
|
||
return [result[1] for result in results[:limit]]
|
||
except Exception as e:
|
||
print(f"⌘ 搜索对话失败: {e}")
|
||
return []
|
||
|
||
def cleanup_old_conversations(self, days: int = 30) -> int:
|
||
"""
|
||
清理旧对话（可选功能）
|
||
|
||
Args:
|
||
days: 保留天数
|
||
|
||
Returns:
|
||
int: 清理的对话数量
|
||
"""
|
||
try:
|
||
from datetime import datetime, timedelta
|
||
cutoff_date = datetime.now() - timedelta(days=days)
|
||
cutoff_iso = cutoff_date.isoformat()
|
||
|
||
index = self._load_index()
|
||
to_delete = []
|
||
|
||
for conv_id, metadata in index.items():
|
||
updated_at = metadata.get("updated_at", "")
|
||
if updated_at < cutoff_iso and metadata.get("status") != "archived":
|
||
to_delete.append(conv_id)
|
||
|
||
deleted_count = 0
|
||
for conv_id in to_delete:
|
||
if self.delete_conversation(conv_id):
|
||
deleted_count += 1
|
||
|
||
if deleted_count > 0:
|
||
print(f"🧹 清理了 {deleted_count} 个旧对话")
|
||
|
||
return deleted_count
|
||
except Exception as e:
|
||
print(f"⌘ 清理旧对话失败: {e}")
|
||
return 0
|