diff --git a/.gitignore b/.gitignore index 08b6b387..196407e9 100644 --- a/.gitignore +++ b/.gitignore @@ -80,7 +80,6 @@ research/ # 含私人信息 / 个人工作流约定（脱敏前不进仓库） CLAUDE.md api_doc/ -android-webview-app/ config/host_workspaces.json.example upload_android_apk.sh scripts/zip_agents.sh diff --git a/AGENTS.md b/AGENTS.md index 63cbd27d..88c03409 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -305,8 +305,10 @@ AI 执行以下流程时，每一步都要向用户说明在做什么： - `android-webview-app/APP_CHANGELOG.md`：在顶部新增当前版本说明 3. 完成修改后提醒用户运行上传脚本 - 在发布流程中运行：`bash ./upload_android_apk.sh`（本地私有脚本，未随仓库发布） + - 脚本除上传 APK 外，还会同步 `app/build.gradle.kts` 与 `APP_CHANGELOG.md` 到服务器（2026-09-02 修复） > 禁止只改前端代码而不更新版本号/更新说明，否则会导致客户端更新提示与分发信息不一致。 +> 服务端 `/api/app/version` 的版本号是从服务器文件系统上的 `build.gradle.kts` 解析的（实现见 `server/status/docker.py`），**不是从 APK 包解析**——只传 APK 不同步元数据文件会导致 App 检测不到新版本。 ## 9) 给 Agent 的硬性要求 diff --git a/android-webview-app/.gitignore b/android-webview-app/.gitignore new file mode 100644 index 00000000..1741fc1d --- /dev/null +++ b/android-webview-app/.gitignore @@ -0,0 +1,25 @@ +# macOS +.DS_Store + +# Local Android/IDE state +local.properties +.idea/ +.gradle/ + +# Build outputs +build/ +**/build/ + +# Signing keys and secrets +*.jks +*.keystore + +# Logs +*.log + +# Release artifacts +*.apk +*.aab +*.apks +*.dm +app/release/ diff --git a/android-webview-app/APP_CHANGELOG.md b/android-webview-app/APP_CHANGELOG.md new file mode 100644 index 00000000..9115b8ca --- /dev/null +++ b/android-webview-app/APP_CHANGELOG.md @@ -0,0 +1,206 @@ +# App 更新说明 + +# 1.0.43 +- 应用更名为 Astrion（原 CYJ AI Agent） + +# 1.0.42 +- 修复 App 内点击外部链接无反应的问题：对话中的网页链接、引用来源链接现在会跳转系统浏览器打开，不再被静默吞掉 +- 站内页面仍在 App 内打开，且站内外判断改为域名精确匹配，防止伪造域名绕过 + +# 1.0.41 +- 修复 Android App 内文件下载报错 404 的问题：下载地址不再错误地把当前对话路径拼进去 +- 修复下载时 `CookieManager` 在非主线程调用导致的崩溃提示 + +# 1.0.40 +- 修复 Android App 内文件下载后系统提示「下载失败」的问题： + - 放弃系统 DownloadManager，改为应用内下载到私有目录 + - 下载完成后弹出系统「分享文件」面板，让用户选择保存到下载管理、QQ浏览器、微信等目标 + - 仍保留 WebView Cookie，下载接口可正常鉴权 + +# 1.0.39 +- 修复 Android App 内图片/视频「从本地发送」无响应的问题：在唤起系统文件选择器前主动请求 `READ_MEDIA_IMAGES` / `READ_MEDIA_VIDEO` 权限，拒绝后支持再次请求并引导到系统设置 +- 修复 Android App 内所有下载（文件卡片、markdown 下载链接）提示「下载启动失败」的问题： + - 前端在 App 内优先调用 `AndroidDownloadBridge` + - Android 端将相对下载地址补全为绝对地址，并携带 WebView 登录 Cookie，避免 401 / 无效链接 + - Android 10+ 下载目录异常时自动回退到 DownloadManager 默认位置 + - Android 9 及以下未授予存储权限时先请求权限再开始下载 + +# 1.0.38 +- PDF 文件卡片改为前端 `vue-pdf-embed` 渲染：在卡片下方直接滚动预览多页，不再调用浏览器原生 PDF 预览或 Android 原生 PDF 预览 Activity +- 与桌面端保持一致的文件卡片交互：PDF 和其他类型文件统一头部 + 下方预览布局 + +# 1.0.37 +- 修复 Android App 内 PDF 文件卡片无法 inline 显示的问题：WebView 不支持 iframe PDF，改为卡片下方显示占位区，点击后调原生 PDF 预览 +- 修复进入含 PDF 卡片的对话自动弹出「已开始下载」的误报：移除 WebView `setDownloadListener`，避免把 PDF 加载请求误判为下载 +- 修复文件卡片点击下载无反应的问题：保留 `AndroidDownloadBridge` 走系统 DownloadManager +- 修复图片/视频「从本地发送」仍然唤起相册/Photo Picker 的问题：Android 端文件选择器改用 `ACTION_OPEN_DOCUMENT` 走系统文件管理器 +- 前端常规设置新增「上传诊断日志」按钮，便于排查 App 端问题 + +# 1.0.36 +- 修复 Android App 内文件卡片点击「下载」无反应的问题：前端优先调用 Android DownloadBridge，WebView 同时注册 `setDownloadListener` 兜底，使用系统 DownloadManager 下载到 Download 目录 +- PDF 文件卡片移除单独的「预览」按钮，改为与桌面端一致的卡片下方 inline 预览（iframe），视觉和交互统一 +- 修复 Android App 内图片/视频「从本地发送」容易失败的问题： + - 延长文件选择器关闭后的 backdrop 点击屏蔽时长，避免 touch 穿透误关选择弹窗导致上传中断 + - Android 端统一使用 `ACTION_GET_CONTENT` 选择器（按 `image/*` / `video/*` 过滤），避免不同 ROM 的相册/Photo Picker 返回临时 URI 导致上传失效 + - 上传链路增强对 Android WebView 返回文件无文件名/无扩展名的兼容：前端根据 MIME 类型补全文件名，后端在 filename 缺失时自动推断兜底文件名 + +# 1.0.35 +- 新增 PDF 文件原生预览：集成 AndroidPdfViewer，点击卡片「预览」按钮可在 App 内直接查看 PDF，不再依赖浏览器 + +# 1.0.32 +- 识别引擎改为预初始化：模型下载完成后立即在后台加载，用户点击时直接可用（不再每次点击等3-5秒初始化） +- startRecordingInternal 增加详细日志，便于排查录音问题 + +# 1.0.31 +- 修复语音识别无结果：去掉多余的 decode() 调用，对齐 sherpa-onnx Android 官方 API（getResult 内部已含 decode） +- recognizeSegment 增加识别结果详细日志 + +# 1.0.30 +- 修复语音识别无声问题：前端处理 initializing 状态，防止初始化期间重复触发 startListening 导致录音永不停止 +- Android 端增加启动取消机制，初始化期间按停止不会意外开始录音 +- 完善录音链路日志，便于后续调试 + +# 1.0.29 +- 暂时禁用 VAD（Android 端 Silero VAD 存在 Native 兼容性问题），改为整段识别模式 +- 录音结束后一次性识别，效果无差异，彻底消除闪退 + +# 1.0.28 +- 修复 VAD 初始化导致闪退：maxSpeechDuration 30f→5.0f（默认值），VAD 失败时降级为整段识别不崩溃 + +# 1.0.27 +- 语音调试日志改为直接上传服务器（/api/voice_debug），无需手动找文件 + +# 1.0.26 +- 修复点击麦克风后闪退问题：initEngine 增加文件完整性前置校验，numThreads 降为 1 降低内存压力 +- 新增「保存日志」按钮，调试日志写入手机 Download/voice_debug.log +- 下载/删除前自动释放引擎资源，避免文件锁定导致异常 + +# 1.0.25 +- 修复下载进度一直显示 0% 的问题（整数除法 bug） +- 新增模型文件完整性校验（文件大小 ±5% 容差），避免下载中断后显示“已下载” +- 新增「删除模型」按钮，可清理不完整的模型文件 +- 模型文件移至 agents 代码目录外（/opt/agent/voice_models/），避免部署时被覆盖 + +# 1.0.24 +- 修复语音按钮在首次启动（模型未下载）时不显示的问题：VoiceBridge 改为随 App 启动即注册，不再等待模型下载完成 +- 修复个人空间中「下载语音模型」按钮误提示「仅在 App 内支持」的问题 + +# 1.0.23 +- 新增端侧语音输入功能：集成 sherpa-onnx SenseVoice int8 离线语音识别 +- 支持中英混说 + 自动标点，完全离线运行，无需网络 +- 语音模型约 228MB，首次使用自动下载（可在「个人空间 → 语音模型」手动下载） +- 输入框语音按钮仅在 App 端显示，桌面端隐藏 +- 新增录音权限请求（RECORD_AUDIO） + +# 1.0.22 +- 修复 Android App 内从本地发送图片/视频无效的问题：补充 Android 13+ 所需的媒体读取权限（`READ_MEDIA_IMAGES` / `READ_MEDIA_VIDEO`） +- 修复 Android App 文件选择器在 Android 13+ 上回调丢失的问题：将已废弃的 `startActivityForResult` 升级为 `registerForActivityResult`（Activity Result API） + +# 1.0.21 +- 修复 Android App 内手机端展开“对话记录”时仍沿用旧宽度的问题，面板宽度与桌面端对话侧栏保持一致 +- 修复 Android App 内对话记录展开面板顶部残留系统栏空白的问题，内容从面板顶端开始显示 + +# 1.0.20 +- 新增“新用户欢迎弹窗 + 新手教程”持久化机制：普通账号首次进入会提示是否开始教程，选择“开始吧”或“不再提示”后将状态写入 `data/users.json` +- 新增后端教程状态接口（查询/更新），并排除宿主机模式用户，避免 host 模式触发教程弹窗 +- 修复宿主机模式切回普通账号登录时可能遗留 `host_mode` 状态，导致教程提示判定异常的问题 +- 优化新手教程欢迎弹窗视觉风格：按钮与系统现有主题风格统一，支持多主题配色 + +# 1.0.19 +- 修复 Android APK 内「个人空间-新手教程」期间无法上下滚动的问题：教程遮罩层现在会正确转发触摸滑动到可滚动容器 +- 优化教程滚动目标识别：优先命中当前可滚动页面区域，减少 WebView 场景下滑动失效 +- 调整软件更新页版本展示：移除括号内的构建代数，仅显示语义版本号（如 1.0.19） + +# 1.0.18 +- 新增完整「新手教程」系统（高光 + 悬浮说明窗）：覆盖桌面端与手机端核心功能入口 +- 教程交互统一优化：仅保留“下一步”，支持自动点击与点击特效，避免误触真实页面 +- 手机端教程新增专属流程：菜单展开、对话记录、工作文件、新建对话、模型/思考模式、个人空间全链路引导 +- 教程定位与层级修复：高光区域更贴合小目标；教程弹窗始终保持最高图层，不再被移动端菜单遮挡 +- 个人空间教程体验优化：教程期间支持滚动查看内容；移动端标签自动横向滚动到可见后再点击；桌面端标签自动纵向滚动后再点击 +- 新增“连接状态指示灯”教程步骤：绿色表示连接正常，红色表示与后端断开 +- 教程入口页精简：移除“已完成/重新开始”状态文案，统一为“开始新手教程” +- 主题命名文案统一：将“Claude 经典”调整为“经典”，并同步更新相关引导文案 +- 模型偏好页优化：移除“默认思考模型”中“思考模式”的“推荐”标记 +- 修复“对话回顾”弹窗多主题适配问题：经典/明亮/夜间主题下的背景、边框、文本、按钮与状态样式统一跟随主题变量 + +# 1.0.17 +- 登录/注册页改为 Vue 实现，统一前端技术栈 +- 登录/注册页接入主题变量体系，自动跟随已保存主题（Claude / 浅色 / 深色） +- 移除登录/注册页独立主题切换控件，主题入口统一由主界面个人空间管理 + +# 1.0.16 +- 工作区顶部品牌图标改为内联 SVG 渲染（不再依赖静态路径加载） +- 修复深色/浅色主题下 logo 颜色适配问题，跟随主题文本色显示 + +# 1.0.15 +- 修复图片查看链路：`view_image` 工具结果可正确附带图片内容，降低“识别成其他图片/内容”的误判 +- 新增「强约束系统」开关（个人空间）：可分别约束 terminal 系列与子智能体系列工具，要求先阅读对应 `skills/*/SKILL.md`（且仅对已启用 skill 生效） +- 优化输入与消息气泡换行策略：达到最大宽度即换行 +- 修复深色模式下 `run_python` 结果区显示异常，代码与输出统一为白色文字 + +# 1.0.14 +- 移除前端「实验功能」中的 Liquid Glass 实验页与悬浮组件，减少无关试验功能干扰 +- 删除 `scripts/liquid-glass-demo` 目录（含 node_modules/.next），显著降低部署压缩包体积 + +# 1.0.13 +- 修复工作区连接状态指示灯在轮询模式下常亮问题：新增连接探活心跳，断连时可正确切换为离线状态 +- 调整状态灯视觉：离线红灯与在线绿灯均支持扩散呼吸特效 + +# 1.0.12 +- 进一步修复手机端菜单点击反馈：增强去除系统默认蓝色点击高亮框（含快捷菜单按钮按压态） + +# 1.0.11 +- 优化手机端点击反馈样式：移除默认蓝色点击高亮矩形，减少触控干扰感 + +# 1.0.10 +- 手机 App 左上角菜单新增“刷新页面”入口（仅 App 端显示） +- 加载界面（正在连接服务器）新增“刷新页面”按钮 +- 继续优化 App 内更新下载链路稳定性 + +# 1.0.9-test1 +- 测试版本：验证 App 内下载更新链路（DownloadManager）是否在不同浏览器环境下稳定工作 + +# 1.0.9 +- 修复 App 内点击“更新”后依赖外部浏览器下载导致部分机型失败的问题 +- 新增 Android JS Bridge 下载通道：优先使用系统 DownloadManager 直接下载 APK +- 前端下载逻辑增加桥接优先与浏览器回退，提升下载成功率 + +# 1.0.8 +- 同步最新对话压缩体验：压缩中状态提示更稳定，压缩完成后反馈更清晰 +- 优化压缩后会话切换与列表刷新表现，减少需要手动刷新的情况 +- 修复电脑端摘要行与正式输出内容的对齐问题 + +# 1.0.7 +- 应用图标替换为新的机器人插画 PNG 版本 + +# 1.0.6 +- 应用图标替换为指定 JPG 图片资源 +- 更新 AndroidManifest 图标引用（icon / roundIcon） + +# 1.0.5 +- 手机端「工作文件」侧栏最大宽度由 86% 提升到 100% +- 保持默认 60% 起始宽度，并在内容需要时继续自适应扩展 + +# 1.0.4 +- 调整手机端「工作文件」侧栏宽度策略：默认按 60% 屏宽展开 +- 当内容较宽时可自适应扩展侧栏宽度（上限 86%），减少内容被截断 +- 保持「对话记录」侧栏宽度逻辑不变 + +# 1.0.3 +- 修复 App 端版本读取时机，避免显示“当前版本未知(?)” +- 进一步隐藏 App 端页面滚动条（保留页面内部可滚动区域） + +# 1.0.2 +- 修复 App 端“当前版本显示未知”问题（版本号改为优先从 Android JS Bridge 读取） +- 优化“软件更新”页更新日志展示：长内容在卡片内滚动，不再无限拉长 +- 强化 App 页面滚动隐藏补丁，减少出现系统滚动条的情况 +- 更新上传脚本：支持自动打包 + 上传，并同步版本号与更新说明 + +# 1.0.1 +- 移除 App 页面上下滚动条（仅 App 壳生效） +- 优化 App 内页面样式补丁与资源版本参数 + +## 1.0.0 +- 首次发布 Android WebView 壳应用 +- 适配移动端抽屉宽度与状态栏主题联动 +- 新增个人空间「软件更新」页（仅 App 端显示） diff --git a/android-webview-app/README.md b/android-webview-app/README.md new file mode 100644 index 00000000..c3bfe906 --- /dev/null +++ b/android-webview-app/README.md @@ -0,0 +1,45 @@ +# Astrion Android WebView 壳应用 + +这个目录是一个最小可用的 Android APK 壳工程： +- 后端仍运行在你的服务器 +- APK 只承载 WebView 前端 + +## 1) 打开工程 + +Android Studio -> Open -> 选择 `android-webview-app`。 + +首次打开会自动下载 Gradle 依赖（需要联网）。 + +## 2) 配置后端地址（必须） + +仓库代码中不包含真实后端域名，构建前需要通过以下任一方式注入（优先级从高到低）： + +1. 命令行参数：`./gradlew assembleRelease -PHOME_URL=https://your-server.example.com` +2. `local.properties`（本机文件，不纳入版本控制）：添加一行 `HOME_URL=https://your-server.example.com` +3. 环境变量：`export HOME_URL=https://your-server.example.com` + +不配置则使用占位地址 `https://agent.example.com`，App 无法连接真实服务。 + +## 3) 可改项 + +- 应用名：`app/src/main/res/values/strings.xml` +- 后端地址：见上文「配置后端地址」，最终编译进 `BuildConfig.HOME_URL` + +## 4) 调试打包 + +- 连接手机/模拟器后，点击 Run +- 生成 release APK： + - 配置签名环境变量 `ANDROID_KEYSTORE_PATH` / `ANDROID_KEYSTORE_PASSWORD` / `ANDROID_KEY_ALIAS` / `ANDROID_KEY_PASSWORD` + - `./gradlew assembleRelease` + +## 5) 已处理能力 + +- JS / DOM Storage 已开启 +- Cookie 已开启 +- 文件上传 (``) 已支持 +- 物理返回键支持网页回退 +- 外部链接自动跳转系统浏览器打开 + +## 6) Nginx 建议 + +如后续出现 WebSocket 不稳定，可把 `Connection 'upgrade'` 改为官方 map 写法（按 upgrade 条件设置）。 diff --git a/android-webview-app/app/build.gradle.kts b/android-webview-app/app/build.gradle.kts new file mode 100644 index 00000000..519d81c6 --- /dev/null +++ b/android-webview-app/app/build.gradle.kts @@ -0,0 +1,100 @@ +import java.util.Properties + +plugins { + id("com.android.application") + id("org.jetbrains.kotlin.android") +} + +fun resolveConfig(name: String): String? { + return (project.findProperty(name) as String?)?.takeIf { it.isNotBlank() } + ?: System.getenv(name)?.takeIf { it.isNotBlank() } +} + +// 后端服务地址（构建期注入，仓库中只保留占位符）： +// 优先级：-PHOME_URL 命令行参数 > local.properties 的 HOME_URL > 环境变量 HOME_URL > 占位默认值 +// 自部署构建示例：./gradlew assembleRelease -PHOME_URL=https://your-server.example.com +fun resolveHomeUrl(): String { + (project.findProperty("HOME_URL") as String?)?.takeIf { it.isNotBlank() }?.let { return it } + val localPropsFile = rootProject.file("local.properties") + if (localPropsFile.exists()) { + val props = Properties() + localPropsFile.inputStream().use { props.load(it) } + props.getProperty("HOME_URL")?.takeIf { it.isNotBlank() }?.let { return it } + } + System.getenv("HOME_URL")?.takeIf { it.isNotBlank() }?.let { return it } + return "https://agent.example.com" +} + +val homeUrl: String = resolveHomeUrl() + +android { + namespace = "com.cyjai.agent" + compileSdk = 35 + + defaultConfig { + applicationId = "com.cyjai.agent" + minSdk = 24 + targetSdk = 35 + versionCode = 45 + versionName = "1.0.43" + + testInstrumentationRunner = "androidx.test.runner.AndroidJUnitRunner" + buildConfigField("String", "HOME_URL", "\"$homeUrl\"") + } + + buildFeatures { + buildConfig = true + } + + signingConfigs { + create("release") { + val storeFilePath = resolveConfig("ANDROID_KEYSTORE_PATH") + val storePass = resolveConfig("ANDROID_KEYSTORE_PASSWORD") + val keyAliasValue = resolveConfig("ANDROID_KEY_ALIAS") + val keyPass = resolveConfig("ANDROID_KEY_PASSWORD") + + if (!storeFilePath.isNullOrBlank()) { + storeFile = file(storeFilePath) + } + storePassword = storePass + keyAlias = keyAliasValue + keyPassword = keyPass + } + } + + buildTypes { + release { + isMinifyEnabled = false + signingConfig = signingConfigs.getByName("release") + proguardFiles( + getDefaultProguardFile("proguard-android-optimize.txt"), + "proguard-rules.pro" + ) + } + } + + compileOptions { + sourceCompatibility = JavaVersion.VERSION_17 + targetCompatibility = JavaVersion.VERSION_17 + } + kotlinOptions { + jvmTarget = "17" + } +} + +dependencies { + implementation("androidx.core:core-ktx:1.13.1") + implementation("androidx.appcompat:appcompat:1.7.0") + implementation("com.google.android.material:material:1.12.0") + implementation("androidx.activity:activity-ktx:1.9.2") + implementation("androidx.lifecycle:lifecycle-runtime-ktx:2.8.0") + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-android:1.8.1") + + // PDF 预览（不依赖浏览器） + implementation("com.github.mhiew:android-pdf-viewer:3.2.0-beta.3") + + // sherpa-onnx 语音识别库（需手动下载 AAR 放到 app/libs/） + // 下载地址：https://huggingface.co/csukuangfj/sherpa-onnx-libs/tree/main/android/aar + // 下载最新 sherpa-onnx-*.aar 放到 app/libs/ 目录即可 + implementation(fileTree(mapOf("dir" to "libs", "include" to listOf("*.aar")))) +} diff --git a/android-webview-app/app/libs/sherpa-onnx-1.10.45.aar b/android-webview-app/app/libs/sherpa-onnx-1.10.45.aar new file mode 100644 index 00000000..f50d1b2c Binary files /dev/null and b/android-webview-app/app/libs/sherpa-onnx-1.10.45.aar differ diff --git a/android-webview-app/app/proguard-rules.pro b/android-webview-app/app/proguard-rules.pro new file mode 100644 index 00000000..2cb8d493 --- /dev/null +++ b/android-webview-app/app/proguard-rules.pro @@ -0,0 +1 @@ +# Intentionally minimal for WebView shell app diff --git a/android-webview-app/app/src/main/AndroidManifest.xml b/android-webview-app/app/src/main/AndroidManifest.xml new file mode 100644 index 00000000..4835d3af --- /dev/null +++ b/android-webview-app/app/src/main/AndroidManifest.xml @@ -0,0 +1,49 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/android-webview-app/app/src/main/java/com/cyjai/agent/MainActivity.kt b/android-webview-app/app/src/main/java/com/cyjai/agent/MainActivity.kt new file mode 100644 index 00000000..479576ed --- /dev/null +++ b/android-webview-app/app/src/main/java/com/cyjai/agent/MainActivity.kt @@ -0,0 +1,686 @@ +package com.cyjai.agent + +import android.Manifest +import android.annotation.SuppressLint +import android.content.ActivityNotFoundException +import android.content.Intent +import android.content.pm.PackageManager +import android.graphics.Color +import android.net.Uri +import android.os.Build +import android.os.Bundle +import android.provider.DocumentsContract +import android.provider.Settings +import android.util.Log +import android.view.ViewGroup +import android.view.View +import android.content.Context +import android.webkit.CookieManager +import android.webkit.JavascriptInterface +import android.webkit.PermissionRequest +import android.webkit.ValueCallback +import android.webkit.WebChromeClient +import android.webkit.WebResourceRequest +import android.webkit.WebSettings +import android.webkit.WebView +import android.webkit.WebViewClient +import android.widget.Toast +import androidx.activity.OnBackPressedCallback +import androidx.activity.result.ActivityResultLauncher +import androidx.activity.result.contract.ActivityResultContracts +import androidx.appcompat.app.AppCompatActivity +import androidx.core.app.ActivityCompat +import androidx.core.content.ContextCompat +import androidx.core.content.FileProvider +import androidx.core.view.ViewCompat +import androidx.core.view.WindowInsetsCompat +import androidx.core.view.WindowCompat +import androidx.lifecycle.lifecycleScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext +import java.io.File +import java.net.HttpURLConnection +import java.net.URL +import java.net.URLConnection +import java.util.Locale + +class MainActivity : AppCompatActivity() { + + private lateinit var webView: WebView + private var filePathCallback: ValueCallback>? = null + private var voiceBridge: VoiceBridge? = null + private var pendingFileChooserParams: WebChromeClient.FileChooserParams? = null + + companion object { + private const val TAG = "MainActivity" + private const val REQUEST_RECORD_AUDIO = 300 + private const val REQUEST_WRITE_STORAGE = 301 + private const val REQUEST_READ_MEDIA = 302 + private val HOME_URL: String = BuildConfig.HOME_URL + private val HOME_HOST: String? = Uri.parse(HOME_URL).host + private const val WEB_ASSET_VERSION = "20260624_2" + } + + private val fileChooserLauncher: ActivityResultLauncher = registerForActivityResult( + ActivityResultContracts.StartActivityForResult() + ) { result -> + val results = WebChromeClient.FileChooserParams.parseResult(result.resultCode, result.data) + filePathCallback?.onReceiveValue(results) + filePathCallback = null + } + + @SuppressLint("SetJavaScriptEnabled") + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + + webView = WebView(this) + webView.layoutParams = ViewGroup.LayoutParams( + ViewGroup.LayoutParams.MATCH_PARENT, + ViewGroup.LayoutParams.MATCH_PARENT + ) + webView.isVerticalScrollBarEnabled = false + webView.isHorizontalScrollBarEnabled = false + webView.overScrollMode = View.OVER_SCROLL_NEVER + setContentView(webView) + + ViewCompat.setOnApplyWindowInsetsListener(webView) { view, insets -> + val systemBars = insets.getInsets(WindowInsetsCompat.Type.systemBars()) + view.setPadding(0, 0, 0, systemBars.bottom) + insets + } + ViewCompat.requestApplyInsets(webView) + + val cookieManager = CookieManager.getInstance() + cookieManager.setAcceptCookie(true) + cookieManager.setAcceptThirdPartyCookies(webView, true) + + webView.settings.apply { + javaScriptEnabled = true + domStorageEnabled = true + databaseEnabled = true + mediaPlaybackRequiresUserGesture = false + // 你的 Nginx 对静态资源设置了 immutable 长缓存，这里对 App 侧禁用缓存，确保样式更新及时生效 + cacheMode = WebSettings.LOAD_NO_CACHE + useWideViewPort = true + loadWithOverviewMode = true + setSupportZoom(false) + builtInZoomControls = false + displayZoomControls = false + if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) { + safeBrowsingEnabled = true + } + } + webView.clearCache(true) + + webView.addJavascriptInterface(ThemeBridge(), "AndroidThemeBridge") + webView.addJavascriptInterface(PdfPreviewBridge(), "AndroidPdfBridge") + webView.addJavascriptInterface(DownloadBridge(), "AndroidDownloadBridge") + + // 语音识别桥接（立即注册，前端始终可检测到；引擎在首次使用时懒初始化） + voiceBridge = VoiceBridge(this@MainActivity, webView) + webView.addJavascriptInterface(voiceBridge!!, "AndroidVoiceBridge") + + // 后台检查并下载模型 + lifecycleScope.launch(Dispatchers.IO) { + ModelManager.init(this@MainActivity) + if (!ModelManager.isModelReady(this@MainActivity)) { + Log.i(TAG, "首次启动，开始下载语音模型...") + withContext(Dispatchers.Main) { + injectVoiceStatus("downloading") + } + val success = ModelManager.downloadModels(this@MainActivity) { pct, msg -> + Log.i(TAG, "模型下载: ${pct}% - $msg") + runOnUiThread { + injectVoiceDownloadProgress(pct, msg) + } + } + if (!success) { + Log.e(TAG, "模型下载失败") + withContext(Dispatchers.Main) { + injectVoiceStatus("error") + } + return@launch + } + } + Log.i(TAG, "模型就绪") + withContext(Dispatchers.Main) { + injectVoiceStatus("ready") + // 模型就绪后立即预初始化识别引擎，用户点击时直接可用 + voiceBridge?.ensureEngine() + } + } + + // 请求录音权限 + if (ContextCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) + != PackageManager.PERMISSION_GRANTED + ) { + ActivityCompat.requestPermissions( + this, + arrayOf(Manifest.permission.RECORD_AUDIO), + REQUEST_RECORD_AUDIO + ) + } + + // Android 9 及以下需要写外部存储权限才能使用 DownloadManager + if (Build.VERSION.SDK_INT <= Build.VERSION_CODES.P && + ContextCompat.checkSelfPermission(this, Manifest.permission.WRITE_EXTERNAL_STORAGE) + != PackageManager.PERMISSION_GRANTED + ) { + ActivityCompat.requestPermissions( + this, + arrayOf(Manifest.permission.WRITE_EXTERNAL_STORAGE), + REQUEST_WRITE_STORAGE + ) + } + + webView.webViewClient = object : WebViewClient() { + override fun onPageFinished(view: WebView?, url: String?) { + super.onPageFinished(view, url) + injectThemeObserver() + injectMobileOverlayWidthPatch() + injectNoPageScrollPatch() + } + + override fun shouldOverrideUrlLoading(view: WebView?, request: WebResourceRequest?): Boolean { + val url = request?.url?.toString() ?: return false + val uri = Uri.parse(url) + // APK 下载优先：虽然是站内 host，也要交给系统下载器/浏览器 + if (isApkUrl(url)) { + startActivity(Intent(Intent.ACTION_VIEW, uri)) + return true + } + // 站内页面：WebView 内加载。用 host 精确比较，防止 example.com.evil.com 之类伪造前缀 + if ((uri.scheme == "http" || uri.scheme == "https") && uri.host == HOME_HOST) { + return false + } + // 外部链接：跳系统浏览器（或对应 app，如 mailto/tel）打开； + // 无 handler 的自定义 scheme（如 download://）直接吞掉，维持原行为 + return try { + startActivity(Intent(Intent.ACTION_VIEW, uri)) + true + } catch (e: ActivityNotFoundException) { + true + } + } + } + + webView.webChromeClient = object : WebChromeClient() { + override fun onPermissionRequest(request: PermissionRequest) { + // 按需放行媒体权限（例如文件上传触发的媒体访问） + request.grant(request.resources) + } + + override fun onShowFileChooser( + webView: WebView?, + filePathCallback: ValueCallback>?, + fileChooserParams: FileChooserParams? + ): Boolean { + this@MainActivity.filePathCallback?.onReceiveValue(null) + this@MainActivity.filePathCallback = filePathCallback + + // 对图片/视频选择，先检查并请求媒体权限；部分国产 ROM 的文件管理器会要求 + // 应用持有 READ_MEDIA_IMAGES/READ_MEDIA_VIDEO 才能展示本地媒体。 + val acceptTypes = fileChooserParams?.acceptTypes?.filter { it.isNotBlank() }?.takeIf { it.isNotEmpty() } + ?: listOf("*/*") + if (needsMediaPermission(acceptTypes) && !hasMediaPermission(acceptTypes)) { + pendingFileChooserParams = fileChooserParams + requestMediaPermissions(acceptTypes) + return true + } + + launchFileChooser(fileChooserParams) + return true + } + } + + onBackPressedDispatcher.addCallback(this, object : OnBackPressedCallback(true) { + override fun handleOnBackPressed() { + if (webView.canGoBack()) { + webView.goBack() + } else { + finish() + } + } + }) + + if (savedInstanceState == null) { + webView.loadUrl(buildHomeUrl(), mapOf( + "Cache-Control" to "no-cache, no-store, must-revalidate", + "Pragma" to "no-cache" + )) + } else { + webView.restoreState(savedInstanceState) + } + } + + override fun onSaveInstanceState(outState: Bundle) { + webView.saveState(outState) + super.onSaveInstanceState(outState) + } + + override fun onDestroy() { + voiceBridge?.destroy() + webView.destroy() + super.onDestroy() + } + + private fun needsMediaPermission(acceptTypes: List): Boolean { + // 仅当选择器明确针对图片或视频时才需要媒体权限；通配类型仍走 DocumentsUI 的临时授权 + if (acceptTypes.isEmpty() || acceptTypes.any { it == "*/*" }) return false + return acceptTypes.any { it.startsWith("image/") || it.startsWith("video/") } + } + + private fun getRequiredMediaPermissions(acceptTypes: List): Array { + val needsImage = acceptTypes.any { it.startsWith("image/") } + val needsVideo = acceptTypes.any { it.startsWith("video/") } + return when { + Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU -> { + val perms = mutableListOf() + if (needsImage) perms.add(Manifest.permission.READ_MEDIA_IMAGES) + if (needsVideo) perms.add(Manifest.permission.READ_MEDIA_VIDEO) + perms.toTypedArray() + } + Build.VERSION.SDK_INT >= Build.VERSION_CODES.M -> { + arrayOf(Manifest.permission.READ_EXTERNAL_STORAGE) + } + else -> emptyArray() + } + } + + private fun hasMediaPermission(acceptTypes: List): Boolean { + val perms = getRequiredMediaPermissions(acceptTypes) + if (perms.isEmpty()) return true + return perms.all { + ContextCompat.checkSelfPermission(this, it) == PackageManager.PERMISSION_GRANTED + } + } + + private fun requestMediaPermissions(acceptTypes: List) { + val perms = getRequiredMediaPermissions(acceptTypes) + if (perms.isEmpty()) return + ActivityCompat.requestPermissions(this, perms, REQUEST_READ_MEDIA) + } + + private fun launchFileChooser(fileChooserParams: WebChromeClient.FileChooserParams?) { + pendingFileChooserParams = null + val acceptTypes = fileChooserParams?.acceptTypes?.filter { it.isNotBlank() }?.takeIf { it.isNotEmpty() } + ?: listOf("*/*") + val allowMultiple = fileChooserParams?.mode == WebChromeClient.FileChooserParams.MODE_OPEN_MULTIPLE + val intent = Intent(Intent.ACTION_OPEN_DOCUMENT).apply { + addCategory(Intent.CATEGORY_OPENABLE) + type = acceptTypes.first() + if (allowMultiple) { + putExtra(Intent.EXTRA_ALLOW_MULTIPLE, true) + } + if (acceptTypes.size > 1) { + putExtra(Intent.EXTRA_MIME_TYPES, acceptTypes.toTypedArray()) + } + } + fileChooserLauncher.launch(intent) + } + + override fun onRequestPermissionsResult( + requestCode: Int, + permissions: Array, + grantResults: IntArray + ) { + super.onRequestPermissionsResult(requestCode, permissions, grantResults) + when (requestCode) { + REQUEST_READ_MEDIA -> { + val params = pendingFileChooserParams + if (params != null) { + if (grantResults.isNotEmpty() && grantResults.all { it == PackageManager.PERMISSION_GRANTED }) { + launchFileChooser(params) + } else { + filePathCallback?.onReceiveValue(null) + filePathCallback = null + pendingFileChooserParams = null + val permanentlyDenied = grantResults.isNotEmpty() && + permissions.isNotEmpty() && + !ActivityCompat.shouldShowRequestPermissionRationale(this, permissions[0]) + if (permanentlyDenied) { + Toast.makeText(this, "请在系统设置中开启媒体访问权限", Toast.LENGTH_LONG).show() + openAppSettings() + } else { + Toast.makeText(this, "需要媒体访问权限才能选择本地文件", Toast.LENGTH_LONG).show() + } + } + } + } + REQUEST_WRITE_STORAGE, REQUEST_RECORD_AUDIO -> { + // 启动时请求的权限，无需额外处理 + } + } + } + + private fun openAppSettings() { + try { + val intent = Intent(Settings.ACTION_APPLICATION_DETAILS_SETTINGS).apply { + data = Uri.fromParts("package", packageName, null) + } + startActivity(intent) + } catch (_: Exception) { + // ignore + } + } + + private fun applySystemBarTheme(rawTheme: String?) { + val theme = (rawTheme ?: "").lowercase(Locale.getDefault()) + val isDark = theme == "dark" + val bgColor = if (isDark) Color.BLACK else Color.WHITE + + window.statusBarColor = bgColor + window.navigationBarColor = bgColor + WindowCompat.getInsetsController(window, window.decorView)?.let { controller -> + controller.isAppearanceLightStatusBars = !isDark + controller.isAppearanceLightNavigationBars = !isDark + } + } + + private fun injectThemeObserver() { + val script = """ + (function() { + function readTheme() { + var t = document.documentElement.getAttribute('data-theme') + || document.body.getAttribute('data-theme') + || localStorage.getItem('agents_ui_theme') + || 'claude'; + return String(t).toLowerCase(); + } + function notify() { + if (window.AndroidThemeBridge && window.AndroidThemeBridge.onThemeChanged) { + window.AndroidThemeBridge.onThemeChanged(readTheme()); + } + } + notify(); + if (!window.__androidThemeObserverInstalled) { + window.__androidThemeObserverInstalled = true; + var ob = new MutationObserver(function() { notify(); }); + ob.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme'] }); + if (document.body) { + ob.observe(document.body, { attributes: true, attributeFilter: ['data-theme'] }); + } + window.addEventListener('storage', function(e){ + if (e && e.key === 'agents_ui_theme') notify(); + }); + } + })(); + """.trimIndent() + webView.evaluateJavascript(script, null) + } + + private fun injectMobileOverlayWidthPatch() { + val script = """ + (function() { + var styleId = '__android_mobile_overlay_width_patch'; + var css = [ + '.mobile-panel-sheet.mobile-panel-sheet--conversation {', + ' width: min(var(--conversation-expanded-width, 306px), 100vw) !important;', + ' max-width: 100vw !important;', + ' padding-top: 0 !important;', + '}', + '.mobile-panel-sheet--conversation .mobile-overlay-content {', + ' height: 100% !important;', + '}', + '.mobile-panel-sheet.mobile-panel-sheet--workspace {', + ' width: fit-content !important;', + ' min-width: min(420px, 60vw) !important;', + ' max-width: 100vw !important;', + '}' + ].join('\n'); + var existing = document.getElementById(styleId); + if (existing) { + existing.textContent = css; + return; + } + var style = document.createElement('style'); + style.id = styleId; + style.type = 'text/css'; + style.textContent = css; + (document.head || document.documentElement).appendChild(style); + })(); + """.trimIndent() + webView.evaluateJavascript(script, null) + } + + private fun injectNoPageScrollPatch() { + val script = """ + (function() { + var styleId = '__android_no_page_scroll_patch'; + var css = [ + 'html, body {', + ' overflow: hidden !important;', + ' height: 100% !important;', + ' max-height: 100% !important;', + '}', + 'body {', + ' position: fixed !important;', + ' width: 100% !important;', + ' inset: 0 !important;', + '}', + '* {', + ' scrollbar-width: none !important;', + '}', + '*::-webkit-scrollbar {', + ' width: 0 !important;', + ' height: 0 !important;', + ' display: none !important;', + '}', + '#app, .app-root, .chat-container {', + ' max-height: 100% !important;', + ' overflow: hidden !important;', + '}' + ].join('\n'); + var existing = document.getElementById(styleId); + if (existing) { + existing.textContent = css; + return; + } + var style = document.createElement('style'); + style.id = styleId; + style.type = 'text/css'; + style.textContent = css; + (document.head || document.documentElement).appendChild(style); + })(); + """.trimIndent() + webView.evaluateJavascript(script, null) + } + + // ═══════════════════ 语音状态通知（JS 注入） ═══════════════════ + + private fun injectVoiceStatus(status: String) { + val script = """ + (function() { + if (window.__onVoiceStatus) window.__onVoiceStatus('$status'); + if (window.dispatchEvent) { + window.dispatchEvent(new CustomEvent('voicebridge:status', { detail: '$status' })); + } + })(); + """.trimIndent() + webView.evaluateJavascript(script, null) + } + + private fun injectVoiceDownloadProgress(pct: Int, msg: String) { + val safeMsg = msg.replace("'", "\\'") + val script = """ + (function() { + if (window.__onVoiceDownloadProgress) { + window.__onVoiceDownloadProgress($pct, '$safeMsg'); + } + })(); + """.trimIndent() + webView.evaluateJavascript(script, null) + } + + inner class PdfPreviewBridge { + @JavascriptInterface + fun previewPdf(pdfUrl: String?) { + val url = pdfUrl ?: return + runOnUiThread { + val intent = Intent(this@MainActivity, PdfPreviewActivity::class.java) + intent.putExtra(PdfPreviewActivity.EXTRA_PDF_URL, url) + startActivity(intent) + } + } + + @JavascriptInterface + fun isPdfPreviewSupported(): Boolean = true + } + + inner class DownloadBridge { + @JavascriptInterface + fun downloadFile(fileUrl: String?, fileName: String?) { + val url = fileUrl ?: return + val name = fileName?.takeIf { it.isNotBlank() } + ?: deriveFileName("", null, url) + runOnUiThread { + startSystemDownload(url, name) + } + } + } + + private fun startSystemDownload(rawUrl: String, fileName: String) { + // 不再使用系统 DownloadManager（国产 ROM 上 enqueue 后经常静默失败）。 + // 改为应用内下载到私有目录，然后通过系统分享 sheet 让用户选择保存位置。 + lifecycleScope.launch(Dispatchers.IO) { + try { + val absoluteUrl = resolveAbsoluteUrl(rawUrl) + if (absoluteUrl.isBlank()) { + withContext(Dispatchers.Main) { + Toast.makeText(this@MainActivity, "下载链接无效", Toast.LENGTH_LONG).show() + } + return@launch + } + + withContext(Dispatchers.Main) { + Toast.makeText(this@MainActivity, "正在下载：$fileName", Toast.LENGTH_SHORT).show() + } + + // CookieManager 必须在主线程访问 + val cookie = withContext(Dispatchers.Main) { + CookieManager.getInstance().getCookie(absoluteUrl) + } + + val downloadedFile = downloadFileToAppDir(absoluteUrl, fileName, cookie) + + withContext(Dispatchers.Main) { + shareDownloadedFile(downloadedFile) + } + } catch (e: Exception) { + Log.e(TAG, "下载失败: ${e.message}", e) + withContext(Dispatchers.Main) { + Toast.makeText(this@MainActivity, "下载失败：${e.message}", Toast.LENGTH_LONG).show() + // 兜底：尝试用浏览器打开 + try { + val intent = Intent(Intent.ACTION_VIEW, Uri.parse(resolveAbsoluteUrl(rawUrl))) + startActivity(intent) + } catch (_: Exception) {} + } + } + } + } + + private fun downloadFileToAppDir(urlString: String, preferredName: String, cookie: String?): File { + val dir = File(filesDir, "downloads").apply { mkdirs() } + val connection = URL(urlString).openConnection() as HttpURLConnection + connection.connectTimeout = 30000 + connection.readTimeout = 30000 + connection.instanceFollowRedirects = true + connection.setRequestProperty("Cookie", cookie ?: "") + connection.connect() + + val finalUrl = connection.url.toString() + val contentDisposition = connection.getHeaderField("Content-Disposition") + val fileName = deriveFileName(preferredName, contentDisposition, finalUrl) + + val file = File(dir, fileName) + connection.inputStream.use { input -> + file.outputStream().use { output -> + input.copyTo(output) + } + } + connection.disconnect() + return file + } + + private fun shareDownloadedFile(file: File) { + try { + val uri = FileProvider.getUriForFile(this, "${packageName}.fileprovider", file) + val mimeType = URLConnection.guessContentTypeFromName(file.name) ?: "application/octet-stream" + val intent = Intent(Intent.ACTION_SEND).apply { + type = mimeType + putExtra(Intent.EXTRA_STREAM, uri) + putExtra(Intent.EXTRA_TITLE, file.name) + addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION) + } + val chooser = Intent.createChooser(intent, "分享文件").apply { + addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION) + } + startActivity(chooser) + } catch (e: ActivityNotFoundException) { + Toast.makeText(this, "没有可用的文件分享应用", Toast.LENGTH_LONG).show() + } catch (e: Exception) { + Log.e(TAG, "分享文件失败: ${e.message}", e) + Toast.makeText(this, "分享文件失败：${e.message}", Toast.LENGTH_LONG).show() + } + } + + private fun deriveFileName(preferredName: String, contentDisposition: String?, url: String): String { + contentDisposition?.let { + val regex = Regex("filename\\*?=\\s*\"?([^\";]+)\"?", RegexOption.IGNORE_CASE) + regex.find(it)?.groupValues?.get(1)?.trim()?.takeIf { name -> name.isNotBlank() }?.let { return it } + } + if (preferredName.isNotBlank() && preferredName.contains(".")) return preferredName + Uri.parse(url).lastPathSegment?.takeIf { it.isNotBlank() && it.contains(".") }?.let { return it } + Uri.parse(url).getQueryParameter("path")?.split("/")?.lastOrNull()?.takeIf { it.isNotBlank() }?.let { return it } + return "download_${System.currentTimeMillis()}" + } + + private fun resolveAbsoluteUrl(rawUrl: String): String { + if (rawUrl.startsWith("http://") || rawUrl.startsWith("https://")) return rawUrl + // API 始终位于域名根路径，不能拿当前 WebView URL（可能包含对话路径 /conv_id）作为 base， + // 否则会把 /api/download/file 拼到对话路径后面，导致 404。 + val base = HOME_URL.trimEnd('/') + return if (rawUrl.startsWith("/")) "$base$rawUrl" else "$base/$rawUrl" + } + + inner class ThemeBridge { + @JavascriptInterface + fun onThemeChanged(theme: String?) { + runOnUiThread { + applySystemBarTheme(theme) + } + } + + @JavascriptInterface + fun getAppVersionCode(): String { + return getInstalledVersionCode().toString() + } + + @JavascriptInterface + fun getAppVersionName(): String { + return getInstalledVersionName() + } + } + + private fun buildHomeUrl(): String { + val vc = getInstalledVersionCode() + val vn = Uri.encode(getInstalledVersionName()) + return "$HOME_URL/?app_shell=$WEB_ASSET_VERSION&app_vc=$vc&app_vn=$vn" + } + + private fun getInstalledVersionCode(): Long { + val pkgInfo = packageManager.getPackageInfo(packageName, 0) + return if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.P) pkgInfo.longVersionCode else @Suppress("DEPRECATION") pkgInfo.versionCode.toLong() + } + + private fun getInstalledVersionName(): String { + val pkgInfo = packageManager.getPackageInfo(packageName, 0) + return pkgInfo.versionName ?: "unknown" + } + + private fun isApkUrl(url: String): Boolean { + return url.lowercase(Locale.getDefault()).endsWith(".apk") || url.contains("/api/app/apk/latest") + } +} diff --git a/android-webview-app/app/src/main/java/com/cyjai/agent/ModelManager.kt b/android-webview-app/app/src/main/java/com/cyjai/agent/ModelManager.kt new file mode 100644 index 00000000..06ca5f2d --- /dev/null +++ b/android-webview-app/app/src/main/java/com/cyjai/agent/ModelManager.kt @@ -0,0 +1,194 @@ +package com.cyjai.agent + +import android.content.Context +import android.util.Log +import kotlinx.coroutines.* +import java.io.* +import java.net.HttpURLConnection +import java.net.URL + +/** + * 语音模型下载管理器 + * 首次启动时自动下载 SenseVoice + Silero VAD 模型到内部存储 + * + * 模型来源：HuggingFace (csukuangfj) + * 总大小：约 230MB（SenseVoice int8 228MB + tokens 308KB + Silero VAD 1.5MB） + */ +object ModelManager { + private const val TAG = "ModelManager" + + // 模型下载地址（从自有服务器下载，3 个文件约 230MB） + // 需提前上传到服务器静态目录：/static/voice_models/ + private val BASE_URL = BuildConfig.HOME_URL.trimEnd('/') + "/static/voice_models" + private val SENSEVOICE_MODEL_URL = "$BASE_URL/model.int8.onnx" + private val SENSEVOICE_TOKENS_URL = "$BASE_URL/tokens.txt" + private val SILERO_VAD_URL = "$BASE_URL/silero_vad.onnx" + + private var _modelDir: File? = null + + fun init(context: Context) { + _modelDir = File(context.filesDir, "voice_models") + } + + private fun getModelDir(context: Context): File { + return _modelDir ?: File(context.filesDir, "voice_models").also { _modelDir = it } + } + + // 模型文件预期大小（字节），用于校验下载完整性（±5% 容差） + private const val EXPECTED_MODEL_SIZE = 239_233_841L // model.int8.onnx ~228MB + private const val EXPECTED_TOKENS_SIZE = 316_000L // tokens.txt ~308KB + private const val EXPECTED_VAD_SIZE = 644_000L // silero_vad.onnx ~629KB + private const val SIZE_TOLERANCE = 0.05 // 5% 容差 + + private fun isSizeValid(file: File, expected: Long): Boolean { + if (!file.exists()) return false + val size = file.length() + val lower = (expected * (1.0 - SIZE_TOLERANCE)).toLong() + val upper = (expected * (1.0 + SIZE_TOLERANCE)).toLong() + return size in lower..upper + } + + /** + * 检查模型是否已下载完毕且文件完整 + */ + fun isModelReady(context: Context): Boolean { + val dir = getModelDir(context) + val senseVoiceModel = File(dir, "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/model.int8.onnx") + val senseVoiceTokens = File(dir, "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/tokens.txt") + val sileroVad = File(dir, "silero_vad.onnx") + return isSizeValid(senseVoiceModel, EXPECTED_MODEL_SIZE) + && isSizeValid(senseVoiceTokens, EXPECTED_TOKENS_SIZE) + && isSizeValid(sileroVad, EXPECTED_VAD_SIZE) + } + + /** + * 删除所有已下载的模型文件 + */ + fun deleteModels(context: Context): Boolean { + val dir = getModelDir(context) + return try { + dir.deleteRecursively() + } catch (e: Exception) { + Log.e(TAG, "删除模型文件失败", e) + false + } + } + + /** + * 下载所有模型（应在后台线程调用） + * @param onProgress 进度回调 (总百分比: Int, 阶段描述: String) + * @return 是否全部成功 + */ + suspend fun downloadModels( + context: Context, + onProgress: (Int, String) -> Unit = { _, _ -> } + ): Boolean = withContext(Dispatchers.IO) { + val dir = getModelDir(context) + val senseVoiceDir = File(dir, "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17") + senseVoiceDir.mkdirs() + + try { + // 1. Silero VAD (~1.5MB) + val sileroFile = File(dir, "silero_vad.onnx") + if (!sileroFile.exists()) { + onProgress(0, "下载 VAD 模型...") + downloadFile(SILERO_VAD_URL, sileroFile) { pct -> + onProgress(pct * 5 / 100, "下载 VAD 模型... ${pct}%") + } + } + + // 2. tokens.txt (~308KB) + val tokensFile = File(senseVoiceDir, "tokens.txt") + if (!tokensFile.exists()) { + onProgress(5, "下载 tokens...") + downloadFile(SENSEVOICE_TOKENS_URL, tokensFile) { pct -> + onProgress(5 + pct * 2 / 100, "下载 tokens... ${pct}%") + } + } + + // 3. SenseVoice int8 (~228MB) + val modelFile = File(senseVoiceDir, "model.int8.onnx") + if (!modelFile.exists()) { + onProgress(7, "下载 SenseVoice 模型（约 228MB）...") + downloadFile(SENSEVOICE_MODEL_URL, modelFile) { pct -> + onProgress(7 + pct * 93 / 100, "下载 SenseVoice 模型... ${pct}%") + } + } + + onProgress(100, "模型就绪") + Log.i(TAG, "模型下载完成") + true + } catch (e: Exception) { + Log.e(TAG, "模型下载失败", e) + false + } + } + + private fun downloadFile(urlStr: String, dest: File, onProgress: (Int) -> Unit) { + var attempt = 0 + val maxAttempts = 3 + + while (attempt < maxAttempts) { + try { + attempt++ + doDownload(urlStr, dest, onProgress) + return + } catch (e: Exception) { + Log.w(TAG, "下载失败 (第${attempt}次): ${dest.name}", e) + if (attempt >= maxAttempts) throw e + Thread.sleep(2000) // 重试前等待 + } + } + } + + private fun doDownload(urlStr: String, dest: File, onProgress: (Int) -> Unit) { + val url = URL(urlStr) + val conn = url.openConnection() as HttpURLConnection + conn.requestMethod = "GET" + conn.connectTimeout = 15000 + conn.readTimeout = 120000 + + // 断点续传 + var downloaded = if (dest.exists()) dest.length() else 0L + if (downloaded > 0) { + conn.setRequestProperty("Range", "bytes=$downloaded-") + } + + conn.connect() + + val totalSize = if (downloaded > 0) { + conn.getHeaderField("Content-Range")?.substringAfter("/")?.toLongOrNull() + ?: (conn.contentLengthLong + downloaded) + } else { + conn.contentLengthLong + } + + val inputStream = conn.inputStream + val outputStream = FileOutputStream(dest, downloaded > 0) + val buffer = ByteArray(65536) + var bytesRead: Int + var totalRead = downloaded + var lastProgressReport = System.currentTimeMillis() + + while (inputStream.read(buffer).also { bytesRead = it } != -1) { + outputStream.write(buffer, 0, bytesRead) + totalRead += bytesRead + + // 限流进度回调（每 200ms 最多报一次） + val now = System.currentTimeMillis() + if (totalSize > 0 && now - lastProgressReport > 200) { + val pct = (totalRead * 100 / totalSize).toInt() + onProgress(pct.coerceIn(0, 100)) + lastProgressReport = now + } + } + + if (totalSize > 0 && totalRead < totalSize) { + throw IOException("下载不完整: 期望 ${totalSize} 字节，实际 ${totalRead} 字节") + } + + inputStream.close() + outputStream.close() + conn.disconnect() + } +} diff --git a/android-webview-app/app/src/main/java/com/cyjai/agent/PdfPreviewActivity.kt b/android-webview-app/app/src/main/java/com/cyjai/agent/PdfPreviewActivity.kt new file mode 100644 index 00000000..e1f31ba4 --- /dev/null +++ b/android-webview-app/app/src/main/java/com/cyjai/agent/PdfPreviewActivity.kt @@ -0,0 +1,150 @@ +package com.cyjai.agent + +import android.net.Uri +import android.os.Bundle +import android.view.MenuItem +import android.view.View +import android.widget.LinearLayout +import android.widget.ProgressBar +import android.widget.TextView +import android.widget.Toast +import androidx.appcompat.app.AppCompatActivity +import androidx.lifecycle.lifecycleScope +import com.github.barteksc.pdfviewer.PDFView +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext +import java.io.File +import java.io.FileOutputStream +import java.net.HttpURLConnection +import java.net.URL + +class PdfPreviewActivity : AppCompatActivity() { + + companion object { + const val EXTRA_PDF_URL = "pdf_url" + private val HOME_URL: String = BuildConfig.HOME_URL + } + + private lateinit var pdfView: PDFView + private lateinit var loadingContainer: LinearLayout + private lateinit var progressBar: ProgressBar + private lateinit var statusText: TextView + + override fun onCreate(savedInstanceState: Bundle?) { + super.onCreate(savedInstanceState) + setContentView(R.layout.activity_pdf_preview) + + supportActionBar?.apply { + setDisplayHomeAsUpEnabled(true) + title = "PDF 预览" + } + + pdfView = findViewById(R.id.pdfView) + loadingContainer = findViewById(R.id.loadingContainer) + progressBar = findViewById(R.id.progressBar) + statusText = findViewById(R.id.statusText) + + val rawUrl = intent.getStringExtra(EXTRA_PDF_URL) ?: "" + if (rawUrl.isBlank()) { + showError("PDF 地址为空") + return + } + + val url = if (rawUrl.startsWith("http://") || rawUrl.startsWith("https://")) { + rawUrl + } else { + HOME_URL.trimEnd('/') + (if (rawUrl.startsWith("/")) rawUrl else "/$rawUrl") + } + + loadPdf(url) + } + + private fun showLoading(msg: String) { + loadingContainer.visibility = View.VISIBLE + progressBar.visibility = View.VISIBLE + statusText.visibility = View.VISIBLE + statusText.text = msg + pdfView.visibility = View.INVISIBLE + } + + private fun hideLoading() { + loadingContainer.visibility = View.GONE + progressBar.visibility = View.GONE + statusText.visibility = View.GONE + pdfView.visibility = View.VISIBLE + } + + private fun showError(msg: String) { + loadingContainer.visibility = View.VISIBLE + progressBar.visibility = View.GONE + statusText.visibility = View.VISIBLE + statusText.text = msg + pdfView.visibility = View.INVISIBLE + Toast.makeText(this, msg, Toast.LENGTH_LONG).show() + } + + private fun loadPdf(url: String) { + showLoading("加载中...") + lifecycleScope.launch(Dispatchers.IO) { + try { + val file = downloadToCache(url) + withContext(Dispatchers.Main) { + showPdf(file) + } + } catch (e: Exception) { + withContext(Dispatchers.Main) { + showError("PDF 加载失败: ${e.message}") + } + } + } + } + + private fun downloadToCache(url: String): File { + val conn = URL(url).openConnection() as HttpURLConnection + conn.connectTimeout = 30_000 + conn.readTimeout = 60_000 + conn.setRequestProperty("Accept", "application/pdf,*/*") + conn.connect() + + if (conn.responseCode !in 200..299) { + throw RuntimeException("HTTP ${conn.responseCode}") + } + + val pathParam = Uri.parse(url).getQueryParameter("path") + val fileName = if (!pathParam.isNullOrBlank()) { + "preview_" + pathParam.replace("/", "_") + } else { + "preview_${System.currentTimeMillis()}.pdf" + } + val file = File(cacheDir, fileName) + + FileOutputStream(file).use { out -> + conn.inputStream.use { input -> + input.copyTo(out) + } + } + conn.disconnect() + return file + } + + private fun showPdf(file: File) { + pdfView.fromFile(file) + .enableSwipe(true) + .swipeHorizontal(false) + .enableDoubletap(true) + .defaultPage(0) + .onError { showError("PDF 渲染失败") } + .onLoad { hideLoading() } + .load() + } + + override fun onOptionsItemSelected(item: MenuItem): Boolean { + return if (item.itemId == android.R.id.home) { + finish() + true + } else { + super.onOptionsItemSelected(item) + } + } +} diff --git a/android-webview-app/app/src/main/java/com/cyjai/agent/VoiceBridge.kt b/android-webview-app/app/src/main/java/com/cyjai/agent/VoiceBridge.kt new file mode 100644 index 00000000..95be6d85 --- /dev/null +++ b/android-webview-app/app/src/main/java/com/cyjai/agent/VoiceBridge.kt @@ -0,0 +1,501 @@ +package com.cyjai.agent + +import android.Manifest +import android.content.Context +import android.content.pm.PackageManager +import android.media.AudioFormat +import android.media.AudioRecord +import android.media.MediaRecorder +import android.util.Log +import android.webkit.JavascriptInterface +import android.webkit.WebView +import androidx.core.app.ActivityCompat +import com.k2fsa.sherpa.onnx.OfflineRecognizer +import com.k2fsa.sherpa.onnx.OfflineRecognizerConfig +import com.k2fsa.sherpa.onnx.OfflineSenseVoiceModelConfig +import com.k2fsa.sherpa.onnx.OfflineModelConfig +import com.k2fsa.sherpa.onnx.FeatureConfig +import com.k2fsa.sherpa.onnx.Vad +import com.k2fsa.sherpa.onnx.VadModelConfig +import com.k2fsa.sherpa.onnx.SileroVadModelConfig +import com.k2fsa.sherpa.onnx.SpeechSegment +import android.os.Environment +import kotlinx.coroutines.* +import java.io.File +import java.io.FileWriter +import java.text.SimpleDateFormat +import java.util.Date +import java.util.Locale +import java.util.concurrent.atomic.AtomicBoolean + +/** + * 语音识别桥接 — 通过 JS Bridge 暴露给 WebView + * + * 前端调用： + * window.AndroidVoiceBridge.startListening() // 开始录音 + * window.AndroidVoiceBridge.stopListening() // 停止录音 + * window.AndroidVoiceBridge.isSupported() // 是否支持 → true + * + * 结果通过全局回调传回前端： + * window.__onVoiceResult(text) // 识别结果 + * window.__onVoiceStatus(status) // 状态变化: "idle"|"listening"|"processing" + * window.__onVoiceError(error) // 错误 + */ +class VoiceBridge( + private val context: Context, + private val webView: WebView +) { + companion object { + private const val TAG = "VoiceBridge" + private const val SAMPLE_RATE = 16000 + private const val CHUNK_SIZE = 512 // 每次读取的采样数 + } + + // ── 状态 ── + private val isRecording = AtomicBoolean(false) + private var audioRecord: AudioRecord? = null + private var recordingJob: Job? = null + private val scope = CoroutineScope(Dispatchers.IO + SupervisorJob()) + private var recordingStartTime = 0L // 录音开始时间戳（用于防抖） + private val MIN_RECORDING_MS = 400L // 最小录音时长（防抖） + + // ── sherpa-onnx 引擎 ── + private var recognizer: OfflineRecognizer? = null + private var vad: Vad? = null + private var initialized = false + private var engineInitJob: Job? = null // 预初始化任务 + + // ── 调试日志 ── + private val logFile: File + get() = File(context.getExternalFilesDir(Environment.DIRECTORY_DOWNLOADS), "voice_debug.log") + private val dateFmt = SimpleDateFormat("MM-dd HH:mm:ss.SSS", Locale.getDefault()) + + private fun logToFile(msg: String) { + try { + val ts = dateFmt.format(Date()) + logFile.parentFile?.mkdirs() + FileWriter(logFile, true).use { it.write("[$ts] $msg\n") } + } catch (_: Exception) {} + } + + // ── 模型路径 ── + private val modelDir: File + get() = File(context.filesDir, "voice_models") + private val senseVoiceModel: File + get() = File(modelDir, "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/model.int8.onnx") + private val senseVoiceTokens: File + get() = File(modelDir, "sherpa-onnx-sense-voice-zh-en-ja-ko-yue-int8-2024-07-17/tokens.txt") + private val sileroVadModel: File + get() = File(modelDir, "silero_vad.onnx") + + // ═══════════════════ JS Bridge 接口 ═══════════════════ + + @JavascriptInterface + fun isSupported(): Boolean = true + + /** 模型是否已就绪（含文件大小校验） */ + @JavascriptInterface + fun isModelReady(): Boolean { + return ModelManager.isModelReady(context) + } + + /** 是否有模型文件残留（存在但不完整，需清理） */ + @JavascriptInterface + fun isModelPartial(): Boolean { + val filesExist = senseVoiceModel.exists() || senseVoiceTokens.exists() || sileroVadModel.exists() + return filesExist && !ModelManager.isModelReady(context) + } + + /** 删除已下载的模型文件 */ + @JavascriptInterface + fun deleteModel(): Boolean { + releaseEngine() + return ModelManager.deleteModels(context) + } + + /** 触发模型下载（供个人空间手动下载），下载前先释放引擎并清理旧文件 */ + @JavascriptInterface + fun downloadModel() { + scope.launch { + try { + // 先释放引擎（避免文件被 onnxruntime 锁定导致删除失败） + releaseEngine() + ModelManager.deleteModels(context) + postToJs("window.__onVoiceDownloadProgress(0, '开始下载...')") + val success = ModelManager.downloadModels(context) { pct, msg -> + postToJs("window.__onVoiceDownloadProgress($pct, '${escapeJs(msg)}')") + } + if (success) { + postToJs("window.__onVoiceDownloadProgress(100, '下载完成')") + // 下载完成后立即预初始化引擎，这样用户点击时直接可用 + ensureEngine() + } else { + postToJs("window.__onVoiceError('模型下载失败，请检查网络后重试')") + } + } catch (e: Exception) { + Log.e(TAG, "下载流程异常", e) + postToJs("window.__onVoiceError('${escapeJs(e.message ?: "未知错误")}')") + } + } + } + + /** 释放识别引擎资源 */ + private fun releaseEngine() { + initialized = false + engineInitJob?.cancel() + engineInitJob = null + try { recognizer?.release() } catch (_: Exception) {} + recognizer = null + try { vad?.release() } catch (_: Exception) {} + vad = null + } + + /** 预初始化引擎（后台，不阻塞调用者）。下载完成后或 App 启动时调用 */ + fun ensureEngine() { + if (initialized) return + if (!isModelReady()) { + logToFile("ensureEngine: 模型未就绪，跳过") + return + } + if (engineInitJob?.isActive == true) { + logToFile("ensureEngine: 已有初始化任务运行中") + return + } + logToFile("ensureEngine: 开始后台预初始化引擎") + engineInitJob = scope.launch(Dispatchers.IO) { + val ok = initEngine() + logToFile("ensureEngine: initEngine 返回 $ok") + if (ok) { + withContext(Dispatchers.Main) { + postToJs("window.__onVoiceModelReady()") + } + } + } + } + + /** 获取模型大小（用于 UI 展示） */ + @JavascriptInterface + fun getModelSizeMB(): Double { + return 228.0 + } + + /** 收集调试日志并上传到服务器 */ + @JavascriptInterface + fun debugLog(msg: String) { + logToFile("[JS] $msg") + } + + @JavascriptInterface + fun collectDebugLog(): String { + val sb = StringBuilder() + sb.appendLine("=== 语音调试日志 ===") + sb.appendLine("时间: ${dateFmt.format(Date())}") + sb.appendLine("模型目录: ${modelDir.absolutePath}") + sb.appendLine("senseVoice model: 存在=${senseVoiceModel.exists()}, 大小=${if (senseVoiceModel.exists()) senseVoiceModel.length() else -1}") + sb.appendLine("senseVoice tokens: 存在=${senseVoiceTokens.exists()}, 大小=${if (senseVoiceTokens.exists()) senseVoiceTokens.length() else -1}") + sb.appendLine("sileroVad: 存在=${sileroVadModel.exists()}, 大小=${if (sileroVadModel.exists()) sileroVadModel.length() else -1}") + sb.appendLine("isModelReady: ${ModelManager.isModelReady(context)}") + sb.appendLine("引擎已初始化: $initialized") + sb.appendLine("recognizer: ${recognizer != null}, vad: ${vad != null}") + sb.appendLine("录音权限: ${hasRecordPermission()}") + sb.appendLine("isRecording: ${isRecording.get()}") + sb.appendLine("") + sb.appendLine("=== 文件日志 ===") + if (logFile.exists()) { + try { sb.append(logFile.readText()) } catch (e: Exception) { sb.appendLine("(读取日志失败: ${e.message})") } + } else { + sb.appendLine("(无文件日志)") + } + return sb.toString() + } + + @JavascriptInterface + fun startListening() { + logToFile("startListening 被调用, initialized=$initialized") + if (!hasRecordPermission()) { + logToFile("startListening: 缺少录音权限") + postToJs("window.__onVoiceError('缺少录音权限')") + Log.e(TAG, "缺少录音权限") + return + } + + // 检查模型是否就绪 + if (!isModelReady()) { + logToFile("startListening: 模型未就绪") + postToJs("window.__onVoiceStatus('model_not_ready')") + Log.w(TAG, "模型未就绪") + return + } + + if (isRecording.get()) { + logToFile("startListening: 已在录音中，忽略") + Log.w(TAG, "已经在录音中") + return + } + + if (!initialized) { + // 触发后台初始化并等待，完成后自动开始录音 + logToFile("startListening: 引擎未初始化，触发后台初始化并等待") + postToJs("window.__onVoiceStatus('initializing')") + pendingStartJob?.cancel() + pendingStartJob = scope.launch(Dispatchers.IO) { + val ok = initEngine() + logToFile("startListening: initEngine 返回 $ok") + withContext(Dispatchers.Main) { + if (pendingStartJob == null || !pendingStartJob!!.isActive) { + logToFile("startListening: 启动请求在初始化期间被取消") + return@withContext + } + pendingStartJob = null + if (!ok) { + postToJs("window.__onVoiceError('模型初始化失败')") + return@withContext + } + startRecordingInternal() + } + } + } else { + startRecordingInternal() + } + } + + @JavascriptInterface + fun stopListening() { + logToFile("stopListening 被调用, isRecording=${isRecording.get()}") + + // 取消待处理的启动（初始化期间的取消） + pendingStartJob?.cancel() + pendingStartJob = null + + // 如果还没开始录音（初始化中），直接通知前端停止 + if (!isRecording.get()) { + logToFile("stopListening: 录音尚未开始，取消启动请求") + postToJs("window.__onVoiceStatus('idle')") + return + } + + // 防抖：录音开始后 MIN_RECORDING_MS 内不允许停止 + val elapsed = System.currentTimeMillis() - recordingStartTime + if (elapsed < MIN_RECORDING_MS) { + logToFile("stopListening: 录音时长不足 ${MIN_RECORDING_MS}ms，忽略 (已录 ${elapsed}ms)") + Log.w(TAG, "录音时长不足 ${MIN_RECORDING_MS}ms，忽略停止请求 (已录 ${elapsed}ms)") + return + } + stopRecordingInternal() + } + + // ═══════════════════ 引擎初始化 ═══════════════════ + + private suspend fun initEngine(): Boolean = withContext(Dispatchers.IO) { + try { + // 先做文件大小校验，避免用残缺文件初始化导致 Native crash + if (!ModelManager.isModelReady(context)) { + logToFile("initEngine: 模型文件不完整，拒绝初始化") + Log.e(TAG, "模型文件不完整，拒绝初始化") + return@withContext false + } + + logToFile("initEngine: 模型文件完整，开始加载 SenseVoice...") + Log.i(TAG, "初始化 SenseVoice 识别器... model=${senseVoiceModel.length()} tokens=${senseVoiceTokens.length()}") + + // SenseVoice 模型配置 + val senseVoiceConfig = OfflineSenseVoiceModelConfig( + model = senseVoiceModel.absolutePath, + useInverseTextNormalization = true + ) + + val featConfig = FeatureConfig( + sampleRate = SAMPLE_RATE, + featureDim = 80 + ) + + val modelConfig = OfflineModelConfig() + modelConfig.senseVoice = senseVoiceConfig + modelConfig.tokens = senseVoiceTokens.absolutePath + modelConfig.numThreads = 1 // 单线程降低内存压力 + modelConfig.provider = "cpu" + + val config = OfflineRecognizerConfig( + featConfig = featConfig, + modelConfig = modelConfig + ) + + Log.i(TAG, "开始创建 OfflineRecognizer...") + logToFile("initEngine: 开始创建 OfflineRecognizer (numThreads=1)...") + recognizer = OfflineRecognizer( + assetManager = null, + config = config + ) + logToFile("initEngine: OfflineRecognizer 创建完成") + Log.i(TAG, "SenseVoice 识别器初始化完成") + + // VAD 暂时禁用（Android 端 Silero VAD 存在兼容性问题，改用整段识别） + logToFile("initEngine: 跳过 VAD，使用整段识别模式") + vad = null + + initialized = true + logToFile("initEngine: 全部初始化完成") + true + } catch (e: Exception) { + logToFile("initEngine: 异常 ${e.javaClass.simpleName}: ${e.message}") + false + } + } + + // ═══════════════════ 录音 ═══════════════════ + + // ── 待处理的启动任务（用于在初始化期间取消）── + private var pendingStartJob: Job? = null + + // ── 录音数据缓冲（无 VAD 模式，整段识别）── + private val audioBuffer = mutableListOf() + + private fun startRecordingInternal() { + logToFile("startRecordingInternal: 开始, isRecording=${isRecording.get()}") + if (isRecording.getAndSet(true)) { + logToFile("startRecordingInternal: 已在录音中，跳过") + return + } + + val bufferSize = AudioRecord.getMinBufferSize( + SAMPLE_RATE, + AudioFormat.CHANNEL_IN_MONO, + AudioFormat.ENCODING_PCM_16BIT + ) + logToFile("startRecordingInternal: bufferSize=$bufferSize") + + audioRecord = AudioRecord( + MediaRecorder.AudioSource.MIC, + SAMPLE_RATE, + AudioFormat.CHANNEL_IN_MONO, + AudioFormat.ENCODING_PCM_16BIT, + bufferSize * 2 + ) + + if (audioRecord?.state != AudioRecord.STATE_INITIALIZED) { + logToFile("startRecordingInternal: AudioRecord 初始化失败 state=${audioRecord?.state}") + Log.e(TAG, "AudioRecord 初始化失败") + isRecording.set(false) + postToJs("window.__onVoiceError('麦克风初始化失败')") + return + } + + audioRecord?.startRecording() + recordingStartTime = System.currentTimeMillis() + audioBuffer.clear() + postToJs("window.__onVoiceStatus('listening')") + logToFile("startRecordingInternal: 录音已开始") + Log.i(TAG, "开始录音") + + recordingJob = scope.launch { + processAudioLoop() + } + } + + private suspend fun processAudioLoop() { + val buffer = ShortArray(CHUNK_SIZE) + while (isRecording.get()) { + val readCount = audioRecord?.read(buffer, 0, buffer.size) ?: -1 + if (readCount <= 0) continue + for (i in 0 until readCount) { + audioBuffer.add(buffer[i]) + } + } + } + + private fun recognizeFullAudio() { + logToFile("recognizeFullAudio: buffer size=${audioBuffer.size}") + if (audioBuffer.isEmpty()) { + logToFile("recognizeFullAudio: buffer 为空") + postToJs("window.__onVoiceError('未检测到语音')") + return + } + scope.launch { + postToJs("window.__onVoiceStatus('processing')") + val samples = FloatArray(audioBuffer.size) { audioBuffer[it] / 32768.0f } + logToFile("recognizeFullAudio: 开始识别 ${samples.size} 采样 (${samples.size / SAMPLE_RATE}s)") + val text = withContext(Dispatchers.IO) { recognizeSegment(samples) } + logToFile("recognizeFullAudio: 识别结果 text='$text' length=${text.length}") + if (text.isNotBlank()) { + Log.i(TAG, "识别结果: $text") + logToFile("recognizeFullAudio: 准备调用 postToJs __onVoiceResult") + postToJs("window.__onVoiceResult('${escapeJs(text)}')") + logToFile("recognizeFullAudio: postToJs __onVoiceResult 已提交") + } else { + logToFile("recognizeFullAudio: 识别结果为空") + postToJs("window.__onVoiceError('未识别到语音内容')") + } + postToJs("window.__onVoiceStatus('idle')") + } + } + + private fun recognizeSegment(samples: FloatArray): String { + val rec = recognizer ?: return "" + return try { + val stream = rec.createStream() + stream.acceptWaveform(samples, SAMPLE_RATE) + rec.decode(stream) + val result = rec.getResult(stream) + stream.release() + val text = result.text?.trim() ?: "" + logToFile("recognizeSegment: text='$text' lang='${result.lang}' emotion='${result.emotion}' event='${result.event}'") + text + } catch (e: Exception) { + logToFile("recognizeSegment: 异常 ${e.javaClass.simpleName}: ${e.message}") + Log.e(TAG, "识别错误", e) + "" + } + } + + private fun stopRecordingInternal() { + logToFile("stopRecordingInternal 被调用, isRecording=${isRecording.get()}") + if (!isRecording.getAndSet(false)) { + logToFile("stopRecordingInternal: isRecording 已为 false，跳过") + return + } + + recordingJob?.cancel() + recordingJob = null + + try { + audioRecord?.stop() + audioRecord?.release() + } catch (e: Exception) { + Log.w(TAG, "AudioRecord 释放异常", e) + logToFile("stopRecordingInternal: AudioRecord 释放异常 ${e.message}") + } + audioRecord = null + + logToFile("stopRecordingInternal: 录音已停止，共 ${audioBuffer.size} 采样") + Log.i(TAG, "录音已停止，共 ${audioBuffer.size} 采样") + recognizeFullAudio() + } + + // ═══════════════════ 工具方法 ═══════════════════ + + private fun hasRecordPermission(): Boolean { + return ActivityCompat.checkSelfPermission( + context, Manifest.permission.RECORD_AUDIO + ) == PackageManager.PERMISSION_GRANTED + } + + private fun postToJs(script: String) { + webView.post { + webView.evaluateJavascript(script, null) + } + } + + private fun escapeJs(text: String): String { + return text + .replace("\\", "\\\\") + .replace("'", "\\'") + .replace("\n", "\\n") + .replace("\r", "\\r") + } + + fun destroy() { + stopRecordingInternal() + scope.cancel() + releaseEngine() + } +} diff --git a/android-webview-app/app/src/main/res/drawable/ic_bot_icon.xml b/android-webview-app/app/src/main/res/drawable/ic_bot_icon.xml new file mode 100644 index 00000000..a3ead3e5 --- /dev/null +++ b/android-webview-app/app/src/main/res/drawable/ic_bot_icon.xml @@ -0,0 +1,67 @@ + + + + + + + + + + + + + + + + + + + + + diff --git a/android-webview-app/app/src/main/res/drawable/ic_launcher_photo.png b/android-webview-app/app/src/main/res/drawable/ic_launcher_photo.png new file mode 100644 index 00000000..19f1c7e5 Binary files /dev/null and b/android-webview-app/app/src/main/res/drawable/ic_launcher_photo.png differ diff --git a/android-webview-app/app/src/main/res/drawable/ic_launcher_photo_v2.png b/android-webview-app/app/src/main/res/drawable/ic_launcher_photo_v2.png new file mode 100644 index 00000000..19f1c7e5 Binary files /dev/null and b/android-webview-app/app/src/main/res/drawable/ic_launcher_photo_v2.png differ diff --git a/android-webview-app/app/src/main/res/layout/activity_pdf_preview.xml b/android-webview-app/app/src/main/res/layout/activity_pdf_preview.xml new file mode 100644 index 00000000..b403f5eb --- /dev/null +++ b/android-webview-app/app/src/main/res/layout/activity_pdf_preview.xml @@ -0,0 +1,34 @@ + + + + + + + + + + + + + diff --git a/android-webview-app/app/src/main/res/values/strings.xml b/android-webview-app/app/src/main/res/values/strings.xml new file mode 100644 index 00000000..fe0c6fe2 --- /dev/null +++ b/android-webview-app/app/src/main/res/values/strings.xml @@ -0,0 +1,4 @@ + + + Astrion + diff --git a/android-webview-app/app/src/main/res/values/themes.xml b/android-webview-app/app/src/main/res/values/themes.xml new file mode 100644 index 00000000..3a2592dc --- /dev/null +++ b/android-webview-app/app/src/main/res/values/themes.xml @@ -0,0 +1,6 @@ + + + + diff --git a/android-webview-app/app/src/main/res/xml/file_paths.xml b/android-webview-app/app/src/main/res/xml/file_paths.xml new file mode 100644 index 00000000..d9c45e87 --- /dev/null +++ b/android-webview-app/app/src/main/res/xml/file_paths.xml @@ -0,0 +1,6 @@ + + + + + + diff --git a/android-webview-app/app/src/main/res/xml/network_security_config.xml b/android-webview-app/app/src/main/res/xml/network_security_config.xml new file mode 100644 index 00000000..61159508 --- /dev/null +++ b/android-webview-app/app/src/main/res/xml/network_security_config.xml @@ -0,0 +1,4 @@ + + + + diff --git a/android-webview-app/build.gradle.kts b/android-webview-app/build.gradle.kts new file mode 100644 index 00000000..017d9099 --- /dev/null +++ b/android-webview-app/build.gradle.kts @@ -0,0 +1,4 @@ +plugins { + id("com.android.application") version "8.5.2" apply false + id("org.jetbrains.kotlin.android") version "1.9.24" apply false +} diff --git a/android-webview-app/gradle.properties b/android-webview-app/gradle.properties new file mode 100644 index 00000000..f0a2e55f --- /dev/null +++ b/android-webview-app/gradle.properties @@ -0,0 +1,4 @@ +org.gradle.jvmargs=-Xmx2048m -Dfile.encoding=UTF-8 +android.useAndroidX=true +kotlin.code.style=official +android.nonTransitiveRClass=true diff --git a/android-webview-app/gradle/wrapper/gradle-wrapper.jar b/android-webview-app/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 00000000..980502d1 Binary files /dev/null and b/android-webview-app/gradle/wrapper/gradle-wrapper.jar differ diff --git a/android-webview-app/gradle/wrapper/gradle-wrapper.properties b/android-webview-app/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 00000000..128196a7 --- /dev/null +++ b/android-webview-app/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,7 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-9.0-milestone-1-bin.zip +networkTimeout=10000 +validateDistributionUrl=true +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/android-webview-app/gradlew b/android-webview-app/gradlew new file mode 100755 index 00000000..faf93008 --- /dev/null +++ b/android-webview-app/gradlew @@ -0,0 +1,251 @@ +#!/bin/sh + +# +# Copyright © 2015-2021 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# SPDX-License-Identifier: Apache-2.0 +# + +############################################################################## +# +# Gradle start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh Gradle +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/HEAD/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +# This is normally unused +# shellcheck disable=SC2034 +APP_BASE_NAME=${0##*/} +# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036) +APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + if ! command -v java >/dev/null 2>&1 + then + die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + # In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + # In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked. + # shellcheck disable=SC2039,SC3045 + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Collect all arguments for the java command: +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments, +# and any embedded shellness will be escaped. +# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be +# treated as '${Hostname}' itself on the command line. + +set -- \ + "-Dorg.gradle.appname=$APP_BASE_NAME" \ + -classpath "$CLASSPATH" \ + org.gradle.wrapper.GradleWrapperMain \ + "$@" + +# Stop when "xargs" is not available. +if ! command -v xargs >/dev/null 2>&1 +then + die "xargs is not available" +fi + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/android-webview-app/gradlew.bat b/android-webview-app/gradlew.bat new file mode 100644 index 00000000..9b42019c --- /dev/null +++ b/android-webview-app/gradlew.bat @@ -0,0 +1,94 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem +@rem SPDX-License-Identifier: Apache-2.0 +@rem + +@if "%DEBUG%"=="" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%"=="" set DIRNAME=. +@rem This is normally unused +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if %ERRORLEVEL% equ 0 goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. 1>&2 +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2 +echo. 1>&2 +echo Please set the JAVA_HOME variable in your environment to match the 1>&2 +echo location of your Java installation. 1>&2 + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* + +:end +@rem End local scope for the variables with windows NT shell +if %ERRORLEVEL% equ 0 goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +set EXIT_CODE=%ERRORLEVEL% +if %EXIT_CODE% equ 0 set EXIT_CODE=1 +if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE% +exit /b %EXIT_CODE% + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/android-webview-app/settings.gradle.kts b/android-webview-app/settings.gradle.kts new file mode 100644 index 00000000..c6d5a5d8 --- /dev/null +++ b/android-webview-app/settings.gradle.kts @@ -0,0 +1,18 @@ +pluginManagement { + repositories { + google() + mavenCentral() + gradlePluginPortal() + } +} + +dependencyResolutionManagement { + repositoriesMode.set(RepositoriesMode.FAIL_ON_PROJECT_REPOS) + repositories { + google() + mavenCentral() + } +} + +rootProject.name = "AgentWebView" +include(":app")