Compare commits
22 Commits
2b3067fdcc
...
v1.1.4
| Author | SHA1 | Date | |
|---|---|---|---|
| b5fc83065c | |||
| ef31a054c0 | |||
| ff35510ef0 | |||
| 21592ae8a0 | |||
| f01547df35 | |||
| 4a2532a83b | |||
| b962265168 | |||
| 38acca6484 | |||
| 8d36ef495d | |||
| 7ac5d54a84 | |||
| ac3c7e2b4c | |||
| d8ea772c24 | |||
| 74c1db2f14 | |||
| 0bed6eba94 | |||
| 238589b82e | |||
| 904bc54248 | |||
| e5fa9429ae | |||
| 3996c4fc2f | |||
| 26595e4e2f | |||
| 16ec3295ce | |||
| 9b08975480 | |||
| 87a5bfa2c8 |
7
.gitignore
vendored
Normal file
7
.gitignore
vendored
Normal file
@@ -0,0 +1,7 @@
|
||||
.DS_Store
|
||||
bin/
|
||||
agent
|
||||
node.log
|
||||
node.pid
|
||||
config.yaml
|
||||
.DS_Store
|
||||
50
INSTALL.md
50
INSTALL.md
@@ -169,6 +169,43 @@ EOF
|
||||
|
||||
**注意:** 使用 `run.sh` 启动的好处是每次启动会自动拉取最新代码并重新编译。
|
||||
|
||||
### 3.1. 配置说明
|
||||
|
||||
**配置优先级(从高到低):**
|
||||
1. 环境变量 `BACKEND_URL`(最高优先级)
|
||||
2. 配置文件 `config.yaml` 中的 `backend.url`
|
||||
3. 默认值
|
||||
|
||||
**重要说明:**
|
||||
- 环境变量 `BACKEND_URL` 会**覆盖**配置文件中的设置
|
||||
- 即使配置文件存在,设置环境变量后也会优先使用环境变量的值
|
||||
- 这确保了编译后的二进制文件不会硬编码后端地址
|
||||
- 配置文件不会被编译进二进制文件,是运行时读取的
|
||||
|
||||
**使用环境变量(推荐):**
|
||||
```bash
|
||||
# 在 systemd 服务文件中设置
|
||||
Environment="BACKEND_URL=http://your-backend-server:8080"
|
||||
|
||||
# 或在命令行中设置
|
||||
BACKEND_URL=http://your-backend-server:8080 ./run.sh start
|
||||
```
|
||||
|
||||
**使用配置文件:**
|
||||
创建 `/opt/linkmaster-node/config.yaml`:
|
||||
```yaml
|
||||
server:
|
||||
port: 2200
|
||||
backend:
|
||||
url: http://your-backend-server:8080
|
||||
heartbeat:
|
||||
interval: 60
|
||||
log:
|
||||
file: node.log
|
||||
level: info
|
||||
debug: false
|
||||
```
|
||||
|
||||
### 4. 启动服务
|
||||
|
||||
```bash
|
||||
@@ -177,7 +214,11 @@ sudo systemctl enable linkmaster-node
|
||||
sudo systemctl start linkmaster-node
|
||||
```
|
||||
|
||||
**注意:** 确保 `BACKEND_URL` 环境变量指向后端服务器的实际地址和端口(默认 8080),不是前端地址。
|
||||
**重要说明:**
|
||||
- 确保 `BACKEND_URL` 环境变量指向后端服务器的实际地址和端口(默认 8080),不是前端地址
|
||||
- `BACKEND_URL` 环境变量会**覆盖**配置文件中的 `backend.url` 设置(优先级最高)
|
||||
- 即使配置文件存在,设置环境变量后也会优先使用环境变量的值
|
||||
- 这确保了编译后的二进制文件不会硬编码后端地址
|
||||
|
||||
## 防火墙配置
|
||||
|
||||
@@ -238,12 +279,19 @@ sudo lsof -i :2200
|
||||
|
||||
**解决:**
|
||||
- 检查后端地址是否正确(应该是 `http://backend-server:8080`,不是前端地址)
|
||||
- 检查环境变量 `BACKEND_URL` 是否设置正确(优先级最高)
|
||||
- 检查配置文件 `config.yaml` 中的 `backend.url` 是否正确
|
||||
- 检查网络连通性:`ping your-backend-server`
|
||||
- 检查端口是否开放:`telnet your-backend-server 8080` 或 `nc -zv your-backend-server 8080`
|
||||
- 检查防火墙规则(确保后端服务器的 8080 端口开放)
|
||||
- 检查后端服务是否运行:`curl http://your-backend-server:8080/api/public/nodes/online`
|
||||
- 如果使用前端代理,节点端仍需要直接连接后端,不能使用前端地址
|
||||
|
||||
**配置优先级说明:**
|
||||
- 环境变量 `BACKEND_URL` 优先级最高,会覆盖配置文件中的设置
|
||||
- 如果同时设置了环境变量和配置文件,优先使用环境变量的值
|
||||
- 这确保了编译后的二进制文件不会硬编码后端地址
|
||||
|
||||
## 卸载
|
||||
|
||||
```bash
|
||||
|
||||
5
Makefile
5
Makefile
@@ -1,4 +1,4 @@
|
||||
.PHONY: build build-linux clean
|
||||
.PHONY: build build-linux build-all clean
|
||||
|
||||
build:
|
||||
go build -o bin/linkmaster-node ./cmd/agent
|
||||
@@ -6,6 +6,9 @@ build:
|
||||
build-linux:
|
||||
GOOS=linux GOARCH=amd64 go build -o bin/linkmaster-node-linux ./cmd/agent
|
||||
|
||||
build-all:
|
||||
@./all-build.sh
|
||||
|
||||
clean:
|
||||
rm -rf bin/
|
||||
|
||||
|
||||
530
README.md
530
README.md
@@ -13,6 +13,8 @@ LinkMaster 节点服务,用于执行网络测试任务。
|
||||
- FindPing IP段批量ping检测
|
||||
- 持续 Ping/TCPing 测试
|
||||
- 心跳上报
|
||||
- 日志文件输出(支持配置日志文件路径和级别)
|
||||
- 心跳故障排查工具
|
||||
|
||||
## 安装
|
||||
|
||||
@@ -83,10 +85,24 @@ BACKEND_URL=http://your-backend-server:8080 ./run.sh start
|
||||
|
||||
## 配置
|
||||
|
||||
### 配置优先级
|
||||
|
||||
配置按以下优先级加载(高优先级会覆盖低优先级):
|
||||
|
||||
1. **环境变量**(最高优先级)
|
||||
2. **配置文件** `config.yaml`
|
||||
3. **默认值**
|
||||
|
||||
### 环境变量
|
||||
|
||||
- `BACKEND_URL`: 后端服务地址(必需,默认: http://localhost:8080)
|
||||
- `BACKEND_URL`: 后端服务地址(**优先级最高**,会覆盖配置文件中的设置)
|
||||
- `CONFIG_PATH`: 配置文件路径(可选,默认: config.yaml)
|
||||
- `LOG_FILE`: 日志文件路径(可选,默认: node.log)
|
||||
|
||||
**重要说明:**
|
||||
- `BACKEND_URL` 环境变量会**覆盖**配置文件中的 `backend.url` 设置
|
||||
- 即使配置文件存在,设置环境变量后也会优先使用环境变量的值
|
||||
- 这确保了编译后的二进制文件不会硬编码后端地址
|
||||
|
||||
### 配置文件(可选)
|
||||
|
||||
@@ -96,12 +112,29 @@ BACKEND_URL=http://your-backend-server:8080 ./run.sh start
|
||||
server:
|
||||
port: 2200
|
||||
backend:
|
||||
url: http://your-backend-server:8080
|
||||
url: http://your-backend-server:8080 # 会被 BACKEND_URL 环境变量覆盖
|
||||
heartbeat:
|
||||
interval: 60
|
||||
log:
|
||||
file: node.log # 日志文件路径(默认: node.log,空则输出到标准错误)
|
||||
level: info # 日志级别: debug, info, warn, error(默认: info)
|
||||
debug: false
|
||||
node:
|
||||
id: 0 # 节点ID(通过心跳自动获取)
|
||||
ip: "" # 节点IP(通过心跳自动获取)
|
||||
country: "" # 国家(通过心跳自动获取)
|
||||
province: "" # 省份(通过心跳自动获取)
|
||||
city: "" # 城市(通过心跳自动获取)
|
||||
isp: "" # ISP(通过心跳自动获取)
|
||||
```
|
||||
|
||||
**配置说明:**
|
||||
- `backend.url`: 后端服务地址,会被 `BACKEND_URL` 环境变量覆盖
|
||||
- `log.file`: 日志文件路径。如果为空,日志将输出到标准错误(stderr)
|
||||
- `log.level`: 日志级别,支持 `debug`、`info`、`warn`、`error`
|
||||
- `node.*`: 节点信息通过心跳自动获取并保存,无需手动配置
|
||||
- 配置文件不会被编译进二进制文件,是运行时读取的
|
||||
|
||||
## 运行脚本
|
||||
|
||||
使用 `run.sh` 脚本管理节点端。**每次启动时会自动拉取最新代码并重新编译**:
|
||||
@@ -136,6 +169,320 @@ BACKEND_URL=http://192.168.1.100:8080 ./run.sh start
|
||||
- 如果 Git 拉取失败(如网络问题),会使用当前代码继续编译
|
||||
- 如果编译失败,服务不会启动
|
||||
|
||||
## 脚本工具
|
||||
|
||||
项目提供了多个脚本工具,方便安装、卸载、运行、编译和发布。
|
||||
|
||||
### 1. install.sh - 一键安装脚本
|
||||
|
||||
自动安装 LinkMaster 节点端到系统,包括依赖安装、源码编译、systemd 服务配置等。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 通过 curl 下载并安装(推荐)
|
||||
curl -fsSL https://gitee.nas.cpolar.cn/yoyo/linkmaster-node/raw/branch/main/install.sh | bash -s -- http://your-backend-server:8080
|
||||
|
||||
# 本地运行安装脚本
|
||||
./install.sh http://your-backend-server:8080
|
||||
|
||||
# 指定分支安装
|
||||
GITHUB_BRANCH=develop curl -fsSL https://gitee.nas.cpolar.cn/yoyo/linkmaster-node/raw/branch/main/install.sh | bash -s -- http://your-backend-server:8080
|
||||
```
|
||||
|
||||
**功能特性:**
|
||||
- 自动检测系统类型和架构(Linux/macOS, amd64/arm64)
|
||||
- 自动检测并配置最快的镜像源(Ubuntu/Debian/CentOS)
|
||||
- 自动安装系统依赖(curl, wget, git, ping, traceroute 等)
|
||||
- 自动安装 Go 环境(优先使用系统包管理器,失败则从官网下载)
|
||||
- 优先从 Releases 下载预编译二进制文件,失败则从源码编译
|
||||
- 自动创建 systemd 服务并配置自启动
|
||||
- 自动配置防火墙规则(开放 2200 端口)
|
||||
- 自动登记节点到后端服务器
|
||||
- 自动启动服务并验证安装
|
||||
|
||||
**安装位置:**
|
||||
- 二进制文件:`/usr/local/bin/linkmaster-node`
|
||||
- 源码目录:`/opt/linkmaster-node`
|
||||
- 服务文件:`/etc/systemd/system/linkmaster-node.service`
|
||||
- 配置文件:`/opt/linkmaster-node/config.yaml`
|
||||
|
||||
### 2. uninstall.sh - 一键卸载脚本
|
||||
|
||||
完全卸载 LinkMaster 节点端,包括停止服务、删除文件、清理配置等。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 通过 curl 下载并运行
|
||||
curl -fsSL https://gitee.nas.cpolar.cn/yoyo/linkmaster-node/raw/branch/main/uninstall.sh | bash
|
||||
|
||||
# 本地运行卸载脚本
|
||||
./uninstall.sh
|
||||
|
||||
# 卸载并删除防火墙规则
|
||||
./uninstall.sh --remove-firewall
|
||||
```
|
||||
|
||||
**功能特性:**
|
||||
- 停止并禁用 systemd 服务
|
||||
- 删除 systemd 服务文件和配置目录
|
||||
- 删除二进制文件(`/usr/local/bin/linkmaster-node`)
|
||||
- 删除源码目录(`/opt/linkmaster-node`)
|
||||
- 清理所有残留进程
|
||||
- 重新加载 systemd daemon
|
||||
- 可选:删除防火墙规则(默认保留)
|
||||
|
||||
**注意事项:**
|
||||
- 卸载前会询问确认(交互式环境)
|
||||
- 默认保留防火墙规则,避免影响其他服务
|
||||
- 使用 `--remove-firewall` 参数可删除防火墙规则
|
||||
|
||||
### 3. run.sh - 运行管理脚本
|
||||
|
||||
用于管理节点端的启动、停止、重启、状态查看和日志查看。**每次启动时会自动拉取最新代码并重新编译**。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 启动服务(会自动拉取最新代码并编译)
|
||||
./run.sh start
|
||||
|
||||
# 停止服务
|
||||
./run.sh stop
|
||||
|
||||
# 重启服务(会拉取最新代码并重新编译)
|
||||
./run.sh restart
|
||||
|
||||
# 查看运行状态
|
||||
./run.sh status
|
||||
|
||||
# 实时查看日志
|
||||
./run.sh logs
|
||||
|
||||
# 查看完整日志
|
||||
./run.sh logs-all
|
||||
|
||||
# 显示帮助信息
|
||||
./run.sh help
|
||||
|
||||
# 指定后端地址启动
|
||||
BACKEND_URL=http://192.168.1.100:8080 ./run.sh start
|
||||
```
|
||||
|
||||
**功能特性:**
|
||||
- 启动时自动执行 `git pull` 拉取最新代码
|
||||
- 自动执行 `go mod download` 更新依赖
|
||||
- 自动编译生成新的二进制文件
|
||||
- 自动检测端口占用并提示处理
|
||||
- 后台运行并保存 PID 文件
|
||||
- 健康检查验证服务状态
|
||||
- 支持通过环境变量 `BACKEND_URL` 指定后端地址
|
||||
|
||||
**环境变量:**
|
||||
- `BACKEND_URL`: 后端服务地址(默认: `http://localhost:8080`)
|
||||
|
||||
### 4. start-systemd.sh - systemd 启动脚本
|
||||
|
||||
用于 systemd 服务启动,直接运行二进制文件。如果二进制文件不存在,会自动拉取代码并编译。
|
||||
|
||||
**使用场景:**
|
||||
- 由 systemd 服务自动调用
|
||||
- 不需要手动运行
|
||||
|
||||
**功能特性:**
|
||||
- 检查二进制文件是否存在且有效
|
||||
- 如果二进制文件不存在,自动拉取代码并编译
|
||||
- 使用 vendor 目录编译(无需网络连接)
|
||||
- 直接运行二进制文件(systemd 管理进程)
|
||||
|
||||
**注意事项:**
|
||||
- 此脚本由 systemd 服务调用,通常不需要手动运行
|
||||
- 需要确保源码目录存在且是 Git 仓库
|
||||
- 需要 Go 环境已安装并在 PATH 中
|
||||
|
||||
### 5. all-build.sh - 跨平台编译脚本
|
||||
|
||||
编译多个操作系统和架构的二进制文件,支持并行编译。**版本号自动从 `version.json` 读取**。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 编译所有平台(自动使用 version.json 中的版本号)
|
||||
./all-build.sh
|
||||
|
||||
# 只编译指定平台
|
||||
./all-build.sh -p linux/amd64
|
||||
|
||||
# 编译前清理输出目录
|
||||
./all-build.sh -c
|
||||
|
||||
# 设置并行编译数量
|
||||
./all-build.sh -j 2
|
||||
|
||||
# 覆盖版本号(覆盖 version.json 中的版本)
|
||||
./all-build.sh -v 1.0.0
|
||||
|
||||
# 只生成不带版本号的文件
|
||||
./all-build.sh -s
|
||||
|
||||
# 列出所有支持的平台
|
||||
./all-build.sh -l
|
||||
|
||||
# 显示帮助信息
|
||||
./all-build.sh -h
|
||||
```
|
||||
|
||||
**支持的平台:**
|
||||
- `linux/amd64` - Linux x86_64
|
||||
- `linux/arm64` - Linux ARM64
|
||||
- `darwin/amd64` - macOS Intel
|
||||
- `darwin/arm64` - macOS Apple Silicon
|
||||
- `windows/amd64` - Windows x86_64
|
||||
- `windows/arm64` - Windows ARM64
|
||||
|
||||
**功能特性:**
|
||||
- ✅ **自动从 `version.json` 读取版本号**(无需手动指定)
|
||||
- ✅ 支持并行编译(默认 4 个任务)
|
||||
- ✅ 自动生成带版本号和不带版本号的文件
|
||||
- ✅ 输出到 `bin/` 目录
|
||||
- ✅ 显示编译进度和结果
|
||||
- ✅ 支持清理输出目录
|
||||
|
||||
**输出文件:**
|
||||
- `bin/agent-{os}-{arch}` - 不带版本号的二进制文件
|
||||
- `bin/agent-{os}-{arch}-{version}` - 带版本号的二进制文件
|
||||
- Windows 平台会自动添加 `.exe` 扩展名
|
||||
|
||||
**版本管理:**
|
||||
版本号统一从 `version.json` 文件读取:
|
||||
```json
|
||||
{
|
||||
"version": "1.1.3",
|
||||
"tag": "v1.1.3"
|
||||
}
|
||||
```
|
||||
|
||||
### 6. all-upload-release.sh - 发布上传脚本
|
||||
|
||||
将编译好的二进制文件上传到 Releases 或通过其他方式发布。**版本号和标签自动从 `version.json` 读取,Token 已硬编码**。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 上传到 Gitea Releases(自动从 version.json 和 .git/config 读取信息)
|
||||
./all-upload-release.sh -m gitea
|
||||
|
||||
# 上传到 Gitea Releases(覆盖版本号和标签)
|
||||
./all-upload-release.sh -m gitea -t v1.2.0 -v 1.2.0
|
||||
|
||||
# 上传到 GitHub Releases
|
||||
./all-upload-release.sh -m github -r owner/repo -t v1.0.0 -v 1.0.0
|
||||
|
||||
# 通过 SCP 上传
|
||||
./all-upload-release.sh -m scp -H example.com -u user -d /path/to/release
|
||||
|
||||
# 通过 SCP 上传(指定私钥)
|
||||
./all-upload-release.sh -m scp -H example.com -u user -d /path/to/release -k ~/.ssh/id_rsa
|
||||
|
||||
# 通过 FTP 上传
|
||||
./all-upload-release.sh -m ftp -H ftp.example.com -u user -d /path/to/release
|
||||
|
||||
# 复制到本地目录
|
||||
./all-upload-release.sh -m local -d /path/to/release
|
||||
|
||||
# 只打包不上传
|
||||
./all-upload-release.sh --pack-only
|
||||
|
||||
# 不上传压缩包,直接上传二进制文件
|
||||
./all-upload-release.sh -m scp --no-pack -H example.com -u user -d /path/to/release
|
||||
|
||||
# 显示帮助信息
|
||||
./all-upload-release.sh -h
|
||||
```
|
||||
|
||||
**支持的上传方式:**
|
||||
- `gitea` - Gitea Releases(自动从 .git/config 读取仓库信息)
|
||||
- `github` - GitHub Releases(需要 GitHub CLI `gh`)
|
||||
- `scp` - 通过 SCP 上传到远程服务器
|
||||
- `ftp` - 通过 FTP 上传
|
||||
- `local` - 复制到本地目录
|
||||
|
||||
**功能特性:**
|
||||
- ✅ **自动从 `version.json` 读取版本号和标签**(无需手动指定)
|
||||
- ✅ **Token 已硬编码**(无需手动指定)
|
||||
- ✅ 自动打包二进制文件(tar.gz 或 zip)
|
||||
- ✅ 自动创建发布说明
|
||||
- ✅ 支持指定平台上传
|
||||
- ✅ 支持自定义版本号和标签(覆盖配置文件)
|
||||
- ✅ 支持自定义发布说明
|
||||
- ✅ 自动检测并处理已存在的 Release
|
||||
|
||||
**参数说明:**
|
||||
- `-m, --method`: 上传方式(gitea|github|scp|ftp|local,默认: gitea)
|
||||
- `-v, --version`: 版本号(默认: 从 version.json 读取)
|
||||
- `-t, --tag`: Git 标签(默认: 从 version.json 读取)
|
||||
- `-p, --platform`: 只上传指定平台
|
||||
- `-T, --token`: 访问令牌(已硬编码,此选项已废弃)
|
||||
- `-H, --host`: 主机地址(SCP/FTP)
|
||||
- `-u, --user`: 用户名(SCP/FTP)
|
||||
- `-d, --dest`: 目标路径(SCP/FTP/local)
|
||||
- `-k, --key`: 私钥路径(SCP)
|
||||
- `--pack-only`: 只打包不上传
|
||||
- `--no-pack`: 不上传压缩包,直接上传二进制文件
|
||||
|
||||
**版本管理:**
|
||||
版本号和标签统一从 `version.json` 文件读取:
|
||||
```json
|
||||
{
|
||||
"version": "1.1.3",
|
||||
"tag": "v1.1.3"
|
||||
}
|
||||
```
|
||||
|
||||
**典型工作流程:**
|
||||
```bash
|
||||
# 1. 编译所有平台(自动使用 version.json 中的版本号)
|
||||
./all-build.sh
|
||||
|
||||
# 2. 上传到 Gitea Releases(自动使用 version.json 中的版本号和标签)
|
||||
./all-upload-release.sh -m gitea
|
||||
```
|
||||
|
||||
### 7. vendor.sh - Vendor 依赖打包脚本
|
||||
|
||||
将项目依赖下载到 vendor 目录,客户端克隆后可直接编译,无需网络连接。
|
||||
|
||||
**使用方法:**
|
||||
|
||||
```bash
|
||||
# 运行脚本(会自动下载依赖并创建 vendor 目录)
|
||||
./vendor.sh
|
||||
```
|
||||
|
||||
**功能特性:**
|
||||
- 检查 Go 环境
|
||||
- 配置 Go 代理(使用官方源)
|
||||
- 下载所有依赖包
|
||||
- 创建 vendor 目录
|
||||
- 更新 .gitignore(允许 vendor 目录被提交)
|
||||
- 自动添加到 Git 暂存区
|
||||
|
||||
**使用场景:**
|
||||
- 项目需要离线编译能力
|
||||
- 需要确保依赖版本一致性
|
||||
- 客户端环境网络受限
|
||||
|
||||
**注意事项:**
|
||||
- 需要 Go 环境已安装
|
||||
- vendor 目录会比较大,需要提交到 Git
|
||||
- 编译时使用 `-mod=vendor` 标志
|
||||
|
||||
**编译命令(使用 vendor):**
|
||||
```bash
|
||||
go build -mod=vendor -o agent ./cmd/agent
|
||||
```
|
||||
|
||||
## API
|
||||
|
||||
### POST /api/test
|
||||
@@ -180,5 +527,180 @@ BACKEND_URL=http://192.168.1.100:8080 ./run.sh start
|
||||
### GET /api/health
|
||||
|
||||
健康检查
|
||||
# linkmaster-node
|
||||
# linkmaster-node
|
||||
|
||||
## 故障排查
|
||||
|
||||
### 心跳同步问题排查
|
||||
|
||||
如果节点无法同步心跳,可以使用排查脚本进行诊断:
|
||||
|
||||
```bash
|
||||
# 运行心跳故障排查脚本
|
||||
./check-heartbeat.sh
|
||||
```
|
||||
|
||||
排查脚本会自动检查以下项目:
|
||||
|
||||
1. **进程状态** - 检查节点进程是否正在运行
|
||||
2. **配置文件** - 检查配置文件是否存在和正确
|
||||
3. **网络连接** - 检查能否连接到后端服务器
|
||||
4. **日志分析** - 分析日志中的心跳相关错误
|
||||
5. **手动测试** - 手动发送心跳测试连接
|
||||
6. **系统资源** - 检查磁盘空间和内存使用情况
|
||||
|
||||
**常见问题及解决方案:**
|
||||
|
||||
1. **进程未运行**
|
||||
```bash
|
||||
./run.sh start
|
||||
```
|
||||
|
||||
2. **网络连接失败**
|
||||
- 检查后端服务是否正常运行
|
||||
- 检查防火墙规则(确保可以访问后端端口)
|
||||
- 检查 BACKEND_URL 配置是否正确
|
||||
|
||||
3. **心跳发送失败**
|
||||
- 查看日志: `./run.sh logs`
|
||||
- 检查后端服务日志
|
||||
- 确认后端 `/api/node/heartbeat` 接口正常
|
||||
|
||||
4. **配置文件问题**
|
||||
- 检查 `config.yaml` 文件格式是否正确
|
||||
- 确认 `BACKEND_URL` 环境变量或配置文件中的 URL 正确
|
||||
|
||||
5. **查看详细日志**
|
||||
```bash
|
||||
# 实时查看日志
|
||||
./run.sh logs
|
||||
|
||||
# 查看完整日志
|
||||
./run.sh logs-all
|
||||
```
|
||||
|
||||
### 日志功能
|
||||
|
||||
节点端支持将日志直接写入文件,便于排查问题和监控运行状态。
|
||||
|
||||
**日志配置方式:**
|
||||
|
||||
1. **环境变量**(推荐)
|
||||
```bash
|
||||
LOG_FILE=/var/log/linkmaster-node.log ./run.sh start
|
||||
```
|
||||
|
||||
2. **配置文件**
|
||||
在 `config.yaml` 中配置:
|
||||
```yaml
|
||||
log:
|
||||
file: node.log # 日志文件路径
|
||||
level: info # 日志级别: debug, info, warn, error
|
||||
```
|
||||
|
||||
3. **默认行为**
|
||||
- 默认日志文件:`node.log`(当前目录)
|
||||
- 默认日志级别:`info`
|
||||
- 如果未设置日志文件,日志输出到标准错误(stderr)
|
||||
|
||||
**日志特性:**
|
||||
- ✅ 自动创建日志文件和目录
|
||||
- ✅ 追加模式,不会覆盖已有日志
|
||||
- ✅ JSON 格式,便于日志分析
|
||||
- ✅ 包含调用信息(文件名和行号)
|
||||
- ✅ Error 级别日志包含堆栈信息
|
||||
|
||||
**查看日志:**
|
||||
```bash
|
||||
# 实时查看日志
|
||||
tail -f node.log
|
||||
|
||||
# 查看心跳相关日志
|
||||
grep -i "心跳" node.log
|
||||
|
||||
# 查看错误日志
|
||||
grep -i "error" node.log
|
||||
|
||||
# 查看最后100行
|
||||
tail -n 100 node.log
|
||||
```
|
||||
|
||||
## 心跳机制
|
||||
|
||||
节点会定期向后端发送心跳,上报节点状态和获取节点信息。
|
||||
|
||||
### 心跳请求字段
|
||||
|
||||
心跳请求包含以下字段:
|
||||
|
||||
- `type`: 固定值 `pingServer`
|
||||
- `version`: 协议版本号,固定值 `2`
|
||||
- `host_name`: 节点主机名(自动读取系统主机名)
|
||||
|
||||
### 心跳响应
|
||||
|
||||
心跳响应包含以下节点信息:
|
||||
|
||||
- `node_id`: 节点ID
|
||||
- `node_ip`: 节点外网IP
|
||||
- `country`: 国家
|
||||
- `province`: 省份
|
||||
- `city`: 城市
|
||||
- `isp`: ISP
|
||||
|
||||
这些信息会自动保存到配置文件中,用于后续的数据推送。
|
||||
|
||||
## 持续测试功能
|
||||
|
||||
节点支持持续 Ping 和 TCPing 测试,测试结果会自动推送到后端服务器。
|
||||
|
||||
### 功能特性
|
||||
|
||||
- ✅ 实时推送测试结果到后端
|
||||
- ✅ 批量推送优化(减少HTTP请求频率)
|
||||
- ✅ 自动清理超时任务
|
||||
- ✅ 资源自动清理(防止内存泄漏)
|
||||
- ✅ 详细的调试日志(debug模式)
|
||||
|
||||
### 数据推送
|
||||
|
||||
- 测试结果会自动推送到后端 `/api/public/node/continuous/result` 接口
|
||||
- 推送包含节点ID、IP、位置信息和测试结果
|
||||
- 如果后端任务不存在,节点端会自动停止对应任务
|
||||
|
||||
## 更新日志
|
||||
|
||||
### v1.1.4 (最新)
|
||||
|
||||
**新增功能:**
|
||||
- ✨ 心跳请求新增 `version` 字段(协议版本号,默认值:2)
|
||||
- ✨ 心跳请求新增 `host_name` 字段(自动读取系统主机名)
|
||||
- ✨ 支持环境变量 `BACKEND_URL` 覆盖配置文件中的后端地址
|
||||
- ✨ 持续测试功能增强,支持批量推送和自动清理
|
||||
|
||||
**改进:**
|
||||
- 🔧 修复持续测试数据推送的锁管理问题
|
||||
- 🔧 修复任务停止时未清理推送缓冲的内存泄漏问题
|
||||
- 🔧 优化配置加载逻辑,环境变量优先级最高
|
||||
- 🔧 增强日志记录,添加详细的调试信息
|
||||
- 📝 完善文档,添加配置优先级和心跳机制说明
|
||||
|
||||
### v1.1.3
|
||||
|
||||
**新增功能:**
|
||||
- ✨ 添加日志文件输出功能,支持配置日志文件路径和级别
|
||||
- ✨ 添加心跳故障排查工具 `check-heartbeat.sh`
|
||||
- ✨ 支持通过环境变量 `LOG_FILE` 设置日志文件路径
|
||||
- ✨ 日志自动创建目录,支持相对路径和绝对路径
|
||||
|
||||
**改进:**
|
||||
- 🔧 优化日志初始化逻辑,支持直接写入文件
|
||||
- 🔧 改进配置加载,支持日志配置项
|
||||
- 📝 完善文档,添加故障排查章节
|
||||
|
||||
### v1.0.0
|
||||
|
||||
- 🎉 初始版本发布
|
||||
- ✅ 支持 HTTP GET/POST 测试
|
||||
- ✅ 支持 Ping、DNS、Traceroute 等网络测试
|
||||
- ✅ 支持持续 Ping/TCPing 测试
|
||||
- ✅ 支持心跳上报
|
||||
|
||||
308
all-build.sh
Executable file
308
all-build.sh
Executable file
@@ -0,0 +1,308 @@
|
||||
#!/bin/bash
|
||||
|
||||
# 跨平台编译脚本
|
||||
# 支持编译多个操作系统和架构的版本
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色输出
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 项目信息
|
||||
PROJECT_NAME="agent"
|
||||
BUILD_DIR="bin"
|
||||
MAIN_PACKAGE="./cmd/agent"
|
||||
|
||||
# 版本配置文件路径
|
||||
VERSION_FILE="version.json"
|
||||
|
||||
# 从版本配置文件读取版本信息
|
||||
read_version_config() {
|
||||
local version_file="${VERSION_FILE}"
|
||||
|
||||
if [ ! -f "$version_file" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
# 检查是否有 jq 命令
|
||||
if command -v jq &> /dev/null; then
|
||||
local version=$(jq -r '.version' "$version_file" 2>/dev/null)
|
||||
|
||||
if [ -n "$version" ] && [ "$version" != "null" ]; then
|
||||
echo "$version"
|
||||
return 0
|
||||
fi
|
||||
else
|
||||
# 如果没有 jq,使用 grep 和 sed 解析 JSON
|
||||
local version=$(grep -o '"version"[[:space:]]*:[[:space:]]*"[^"]*"' "$version_file" 2>/dev/null | sed 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/')
|
||||
|
||||
if [ -n "$version" ]; then
|
||||
echo "$version"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
# 初始化版本号(从配置文件读取,如果失败则使用时间戳)
|
||||
VERSION_CONFIG=$(read_version_config)
|
||||
if [ $? -eq 0 ] && [ -n "$VERSION_CONFIG" ]; then
|
||||
VERSION="${VERSION:-$VERSION_CONFIG}"
|
||||
else
|
||||
VERSION="${VERSION:-$(date +%Y%m%d-%H%M%S)}"
|
||||
fi
|
||||
|
||||
# 支持的平台列表
|
||||
# 格式: OS/ARCH
|
||||
PLATFORMS=(
|
||||
"linux/amd64"
|
||||
"linux/arm64"
|
||||
"darwin/amd64"
|
||||
"darwin/arm64"
|
||||
"windows/amd64"
|
||||
"windows/arm64"
|
||||
)
|
||||
|
||||
# 显示使用说明
|
||||
usage() {
|
||||
echo -e "${BLUE}使用方法:${NC}"
|
||||
echo " $0 [选项]"
|
||||
echo ""
|
||||
echo -e "${BLUE}选项:${NC}"
|
||||
echo " -h, --help 显示帮助信息"
|
||||
echo " -p, --platform PLATFORM 只编译指定平台 (例如: linux/amd64)"
|
||||
echo " -l, --list 列出所有支持的平台"
|
||||
echo " -c, --clean 编译前清理输出目录"
|
||||
echo " -j, --jobs N 并行编译数量 (默认: 4)"
|
||||
echo " -v, --version VERSION 设置版本号 (默认: 从 version.json 读取)"
|
||||
echo " -s, --simple-only 只生成不带版本号的文件(默认生成两个)"
|
||||
echo ""
|
||||
echo -e "${BLUE}示例:${NC}"
|
||||
echo " $0 # 编译所有平台"
|
||||
echo " $0 -p linux/amd64 # 只编译 Linux AMD64"
|
||||
echo " $0 -j 2 # 使用2个并行任务"
|
||||
echo " $0 -c # 清理后编译"
|
||||
}
|
||||
|
||||
# 列出所有平台
|
||||
list_platforms() {
|
||||
echo -e "${BLUE}支持的平台:${NC}"
|
||||
for platform in "${PLATFORMS[@]}"; do
|
||||
echo " - $platform"
|
||||
done
|
||||
}
|
||||
|
||||
# 清理输出目录
|
||||
clean_build() {
|
||||
if [ -d "$BUILD_DIR" ]; then
|
||||
echo -e "${YELLOW}清理输出目录...${NC}"
|
||||
rm -rf "$BUILD_DIR"
|
||||
fi
|
||||
mkdir -p "$BUILD_DIR"
|
||||
}
|
||||
|
||||
# 编译单个平台
|
||||
build_platform() {
|
||||
local os_arch=$1
|
||||
local simple_only=$2 # 是否只生成不带版本号的文件
|
||||
local os=$(echo $os_arch | cut -d'/' -f1)
|
||||
local arch=$(echo $os_arch | cut -d'/' -f2)
|
||||
|
||||
local output_path
|
||||
if [ "$simple_only" = "true" ]; then
|
||||
# 只生成不带版本号的文件
|
||||
output_path="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
output_path="${output_path}.exe"
|
||||
fi
|
||||
else
|
||||
# 生成带版本号的文件
|
||||
output_path="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}-${VERSION}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
output_path="${output_path}.exe"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo -e "${BLUE}[编译]${NC} ${os}/${arch} -> ${output_path}"
|
||||
|
||||
if GOOS=$os GOARCH=$arch go build -ldflags "-s -w -X main.version=${VERSION}" \
|
||||
-o "$output_path" "$MAIN_PACKAGE" 2>&1; then
|
||||
echo -e "${GREEN}[成功]${NC} ${os}/${arch}"
|
||||
|
||||
# 如果不是只生成简单版本,则创建不带版本号的副本(方便使用)
|
||||
if [ "$simple_only" != "true" ]; then
|
||||
local simple_name="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
simple_name="${simple_name}.exe"
|
||||
fi
|
||||
cp "$output_path" "$simple_name" 2>/dev/null || true
|
||||
fi
|
||||
return 0
|
||||
else
|
||||
echo -e "${RED}[失败]${NC} ${os}/${arch}"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 主函数
|
||||
main() {
|
||||
local selected_platforms=()
|
||||
local clean=false
|
||||
local jobs=4
|
||||
local list_only=false
|
||||
local simple_only=false
|
||||
|
||||
# 解析参数
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
-p|--platform)
|
||||
selected_platforms+=("$2")
|
||||
shift 2
|
||||
;;
|
||||
-l|--list)
|
||||
list_only=true
|
||||
shift
|
||||
;;
|
||||
-c|--clean)
|
||||
clean=true
|
||||
shift
|
||||
;;
|
||||
-j|--jobs)
|
||||
jobs="$2"
|
||||
shift 2
|
||||
;;
|
||||
-v|--version)
|
||||
VERSION="$2"
|
||||
shift 2
|
||||
;;
|
||||
-s|--simple-only)
|
||||
simple_only=true
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}未知参数: $1${NC}"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# 如果只是列出平台,则退出
|
||||
if [ "$list_only" = true ]; then
|
||||
list_platforms
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 确定要编译的平台
|
||||
if [ ${#selected_platforms[@]} -eq 0 ]; then
|
||||
selected_platforms=("${PLATFORMS[@]}")
|
||||
else
|
||||
# 验证平台是否支持
|
||||
for platform in "${selected_platforms[@]}"; do
|
||||
local found=false
|
||||
for p in "${PLATFORMS[@]}"; do
|
||||
if [ "$p" = "$platform" ]; then
|
||||
found=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
if [ "$found" = false ]; then
|
||||
echo -e "${RED}错误: 不支持的平台 '$platform'${NC}"
|
||||
echo ""
|
||||
list_platforms
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# 清理(如果需要)
|
||||
if [ "$clean" = true ]; then
|
||||
clean_build
|
||||
else
|
||||
mkdir -p "$BUILD_DIR"
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}开始跨平台编译${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "项目名称: ${BLUE}${PROJECT_NAME}${NC}"
|
||||
echo -e "版本号: ${BLUE}${VERSION}${NC}"
|
||||
echo -e "输出目录: ${BLUE}${BUILD_DIR}${NC}"
|
||||
echo -e "并行任务数: ${BLUE}${jobs}${NC}"
|
||||
echo -e "平台数量: ${BLUE}${#selected_platforms[@]}${NC}"
|
||||
echo ""
|
||||
|
||||
# 导出函数和变量供后台任务使用
|
||||
export -f build_platform
|
||||
export PROJECT_NAME BUILD_DIR VERSION MAIN_PACKAGE RED GREEN YELLOW BLUE NC
|
||||
|
||||
# 使用后台任务进行并行编译
|
||||
local success_count=0
|
||||
local fail_count=0
|
||||
local temp_file=$(mktemp)
|
||||
|
||||
# 导出变量供后台任务使用
|
||||
export simple_only
|
||||
|
||||
# 启动编译任务
|
||||
for platform in "${selected_platforms[@]}"; do
|
||||
(
|
||||
if build_platform "$platform" "$simple_only"; then
|
||||
echo "SUCCESS $platform" >> "$temp_file"
|
||||
else
|
||||
echo "FAIL $platform" >> "$temp_file"
|
||||
fi
|
||||
) &
|
||||
|
||||
# 控制并行数量
|
||||
while [ $(jobs -r | wc -l | tr -d ' ') -ge "$jobs" ]; do
|
||||
sleep 0.1
|
||||
done
|
||||
done
|
||||
|
||||
# 等待所有后台任务完成
|
||||
wait
|
||||
|
||||
# 统计结果
|
||||
if [ -f "$temp_file" ]; then
|
||||
while IFS= read -r line; do
|
||||
if [[ $line == SUCCESS* ]]; then
|
||||
((success_count++))
|
||||
elif [[ $line == FAIL* ]]; then
|
||||
((fail_count++))
|
||||
fi
|
||||
done < "$temp_file"
|
||||
rm -f "$temp_file"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}编译完成${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "成功: ${GREEN}${success_count}${NC}"
|
||||
echo -e "失败: ${RED}${fail_count}${NC}"
|
||||
echo ""
|
||||
|
||||
if [ $fail_count -eq 0 ]; then
|
||||
echo -e "${GREEN}所有平台编译成功!${NC}"
|
||||
echo ""
|
||||
echo -e "${BLUE}编译输出文件:${NC}"
|
||||
ls -lh "$BUILD_DIR" | grep "$PROJECT_NAME" | awk '{print " " $9 " (" $5 ")"}'
|
||||
else
|
||||
echo -e "${RED}部分平台编译失败,请检查错误信息${NC}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 运行主函数
|
||||
main "$@"
|
||||
|
||||
949
all-upload-release.sh
Executable file
949
all-upload-release.sh
Executable file
@@ -0,0 +1,949 @@
|
||||
#!/bin/bash
|
||||
|
||||
# 发布上传脚本
|
||||
# 支持多种上传方式:GitHub Releases、Gitea Releases、SCP、FTP、本地复制
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色输出
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 项目信息
|
||||
PROJECT_NAME="agent"
|
||||
BUILD_DIR="bin"
|
||||
RELEASE_DIR="release"
|
||||
TEMP_DIR=$(mktemp -d)
|
||||
|
||||
|
||||
|
||||
# Gitea Token (硬编码)
|
||||
GITEA_TOKEN="3becb08eee31b422481ce1b8986de1cd645b468e"
|
||||
|
||||
# 版本配置文件路径
|
||||
VERSION_FILE="version.json"
|
||||
|
||||
# 从版本配置文件读取版本信息
|
||||
read_version_config() {
|
||||
local version_file="${VERSION_FILE}"
|
||||
|
||||
if [ ! -f "$version_file" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
# 检查是否有 jq 命令
|
||||
if command -v jq &> /dev/null; then
|
||||
local version=$(jq -r '.version' "$version_file" 2>/dev/null)
|
||||
local tag=$(jq -r '.tag' "$version_file" 2>/dev/null)
|
||||
|
||||
if [ -n "$version" ] && [ "$version" != "null" ]; then
|
||||
echo "$version|$tag"
|
||||
return 0
|
||||
fi
|
||||
else
|
||||
# 如果没有 jq,使用 grep 和 sed 解析 JSON
|
||||
local version=$(grep -o '"version"[[:space:]]*:[[:space:]]*"[^"]*"' "$version_file" 2>/dev/null | sed 's/.*"version"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/')
|
||||
local tag=$(grep -o '"tag"[[:space:]]*:[[:space:]]*"[^"]*"' "$version_file" 2>/dev/null | sed 's/.*"tag"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/')
|
||||
|
||||
if [ -n "$version" ]; then
|
||||
echo "$version|$tag"
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
# 初始化版本号(从配置文件读取,如果失败则使用时间戳)
|
||||
VERSION_CONFIG=$(read_version_config)
|
||||
if [ $? -eq 0 ] && [ -n "$VERSION_CONFIG" ]; then
|
||||
IFS='|' read -r config_version config_tag <<< "$VERSION_CONFIG"
|
||||
VERSION="${VERSION:-$config_version}"
|
||||
DEFAULT_TAG="${config_tag}"
|
||||
else
|
||||
VERSION="${VERSION:-$(date +%Y%m%d-%H%M%S)}"
|
||||
DEFAULT_TAG=""
|
||||
fi
|
||||
|
||||
# 支持的平台列表
|
||||
PLATFORMS=(
|
||||
"linux/amd64"
|
||||
"linux/arm64"
|
||||
"darwin/amd64"
|
||||
"darwin/arm64"
|
||||
"windows/amd64"
|
||||
"windows/arm64"
|
||||
)
|
||||
|
||||
# 清理函数
|
||||
cleanup() {
|
||||
if [ -d "$TEMP_DIR" ]; then
|
||||
rm -rf "$TEMP_DIR"
|
||||
fi
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# 显示使用说明
|
||||
usage() {
|
||||
echo -e "${BLUE}使用方法:${NC}"
|
||||
echo " $0 [选项]"
|
||||
echo ""
|
||||
echo -e "${BLUE}选项:${NC}"
|
||||
echo " -h, --help 显示帮助信息"
|
||||
echo " -m, --method METHOD 上传方式: github|gitea|scp|ftp|local (默认: gitea)"
|
||||
echo " -v, --version VERSION 版本号 (默认: 时间戳)"
|
||||
echo " -p, --platform PLATFORM 只上传指定平台 (例如: linux/amd64)"
|
||||
echo " -t, --tag TAG Git标签 (GitHub/Gitea Releases需要)"
|
||||
echo " -r, --repo REPO 仓库 (格式: owner/repo,默认从.git/config读取)"
|
||||
echo " -b, --base-url URL Gitea基础URL (默认从.git/config读取)"
|
||||
echo " -T, --token TOKEN 访问令牌 (已硬编码,此选项已废弃)"
|
||||
echo " -d, --dest DEST 目标路径 (SCP/FTP/local需要)"
|
||||
echo " -H, --host HOST 主机地址 (SCP/FTP需要)"
|
||||
echo " -u, --user USER 用户名 (SCP/FTP需要)"
|
||||
echo " -P, --port PORT 端口号 (SCP/FTP需要,默认: SCP=22, FTP=21)"
|
||||
echo " -k, --key KEY 私钥路径 (SCP需要)"
|
||||
echo " --pack-only 只打包不上传"
|
||||
echo " --no-pack 不上传压缩包,直接上传二进制文件"
|
||||
echo " --notes NOTES 发布说明 (GitHub/Gitea Releases)"
|
||||
echo " --notes-file FILE 从文件读取发布说明"
|
||||
echo ""
|
||||
echo -e "${BLUE}上传方式说明:${NC}"
|
||||
echo ""
|
||||
echo -e "${CYAN}Gitea Releases (自动从.git/config和version.json读取):${NC}"
|
||||
echo " $0 -m gitea"
|
||||
echo " $0 -m gitea -t v1.0.0 -v 1.0.0"
|
||||
echo ""
|
||||
echo -e "${CYAN}GitHub Releases:${NC}"
|
||||
echo " $0 -m github -r owner/repo -t v1.0.0 -v 1.0.0"
|
||||
echo ""
|
||||
echo -e "${CYAN}SCP上传:${NC}"
|
||||
echo " $0 -m scp -H example.com -u user -d /path/to/release"
|
||||
echo " $0 -m scp -H example.com -u user -d /path/to/release -k ~/.ssh/id_rsa"
|
||||
echo ""
|
||||
echo -e "${CYAN}FTP上传:${NC}"
|
||||
echo " $0 -m ftp -H ftp.example.com -u user -d /path/to/release"
|
||||
echo ""
|
||||
echo -e "${CYAN}本地复制:${NC}"
|
||||
echo " $0 -m local -d /path/to/release"
|
||||
echo ""
|
||||
echo -e "${CYAN}只打包:${NC}"
|
||||
echo " $0 --pack-only -v 1.0.0"
|
||||
}
|
||||
|
||||
# 从 .git/config 读取 Git 信息
|
||||
read_git_info() {
|
||||
local git_config=".git/config"
|
||||
|
||||
if [ ! -f "$git_config" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
# 读取 origin URL
|
||||
local url=$(git config --get remote.origin.url 2>/dev/null || echo "")
|
||||
|
||||
if [ -z "$url" ]; then
|
||||
return 1
|
||||
fi
|
||||
|
||||
# 解析 URL
|
||||
# 支持格式:
|
||||
# - https://user:pass@host/owner/repo.git
|
||||
# - https://host/owner/repo.git
|
||||
# - git@host:owner/repo.git
|
||||
# - ssh://user@host/owner/repo.git
|
||||
|
||||
local base_url=""
|
||||
local owner=""
|
||||
local repo_name=""
|
||||
local token=""
|
||||
|
||||
# 提取 token (如果有)
|
||||
if [[ "$url" =~ https://([^:]+):([^@]+)@(.+) ]]; then
|
||||
token="${BASH_REMATCH[2]}"
|
||||
url="https://${BASH_REMATCH[1]}@${BASH_REMATCH[3]}"
|
||||
fi
|
||||
|
||||
# 提取 base_url, owner, repo
|
||||
if [[ "$url" =~ https://([^/]+)/([^/]+)/([^/]+)\.git ]]; then
|
||||
base_url="https://${BASH_REMATCH[1]}"
|
||||
owner="${BASH_REMATCH[2]}"
|
||||
repo_name="${BASH_REMATCH[3]}"
|
||||
elif [[ "$url" =~ git@([^:]+):([^/]+)/([^/]+)\.git ]]; then
|
||||
base_url="https://${BASH_REMATCH[1]}"
|
||||
owner="${BASH_REMATCH[2]}"
|
||||
repo_name="${BASH_REMATCH[3]}"
|
||||
elif [[ "$url" =~ ssh://[^@]+@([^/]+)/([^/]+)/([^/]+)\.git ]]; then
|
||||
base_url="https://${BASH_REMATCH[1]}"
|
||||
owner="${BASH_REMATCH[2]}"
|
||||
repo_name="${BASH_REMATCH[3]}"
|
||||
fi
|
||||
|
||||
if [ -n "$base_url" ] && [ -n "$owner" ] && [ -n "$repo_name" ]; then
|
||||
echo "$base_url|$owner|$repo_name|$token"
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
# 检查必要的工具
|
||||
check_dependencies() {
|
||||
local method=$1
|
||||
local missing_tools=()
|
||||
|
||||
case $method in
|
||||
github)
|
||||
if ! command -v gh &> /dev/null; then
|
||||
missing_tools+=("gh (GitHub CLI)")
|
||||
fi
|
||||
;;
|
||||
gitea)
|
||||
if ! command -v curl &> /dev/null; then
|
||||
missing_tools+=("curl")
|
||||
fi
|
||||
if ! command -v jq &> /dev/null; then
|
||||
echo -e "${YELLOW}警告: jq 未安装,某些功能可能受限${NC}"
|
||||
fi
|
||||
;;
|
||||
scp)
|
||||
if ! command -v scp &> /dev/null; then
|
||||
missing_tools+=("scp")
|
||||
fi
|
||||
;;
|
||||
ftp)
|
||||
if ! command -v curl &> /dev/null && ! command -v ftp &> /dev/null; then
|
||||
missing_tools+=("curl 或 ftp")
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
if [ ${#missing_tools[@]} -gt 0 ]; then
|
||||
echo -e "${RED}错误: 缺少必要的工具:${NC}"
|
||||
for tool in "${missing_tools[@]}"; do
|
||||
echo " - $tool"
|
||||
done
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 检查构建文件是否存在
|
||||
check_build_files() {
|
||||
if [ ! -d "$BUILD_DIR" ] || [ -z "$(ls -A $BUILD_DIR 2>/dev/null)" ]; then
|
||||
echo -e "${RED}错误: 构建目录为空或不存在${NC}"
|
||||
echo "请先运行 ./all-build.sh 编译项目"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
local found=0
|
||||
for platform in "${PLATFORMS[@]}"; do
|
||||
local os=$(echo $platform | cut -d'/' -f1)
|
||||
local arch=$(echo $platform | cut -d'/' -f2)
|
||||
local file="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
file="${file}.exe"
|
||||
fi
|
||||
|
||||
if [ -f "$file" ]; then
|
||||
found=1
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $found -eq 0 ]; then
|
||||
echo -e "${RED}错误: 未找到任何构建文件${NC}"
|
||||
echo "请先运行 ./all-build.sh 编译项目"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 打包文件
|
||||
pack_files() {
|
||||
local platform=$1
|
||||
local os=$(echo $platform | cut -d'/' -f1)
|
||||
local arch=$(echo $platform | cut -d'/' -f2)
|
||||
local binary="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
|
||||
if [ "$os" = "windows" ]; then
|
||||
binary="${binary}.exe"
|
||||
fi
|
||||
|
||||
if [ ! -f "$binary" ]; then
|
||||
echo -e "${YELLOW}[跳过]${NC} ${platform} - 文件不存在"
|
||||
return 1
|
||||
fi
|
||||
|
||||
local pack_name="${PROJECT_NAME}-${os}-${arch}-${VERSION}"
|
||||
local pack_file
|
||||
|
||||
if [ "$os" = "windows" ]; then
|
||||
pack_file="${TEMP_DIR}/${pack_name}.zip"
|
||||
echo -e "${BLUE}[打包]${NC} ${platform} -> ${pack_name}.zip"
|
||||
(cd "$BUILD_DIR" && zip -q "${pack_file}" "$(basename $binary)")
|
||||
else
|
||||
pack_file="${TEMP_DIR}/${pack_name}.tar.gz"
|
||||
echo -e "${BLUE}[打包]${NC} ${platform} -> ${pack_name}.tar.gz"
|
||||
tar -czf "$pack_file" -C "$BUILD_DIR" "$(basename $binary)"
|
||||
fi
|
||||
|
||||
echo "$pack_file"
|
||||
return 0
|
||||
}
|
||||
|
||||
# 创建发布说明
|
||||
create_release_notes() {
|
||||
local notes_file="${TEMP_DIR}/release_notes.md"
|
||||
cat > "$notes_file" <<EOF
|
||||
# ${PROJECT_NAME} ${VERSION}
|
||||
|
||||
## 版本信息
|
||||
- **版本号**: ${VERSION}
|
||||
- **发布日期**: $(date '+%Y-%m-%d %H:%M:%S')
|
||||
- **Git提交**: $(git rev-parse --short HEAD 2>/dev/null || echo "N/A")
|
||||
- **Git分支**: $(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo "N/A")
|
||||
|
||||
## 支持的平台
|
||||
EOF
|
||||
|
||||
for platform in "${PLATFORMS[@]}"; do
|
||||
local os=$(echo $platform | cut -d'/' -f1)
|
||||
local arch=$(echo $platform | cut -d'/' -f2)
|
||||
local binary="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
binary="${binary}.exe"
|
||||
fi
|
||||
|
||||
if [ -f "$binary" ]; then
|
||||
local size=$(ls -lh "$binary" | awk '{print $5}')
|
||||
echo "- ${os}/${arch} (${size})" >> "$notes_file"
|
||||
fi
|
||||
done
|
||||
|
||||
echo "" >> "$notes_file"
|
||||
echo "## 安装说明" >> "$notes_file"
|
||||
echo "" >> "$notes_file"
|
||||
echo "### Linux/macOS" >> "$notes_file"
|
||||
echo "\`\`\`bash" >> "$notes_file"
|
||||
echo "tar -xzf ${PROJECT_NAME}-linux-amd64-${VERSION}.tar.gz" >> "$notes_file"
|
||||
echo "chmod +x ${PROJECT_NAME}-linux-amd64" >> "$notes_file"
|
||||
echo "./${PROJECT_NAME}-linux-amd64" >> "$notes_file"
|
||||
echo "\`\`\`" >> "$notes_file"
|
||||
echo "" >> "$notes_file"
|
||||
echo "### Windows" >> "$notes_file"
|
||||
echo "\`\`\`powershell" >> "$notes_file"
|
||||
echo "Expand-Archive ${PROJECT_NAME}-windows-amd64-${VERSION}.zip" >> "$notes_file"
|
||||
echo ".\\${PROJECT_NAME}-windows-amd64.exe" >> "$notes_file"
|
||||
echo "\`\`\`" >> "$notes_file"
|
||||
|
||||
echo "$notes_file"
|
||||
}
|
||||
|
||||
# GitHub Releases 上传
|
||||
upload_github() {
|
||||
local repo=$1
|
||||
local tag=$2
|
||||
local notes_file=$3
|
||||
|
||||
if [ -z "$repo" ]; then
|
||||
echo -e "${RED}错误: GitHub仓库未指定,使用 -r owner/repo${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$tag" ]; then
|
||||
echo -e "${RED}错误: Git标签未指定,使用 -t TAG${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}上传到 GitHub Releases${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "仓库: ${BLUE}${repo}${NC}"
|
||||
echo -e "标签: ${BLUE}${tag}${NC}"
|
||||
echo -e "版本: ${BLUE}${VERSION}${NC}"
|
||||
echo ""
|
||||
|
||||
# 检查是否已存在release
|
||||
if gh release view "$tag" --repo "$repo" &>/dev/null; then
|
||||
echo -e "${YELLOW}警告: Release ${tag} 已存在${NC}"
|
||||
read -p "是否删除并重新创建? (y/N): " confirm
|
||||
if [[ "$confirm" =~ ^[Yy]$ ]]; then
|
||||
gh release delete "$tag" --repo "$repo" --yes
|
||||
else
|
||||
echo "取消上传"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# 创建release
|
||||
echo -e "${BLUE}[创建Release]${NC} ${tag}"
|
||||
if [ -f "$notes_file" ]; then
|
||||
gh release create "$tag" \
|
||||
--repo "$repo" \
|
||||
--title "${PROJECT_NAME} ${VERSION}" \
|
||||
--notes-file "$notes_file" \
|
||||
--draft=false \
|
||||
--prerelease=false
|
||||
else
|
||||
gh release create "$tag" \
|
||||
--repo "$repo" \
|
||||
--title "${PROJECT_NAME} ${VERSION}" \
|
||||
--notes "Release ${VERSION}" \
|
||||
--draft=false \
|
||||
--prerelease=false
|
||||
fi
|
||||
|
||||
# 上传文件
|
||||
local upload_count=0
|
||||
shopt -s nullglob
|
||||
for file in "${TEMP_DIR}"/*.tar.gz "${TEMP_DIR}"/*.zip; do
|
||||
if [ -f "$file" ]; then
|
||||
echo -e "${BLUE}[上传]${NC} $(basename $file)"
|
||||
gh release upload "$tag" "$file" --repo "$repo" --clobber
|
||||
((upload_count++))
|
||||
fi
|
||||
done
|
||||
shopt -u nullglob
|
||||
|
||||
if [ $upload_count -eq 0 ] && [ "$NO_PACK" != "true" ]; then
|
||||
echo -e "${YELLOW}警告: 没有找到要上传的文件${NC}"
|
||||
else
|
||||
echo -e "${GREEN}[完成]${NC} 已上传 ${upload_count} 个文件"
|
||||
echo -e "${CYAN}Release链接:${NC} https://github.com/${repo}/releases/tag/${tag}"
|
||||
fi
|
||||
}
|
||||
|
||||
# Gitea Releases 上传
|
||||
upload_gitea() {
|
||||
local base_url=$1
|
||||
local owner=$2
|
||||
local repo=$3
|
||||
local tag=$4
|
||||
local token=$5
|
||||
local notes_file=$6
|
||||
|
||||
if [ -z "$base_url" ] || [ -z "$owner" ] || [ -z "$repo" ]; then
|
||||
echo -e "${RED}错误: Gitea仓库信息不完整${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$tag" ]; then
|
||||
echo -e "${RED}错误: Git标签未指定,使用 -t TAG${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Token 已硬编码,确保使用硬编码的 token
|
||||
if [ -z "$token" ]; then
|
||||
token="${GITEA_TOKEN}"
|
||||
fi
|
||||
|
||||
if [ -z "$token" ]; then
|
||||
echo -e "${RED}错误: 访问令牌未配置${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}上传到 Gitea Releases${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "基础URL: ${BLUE}${base_url}${NC}"
|
||||
echo -e "仓库: ${BLUE}${owner}/${repo}${NC}"
|
||||
echo -e "标签: ${BLUE}${tag}${NC}"
|
||||
echo -e "版本: ${BLUE}${VERSION}${NC}"
|
||||
echo ""
|
||||
|
||||
local api_base="${base_url}/api/v1"
|
||||
local repo_api="${api_base}/repos/${owner}/${repo}"
|
||||
|
||||
# 读取发布说明
|
||||
local notes="Release ${VERSION}"
|
||||
if [ -f "$notes_file" ]; then
|
||||
notes=$(cat "$notes_file")
|
||||
fi
|
||||
|
||||
# 转义 JSON 字符串(不使用 jq)
|
||||
local notes_escaped=$(echo "$notes" | sed 's/\\/\\\\/g' | sed 's/"/\\"/g' | sed ':a;N;$!ba;s/\n/\\n/g')
|
||||
|
||||
# 检查是否已存在release
|
||||
local existing_release=$(curl -s -X GET \
|
||||
"${repo_api}/releases/tags/${tag}" \
|
||||
-H "Authorization: token ${token}" \
|
||||
-H "Content-Type: application/json" 2>/dev/null)
|
||||
|
||||
if echo "$existing_release" | grep -q '"id"'; then
|
||||
echo -e "${YELLOW}警告: Release ${tag} 已存在${NC}"
|
||||
read -p "是否删除并重新创建? (y/N): " confirm
|
||||
if [[ "$confirm" =~ ^[Yy]$ ]]; then
|
||||
local release_id=$(echo "$existing_release" | grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2)
|
||||
if [ -n "$release_id" ]; then
|
||||
curl -s -X DELETE \
|
||||
"${repo_api}/releases/${release_id}" \
|
||||
-H "Authorization: token ${token}" > /dev/null
|
||||
echo -e "${BLUE}[删除]${NC} 已删除现有 Release"
|
||||
fi
|
||||
else
|
||||
echo "取消上传"
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
# 创建release
|
||||
echo -e "${BLUE}[创建Release]${NC} ${tag}"
|
||||
local release_data=$(cat <<EOF
|
||||
{
|
||||
"tag_name": "${tag}",
|
||||
"target_commitish": "main",
|
||||
"name": "${PROJECT_NAME} ${VERSION}",
|
||||
"body": "${notes_escaped}",
|
||||
"draft": false,
|
||||
"prerelease": false
|
||||
}
|
||||
EOF
|
||||
)
|
||||
|
||||
local create_response=$(curl -s -X POST \
|
||||
"${repo_api}/releases" \
|
||||
-H "Authorization: token ${token}" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d "$release_data")
|
||||
|
||||
if echo "$create_response" | grep -q '"id"'; then
|
||||
local release_id=$(echo "$create_response" | grep -o '"id":[0-9]*' | head -1 | cut -d':' -f2)
|
||||
echo -e "${GREEN}[成功]${NC} Release 已创建 (ID: ${release_id})"
|
||||
else
|
||||
echo -e "${RED}[失败]${NC} 创建 Release 失败"
|
||||
echo "$create_response" | head -20
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 上传文件
|
||||
local upload_count=0
|
||||
shopt -s nullglob
|
||||
for file in "${TEMP_DIR}"/*.tar.gz "${TEMP_DIR}"/*.zip; do
|
||||
if [ -f "$file" ]; then
|
||||
echo -e "${BLUE}[上传]${NC} $(basename $file)"
|
||||
local upload_response=$(curl -s -X POST \
|
||||
"${repo_api}/releases/${release_id}/assets?name=$(basename $file)" \
|
||||
-H "Authorization: token ${token}" \
|
||||
-H "Content-Type: application/octet-stream" \
|
||||
--data-binary "@${file}")
|
||||
|
||||
if echo "$upload_response" | grep -q '"id"'; then
|
||||
echo -e " ${GREEN}✓${NC} 上传成功"
|
||||
((upload_count++))
|
||||
else
|
||||
echo -e " ${RED}✗${NC} 上传失败"
|
||||
echo "$upload_response" | head -5
|
||||
fi
|
||||
fi
|
||||
done
|
||||
shopt -u nullglob
|
||||
|
||||
if [ $upload_count -eq 0 ] && [ "$NO_PACK" != "true" ]; then
|
||||
echo -e "${YELLOW}警告: 没有找到要上传的文件${NC}"
|
||||
else
|
||||
echo -e "${GREEN}[完成]${NC} 已上传 ${upload_count} 个文件"
|
||||
echo -e "${CYAN}Release链接:${NC} ${base_url}/${owner}/${repo}/releases/tag/${tag}"
|
||||
fi
|
||||
}
|
||||
|
||||
# SCP上传
|
||||
upload_scp() {
|
||||
local host=$1
|
||||
local user=$2
|
||||
local dest=$3
|
||||
local port=${4:-22}
|
||||
local key=$5
|
||||
|
||||
if [ -z "$host" ] || [ -z "$user" ] || [ -z "$dest" ]; then
|
||||
echo -e "${RED}错误: SCP需要指定主机(-H)、用户(-u)和目标路径(-d)${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}上传到 SCP${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "主机: ${BLUE}${user}@${host}:${port}${NC}"
|
||||
echo -e "目标: ${BLUE}${dest}${NC}"
|
||||
echo ""
|
||||
|
||||
local scp_opts="-P $port"
|
||||
if [ -n "$key" ]; then
|
||||
scp_opts="$scp_opts -i $key"
|
||||
fi
|
||||
|
||||
# 创建远程目录
|
||||
local ssh_cmd="ssh"
|
||||
if [ -n "$key" ]; then
|
||||
ssh_cmd="$ssh_cmd -i $key"
|
||||
fi
|
||||
ssh -p "$port" $([ -n "$key" ] && echo "-i $key") "$user@$host" "mkdir -p $dest" || true
|
||||
|
||||
# 上传文件
|
||||
local upload_count=0
|
||||
if [ "$NO_PACK" = "true" ]; then
|
||||
# 直接上传二进制文件
|
||||
for platform in "${SELECTED_PLATFORMS[@]}"; do
|
||||
local os=$(echo $platform | cut -d'/' -f1)
|
||||
local arch=$(echo $platform | cut -d'/' -f2)
|
||||
local binary="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
binary="${binary}.exe"
|
||||
fi
|
||||
|
||||
if [ -f "$binary" ]; then
|
||||
echo -e "${BLUE}[上传]${NC} $(basename $binary)"
|
||||
scp $scp_opts "$binary" "$user@$host:$dest/"
|
||||
((upload_count++))
|
||||
fi
|
||||
done
|
||||
else
|
||||
# 上传压缩包
|
||||
shopt -s nullglob
|
||||
for file in "${TEMP_DIR}"/*.tar.gz "${TEMP_DIR}"/*.zip; do
|
||||
if [ -f "$file" ]; then
|
||||
echo -e "${BLUE}[上传]${NC} $(basename $file)"
|
||||
scp $scp_opts "$file" "$user@$host:$dest/"
|
||||
((upload_count++))
|
||||
fi
|
||||
done
|
||||
shopt -u nullglob
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}[完成]${NC} 已上传 ${upload_count} 个文件"
|
||||
}
|
||||
|
||||
# FTP上传
|
||||
upload_ftp() {
|
||||
local host=$1
|
||||
local user=$2
|
||||
local dest=$3
|
||||
local port=${4:-21}
|
||||
|
||||
if [ -z "$host" ] || [ -z "$user" ] || [ -z "$dest" ]; then
|
||||
echo -e "${RED}错误: FTP需要指定主机(-H)、用户(-u)和目标路径(-d)${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}上传到 FTP${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "主机: ${BLUE}${host}:${port}${NC}"
|
||||
echo -e "用户: ${BLUE}${user}${NC}"
|
||||
echo -e "目标: ${BLUE}${dest}${NC}"
|
||||
echo ""
|
||||
|
||||
read -sp "请输入FTP密码: " ftp_pass
|
||||
echo ""
|
||||
|
||||
local upload_count=0
|
||||
|
||||
if [ "$NO_PACK" = "true" ]; then
|
||||
# 直接上传二进制文件
|
||||
for platform in "${SELECTED_PLATFORMS[@]}"; do
|
||||
local os=$(echo $platform | cut -d'/' -f1)
|
||||
local arch=$(echo $platform | cut -d'/' -f2)
|
||||
local binary="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
binary="${binary}.exe"
|
||||
fi
|
||||
|
||||
if [ -f "$binary" ]; then
|
||||
echo -e "${BLUE}[上传]${NC} $(basename $binary)"
|
||||
curl -T "$binary" \
|
||||
"ftp://${host}:${port}${dest}/$(basename $binary)" \
|
||||
--user "${user}:${ftp_pass}" \
|
||||
--ftp-create-dirs
|
||||
((upload_count++))
|
||||
fi
|
||||
done
|
||||
else
|
||||
# 上传压缩包
|
||||
shopt -s nullglob
|
||||
for file in "${TEMP_DIR}"/*.tar.gz "${TEMP_DIR}"/*.zip; do
|
||||
if [ -f "$file" ]; then
|
||||
echo -e "${BLUE}[上传]${NC} $(basename $file)"
|
||||
curl -T "$file" \
|
||||
"ftp://${host}:${port}${dest}/$(basename $file)" \
|
||||
--user "${user}:${ftp_pass}" \
|
||||
--ftp-create-dirs
|
||||
((upload_count++))
|
||||
fi
|
||||
done
|
||||
shopt -u nullglob
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}[完成]${NC} 已上传 ${upload_count} 个文件"
|
||||
}
|
||||
|
||||
# 本地复制
|
||||
upload_local() {
|
||||
local dest=$1
|
||||
|
||||
if [ -z "$dest" ]; then
|
||||
echo -e "${RED}错误: 本地复制需要指定目标路径(-d)${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}复制到本地目录${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "目标: ${BLUE}${dest}${NC}"
|
||||
echo ""
|
||||
|
||||
mkdir -p "$dest"
|
||||
|
||||
local copy_count=0
|
||||
|
||||
if [ "$NO_PACK" = "true" ]; then
|
||||
# 直接复制二进制文件
|
||||
for platform in "${SELECTED_PLATFORMS[@]}"; do
|
||||
local os=$(echo $platform | cut -d'/' -f1)
|
||||
local arch=$(echo $platform | cut -d'/' -f2)
|
||||
local binary="${BUILD_DIR}/${PROJECT_NAME}-${os}-${arch}"
|
||||
if [ "$os" = "windows" ]; then
|
||||
binary="${binary}.exe"
|
||||
fi
|
||||
|
||||
if [ -f "$binary" ]; then
|
||||
echo -e "${BLUE}[复制]${NC} $(basename $binary)"
|
||||
cp "$binary" "$dest/"
|
||||
((copy_count++))
|
||||
fi
|
||||
done
|
||||
else
|
||||
# 复制压缩包
|
||||
shopt -s nullglob
|
||||
for file in "${TEMP_DIR}"/*.tar.gz "${TEMP_DIR}"/*.zip; do
|
||||
if [ -f "$file" ]; then
|
||||
echo -e "${BLUE}[复制]${NC} $(basename $file)"
|
||||
cp "$file" "$dest/"
|
||||
((copy_count++))
|
||||
fi
|
||||
done
|
||||
shopt -u nullglob
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}[完成]${NC} 已复制 ${copy_count} 个文件"
|
||||
}
|
||||
|
||||
# 主函数
|
||||
main() {
|
||||
local method="gitea"
|
||||
local selected_platforms=()
|
||||
local tag="${DEFAULT_TAG}"
|
||||
local repo=""
|
||||
local base_url=""
|
||||
local token="${GITEA_TOKEN}" # 使用硬编码的 token
|
||||
local dest=""
|
||||
local host=""
|
||||
local user=""
|
||||
local port=""
|
||||
local key=""
|
||||
local pack_only=false
|
||||
local notes=""
|
||||
local notes_file=""
|
||||
|
||||
# 解析参数
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case $1 in
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
-m|--method)
|
||||
method="$2"
|
||||
shift 2
|
||||
;;
|
||||
-v|--version)
|
||||
VERSION="$2"
|
||||
shift 2
|
||||
;;
|
||||
-p|--platform)
|
||||
selected_platforms+=("$2")
|
||||
shift 2
|
||||
;;
|
||||
-t|--tag)
|
||||
tag="$2"
|
||||
shift 2
|
||||
;;
|
||||
-r|--repo)
|
||||
repo="$2"
|
||||
shift 2
|
||||
;;
|
||||
-b|--base-url)
|
||||
base_url="$2"
|
||||
shift 2
|
||||
;;
|
||||
-T|--token)
|
||||
echo -e "${YELLOW}警告: Token 已硬编码,-T 参数将被忽略${NC}"
|
||||
shift 2
|
||||
;;
|
||||
-d|--dest)
|
||||
dest="$2"
|
||||
shift 2
|
||||
;;
|
||||
-H|--host)
|
||||
host="$2"
|
||||
shift 2
|
||||
;;
|
||||
-u|--user)
|
||||
user="$2"
|
||||
shift 2
|
||||
;;
|
||||
-P|--port)
|
||||
port="$2"
|
||||
shift 2
|
||||
;;
|
||||
-k|--key)
|
||||
key="$2"
|
||||
shift 2
|
||||
;;
|
||||
--pack-only)
|
||||
pack_only=true
|
||||
shift
|
||||
;;
|
||||
--no-pack)
|
||||
NO_PACK=true
|
||||
shift
|
||||
;;
|
||||
--notes)
|
||||
notes="$2"
|
||||
shift 2
|
||||
;;
|
||||
--notes-file)
|
||||
notes_file="$2"
|
||||
shift 2
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}未知参数: $1${NC}"
|
||||
usage
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# 确定要处理的平台
|
||||
if [ ${#selected_platforms[@]} -eq 0 ]; then
|
||||
SELECTED_PLATFORMS=("${PLATFORMS[@]}")
|
||||
else
|
||||
SELECTED_PLATFORMS=("${selected_platforms[@]}")
|
||||
fi
|
||||
|
||||
# 检查构建文件
|
||||
check_build_files
|
||||
|
||||
# 检查依赖
|
||||
if [ "$pack_only" != "true" ]; then
|
||||
check_dependencies "$method"
|
||||
fi
|
||||
|
||||
# 打包文件
|
||||
if [ "$NO_PACK" != "true" ]; then
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}开始打包${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "版本: ${BLUE}${VERSION}${NC}"
|
||||
echo -e "平台数量: ${BLUE}${#SELECTED_PLATFORMS[@]}${NC}"
|
||||
echo ""
|
||||
|
||||
for platform in "${SELECTED_PLATFORMS[@]}"; do
|
||||
pack_files "$platform" > /dev/null
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo -e "${GREEN}打包完成${NC}"
|
||||
echo -e "${BLUE}打包文件:${NC}"
|
||||
shopt -s nullglob
|
||||
ls -lh "${TEMP_DIR}"/*.tar.gz "${TEMP_DIR}"/*.zip 2>/dev/null | awk '{print " " $9 " (" $5 ")"}'
|
||||
shopt -u nullglob
|
||||
echo ""
|
||||
fi
|
||||
|
||||
# 如果只是打包,则复制到release目录
|
||||
if [ "$pack_only" = "true" ]; then
|
||||
mkdir -p "$RELEASE_DIR"
|
||||
shopt -s nullglob
|
||||
cp "${TEMP_DIR}"/*.tar.gz "${TEMP_DIR}"/*.zip "$RELEASE_DIR/" 2>/dev/null || true
|
||||
shopt -u nullglob
|
||||
echo -e "${GREEN}文件已复制到 ${RELEASE_DIR} 目录${NC}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 创建发布说明
|
||||
local release_notes=""
|
||||
if [ -n "$notes_file" ] && [ -f "$notes_file" ]; then
|
||||
release_notes="$notes_file"
|
||||
elif [ -n "$notes" ]; then
|
||||
echo "$notes" > "${TEMP_DIR}/release_notes.md"
|
||||
release_notes="${TEMP_DIR}/release_notes.md"
|
||||
elif [ "$method" = "github" ] || [ "$method" = "gitea" ]; then
|
||||
release_notes=$(create_release_notes)
|
||||
fi
|
||||
|
||||
# 根据方法上传
|
||||
case $method in
|
||||
gitea)
|
||||
# 从 .git/config 读取信息(如果未指定)
|
||||
local git_info=""
|
||||
if [ -z "$base_url" ] || [ -z "$repo" ]; then
|
||||
git_info=$(read_git_info)
|
||||
if [ $? -eq 0 ] && [ -n "$git_info" ]; then
|
||||
IFS='|' read -r git_base_url git_owner git_repo_name git_token <<< "$git_info"
|
||||
if [ -z "$base_url" ]; then
|
||||
base_url="$git_base_url"
|
||||
fi
|
||||
if [ -z "$repo" ]; then
|
||||
repo="${git_owner}/${git_repo_name}"
|
||||
fi
|
||||
# Token 已硬编码,不从 git config 读取
|
||||
# if [ -z "$token" ] && [ -n "$git_token" ]; then
|
||||
# token="$git_token"
|
||||
# fi
|
||||
echo -e "${CYAN}[信息]${NC} 从 .git/config 读取仓库信息: ${repo}"
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ -z "$base_url" ] || [ -z "$repo" ]; then
|
||||
echo -e "${RED}错误: Gitea仓库信息不完整${NC}"
|
||||
echo "请指定 -b BASE_URL 和 -r owner/repo,或确保 .git/config 中有正确的远程仓库配置"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
IFS='/' read -r owner repo_name <<< "$repo"
|
||||
if [ -z "$owner" ] || [ -z "$repo_name" ]; then
|
||||
echo -e "${RED}错误: 仓库格式不正确,应为 owner/repo${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
upload_gitea "$base_url" "$owner" "$repo_name" "$tag" "$token" "$release_notes"
|
||||
;;
|
||||
github)
|
||||
upload_github "$repo" "$tag" "$release_notes"
|
||||
;;
|
||||
scp)
|
||||
upload_scp "$host" "$user" "$dest" "$port" "$key"
|
||||
;;
|
||||
ftp)
|
||||
upload_ftp "$host" "$user" "$dest" "$port"
|
||||
;;
|
||||
local)
|
||||
upload_local "$dest"
|
||||
;;
|
||||
*)
|
||||
echo -e "${RED}错误: 不支持的上传方式: ${method}${NC}"
|
||||
echo "支持的方式: gitea, github, scp, ftp, local"
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo ""
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN}发布完成${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
}
|
||||
|
||||
# 运行主函数
|
||||
main "$@"
|
||||
|
||||
512
check-heartbeat.sh
Executable file
512
check-heartbeat.sh
Executable file
@@ -0,0 +1,512 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ============================================
|
||||
# LinkMaster 节点心跳故障排查脚本
|
||||
# 用途:诊断节点心跳同步问题
|
||||
# ============================================
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色输出
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
CYAN='\033[0;36m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
# 脚本目录
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
# 配置
|
||||
BINARY_NAME="agent"
|
||||
LOG_FILE="node.log"
|
||||
PID_FILE="node.pid"
|
||||
CONFIG_FILE="${CONFIG_PATH:-config.yaml}"
|
||||
|
||||
# 检查结果
|
||||
ISSUES=0
|
||||
WARNINGS=0
|
||||
|
||||
# 打印分隔线
|
||||
print_separator() {
|
||||
echo -e "${CYAN}========================================${NC}"
|
||||
}
|
||||
|
||||
# 打印检查项标题
|
||||
print_check_title() {
|
||||
echo -e "\n${BLUE}▶ $1${NC}"
|
||||
}
|
||||
|
||||
# 打印成功信息
|
||||
print_success() {
|
||||
echo -e "${GREEN}✓ $1${NC}"
|
||||
}
|
||||
|
||||
# 打印警告信息
|
||||
print_warning() {
|
||||
echo -e "${YELLOW}⚠ $1${NC}"
|
||||
((WARNINGS++))
|
||||
}
|
||||
|
||||
# 打印错误信息
|
||||
print_error() {
|
||||
echo -e "${RED}✗ $1${NC}"
|
||||
((ISSUES++))
|
||||
}
|
||||
|
||||
# 打印信息
|
||||
print_info() {
|
||||
echo -e "${CYAN}ℹ $1${NC}"
|
||||
}
|
||||
|
||||
# 获取PID
|
||||
get_pid() {
|
||||
if [ -f "$PID_FILE" ]; then
|
||||
PID=$(cat "$PID_FILE")
|
||||
if ps -p "$PID" > /dev/null 2>&1; then
|
||||
echo "$PID"
|
||||
else
|
||||
rm -f "$PID_FILE"
|
||||
echo ""
|
||||
fi
|
||||
else
|
||||
echo ""
|
||||
fi
|
||||
}
|
||||
|
||||
# 1. 检查进程状态
|
||||
check_process() {
|
||||
print_check_title "检查进程状态"
|
||||
|
||||
PID=$(get_pid)
|
||||
if [ -z "$PID" ]; then
|
||||
print_error "节点进程未运行"
|
||||
print_info "请使用 ./run.sh start 启动服务"
|
||||
return 1
|
||||
else
|
||||
print_success "节点进程正在运行 (PID: $PID)"
|
||||
|
||||
# 检查进程运行时间
|
||||
if command -v ps > /dev/null 2>&1; then
|
||||
RUNTIME=$(ps -o etime= -p "$PID" 2>/dev/null | tr -d ' ')
|
||||
if [ -n "$RUNTIME" ]; then
|
||||
print_info "进程运行时间: $RUNTIME"
|
||||
fi
|
||||
fi
|
||||
|
||||
# 检查进程资源使用
|
||||
if command -v ps > /dev/null 2>&1; then
|
||||
CPU_MEM=$(ps -o %cpu,%mem= -p "$PID" 2>/dev/null | tr -d ' ')
|
||||
if [ -n "$CPU_MEM" ]; then
|
||||
print_info "CPU/内存使用: $CPU_MEM"
|
||||
fi
|
||||
fi
|
||||
|
||||
return 0
|
||||
fi
|
||||
}
|
||||
|
||||
# 2. 检查配置文件
|
||||
check_config() {
|
||||
print_check_title "检查配置文件"
|
||||
|
||||
if [ ! -f "$CONFIG_FILE" ]; then
|
||||
print_warning "配置文件不存在: $CONFIG_FILE"
|
||||
print_info "将使用环境变量和默认配置"
|
||||
|
||||
# 检查环境变量
|
||||
if [ -n "$BACKEND_URL" ]; then
|
||||
print_info "使用环境变量 BACKEND_URL: $BACKEND_URL"
|
||||
else
|
||||
print_warning "未设置 BACKEND_URL 环境变量,将使用默认值: http://localhost:8080"
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
|
||||
print_success "配置文件存在: $CONFIG_FILE"
|
||||
|
||||
# 检查配置文件内容
|
||||
if command -v yq > /dev/null 2>&1; then
|
||||
BACKEND_URL_FROM_CONFIG=$(yq eval '.backend.url' "$CONFIG_FILE" 2>/dev/null || echo "")
|
||||
HEARTBEAT_INTERVAL=$(yq eval '.heartbeat.interval' "$CONFIG_FILE" 2>/dev/null || echo "")
|
||||
NODE_ID=$(yq eval '.node.id' "$CONFIG_FILE" 2>/dev/null || echo "")
|
||||
NODE_IP=$(yq eval '.node.ip' "$CONFIG_FILE" 2>/dev/null || echo "")
|
||||
else
|
||||
# 使用 grep 和 sed 简单解析
|
||||
BACKEND_URL_FROM_CONFIG=$(grep -E "^\s*url:" "$CONFIG_FILE" | head -1 | sed 's/.*url:\s*//' | tr -d '"' | tr -d "'" || echo "")
|
||||
HEARTBEAT_INTERVAL=$(grep -E "^\s*interval:" "$CONFIG_FILE" | head -1 | sed 's/.*interval:\s*//' | tr -d '"' | tr -d "'" || echo "")
|
||||
NODE_ID=$(grep -E "^\s*id:" "$CONFIG_FILE" | head -1 | sed 's/.*id:\s*//' | tr -d '"' | tr -d "'" || echo "")
|
||||
NODE_IP=$(grep -E "^\s*ip:" "$CONFIG_FILE" | head -1 | sed 's/.*ip:\s*//' | tr -d '"' | tr -d "'" || echo "")
|
||||
fi
|
||||
|
||||
# 确定使用的后端URL
|
||||
if [ -n "$BACKEND_URL" ]; then
|
||||
FINAL_BACKEND_URL="$BACKEND_URL"
|
||||
print_info "使用环境变量 BACKEND_URL: $FINAL_BACKEND_URL"
|
||||
elif [ -n "$BACKEND_URL_FROM_CONFIG" ]; then
|
||||
FINAL_BACKEND_URL="$BACKEND_URL_FROM_CONFIG"
|
||||
print_info "使用配置文件中的后端URL: $FINAL_BACKEND_URL"
|
||||
else
|
||||
FINAL_BACKEND_URL="http://localhost:8080"
|
||||
print_warning "未找到后端URL配置,使用默认值: $FINAL_BACKEND_URL"
|
||||
fi
|
||||
|
||||
if [ -n "$HEARTBEAT_INTERVAL" ]; then
|
||||
print_info "心跳间隔: ${HEARTBEAT_INTERVAL}秒"
|
||||
else
|
||||
print_info "心跳间隔: 60秒 (默认值)"
|
||||
fi
|
||||
|
||||
if [ -n "$NODE_ID" ] && [ "$NODE_ID" != "0" ] && [ "$NODE_ID" != "null" ]; then
|
||||
print_success "节点ID已配置: $NODE_ID"
|
||||
else
|
||||
print_warning "节点ID未配置或为0,将在首次心跳时获取"
|
||||
fi
|
||||
|
||||
if [ -n "$NODE_IP" ] && [ "$NODE_IP" != "null" ]; then
|
||||
print_success "节点IP已配置: $NODE_IP"
|
||||
else
|
||||
print_warning "节点IP未配置,将在首次心跳时获取"
|
||||
fi
|
||||
|
||||
export FINAL_BACKEND_URL
|
||||
}
|
||||
|
||||
# 3. 检查网络连接
|
||||
check_network() {
|
||||
print_check_title "检查网络连接"
|
||||
|
||||
if [ -z "$FINAL_BACKEND_URL" ]; then
|
||||
print_error "无法确定后端URL,跳过网络检查"
|
||||
return 1
|
||||
fi
|
||||
|
||||
# 提取主机和端口
|
||||
BACKEND_HOST=$(echo "$FINAL_BACKEND_URL" | sed -E 's|https?://||' | cut -d'/' -f1 | cut -d':' -f1)
|
||||
BACKEND_PORT=$(echo "$FINAL_BACKEND_URL" | sed -E 's|https?://||' | cut -d'/' -f1 | cut -d':' -f2)
|
||||
|
||||
if [ -z "$BACKEND_PORT" ]; then
|
||||
if echo "$FINAL_BACKEND_URL" | grep -q "https://"; then
|
||||
BACKEND_PORT=443
|
||||
else
|
||||
BACKEND_PORT=80
|
||||
fi
|
||||
fi
|
||||
|
||||
print_info "后端地址: $BACKEND_HOST:$BACKEND_PORT"
|
||||
|
||||
# 检查DNS解析
|
||||
if command -v nslookup > /dev/null 2>&1 || command -v host > /dev/null 2>&1; then
|
||||
if command -v nslookup > /dev/null 2>&1; then
|
||||
if nslookup "$BACKEND_HOST" > /dev/null 2>&1; then
|
||||
print_success "DNS解析成功: $BACKEND_HOST"
|
||||
else
|
||||
print_error "DNS解析失败: $BACKEND_HOST"
|
||||
return 1
|
||||
fi
|
||||
elif command -v host > /dev/null 2>&1; then
|
||||
if host "$BACKEND_HOST" > /dev/null 2>&1; then
|
||||
print_success "DNS解析成功: $BACKEND_HOST"
|
||||
else
|
||||
print_error "DNS解析失败: $BACKEND_HOST"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# 检查端口连通性
|
||||
if command -v nc > /dev/null 2>&1; then
|
||||
if nc -z -w 3 "$BACKEND_HOST" "$BACKEND_PORT" 2>/dev/null; then
|
||||
print_success "端口连通性检查通过: $BACKEND_HOST:$BACKEND_PORT"
|
||||
else
|
||||
print_error "端口无法连接: $BACKEND_HOST:$BACKEND_PORT"
|
||||
print_info "可能原因: 防火墙阻止、后端服务未启动、网络不通"
|
||||
return 1
|
||||
fi
|
||||
elif command -v timeout > /dev/null 2>&1 && command -v bash > /dev/null 2>&1; then
|
||||
# 使用 bash 内置的 TCP 连接测试
|
||||
if timeout 3 bash -c "echo > /dev/tcp/$BACKEND_HOST/$BACKEND_PORT" 2>/dev/null; then
|
||||
print_success "端口连通性检查通过: $BACKEND_HOST:$BACKEND_PORT"
|
||||
else
|
||||
print_error "端口无法连接: $BACKEND_HOST:$BACKEND_PORT"
|
||||
print_info "可能原因: 防火墙阻止、后端服务未启动、网络不通"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
print_warning "无法检查端口连通性(需要 nc 或 timeout 命令)"
|
||||
fi
|
||||
|
||||
# 检查HTTP连接
|
||||
HEARTBEAT_URL="${FINAL_BACKEND_URL%/}/api/node/heartbeat"
|
||||
print_info "测试心跳接口: $HEARTBEAT_URL"
|
||||
|
||||
if command -v curl > /dev/null 2>&1; then
|
||||
HTTP_CODE=$(curl -s -o /dev/null -w "%{http_code}" --connect-timeout 5 --max-time 10 \
|
||||
-X POST \
|
||||
-H "Content-Type: application/x-www-form-urlencoded" \
|
||||
-d "type=pingServer" \
|
||||
"$HEARTBEAT_URL" 2>/dev/null || echo "000")
|
||||
|
||||
if [ "$HTTP_CODE" = "200" ]; then
|
||||
print_success "心跳接口响应正常 (HTTP 200)"
|
||||
elif [ "$HTTP_CODE" = "000" ]; then
|
||||
print_error "无法连接到心跳接口"
|
||||
print_info "可能原因: 网络不通、后端服务未启动、防火墙阻止"
|
||||
return 1
|
||||
else
|
||||
print_warning "心跳接口返回异常状态码: HTTP $HTTP_CODE"
|
||||
print_info "这可能是正常的,取决于后端实现"
|
||||
fi
|
||||
elif command -v wget > /dev/null 2>&1; then
|
||||
HTTP_CODE=$(wget --spider --server-response --timeout=5 --tries=1 \
|
||||
--post-data="type=pingServer" \
|
||||
--header="Content-Type: application/x-www-form-urlencoded" \
|
||||
"$HEARTBEAT_URL" 2>&1 | grep -E "HTTP/" | tail -1 | awk '{print $2}' || echo "000")
|
||||
|
||||
if [ "$HTTP_CODE" = "200" ]; then
|
||||
print_success "心跳接口响应正常 (HTTP 200)"
|
||||
elif [ "$HTTP_CODE" = "000" ]; then
|
||||
print_error "无法连接到心跳接口"
|
||||
return 1
|
||||
else
|
||||
print_warning "心跳接口返回异常状态码: HTTP $HTTP_CODE"
|
||||
fi
|
||||
else
|
||||
print_warning "无法测试HTTP连接(需要 curl 或 wget 命令)"
|
||||
fi
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
# 4. 检查日志
|
||||
check_logs() {
|
||||
print_check_title "检查日志文件"
|
||||
|
||||
if [ ! -f "$LOG_FILE" ]; then
|
||||
print_warning "日志文件不存在: $LOG_FILE"
|
||||
print_info "如果服务刚启动,日志文件可能还未创建"
|
||||
return 0
|
||||
fi
|
||||
|
||||
print_success "日志文件存在: $LOG_FILE"
|
||||
|
||||
# 检查日志文件大小
|
||||
LOG_SIZE=$(stat -f%z "$LOG_FILE" 2>/dev/null || stat -c%s "$LOG_FILE" 2>/dev/null || echo "0")
|
||||
if [ "$LOG_SIZE" -gt 10485760 ]; then
|
||||
print_warning "日志文件较大: $(($LOG_SIZE / 1024 / 1024))MB"
|
||||
fi
|
||||
|
||||
# 检查最近的心跳记录
|
||||
print_info "查找最近的心跳记录..."
|
||||
|
||||
HEARTBEAT_SUCCESS=$(grep -i "心跳发送成功\|heartbeat.*success\|心跳响应" "$LOG_FILE" 2>/dev/null | tail -5 || true)
|
||||
HEARTBEAT_FAILED=$(grep -i "心跳发送失败\|heartbeat.*fail\|发送心跳失败" "$LOG_FILE" 2>/dev/null | tail -5 || true)
|
||||
HEARTBEAT_ERROR=$(grep -i "error.*heartbeat\|心跳.*error" "$LOG_FILE" 2>/dev/null | tail -5 || true)
|
||||
|
||||
if [ -n "$HEARTBEAT_SUCCESS" ]; then
|
||||
echo -e "${GREEN}最近成功的心跳记录:${NC}"
|
||||
echo "$HEARTBEAT_SUCCESS" | while IFS= read -r line; do
|
||||
echo " $line"
|
||||
done
|
||||
fi
|
||||
|
||||
if [ -n "$HEARTBEAT_FAILED" ]; then
|
||||
echo -e "${YELLOW}最近失败的心跳记录:${NC}"
|
||||
echo "$HEARTBEAT_FAILED" | while IFS= read -r line; do
|
||||
echo " $line"
|
||||
done
|
||||
((WARNINGS++))
|
||||
fi
|
||||
|
||||
if [ -n "$HEARTBEAT_ERROR" ]; then
|
||||
echo -e "${RED}最近的心跳错误记录:${NC}"
|
||||
echo "$HEARTBEAT_ERROR" | while IFS= read -r line; do
|
||||
echo " $line"
|
||||
done
|
||||
((ISSUES++))
|
||||
fi
|
||||
|
||||
# 检查最近的错误
|
||||
RECENT_ERRORS=$(grep -i "error\|fail\|panic" "$LOG_FILE" 2>/dev/null | tail -10 || true)
|
||||
if [ -n "$RECENT_ERRORS" ]; then
|
||||
echo -e "${YELLOW}最近的错误记录(最后10条):${NC}"
|
||||
echo "$RECENT_ERRORS" | while IFS= read -r line; do
|
||||
echo " $line"
|
||||
done
|
||||
fi
|
||||
|
||||
# 检查最后的心跳时间
|
||||
LAST_HEARTBEAT=$(grep -i "心跳" "$LOG_FILE" 2>/dev/null | tail -1 || true)
|
||||
if [ -n "$LAST_HEARTBEAT" ]; then
|
||||
print_info "最后的心跳日志: $LAST_HEARTBEAT"
|
||||
else
|
||||
print_warning "日志中未找到心跳记录"
|
||||
fi
|
||||
}
|
||||
|
||||
# 5. 手动测试心跳
|
||||
test_heartbeat() {
|
||||
print_check_title "手动测试心跳发送"
|
||||
|
||||
if [ -z "$FINAL_BACKEND_URL" ]; then
|
||||
print_error "无法确定后端URL,跳过心跳测试"
|
||||
return 1
|
||||
fi
|
||||
|
||||
HEARTBEAT_URL="${FINAL_BACKEND_URL%/}/api/node/heartbeat"
|
||||
print_info "发送测试心跳到: $HEARTBEAT_URL"
|
||||
|
||||
if command -v curl > /dev/null 2>&1; then
|
||||
RESPONSE=$(curl -s -w "\n%{http_code}" --connect-timeout 10 --max-time 15 \
|
||||
-X POST \
|
||||
-H "Content-Type: application/x-www-form-urlencoded" \
|
||||
-d "type=pingServer" \
|
||||
"$HEARTBEAT_URL" 2>&1)
|
||||
|
||||
HTTP_CODE=$(echo "$RESPONSE" | tail -1)
|
||||
BODY=$(echo "$RESPONSE" | sed '$d')
|
||||
|
||||
if [ "$HTTP_CODE" = "200" ]; then
|
||||
print_success "心跳发送成功 (HTTP 200)"
|
||||
if [ -n "$BODY" ]; then
|
||||
print_info "响应内容: $BODY"
|
||||
|
||||
# 尝试解析JSON响应
|
||||
if echo "$BODY" | grep -q "node_id\|node_ip"; then
|
||||
print_success "响应包含节点信息"
|
||||
echo "$BODY" | grep -o '"node_id":[0-9]*\|"node_ip":"[^"]*"' 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
else
|
||||
print_error "心跳发送失败 (HTTP $HTTP_CODE)"
|
||||
if [ -n "$BODY" ]; then
|
||||
print_info "响应内容: $BODY"
|
||||
fi
|
||||
return 1
|
||||
fi
|
||||
elif command -v wget > /dev/null 2>&1; then
|
||||
RESPONSE=$(wget -qO- --post-data="type=pingServer" \
|
||||
--header="Content-Type: application/x-www-form-urlencoded" \
|
||||
--timeout=15 \
|
||||
"$HEARTBEAT_URL" 2>&1)
|
||||
|
||||
if [ $? -eq 0 ]; then
|
||||
print_success "心跳发送成功"
|
||||
if [ -n "$RESPONSE" ]; then
|
||||
print_info "响应内容: $RESPONSE"
|
||||
fi
|
||||
else
|
||||
print_error "心跳发送失败"
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
print_warning "无法测试心跳(需要 curl 或 wget 命令)"
|
||||
return 1
|
||||
fi
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
# 6. 检查系统资源
|
||||
check_resources() {
|
||||
print_check_title "检查系统资源"
|
||||
|
||||
# 检查磁盘空间
|
||||
if command -v df > /dev/null 2>&1; then
|
||||
DISK_USAGE=$(df -h . | tail -1 | awk '{print $5}' | sed 's/%//')
|
||||
if [ "$DISK_USAGE" -gt 90 ]; then
|
||||
print_error "磁盘空间不足: ${DISK_USAGE}%"
|
||||
elif [ "$DISK_USAGE" -gt 80 ]; then
|
||||
print_warning "磁盘空间紧张: ${DISK_USAGE}%"
|
||||
else
|
||||
print_success "磁盘空间充足: ${DISK_USAGE}%"
|
||||
fi
|
||||
fi
|
||||
|
||||
# 检查内存
|
||||
if command -v free > /dev/null 2>&1; then
|
||||
MEM_INFO=$(free -m | grep Mem)
|
||||
MEM_TOTAL=$(echo "$MEM_INFO" | awk '{print $2}')
|
||||
MEM_AVAIL=$(echo "$MEM_INFO" | awk '{print $7}')
|
||||
if [ -z "$MEM_AVAIL" ]; then
|
||||
MEM_AVAIL=$(echo "$MEM_INFO" | awk '{print $4}')
|
||||
fi
|
||||
|
||||
if [ -n "$MEM_TOTAL" ] && [ -n "$MEM_AVAIL" ]; then
|
||||
MEM_PERCENT=$((MEM_AVAIL * 100 / MEM_TOTAL))
|
||||
if [ "$MEM_PERCENT" -lt 10 ]; then
|
||||
print_error "可用内存不足: ${MEM_AVAIL}MB / ${MEM_TOTAL}MB (${MEM_PERCENT}%)"
|
||||
elif [ "$MEM_PERCENT" -lt 20 ]; then
|
||||
print_warning "可用内存紧张: ${MEM_AVAIL}MB / ${MEM_TOTAL}MB (${MEM_PERCENT}%)"
|
||||
else
|
||||
print_success "内存充足: ${MEM_AVAIL}MB / ${MEM_TOTAL}MB (${MEM_PERCENT}%)"
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
}
|
||||
|
||||
# 主函数
|
||||
main() {
|
||||
echo -e "${CYAN}"
|
||||
echo "========================================"
|
||||
echo " LinkMaster 节点心跳故障排查工具"
|
||||
echo "========================================"
|
||||
echo -e "${NC}"
|
||||
|
||||
# 执行各项检查
|
||||
check_process
|
||||
PROCESS_OK=$?
|
||||
|
||||
check_config
|
||||
|
||||
if [ $PROCESS_OK -eq 0 ]; then
|
||||
check_network
|
||||
NETWORK_OK=$?
|
||||
|
||||
check_logs
|
||||
|
||||
if [ $NETWORK_OK -eq 0 ]; then
|
||||
echo ""
|
||||
read -p "是否执行手动心跳测试? (y/N): " -n 1 -r
|
||||
echo ""
|
||||
if [[ $REPLY =~ ^[Yy]$ ]]; then
|
||||
test_heartbeat
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
check_resources
|
||||
|
||||
# 总结
|
||||
print_separator
|
||||
echo -e "\n${BLUE}排查总结:${NC}"
|
||||
|
||||
if [ $ISSUES -eq 0 ] && [ $WARNINGS -eq 0 ]; then
|
||||
echo -e "${GREEN}✓ 未发现明显问题${NC}"
|
||||
echo -e "${CYAN}如果心跳仍然无法同步,请检查:${NC}"
|
||||
echo " 1. 后端服务是否正常运行"
|
||||
echo " 2. 后端数据库是否正常"
|
||||
echo " 3. 防火墙规则是否正确配置"
|
||||
echo " 4. 查看完整日志: ./run.sh logs-all"
|
||||
else
|
||||
if [ $ISSUES -gt 0 ]; then
|
||||
echo -e "${RED}发现 $ISSUES 个严重问题${NC}"
|
||||
fi
|
||||
if [ $WARNINGS -gt 0 ]; then
|
||||
echo -e "${YELLOW}发现 $WARNINGS 个警告${NC}"
|
||||
fi
|
||||
|
||||
echo -e "\n${CYAN}建议操作:${NC}"
|
||||
echo " 1. 根据上述检查结果修复问题"
|
||||
echo " 2. 重启服务: ./run.sh restart"
|
||||
echo " 3. 查看实时日志: ./run.sh logs"
|
||||
echo " 4. 查看完整日志: ./run.sh logs-all"
|
||||
fi
|
||||
|
||||
print_separator
|
||||
}
|
||||
|
||||
# 运行主函数
|
||||
main
|
||||
@@ -5,6 +5,7 @@ import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
@@ -14,8 +15,11 @@ import (
|
||||
"linkmaster-node/internal/server"
|
||||
|
||||
"go.uber.org/zap"
|
||||
"go.uber.org/zap/zapcore"
|
||||
)
|
||||
|
||||
var version = "1.1.0" // 编译时通过 -ldflags "-X main.version=xxx" 设置
|
||||
|
||||
func main() {
|
||||
// 加载配置
|
||||
cfg, err := config.Load()
|
||||
@@ -32,7 +36,7 @@ func main() {
|
||||
}
|
||||
defer logger.Sync()
|
||||
|
||||
logger.Info("节点服务启动", zap.String("version", "1.0.0"))
|
||||
logger.Info("节点服务启动", zap.String("version", version))
|
||||
|
||||
// 初始化错误恢复
|
||||
recovery.Init()
|
||||
@@ -80,9 +84,69 @@ func main() {
|
||||
}
|
||||
|
||||
func initLogger(cfg *config.Config) (*zap.Logger, error) {
|
||||
if cfg.Debug {
|
||||
return zap.NewDevelopment()
|
||||
// 确定日志级别
|
||||
var level zapcore.Level
|
||||
logLevel := cfg.Log.Level
|
||||
if logLevel == "" {
|
||||
if cfg.Debug {
|
||||
logLevel = "debug"
|
||||
} else {
|
||||
logLevel = "info"
|
||||
}
|
||||
}
|
||||
return zap.NewProduction()
|
||||
}
|
||||
|
||||
switch logLevel {
|
||||
case "debug":
|
||||
level = zapcore.DebugLevel
|
||||
case "info":
|
||||
level = zapcore.InfoLevel
|
||||
case "warn":
|
||||
level = zapcore.WarnLevel
|
||||
case "error":
|
||||
level = zapcore.ErrorLevel
|
||||
default:
|
||||
level = zapcore.InfoLevel
|
||||
}
|
||||
|
||||
// 编码器配置
|
||||
encoderConfig := zap.NewProductionEncoderConfig()
|
||||
if cfg.Debug {
|
||||
encoderConfig = zap.NewDevelopmentEncoderConfig()
|
||||
}
|
||||
encoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
|
||||
encoderConfig.EncodeLevel = zapcore.CapitalLevelEncoder
|
||||
|
||||
// 确定输出目标
|
||||
var writeSyncer zapcore.WriteSyncer
|
||||
if cfg.Log.File != "" {
|
||||
// 确保日志目录存在
|
||||
logDir := filepath.Dir(cfg.Log.File)
|
||||
if logDir != "." && logDir != "" {
|
||||
if err := os.MkdirAll(logDir, 0755); err != nil {
|
||||
return nil, fmt.Errorf("创建日志目录失败: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
// 打开日志文件(追加模式)
|
||||
logFile, err := os.OpenFile(cfg.Log.File, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("打开日志文件失败: %w", err)
|
||||
}
|
||||
writeSyncer = zapcore.AddSync(logFile)
|
||||
} else {
|
||||
// 输出到标准错误(兼容原有行为)
|
||||
writeSyncer = zapcore.AddSync(os.Stderr)
|
||||
}
|
||||
|
||||
// 创建核心
|
||||
core := zapcore.NewCore(
|
||||
zapcore.NewJSONEncoder(encoderConfig),
|
||||
writeSyncer,
|
||||
level,
|
||||
)
|
||||
|
||||
// 创建 logger
|
||||
logger := zap.New(core, zap.AddCaller(), zap.AddStacktrace(zapcore.ErrorLevel))
|
||||
|
||||
return logger, nil
|
||||
}
|
||||
|
||||
1046
install.sh
1046
install.sh
File diff suppressed because it is too large
Load Diff
@@ -21,6 +21,11 @@ type Config struct {
|
||||
Interval int `yaml:"interval"` // 心跳间隔(秒)
|
||||
} `yaml:"heartbeat"`
|
||||
|
||||
Log struct {
|
||||
File string `yaml:"file"` // 日志文件路径(空则输出到标准错误)
|
||||
Level string `yaml:"level"` // 日志级别:debug, info, warn, error(默认: info)
|
||||
} `yaml:"log"`
|
||||
|
||||
Debug bool `yaml:"debug"`
|
||||
|
||||
// 节点信息(通过心跳获取并持久化)
|
||||
@@ -42,12 +47,13 @@ func Load() (*Config, error) {
|
||||
cfg.Heartbeat.Interval = 60
|
||||
cfg.Debug = false
|
||||
|
||||
// 从环境变量读取后端URL
|
||||
backendURL := os.Getenv("BACKEND_URL")
|
||||
if backendURL == "" {
|
||||
backendURL = "http://localhost:8080"
|
||||
// 默认日志配置
|
||||
logFile := os.Getenv("LOG_FILE")
|
||||
if logFile == "" {
|
||||
logFile = "node.log"
|
||||
}
|
||||
cfg.Backend.URL = backendURL
|
||||
cfg.Log.File = logFile
|
||||
cfg.Log.Level = "info"
|
||||
|
||||
// 尝试从配置文件读取
|
||||
configPath := os.Getenv("CONFIG_PATH")
|
||||
@@ -66,6 +72,30 @@ func Load() (*Config, error) {
|
||||
}
|
||||
}
|
||||
|
||||
// 环境变量优先级最高,覆盖配置文件中的设置
|
||||
// 支持 BACKEND_URL 环境变量覆盖后端地址
|
||||
if backendURL := os.Getenv("BACKEND_URL"); backendURL != "" {
|
||||
cfg.Backend.URL = backendURL
|
||||
}
|
||||
|
||||
// 如果配置文件中没有设置日志文件,使用环境变量或默认值
|
||||
if cfg.Log.File == "" {
|
||||
logFile := os.Getenv("LOG_FILE")
|
||||
if logFile == "" {
|
||||
logFile = "node.log"
|
||||
}
|
||||
cfg.Log.File = logFile
|
||||
}
|
||||
|
||||
// 如果配置文件中没有设置日志级别,使用默认值
|
||||
if cfg.Log.Level == "" {
|
||||
if cfg.Debug {
|
||||
cfg.Log.Level = "debug"
|
||||
} else {
|
||||
cfg.Log.Level = "info"
|
||||
}
|
||||
}
|
||||
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
@@ -102,4 +132,3 @@ func GetConfigPath() string {
|
||||
}
|
||||
return configPath
|
||||
}
|
||||
|
||||
|
||||
@@ -28,14 +28,47 @@ type TCPingTask struct {
|
||||
}
|
||||
|
||||
func NewTCPingTask(taskID, target string, interval, maxDuration time.Duration) (*TCPingTask, error) {
|
||||
// 解析host:port
|
||||
parts := strings.Split(target, ":")
|
||||
if len(parts) != 2 {
|
||||
return nil, fmt.Errorf("无效的target格式,需要 host:port")
|
||||
// 解析host:port,如果没有端口则默认80
|
||||
var host string
|
||||
var portStr string
|
||||
var port int
|
||||
|
||||
// 检查是否是IPv6格式(如 [::1]:8080)
|
||||
if strings.HasPrefix(target, "[") {
|
||||
// IPv6格式 - 使用 Index 而不是 LastIndex 来找到第一个闭合括号
|
||||
closeBracket := strings.Index(target, "]")
|
||||
if closeBracket == -1 {
|
||||
return nil, fmt.Errorf("无效的target格式,IPv6地址格式应为 [host]:port")
|
||||
}
|
||||
host = target[1:closeBracket]
|
||||
if closeBracket+1 < len(target) && target[closeBracket+1] == ':' {
|
||||
portStr = target[closeBracket+2:]
|
||||
// 如果端口部分为空,使用默认端口80(修复 Bug 1)
|
||||
if portStr == "" {
|
||||
portStr = "80"
|
||||
}
|
||||
} else {
|
||||
portStr = "80" // 默认端口
|
||||
}
|
||||
} else {
|
||||
// 普通格式 host:port 或 host
|
||||
lastColonIndex := strings.LastIndex(target, ":")
|
||||
if lastColonIndex == -1 {
|
||||
// 没有冒号,使用默认端口80
|
||||
host = target
|
||||
portStr = "80"
|
||||
} else {
|
||||
host = target[:lastColonIndex]
|
||||
portStr = target[lastColonIndex+1:]
|
||||
// 如果端口部分为空,使用默认端口80
|
||||
if portStr == "" {
|
||||
portStr = "80"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
host := parts[0]
|
||||
port, err := strconv.Atoi(parts[1])
|
||||
var err error
|
||||
port, err = strconv.Atoi(portStr)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("无效的端口: %v", err)
|
||||
}
|
||||
@@ -185,4 +218,3 @@ func (t *TCPingTask) executeTCPing() map[string]interface{} {
|
||||
"ip": targetIP,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
@@ -18,6 +19,7 @@ import (
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"go.uber.org/zap"
|
||||
"go.uber.org/zap/zapcore"
|
||||
)
|
||||
|
||||
var continuousTasks = make(map[string]*ContinuousTask)
|
||||
@@ -46,7 +48,52 @@ const (
|
||||
|
||||
func InitContinuousHandler(cfg *config.Config) {
|
||||
backendURL = cfg.Backend.URL
|
||||
logger, _ = zap.NewProduction()
|
||||
|
||||
// 根据配置创建logger
|
||||
var level zapcore.Level
|
||||
logLevel := cfg.Log.Level
|
||||
if logLevel == "" {
|
||||
if cfg.Debug {
|
||||
logLevel = "debug"
|
||||
} else {
|
||||
logLevel = "info"
|
||||
}
|
||||
}
|
||||
|
||||
switch logLevel {
|
||||
case "debug":
|
||||
level = zapcore.DebugLevel
|
||||
case "info":
|
||||
level = zapcore.InfoLevel
|
||||
case "warn":
|
||||
level = zapcore.WarnLevel
|
||||
case "error":
|
||||
level = zapcore.ErrorLevel
|
||||
default:
|
||||
level = zapcore.InfoLevel
|
||||
}
|
||||
|
||||
// 创建编码器配置
|
||||
encoderConfig := zap.NewProductionEncoderConfig()
|
||||
if cfg.Debug {
|
||||
encoderConfig = zap.NewDevelopmentEncoderConfig()
|
||||
}
|
||||
encoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
|
||||
encoderConfig.EncodeLevel = zapcore.CapitalLevelEncoder
|
||||
|
||||
// 创建核心 - 输出到标准错误(日志文件由main.go统一管理,这里输出到stderr便于调试)
|
||||
core := zapcore.NewCore(
|
||||
zapcore.NewJSONEncoder(encoderConfig),
|
||||
zapcore.AddSync(os.Stderr),
|
||||
level,
|
||||
)
|
||||
|
||||
// 创建logger
|
||||
logger = zap.New(core, zap.AddCaller(), zap.AddStacktrace(zapcore.ErrorLevel))
|
||||
|
||||
logger.Info("持续测试处理器已初始化",
|
||||
zap.String("backend_url", backendURL),
|
||||
zap.String("log_level", logLevel))
|
||||
}
|
||||
|
||||
type ContinuousTask struct {
|
||||
@@ -160,7 +207,15 @@ func HandleContinuousStop(c *gin.Context) {
|
||||
if task.tcpingTask != nil {
|
||||
task.tcpingTask.Stop()
|
||||
}
|
||||
close(task.StopCh)
|
||||
|
||||
// 关闭停止通道
|
||||
select {
|
||||
case <-task.StopCh:
|
||||
// 已经关闭
|
||||
default:
|
||||
close(task.StopCh)
|
||||
}
|
||||
|
||||
delete(continuousTasks, req.TaskID)
|
||||
}
|
||||
taskMutex.Unlock()
|
||||
@@ -170,6 +225,17 @@ func HandleContinuousStop(c *gin.Context) {
|
||||
return
|
||||
}
|
||||
|
||||
// 清理推送缓冲
|
||||
bufferMutex.Lock()
|
||||
if buffer, exists := pushBuffers[req.TaskID]; exists {
|
||||
if buffer.pushTimer != nil {
|
||||
buffer.pushTimer.Stop()
|
||||
}
|
||||
delete(pushBuffers, req.TaskID)
|
||||
logger.Debug("已清理任务推送缓冲", zap.String("task_id", req.TaskID))
|
||||
}
|
||||
bufferMutex.Unlock()
|
||||
|
||||
c.JSON(http.StatusOK, gin.H{"message": "任务已停止"})
|
||||
}
|
||||
|
||||
@@ -237,7 +303,8 @@ func pushResultToBackend(taskID string, result map[string]interface{}) {
|
||||
logger.Warn("节点ID未获取,跳过推送结果",
|
||||
zap.String("task_id", taskID),
|
||||
zap.String("node_ip", nodeIP),
|
||||
zap.String("hint", "等待心跳返回node_id后再推送"))
|
||||
zap.String("hint", "等待心跳返回node_id后再推送"),
|
||||
zap.Any("result", result))
|
||||
return
|
||||
}
|
||||
|
||||
@@ -246,10 +313,18 @@ func pushResultToBackend(taskID string, result map[string]interface{}) {
|
||||
logger.Warn("节点IP未获取,跳过推送结果",
|
||||
zap.String("task_id", taskID),
|
||||
zap.Uint("node_id", nodeID),
|
||||
zap.String("hint", "等待心跳返回node_ip后再推送"))
|
||||
zap.String("hint", "等待心跳返回node_ip后再推送"),
|
||||
zap.Any("result", result))
|
||||
return
|
||||
}
|
||||
|
||||
// 记录调试信息
|
||||
logger.Debug("准备推送结果到后端",
|
||||
zap.String("task_id", taskID),
|
||||
zap.Uint("node_id", nodeID),
|
||||
zap.String("node_ip", nodeIP),
|
||||
zap.Any("result", result))
|
||||
|
||||
// 添加到批量推送缓冲
|
||||
addToPushBuffer(taskID, nodeID, nodeIP, result)
|
||||
}
|
||||
@@ -269,28 +344,43 @@ func addToPushBuffer(taskID string, nodeID uint, nodeIP string, result map[strin
|
||||
bufferMutex.Unlock()
|
||||
|
||||
buffer.mu.Lock()
|
||||
defer buffer.mu.Unlock()
|
||||
|
||||
// 添加结果到缓冲
|
||||
buffer.results = append(buffer.results, result)
|
||||
|
||||
// 如果缓冲已满,立即推送
|
||||
shouldFlush := len(buffer.results) >= batchPushMaxSize
|
||||
buffer.mu.Unlock()
|
||||
|
||||
if shouldFlush {
|
||||
flushPushBuffer(taskID, nodeID, nodeIP)
|
||||
// 复制结果列表
|
||||
results := make([]map[string]interface{}, len(buffer.results))
|
||||
copy(results, buffer.results)
|
||||
buffer.results = buffer.results[:0] // 清空缓冲
|
||||
|
||||
// 停止定时器
|
||||
if buffer.pushTimer != nil {
|
||||
buffer.pushTimer.Stop()
|
||||
buffer.pushTimer = nil
|
||||
}
|
||||
|
||||
buffer.lastPush = time.Now()
|
||||
buffer.mu.Unlock()
|
||||
|
||||
// 批量推送结果
|
||||
for _, r := range results {
|
||||
pushSingleResult(taskID, nodeID, nodeIP, r)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
buffer.mu.Lock()
|
||||
|
||||
// 如果距离上次推送超过间隔时间,启动定时器推送
|
||||
if buffer.pushTimer == nil {
|
||||
buffer.pushTimer = time.AfterFunc(batchPushInterval, func() {
|
||||
flushPushBuffer(taskID, nodeID, nodeIP)
|
||||
})
|
||||
}
|
||||
|
||||
buffer.mu.Unlock()
|
||||
}
|
||||
|
||||
// flushPushBuffer 刷新并推送缓冲中的结果
|
||||
@@ -362,13 +452,21 @@ func pushSingleResult(taskID string, nodeID uint, nodeIP string, result map[stri
|
||||
|
||||
jsonData, err := json.Marshal(data)
|
||||
if err != nil {
|
||||
logger.Error("序列化结果失败", zap.Error(err), zap.String("task_id", taskID))
|
||||
logger.Error("序列化结果失败",
|
||||
zap.Error(err),
|
||||
zap.String("task_id", taskID),
|
||||
zap.Uint("node_id", nodeID),
|
||||
zap.String("node_ip", nodeIP),
|
||||
zap.Any("data", data))
|
||||
return
|
||||
}
|
||||
|
||||
req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonData))
|
||||
if err != nil {
|
||||
logger.Error("创建请求失败", zap.Error(err), zap.String("task_id", taskID))
|
||||
logger.Error("创建请求失败",
|
||||
zap.Error(err),
|
||||
zap.String("task_id", taskID),
|
||||
zap.String("url", url))
|
||||
return
|
||||
}
|
||||
|
||||
@@ -380,7 +478,9 @@ func pushSingleResult(taskID string, nodeID uint, nodeIP string, result map[stri
|
||||
logger.Warn("推送结果失败,继续运行",
|
||||
zap.Error(err),
|
||||
zap.String("task_id", taskID),
|
||||
zap.String("url", url))
|
||||
zap.String("url", url),
|
||||
zap.Uint("node_id", nodeID),
|
||||
zap.String("node_ip", nodeIP))
|
||||
// 推送失败不停止任务,继续运行
|
||||
return
|
||||
}
|
||||
@@ -394,7 +494,9 @@ func pushSingleResult(taskID string, nodeID uint, nodeIP string, result map[stri
|
||||
if containsTaskNotFoundError(bodyStr) {
|
||||
logger.Warn("后端任务不存在,停止节点端任务",
|
||||
zap.String("task_id", taskID),
|
||||
zap.String("response", bodyStr))
|
||||
zap.String("response", bodyStr),
|
||||
zap.Uint("node_id", nodeID),
|
||||
zap.String("node_ip", nodeIP))
|
||||
// 停止对应的持续测试任务
|
||||
stopTaskByTaskID(taskID)
|
||||
return
|
||||
@@ -404,12 +506,18 @@ func pushSingleResult(taskID string, nodeID uint, nodeIP string, result map[stri
|
||||
zap.Int("status", resp.StatusCode),
|
||||
zap.String("task_id", taskID),
|
||||
zap.String("url", url),
|
||||
zap.String("response", bodyStr))
|
||||
zap.String("response", bodyStr),
|
||||
zap.Uint("node_id", nodeID),
|
||||
zap.String("node_ip", nodeIP))
|
||||
// 其他错误不停止任务,继续运行
|
||||
return
|
||||
}
|
||||
|
||||
logger.Debug("推送结果成功", zap.String("task_id", taskID))
|
||||
logger.Debug("推送结果成功",
|
||||
zap.String("task_id", taskID),
|
||||
zap.Uint("node_id", nodeID),
|
||||
zap.String("node_ip", nodeIP),
|
||||
zap.Any("result", result))
|
||||
}
|
||||
|
||||
// containsTaskNotFoundError 检查响应中是否包含任务不存在的错误
|
||||
@@ -522,23 +630,20 @@ func StartTaskCleanup() {
|
||||
for range ticker.C {
|
||||
now := time.Now()
|
||||
taskMutex.Lock()
|
||||
var tasksToDelete []string
|
||||
for taskID, task := range continuousTasks {
|
||||
shouldDelete := false
|
||||
// 检查最大运行时长
|
||||
if now.Sub(task.StartTime) > task.MaxDuration {
|
||||
logger.Info("任务达到最大运行时长,自动停止", zap.String("task_id", taskID))
|
||||
task.IsRunning = false
|
||||
if task.pingTask != nil {
|
||||
task.pingTask.Stop()
|
||||
}
|
||||
if task.tcpingTask != nil {
|
||||
task.tcpingTask.Stop()
|
||||
}
|
||||
delete(continuousTasks, taskID)
|
||||
continue
|
||||
}
|
||||
// 检查无客户端连接(30分钟无请求)
|
||||
if now.Sub(task.LastRequest) > 30*time.Minute {
|
||||
shouldDelete = true
|
||||
} else if now.Sub(task.LastRequest) > 30*time.Minute {
|
||||
// 检查无客户端连接(30分钟无请求)
|
||||
logger.Info("任务无客户端连接,自动停止", zap.String("task_id", taskID))
|
||||
shouldDelete = true
|
||||
}
|
||||
|
||||
if shouldDelete {
|
||||
task.IsRunning = false
|
||||
if task.pingTask != nil {
|
||||
task.pingTask.Stop()
|
||||
@@ -546,10 +651,41 @@ func StartTaskCleanup() {
|
||||
if task.tcpingTask != nil {
|
||||
task.tcpingTask.Stop()
|
||||
}
|
||||
delete(continuousTasks, taskID)
|
||||
|
||||
// 关闭停止通道
|
||||
select {
|
||||
case <-task.StopCh:
|
||||
// 已经关闭
|
||||
default:
|
||||
close(task.StopCh)
|
||||
}
|
||||
|
||||
tasksToDelete = append(tasksToDelete, taskID)
|
||||
}
|
||||
}
|
||||
taskMutex.Unlock()
|
||||
|
||||
// 清理任务和推送缓冲
|
||||
if len(tasksToDelete) > 0 {
|
||||
taskMutex.Lock()
|
||||
for _, taskID := range tasksToDelete {
|
||||
delete(continuousTasks, taskID)
|
||||
}
|
||||
taskMutex.Unlock()
|
||||
|
||||
// 清理推送缓冲
|
||||
bufferMutex.Lock()
|
||||
for _, taskID := range tasksToDelete {
|
||||
if buffer, exists := pushBuffers[taskID]; exists {
|
||||
if buffer.pushTimer != nil {
|
||||
buffer.pushTimer.Stop()
|
||||
}
|
||||
delete(pushBuffers, taskID)
|
||||
logger.Debug("已清理任务推送缓冲", zap.String("task_id", taskID))
|
||||
}
|
||||
}
|
||||
bufferMutex.Unlock()
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
@@ -145,11 +145,8 @@ func handleGet(c *gin.Context, urlStr string, params map[string]interface{}) {
|
||||
Transport: timingTransport,
|
||||
Timeout: 15 * time.Second,
|
||||
CheckRedirect: func(req *http.Request, via []*http.Request) error {
|
||||
// 跟随重定向,最多20次
|
||||
if len(via) >= 20 {
|
||||
return fmt.Errorf("重定向次数过多")
|
||||
}
|
||||
return nil
|
||||
// 不跟随重定向,返回第一个状态码和 header
|
||||
return http.ErrUseLastResponse
|
||||
},
|
||||
}
|
||||
|
||||
@@ -181,8 +178,11 @@ func handleGet(c *gin.Context, urlStr string, params map[string]interface{}) {
|
||||
// 执行请求
|
||||
startTime := time.Now()
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
// 错误处理
|
||||
|
||||
// 处理重定向错误:当 CheckRedirect 返回 ErrUseLastResponse 时,
|
||||
// client.Do 会返回响应和错误,但响应仍然有效(包含重定向状态码和 header)
|
||||
if err != nil && resp == nil {
|
||||
// 真正的错误,没有响应
|
||||
errMsg := err.Error()
|
||||
if strings.Contains(errMsg, "no such host") {
|
||||
result["ip"] = "域名无法解析"
|
||||
@@ -204,7 +204,24 @@ func handleGet(c *gin.Context, urlStr string, params map[string]interface{}) {
|
||||
c.JSON(200, result)
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// 如果有响应(包括重定向响应),继续处理
|
||||
if resp != nil {
|
||||
defer resp.Body.Close()
|
||||
} else {
|
||||
// 没有响应也没有错误,不应该发生
|
||||
result["error"] = "未知错误"
|
||||
result["ip"] = "访问失败"
|
||||
result["totaltime"] = "*"
|
||||
result["downtime"] = "*"
|
||||
result["downsize"] = "*"
|
||||
result["downspeed"] = "*"
|
||||
result["firstbytetime"] = "*"
|
||||
result["conntime"] = "*"
|
||||
result["size"] = "*"
|
||||
c.JSON(200, result)
|
||||
return
|
||||
}
|
||||
|
||||
// 获取时间信息
|
||||
timingTransport.mu.Lock()
|
||||
@@ -237,7 +254,7 @@ func handleGet(c *gin.Context, urlStr string, params map[string]interface{}) {
|
||||
bodyReader := io.LimitReader(resp.Body, 1024*1024) // 限制1MB
|
||||
bodyStartTime := time.Now()
|
||||
body, err := io.ReadAll(bodyReader)
|
||||
bodyReadTime := time.Now().Sub(bodyStartTime)
|
||||
bodyReadTime := time.Since(bodyStartTime)
|
||||
if err != nil && err != io.EOF {
|
||||
result["error"] = err.Error()
|
||||
}
|
||||
@@ -333,10 +350,8 @@ func handlePost(c *gin.Context, urlStr string, params map[string]interface{}) {
|
||||
Transport: timingTransport,
|
||||
Timeout: 15 * time.Second,
|
||||
CheckRedirect: func(req *http.Request, via []*http.Request) error {
|
||||
if len(via) >= 20 {
|
||||
return fmt.Errorf("重定向次数过多")
|
||||
}
|
||||
return nil
|
||||
// 不跟随重定向,返回第一个状态码和 header
|
||||
return http.ErrUseLastResponse
|
||||
},
|
||||
}
|
||||
|
||||
@@ -363,7 +378,11 @@ func handlePost(c *gin.Context, urlStr string, params map[string]interface{}) {
|
||||
// 执行请求
|
||||
startTime := time.Now()
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
|
||||
// 处理重定向错误:当 CheckRedirect 返回 ErrUseLastResponse 时,
|
||||
// client.Do 会返回响应和错误,但响应仍然有效(包含重定向状态码和 header)
|
||||
if err != nil && resp == nil {
|
||||
// 真正的错误,没有响应
|
||||
errMsg := err.Error()
|
||||
if strings.Contains(errMsg, "no such host") {
|
||||
result["ip"] = "域名无法解析"
|
||||
@@ -385,7 +404,24 @@ func handlePost(c *gin.Context, urlStr string, params map[string]interface{}) {
|
||||
c.JSON(200, result)
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// 如果有响应(包括重定向响应),继续处理
|
||||
if resp != nil {
|
||||
defer resp.Body.Close()
|
||||
} else {
|
||||
// 没有响应也没有错误,不应该发生
|
||||
result["error"] = "未知错误"
|
||||
result["ip"] = "访问失败"
|
||||
result["totaltime"] = "*"
|
||||
result["downtime"] = "*"
|
||||
result["downsize"] = "*"
|
||||
result["downspeed"] = "*"
|
||||
result["firstbytetime"] = "*"
|
||||
result["conntime"] = "*"
|
||||
result["size"] = "*"
|
||||
c.JSON(200, result)
|
||||
return
|
||||
}
|
||||
|
||||
// 获取时间信息
|
||||
timingTransport.mu.Lock()
|
||||
|
||||
@@ -16,27 +16,59 @@ func handleTCPing(c *gin.Context, url string, params map[string]interface{}) {
|
||||
seq = seqVal
|
||||
}
|
||||
|
||||
// 解析host:port格式
|
||||
parts := strings.Split(url, ":")
|
||||
if len(parts) != 2 {
|
||||
c.JSON(200, gin.H{
|
||||
"seq": seq,
|
||||
"type": "ceTCPing",
|
||||
"url": url,
|
||||
"error": "格式错误,需要 host:port",
|
||||
})
|
||||
return
|
||||
// 解析host:port格式,如果没有端口则默认80
|
||||
var host string
|
||||
var portStr string
|
||||
var port int
|
||||
|
||||
// 检查是否是IPv6格式(如 [::1]:8080)
|
||||
if strings.HasPrefix(url, "[") {
|
||||
// IPv6格式 - 使用 Index 而不是 LastIndex 来找到第一个闭合括号
|
||||
closeBracket := strings.Index(url, "]")
|
||||
if closeBracket == -1 {
|
||||
c.JSON(200, gin.H{
|
||||
"seq": seq,
|
||||
"type": "ceTCPing",
|
||||
"url": url,
|
||||
"error": "格式错误,IPv6地址格式应为 [host]:port",
|
||||
})
|
||||
return
|
||||
}
|
||||
host = url[1:closeBracket]
|
||||
if closeBracket+1 < len(url) && url[closeBracket+1] == ':' {
|
||||
portStr = url[closeBracket+2:]
|
||||
// 如果端口部分为空,使用默认端口80(修复 Bug 1)
|
||||
if portStr == "" {
|
||||
portStr = "80"
|
||||
}
|
||||
} else {
|
||||
portStr = "80" // 默认端口
|
||||
}
|
||||
} else {
|
||||
// 普通格式 host:port 或 host
|
||||
lastColonIndex := strings.LastIndex(url, ":")
|
||||
if lastColonIndex == -1 {
|
||||
// 没有冒号,使用默认端口80
|
||||
host = url
|
||||
portStr = "80"
|
||||
} else {
|
||||
host = url[:lastColonIndex]
|
||||
portStr = url[lastColonIndex+1:]
|
||||
// 如果端口部分为空,使用默认端口80
|
||||
if portStr == "" {
|
||||
portStr = "80"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
host := parts[0]
|
||||
portStr := parts[1]
|
||||
port, err := strconv.Atoi(portStr)
|
||||
var err error
|
||||
port, err = strconv.Atoi(portStr)
|
||||
if err != nil {
|
||||
c.JSON(200, gin.H{
|
||||
"seq": seq,
|
||||
"type": "ceTCPing",
|
||||
"url": url,
|
||||
"error": "端口格式错误",
|
||||
"seq": seq,
|
||||
"type": "ceTCPing",
|
||||
"url": url,
|
||||
"error": "端口格式错误",
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -131,12 +163,12 @@ func handleTCPing(c *gin.Context, url string, params map[string]interface{}) {
|
||||
|
||||
// 返回格式和PING一致
|
||||
result := gin.H{
|
||||
"seq": seq,
|
||||
"type": "ceTCPing",
|
||||
"url": url,
|
||||
"ip": primaryIP,
|
||||
"host": host,
|
||||
"port": port,
|
||||
"seq": seq,
|
||||
"type": "ceTCPing",
|
||||
"url": url,
|
||||
"ip": primaryIP,
|
||||
"host": host,
|
||||
"port": port,
|
||||
"packets_total": strconv.Itoa(packetsTotal),
|
||||
"packets_recv": strconv.Itoa(packetsRecv),
|
||||
"packets_losrat": packetsLosrat, // float64类型,百分比值(如10.5表示10.5%)
|
||||
@@ -160,4 +192,3 @@ func handleTCPing(c *gin.Context, url string, params map[string]interface{}) {
|
||||
|
||||
c.JSON(200, result)
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,8 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"os"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
@@ -18,13 +20,13 @@ import (
|
||||
// 节点信息存储(通过心跳更新,优先从配置文件读取)
|
||||
var nodeInfo struct {
|
||||
sync.RWMutex
|
||||
nodeID uint
|
||||
nodeIP string
|
||||
country string
|
||||
province string
|
||||
city string
|
||||
isp string
|
||||
cfg *config.Config
|
||||
nodeID uint
|
||||
nodeIP string
|
||||
country string
|
||||
province string
|
||||
city string
|
||||
isp string
|
||||
cfg *config.Config
|
||||
initialized bool
|
||||
}
|
||||
|
||||
@@ -110,10 +112,25 @@ func (r *Reporter) Stop() {
|
||||
close(r.stopCh)
|
||||
}
|
||||
|
||||
// buildHeartbeatBody 构建心跳请求体
|
||||
func buildHeartbeatBody() string {
|
||||
hostname, err := os.Hostname()
|
||||
if err != nil {
|
||||
hostname = "unknown"
|
||||
}
|
||||
|
||||
values := url.Values{}
|
||||
values.Set("type", "pingServer")
|
||||
values.Set("version", "2")
|
||||
values.Set("host_name", hostname)
|
||||
|
||||
return values.Encode()
|
||||
}
|
||||
|
||||
// RegisterNode 注册节点(安装时或首次启动时调用)
|
||||
func RegisterNode(cfg *config.Config) error {
|
||||
url := fmt.Sprintf("%s/api/node/heartbeat", cfg.Backend.URL)
|
||||
req, err := http.NewRequest("POST", url, bytes.NewBufferString("type=pingServer"))
|
||||
req, err := http.NewRequest("POST", url, bytes.NewBufferString(buildHeartbeatBody()))
|
||||
if err != nil {
|
||||
return fmt.Errorf("创建心跳请求失败: %w", err)
|
||||
}
|
||||
@@ -123,7 +140,7 @@ func RegisterNode(cfg *config.Config) error {
|
||||
client := &http.Client{Timeout: 10 * time.Second}
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("发送心跳失败: %w", err)
|
||||
return fmt.Errorf("发送心跳失败 (URL: %s): %w", url, err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
@@ -173,16 +190,27 @@ func RegisterNode(cfg *config.Config) error {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("心跳响应格式无效或节点信息不完整")
|
||||
return fmt.Errorf("心跳响应格式无效或节点信息不完整 (响应体: %s)", string(body))
|
||||
}
|
||||
|
||||
return fmt.Errorf("心跳请求失败,状态码: %d", resp.StatusCode)
|
||||
// 读取响应体以便记录错误详情
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
bodyStr := ""
|
||||
if err == nil && len(body) > 0 {
|
||||
// 限制响应体长度,避免错误信息过长
|
||||
if len(body) > 500 {
|
||||
bodyStr = string(body[:500]) + "..."
|
||||
} else {
|
||||
bodyStr = string(body)
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("心跳请求失败,状态码: %d, URL: %s, 响应体: %s", resp.StatusCode, url, bodyStr)
|
||||
}
|
||||
|
||||
func (r *Reporter) sendHeartbeat() {
|
||||
// 发送心跳(使用Form格式,兼容旧接口)
|
||||
url := fmt.Sprintf("%s/api/node/heartbeat", r.cfg.Backend.URL)
|
||||
req, err := http.NewRequest("POST", url, bytes.NewBufferString("type=pingServer"))
|
||||
req, err := http.NewRequest("POST", url, bytes.NewBufferString(buildHeartbeatBody()))
|
||||
if err != nil {
|
||||
r.logger.Error("创建心跳请求失败", zap.Error(err))
|
||||
return
|
||||
@@ -192,7 +220,9 @@ func (r *Reporter) sendHeartbeat() {
|
||||
|
||||
resp, err := r.client.Do(req)
|
||||
if err != nil {
|
||||
r.logger.Warn("发送心跳失败", zap.Error(err))
|
||||
r.logger.Warn("发送心跳失败",
|
||||
zap.String("url", url),
|
||||
zap.Error(err))
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
@@ -260,7 +290,21 @@ func (r *Reporter) sendHeartbeat() {
|
||||
}
|
||||
r.logger.Debug("心跳发送成功")
|
||||
} else {
|
||||
r.logger.Warn("心跳发送失败", zap.Int("status", resp.StatusCode))
|
||||
// 读取响应体以便记录错误详情
|
||||
body, err := io.ReadAll(resp.Body)
|
||||
bodyStr := ""
|
||||
if err == nil && len(body) > 0 {
|
||||
// 限制响应体长度,避免日志过长
|
||||
if len(body) > 500 {
|
||||
bodyStr = string(body[:500]) + "..."
|
||||
} else {
|
||||
bodyStr = string(body)
|
||||
}
|
||||
}
|
||||
|
||||
r.logger.Warn("心跳发送失败",
|
||||
zap.Int("status", resp.StatusCode),
|
||||
zap.String("url", url),
|
||||
zap.String("response_body", bodyStr))
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
6
run.sh
6
run.sh
@@ -191,12 +191,14 @@ start() {
|
||||
|
||||
echo -e "${BLUE}启动节点端服务...${NC}"
|
||||
echo -e "${BLUE}后端地址: $BACKEND_URL${NC}"
|
||||
echo -e "${BLUE}日志文件: $LOG_FILE${NC}"
|
||||
|
||||
# 设置环境变量
|
||||
export BACKEND_URL="$BACKEND_URL"
|
||||
export LOG_FILE="$LOG_FILE"
|
||||
|
||||
# 后台运行
|
||||
nohup ./"$BINARY_NAME" > "$LOG_FILE" 2>&1 &
|
||||
# 后台运行(日志现在由程序直接写入文件,这里保留重定向作为备份)
|
||||
nohup ./"$BINARY_NAME" >> "$LOG_FILE" 2>&1 &
|
||||
NEW_PID=$!
|
||||
|
||||
# 保存PID
|
||||
|
||||
@@ -15,11 +15,38 @@ cd "$SCRIPT_DIR"
|
||||
BINARY_NAME="agent"
|
||||
BACKEND_URL="${BACKEND_URL:-http://localhost:8080}"
|
||||
|
||||
# 检查二进制文件是否存在且有效
|
||||
check_binary() {
|
||||
if [ -f "$BINARY_NAME" ] && [ -x "$BINARY_NAME" ] && [ -s "$BINARY_NAME" ]; then
|
||||
# 验证二进制文件类型(Linux 应该是 ELF 文件)
|
||||
if command -v file > /dev/null 2>&1; then
|
||||
local file_type=$(file "$BINARY_NAME" 2>/dev/null || echo "")
|
||||
if [ -n "$file_type" ] && echo "$file_type" | grep -qi "ELF"; then
|
||||
return 0
|
||||
fi
|
||||
else
|
||||
# 如果没有 file 命令,至少检查文件大小(应该大于 1MB)
|
||||
local file_size=$(stat -c%s "$BINARY_NAME" 2>/dev/null || stat -f%z "$BINARY_NAME" 2>/dev/null || echo "0")
|
||||
if [ "$file_size" -gt 1048576 ]; then
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
return 1
|
||||
}
|
||||
|
||||
# 拉取最新源码并编译
|
||||
update_and_build() {
|
||||
# 如果二进制文件已存在且有效,跳过编译
|
||||
if check_binary; then
|
||||
echo "二进制文件已存在,跳过编译步骤"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# 检查是否在 Git 仓库中
|
||||
if [ ! -d ".git" ]; then
|
||||
return 0
|
||||
echo "错误: 不在 Git 仓库中,无法更新代码" >&2
|
||||
return 1
|
||||
fi
|
||||
|
||||
# 配置 Git safe.directory,解决所有权问题
|
||||
@@ -31,14 +58,41 @@ update_and_build() {
|
||||
echo "代码更新完成"
|
||||
fi
|
||||
|
||||
# 确保 Go 在 PATH 中(systemd 可能没有设置 PATH)
|
||||
if [ -d "/usr/local/go/bin" ] && ! echo "$PATH" | grep -q "/usr/local/go/bin"; then
|
||||
export PATH="/usr/local/go/bin:$PATH"
|
||||
fi
|
||||
|
||||
# 检查 Go 环境
|
||||
if ! command -v go > /dev/null 2>&1; then
|
||||
echo "错误: 未找到 Go 环境,无法编译" >&2
|
||||
echo "PATH: $PATH" >&2
|
||||
echo "请确保 Go 已安装: /usr/local/go/bin 或系统 PATH 中包含 go 命令" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 更新依赖
|
||||
go mod download 2>&1 > /dev/null || true
|
||||
# 检查是否有 vendor 目录
|
||||
local use_vendor=false
|
||||
if [ -d "./vendor" ] && [ -f "./vendor/modules.txt" ]; then
|
||||
use_vendor=true
|
||||
echo "使用 vendor 目录编译(无需网络连接)"
|
||||
fi
|
||||
|
||||
# 更新依赖(仅在非 vendor 模式下,并设置超时避免卡住)
|
||||
if [ "$use_vendor" != "true" ]; then
|
||||
echo "更新 Go 依赖..."
|
||||
# 使用 timeout 命令设置超时(30秒),如果系统没有 timeout 命令则直接执行
|
||||
if command -v timeout > /dev/null 2>&1; then
|
||||
timeout 30 go mod download 2>&1 > /dev/null || {
|
||||
echo "警告: 依赖更新失败或超时,尝试继续编译" >&2
|
||||
}
|
||||
else
|
||||
# 如果没有 timeout 命令,直接执行(可能卡住,但至少能工作)
|
||||
go mod download 2>&1 > /dev/null || {
|
||||
echo "警告: 依赖更新失败,尝试继续编译" >&2
|
||||
}
|
||||
fi
|
||||
fi
|
||||
|
||||
# 编译
|
||||
ARCH=$(uname -m)
|
||||
@@ -54,21 +108,33 @@ update_and_build() {
|
||||
;;
|
||||
esac
|
||||
|
||||
if GOOS=linux GOARCH=${ARCH} CGO_ENABLED=0 go build -buildvcs=false -ldflags="-w -s" -o "$BINARY_NAME" ./cmd/agent 2>&1; then
|
||||
# 构建编译命令
|
||||
local build_cmd="GOOS=linux GOARCH=${ARCH} CGO_ENABLED=0 go build -buildvcs=false -ldflags=\"-w -s\""
|
||||
if [ "$use_vendor" = "true" ]; then
|
||||
build_cmd="$build_cmd -mod=vendor"
|
||||
fi
|
||||
build_cmd="$build_cmd -o \"$BINARY_NAME\" ./cmd/agent"
|
||||
|
||||
echo "开始编译(架构: ${ARCH})..."
|
||||
if eval "$build_cmd" 2>&1; then
|
||||
if [ -f "$BINARY_NAME" ] && [ -s "$BINARY_NAME" ]; then
|
||||
chmod +x "$BINARY_NAME"
|
||||
echo "编译成功"
|
||||
return 0
|
||||
else
|
||||
echo "错误: 编译失败,未生成二进制文件" >&2
|
||||
exit 1
|
||||
return 1
|
||||
fi
|
||||
else
|
||||
echo "错误: 编译失败" >&2
|
||||
exit 1
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 拉取最新源码并编译
|
||||
update_and_build
|
||||
# 检查并更新/编译
|
||||
if ! check_binary; then
|
||||
update_and_build
|
||||
fi
|
||||
|
||||
# 设置环境变量
|
||||
export BACKEND_URL="$BACKEND_URL"
|
||||
|
||||
318
uninstall.sh
Executable file
318
uninstall.sh
Executable file
@@ -0,0 +1,318 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ============================================
|
||||
# LinkMaster 节点端一键卸载脚本
|
||||
# 使用方法: curl -fsSL https://gitee.nas.cpolar.cn/yoyo/linkmaster-node/raw/branch/main/uninstall.sh | bash
|
||||
# 或: ./uninstall.sh
|
||||
# ============================================
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色输出
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
# 配置
|
||||
BINARY_NAME="linkmaster-node"
|
||||
INSTALL_DIR="/usr/local/bin"
|
||||
SERVICE_NAME="linkmaster-node"
|
||||
SOURCE_DIR="/opt/linkmaster-node"
|
||||
CONFIG_FILE="${SOURCE_DIR}/config.yaml"
|
||||
LOG_FILE="${SOURCE_DIR}/node.log"
|
||||
PID_FILE="${SOURCE_DIR}/node.pid"
|
||||
|
||||
# 检查是否已安装
|
||||
check_installed() {
|
||||
local installed=false
|
||||
|
||||
# 检查服务文件是否存在
|
||||
if [ -f "/etc/systemd/system/${SERVICE_NAME}.service" ]; then
|
||||
installed=true
|
||||
fi
|
||||
|
||||
# 检查二进制文件是否存在
|
||||
if [ -f "$INSTALL_DIR/$BINARY_NAME" ]; then
|
||||
installed=true
|
||||
fi
|
||||
|
||||
# 检查源码目录是否存在
|
||||
if [ -d "$SOURCE_DIR" ]; then
|
||||
installed=true
|
||||
fi
|
||||
|
||||
if [ "$installed" = false ]; then
|
||||
return 1
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
# 停止服务
|
||||
stop_service() {
|
||||
echo -e "${BLUE}停止服务...${NC}"
|
||||
|
||||
# 停止 systemd 服务
|
||||
if systemctl is-active --quiet ${SERVICE_NAME} 2>/dev/null; then
|
||||
echo -e "${BLUE} 正在停止 systemd 服务...${NC}"
|
||||
sudo systemctl stop ${SERVICE_NAME} 2>/dev/null || true
|
||||
sleep 2
|
||||
fi
|
||||
|
||||
# 清理残留进程
|
||||
if pgrep -f "$BINARY_NAME" > /dev/null 2>&1; then
|
||||
echo -e "${BLUE} 清理残留进程...${NC}"
|
||||
sudo pkill -f "$BINARY_NAME" 2>/dev/null || true
|
||||
sleep 1
|
||||
|
||||
# 再次检查,如果还有进程则强制杀死
|
||||
if pgrep -f "$BINARY_NAME" > /dev/null 2>&1; then
|
||||
echo -e "${YELLOW} 强制清理残留进程...${NC}"
|
||||
sudo pkill -9 -f "$BINARY_NAME" 2>/dev/null || true
|
||||
sleep 1
|
||||
fi
|
||||
fi
|
||||
|
||||
# 如果使用 run.sh 启动的进程(通过 PID 文件)
|
||||
if [ -f "$PID_FILE" ]; then
|
||||
local pid=$(cat "$PID_FILE" 2>/dev/null || echo "")
|
||||
if [ -n "$pid" ] && kill -0 "$pid" 2>/dev/null; then
|
||||
echo -e "${BLUE} 停止通过 run.sh 启动的进程 (PID: $pid)...${NC}"
|
||||
kill "$pid" 2>/dev/null || true
|
||||
sleep 1
|
||||
if kill -0 "$pid" 2>/dev/null; then
|
||||
kill -9 "$pid" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
echo -e "${GREEN}✓ 服务已停止${NC}"
|
||||
}
|
||||
|
||||
# 禁用服务
|
||||
disable_service() {
|
||||
echo -e "${BLUE}禁用服务...${NC}"
|
||||
|
||||
if systemctl is-enabled --quiet ${SERVICE_NAME} 2>/dev/null; then
|
||||
sudo systemctl disable ${SERVICE_NAME} 2>/dev/null || true
|
||||
echo -e "${GREEN}✓ 服务已禁用${NC}"
|
||||
else
|
||||
echo -e "${BLUE} 服务未启用,跳过${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 删除服务文件
|
||||
remove_service_files() {
|
||||
echo -e "${BLUE}删除服务文件...${NC}"
|
||||
|
||||
local removed=false
|
||||
|
||||
# 删除 systemd 服务文件
|
||||
if [ -f "/etc/systemd/system/${SERVICE_NAME}.service" ]; then
|
||||
sudo rm -f /etc/systemd/system/${SERVICE_NAME}.service
|
||||
echo -e "${GREEN}✓ 已删除服务文件: /etc/systemd/system/${SERVICE_NAME}.service${NC}"
|
||||
removed=true
|
||||
fi
|
||||
|
||||
# 删除可能的 override 配置目录
|
||||
if [ -d "/etc/systemd/system/${SERVICE_NAME}.service.d" ]; then
|
||||
sudo rm -rf /etc/systemd/system/${SERVICE_NAME}.service.d
|
||||
echo -e "${GREEN}✓ 已删除服务配置目录: /etc/systemd/system/${SERVICE_NAME}.service.d${NC}"
|
||||
removed=true
|
||||
fi
|
||||
|
||||
if [ "$removed" = true ]; then
|
||||
# 重新加载 systemd daemon
|
||||
sudo systemctl daemon-reload
|
||||
echo -e "${GREEN}✓ systemd daemon 已重新加载${NC}"
|
||||
else
|
||||
echo -e "${BLUE} 未找到服务文件,跳过${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 删除二进制文件
|
||||
remove_binary() {
|
||||
echo -e "${BLUE}删除二进制文件...${NC}"
|
||||
|
||||
if [ -f "$INSTALL_DIR/$BINARY_NAME" ]; then
|
||||
sudo rm -f "$INSTALL_DIR/$BINARY_NAME"
|
||||
echo -e "${GREEN}✓ 已删除二进制文件: $INSTALL_DIR/$BINARY_NAME${NC}"
|
||||
else
|
||||
echo -e "${BLUE} 未找到二进制文件,跳过${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 删除源码目录和配置文件
|
||||
remove_source_directory() {
|
||||
echo -e "${BLUE}删除源码目录和配置文件...${NC}"
|
||||
|
||||
if [ -d "$SOURCE_DIR" ]; then
|
||||
# 显示将要删除的内容
|
||||
echo -e "${BLUE} 源码目录: $SOURCE_DIR${NC}"
|
||||
|
||||
# 删除整个源码目录
|
||||
sudo rm -rf "$SOURCE_DIR"
|
||||
echo -e "${GREEN}✓ 已删除源码目录: $SOURCE_DIR${NC}"
|
||||
else
|
||||
echo -e "${BLUE} 未找到源码目录,跳过${NC}"
|
||||
fi
|
||||
|
||||
# 额外检查配置文件(如果单独存在)
|
||||
if [ -f "$CONFIG_FILE" ]; then
|
||||
sudo rm -f "$CONFIG_FILE"
|
||||
echo -e "${GREEN}✓ 已删除配置文件: $CONFIG_FILE${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 清理防火墙规则(可选,默认不删除)
|
||||
remove_firewall_rules() {
|
||||
local remove_firewall="${1:-false}"
|
||||
|
||||
if [ "$remove_firewall" != "true" ]; then
|
||||
echo -e "${BLUE}跳过防火墙规则清理(默认保留)${NC}"
|
||||
echo -e "${YELLOW} 如需删除防火墙规则,请使用: ./uninstall.sh --remove-firewall${NC}"
|
||||
return
|
||||
fi
|
||||
|
||||
echo -e "${BLUE}清理防火墙规则(开放2200端口)...${NC}"
|
||||
|
||||
PORT=2200
|
||||
FIREWALL_REMOVED=false
|
||||
|
||||
# 检测 firewalld (CentOS/RHEL 7+, Fedora)
|
||||
if command -v firewall-cmd > /dev/null 2>&1; then
|
||||
if sudo systemctl is-active --quiet firewalld 2>/dev/null; then
|
||||
echo -e "${BLUE} 从 firewalld 删除端口规则...${NC}"
|
||||
if sudo firewall-cmd --permanent --remove-port=${PORT}/tcp > /dev/null 2>&1; then
|
||||
sudo firewall-cmd --reload > /dev/null 2>&1
|
||||
echo -e "${GREEN}✓ firewalld 规则已删除${NC}"
|
||||
FIREWALL_REMOVED=true
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# 检测 ufw (Ubuntu/Debian)
|
||||
if command -v ufw > /dev/null 2>&1; then
|
||||
if sudo ufw status > /dev/null 2>&1; then
|
||||
echo -e "${BLUE} 从 ufw 删除端口规则...${NC}"
|
||||
if sudo ufw delete allow ${PORT}/tcp > /dev/null 2>&1; then
|
||||
echo -e "${GREEN}✓ ufw 规则已删除${NC}"
|
||||
FIREWALL_REMOVED=true
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
# 检测 iptables(较老的系统)
|
||||
if command -v iptables > /dev/null 2>&1 && [ "$FIREWALL_REMOVED" = false ]; then
|
||||
if sudo iptables -C INPUT -p tcp --dport ${PORT} -j ACCEPT > /dev/null 2>&1; then
|
||||
echo -e "${BLUE} 从 iptables 删除端口规则...${NC}"
|
||||
if sudo iptables -D INPUT -p tcp --dport ${PORT} -j ACCEPT > /dev/null 2>&1; then
|
||||
echo -e "${GREEN}✓ iptables 规则已删除${NC}"
|
||||
# 尝试保存规则
|
||||
if command -v iptables-save > /dev/null 2>&1; then
|
||||
if [ -f /etc/redhat-release ]; then
|
||||
sudo iptables-save > /etc/sysconfig/iptables 2>/dev/null || true
|
||||
elif [ -f /etc/debian_version ]; then
|
||||
sudo iptables-save > /etc/iptables/rules.v4 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
FIREWALL_REMOVED=true
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$FIREWALL_REMOVED" = false ]; then
|
||||
echo -e "${YELLOW}⚠ 未找到防火墙规则或防火墙未启用${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 显示卸载摘要
|
||||
show_summary() {
|
||||
echo ""
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN} 卸载完成!${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${BLUE}已删除的内容:${NC}"
|
||||
echo " - systemd 服务文件"
|
||||
echo " - 二进制文件: $INSTALL_DIR/$BINARY_NAME"
|
||||
echo " - 源码目录: $SOURCE_DIR"
|
||||
echo ""
|
||||
echo -e "${YELLOW}注意:${NC}"
|
||||
echo " - 防火墙规则已保留(如需删除请使用 --remove-firewall 参数)"
|
||||
echo " - 如果手动修改过系统配置,请手动清理"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# 主卸载流程
|
||||
main() {
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN} LinkMaster 节点端卸载程序${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo ""
|
||||
|
||||
# 检查是否已安装
|
||||
if ! check_installed; then
|
||||
echo -e "${YELLOW}未检测到已安装的 LinkMaster 节点端${NC}"
|
||||
echo -e "${YELLOW}无需卸载${NC}"
|
||||
exit 0
|
||||
fi
|
||||
|
||||
# 检查是否需要删除防火墙规则
|
||||
REMOVE_FIREWALL=false
|
||||
if [ "$1" = "--remove-firewall" ]; then
|
||||
REMOVE_FIREWALL=true
|
||||
fi
|
||||
|
||||
# 确认卸载
|
||||
echo -e "${YELLOW}即将卸载 LinkMaster 节点端,此操作将:${NC}"
|
||||
echo " - 停止并禁用服务"
|
||||
echo " - 删除 systemd 服务文件"
|
||||
echo " - 删除二进制文件"
|
||||
echo " - 删除源码目录和配置文件"
|
||||
echo ""
|
||||
|
||||
# 如果在非交互式环境中,自动确认
|
||||
if [ -t 0 ]; then
|
||||
read -p "是否继续?(y/N): " -n 1 -r
|
||||
echo ""
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
echo -e "${YELLOW}已取消卸载${NC}"
|
||||
exit 0
|
||||
fi
|
||||
else
|
||||
echo -e "${BLUE}非交互式环境,自动确认卸载${NC}"
|
||||
fi
|
||||
|
||||
echo ""
|
||||
|
||||
# 执行卸载步骤
|
||||
stop_service
|
||||
disable_service
|
||||
remove_service_files
|
||||
remove_binary
|
||||
remove_source_directory
|
||||
remove_firewall_rules "$REMOVE_FIREWALL"
|
||||
|
||||
# 最终检查
|
||||
echo ""
|
||||
echo -e "${BLUE}最终检查...${NC}"
|
||||
|
||||
local still_installed=false
|
||||
if check_installed; then
|
||||
still_installed=true
|
||||
fi
|
||||
|
||||
if [ "$still_installed" = false ]; then
|
||||
show_summary
|
||||
exit 0
|
||||
else
|
||||
echo -e "${YELLOW}⚠ 部分文件可能未完全删除,请手动检查${NC}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 执行卸载
|
||||
main "$@"
|
||||
|
||||
220
vendor.sh
Executable file
220
vendor.sh
Executable file
@@ -0,0 +1,220 @@
|
||||
#!/bin/bash
|
||||
|
||||
# ============================================
|
||||
# LinkMaster Node Vendor 依赖打包脚本
|
||||
# 用途:将项目依赖下载到 vendor 目录,客户端克隆后可直接编译,无需网络
|
||||
# 使用方法: ./vendor.sh
|
||||
# ============================================
|
||||
|
||||
set -e
|
||||
|
||||
# 颜色输出
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
BLUE='\033[0;34m'
|
||||
NC='\033[0m'
|
||||
|
||||
# 脚本目录
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
# 检查 Go 环境
|
||||
check_go() {
|
||||
if ! command -v go > /dev/null 2>&1; then
|
||||
echo -e "${RED}错误: 未找到 Go 环境${NC}"
|
||||
echo -e "${YELLOW}请先安装 Go: https://golang.org/dl/${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
GO_VERSION=$(go version 2>/dev/null | head -1)
|
||||
echo -e "${GREEN}✓ 检测到 Go: ${GO_VERSION}${NC}"
|
||||
}
|
||||
|
||||
# 配置 Go 环境(海外网络,直接使用官方源)
|
||||
setup_go_env() {
|
||||
echo -e "${BLUE}配置 Go 环境(使用官方源)...${NC}"
|
||||
|
||||
# 海外网络可以直接使用官方源,无需代理
|
||||
# 使用官方 Go 代理和校验和数据库
|
||||
export GOPROXY="https://proxy.golang.org,direct"
|
||||
export GOSUMDB="sum.golang.org"
|
||||
|
||||
# 使用 go env -w 永久设置
|
||||
go env -w GOPROXY="https://proxy.golang.org,direct" 2>/dev/null || true
|
||||
go env -w GOSUMDB="sum.golang.org" 2>/dev/null || true
|
||||
|
||||
echo -e "${GREEN}✓ Go 环境配置完成(使用官方源)${NC}"
|
||||
echo -e "${BLUE} GOPROXY=https://proxy.golang.org,direct${NC}"
|
||||
echo -e "${BLUE} GOSUMDB=sum.golang.org${NC}"
|
||||
}
|
||||
|
||||
# 下载依赖
|
||||
download_dependencies() {
|
||||
echo -e "${BLUE}下载 Go 依赖包...${NC}"
|
||||
|
||||
# 先整理依赖
|
||||
echo -e "${BLUE}整理 Go 模块依赖...${NC}"
|
||||
if ! go mod tidy -v 2>&1; then
|
||||
echo -e "${YELLOW}⚠ go mod tidy 失败,继续尝试下载依赖...${NC}"
|
||||
fi
|
||||
|
||||
# 下载依赖
|
||||
echo -e "${BLUE}下载所有依赖包(这可能需要一些时间)...${NC}"
|
||||
DOWNLOAD_START_TIME=$(date +%s)
|
||||
|
||||
if go mod download -x 2>&1; then
|
||||
DOWNLOAD_END_TIME=$(date +%s)
|
||||
DOWNLOAD_DURATION=$((DOWNLOAD_END_TIME - DOWNLOAD_START_TIME))
|
||||
echo -e "${GREEN}✓ 依赖下载完成 (耗时: ${DOWNLOAD_DURATION}秒)${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ 依赖下载失败${NC}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 创建 vendor 目录
|
||||
create_vendor() {
|
||||
echo -e "${BLUE}创建 vendor 目录(打包所有依赖)...${NC}"
|
||||
|
||||
# 删除旧的 vendor 目录(如果存在)
|
||||
if [ -d "vendor" ]; then
|
||||
echo -e "${YELLOW}清理旧的 vendor 目录...${NC}"
|
||||
rm -rf vendor
|
||||
fi
|
||||
|
||||
# 创建 vendor 目录
|
||||
if go mod vendor -v 2>&1; then
|
||||
VENDOR_COUNT=$(find vendor -type d -mindepth 2 2>/dev/null | wc -l | tr -d '\n' || echo "0")
|
||||
VENDOR_COUNT=${VENDOR_COUNT:-0}
|
||||
|
||||
# 计算 vendor 目录大小
|
||||
VENDOR_SIZE=$(du -sh vendor 2>/dev/null | cut -f1 || echo "未知")
|
||||
|
||||
echo -e "${GREEN}✓ vendor 目录创建成功${NC}"
|
||||
echo -e "${BLUE}统计信息:${NC}"
|
||||
echo -e " 依赖包数量: ${GREEN}${VENDOR_COUNT}${NC}"
|
||||
echo -e " 目录大小: ${GREEN}${VENDOR_SIZE}${NC}"
|
||||
else
|
||||
echo -e "${RED}✗ vendor 创建失败${NC}"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
# 更新 .gitignore
|
||||
update_gitignore() {
|
||||
echo -e "${BLUE}更新 .gitignore...${NC}"
|
||||
|
||||
if [ ! -f ".gitignore" ]; then
|
||||
echo -e "${BLUE}创建 .gitignore 文件...${NC}"
|
||||
touch .gitignore
|
||||
fi
|
||||
|
||||
# 检查 .gitignore 是否忽略了 vendor
|
||||
if grep -q "^vendor$" .gitignore 2>/dev/null || grep -q "^vendor/" .gitignore 2>/dev/null; then
|
||||
echo -e "${YELLOW}⚠ 检测到 .gitignore 中忽略了 vendor 目录${NC}"
|
||||
echo -e "${BLUE}正在更新 .gitignore,允许 vendor 目录被提交...${NC}"
|
||||
# 移除 vendor 相关的忽略规则
|
||||
sed -i.bak '/^vendor$/d; /^vendor\//d' .gitignore 2>/dev/null || \
|
||||
sed -i '' '/^vendor$/d; /^vendor\//d' .gitignore 2>/dev/null || true
|
||||
rm -f .gitignore.bak 2>/dev/null || true
|
||||
echo -e "${GREEN}✓ 已更新 .gitignore${NC}"
|
||||
else
|
||||
echo -e "${GREEN}✓ .gitignore 配置正确${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 添加到 Git(可选)
|
||||
add_to_git() {
|
||||
if [ -d ".git" ]; then
|
||||
echo -e "${BLUE}将 vendor 目录添加到 Git...${NC}"
|
||||
|
||||
# 检查是否有未提交的更改
|
||||
if git status --porcelain vendor 2>/dev/null | grep -q vendor; then
|
||||
git add vendor/ 2>/dev/null || true
|
||||
echo -e "${GREEN}✓ vendor 目录已添加到 Git 暂存区${NC}"
|
||||
echo -e "${YELLOW}提示: 请运行 'git commit -m \"chore: add vendor dependencies\"' 提交 vendor 目录${NC}"
|
||||
else
|
||||
echo -e "${BLUE}vendor 目录已在 Git 中${NC}"
|
||||
fi
|
||||
else
|
||||
echo -e "${YELLOW}⚠ 当前目录不是 Git 仓库,跳过 Git 操作${NC}"
|
||||
fi
|
||||
}
|
||||
|
||||
# 显示进度显示
|
||||
show_progress() {
|
||||
echo -e "${BLUE}正在下载依赖包,显示进度...${NC}"
|
||||
|
||||
# 使用后台进程显示进度
|
||||
(
|
||||
DOWNLOADED_COUNT=0
|
||||
LAST_COUNT=0
|
||||
LAST_PACKAGE=""
|
||||
START_TIME=$(date +%s)
|
||||
|
||||
while true; do
|
||||
sleep 1
|
||||
|
||||
# 从 go mod download 的输出中统计
|
||||
CURRENT_COUNT=$(ps aux | grep -c "[g]o mod download" || echo "0")
|
||||
|
||||
# 检查进程是否还在运行
|
||||
if ! pgrep -f "go mod download" > /dev/null 2>&1; then
|
||||
break
|
||||
fi
|
||||
|
||||
CURRENT_TIME=$(date +%s)
|
||||
ELAPSED=$((CURRENT_TIME - START_TIME))
|
||||
|
||||
if [ "$ELAPSED" -gt 0 ]; then
|
||||
echo -ne "\r${BLUE}进度: 下载中... | 耗时: ${ELAPSED}秒 "
|
||||
fi
|
||||
done
|
||||
) &
|
||||
PROGRESS_PID=$!
|
||||
|
||||
# 等待下载完成
|
||||
wait $PROGRESS_PID 2>/dev/null || true
|
||||
}
|
||||
|
||||
# 主函数
|
||||
main() {
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN} LinkMaster Node Vendor 依赖打包工具${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo ""
|
||||
|
||||
# 检查是否在项目根目录
|
||||
if [ ! -f "go.mod" ]; then
|
||||
echo -e "${RED}错误: 未找到 go.mod 文件${NC}"
|
||||
echo -e "${YELLOW}请在项目根目录运行此脚本${NC}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
check_go
|
||||
setup_go_env
|
||||
download_dependencies
|
||||
create_vendor
|
||||
update_gitignore
|
||||
add_to_git
|
||||
|
||||
echo ""
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo -e "${GREEN} ✓ Vendor 依赖打包完成!${NC}"
|
||||
echo -e "${GREEN}========================================${NC}"
|
||||
echo ""
|
||||
echo -e "${BLUE}下一步操作:${NC}"
|
||||
echo " 1. 检查 vendor 目录: ls -lh vendor/"
|
||||
echo " 2. 提交到 Git: git add vendor/ && git commit -m \"chore: add vendor dependencies\""
|
||||
echo " 3. 推送到远程: git push"
|
||||
echo ""
|
||||
echo -e "${BLUE}客户端使用:${NC}"
|
||||
echo " 客户端克隆项目后,可以使用以下命令编译(无需网络):"
|
||||
echo " go build -mod=vendor -o agent ./cmd/agent"
|
||||
echo ""
|
||||
}
|
||||
|
||||
# 执行主函数
|
||||
main
|
||||
|
||||
52
vendor/github.com/bytedance/sonic/.gitignore
generated
vendored
Normal file
52
vendor/github.com/bytedance/sonic/.gitignore
generated
vendored
Normal file
@@ -0,0 +1,52 @@
|
||||
*.o
|
||||
*.swp
|
||||
*.swm
|
||||
*.swn
|
||||
*.a
|
||||
*.so
|
||||
_obj
|
||||
_test
|
||||
*.[568vq]
|
||||
[568vq].out
|
||||
*.cgo1.go
|
||||
*.cgo2.c
|
||||
_cgo_defun.c
|
||||
_cgo_gotypes.go
|
||||
_cgo_export.*
|
||||
_testmain.go
|
||||
*.exe
|
||||
*.exe~
|
||||
*.test
|
||||
*.prof
|
||||
*.rar
|
||||
*.zip
|
||||
*.gz
|
||||
*.psd
|
||||
*.bmd
|
||||
*.cfg
|
||||
*.pptx
|
||||
*.log
|
||||
*nohup.out
|
||||
*settings.pyc
|
||||
*.sublime-project
|
||||
*.sublime-workspace
|
||||
.DS_Store
|
||||
/.idea/
|
||||
/.vscode/
|
||||
/output/
|
||||
/vendor/
|
||||
/Gopkg.lock
|
||||
/Gopkg.toml
|
||||
coverage.html
|
||||
coverage.out
|
||||
coverage.xml
|
||||
junit.xml
|
||||
*.profile
|
||||
*.svg
|
||||
*.out
|
||||
ast/test.out
|
||||
ast/bench.sh
|
||||
|
||||
!testdata/*.json.gz
|
||||
fuzz/testdata
|
||||
*__debug_bin
|
||||
3
vendor/github.com/bytedance/sonic/.gitmodules
generated
vendored
Normal file
3
vendor/github.com/bytedance/sonic/.gitmodules
generated
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
[submodule "tools/asm2asm"]
|
||||
path = tools/asm2asm
|
||||
url = https://github.com/chenzhuoyu/asm2asm
|
||||
24
vendor/github.com/bytedance/sonic/.licenserc.yaml
generated
vendored
Normal file
24
vendor/github.com/bytedance/sonic/.licenserc.yaml
generated
vendored
Normal file
@@ -0,0 +1,24 @@
|
||||
header:
|
||||
license:
|
||||
spdx-id: Apache-2.0
|
||||
copyright-owner: ByteDance Inc.
|
||||
|
||||
paths:
|
||||
- '**/*.go'
|
||||
- '**/*.s'
|
||||
|
||||
paths-ignore:
|
||||
- 'ast/asm.s' # empty file
|
||||
- 'decoder/asm.s' # empty file
|
||||
- 'encoder/asm.s' # empty file
|
||||
- 'internal/caching/asm.s' # empty file
|
||||
- 'internal/jit/asm.s' # empty file
|
||||
- 'internal/native/avx/native_amd64.s' # auto-generated by asm2asm
|
||||
- 'internal/native/avx/native_subr_amd64.go' # auto-generated by asm2asm
|
||||
- 'internal/native/avx2/native_amd64.s' # auto-generated by asm2asm
|
||||
- 'internal/native/avx2/native_subr_amd64.go' # auto-generated by asm2asm
|
||||
- 'internal/resolver/asm.s' # empty file
|
||||
- 'internal/rt/asm.s' # empty file
|
||||
- 'internal/loader/asm.s' # empty file
|
||||
|
||||
comment: on-failure
|
||||
128
vendor/github.com/bytedance/sonic/CODE_OF_CONDUCT.md
generated
vendored
Normal file
128
vendor/github.com/bytedance/sonic/CODE_OF_CONDUCT.md
generated
vendored
Normal file
@@ -0,0 +1,128 @@
|
||||
# Contributor Covenant Code of Conduct
|
||||
|
||||
## Our Pledge
|
||||
|
||||
We as members, contributors, and leaders pledge to make participation in our
|
||||
community a harassment-free experience for everyone, regardless of age, body
|
||||
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
||||
identity and expression, level of experience, education, socio-economic status,
|
||||
nationality, personal appearance, race, religion, or sexual identity
|
||||
and orientation.
|
||||
|
||||
We pledge to act and interact in ways that contribute to an open, welcoming,
|
||||
diverse, inclusive, and healthy community.
|
||||
|
||||
## Our Standards
|
||||
|
||||
Examples of behavior that contributes to a positive environment for our
|
||||
community include:
|
||||
|
||||
* Demonstrating empathy and kindness toward other people
|
||||
* Being respectful of differing opinions, viewpoints, and experiences
|
||||
* Giving and gracefully accepting constructive feedback
|
||||
* Accepting responsibility and apologizing to those affected by our mistakes,
|
||||
and learning from the experience
|
||||
* Focusing on what is best not just for us as individuals, but for the
|
||||
overall community
|
||||
|
||||
Examples of unacceptable behavior include:
|
||||
|
||||
* The use of sexualized language or imagery, and sexual attention or
|
||||
advances of any kind
|
||||
* Trolling, insulting or derogatory comments, and personal or political attacks
|
||||
* Public or private harassment
|
||||
* Publishing others' private information, such as a physical or email
|
||||
address, without their explicit permission
|
||||
* Other conduct which could reasonably be considered inappropriate in a
|
||||
professional setting
|
||||
|
||||
## Enforcement Responsibilities
|
||||
|
||||
Community leaders are responsible for clarifying and enforcing our standards of
|
||||
acceptable behavior and will take appropriate and fair corrective action in
|
||||
response to any behavior that they deem inappropriate, threatening, offensive,
|
||||
or harmful.
|
||||
|
||||
Community leaders have the right and responsibility to remove, edit, or reject
|
||||
comments, commits, code, wiki edits, issues, and other contributions that are
|
||||
not aligned to this Code of Conduct, and will communicate reasons for moderation
|
||||
decisions when appropriate.
|
||||
|
||||
## Scope
|
||||
|
||||
This Code of Conduct applies within all community spaces, and also applies when
|
||||
an individual is officially representing the community in public spaces.
|
||||
Examples of representing our community include using an official e-mail address,
|
||||
posting via an official social media account, or acting as an appointed
|
||||
representative at an online or offline event.
|
||||
|
||||
## Enforcement
|
||||
|
||||
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
||||
reported to the community leaders responsible for enforcement at
|
||||
wudi.daniel@bytedance.com.
|
||||
All complaints will be reviewed and investigated promptly and fairly.
|
||||
|
||||
All community leaders are obligated to respect the privacy and security of the
|
||||
reporter of any incident.
|
||||
|
||||
## Enforcement Guidelines
|
||||
|
||||
Community leaders will follow these Community Impact Guidelines in determining
|
||||
the consequences for any action they deem in violation of this Code of Conduct:
|
||||
|
||||
### 1. Correction
|
||||
|
||||
**Community Impact**: Use of inappropriate language or other behavior deemed
|
||||
unprofessional or unwelcome in the community.
|
||||
|
||||
**Consequence**: A private, written warning from community leaders, providing
|
||||
clarity around the nature of the violation and an explanation of why the
|
||||
behavior was inappropriate. A public apology may be requested.
|
||||
|
||||
### 2. Warning
|
||||
|
||||
**Community Impact**: A violation through a single incident or series
|
||||
of actions.
|
||||
|
||||
**Consequence**: A warning with consequences for continued behavior. No
|
||||
interaction with the people involved, including unsolicited interaction with
|
||||
those enforcing the Code of Conduct, for a specified period of time. This
|
||||
includes avoiding interactions in community spaces as well as external channels
|
||||
like social media. Violating these terms may lead to a temporary or
|
||||
permanent ban.
|
||||
|
||||
### 3. Temporary Ban
|
||||
|
||||
**Community Impact**: A serious violation of community standards, including
|
||||
sustained inappropriate behavior.
|
||||
|
||||
**Consequence**: A temporary ban from any sort of interaction or public
|
||||
communication with the community for a specified period of time. No public or
|
||||
private interaction with the people involved, including unsolicited interaction
|
||||
with those enforcing the Code of Conduct, is allowed during this period.
|
||||
Violating these terms may lead to a permanent ban.
|
||||
|
||||
### 4. Permanent Ban
|
||||
|
||||
**Community Impact**: Demonstrating a pattern of violation of community
|
||||
standards, including sustained inappropriate behavior, harassment of an
|
||||
individual, or aggression toward or disparagement of classes of individuals.
|
||||
|
||||
**Consequence**: A permanent ban from any sort of public interaction within
|
||||
the community.
|
||||
|
||||
## Attribution
|
||||
|
||||
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
||||
version 2.0, available at
|
||||
https://www.contributor-covenant.org/version/2/0/code_of_conduct.html.
|
||||
|
||||
Community Impact Guidelines were inspired by [Mozilla's code of conduct
|
||||
enforcement ladder](https://github.com/mozilla/diversity).
|
||||
|
||||
[homepage]: https://www.contributor-covenant.org
|
||||
|
||||
For answers to common questions about this code of conduct, see the FAQ at
|
||||
https://www.contributor-covenant.org/faq. Translations are available at
|
||||
https://www.contributor-covenant.org/translations.
|
||||
63
vendor/github.com/bytedance/sonic/CONTRIBUTING.md
generated
vendored
Normal file
63
vendor/github.com/bytedance/sonic/CONTRIBUTING.md
generated
vendored
Normal file
@@ -0,0 +1,63 @@
|
||||
# How to Contribute
|
||||
|
||||
## Your First Pull Request
|
||||
We use GitHub for our codebase. You can start by reading [How To Pull Request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests).
|
||||
|
||||
## Without Semantic Versioning
|
||||
We keep the stable code in branch `main` like `golang.org/x`. Development base on branch `develop`. We promise the **Forward Compatibility** by adding new package directory with suffix `v2/v3` when code has break changes.
|
||||
|
||||
## Branch Organization
|
||||
We use [git-flow](https://nvie.com/posts/a-successful-git-branching-model/) as our branch organization, as known as [FDD](https://en.wikipedia.org/wiki/Feature-driven_development)
|
||||
|
||||
|
||||
## Bugs
|
||||
### 1. How to Find Known Issues
|
||||
We are using [Github Issues](https://github.com/bytedance/sonic/issues) for our public bugs. We keep a close eye on this and try to make it clear when we have an internal fix in progress. Before filing a new task, try to make sure your problem doesn’t already exist.
|
||||
|
||||
### 2. Reporting New Issues
|
||||
Providing a reduced test code is a recommended way for reporting issues. Then can be placed in:
|
||||
- Just in issues
|
||||
- [Golang Playground](https://play.golang.org/)
|
||||
|
||||
### 3. Security Bugs
|
||||
Please do not report the safe disclosure of bugs to public issues. Contact us by [Support Email](mailto:sonic@bytedance.com)
|
||||
|
||||
## How to Get in Touch
|
||||
- [Email](mailto:wudi.daniel@bytedance.com)
|
||||
|
||||
## Submit a Pull Request
|
||||
Before you submit your Pull Request (PR) consider the following guidelines:
|
||||
1. Search [GitHub](https://github.com/bytedance/sonic/pulls) for an open or closed PR that relates to your submission. You don't want to duplicate existing efforts.
|
||||
2. Be sure that an issue describes the problem you're fixing, or documents the design for the feature you'd like to add. Discussing the design upfront helps to ensure that we're ready to accept your work.
|
||||
3. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the bytedance/sonic repo.
|
||||
4. In your forked repository, make your changes in a new git branch:
|
||||
```
|
||||
git checkout -b bugfix/security_bug develop
|
||||
```
|
||||
5. Create your patch, including appropriate test cases.
|
||||
6. Follow our [Style Guides](#code-style-guides).
|
||||
7. Commit your changes using a descriptive commit message that follows [AngularJS Git Commit Message Conventions](https://docs.google.com/document/d/1QrDFcIiPjSLDn3EL15IJygNPiHORgU1_OOAqWjiDU5Y/edit).
|
||||
Adherence to these conventions is necessary because release notes will be automatically generated from these messages.
|
||||
8. Push your branch to GitHub:
|
||||
```
|
||||
git push origin bugfix/security_bug
|
||||
```
|
||||
9. In GitHub, send a pull request to `sonic:main`
|
||||
|
||||
Note: you must use one of `optimize/feature/bugfix/doc/ci/test/refactor` following a slash(`/`) as the branch prefix.
|
||||
|
||||
Your pr title and commit message should follow https://www.conventionalcommits.org/.
|
||||
|
||||
## Contribution Prerequisites
|
||||
- Our development environment keeps up with [Go Official](https://golang.org/project/).
|
||||
- You need fully checking with lint tools before submit your pull request. [gofmt](https://golang.org/pkg/cmd/gofmt/) & [golangci-lint](https://github.com/golangci/golangci-lint)
|
||||
- You are familiar with [Github](https://github.com)
|
||||
- Maybe you need familiar with [Actions](https://github.com/features/actions)(our default workflow tool).
|
||||
|
||||
## Code Style Guides
|
||||
See [Go Code Review Comments](https://github.com/golang/go/wiki/CodeReviewComments).
|
||||
|
||||
Good resources:
|
||||
- [Effective Go](https://golang.org/doc/effective_go)
|
||||
- [Pingcap General advice](https://pingcap.github.io/style-guide/general.html)
|
||||
- [Uber Go Style Guide](https://github.com/uber-go/guide/blob/master/style.md)
|
||||
0
vendor/github.com/bytedance/sonic/CREDITS
generated
vendored
Normal file
0
vendor/github.com/bytedance/sonic/CREDITS
generated
vendored
Normal file
201
vendor/github.com/bytedance/sonic/LICENSE
generated
vendored
Normal file
201
vendor/github.com/bytedance/sonic/LICENSE
generated
vendored
Normal file
@@ -0,0 +1,201 @@
|
||||
Apache License
|
||||
Version 2.0, January 2004
|
||||
http://www.apache.org/licenses/
|
||||
|
||||
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
||||
|
||||
1. Definitions.
|
||||
|
||||
"License" shall mean the terms and conditions for use, reproduction,
|
||||
and distribution as defined by Sections 1 through 9 of this document.
|
||||
|
||||
"Licensor" shall mean the copyright owner or entity authorized by
|
||||
the copyright owner that is granting the License.
|
||||
|
||||
"Legal Entity" shall mean the union of the acting entity and all
|
||||
other entities that control, are controlled by, or are under common
|
||||
control with that entity. For the purposes of this definition,
|
||||
"control" means (i) the power, direct or indirect, to cause the
|
||||
direction or management of such entity, whether by contract or
|
||||
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
||||
outstanding shares, or (iii) beneficial ownership of such entity.
|
||||
|
||||
"You" (or "Your") shall mean an individual or Legal Entity
|
||||
exercising permissions granted by this License.
|
||||
|
||||
"Source" form shall mean the preferred form for making modifications,
|
||||
including but not limited to software source code, documentation
|
||||
source, and configuration files.
|
||||
|
||||
"Object" form shall mean any form resulting from mechanical
|
||||
transformation or translation of a Source form, including but
|
||||
not limited to compiled object code, generated documentation,
|
||||
and conversions to other media types.
|
||||
|
||||
"Work" shall mean the work of authorship, whether in Source or
|
||||
Object form, made available under the License, as indicated by a
|
||||
copyright notice that is included in or attached to the work
|
||||
(an example is provided in the Appendix below).
|
||||
|
||||
"Derivative Works" shall mean any work, whether in Source or Object
|
||||
form, that is based on (or derived from) the Work and for which the
|
||||
editorial revisions, annotations, elaborations, or other modifications
|
||||
represent, as a whole, an original work of authorship. For the purposes
|
||||
of this License, Derivative Works shall not include works that remain
|
||||
separable from, or merely link (or bind by name) to the interfaces of,
|
||||
the Work and Derivative Works thereof.
|
||||
|
||||
"Contribution" shall mean any work of authorship, including
|
||||
the original version of the Work and any modifications or additions
|
||||
to that Work or Derivative Works thereof, that is intentionally
|
||||
submitted to Licensor for inclusion in the Work by the copyright owner
|
||||
or by an individual or Legal Entity authorized to submit on behalf of
|
||||
the copyright owner. For the purposes of this definition, "submitted"
|
||||
means any form of electronic, verbal, or written communication sent
|
||||
to the Licensor or its representatives, including but not limited to
|
||||
communication on electronic mailing lists, source code control systems,
|
||||
and issue tracking systems that are managed by, or on behalf of, the
|
||||
Licensor for the purpose of discussing and improving the Work, but
|
||||
excluding communication that is conspicuously marked or otherwise
|
||||
designated in writing by the copyright owner as "Not a Contribution."
|
||||
|
||||
"Contributor" shall mean Licensor and any individual or Legal Entity
|
||||
on behalf of whom a Contribution has been received by Licensor and
|
||||
subsequently incorporated within the Work.
|
||||
|
||||
2. Grant of Copyright License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
copyright license to reproduce, prepare Derivative Works of,
|
||||
publicly display, publicly perform, sublicense, and distribute the
|
||||
Work and such Derivative Works in Source or Object form.
|
||||
|
||||
3. Grant of Patent License. Subject to the terms and conditions of
|
||||
this License, each Contributor hereby grants to You a perpetual,
|
||||
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
||||
(except as stated in this section) patent license to make, have made,
|
||||
use, offer to sell, sell, import, and otherwise transfer the Work,
|
||||
where such license applies only to those patent claims licensable
|
||||
by such Contributor that are necessarily infringed by their
|
||||
Contribution(s) alone or by combination of their Contribution(s)
|
||||
with the Work to which such Contribution(s) was submitted. If You
|
||||
institute patent litigation against any entity (including a
|
||||
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
||||
or a Contribution incorporated within the Work constitutes direct
|
||||
or contributory patent infringement, then any patent licenses
|
||||
granted to You under this License for that Work shall terminate
|
||||
as of the date such litigation is filed.
|
||||
|
||||
4. Redistribution. You may reproduce and distribute copies of the
|
||||
Work or Derivative Works thereof in any medium, with or without
|
||||
modifications, and in Source or Object form, provided that You
|
||||
meet the following conditions:
|
||||
|
||||
(a) You must give any other recipients of the Work or
|
||||
Derivative Works a copy of this License; and
|
||||
|
||||
(b) You must cause any modified files to carry prominent notices
|
||||
stating that You changed the files; and
|
||||
|
||||
(c) You must retain, in the Source form of any Derivative Works
|
||||
that You distribute, all copyright, patent, trademark, and
|
||||
attribution notices from the Source form of the Work,
|
||||
excluding those notices that do not pertain to any part of
|
||||
the Derivative Works; and
|
||||
|
||||
(d) If the Work includes a "NOTICE" text file as part of its
|
||||
distribution, then any Derivative Works that You distribute must
|
||||
include a readable copy of the attribution notices contained
|
||||
within such NOTICE file, excluding those notices that do not
|
||||
pertain to any part of the Derivative Works, in at least one
|
||||
of the following places: within a NOTICE text file distributed
|
||||
as part of the Derivative Works; within the Source form or
|
||||
documentation, if provided along with the Derivative Works; or,
|
||||
within a display generated by the Derivative Works, if and
|
||||
wherever such third-party notices normally appear. The contents
|
||||
of the NOTICE file are for informational purposes only and
|
||||
do not modify the License. You may add Your own attribution
|
||||
notices within Derivative Works that You distribute, alongside
|
||||
or as an addendum to the NOTICE text from the Work, provided
|
||||
that such additional attribution notices cannot be construed
|
||||
as modifying the License.
|
||||
|
||||
You may add Your own copyright statement to Your modifications and
|
||||
may provide additional or different license terms and conditions
|
||||
for use, reproduction, or distribution of Your modifications, or
|
||||
for any such Derivative Works as a whole, provided Your use,
|
||||
reproduction, and distribution of the Work otherwise complies with
|
||||
the conditions stated in this License.
|
||||
|
||||
5. Submission of Contributions. Unless You explicitly state otherwise,
|
||||
any Contribution intentionally submitted for inclusion in the Work
|
||||
by You to the Licensor shall be under the terms and conditions of
|
||||
this License, without any additional terms or conditions.
|
||||
Notwithstanding the above, nothing herein shall supersede or modify
|
||||
the terms of any separate license agreement you may have executed
|
||||
with Licensor regarding such Contributions.
|
||||
|
||||
6. Trademarks. This License does not grant permission to use the trade
|
||||
names, trademarks, service marks, or product names of the Licensor,
|
||||
except as required for reasonable and customary use in describing the
|
||||
origin of the Work and reproducing the content of the NOTICE file.
|
||||
|
||||
7. Disclaimer of Warranty. Unless required by applicable law or
|
||||
agreed to in writing, Licensor provides the Work (and each
|
||||
Contributor provides its Contributions) on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
||||
implied, including, without limitation, any warranties or conditions
|
||||
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
||||
PARTICULAR PURPOSE. You are solely responsible for determining the
|
||||
appropriateness of using or redistributing the Work and assume any
|
||||
risks associated with Your exercise of permissions under this License.
|
||||
|
||||
8. Limitation of Liability. In no event and under no legal theory,
|
||||
whether in tort (including negligence), contract, or otherwise,
|
||||
unless required by applicable law (such as deliberate and grossly
|
||||
negligent acts) or agreed to in writing, shall any Contributor be
|
||||
liable to You for damages, including any direct, indirect, special,
|
||||
incidental, or consequential damages of any character arising as a
|
||||
result of this License or out of the use or inability to use the
|
||||
Work (including but not limited to damages for loss of goodwill,
|
||||
work stoppage, computer failure or malfunction, or any and all
|
||||
other commercial damages or losses), even if such Contributor
|
||||
has been advised of the possibility of such damages.
|
||||
|
||||
9. Accepting Warranty or Additional Liability. While redistributing
|
||||
the Work or Derivative Works thereof, You may choose to offer,
|
||||
and charge a fee for, acceptance of support, warranty, indemnity,
|
||||
or other liability obligations and/or rights consistent with this
|
||||
License. However, in accepting such obligations, You may act only
|
||||
on Your own behalf and on Your sole responsibility, not on behalf
|
||||
of any other Contributor, and only if You agree to indemnify,
|
||||
defend, and hold each Contributor harmless for any liability
|
||||
incurred by, or claims asserted against, such Contributor by reason
|
||||
of your accepting any such warranty or additional liability.
|
||||
|
||||
END OF TERMS AND CONDITIONS
|
||||
|
||||
APPENDIX: How to apply the Apache License to your work.
|
||||
|
||||
To apply the Apache License to your work, attach the following
|
||||
boilerplate notice, with the fields enclosed by brackets "[]"
|
||||
replaced with your own identifying information. (Don't include
|
||||
the brackets!) The text should be enclosed in the appropriate
|
||||
comment syntax for the file format. We also recommend that a
|
||||
file or class name and description of purpose be included on the
|
||||
same "printed page" as the copyright notice for easier
|
||||
identification within third-party archives.
|
||||
|
||||
Copyright [yyyy] [name of copyright owner]
|
||||
|
||||
Licensed under the Apache License, Version 2.0 (the "License");
|
||||
you may not use this file except in compliance with the License.
|
||||
You may obtain a copy of the License at
|
||||
|
||||
http://www.apache.org/licenses/LICENSE-2.0
|
||||
|
||||
Unless required by applicable law or agreed to in writing, software
|
||||
distributed under the License is distributed on an "AS IS" BASIS,
|
||||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
See the License for the specific language governing permissions and
|
||||
limitations under the License.
|
||||
112
vendor/github.com/bytedance/sonic/Makefile
generated
vendored
Normal file
112
vendor/github.com/bytedance/sonic/Makefile
generated
vendored
Normal file
@@ -0,0 +1,112 @@
|
||||
#
|
||||
# Copyright 2021 ByteDance Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
|
||||
ARCH := avx avx2 sse
|
||||
TMP_DIR := output
|
||||
OUT_DIR := internal/native
|
||||
SRC_FILE := native/native.c
|
||||
|
||||
CPU_avx := amd64
|
||||
CPU_avx2 := amd64
|
||||
CPU_sse := amd64
|
||||
|
||||
TMPL_avx := fastint_amd64_test fastfloat_amd64_test native_amd64_test native_export_amd64
|
||||
TMPL_avx2 := fastint_amd64_test fastfloat_amd64_test native_amd64_test native_export_amd64
|
||||
TMPL_sse := fastint_amd64_test fastfloat_amd64_test native_amd64_test native_export_amd64
|
||||
|
||||
CFLAGS_avx := -msse -mno-sse4 -mavx -mpclmul -mno-avx2 -DUSE_AVX=1 -DUSE_AVX2=0
|
||||
CFLAGS_avx2 := -msse -mno-sse4 -mavx -mpclmul -mavx2 -DUSE_AVX=1 -DUSE_AVX2=1
|
||||
CFLAGS_sse := -msse -mno-sse4 -mno-avx -mno-avx2 -mpclmul
|
||||
|
||||
CC_amd64 := clang
|
||||
ASM2ASM_amd64 := tools/asm2asm/asm2asm.py
|
||||
|
||||
CFLAGS := -mno-red-zone
|
||||
CFLAGS += -target x86_64-apple-macos11
|
||||
CFLAGS += -fno-asynchronous-unwind-tables
|
||||
CFLAGS += -fno-builtin
|
||||
CFLAGS += -fno-exceptions
|
||||
CFLAGS += -fno-rtti
|
||||
CFLAGS += -fno-stack-protector
|
||||
CFLAGS += -nostdlib
|
||||
CFLAGS += -O3
|
||||
CFLAGS += -Wall -Werror
|
||||
|
||||
NATIVE_SRC := $(wildcard native/*.h)
|
||||
NATIVE_SRC += $(wildcard native/*.c)
|
||||
|
||||
.PHONY: all clean ${ARCH}
|
||||
|
||||
define build_tmpl
|
||||
$(eval @arch := $(1))
|
||||
$(eval @tmpl := $(2))
|
||||
$(eval @dest := $(3))
|
||||
|
||||
${@dest}: ${@tmpl}
|
||||
mkdir -p $(dir ${@dest})
|
||||
echo '// Code generated by Makefile, DO NOT EDIT.' > ${@dest}
|
||||
echo >> ${@dest}
|
||||
sed -e 's/{{PACKAGE}}/${@arch}/g' ${@tmpl} >> ${@dest}
|
||||
endef
|
||||
|
||||
define build_arch
|
||||
$(eval @cpu := $(value CPU_$(1)))
|
||||
$(eval @deps := $(foreach tmpl,$(value TMPL_$(1)),${OUT_DIR}/$(1)/${tmpl}.go))
|
||||
$(eval @asmin := ${TMP_DIR}/$(1)/native.s)
|
||||
$(eval @asmout := ${OUT_DIR}/$(1)/native_${@cpu}.s)
|
||||
$(eval @stubin := ${OUT_DIR}/native_${@cpu}.tmpl)
|
||||
$(eval @stubout := ${OUT_DIR}/$(1)/native_${@cpu}.go)
|
||||
|
||||
$(1): ${@asmout} ${@deps}
|
||||
|
||||
${@asmout}: ${@stubout} ${NATIVE_SRC}
|
||||
mkdir -p ${TMP_DIR}/$(1)
|
||||
$${CC_${@cpu}} $${CFLAGS} $${CFLAGS_$(1)} -S -o ${TMP_DIR}/$(1)/native.s ${SRC_FILE}
|
||||
python3 $${ASM2ASM_${@cpu}} ${@asmout} ${TMP_DIR}/$(1)/native.s
|
||||
asmfmt -w ${@asmout}
|
||||
|
||||
$(eval $(call \
|
||||
build_tmpl, \
|
||||
$(1), \
|
||||
${@stubin}, \
|
||||
${@stubout} \
|
||||
))
|
||||
|
||||
$(foreach \
|
||||
tmpl, \
|
||||
$(value TMPL_$(1)), \
|
||||
$(eval $(call \
|
||||
build_tmpl, \
|
||||
$(1), \
|
||||
${OUT_DIR}/${tmpl}.tmpl, \
|
||||
${OUT_DIR}/$(1)/${tmpl}.go \
|
||||
)) \
|
||||
)
|
||||
endef
|
||||
|
||||
all: ${ARCH}
|
||||
|
||||
clean:
|
||||
for arch in ${ARCH}; do \
|
||||
rm -vfr ${TMP_DIR}/$${arch}; \
|
||||
rm -vfr ${OUT_DIR}/$${arch}; \
|
||||
done
|
||||
|
||||
$(foreach \
|
||||
arch, \
|
||||
${ARCH}, \
|
||||
$(eval $(call build_arch,${arch})) \
|
||||
)
|
||||
362
vendor/github.com/bytedance/sonic/README.md
generated
vendored
Normal file
362
vendor/github.com/bytedance/sonic/README.md
generated
vendored
Normal file
@@ -0,0 +1,362 @@
|
||||
# Sonic
|
||||
|
||||
English | [中文](README_ZH_CN.md)
|
||||
|
||||
A blazingly fast JSON serializing & deserializing library, accelerated by JIT (just-in-time compiling) and SIMD (single-instruction-multiple-data).
|
||||
|
||||
## Requirement
|
||||
- Go 1.15~1.20
|
||||
- Linux/MacOS/Windows
|
||||
- Amd64 ARCH
|
||||
|
||||
## Features
|
||||
- Runtime object binding without code generation
|
||||
- Complete APIs for JSON value manipulation
|
||||
- Fast, fast, fast!
|
||||
|
||||
## Benchmarks
|
||||
For **all sizes** of json and **all scenarios** of usage, **Sonic performs best**.
|
||||
- [Medium](https://github.com/bytedance/sonic/blob/main/decoder/testdata_test.go#L19) (13KB, 300+ key, 6 layers)
|
||||
```powershell
|
||||
goversion: 1.17.1
|
||||
goos: darwin
|
||||
goarch: amd64
|
||||
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
|
||||
BenchmarkEncoder_Generic_Sonic-16 32393 ns/op 402.40 MB/s 11965 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Generic_Sonic_Fast-16 21668 ns/op 601.57 MB/s 10940 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Generic_JsonIter-16 42168 ns/op 309.12 MB/s 14345 B/op 115 allocs/op
|
||||
BenchmarkEncoder_Generic_GoJson-16 65189 ns/op 199.96 MB/s 23261 B/op 16 allocs/op
|
||||
BenchmarkEncoder_Generic_StdLib-16 106322 ns/op 122.60 MB/s 49136 B/op 789 allocs/op
|
||||
BenchmarkEncoder_Binding_Sonic-16 6269 ns/op 2079.26 MB/s 14173 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Binding_Sonic_Fast-16 5281 ns/op 2468.16 MB/s 12322 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Binding_JsonIter-16 20056 ns/op 649.93 MB/s 9488 B/op 2 allocs/op
|
||||
BenchmarkEncoder_Binding_GoJson-16 8311 ns/op 1568.32 MB/s 9481 B/op 1 allocs/op
|
||||
BenchmarkEncoder_Binding_StdLib-16 16448 ns/op 792.52 MB/s 9479 B/op 1 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_Sonic-16 6681 ns/op 1950.93 MB/s 12738 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_Sonic_Fast-16 4179 ns/op 3118.99 MB/s 10757 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_JsonIter-16 9861 ns/op 1321.84 MB/s 14362 B/op 115 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_GoJson-16 18850 ns/op 691.52 MB/s 23278 B/op 16 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_StdLib-16 45902 ns/op 283.97 MB/s 49174 B/op 789 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_Sonic-16 1480 ns/op 8810.09 MB/s 13049 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_Sonic_Fast-16 1209 ns/op 10785.23 MB/s 11546 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_JsonIter-16 6170 ns/op 2112.58 MB/s 9504 B/op 2 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_GoJson-16 3321 ns/op 3925.52 MB/s 9496 B/op 1 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_StdLib-16 3739 ns/op 3486.49 MB/s 9480 B/op 1 allocs/op
|
||||
|
||||
BenchmarkDecoder_Generic_Sonic-16 66812 ns/op 195.10 MB/s 57602 B/op 723 allocs/op
|
||||
BenchmarkDecoder_Generic_Sonic_Fast-16 54523 ns/op 239.07 MB/s 49786 B/op 313 allocs/op
|
||||
BenchmarkDecoder_Generic_StdLib-16 124260 ns/op 104.90 MB/s 50869 B/op 772 allocs/op
|
||||
BenchmarkDecoder_Generic_JsonIter-16 91274 ns/op 142.81 MB/s 55782 B/op 1068 allocs/op
|
||||
BenchmarkDecoder_Generic_GoJson-16 88569 ns/op 147.17 MB/s 66367 B/op 973 allocs/op
|
||||
BenchmarkDecoder_Binding_Sonic-16 32557 ns/op 400.38 MB/s 28302 B/op 137 allocs/op
|
||||
BenchmarkDecoder_Binding_Sonic_Fast-16 28649 ns/op 455.00 MB/s 24999 B/op 34 allocs/op
|
||||
BenchmarkDecoder_Binding_StdLib-16 111437 ns/op 116.97 MB/s 10576 B/op 208 allocs/op
|
||||
BenchmarkDecoder_Binding_JsonIter-16 35090 ns/op 371.48 MB/s 14673 B/op 385 allocs/op
|
||||
BenchmarkDecoder_Binding_GoJson-16 28738 ns/op 453.59 MB/s 22039 B/op 49 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_Sonic-16 12321 ns/op 1057.91 MB/s 57233 B/op 723 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_Sonic_Fast-16 10644 ns/op 1224.64 MB/s 49362 B/op 313 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_StdLib-16 57587 ns/op 226.35 MB/s 50874 B/op 772 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_JsonIter-16 38666 ns/op 337.12 MB/s 55789 B/op 1068 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_GoJson-16 30259 ns/op 430.79 MB/s 66370 B/op 974 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_Sonic-16 5965 ns/op 2185.28 MB/s 27747 B/op 137 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_Sonic_Fast-16 5170 ns/op 2521.31 MB/s 24715 B/op 34 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_StdLib-16 27582 ns/op 472.58 MB/s 10576 B/op 208 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_JsonIter-16 13571 ns/op 960.51 MB/s 14685 B/op 385 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_GoJson-16 10031 ns/op 1299.51 MB/s 22111 B/op 49 allocs/op
|
||||
|
||||
BenchmarkGetOne_Sonic-16 3276 ns/op 3975.78 MB/s 24 B/op 1 allocs/op
|
||||
BenchmarkGetOne_Gjson-16 9431 ns/op 1380.81 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkGetOne_Jsoniter-16 51178 ns/op 254.46 MB/s 27936 B/op 647 allocs/op
|
||||
BenchmarkGetOne_Parallel_Sonic-16 216.7 ns/op 60098.95 MB/s 24 B/op 1 allocs/op
|
||||
BenchmarkGetOne_Parallel_Gjson-16 1076 ns/op 12098.62 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkGetOne_Parallel_Jsoniter-16 17741 ns/op 734.06 MB/s 27945 B/op 647 allocs/op
|
||||
BenchmarkSetOne_Sonic-16 9571 ns/op 1360.61 MB/s 1584 B/op 17 allocs/op
|
||||
BenchmarkSetOne_Sjson-16 36456 ns/op 357.22 MB/s 52180 B/op 9 allocs/op
|
||||
BenchmarkSetOne_Jsoniter-16 79475 ns/op 163.86 MB/s 45862 B/op 964 allocs/op
|
||||
BenchmarkSetOne_Parallel_Sonic-16 850.9 ns/op 15305.31 MB/s 1584 B/op 17 allocs/op
|
||||
BenchmarkSetOne_Parallel_Sjson-16 18194 ns/op 715.77 MB/s 52247 B/op 9 allocs/op
|
||||
BenchmarkSetOne_Parallel_Jsoniter-16 33560 ns/op 388.05 MB/s 45892 B/op 964 allocs/op
|
||||
```
|
||||
- [Small](https://github.com/bytedance/sonic/blob/main/testdata/small.go) (400B, 11 keys, 3 layers)
|
||||

|
||||
- [Large](https://github.com/bytedance/sonic/blob/main/testdata/twitter.json) (635KB, 10000+ key, 6 layers)
|
||||

|
||||
|
||||
See [bench.sh](https://github.com/bytedance/sonic/blob/main/bench.sh) for benchmark codes.
|
||||
|
||||
## How it works
|
||||
See [INTRODUCTION.md](./docs/INTRODUCTION.md).
|
||||
|
||||
## Usage
|
||||
|
||||
### Marshal/Unmarshal
|
||||
|
||||
Default behaviors are mostly consistent with `encoding/json`, except HTML escaping form (see [Escape HTML](https://github.com/bytedance/sonic/blob/main/README.md#escape-html)) and `SortKeys` feature (optional support see [Sort Keys](https://github.com/bytedance/sonic/blob/main/README.md#sort-keys)) that is **NOT** in conformity to [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259).
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
var data YourSchema
|
||||
// Marshal
|
||||
output, err := sonic.Marshal(&data)
|
||||
// Unmarshal
|
||||
err := sonic.Unmarshal(output, &data)
|
||||
```
|
||||
|
||||
### Streaming IO
|
||||
Sonic supports decoding json from `io.Reader` or encoding objects into `io.`Writer`, aims at handling multiple values as well as reducing memory consumption.
|
||||
- encoder
|
||||
```go
|
||||
var o1 = map[string]interface{}{
|
||||
"a": "b",
|
||||
}
|
||||
var o2 = 1
|
||||
var w = bytes.NewBuffer(nil)
|
||||
var enc = sonic.ConfigDefault.NewEncoder(w)
|
||||
enc.Encode(o1)
|
||||
enc.Encode(o2)
|
||||
fmt.Println(w.String())
|
||||
// Output:
|
||||
// {"a":"b"}
|
||||
// 1
|
||||
```
|
||||
- decoder
|
||||
```go
|
||||
var o = map[string]interface{}{}
|
||||
var r = strings.NewReader(`{"a":"b"}{"1":"2"}`)
|
||||
var dec = sonic.ConfigDefault.NewDecoder(r)
|
||||
dec.Decode(&o)
|
||||
dec.Decode(&o)
|
||||
fmt.Printf("%+v", o)
|
||||
// Output:
|
||||
// map[1:2 a:b]
|
||||
```
|
||||
|
||||
### Use Number/Use Int64
|
||||
```go
|
||||
import "github.com/bytedance/sonic/decoder"
|
||||
|
||||
var input = `1`
|
||||
var data interface{}
|
||||
|
||||
// default float64
|
||||
dc := decoder.NewDecoder(input)
|
||||
dc.Decode(&data) // data == float64(1)
|
||||
// use json.Number
|
||||
dc = decoder.NewDecoder(input)
|
||||
dc.UseNumber()
|
||||
dc.Decode(&data) // data == json.Number("1")
|
||||
// use int64
|
||||
dc = decoder.NewDecoder(input)
|
||||
dc.UseInt64()
|
||||
dc.Decode(&data) // data == int64(1)
|
||||
|
||||
root, err := sonic.GetFromString(input)
|
||||
// Get json.Number
|
||||
jn := root.Number()
|
||||
jm := root.InterfaceUseNumber().(json.Number) // jn == jm
|
||||
// Get float64
|
||||
fn := root.Float64()
|
||||
fm := root.Interface().(float64) // jn == jm
|
||||
```
|
||||
|
||||
### Sort Keys
|
||||
On account of the performance loss from sorting (roughly 10%), sonic doesn't enable this feature by default. If your component depends on it to work (like [zstd](https://github.com/facebook/zstd)), Use it like this:
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
import "github.com/bytedance/sonic/encoder"
|
||||
|
||||
// Binding map only
|
||||
m := map[string]interface{}{}
|
||||
v, err := encoder.Encode(m, encoder.SortMapKeys)
|
||||
|
||||
// Or ast.Node.SortKeys() before marshal
|
||||
var root := sonic.Get(JSON)
|
||||
err := root.SortKeys()
|
||||
```
|
||||
### Escape HTML
|
||||
On account of the performance loss (roughly 15%), sonic doesn't enable this feature by default. You can use `encoder.EscapeHTML` option to open this feature (align with `encoding/json.HTMLEscape`).
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
v := map[string]string{"&&":"<>"}
|
||||
ret, err := Encode(v, EscapeHTML) // ret == `{"\u0026\u0026":{"X":"\u003c\u003e"}}`
|
||||
```
|
||||
### Compact Format
|
||||
Sonic encodes primitive objects (struct/map...) as compact-format JSON by default, except marshaling `json.RawMessage` or `json.Marshaler`: sonic ensures validating their output JSON but **DONOT** compacting them for performance concerns. We provide the option `encoder.CompactMarshaler` to add compacting process.
|
||||
|
||||
### Print Error
|
||||
If there invalid syntax in input JSON, sonic will return `decoder.SyntaxError`, which supports pretty-printing of error position
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
import "github.com/bytedance/sonic/decoder"
|
||||
|
||||
var data interface{}
|
||||
err := sonic.UnmarshalString("[[[}]]", &data)
|
||||
if err != nil {
|
||||
/* One line by default */
|
||||
println(e.Error()) // "Syntax error at index 3: invalid char\n\n\t[[[}]]\n\t...^..\n"
|
||||
/* Pretty print */
|
||||
if e, ok := err.(decoder.SyntaxError); ok {
|
||||
/*Syntax error at index 3: invalid char
|
||||
|
||||
[[[}]]
|
||||
...^..
|
||||
*/
|
||||
print(e.Description())
|
||||
} else if me, ok := err.(*decoder.MismatchTypeError); ok {
|
||||
// decoder.MismatchTypeError is new to Sonic v1.6.0
|
||||
print(me.Description())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### Mismatched Types [Sonic v1.6.0]
|
||||
If there a **mismatch-typed** value for a given key, sonic will report `decoder.MismatchTypeError` (if there are many, report the last one), but still skip wrong the value and keep decoding next JSON.
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
import "github.com/bytedance/sonic/decoder"
|
||||
|
||||
var data = struct{
|
||||
A int
|
||||
B int
|
||||
}{}
|
||||
err := UnmarshalString(`{"A":"1","B":1}`, &data)
|
||||
println(err.Error()) // Mismatch type int with value string "at index 5: mismatched type with value\n\n\t{\"A\":\"1\",\"B\":1}\n\t.....^.........\n"
|
||||
fmt.Printf("%+v", data) // {A:0 B:1}
|
||||
```
|
||||
### Ast.Node
|
||||
Sonic/ast.Node is a completely self-contained AST for JSON. It implements serialization and deserialization both and provides robust APIs for obtaining and modification of generic data.
|
||||
#### Get/Index
|
||||
Search partial JSON by given paths, which must be non-negative integer or string, or nil
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
input := []byte(`{"key1":[{},{"key2":{"key3":[1,2,3]}}]}`)
|
||||
|
||||
// no path, returns entire json
|
||||
root, err := sonic.Get(input)
|
||||
raw := root.Raw() // == string(input)
|
||||
|
||||
// multiple paths
|
||||
root, err := sonic.Get(input, "key1", 1, "key2")
|
||||
sub := root.Get("key3").Index(2).Int64() // == 3
|
||||
```
|
||||
**Tip**: since `Index()` uses offset to locate data, which is much faster than scanning like `Get()`, we suggest you use it as much as possible. And sonic also provides another API `IndexOrGet()` to underlying use offset as well as ensure the key is matched.
|
||||
|
||||
#### Set/Unset
|
||||
Modify the json content by Set()/Unset()
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
// Set
|
||||
exist, err := root.Set("key4", NewBool(true)) // exist == false
|
||||
alias1 := root.Get("key4")
|
||||
println(alias1.Valid()) // true
|
||||
alias2 := root.Index(1)
|
||||
println(alias1 == alias2) // true
|
||||
|
||||
// Unset
|
||||
exist, err := root.UnsetByIndex(1) // exist == true
|
||||
println(root.Get("key4").Check()) // "value not exist"
|
||||
```
|
||||
|
||||
#### Serialize
|
||||
To encode `ast.Node` as json, use `MarshalJson()` or `json.Marshal()` (MUST pass the node's pointer)
|
||||
```go
|
||||
import (
|
||||
"encoding/json"
|
||||
"github.com/bytedance/sonic"
|
||||
)
|
||||
|
||||
buf, err := root.MarshalJson()
|
||||
println(string(buf)) // {"key1":[{},{"key2":{"key3":[1,2,3]}}]}
|
||||
exp, err := json.Marshal(&root) // WARN: use pointer
|
||||
println(string(buf) == string(exp)) // true
|
||||
```
|
||||
|
||||
#### APIs
|
||||
- validation: `Check()`, `Error()`, `Valid()`, `Exist()`
|
||||
- searching: `Index()`, `Get()`, `IndexPair()`, `IndexOrGet()`, `GetByPath()`
|
||||
- go-type casting: `Int64()`, `Float64()`, `String()`, `Number()`, `Bool()`, `Map[UseNumber|UseNode]()`, `Array[UseNumber|UseNode]()`, `Interface[UseNumber|UseNode]()`
|
||||
- go-type packing: `NewRaw()`, `NewNumber()`, `NewNull()`, `NewBool()`, `NewString()`, `NewObject()`, `NewArray()`
|
||||
- iteration: `Values()`, `Properties()`, `ForEach()`, `SortKeys()`
|
||||
- modification: `Set()`, `SetByIndex()`, `Add()`
|
||||
|
||||
## Compatibility
|
||||
Sonic **DOES NOT** ensure to support all environments, due to the difficulty of developing high-performance codes. For developers who use sonic to build their applications in different environments, we have the following suggestions:
|
||||
|
||||
- Developing on **Mac M1**: Make sure you have Rosetta 2 installed on your machine, and set `GOARCH=amd64` when building your application. Rosetta 2 can automatically translate x86 binaries to arm64 binaries and run x86 applications on Mac M1.
|
||||
- Developing on **Linux arm64**: You can install qemu and use the `qemu-x86_64 -cpu max` command to convert x86 binaries to amr64 binaries for applications built with sonic. The qemu can achieve a similar transfer effect to Rosetta 2 on Mac M1.
|
||||
|
||||
For developers who want to use sonic on Linux arm64 without qemu, or those who want to handle JSON strictly consistent with `encoding/json`, we provide some compatible APIs as `sonic.API`
|
||||
- `ConfigDefault`: the sonic's default config (`EscapeHTML=false`,`SortKeys=false`...) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options like `SortKeys=false` will be invalid.
|
||||
- `ConfigStd`: the std-compatible config (`EscapeHTML=true`,`SortKeys=true`...) to run on sonic-supporting environment. It will fall back to `encoding/json`.
|
||||
- `ConfigFastest`: the fastest config (`NoQuoteTextMarshaler=true`) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options will be invalid.
|
||||
|
||||
## Tips
|
||||
|
||||
### Pretouch
|
||||
Since Sonic uses [golang-asm](https://github.com/twitchyliquid64/golang-asm) as a JIT assembler, which is NOT very suitable for runtime compiling, first-hit running of a huge schema may cause request-timeout or even process-OOM. For better stability, we advise **using `Pretouch()` for huge-schema or compact-memory applications** before `Marshal()/Unmarshal()`.
|
||||
```go
|
||||
import (
|
||||
"reflect"
|
||||
"github.com/bytedance/sonic"
|
||||
"github.com/bytedance/sonic/option"
|
||||
)
|
||||
|
||||
func init() {
|
||||
var v HugeStruct
|
||||
|
||||
// For most large types (nesting depth <= option.DefaultMaxInlineDepth)
|
||||
err := sonic.Pretouch(reflect.TypeOf(v))
|
||||
|
||||
// with more CompileOption...
|
||||
err := sonic.Pretouch(reflect.TypeOf(v),
|
||||
// If the type is too deep nesting (nesting depth > option.DefaultMaxInlineDepth),
|
||||
// you can set compile recursive loops in Pretouch for better stability in JIT.
|
||||
option.WithCompileRecursiveDepth(loop),
|
||||
// For a large nested struct, try to set a smaller depth to reduce compiling time.
|
||||
option.WithCompileMaxInlineDepth(depth),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### Copy string
|
||||
When decoding **string values without any escaped characters**, sonic references them from the origin JSON buffer instead of mallocing a new buffer to copy. This helps a lot for CPU performance but may leave the whole JSON buffer in memory as long as the decoded objects are being used. In practice, we found the extra memory introduced by referring JSON buffer is usually 20% ~ 80% of decoded objects. Once an application holds these objects for a long time (for example, cache the decoded objects for reusing), its in-use memory on the server may go up. We provide the option `decoder.CopyString()` for users to choose not to reference the JSON buffer, which may cause a decline in CPU performance to some degree.
|
||||
|
||||
### Pass string or []byte?
|
||||
For alignment to `encoding/json`, we provide API to pass `[]byte` as an argument, but the string-to-bytes copy is conducted at the same time considering safety, which may lose performance when the origin JSON is huge. Therefore, you can use `UnmarshalString()` and `GetFromString()` to pass a string, as long as your origin data is a string or **nocopy-cast** is safe for your []byte. We also provide API `MarshalString()` for convenient **nocopy-cast** of encoded JSON []byte, which is safe since sonic's output bytes is always duplicated and unique.
|
||||
|
||||
### Accelerate `encoding.TextMarshaler`
|
||||
To ensure data security, sonic.Encoder quotes and escapes string values from `encoding.TextMarshaler` interfaces by default, which may degrade performance much if most of your data is in form of them. We provide `encoder.NoQuoteTextMarshaler` to skip these operations, which means you **MUST** ensure their output string escaped and quoted following [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259).
|
||||
|
||||
|
||||
### Better performance for generic data
|
||||
In **fully-parsed** scenario, `Unmarshal()` performs better than `Get()`+`Node.Interface()`. But if you only have a part of the schema for specific json, you can combine `Get()` and `Unmarshal()` together:
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
node, err := sonic.GetFromString(_TwitterJson, "statuses", 3, "user")
|
||||
var user User // your partial schema...
|
||||
err = sonic.UnmarshalString(node.Raw(), &user)
|
||||
```
|
||||
Even if you don't have any schema, use `ast.Node` as the container of generic values instead of `map` or `interface`:
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
root, err := sonic.GetFromString(_TwitterJson)
|
||||
user := root.GetByPath("statuses", 3, "user") // === root.Get("status").Index(3).Get("user")
|
||||
err = user.Check()
|
||||
|
||||
// err = user.LoadAll() // only call this when you want to use 'user' concurrently...
|
||||
go someFunc(user)
|
||||
```
|
||||
Why? Because `ast.Node` stores its children using `array`:
|
||||
- `Array`'s performance is **much better** than `Map` when Inserting (Deserialize) and Scanning (Serialize) data;
|
||||
- **Hashing** (`map[x]`) is not as efficient as **Indexing** (`array[x]`), which `ast.Node` can conduct on **both array and object**;
|
||||
- Using `Interface()`/`Map()` means Sonic must parse all the underlying values, while `ast.Node` can parse them **on demand**.
|
||||
|
||||
**CAUTION:** `ast.Node` **DOESN'T** ensure concurrent security directly, due to its **lazy-load** design. However, you can call `Node.Load()`/`Node.LoadAll()` to achieve that, which may bring performance reduction while it still works faster than converting to `map` or `interface{}`
|
||||
|
||||
## Community
|
||||
Sonic is a subproject of [CloudWeGo](https://www.cloudwego.io/). We are committed to building a cloud native ecosystem.
|
||||
382
vendor/github.com/bytedance/sonic/README_ZH_CN.md
generated
vendored
Normal file
382
vendor/github.com/bytedance/sonic/README_ZH_CN.md
generated
vendored
Normal file
@@ -0,0 +1,382 @@
|
||||
# Sonic
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
一个速度奇快的 JSON 序列化/反序列化库,由 JIT (即时编译)和 SIMD (单指令流多数据流)加速。
|
||||
|
||||
## 依赖
|
||||
|
||||
- Go 1.15~1.20
|
||||
- Linux/MacOS/Windows
|
||||
- Amd64 架构
|
||||
|
||||
## 特色
|
||||
|
||||
- 运行时对象绑定,无需代码生成
|
||||
- 完备的 JSON 操作 API
|
||||
- 快,更快,还要更快!
|
||||
|
||||
## 基准测试
|
||||
|
||||
对于**所有大小**的 json 和**所有使用场景**, **Sonic 表现均为最佳**。
|
||||
- [中型](https://github.com/bytedance/sonic/blob/main/decoder/testdata_test.go#L19) (13kB, 300+ 键, 6 层)
|
||||
```powershell
|
||||
goversion: 1.17.1
|
||||
goos: darwin
|
||||
goarch: amd64
|
||||
cpu: Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
|
||||
BenchmarkEncoder_Generic_Sonic-16 32393 ns/op 402.40 MB/s 11965 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Generic_Sonic_Fast-16 21668 ns/op 601.57 MB/s 10940 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Generic_JsonIter-16 42168 ns/op 309.12 MB/s 14345 B/op 115 allocs/op
|
||||
BenchmarkEncoder_Generic_GoJson-16 65189 ns/op 199.96 MB/s 23261 B/op 16 allocs/op
|
||||
BenchmarkEncoder_Generic_StdLib-16 106322 ns/op 122.60 MB/s 49136 B/op 789 allocs/op
|
||||
BenchmarkEncoder_Binding_Sonic-16 6269 ns/op 2079.26 MB/s 14173 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Binding_Sonic_Fast-16 5281 ns/op 2468.16 MB/s 12322 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Binding_JsonIter-16 20056 ns/op 649.93 MB/s 9488 B/op 2 allocs/op
|
||||
BenchmarkEncoder_Binding_GoJson-16 8311 ns/op 1568.32 MB/s 9481 B/op 1 allocs/op
|
||||
BenchmarkEncoder_Binding_StdLib-16 16448 ns/op 792.52 MB/s 9479 B/op 1 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_Sonic-16 6681 ns/op 1950.93 MB/s 12738 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_Sonic_Fast-16 4179 ns/op 3118.99 MB/s 10757 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_JsonIter-16 9861 ns/op 1321.84 MB/s 14362 B/op 115 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_GoJson-16 18850 ns/op 691.52 MB/s 23278 B/op 16 allocs/op
|
||||
BenchmarkEncoder_Parallel_Generic_StdLib-16 45902 ns/op 283.97 MB/s 49174 B/op 789 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_Sonic-16 1480 ns/op 8810.09 MB/s 13049 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_Sonic_Fast-16 1209 ns/op 10785.23 MB/s 11546 B/op 4 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_JsonIter-16 6170 ns/op 2112.58 MB/s 9504 B/op 2 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_GoJson-16 3321 ns/op 3925.52 MB/s 9496 B/op 1 allocs/op
|
||||
BenchmarkEncoder_Parallel_Binding_StdLib-16 3739 ns/op 3486.49 MB/s 9480 B/op 1 allocs/op
|
||||
|
||||
BenchmarkDecoder_Generic_Sonic-16 66812 ns/op 195.10 MB/s 57602 B/op 723 allocs/op
|
||||
BenchmarkDecoder_Generic_Sonic_Fast-16 54523 ns/op 239.07 MB/s 49786 B/op 313 allocs/op
|
||||
BenchmarkDecoder_Generic_StdLib-16 124260 ns/op 104.90 MB/s 50869 B/op 772 allocs/op
|
||||
BenchmarkDecoder_Generic_JsonIter-16 91274 ns/op 142.81 MB/s 55782 B/op 1068 allocs/op
|
||||
BenchmarkDecoder_Generic_GoJson-16 88569 ns/op 147.17 MB/s 66367 B/op 973 allocs/op
|
||||
BenchmarkDecoder_Binding_Sonic-16 32557 ns/op 400.38 MB/s 28302 B/op 137 allocs/op
|
||||
BenchmarkDecoder_Binding_Sonic_Fast-16 28649 ns/op 455.00 MB/s 24999 B/op 34 allocs/op
|
||||
BenchmarkDecoder_Binding_StdLib-16 111437 ns/op 116.97 MB/s 10576 B/op 208 allocs/op
|
||||
BenchmarkDecoder_Binding_JsonIter-16 35090 ns/op 371.48 MB/s 14673 B/op 385 allocs/op
|
||||
BenchmarkDecoder_Binding_GoJson-16 28738 ns/op 453.59 MB/s 22039 B/op 49 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_Sonic-16 12321 ns/op 1057.91 MB/s 57233 B/op 723 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_Sonic_Fast-16 10644 ns/op 1224.64 MB/s 49362 B/op 313 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_StdLib-16 57587 ns/op 226.35 MB/s 50874 B/op 772 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_JsonIter-16 38666 ns/op 337.12 MB/s 55789 B/op 1068 allocs/op
|
||||
BenchmarkDecoder_Parallel_Generic_GoJson-16 30259 ns/op 430.79 MB/s 66370 B/op 974 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_Sonic-16 5965 ns/op 2185.28 MB/s 27747 B/op 137 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_Sonic_Fast-16 5170 ns/op 2521.31 MB/s 24715 B/op 34 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_StdLib-16 27582 ns/op 472.58 MB/s 10576 B/op 208 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_JsonIter-16 13571 ns/op 960.51 MB/s 14685 B/op 385 allocs/op
|
||||
BenchmarkDecoder_Parallel_Binding_GoJson-16 10031 ns/op 1299.51 MB/s 22111 B/op 49 allocs/op
|
||||
|
||||
BenchmarkGetOne_Sonic-16 3276 ns/op 3975.78 MB/s 24 B/op 1 allocs/op
|
||||
BenchmarkGetOne_Gjson-16 9431 ns/op 1380.81 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkGetOne_Jsoniter-16 51178 ns/op 254.46 MB/s 27936 B/op 647 allocs/op
|
||||
BenchmarkGetOne_Parallel_Sonic-16 216.7 ns/op 60098.95 MB/s 24 B/op 1 allocs/op
|
||||
BenchmarkGetOne_Parallel_Gjson-16 1076 ns/op 12098.62 MB/s 0 B/op 0 allocs/op
|
||||
BenchmarkGetOne_Parallel_Jsoniter-16 17741 ns/op 734.06 MB/s 27945 B/op 647 allocs/op
|
||||
BenchmarkSetOne_Sonic-16 9571 ns/op 1360.61 MB/s 1584 B/op 17 allocs/op
|
||||
BenchmarkSetOne_Sjson-16 36456 ns/op 357.22 MB/s 52180 B/op 9 allocs/op
|
||||
BenchmarkSetOne_Jsoniter-16 79475 ns/op 163.86 MB/s 45862 B/op 964 allocs/op
|
||||
BenchmarkSetOne_Parallel_Sonic-16 850.9 ns/op 15305.31 MB/s 1584 B/op 17 allocs/op
|
||||
BenchmarkSetOne_Parallel_Sjson-16 18194 ns/op 715.77 MB/s 52247 B/op 9 allocs/op
|
||||
BenchmarkSetOne_Parallel_Jsoniter-16 33560 ns/op 388.05 MB/s 45892 B/op 964 allocs/op
|
||||
```
|
||||
- [小型](https://github.com/bytedance/sonic/blob/main/testdata/small.go) (400B, 11 个键, 3 层)
|
||||

|
||||
- [大型](https://github.com/bytedance/sonic/blob/main/testdata/twitter.json) (635kB, 10000+ 个键, 6 层)
|
||||

|
||||
|
||||
要查看基准测试代码,请参阅 [bench.sh](https://github.com/bytedance/sonic/blob/main/bench.sh) 。
|
||||
|
||||
## 工作原理
|
||||
|
||||
请参阅 [INTRODUCTION_ZH_CN.md](./docs/INTRODUCTION_ZH_CN.md).
|
||||
|
||||
## 使用方式
|
||||
|
||||
### 序列化/反序列化
|
||||
|
||||
默认的行为基本上与 `encoding/json` 相一致,除了 HTML 转义形式(参见 [Escape HTML](https://github.com/bytedance/sonic/blob/main/README.md#escape-html)) 和 `SortKeys` 功能(参见 [Sort Keys](https://github.com/bytedance/sonic/blob/main/README.md#sort-keys))**没有**遵循 [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259) 。
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
var data YourSchema
|
||||
// Marshal
|
||||
output, err := sonic.Marshal(&data)
|
||||
// Unmarshal
|
||||
err := sonic.Unmarshal(output, &data)
|
||||
```
|
||||
|
||||
### 流式输入输出
|
||||
|
||||
Sonic 支持解码 `io.Reader` 中输入的 json,或将对象编码为 json 后输出至 `io.Writer`,以处理多个值并减少内存消耗。
|
||||
- 编码器
|
||||
```go
|
||||
var o1 = map[string]interface{}{
|
||||
"a": "b",
|
||||
}
|
||||
var o2 = 1
|
||||
var w = bytes.NewBuffer(nil)
|
||||
var enc = sonic.ConfigDefault.NewEncoder(w)
|
||||
enc.Encode(o1)
|
||||
enc.Encode(o2)
|
||||
fmt.Println(w.String())
|
||||
// Output:
|
||||
// {"a":"b"}
|
||||
// 1
|
||||
```
|
||||
- 解码器
|
||||
```go
|
||||
var o = map[string]interface{}{}
|
||||
var r = strings.NewReader(`{"a":"b"}{"1":"2"}`)
|
||||
var dec = sonic.ConfigDefault.NewDecoder(r)
|
||||
dec.Decode(&o)
|
||||
dec.Decode(&o)
|
||||
fmt.Printf("%+v", o)
|
||||
// Output:
|
||||
// map[1:2 a:b]
|
||||
```
|
||||
|
||||
### 使用 `Number` / `int64`
|
||||
|
||||
```go
|
||||
import "github.com/bytedance/sonic/decoder"
|
||||
|
||||
var input = `1`
|
||||
var data interface{}
|
||||
|
||||
// default float64
|
||||
dc := decoder.NewDecoder(input)
|
||||
dc.Decode(&data) // data == float64(1)
|
||||
// use json.Number
|
||||
dc = decoder.NewDecoder(input)
|
||||
dc.UseNumber()
|
||||
dc.Decode(&data) // data == json.Number("1")
|
||||
// use int64
|
||||
dc = decoder.NewDecoder(input)
|
||||
dc.UseInt64()
|
||||
dc.Decode(&data) // data == int64(1)
|
||||
|
||||
root, err := sonic.GetFromString(input)
|
||||
// Get json.Number
|
||||
jn := root.Number()
|
||||
jm := root.InterfaceUseNumber().(json.Number) // jn == jm
|
||||
// Get float64
|
||||
fn := root.Float64()
|
||||
fm := root.Interface().(float64) // jn == jm
|
||||
```
|
||||
|
||||
### 对键排序
|
||||
|
||||
考虑到排序带来的性能损失(约 10% ), sonic 默认不会启用这个功能。如果你的组件依赖这个行为(如 [zstd](https://github.com/facebook/zstd)) ,可以仿照下面的例子:
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
import "github.com/bytedance/sonic/encoder"
|
||||
|
||||
// Binding map only
|
||||
m := map[string]interface{}{}
|
||||
v, err := encoder.Encode(m, encoder.SortMapKeys)
|
||||
|
||||
// Or ast.Node.SortKeys() before marshal
|
||||
var root := sonic.Get(JSON)
|
||||
err := root.SortKeys()
|
||||
```
|
||||
|
||||
### HTML 转义
|
||||
|
||||
考虑到性能损失(约15%), sonic 默认不会启用这个功能。你可以使用 `encoder.EscapeHTML` 选项来开启(与 `encoding/json.HTMLEscape` 行为一致)。
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
v := map[string]string{"&&":"<>"}
|
||||
ret, err := Encode(v, EscapeHTML) // ret == `{"\u0026\u0026":{"X":"\u003c\u003e"}}`
|
||||
```
|
||||
|
||||
### 紧凑格式
|
||||
Sonic 默认将基本类型( `struct` , `map` 等)编码为紧凑格式的 JSON ,除非使用 `json.RawMessage` or `json.Marshaler` 进行编码: sonic 确保输出的 JSON 合法,但出于性能考虑,**不会**加工成紧凑格式。我们提供选项 `encoder.CompactMarshaler` 来添加此过程,
|
||||
|
||||
### 打印错误
|
||||
|
||||
如果输入的 JSON 存在无效的语法,sonic 将返回 `decoder.SyntaxError`,该错误支持错误位置的美化输出。
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
import "github.com/bytedance/sonic/decoder"
|
||||
|
||||
var data interface{}
|
||||
err := sonic.UnmarshalString("[[[}]]", &data)
|
||||
if err != nil {
|
||||
/* One line by default */
|
||||
println(e.Error()) // "Syntax error at index 3: invalid char\n\n\t[[[}]]\n\t...^..\n"
|
||||
/* Pretty print */
|
||||
if e, ok := err.(decoder.SyntaxError); ok {
|
||||
/*Syntax error at index 3: invalid char
|
||||
|
||||
[[[}]]
|
||||
...^..
|
||||
*/
|
||||
print(e.Description())
|
||||
} else if me, ok := err.(*decoder.MismatchTypeError); ok {
|
||||
// decoder.MismatchTypeError is new to Sonic v1.6.0
|
||||
print(me.Description())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
#### 类型不匹配 [Sonic v1.6.0]
|
||||
|
||||
如果给定键中存在**类型不匹配**的值, sonic 会抛出 `decoder.MismatchTypeError` (如果有多个,只会报告最后一个),但仍会跳过错误的值并解码下一个 JSON 。
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
import "github.com/bytedance/sonic/decoder"
|
||||
|
||||
var data = struct{
|
||||
A int
|
||||
B int
|
||||
}{}
|
||||
err := UnmarshalString(`{"A":"1","B":1}`, &data)
|
||||
println(err.Error()) // Mismatch type int with value string "at index 5: mismatched type with value\n\n\t{\"A\":\"1\",\"B\":1}\n\t.....^.........\n"
|
||||
fmt.Printf("%+v", data) // {A:0 B:1}
|
||||
```
|
||||
### `Ast.Node`
|
||||
|
||||
Sonic/ast.Node 是完全独立的 JSON 抽象语法树库。它实现了序列化和反序列化,并提供了获取和修改通用数据的鲁棒的 API。
|
||||
|
||||
#### 查找/索引
|
||||
|
||||
通过给定的路径搜索 JSON 片段,路径必须为非负整数,字符串或 `nil` 。
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
input := []byte(`{"key1":[{},{"key2":{"key3":[1,2,3]}}]}`)
|
||||
|
||||
// no path, returns entire json
|
||||
root, err := sonic.Get(input)
|
||||
raw := root.Raw() // == string(input)
|
||||
|
||||
// multiple paths
|
||||
root, err := sonic.Get(input, "key1", 1, "key2")
|
||||
sub := root.Get("key3").Index(2).Int64() // == 3
|
||||
```
|
||||
**注意**:由于 `Index()` 使用偏移量来定位数据,比使用扫描的 `Get()` 要快的多,建议尽可能的使用 `Index` 。 Sonic 也提供了另一个 API, `IndexOrGet()` ,以偏移量为基础并且也确保键的匹配。
|
||||
|
||||
#### 修改
|
||||
|
||||
使用 ` Set()` / `Unset()` 修改 json 的内容
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
// Set
|
||||
exist, err := root.Set("key4", NewBool(true)) // exist == false
|
||||
alias1 := root.Get("key4")
|
||||
println(alias1.Valid()) // true
|
||||
alias2 := root.Index(1)
|
||||
println(alias1 == alias2) // true
|
||||
|
||||
// Unset
|
||||
exist, err := root.UnsetByIndex(1) // exist == true
|
||||
println(root.Get("key4").Check()) // "value not exist"
|
||||
```
|
||||
|
||||
#### 序列化
|
||||
要将 `ast.Node` 编码为 json ,使用 `MarshalJson()` 或者 `json.Marshal()` (必须传递指向节点的指针)
|
||||
```go
|
||||
import (
|
||||
"encoding/json"
|
||||
"github.com/bytedance/sonic"
|
||||
)
|
||||
|
||||
buf, err := root.MarshalJson()
|
||||
println(string(buf)) // {"key1":[{},{"key2":{"key3":[1,2,3]}}]}
|
||||
exp, err := json.Marshal(&root) // WARN: use pointer
|
||||
println(string(buf) == string(exp)) // true
|
||||
```
|
||||
|
||||
#### APIs
|
||||
- 合法性检查: `Check()`, `Error()`, `Valid()`, `Exist()`
|
||||
- 索引: `Index()`, `Get()`, `IndexPair()`, `IndexOrGet()`, `GetByPath()`
|
||||
- 转换至 go 内置类型: `Int64()`, `Float64()`, `String()`, `Number()`, `Bool()`, `Map[UseNumber|UseNode]()`, `Array[UseNumber|UseNode]()`, `Interface[UseNumber|UseNode]()`
|
||||
- go 类型打包: `NewRaw()`, `NewNumber()`, `NewNull()`, `NewBool()`, `NewString()`, `NewObject()`, `NewArray()`
|
||||
- 迭代: `Values()`, `Properties()`, `ForEach()`, `SortKeys()`
|
||||
- 修改: `Set()`, `SetByIndex()`, `Add()`
|
||||
|
||||
## 兼容性
|
||||
由于开发高性能代码的困难性, Sonic **不**保证对所有环境的支持。对于在不同环境中使用 Sonic 构建应用程序的开发者,我们有以下建议:
|
||||
|
||||
- 在 **Mac M1** 上开发:确保在您的计算机上安装了 Rosetta 2,并在构建时设置 `GOARCH=amd64` 。 Rosetta 2 可以自动将 x86 二进制文件转换为 arm64 二进制文件,并在 Mac M1 上运行 x86 应用程序。
|
||||
- 在 **Linux arm64** 上开发:您可以安装 qemu 并使用 `qemu-x86_64 -cpu max` 命令来将 x86 二进制文件转换为 arm64 二进制文件。qemu可以实现与Mac M1上的Rosetta 2类似的转换效果。
|
||||
|
||||
对于希望在不使用 qemu 下使用 sonic 的开发者,或者希望处理 JSON 时与 `encoding/JSON` 严格保持一致的开发者,我们在 `sonic.API` 中提供了一些兼容性 API
|
||||
- `ConfigDefault`: 在支持 sonic 的环境下 sonic 的默认配置(`EscapeHTML=false`,`SortKeys=false`等)。行为与具有相应配置的 `encoding/json` 一致,一些选项,如 `SortKeys=false` 将无效。
|
||||
- `ConfigStd`: 在支持 sonic 的环境下与标准库兼容的配置(`EscapeHTML=true`,`SortKeys=true`等)。行为与 `encoding/json` 一致。
|
||||
- `ConfigFastest`: 在支持 sonic 的环境下运行最快的配置(`NoQuoteTextMarshaler=true`)。行为与具有相应配置的 `encoding/json` 一致,某些选项将无效。
|
||||
|
||||
## 注意事项
|
||||
|
||||
### 预热
|
||||
由于 Sonic 使用 [golang-asm](https://github.com/twitchyliquid64/golang-asm) 作为 JIT 汇编器,这个库并不适用于运行时编译,第一次运行一个大型模式可能会导致请求超时甚至进程内存溢出。为了更好地稳定性,我们建议在运行大型模式或在内存有限的应用中,在使用 `Marshal()/Unmarshal()` 前运行 `Pretouch()`。
|
||||
```go
|
||||
import (
|
||||
"reflect"
|
||||
"github.com/bytedance/sonic"
|
||||
"github.com/bytedance/sonic/option"
|
||||
)
|
||||
|
||||
func init() {
|
||||
var v HugeStruct
|
||||
|
||||
// For most large types (nesting depth <= option.DefaultMaxInlineDepth)
|
||||
err := sonic.Pretouch(reflect.TypeOf(v))
|
||||
|
||||
// with more CompileOption...
|
||||
err := sonic.Pretouch(reflect.TypeOf(v),
|
||||
// If the type is too deep nesting (nesting depth > option.DefaultMaxInlineDepth),
|
||||
// you can set compile recursive loops in Pretouch for better stability in JIT.
|
||||
option.WithCompileRecursiveDepth(loop),
|
||||
// For a large nested struct, try to set a smaller depth to reduce compiling time.
|
||||
option.WithCompileMaxInlineDepth(depth),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
### 拷贝字符串
|
||||
|
||||
当解码 **没有转义字符的字符串**时, sonic 会从原始的 JSON 缓冲区内引用而不是复制到新的一个缓冲区中。这对 CPU 的性能方面很有帮助,但是可能因此在解码后对象仍在使用的时候将整个 JSON 缓冲区保留在内存中。实践中我们发现,通过引用 JSON 缓冲区引入的额外内存通常是解码后对象的 20% 至 80% ,一旦应用长期保留这些对象(如缓存以备重用),服务器所使用的内存可能会增加。我们提供了选项 `decoder.CopyString()` 供用户选择,不引用 JSON 缓冲区。这可能在一定程度上降低 CPU 性能。
|
||||
|
||||
### 传递字符串还是字节数组?
|
||||
为了和 `encoding/json` 保持一致,我们提供了传递 `[]byte` 作为参数的 API ,但考虑到安全性,字符串到字节的复制是同时进行的,这在原始 JSON 非常大时可能会导致性能损失。因此,你可以使用 `UnmarshalString()` 和 `GetFromString()` 来传递字符串,只要你的原始数据是字符串,或**零拷贝类型转换**对于你的字节数组是安全的。我们也提供了 `MarshalString()` 的 API ,以便对编码的 JSON 字节数组进行**零拷贝类型转换**,因为 sonic 输出的字节始终是重复并且唯一的,所以这样是安全的。
|
||||
|
||||
### 加速 `encoding.TextMarshaler`
|
||||
|
||||
为了保证数据安全性, `sonic.Encoder` 默认会对来自 `encoding.TextMarshaler` 接口的字符串进行引用和转义,如果大部分数据都是这种形式那可能会导致很大的性能损失。我们提供了 `encoder.NoQuoteTextMarshaler` 选项来跳过这些操作,但你**必须**保证他们的输出字符串依照 [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259) 进行了转义和引用。
|
||||
|
||||
|
||||
### 泛型的性能优化
|
||||
|
||||
在 **完全解析**的场景下, `Unmarshal()` 表现得比 `Get()`+`Node.Interface()` 更好。但是如果你只有特定 JSON 的部分模式,你可以将 `Get()` 和 `Unmarshal()` 结合使用:
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
node, err := sonic.GetFromString(_TwitterJson, "statuses", 3, "user")
|
||||
var user User // your partial schema...
|
||||
err = sonic.UnmarshalString(node.Raw(), &user)
|
||||
```
|
||||
甚至如果你没有任何模式,可以用 `ast.Node` 代替 `map` 或 `interface` 作为泛型的容器:
|
||||
```go
|
||||
import "github.com/bytedance/sonic"
|
||||
|
||||
root, err := sonic.GetFromString(_TwitterJson)
|
||||
user := root.GetByPath("statuses", 3, "user") // === root.Get("status").Index(3).Get("user")
|
||||
err = user.Check()
|
||||
|
||||
// err = user.LoadAll() // only call this when you want to use 'user' concurrently...
|
||||
go someFunc(user)
|
||||
```
|
||||
为什么?因为 `ast.Node` 使用 `array` 来存储其子节点:
|
||||
- 在插入(反序列化)和扫描(序列化)数据时,`Array` 的性能比 `Map` **好得多**;
|
||||
- **哈希**(`map[x]`)的效率不如**索引**(`array[x]`)高效,而 `ast.Node` 可以在数组和对象上使用索引;
|
||||
- 使用 `Interface()` / `Map()` 意味着 sonic 必须解析所有的底层值,而 `ast.Node` 可以**按需解析**它们。
|
||||
|
||||
**注意**:由于 `ast.Node` 的惰性加载设计,其**不能**直接保证并发安全性,但你可以调用 `Node.Load()` / `Node.LoadAll()` 来实现并发安全。尽管可能会带来性能损失,但仍比转换成 `map` 或 `interface{}` 更为高效。
|
||||
|
||||
## 社区
|
||||
|
||||
Sonic 是 [CloudWeGo](https://www.cloudwego.io/) 下的一个子项目。我们致力于构建云原生生态系统。
|
||||
186
vendor/github.com/bytedance/sonic/api.go
generated
vendored
Normal file
186
vendor/github.com/bytedance/sonic/api.go
generated
vendored
Normal file
@@ -0,0 +1,186 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package sonic
|
||||
|
||||
import (
|
||||
`io`
|
||||
|
||||
`github.com/bytedance/sonic/ast`
|
||||
)
|
||||
|
||||
// Config is a combination of sonic/encoder.Options and sonic/decoder.Options
|
||||
type Config struct {
|
||||
// EscapeHTML indicates encoder to escape all HTML characters
|
||||
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
EscapeHTML bool
|
||||
|
||||
// SortMapKeys indicates encoder that the keys of a map needs to be sorted
|
||||
// before serializing into JSON.
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
SortMapKeys bool
|
||||
|
||||
// CompactMarshaler indicates encoder that the output JSON from json.Marshaler
|
||||
// is always compact and needs no validation
|
||||
CompactMarshaler bool
|
||||
|
||||
// NoQuoteTextMarshaler indicates encoder that the output text from encoding.TextMarshaler
|
||||
// is always escaped string and needs no quoting
|
||||
NoQuoteTextMarshaler bool
|
||||
|
||||
// NoNullSliceOrMap indicates encoder that all empty Array or Object are encoded as '[]' or '{}',
|
||||
// instead of 'null'
|
||||
NoNullSliceOrMap bool
|
||||
|
||||
// UseInt64 indicates decoder to unmarshal an integer into an interface{} as an
|
||||
// int64 instead of as a float64.
|
||||
UseInt64 bool
|
||||
|
||||
// UseNumber indicates decoder to unmarshal a number into an interface{} as a
|
||||
// json.Number instead of as a float64.
|
||||
UseNumber bool
|
||||
|
||||
// UseUnicodeErrors indicates decoder to return an error when encounter invalid
|
||||
// UTF-8 escape sequences.
|
||||
UseUnicodeErrors bool
|
||||
|
||||
// DisallowUnknownFields indicates decoder to return an error when the destination
|
||||
// is a struct and the input contains object keys which do not match any
|
||||
// non-ignored, exported fields in the destination.
|
||||
DisallowUnknownFields bool
|
||||
|
||||
// CopyString indicates decoder to decode string values by copying instead of referring.
|
||||
CopyString bool
|
||||
|
||||
// ValidateString indicates decoder and encoder to valid string values: decoder will return errors
|
||||
// when unescaped control chars(\u0000-\u001f) in the string value of JSON.
|
||||
ValidateString bool
|
||||
}
|
||||
|
||||
var (
|
||||
// ConfigDefault is the default config of APIs, aiming at efficiency and safty.
|
||||
ConfigDefault = Config{}.Froze()
|
||||
|
||||
// ConfigStd is the standard config of APIs, aiming at being compatible with encoding/json.
|
||||
ConfigStd = Config{
|
||||
EscapeHTML : true,
|
||||
SortMapKeys: true,
|
||||
CompactMarshaler: true,
|
||||
CopyString : true,
|
||||
ValidateString : true,
|
||||
}.Froze()
|
||||
|
||||
// ConfigFastest is the fastest config of APIs, aiming at speed.
|
||||
ConfigFastest = Config{
|
||||
NoQuoteTextMarshaler: true,
|
||||
}.Froze()
|
||||
)
|
||||
|
||||
|
||||
// API is a binding of specific config.
|
||||
// This interface is inspired by github.com/json-iterator/go,
|
||||
// and has same behaviors under equavilent config.
|
||||
type API interface {
|
||||
// MarshalToString returns the JSON encoding string of v
|
||||
MarshalToString(v interface{}) (string, error)
|
||||
// Marshal returns the JSON encoding bytes of v.
|
||||
Marshal(v interface{}) ([]byte, error)
|
||||
// MarshalIndent returns the JSON encoding bytes with indent and prefix.
|
||||
MarshalIndent(v interface{}, prefix, indent string) ([]byte, error)
|
||||
// UnmarshalFromString parses the JSON-encoded bytes and stores the result in the value pointed to by v.
|
||||
UnmarshalFromString(str string, v interface{}) error
|
||||
// Unmarshal parses the JSON-encoded string and stores the result in the value pointed to by v.
|
||||
Unmarshal(data []byte, v interface{}) error
|
||||
// NewEncoder create a Encoder holding writer
|
||||
NewEncoder(writer io.Writer) Encoder
|
||||
// NewDecoder create a Decoder holding reader
|
||||
NewDecoder(reader io.Reader) Decoder
|
||||
// Valid validates the JSON-encoded bytes and reportes if it is valid
|
||||
Valid(data []byte) bool
|
||||
}
|
||||
|
||||
// Encoder encodes JSON into io.Writer
|
||||
type Encoder interface {
|
||||
// Encode writes the JSON encoding of v to the stream, followed by a newline character.
|
||||
Encode(val interface{}) error
|
||||
// SetEscapeHTML specifies whether problematic HTML characters
|
||||
// should be escaped inside JSON quoted strings.
|
||||
// The default behavior NOT ESCAPE
|
||||
SetEscapeHTML(on bool)
|
||||
// SetIndent instructs the encoder to format each subsequent encoded value
|
||||
// as if indented by the package-level function Indent(dst, src, prefix, indent).
|
||||
// Calling SetIndent("", "") disables indentation
|
||||
SetIndent(prefix, indent string)
|
||||
}
|
||||
|
||||
// Decoder decodes JSON from io.Read
|
||||
type Decoder interface {
|
||||
// Decode reads the next JSON-encoded value from its input and stores it in the value pointed to by v.
|
||||
Decode(val interface{}) error
|
||||
// Buffered returns a reader of the data remaining in the Decoder's buffer.
|
||||
// The reader is valid until the next call to Decode.
|
||||
Buffered() io.Reader
|
||||
// DisallowUnknownFields causes the Decoder to return an error when the destination is a struct
|
||||
// and the input contains object keys which do not match any non-ignored, exported fields in the destination.
|
||||
DisallowUnknownFields()
|
||||
// More reports whether there is another element in the current array or object being parsed.
|
||||
More() bool
|
||||
// UseNumber causes the Decoder to unmarshal a number into an interface{} as a Number instead of as a float64.
|
||||
UseNumber()
|
||||
}
|
||||
|
||||
// Marshal returns the JSON encoding bytes of v.
|
||||
func Marshal(val interface{}) ([]byte, error) {
|
||||
return ConfigDefault.Marshal(val)
|
||||
}
|
||||
|
||||
// MarshalString returns the JSON encoding string of v.
|
||||
func MarshalString(val interface{}) (string, error) {
|
||||
return ConfigDefault.MarshalToString(val)
|
||||
}
|
||||
|
||||
// Unmarshal parses the JSON-encoded data and stores the result in the value pointed to by v.
|
||||
// NOTICE: This API copies given buffer by default,
|
||||
// if you want to pass JSON more efficiently, use UnmarshalString instead.
|
||||
func Unmarshal(buf []byte, val interface{}) error {
|
||||
return ConfigDefault.Unmarshal(buf, val)
|
||||
}
|
||||
|
||||
// UnmarshalString is like Unmarshal, except buf is a string.
|
||||
func UnmarshalString(buf string, val interface{}) error {
|
||||
return ConfigDefault.UnmarshalFromString(buf, val)
|
||||
}
|
||||
|
||||
// Get searches the given path from json,
|
||||
// and returns its representing ast.Node.
|
||||
//
|
||||
// Each path arg must be integer or string:
|
||||
// - Integer is target index(>=0), means searching current node as array.
|
||||
// - String is target key, means searching current node as object.
|
||||
//
|
||||
//
|
||||
// Note, the api expects the json is well-formed at least,
|
||||
// otherwise it may return unexpected result.
|
||||
func Get(src []byte, path ...interface{}) (ast.Node, error) {
|
||||
return GetFromString(string(src), path...)
|
||||
}
|
||||
|
||||
// GetFromString is same with Get except src is string,
|
||||
// which can reduce unnecessary memory copy.
|
||||
func GetFromString(src string, path ...interface{}) (ast.Node, error) {
|
||||
return ast.NewSearcher(src).GetByPath(path...)
|
||||
}
|
||||
151
vendor/github.com/bytedance/sonic/ast/api_amd64.go
generated
vendored
Normal file
151
vendor/github.com/bytedance/sonic/ast/api_amd64.go
generated
vendored
Normal file
@@ -0,0 +1,151 @@
|
||||
// +build amd64,go1.15,!go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2022 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`runtime`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/encoder`
|
||||
`github.com/bytedance/sonic/internal/native`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
uq `github.com/bytedance/sonic/unquote`
|
||||
`github.com/chenzhuoyu/base64x`
|
||||
)
|
||||
|
||||
var typeByte = rt.UnpackEface(byte(0)).Type
|
||||
|
||||
//go:nocheckptr
|
||||
func quote(buf *[]byte, val string) {
|
||||
*buf = append(*buf, '"')
|
||||
if len(val) == 0 {
|
||||
*buf = append(*buf, '"')
|
||||
return
|
||||
}
|
||||
|
||||
sp := rt.IndexChar(val, 0)
|
||||
nb := len(val)
|
||||
b := (*rt.GoSlice)(unsafe.Pointer(buf))
|
||||
|
||||
// input buffer
|
||||
for nb > 0 {
|
||||
// output buffer
|
||||
dp := unsafe.Pointer(uintptr(b.Ptr) + uintptr(b.Len))
|
||||
dn := b.Cap - b.Len
|
||||
// call native.Quote, dn is byte count it outputs
|
||||
ret := native.Quote(sp, nb, dp, &dn, 0)
|
||||
// update *buf length
|
||||
b.Len += dn
|
||||
|
||||
// no need more output
|
||||
if ret >= 0 {
|
||||
break
|
||||
}
|
||||
|
||||
// double buf size
|
||||
*b = growslice(typeByte, *b, b.Cap*2)
|
||||
// ret is the complement of consumed input
|
||||
ret = ^ret
|
||||
// update input buffer
|
||||
nb -= ret
|
||||
sp = unsafe.Pointer(uintptr(sp) + uintptr(ret))
|
||||
}
|
||||
|
||||
runtime.KeepAlive(buf)
|
||||
runtime.KeepAlive(sp)
|
||||
*buf = append(*buf, '"')
|
||||
}
|
||||
|
||||
func unquote(src string) (string, types.ParsingError) {
|
||||
return uq.String(src)
|
||||
}
|
||||
|
||||
func decodeBase64(src string) ([]byte, error) {
|
||||
return base64x.StdEncoding.DecodeString(src)
|
||||
}
|
||||
|
||||
func encodeBase64(src []byte) string {
|
||||
return base64x.StdEncoding.EncodeToString(src)
|
||||
}
|
||||
|
||||
func (self *Parser) decodeValue() (val types.JsonState) {
|
||||
sv := (*rt.GoString)(unsafe.Pointer(&self.s))
|
||||
self.p = native.Value(sv.Ptr, sv.Len, self.p, &val, 0)
|
||||
return
|
||||
}
|
||||
|
||||
func (self *Parser) skip() (int, types.ParsingError) {
|
||||
fsm := types.NewStateMachine()
|
||||
start := native.SkipOne(&self.s, &self.p, fsm, 0)
|
||||
types.FreeStateMachine(fsm)
|
||||
|
||||
if start < 0 {
|
||||
return self.p, types.ParsingError(-start)
|
||||
}
|
||||
return start, 0
|
||||
}
|
||||
|
||||
func (self *Node) encodeInterface(buf *[]byte) error {
|
||||
//WARN: NOT compatible with json.Encoder
|
||||
return encoder.EncodeInto(buf, self.packAny(), 0)
|
||||
}
|
||||
|
||||
func (self *Parser) skipFast() (int, types.ParsingError) {
|
||||
start := native.SkipOneFast(&self.s, &self.p)
|
||||
if start < 0 {
|
||||
return self.p, types.ParsingError(-start)
|
||||
}
|
||||
return start, 0
|
||||
}
|
||||
|
||||
func (self *Parser) getByPath(path ...interface{}) (int, types.ParsingError) {
|
||||
fsm := types.NewStateMachine()
|
||||
start := native.GetByPath(&self.s, &self.p, &path, fsm)
|
||||
types.FreeStateMachine(fsm)
|
||||
runtime.KeepAlive(path)
|
||||
if start < 0 {
|
||||
return self.p, types.ParsingError(-start)
|
||||
}
|
||||
return start, 0
|
||||
}
|
||||
|
||||
func (self *Searcher) GetByPath(path ...interface{}) (Node, error) {
|
||||
var err types.ParsingError
|
||||
var start int
|
||||
|
||||
self.parser.p = 0
|
||||
start, err = self.parser.getByPath(path...)
|
||||
if err != 0 {
|
||||
// for compatibility with old version
|
||||
if err == types.ERR_NOT_FOUND {
|
||||
return Node{}, ErrNotExist
|
||||
}
|
||||
if err == types.ERR_UNSUPPORT_TYPE {
|
||||
panic("path must be either int(>=0) or string")
|
||||
}
|
||||
return Node{}, self.parser.syntaxError(err)
|
||||
}
|
||||
|
||||
t := switchRawType(self.parser.s[start])
|
||||
if t == _V_NONE {
|
||||
return Node{}, self.parser.ExportError(err)
|
||||
}
|
||||
return newRawNode(self.parser.s[start:self.parser.p], t), nil
|
||||
}
|
||||
120
vendor/github.com/bytedance/sonic/ast/api_compat.go
generated
vendored
Normal file
120
vendor/github.com/bytedance/sonic/ast/api_compat.go
generated
vendored
Normal file
@@ -0,0 +1,120 @@
|
||||
// +build !amd64 go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2022 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`encoding/base64`
|
||||
`encoding/json`
|
||||
`fmt`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
func quote(buf *[]byte, val string) {
|
||||
quoteString(buf, val)
|
||||
}
|
||||
|
||||
func unquote(src string) (string, types.ParsingError) {
|
||||
sp := rt.IndexChar(src, -1)
|
||||
out, ok := unquoteBytes(rt.BytesFrom(sp, len(src)+2, len(src)+2))
|
||||
if !ok {
|
||||
return "", types.ERR_INVALID_ESCAPE
|
||||
}
|
||||
return rt.Mem2Str(out), 0
|
||||
}
|
||||
|
||||
func decodeBase64(src string) ([]byte, error) {
|
||||
return base64.StdEncoding.DecodeString(src)
|
||||
}
|
||||
|
||||
func encodeBase64(src []byte) string {
|
||||
return base64.StdEncoding.EncodeToString(src)
|
||||
}
|
||||
|
||||
func (self *Parser) decodeValue() (val types.JsonState) {
|
||||
e, v := decodeValue(self.s, self.p)
|
||||
if e < 0 {
|
||||
return v
|
||||
}
|
||||
self.p = e
|
||||
return v
|
||||
}
|
||||
|
||||
func (self *Parser) skip() (int, types.ParsingError) {
|
||||
e, s := skipValue(self.s, self.p)
|
||||
if e < 0 {
|
||||
return self.p, types.ParsingError(-e)
|
||||
}
|
||||
self.p = e
|
||||
return s, 0
|
||||
}
|
||||
|
||||
func (self *Parser) skipFast() (int, types.ParsingError) {
|
||||
e, s := skipValueFast(self.s, self.p)
|
||||
if e < 0 {
|
||||
return self.p, types.ParsingError(-e)
|
||||
}
|
||||
self.p = e
|
||||
return s, 0
|
||||
}
|
||||
|
||||
func (self *Node) encodeInterface(buf *[]byte) error {
|
||||
out, err := json.Marshal(self.packAny())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*buf = append(*buf, out...)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Searcher) GetByPath(path ...interface{}) (Node, error) {
|
||||
self.parser.p = 0
|
||||
|
||||
var err types.ParsingError
|
||||
for _, p := range path {
|
||||
if idx, ok := p.(int); ok && idx >= 0 {
|
||||
if err = self.parser.searchIndex(idx); err != 0 {
|
||||
return Node{}, self.parser.ExportError(err)
|
||||
}
|
||||
} else if key, ok := p.(string); ok {
|
||||
if err = self.parser.searchKey(key); err != 0 {
|
||||
return Node{}, self.parser.ExportError(err)
|
||||
}
|
||||
} else {
|
||||
panic("path must be either int(>=0) or string")
|
||||
}
|
||||
}
|
||||
|
||||
var start = self.parser.p
|
||||
if start, err = self.parser.skip(); err != 0 {
|
||||
return Node{}, self.parser.ExportError(err)
|
||||
}
|
||||
ns := len(self.parser.s)
|
||||
if self.parser.p > ns || start >= ns || start>=self.parser.p {
|
||||
return Node{}, fmt.Errorf("skip %d char out of json boundary", start)
|
||||
}
|
||||
|
||||
t := switchRawType(self.parser.s[start])
|
||||
if t == _V_NONE {
|
||||
return Node{}, self.parser.ExportError(err)
|
||||
}
|
||||
|
||||
return newRawNode(self.parser.s[start:self.parser.p], t), nil
|
||||
}
|
||||
0
vendor/github.com/bytedance/sonic/ast/asm.s
generated
vendored
Normal file
0
vendor/github.com/bytedance/sonic/ast/asm.s
generated
vendored
Normal file
575
vendor/github.com/bytedance/sonic/ast/decode.go
generated
vendored
Normal file
575
vendor/github.com/bytedance/sonic/ast/decode.go
generated
vendored
Normal file
@@ -0,0 +1,575 @@
|
||||
/*
|
||||
* Copyright 2022 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`encoding/base64`
|
||||
`runtime`
|
||||
`strconv`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
const _blankCharsMask = (1 << ' ') | (1 << '\t') | (1 << '\r') | (1 << '\n')
|
||||
|
||||
const (
|
||||
bytesNull = "null"
|
||||
bytesTrue = "true"
|
||||
bytesFalse = "false"
|
||||
bytesObject = "{}"
|
||||
bytesArray = "[]"
|
||||
)
|
||||
|
||||
func isSpace(c byte) bool {
|
||||
return (int(1<<c) & _blankCharsMask) != 0
|
||||
}
|
||||
|
||||
//go:nocheckptr
|
||||
func skipBlank(src string, pos int) int {
|
||||
se := uintptr(rt.IndexChar(src, len(src)))
|
||||
sp := uintptr(rt.IndexChar(src, pos))
|
||||
|
||||
for sp < se {
|
||||
if !isSpace(*(*byte)(unsafe.Pointer(sp))) {
|
||||
break
|
||||
}
|
||||
sp += 1
|
||||
}
|
||||
if sp >= se {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
runtime.KeepAlive(src)
|
||||
return int(sp - uintptr(rt.IndexChar(src, 0)))
|
||||
}
|
||||
|
||||
func decodeNull(src string, pos int) (ret int) {
|
||||
ret = pos + 4
|
||||
if ret > len(src) {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
if src[pos:ret] == bytesNull {
|
||||
return ret
|
||||
} else {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
}
|
||||
|
||||
func decodeTrue(src string, pos int) (ret int) {
|
||||
ret = pos + 4
|
||||
if ret > len(src) {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
if src[pos:ret] == bytesTrue {
|
||||
return ret
|
||||
} else {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func decodeFalse(src string, pos int) (ret int) {
|
||||
ret = pos + 5
|
||||
if ret > len(src) {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
if src[pos:ret] == bytesFalse {
|
||||
return ret
|
||||
}
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
|
||||
//go:nocheckptr
|
||||
func decodeString(src string, pos int) (ret int, v string) {
|
||||
ret, ep := skipString(src, pos)
|
||||
if ep == -1 {
|
||||
(*rt.GoString)(unsafe.Pointer(&v)).Ptr = rt.IndexChar(src, pos+1)
|
||||
(*rt.GoString)(unsafe.Pointer(&v)).Len = ret - pos - 2
|
||||
return ret, v
|
||||
}
|
||||
|
||||
vv, ok := unquoteBytes(rt.Str2Mem(src[pos:ret]))
|
||||
if !ok {
|
||||
return -int(types.ERR_INVALID_CHAR), ""
|
||||
}
|
||||
|
||||
runtime.KeepAlive(src)
|
||||
return ret, rt.Mem2Str(vv)
|
||||
}
|
||||
|
||||
func decodeBinary(src string, pos int) (ret int, v []byte) {
|
||||
var vv string
|
||||
ret, vv = decodeString(src, pos)
|
||||
if ret < 0 {
|
||||
return ret, nil
|
||||
}
|
||||
var err error
|
||||
v, err = base64.StdEncoding.DecodeString(vv)
|
||||
if err != nil {
|
||||
return -int(types.ERR_INVALID_CHAR), nil
|
||||
}
|
||||
return ret, v
|
||||
}
|
||||
|
||||
func isDigit(c byte) bool {
|
||||
return c >= '0' && c <= '9'
|
||||
}
|
||||
|
||||
//go:nocheckptr
|
||||
func decodeInt64(src string, pos int) (ret int, v int64, err error) {
|
||||
sp := uintptr(rt.IndexChar(src, pos))
|
||||
ss := uintptr(sp)
|
||||
se := uintptr(rt.IndexChar(src, len(src)))
|
||||
if uintptr(sp) >= se {
|
||||
return -int(types.ERR_EOF), 0, nil
|
||||
}
|
||||
|
||||
if c := *(*byte)(unsafe.Pointer(sp)); c == '-' {
|
||||
sp += 1
|
||||
}
|
||||
if sp == se {
|
||||
return -int(types.ERR_EOF), 0, nil
|
||||
}
|
||||
|
||||
for ; sp < se; sp += uintptr(1) {
|
||||
if !isDigit(*(*byte)(unsafe.Pointer(sp))) {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if sp < se {
|
||||
if c := *(*byte)(unsafe.Pointer(sp)); c == '.' || c == 'e' || c == 'E' {
|
||||
return -int(types.ERR_INVALID_NUMBER_FMT), 0, nil
|
||||
}
|
||||
}
|
||||
|
||||
var vv string
|
||||
ret = int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
|
||||
(*rt.GoString)(unsafe.Pointer(&vv)).Ptr = unsafe.Pointer(ss)
|
||||
(*rt.GoString)(unsafe.Pointer(&vv)).Len = ret - pos
|
||||
|
||||
v, err = strconv.ParseInt(vv, 10, 64)
|
||||
if err != nil {
|
||||
//NOTICE: allow overflow here
|
||||
if err.(*strconv.NumError).Err == strconv.ErrRange {
|
||||
return ret, 0, err
|
||||
}
|
||||
return -int(types.ERR_INVALID_CHAR), 0, err
|
||||
}
|
||||
|
||||
runtime.KeepAlive(src)
|
||||
return ret, v, nil
|
||||
}
|
||||
|
||||
func isNumberChars(c byte) bool {
|
||||
return (c >= '0' && c <= '9') || c == '+' || c == '-' || c == 'e' || c == 'E' || c == '.'
|
||||
}
|
||||
|
||||
//go:nocheckptr
|
||||
func decodeFloat64(src string, pos int) (ret int, v float64, err error) {
|
||||
sp := uintptr(rt.IndexChar(src, pos))
|
||||
ss := uintptr(sp)
|
||||
se := uintptr(rt.IndexChar(src, len(src)))
|
||||
if uintptr(sp) >= se {
|
||||
return -int(types.ERR_EOF), 0, nil
|
||||
}
|
||||
|
||||
if c := *(*byte)(unsafe.Pointer(sp)); c == '-' {
|
||||
sp += 1
|
||||
}
|
||||
if sp == se {
|
||||
return -int(types.ERR_EOF), 0, nil
|
||||
}
|
||||
|
||||
for ; sp < se; sp += uintptr(1) {
|
||||
if !isNumberChars(*(*byte)(unsafe.Pointer(sp))) {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
var vv string
|
||||
ret = int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
|
||||
(*rt.GoString)(unsafe.Pointer(&vv)).Ptr = unsafe.Pointer(ss)
|
||||
(*rt.GoString)(unsafe.Pointer(&vv)).Len = ret - pos
|
||||
|
||||
v, err = strconv.ParseFloat(vv, 64)
|
||||
if err != nil {
|
||||
//NOTICE: allow overflow here
|
||||
if err.(*strconv.NumError).Err == strconv.ErrRange {
|
||||
return ret, 0, err
|
||||
}
|
||||
return -int(types.ERR_INVALID_CHAR), 0, err
|
||||
}
|
||||
|
||||
runtime.KeepAlive(src)
|
||||
return ret, v, nil
|
||||
}
|
||||
|
||||
func decodeValue(src string, pos int) (ret int, v types.JsonState) {
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, types.JsonState{Vt: types.ValueType(pos)}
|
||||
}
|
||||
switch c := src[pos]; c {
|
||||
case 'n':
|
||||
ret = decodeNull(src, pos)
|
||||
if ret < 0 {
|
||||
return ret, types.JsonState{Vt: types.ValueType(ret)}
|
||||
}
|
||||
return ret, types.JsonState{Vt: types.V_NULL}
|
||||
case '"':
|
||||
var ep int
|
||||
ret, ep = skipString(src, pos)
|
||||
if ret < 0 {
|
||||
return ret, types.JsonState{Vt: types.ValueType(ret)}
|
||||
}
|
||||
return ret, types.JsonState{Vt: types.V_STRING, Iv: int64(pos + 1), Ep: ep}
|
||||
case '{':
|
||||
return pos + 1, types.JsonState{Vt: types.V_OBJECT}
|
||||
case '[':
|
||||
return pos + 1, types.JsonState{Vt: types.V_ARRAY}
|
||||
case 't':
|
||||
ret = decodeTrue(src, pos)
|
||||
if ret < 0 {
|
||||
return ret, types.JsonState{Vt: types.ValueType(ret)}
|
||||
}
|
||||
return ret, types.JsonState{Vt: types.V_TRUE}
|
||||
case 'f':
|
||||
ret = decodeFalse(src, pos)
|
||||
if ret < 0 {
|
||||
return ret, types.JsonState{Vt: types.ValueType(ret)}
|
||||
}
|
||||
return ret, types.JsonState{Vt: types.V_FALSE}
|
||||
case '-', '+', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
|
||||
var iv int64
|
||||
ret, iv, _ = decodeInt64(src, pos)
|
||||
if ret >= 0 {
|
||||
return ret, types.JsonState{Vt: types.V_INTEGER, Iv: iv, Ep: pos}
|
||||
} else if ret != -int(types.ERR_INVALID_NUMBER_FMT) {
|
||||
return ret, types.JsonState{Vt: types.ValueType(ret)}
|
||||
}
|
||||
var fv float64
|
||||
ret, fv, _ = decodeFloat64(src, pos)
|
||||
if ret >= 0 {
|
||||
return ret, types.JsonState{Vt: types.V_DOUBLE, Dv: fv, Ep: pos}
|
||||
} else {
|
||||
return ret, types.JsonState{Vt: types.ValueType(ret)}
|
||||
}
|
||||
default:
|
||||
return -int(types.ERR_INVALID_CHAR), types.JsonState{Vt:-types.ValueType(types.ERR_INVALID_CHAR)}
|
||||
}
|
||||
}
|
||||
|
||||
//go:nocheckptr
|
||||
func skipNumber(src string, pos int) (ret int) {
|
||||
sp := uintptr(rt.IndexChar(src, pos))
|
||||
se := uintptr(rt.IndexChar(src, len(src)))
|
||||
if uintptr(sp) >= se {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
|
||||
if c := *(*byte)(unsafe.Pointer(sp)); c == '-' {
|
||||
sp += 1
|
||||
}
|
||||
ss := sp
|
||||
|
||||
var pointer bool
|
||||
var exponent bool
|
||||
var lastIsDigit bool
|
||||
var nextNeedDigit = true
|
||||
|
||||
for ; sp < se; sp += uintptr(1) {
|
||||
c := *(*byte)(unsafe.Pointer(sp))
|
||||
if isDigit(c) {
|
||||
lastIsDigit = true
|
||||
nextNeedDigit = false
|
||||
continue
|
||||
} else if nextNeedDigit {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
} else if c == '.' {
|
||||
if !lastIsDigit || pointer || exponent || sp == ss {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
pointer = true
|
||||
lastIsDigit = false
|
||||
nextNeedDigit = true
|
||||
continue
|
||||
} else if c == 'e' || c == 'E' {
|
||||
if !lastIsDigit || exponent {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
if sp == se-1 {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
exponent = true
|
||||
lastIsDigit = false
|
||||
nextNeedDigit = false
|
||||
continue
|
||||
} else if c == '-' || c == '+' {
|
||||
if prev := *(*byte)(unsafe.Pointer(sp - 1)); prev != 'e' && prev != 'E' {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
lastIsDigit = false
|
||||
nextNeedDigit = true
|
||||
continue
|
||||
} else {
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if nextNeedDigit {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
|
||||
runtime.KeepAlive(src)
|
||||
return int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
|
||||
}
|
||||
|
||||
//go:nocheckptr
|
||||
func skipString(src string, pos int) (ret int, ep int) {
|
||||
if pos+1 >= len(src) {
|
||||
return -int(types.ERR_EOF), -1
|
||||
}
|
||||
|
||||
sp := uintptr(rt.IndexChar(src, pos))
|
||||
se := uintptr(rt.IndexChar(src, len(src)))
|
||||
|
||||
// not start with quote
|
||||
if *(*byte)(unsafe.Pointer(sp)) != '"' {
|
||||
return -int(types.ERR_INVALID_CHAR), -1
|
||||
}
|
||||
sp += 1
|
||||
|
||||
ep = -1
|
||||
for sp < se {
|
||||
c := *(*byte)(unsafe.Pointer(sp))
|
||||
if c == '\\' {
|
||||
if ep == -1 {
|
||||
ep = int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
|
||||
}
|
||||
sp += 2
|
||||
continue
|
||||
}
|
||||
sp += 1
|
||||
if c == '"' {
|
||||
return int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr)), ep
|
||||
}
|
||||
}
|
||||
|
||||
runtime.KeepAlive(src)
|
||||
// not found the closed quote until EOF
|
||||
return -int(types.ERR_EOF), -1
|
||||
}
|
||||
|
||||
//go:nocheckptr
|
||||
func skipPair(src string, pos int, lchar byte, rchar byte) (ret int) {
|
||||
if pos+1 >= len(src) {
|
||||
return -int(types.ERR_EOF)
|
||||
}
|
||||
|
||||
sp := uintptr(rt.IndexChar(src, pos))
|
||||
se := uintptr(rt.IndexChar(src, len(src)))
|
||||
|
||||
if *(*byte)(unsafe.Pointer(sp)) != lchar {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
|
||||
sp += 1
|
||||
nbrace := 1
|
||||
inquote := false
|
||||
|
||||
for sp < se {
|
||||
c := *(*byte)(unsafe.Pointer(sp))
|
||||
if c == '\\' {
|
||||
sp += 2
|
||||
continue
|
||||
} else if c == '"' {
|
||||
inquote = !inquote
|
||||
} else if c == lchar {
|
||||
if !inquote {
|
||||
nbrace += 1
|
||||
}
|
||||
} else if c == rchar {
|
||||
if !inquote {
|
||||
nbrace -= 1
|
||||
if nbrace == 0 {
|
||||
sp += 1
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
sp += 1
|
||||
}
|
||||
|
||||
if nbrace != 0 {
|
||||
return -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
|
||||
runtime.KeepAlive(src)
|
||||
return int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
|
||||
}
|
||||
|
||||
func skipValueFast(src string, pos int) (ret int, start int) {
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
switch c := src[pos]; c {
|
||||
case 'n':
|
||||
ret = decodeNull(src, pos)
|
||||
case '"':
|
||||
ret, _ = skipString(src, pos)
|
||||
case '{':
|
||||
ret = skipPair(src, pos, '{', '}')
|
||||
case '[':
|
||||
ret = skipPair(src, pos, '[', ']')
|
||||
case 't':
|
||||
ret = decodeTrue(src, pos)
|
||||
case 'f':
|
||||
ret = decodeFalse(src, pos)
|
||||
case '-', '+', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
|
||||
ret = skipNumber(src, pos)
|
||||
default:
|
||||
ret = -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
return ret, pos
|
||||
}
|
||||
|
||||
func skipValue(src string, pos int) (ret int, start int) {
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
switch c := src[pos]; c {
|
||||
case 'n':
|
||||
ret = decodeNull(src, pos)
|
||||
case '"':
|
||||
ret, _ = skipString(src, pos)
|
||||
case '{':
|
||||
ret, _ = skipObject(src, pos)
|
||||
case '[':
|
||||
ret, _ = skipArray(src, pos)
|
||||
case 't':
|
||||
ret = decodeTrue(src, pos)
|
||||
case 'f':
|
||||
ret = decodeFalse(src, pos)
|
||||
case '-', '+', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9':
|
||||
ret = skipNumber(src, pos)
|
||||
default:
|
||||
ret = -int(types.ERR_INVALID_CHAR)
|
||||
}
|
||||
return ret, pos
|
||||
}
|
||||
|
||||
func skipObject(src string, pos int) (ret int, start int) {
|
||||
start = skipBlank(src, pos)
|
||||
if start < 0 {
|
||||
return start, -1
|
||||
}
|
||||
|
||||
if src[start] != '{' {
|
||||
return -int(types.ERR_INVALID_CHAR), -1
|
||||
}
|
||||
|
||||
pos = start + 1
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
if src[pos] == '}' {
|
||||
return pos + 1, start
|
||||
}
|
||||
|
||||
for {
|
||||
pos, _ = skipString(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
if src[pos] != ':' {
|
||||
return -int(types.ERR_INVALID_CHAR), -1
|
||||
}
|
||||
|
||||
pos++
|
||||
pos, _ = skipValue(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
if src[pos] == '}' {
|
||||
return pos + 1, start
|
||||
}
|
||||
if src[pos] != ',' {
|
||||
return -int(types.ERR_INVALID_CHAR), -1
|
||||
}
|
||||
|
||||
pos++
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
func skipArray(src string, pos int) (ret int, start int) {
|
||||
start = skipBlank(src, pos)
|
||||
if start < 0 {
|
||||
return start, -1
|
||||
}
|
||||
|
||||
if src[start] != '[' {
|
||||
return -int(types.ERR_INVALID_CHAR), -1
|
||||
}
|
||||
|
||||
pos = start + 1
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
if src[pos] == ']' {
|
||||
return pos + 1, start
|
||||
}
|
||||
|
||||
for {
|
||||
pos, _ = skipValue(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
|
||||
pos = skipBlank(src, pos)
|
||||
if pos < 0 {
|
||||
return pos, -1
|
||||
}
|
||||
if src[pos] == ']' {
|
||||
return pos + 1, start
|
||||
}
|
||||
if src[pos] != ',' {
|
||||
return -int(types.ERR_INVALID_CHAR), -1
|
||||
}
|
||||
pos++
|
||||
}
|
||||
}
|
||||
259
vendor/github.com/bytedance/sonic/ast/encode.go
generated
vendored
Normal file
259
vendor/github.com/bytedance/sonic/ast/encode.go
generated
vendored
Normal file
@@ -0,0 +1,259 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`sync`
|
||||
`unicode/utf8`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
const (
|
||||
_MaxBuffer = 1024 // 1KB buffer size
|
||||
)
|
||||
|
||||
func quoteString(e *[]byte, s string) {
|
||||
*e = append(*e, '"')
|
||||
start := 0
|
||||
for i := 0; i < len(s); {
|
||||
if b := s[i]; b < utf8.RuneSelf {
|
||||
if safeSet[b] {
|
||||
i++
|
||||
continue
|
||||
}
|
||||
if start < i {
|
||||
*e = append(*e, s[start:i]...)
|
||||
}
|
||||
*e = append(*e, '\\')
|
||||
switch b {
|
||||
case '\\', '"':
|
||||
*e = append(*e, b)
|
||||
case '\n':
|
||||
*e = append(*e, 'n')
|
||||
case '\r':
|
||||
*e = append(*e, 'r')
|
||||
case '\t':
|
||||
*e = append(*e, 't')
|
||||
default:
|
||||
// This encodes bytes < 0x20 except for \t, \n and \r.
|
||||
// If escapeHTML is set, it also escapes <, >, and &
|
||||
// because they can lead to security holes when
|
||||
// user-controlled strings are rendered into JSON
|
||||
// and served to some browsers.
|
||||
*e = append(*e, `u00`...)
|
||||
*e = append(*e, hex[b>>4])
|
||||
*e = append(*e, hex[b&0xF])
|
||||
}
|
||||
i++
|
||||
start = i
|
||||
continue
|
||||
}
|
||||
c, size := utf8.DecodeRuneInString(s[i:])
|
||||
// if c == utf8.RuneError && size == 1 {
|
||||
// if start < i {
|
||||
// e.Write(s[start:i])
|
||||
// }
|
||||
// e.WriteString(`\ufffd`)
|
||||
// i += size
|
||||
// start = i
|
||||
// continue
|
||||
// }
|
||||
if c == '\u2028' || c == '\u2029' {
|
||||
if start < i {
|
||||
*e = append(*e, s[start:i]...)
|
||||
}
|
||||
*e = append(*e, `\u202`...)
|
||||
*e = append(*e, hex[c&0xF])
|
||||
i += size
|
||||
start = i
|
||||
continue
|
||||
}
|
||||
i += size
|
||||
}
|
||||
if start < len(s) {
|
||||
*e = append(*e, s[start:]...)
|
||||
}
|
||||
*e = append(*e, '"')
|
||||
}
|
||||
|
||||
var bytesPool = sync.Pool{}
|
||||
|
||||
func (self *Node) MarshalJSON() ([]byte, error) {
|
||||
buf := newBuffer()
|
||||
err := self.encode(buf)
|
||||
if err != nil {
|
||||
freeBuffer(buf)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
ret := make([]byte, len(*buf))
|
||||
copy(ret, *buf)
|
||||
freeBuffer(buf)
|
||||
return ret, err
|
||||
}
|
||||
|
||||
func newBuffer() *[]byte {
|
||||
if ret := bytesPool.Get(); ret != nil {
|
||||
return ret.(*[]byte)
|
||||
} else {
|
||||
buf := make([]byte, 0, _MaxBuffer)
|
||||
return &buf
|
||||
}
|
||||
}
|
||||
|
||||
func freeBuffer(buf *[]byte) {
|
||||
*buf = (*buf)[:0]
|
||||
bytesPool.Put(buf)
|
||||
}
|
||||
|
||||
func (self *Node) encode(buf *[]byte) error {
|
||||
if self.IsRaw() {
|
||||
return self.encodeRaw(buf)
|
||||
}
|
||||
switch self.Type() {
|
||||
case V_NONE : return ErrNotExist
|
||||
case V_ERROR : return self.Check()
|
||||
case V_NULL : return self.encodeNull(buf)
|
||||
case V_TRUE : return self.encodeTrue(buf)
|
||||
case V_FALSE : return self.encodeFalse(buf)
|
||||
case V_ARRAY : return self.encodeArray(buf)
|
||||
case V_OBJECT: return self.encodeObject(buf)
|
||||
case V_STRING: return self.encodeString(buf)
|
||||
case V_NUMBER: return self.encodeNumber(buf)
|
||||
case V_ANY : return self.encodeInterface(buf)
|
||||
default : return ErrUnsupportType
|
||||
}
|
||||
}
|
||||
|
||||
func (self *Node) encodeRaw(buf *[]byte) error {
|
||||
raw, err := self.Raw()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*buf = append(*buf, raw...)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Node) encodeNull(buf *[]byte) error {
|
||||
*buf = append(*buf, bytesNull...)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Node) encodeTrue(buf *[]byte) error {
|
||||
*buf = append(*buf, bytesTrue...)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Node) encodeFalse(buf *[]byte) error {
|
||||
*buf = append(*buf, bytesFalse...)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Node) encodeNumber(buf *[]byte) error {
|
||||
str := rt.StrFrom(self.p, self.v)
|
||||
*buf = append(*buf, str...)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Node) encodeString(buf *[]byte) error {
|
||||
if self.v == 0 {
|
||||
*buf = append(*buf, '"', '"')
|
||||
return nil
|
||||
}
|
||||
|
||||
quote(buf, rt.StrFrom(self.p, self.v))
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Node) encodeArray(buf *[]byte) error {
|
||||
if self.isLazy() {
|
||||
if err := self.skipAllIndex(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
nb := self.len()
|
||||
if nb == 0 {
|
||||
*buf = append(*buf, bytesArray...)
|
||||
return nil
|
||||
}
|
||||
|
||||
*buf = append(*buf, '[')
|
||||
|
||||
var p = (*Node)(self.p)
|
||||
err := p.encode(buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for i := 1; i < nb; i++ {
|
||||
*buf = append(*buf, ',')
|
||||
p = p.unsafe_next()
|
||||
err := p.encode(buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
*buf = append(*buf, ']')
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *Pair) encode(buf *[]byte) error {
|
||||
if len(*buf) == 0 {
|
||||
*buf = append(*buf, '"', '"', ':')
|
||||
return self.Value.encode(buf)
|
||||
}
|
||||
|
||||
quote(buf, self.Key)
|
||||
*buf = append(*buf, ':')
|
||||
|
||||
return self.Value.encode(buf)
|
||||
}
|
||||
|
||||
func (self *Node) encodeObject(buf *[]byte) error {
|
||||
if self.isLazy() {
|
||||
if err := self.skipAllKey(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
nb := self.len()
|
||||
if nb == 0 {
|
||||
*buf = append(*buf, bytesObject...)
|
||||
return nil
|
||||
}
|
||||
|
||||
*buf = append(*buf, '{')
|
||||
|
||||
var p = (*Pair)(self.p)
|
||||
err := p.encode(buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for i := 1; i < nb; i++ {
|
||||
*buf = append(*buf, ',')
|
||||
p = p.unsafe_next()
|
||||
err := p.encode(buf)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
*buf = append(*buf, '}')
|
||||
return nil
|
||||
}
|
||||
98
vendor/github.com/bytedance/sonic/ast/error.go
generated
vendored
Normal file
98
vendor/github.com/bytedance/sonic/ast/error.go
generated
vendored
Normal file
@@ -0,0 +1,98 @@
|
||||
package ast
|
||||
|
||||
import (
|
||||
`fmt`
|
||||
`strings`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
)
|
||||
|
||||
func (self *Parser) syntaxError(err types.ParsingError) SyntaxError {
|
||||
return SyntaxError{
|
||||
Pos : self.p,
|
||||
Src : self.s,
|
||||
Code: err,
|
||||
}
|
||||
}
|
||||
|
||||
func newSyntaxError(err SyntaxError) *Node {
|
||||
msg := err.Description()
|
||||
return &Node{
|
||||
t: V_ERROR,
|
||||
v: int64(err.Code),
|
||||
p: unsafe.Pointer(&msg),
|
||||
}
|
||||
}
|
||||
|
||||
type SyntaxError struct {
|
||||
Pos int
|
||||
Src string
|
||||
Code types.ParsingError
|
||||
Msg string
|
||||
}
|
||||
|
||||
func (self SyntaxError) Error() string {
|
||||
return fmt.Sprintf("%q", self.Description())
|
||||
}
|
||||
|
||||
func (self SyntaxError) Description() string {
|
||||
return "Syntax error " + self.description()
|
||||
}
|
||||
|
||||
func (self SyntaxError) description() string {
|
||||
i := 16
|
||||
p := self.Pos - i
|
||||
q := self.Pos + i
|
||||
|
||||
/* check for empty source */
|
||||
if self.Src == "" {
|
||||
return fmt.Sprintf("no sources available: %#v", self)
|
||||
}
|
||||
|
||||
/* prevent slicing before the beginning */
|
||||
if p < 0 {
|
||||
p, q, i = 0, q - p, i + p
|
||||
}
|
||||
|
||||
/* prevent slicing beyond the end */
|
||||
if n := len(self.Src); q > n {
|
||||
n = q - n
|
||||
q = len(self.Src)
|
||||
|
||||
/* move the left bound if possible */
|
||||
if p > n {
|
||||
i += n
|
||||
p -= n
|
||||
}
|
||||
}
|
||||
|
||||
/* left and right length */
|
||||
x := clamp_zero(i)
|
||||
y := clamp_zero(q - p - i - 1)
|
||||
|
||||
/* compose the error description */
|
||||
return fmt.Sprintf(
|
||||
"at index %d: %s\n\n\t%s\n\t%s^%s\n",
|
||||
self.Pos,
|
||||
self.Message(),
|
||||
self.Src[p:q],
|
||||
strings.Repeat(".", x),
|
||||
strings.Repeat(".", y),
|
||||
)
|
||||
}
|
||||
|
||||
func (self SyntaxError) Message() string {
|
||||
if self.Msg == "" {
|
||||
return self.Code.Message()
|
||||
}
|
||||
return self.Msg
|
||||
}
|
||||
|
||||
func clamp_zero(v int) int {
|
||||
if v < 0 {
|
||||
return 0
|
||||
} else {
|
||||
return v
|
||||
}
|
||||
}
|
||||
164
vendor/github.com/bytedance/sonic/ast/iterator.go
generated
vendored
Normal file
164
vendor/github.com/bytedance/sonic/ast/iterator.go
generated
vendored
Normal file
@@ -0,0 +1,164 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`fmt`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
)
|
||||
|
||||
type Pair struct {
|
||||
Key string
|
||||
Value Node
|
||||
}
|
||||
|
||||
// Values returns iterator for array's children traversal
|
||||
func (self *Node) Values() (ListIterator, error) {
|
||||
if err := self.should(types.V_ARRAY, "an array"); err != nil {
|
||||
return ListIterator{}, err
|
||||
}
|
||||
return ListIterator{Iterator{p: self}}, nil
|
||||
}
|
||||
|
||||
// Properties returns iterator for object's children traversal
|
||||
func (self *Node) Properties() (ObjectIterator, error) {
|
||||
if err := self.should(types.V_OBJECT, "an object"); err != nil {
|
||||
return ObjectIterator{}, err
|
||||
}
|
||||
return ObjectIterator{Iterator{p: self}}, nil
|
||||
}
|
||||
|
||||
type Iterator struct {
|
||||
i int
|
||||
p *Node
|
||||
}
|
||||
|
||||
func (self *Iterator) Pos() int {
|
||||
return self.i
|
||||
}
|
||||
|
||||
func (self *Iterator) Len() int {
|
||||
return self.p.len()
|
||||
}
|
||||
|
||||
// HasNext reports if it is the end of iteration or has error.
|
||||
func (self *Iterator) HasNext() bool {
|
||||
if !self.p.isLazy() {
|
||||
return self.p.Valid() && self.i < self.p.len()
|
||||
} else if self.p.t == _V_ARRAY_LAZY {
|
||||
return self.p.skipNextNode().Valid()
|
||||
} else if self.p.t == _V_OBJECT_LAZY {
|
||||
pair := self.p.skipNextPair()
|
||||
if pair == nil {
|
||||
return false
|
||||
}
|
||||
return pair.Value.Valid()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ListIterator is specialized iterator for V_ARRAY
|
||||
type ListIterator struct {
|
||||
Iterator
|
||||
}
|
||||
|
||||
// ObjectIterator is specialized iterator for V_ARRAY
|
||||
type ObjectIterator struct {
|
||||
Iterator
|
||||
}
|
||||
|
||||
// Next scans through children of underlying V_ARRAY,
|
||||
// copies each child to v, and returns .HasNext().
|
||||
func (self *ListIterator) Next(v *Node) bool {
|
||||
if !self.HasNext() {
|
||||
return false
|
||||
} else {
|
||||
*v, self.i = *self.p.nodeAt(self.i), self.i + 1
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// Next scans through children of underlying V_OBJECT,
|
||||
// copies each child to v, and returns .HasNext().
|
||||
func (self *ObjectIterator) Next(p *Pair) bool {
|
||||
if !self.HasNext() {
|
||||
return false
|
||||
} else {
|
||||
*p, self.i = *self.p.pairAt(self.i), self.i + 1
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// Sequence represents scanning path of single-layer nodes.
|
||||
// Index indicates the value's order in both V_ARRAY and V_OBJECT json.
|
||||
// Key is the value's key (for V_OBJECT json only, otherwise it will be nil).
|
||||
type Sequence struct {
|
||||
Index int
|
||||
Key *string
|
||||
// Level int
|
||||
}
|
||||
|
||||
// String is string representation of one Sequence
|
||||
func (s Sequence) String() string {
|
||||
k := ""
|
||||
if s.Key != nil {
|
||||
k = *s.Key
|
||||
}
|
||||
return fmt.Sprintf("Sequence(%d, %q)", s.Index, k)
|
||||
}
|
||||
|
||||
type Scanner func(path Sequence, node *Node) bool
|
||||
|
||||
// ForEach scans one V_OBJECT node's children from JSON head to tail,
|
||||
// and pass the Sequence and Node of corresponding JSON value.
|
||||
//
|
||||
// Especailly, if the node is not V_ARRAY or V_OBJECT,
|
||||
// the node itself will be returned and Sequence.Index == -1.
|
||||
func (self *Node) ForEach(sc Scanner) error {
|
||||
switch self.itype() {
|
||||
case types.V_ARRAY:
|
||||
ns, err := self.UnsafeArray()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for i := range ns {
|
||||
if !sc(Sequence{i, nil}, &ns[i]) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case types.V_OBJECT:
|
||||
ns, err := self.UnsafeMap()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for i := range ns {
|
||||
if !sc(Sequence{i, &ns[i].Key}, &ns[i].Value) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
default:
|
||||
sc(Sequence{-1, nil}, self)
|
||||
}
|
||||
return self.Check()
|
||||
}
|
||||
|
||||
type PairSlice []Pair
|
||||
|
||||
func (self PairSlice) Sort() {
|
||||
radixQsort(self, 0, maxDepth(len(self)))
|
||||
}
|
||||
1808
vendor/github.com/bytedance/sonic/ast/node.go
generated
vendored
Normal file
1808
vendor/github.com/bytedance/sonic/ast/node.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
618
vendor/github.com/bytedance/sonic/ast/parser.go
generated
vendored
Normal file
618
vendor/github.com/bytedance/sonic/ast/parser.go
generated
vendored
Normal file
@@ -0,0 +1,618 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`fmt`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
const _DEFAULT_NODE_CAP int = 16
|
||||
|
||||
const (
|
||||
_ERR_NOT_FOUND types.ParsingError = 33
|
||||
_ERR_UNSUPPORT_TYPE types.ParsingError = 34
|
||||
)
|
||||
|
||||
var (
|
||||
ErrNotExist error = newError(_ERR_NOT_FOUND, "value not exists")
|
||||
ErrUnsupportType error = newError(_ERR_UNSUPPORT_TYPE, "unsupported type")
|
||||
)
|
||||
|
||||
type Parser struct {
|
||||
p int
|
||||
s string
|
||||
noLazy bool
|
||||
skipValue bool
|
||||
}
|
||||
|
||||
/** Parser Private Methods **/
|
||||
|
||||
func (self *Parser) delim() types.ParsingError {
|
||||
n := len(self.s)
|
||||
p := self.lspace(self.p)
|
||||
|
||||
/* check for EOF */
|
||||
if p >= n {
|
||||
return types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for the delimtier */
|
||||
if self.s[p] != ':' {
|
||||
return types.ERR_INVALID_CHAR
|
||||
}
|
||||
|
||||
/* update the read pointer */
|
||||
self.p = p + 1
|
||||
return 0
|
||||
}
|
||||
|
||||
func (self *Parser) object() types.ParsingError {
|
||||
n := len(self.s)
|
||||
p := self.lspace(self.p)
|
||||
|
||||
/* check for EOF */
|
||||
if p >= n {
|
||||
return types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for the delimtier */
|
||||
if self.s[p] != '{' {
|
||||
return types.ERR_INVALID_CHAR
|
||||
}
|
||||
|
||||
/* update the read pointer */
|
||||
self.p = p + 1
|
||||
return 0
|
||||
}
|
||||
|
||||
func (self *Parser) array() types.ParsingError {
|
||||
n := len(self.s)
|
||||
p := self.lspace(self.p)
|
||||
|
||||
/* check for EOF */
|
||||
if p >= n {
|
||||
return types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for the delimtier */
|
||||
if self.s[p] != '[' {
|
||||
return types.ERR_INVALID_CHAR
|
||||
}
|
||||
|
||||
/* update the read pointer */
|
||||
self.p = p + 1
|
||||
return 0
|
||||
}
|
||||
|
||||
func (self *Parser) lspace(sp int) int {
|
||||
ns := len(self.s)
|
||||
for ; sp<ns && isSpace(self.s[sp]); sp+=1 {}
|
||||
|
||||
return sp
|
||||
}
|
||||
|
||||
func (self *Parser) decodeArray(ret []Node) (Node, types.ParsingError) {
|
||||
sp := self.p
|
||||
ns := len(self.s)
|
||||
|
||||
/* check for EOF */
|
||||
if self.p = self.lspace(sp); self.p >= ns {
|
||||
return Node{}, types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for empty array */
|
||||
if self.s[self.p] == ']' {
|
||||
self.p++
|
||||
return emptyArrayNode, 0
|
||||
}
|
||||
|
||||
/* allocate array space and parse every element */
|
||||
for {
|
||||
var val Node
|
||||
var err types.ParsingError
|
||||
|
||||
if self.skipValue {
|
||||
/* skip the value */
|
||||
var start int
|
||||
if start, err = self.skipFast(); err != 0 {
|
||||
return Node{}, err
|
||||
}
|
||||
if self.p > ns {
|
||||
return Node{}, types.ERR_EOF
|
||||
}
|
||||
t := switchRawType(self.s[start])
|
||||
if t == _V_NONE {
|
||||
return Node{}, types.ERR_INVALID_CHAR
|
||||
}
|
||||
val = newRawNode(self.s[start:self.p], t)
|
||||
}else{
|
||||
/* decode the value */
|
||||
if val, err = self.Parse(); err != 0 {
|
||||
return Node{}, err
|
||||
}
|
||||
}
|
||||
|
||||
/* add the value to result */
|
||||
ret = append(ret, val)
|
||||
self.p = self.lspace(self.p)
|
||||
|
||||
/* check for EOF */
|
||||
if self.p >= ns {
|
||||
return Node{}, types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for the next character */
|
||||
switch self.s[self.p] {
|
||||
case ',' : self.p++
|
||||
case ']' : self.p++; return NewArray(ret), 0
|
||||
default:
|
||||
if val.isLazy() {
|
||||
return newLazyArray(self, ret), 0
|
||||
}
|
||||
return Node{}, types.ERR_INVALID_CHAR
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *Parser) decodeObject(ret []Pair) (Node, types.ParsingError) {
|
||||
sp := self.p
|
||||
ns := len(self.s)
|
||||
|
||||
/* check for EOF */
|
||||
if self.p = self.lspace(sp); self.p >= ns {
|
||||
return Node{}, types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for empty object */
|
||||
if self.s[self.p] == '}' {
|
||||
self.p++
|
||||
return emptyObjectNode, 0
|
||||
}
|
||||
|
||||
/* decode each pair */
|
||||
for {
|
||||
var val Node
|
||||
var njs types.JsonState
|
||||
var err types.ParsingError
|
||||
|
||||
/* decode the key */
|
||||
if njs = self.decodeValue(); njs.Vt != types.V_STRING {
|
||||
return Node{}, types.ERR_INVALID_CHAR
|
||||
}
|
||||
|
||||
/* extract the key */
|
||||
idx := self.p - 1
|
||||
key := self.s[njs.Iv:idx]
|
||||
|
||||
/* check for escape sequence */
|
||||
if njs.Ep != -1 {
|
||||
if key, err = unquote(key); err != 0 {
|
||||
return Node{}, err
|
||||
}
|
||||
}
|
||||
|
||||
/* expect a ':' delimiter */
|
||||
if err = self.delim(); err != 0 {
|
||||
return Node{}, err
|
||||
}
|
||||
|
||||
|
||||
if self.skipValue {
|
||||
/* skip the value */
|
||||
var start int
|
||||
if start, err = self.skipFast(); err != 0 {
|
||||
return Node{}, err
|
||||
}
|
||||
if self.p > ns {
|
||||
return Node{}, types.ERR_EOF
|
||||
}
|
||||
t := switchRawType(self.s[start])
|
||||
if t == _V_NONE {
|
||||
return Node{}, types.ERR_INVALID_CHAR
|
||||
}
|
||||
val = newRawNode(self.s[start:self.p], t)
|
||||
} else {
|
||||
/* decode the value */
|
||||
if val, err = self.Parse(); err != 0 {
|
||||
return Node{}, err
|
||||
}
|
||||
}
|
||||
|
||||
/* add the value to result */
|
||||
ret = append(ret, Pair{Key: key, Value: val})
|
||||
self.p = self.lspace(self.p)
|
||||
|
||||
/* check for EOF */
|
||||
if self.p >= ns {
|
||||
return Node{}, types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for the next character */
|
||||
switch self.s[self.p] {
|
||||
case ',' : self.p++
|
||||
case '}' : self.p++; return NewObject(ret), 0
|
||||
default:
|
||||
if val.isLazy() {
|
||||
return newLazyObject(self, ret), 0
|
||||
}
|
||||
return Node{}, types.ERR_INVALID_CHAR
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *Parser) decodeString(iv int64, ep int) (Node, types.ParsingError) {
|
||||
p := self.p - 1
|
||||
s := self.s[iv:p]
|
||||
|
||||
/* fast path: no escape sequence */
|
||||
if ep == -1 {
|
||||
return NewString(s), 0
|
||||
}
|
||||
|
||||
/* unquote the string */
|
||||
out, err := unquote(s)
|
||||
|
||||
/* check for errors */
|
||||
if err != 0 {
|
||||
return Node{}, err
|
||||
} else {
|
||||
return newBytes(rt.Str2Mem(out)), 0
|
||||
}
|
||||
}
|
||||
|
||||
/** Parser Interface **/
|
||||
|
||||
func (self *Parser) Pos() int {
|
||||
return self.p
|
||||
}
|
||||
|
||||
func (self *Parser) Parse() (Node, types.ParsingError) {
|
||||
switch val := self.decodeValue(); val.Vt {
|
||||
case types.V_EOF : return Node{}, types.ERR_EOF
|
||||
case types.V_NULL : return nullNode, 0
|
||||
case types.V_TRUE : return trueNode, 0
|
||||
case types.V_FALSE : return falseNode, 0
|
||||
case types.V_STRING : return self.decodeString(val.Iv, val.Ep)
|
||||
case types.V_ARRAY:
|
||||
if self.noLazy {
|
||||
return self.decodeArray(make([]Node, 0, _DEFAULT_NODE_CAP))
|
||||
}
|
||||
return newLazyArray(self, make([]Node, 0, _DEFAULT_NODE_CAP)), 0
|
||||
case types.V_OBJECT:
|
||||
if self.noLazy {
|
||||
return self.decodeObject(make([]Pair, 0, _DEFAULT_NODE_CAP))
|
||||
}
|
||||
return newLazyObject(self, make([]Pair, 0, _DEFAULT_NODE_CAP)), 0
|
||||
case types.V_DOUBLE : return NewNumber(self.s[val.Ep:self.p]), 0
|
||||
case types.V_INTEGER : return NewNumber(self.s[val.Ep:self.p]), 0
|
||||
default : return Node{}, types.ParsingError(-val.Vt)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *Parser) searchKey(match string) types.ParsingError {
|
||||
ns := len(self.s)
|
||||
if err := self.object(); err != 0 {
|
||||
return err
|
||||
}
|
||||
|
||||
/* check for EOF */
|
||||
if self.p = self.lspace(self.p); self.p >= ns {
|
||||
return types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for empty object */
|
||||
if self.s[self.p] == '}' {
|
||||
self.p++
|
||||
return _ERR_NOT_FOUND
|
||||
}
|
||||
|
||||
var njs types.JsonState
|
||||
var err types.ParsingError
|
||||
/* decode each pair */
|
||||
for {
|
||||
|
||||
/* decode the key */
|
||||
if njs = self.decodeValue(); njs.Vt != types.V_STRING {
|
||||
return types.ERR_INVALID_CHAR
|
||||
}
|
||||
|
||||
/* extract the key */
|
||||
idx := self.p - 1
|
||||
key := self.s[njs.Iv:idx]
|
||||
|
||||
/* check for escape sequence */
|
||||
if njs.Ep != -1 {
|
||||
if key, err = unquote(key); err != 0 {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
/* expect a ':' delimiter */
|
||||
if err = self.delim(); err != 0 {
|
||||
return err
|
||||
}
|
||||
|
||||
/* skip value */
|
||||
if key != match {
|
||||
if _, err = self.skipFast(); err != 0 {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
return 0
|
||||
}
|
||||
|
||||
/* check for EOF */
|
||||
self.p = self.lspace(self.p)
|
||||
if self.p >= ns {
|
||||
return types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for the next character */
|
||||
switch self.s[self.p] {
|
||||
case ',':
|
||||
self.p++
|
||||
case '}':
|
||||
self.p++
|
||||
return _ERR_NOT_FOUND
|
||||
default:
|
||||
return types.ERR_INVALID_CHAR
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *Parser) searchIndex(idx int) types.ParsingError {
|
||||
ns := len(self.s)
|
||||
if err := self.array(); err != 0 {
|
||||
return err
|
||||
}
|
||||
|
||||
/* check for EOF */
|
||||
if self.p = self.lspace(self.p); self.p >= ns {
|
||||
return types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for empty array */
|
||||
if self.s[self.p] == ']' {
|
||||
self.p++
|
||||
return _ERR_NOT_FOUND
|
||||
}
|
||||
|
||||
var err types.ParsingError
|
||||
/* allocate array space and parse every element */
|
||||
for i := 0; i < idx; i++ {
|
||||
|
||||
/* decode the value */
|
||||
if _, err = self.skipFast(); err != 0 {
|
||||
return err
|
||||
}
|
||||
|
||||
/* check for EOF */
|
||||
self.p = self.lspace(self.p)
|
||||
if self.p >= ns {
|
||||
return types.ERR_EOF
|
||||
}
|
||||
|
||||
/* check for the next character */
|
||||
switch self.s[self.p] {
|
||||
case ',':
|
||||
self.p++
|
||||
case ']':
|
||||
self.p++
|
||||
return _ERR_NOT_FOUND
|
||||
default:
|
||||
return types.ERR_INVALID_CHAR
|
||||
}
|
||||
}
|
||||
|
||||
return 0
|
||||
}
|
||||
|
||||
func (self *Node) skipNextNode() *Node {
|
||||
if !self.isLazy() {
|
||||
return nil
|
||||
}
|
||||
|
||||
parser, stack := self.getParserAndArrayStack()
|
||||
ret := stack.v
|
||||
sp := parser.p
|
||||
ns := len(parser.s)
|
||||
|
||||
/* check for EOF */
|
||||
if parser.p = parser.lspace(sp); parser.p >= ns {
|
||||
return newSyntaxError(parser.syntaxError(types.ERR_EOF))
|
||||
}
|
||||
|
||||
/* check for empty array */
|
||||
if parser.s[parser.p] == ']' {
|
||||
parser.p++
|
||||
self.setArray(ret)
|
||||
return nil
|
||||
}
|
||||
|
||||
var val Node
|
||||
/* skip the value */
|
||||
if start, err := parser.skipFast(); err != 0 {
|
||||
return newSyntaxError(parser.syntaxError(err))
|
||||
} else {
|
||||
t := switchRawType(parser.s[start])
|
||||
if t == _V_NONE {
|
||||
return newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))
|
||||
}
|
||||
val = newRawNode(parser.s[start:parser.p], t)
|
||||
}
|
||||
|
||||
/* add the value to result */
|
||||
ret = append(ret, val)
|
||||
parser.p = parser.lspace(parser.p)
|
||||
|
||||
/* check for EOF */
|
||||
if parser.p >= ns {
|
||||
return newSyntaxError(parser.syntaxError(types.ERR_EOF))
|
||||
}
|
||||
|
||||
/* check for the next character */
|
||||
switch parser.s[parser.p] {
|
||||
case ',':
|
||||
parser.p++
|
||||
self.setLazyArray(parser, ret)
|
||||
return &ret[len(ret)-1]
|
||||
case ']':
|
||||
parser.p++
|
||||
self.setArray(ret)
|
||||
return &ret[len(ret)-1]
|
||||
default:
|
||||
return newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))
|
||||
}
|
||||
}
|
||||
|
||||
func (self *Node) skipNextPair() (*Pair) {
|
||||
if !self.isLazy() {
|
||||
return nil
|
||||
}
|
||||
|
||||
parser, stack := self.getParserAndObjectStack()
|
||||
ret := stack.v
|
||||
sp := parser.p
|
||||
ns := len(parser.s)
|
||||
|
||||
/* check for EOF */
|
||||
if parser.p = parser.lspace(sp); parser.p >= ns {
|
||||
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_EOF))}
|
||||
}
|
||||
|
||||
/* check for empty object */
|
||||
if parser.s[parser.p] == '}' {
|
||||
parser.p++
|
||||
self.setObject(ret)
|
||||
return nil
|
||||
}
|
||||
|
||||
/* decode one pair */
|
||||
var val Node
|
||||
var njs types.JsonState
|
||||
var err types.ParsingError
|
||||
|
||||
/* decode the key */
|
||||
if njs = parser.decodeValue(); njs.Vt != types.V_STRING {
|
||||
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
|
||||
}
|
||||
|
||||
/* extract the key */
|
||||
idx := parser.p - 1
|
||||
key := parser.s[njs.Iv:idx]
|
||||
|
||||
/* check for escape sequence */
|
||||
if njs.Ep != -1 {
|
||||
if key, err = unquote(key); err != 0 {
|
||||
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
|
||||
}
|
||||
}
|
||||
|
||||
/* expect a ':' delimiter */
|
||||
if err = parser.delim(); err != 0 {
|
||||
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
|
||||
}
|
||||
|
||||
/* skip the value */
|
||||
if start, err := parser.skipFast(); err != 0 {
|
||||
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
|
||||
} else {
|
||||
t := switchRawType(parser.s[start])
|
||||
if t == _V_NONE {
|
||||
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
|
||||
}
|
||||
val = newRawNode(parser.s[start:parser.p], t)
|
||||
}
|
||||
|
||||
/* add the value to result */
|
||||
ret = append(ret, Pair{Key: key, Value: val})
|
||||
parser.p = parser.lspace(parser.p)
|
||||
|
||||
/* check for EOF */
|
||||
if parser.p >= ns {
|
||||
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_EOF))}
|
||||
}
|
||||
|
||||
/* check for the next character */
|
||||
switch parser.s[parser.p] {
|
||||
case ',':
|
||||
parser.p++
|
||||
self.setLazyObject(parser, ret)
|
||||
return &ret[len(ret)-1]
|
||||
case '}':
|
||||
parser.p++
|
||||
self.setObject(ret)
|
||||
return &ret[len(ret)-1]
|
||||
default:
|
||||
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/** Parser Factory **/
|
||||
|
||||
// Loads parse all json into interface{}
|
||||
func Loads(src string) (int, interface{}, error) {
|
||||
ps := &Parser{s: src}
|
||||
np, err := ps.Parse()
|
||||
|
||||
/* check for errors */
|
||||
if err != 0 {
|
||||
return 0, nil, ps.ExportError(err)
|
||||
} else {
|
||||
x, err := np.Interface()
|
||||
if err != nil {
|
||||
return 0, nil, err
|
||||
}
|
||||
return ps.Pos(), x, nil
|
||||
}
|
||||
}
|
||||
|
||||
// LoadsUseNumber parse all json into interface{}, with numeric nodes casted to json.Number
|
||||
func LoadsUseNumber(src string) (int, interface{}, error) {
|
||||
ps := &Parser{s: src}
|
||||
np, err := ps.Parse()
|
||||
|
||||
/* check for errors */
|
||||
if err != 0 {
|
||||
return 0, nil, err
|
||||
} else {
|
||||
x, err := np.InterfaceUseNumber()
|
||||
if err != nil {
|
||||
return 0, nil, err
|
||||
}
|
||||
return ps.Pos(), x, nil
|
||||
}
|
||||
}
|
||||
|
||||
func NewParser(src string) *Parser {
|
||||
return &Parser{s: src}
|
||||
}
|
||||
|
||||
// ExportError converts types.ParsingError to std Error
|
||||
func (self *Parser) ExportError(err types.ParsingError) error {
|
||||
if err == _ERR_NOT_FOUND {
|
||||
return ErrNotExist
|
||||
}
|
||||
return fmt.Errorf("%q", SyntaxError{
|
||||
Pos : self.p,
|
||||
Src : self.s,
|
||||
Code: err,
|
||||
}.Description())
|
||||
}
|
||||
30
vendor/github.com/bytedance/sonic/ast/search.go
generated
vendored
Normal file
30
vendor/github.com/bytedance/sonic/ast/search.go
generated
vendored
Normal file
@@ -0,0 +1,30 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
type Searcher struct {
|
||||
parser Parser
|
||||
}
|
||||
|
||||
func NewSearcher(str string) *Searcher {
|
||||
return &Searcher{
|
||||
parser: Parser{
|
||||
s: str,
|
||||
noLazy: false,
|
||||
},
|
||||
}
|
||||
}
|
||||
206
vendor/github.com/bytedance/sonic/ast/sort.go
generated
vendored
Normal file
206
vendor/github.com/bytedance/sonic/ast/sort.go
generated
vendored
Normal file
@@ -0,0 +1,206 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
// Algorithm 3-way Radix Quicksort, d means the radix.
|
||||
// Reference: https://algs4.cs.princeton.edu/51radix/Quick3string.java.html
|
||||
func radixQsort(kvs PairSlice, d, maxDepth int) {
|
||||
for len(kvs) > 11 {
|
||||
// To avoid the worst case of quickSort (time: O(n^2)), use introsort here.
|
||||
// Reference: https://en.wikipedia.org/wiki/Introsort and
|
||||
// https://github.com/golang/go/issues/467
|
||||
if maxDepth == 0 {
|
||||
heapSort(kvs, 0, len(kvs))
|
||||
return
|
||||
}
|
||||
maxDepth--
|
||||
|
||||
p := pivot(kvs, d)
|
||||
lt, i, gt := 0, 0, len(kvs)
|
||||
for i < gt {
|
||||
c := byteAt(kvs[i].Key, d)
|
||||
if c < p {
|
||||
swap(kvs, lt, i)
|
||||
i++
|
||||
lt++
|
||||
} else if c > p {
|
||||
gt--
|
||||
swap(kvs, i, gt)
|
||||
} else {
|
||||
i++
|
||||
}
|
||||
}
|
||||
|
||||
// kvs[0:lt] < v = kvs[lt:gt] < kvs[gt:len(kvs)]
|
||||
// Native implemention:
|
||||
// radixQsort(kvs[:lt], d, maxDepth)
|
||||
// if p > -1 {
|
||||
// radixQsort(kvs[lt:gt], d+1, maxDepth)
|
||||
// }
|
||||
// radixQsort(kvs[gt:], d, maxDepth)
|
||||
// Optimize as follows: make recursive calls only for the smaller parts.
|
||||
// Reference: https://www.geeksforgeeks.org/quicksort-tail-call-optimization-reducing-worst-case-space-log-n/
|
||||
if p == -1 {
|
||||
if lt > len(kvs) - gt {
|
||||
radixQsort(kvs[gt:], d, maxDepth)
|
||||
kvs = kvs[:lt]
|
||||
} else {
|
||||
radixQsort(kvs[:lt], d, maxDepth)
|
||||
kvs = kvs[gt:]
|
||||
}
|
||||
} else {
|
||||
ml := maxThree(lt, gt-lt, len(kvs)-gt)
|
||||
if ml == lt {
|
||||
radixQsort(kvs[lt:gt], d+1, maxDepth)
|
||||
radixQsort(kvs[gt:], d, maxDepth)
|
||||
kvs = kvs[:lt]
|
||||
} else if ml == gt-lt {
|
||||
radixQsort(kvs[:lt], d, maxDepth)
|
||||
radixQsort(kvs[gt:], d, maxDepth)
|
||||
kvs = kvs[lt:gt]
|
||||
d += 1
|
||||
} else {
|
||||
radixQsort(kvs[:lt], d, maxDepth)
|
||||
radixQsort(kvs[lt:gt], d+1, maxDepth)
|
||||
kvs = kvs[gt:]
|
||||
}
|
||||
}
|
||||
}
|
||||
insertRadixSort(kvs, d)
|
||||
}
|
||||
|
||||
func insertRadixSort(kvs PairSlice, d int) {
|
||||
for i := 1; i < len(kvs); i++ {
|
||||
for j := i; j > 0 && lessFrom(kvs[j].Key, kvs[j-1].Key, d); j-- {
|
||||
swap(kvs, j, j-1)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func pivot(kvs PairSlice, d int) int {
|
||||
m := len(kvs) >> 1
|
||||
if len(kvs) > 40 {
|
||||
// Tukey's ``Ninther,'' median of three mediankvs of three.
|
||||
t := len(kvs) / 8
|
||||
return medianThree(
|
||||
medianThree(byteAt(kvs[0].Key, d), byteAt(kvs[t].Key, d), byteAt(kvs[2*t].Key, d)),
|
||||
medianThree(byteAt(kvs[m].Key, d), byteAt(kvs[m-t].Key, d), byteAt(kvs[m+t].Key, d)),
|
||||
medianThree(byteAt(kvs[len(kvs)-1].Key, d),
|
||||
byteAt(kvs[len(kvs)-1-t].Key, d),
|
||||
byteAt(kvs[len(kvs)-1-2*t].Key, d)))
|
||||
}
|
||||
return medianThree(byteAt(kvs[0].Key, d), byteAt(kvs[m].Key, d), byteAt(kvs[len(kvs)-1].Key, d))
|
||||
}
|
||||
|
||||
func medianThree(i, j, k int) int {
|
||||
if i > j {
|
||||
i, j = j, i
|
||||
} // i < j
|
||||
if k < i {
|
||||
return i
|
||||
}
|
||||
if k > j {
|
||||
return j
|
||||
}
|
||||
return k
|
||||
}
|
||||
|
||||
func maxThree(i, j, k int) int {
|
||||
max := i
|
||||
if max < j {
|
||||
max = j
|
||||
}
|
||||
if max < k {
|
||||
max = k
|
||||
}
|
||||
return max
|
||||
}
|
||||
|
||||
// maxDepth returns a threshold at which quicksort should switch
|
||||
// to heapsort. It returnkvs 2*ceil(lg(n+1)).
|
||||
func maxDepth(n int) int {
|
||||
var depth int
|
||||
for i := n; i > 0; i >>= 1 {
|
||||
depth++
|
||||
}
|
||||
return depth * 2
|
||||
}
|
||||
|
||||
// siftDown implements the heap property on kvs[lo:hi].
|
||||
// first is an offset into the array where the root of the heap lies.
|
||||
func siftDown(kvs PairSlice, lo, hi, first int) {
|
||||
root := lo
|
||||
for {
|
||||
child := 2*root + 1
|
||||
if child >= hi {
|
||||
break
|
||||
}
|
||||
if child+1 < hi && kvs[first+child].Key < kvs[first+child+1].Key {
|
||||
child++
|
||||
}
|
||||
if kvs[first+root].Key >= kvs[first+child].Key {
|
||||
return
|
||||
}
|
||||
swap(kvs, first+root, first+child)
|
||||
root = child
|
||||
}
|
||||
}
|
||||
|
||||
func heapSort(kvs PairSlice, a, b int) {
|
||||
first := a
|
||||
lo := 0
|
||||
hi := b - a
|
||||
|
||||
// Build heap with the greatest element at top.
|
||||
for i := (hi - 1) / 2; i >= 0; i-- {
|
||||
siftDown(kvs, i, hi, first)
|
||||
}
|
||||
|
||||
// Pop elements, the largest first, into end of kvs.
|
||||
for i := hi - 1; i >= 0; i-- {
|
||||
swap(kvs, first, first+i)
|
||||
siftDown(kvs, lo, i, first)
|
||||
}
|
||||
}
|
||||
|
||||
// Note that Pair.Key is NOT pointed to Pair.m when map key is integer after swap
|
||||
func swap(kvs PairSlice, a, b int) {
|
||||
kvs[a].Key, kvs[b].Key = kvs[b].Key, kvs[a].Key
|
||||
kvs[a].Value, kvs[b].Value = kvs[b].Value, kvs[a].Value
|
||||
}
|
||||
|
||||
// Compare two strings from the pos d.
|
||||
func lessFrom(a, b string, d int) bool {
|
||||
l := len(a)
|
||||
if l > len(b) {
|
||||
l = len(b)
|
||||
}
|
||||
for i := d; i < l; i++ {
|
||||
if a[i] == b[i] {
|
||||
continue
|
||||
}
|
||||
return a[i] < b[i]
|
||||
}
|
||||
return len(a) < len(b)
|
||||
}
|
||||
|
||||
func byteAt(b string, p int) int {
|
||||
if p < len(b) {
|
||||
return int(b[p])
|
||||
}
|
||||
return -1
|
||||
}
|
||||
55
vendor/github.com/bytedance/sonic/ast/stubs_go115.go
generated
vendored
Normal file
55
vendor/github.com/bytedance/sonic/ast/stubs_go115.go
generated
vendored
Normal file
@@ -0,0 +1,55 @@
|
||||
// +build !go1.20
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
`unicode/utf8`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memmove runtime.memmove
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname unsafe_NewArray reflect.unsafe_NewArray
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func unsafe_NewArray(typ *rt.GoType, n int) unsafe.Pointer
|
||||
|
||||
//go:linkname growslice runtime.growslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
|
||||
|
||||
//go:nosplit
|
||||
func mem2ptr(s []byte) unsafe.Pointer {
|
||||
return (*rt.GoSlice)(unsafe.Pointer(&s)).Ptr
|
||||
}
|
||||
|
||||
var (
|
||||
//go:linkname safeSet encoding/json.safeSet
|
||||
safeSet [utf8.RuneSelf]bool
|
||||
|
||||
//go:linkname hex encoding/json.hex
|
||||
hex string
|
||||
)
|
||||
|
||||
//go:linkname unquoteBytes encoding/json.unquoteBytes
|
||||
func unquoteBytes(s []byte) (t []byte, ok bool)
|
||||
55
vendor/github.com/bytedance/sonic/ast/stubs_go120.go
generated
vendored
Normal file
55
vendor/github.com/bytedance/sonic/ast/stubs_go120.go
generated
vendored
Normal file
@@ -0,0 +1,55 @@
|
||||
// +build go1.20
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package ast
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
`unicode/utf8`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memmove runtime.memmove
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname unsafe_NewArray reflect.unsafe_NewArray
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func unsafe_NewArray(typ *rt.GoType, n int) unsafe.Pointer
|
||||
|
||||
//go:linkname growslice reflect.growslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
|
||||
|
||||
//go:nosplit
|
||||
func mem2ptr(s []byte) unsafe.Pointer {
|
||||
return (*rt.GoSlice)(unsafe.Pointer(&s)).Ptr
|
||||
}
|
||||
|
||||
var (
|
||||
//go:linkname safeSet encoding/json.safeSet
|
||||
safeSet [utf8.RuneSelf]bool
|
||||
|
||||
//go:linkname hex encoding/json.hex
|
||||
hex string
|
||||
)
|
||||
|
||||
//go:linkname unquoteBytes encoding/json.unquoteBytes
|
||||
func unquoteBytes(s []byte) (t []byte, ok bool)
|
||||
14
vendor/github.com/bytedance/sonic/bench-arm.sh
generated
vendored
Normal file
14
vendor/github.com/bytedance/sonic/bench-arm.sh
generated
vendored
Normal file
@@ -0,0 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
pwd=$(pwd)
|
||||
export SONIC_NO_ASYNC_GC=1
|
||||
|
||||
cd $pwd/ast
|
||||
go test -benchmem -run=^$ -benchtime=1000000x -bench "^(BenchmarkGet.*|BenchmarkSet.*)$"
|
||||
|
||||
go test -benchmem -run=^$ -benchtime=10000x -bench "^(BenchmarkParser_.*|BenchmarkEncode.*)$"
|
||||
|
||||
go test -benchmem -run=^$ -benchtime=10000000x -bench "^(BenchmarkNodeGetByPath|BenchmarkStructGetByPath|BenchmarkNodeIndex|BenchmarkStructIndex|BenchmarkSliceIndex|BenchmarkMapIndex|BenchmarkNodeGet|BenchmarkSliceGet|BenchmarkMapGet|BenchmarkNodeSet|BenchmarkMapSet|BenchmarkNodeSetByIndex|BenchmarkSliceSetByIndex|BenchmarkStructSetByIndex|BenchmarkNodeUnset|BenchmarkMapUnset|BenchmarkNodUnsetByIndex|BenchmarkSliceUnsetByIndex|BenchmarkNodeAdd|BenchmarkSliceAdd|BenchmarkMapAdd)$"
|
||||
|
||||
unset SONIC_NO_ASYNC_GC
|
||||
cd $pwd
|
||||
134
vendor/github.com/bytedance/sonic/bench.py
generated
vendored
Normal file
134
vendor/github.com/bytedance/sonic/bench.py
generated
vendored
Normal file
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
# Copyright 2022 ByteDance Inc.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
import tempfile
|
||||
import os
|
||||
import subprocess
|
||||
import argparse
|
||||
|
||||
gbench_prefix = "SONIC_NO_ASYNC_GC=1 go test -benchmem -run=none "
|
||||
|
||||
def run(cmd):
|
||||
print(cmd)
|
||||
if os.system(cmd):
|
||||
print ("Failed to run cmd: %s"%(cmd))
|
||||
exit(1)
|
||||
|
||||
def run_s(cmd):
|
||||
print (cmd)
|
||||
try:
|
||||
res = os.popen(cmd)
|
||||
except subprocess.CalledProcessError as e:
|
||||
if e.returncode:
|
||||
print (e.output)
|
||||
exit(1)
|
||||
return res.read()
|
||||
|
||||
def run_r(cmd):
|
||||
print (cmd)
|
||||
try:
|
||||
cmds = cmd.split(' ')
|
||||
data = subprocess.check_output(cmds, stderr=subprocess.STDOUT)
|
||||
except subprocess.CalledProcessError as e:
|
||||
if e.returncode:
|
||||
print (e.output)
|
||||
exit(1)
|
||||
return data.decode("utf-8")
|
||||
|
||||
def compare(args):
|
||||
# detech current branch.
|
||||
# result = run_r("git branch")
|
||||
current_branch = run_s("git status | head -n1 | sed 's/On branch //'")
|
||||
# for br in result.split('\n'):
|
||||
# if br.startswith("* "):
|
||||
# current_branch = br.lstrip('* ')
|
||||
# break
|
||||
|
||||
if not current_branch:
|
||||
print ("Failed to detech current branch")
|
||||
return None
|
||||
|
||||
# get the current diff
|
||||
(fd, diff) = tempfile.mkstemp()
|
||||
run("git diff > %s"%diff)
|
||||
|
||||
# early return if currrent is main branch.
|
||||
print ("Current branch: %s"%(current_branch))
|
||||
if current_branch == "main":
|
||||
print ("Cannot compare at the main branch.Please build a new branch")
|
||||
return None
|
||||
|
||||
# benchmark current branch
|
||||
(fd, target) = tempfile.mkstemp(".target.txt")
|
||||
run("%s %s ./... 2>&1 | tee %s" %(gbench_prefix, args, target))
|
||||
|
||||
# trying to switch to the latest main branch
|
||||
run("git checkout -- .")
|
||||
if current_branch != "main":
|
||||
run("git checkout main")
|
||||
run("git pull --allow-unrelated-histories origin main")
|
||||
|
||||
# benchmark main branch
|
||||
(fd, main) = tempfile.mkstemp(".main.txt")
|
||||
run("%s %s ./... 2>&1 | tee %s" %(gbench_prefix, args, main))
|
||||
|
||||
# diff the result
|
||||
# benchstat = "go get golang.org/x/perf/cmd/benchstat && go install golang.org/x/perf/cmd/benchstat"
|
||||
run( "benchstat -sort=delta %s %s"%(main, target))
|
||||
run("git checkout -- .")
|
||||
|
||||
# restore branch
|
||||
if current_branch != "main":
|
||||
run("git checkout %s"%(current_branch))
|
||||
run("patch -p1 < %s" % (diff))
|
||||
return target
|
||||
|
||||
def main():
|
||||
argparser = argparse.ArgumentParser(description='Tools to test the performance. Example: ./bench.py -b Decoder_Generic_Sonic -c')
|
||||
argparser.add_argument('-b', '--bench', dest='filter', required=False,
|
||||
help='Specify the filter for golang benchmark')
|
||||
argparser.add_argument('-c', '--compare', dest='compare', action='store_true', required=False,
|
||||
help='Compare with the main benchmarking')
|
||||
argparser.add_argument('-t', '--times', dest='times', required=False,
|
||||
help='benchmark the times')
|
||||
argparser.add_argument('-r', '--repeat_times', dest='count', required=False,
|
||||
help='benchmark the count')
|
||||
args = argparser.parse_args()
|
||||
|
||||
if args.filter:
|
||||
gbench_args = "-bench=%s"%(args.filter)
|
||||
else:
|
||||
gbench_args = "-bench=."
|
||||
|
||||
if args.times:
|
||||
gbench_args += " -benchtime=%s"%(args.times)
|
||||
|
||||
if args.count:
|
||||
gbench_args += " -count=%s"%(args.count)
|
||||
else:
|
||||
gbench_args += " -count=10"
|
||||
|
||||
if args.compare:
|
||||
target = compare(gbench_args)
|
||||
else:
|
||||
target = None
|
||||
|
||||
if not target:
|
||||
(fd, target) = tempfile.mkstemp(".target.txt")
|
||||
run("%s %s ./... 2>&1 | tee %s" %(gbench_prefix, gbench_args, target))
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
27
vendor/github.com/bytedance/sonic/bench.sh
generated
vendored
Normal file
27
vendor/github.com/bytedance/sonic/bench.sh
generated
vendored
Normal file
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
pwd=$(pwd)
|
||||
export SONIC_NO_ASYNC_GC=1
|
||||
|
||||
cd $pwd/encoder
|
||||
go test -benchmem -run=^$ -benchtime=100000x -bench "^(BenchmarkEncoder_.*)$"
|
||||
|
||||
cd $pwd/decoder
|
||||
go test -benchmem -run=^$ -benchtime=100000x -bench "^(BenchmarkDecoder_.*)$"
|
||||
|
||||
cd $pwd/ast
|
||||
go test -benchmem -run=^$ -benchtime=1000000x -bench "^(BenchmarkGet.*|BenchmarkSet.*)$"
|
||||
|
||||
go test -benchmem -run=^$ -benchtime=10000x -bench "^(BenchmarkParser_.*|BenchmarkEncode.*)$"
|
||||
|
||||
go test -benchmem -run=^$ -benchtime=10000000x -bench "^(BenchmarkNodeGetByPath|BenchmarkStructGetByPath|BenchmarkNodeIndex|BenchmarkStructIndex|BenchmarkSliceIndex|BenchmarkMapIndex|BenchmarkNodeGet|BenchmarkSliceGet|BenchmarkMapGet|BenchmarkNodeSet|BenchmarkMapSet|BenchmarkNodeSetByIndex|BenchmarkSliceSetByIndex|BenchmarkStructSetByIndex|BenchmarkNodeUnset|BenchmarkMapUnset|BenchmarkNodUnsetByIndex|BenchmarkSliceUnsetByIndex|BenchmarkNodeAdd|BenchmarkSliceAdd|BenchmarkMapAdd)$"
|
||||
|
||||
cd $pwd/external_jsonlib_test/benchmark_test
|
||||
go test -benchmem -run=^$ -benchtime=100000x -bench "^(BenchmarkEncoder_.*|BenchmarkDecoder_.*)$"
|
||||
|
||||
go test -benchmem -run=^$ -benchtime=1000000x -bench "^(BenchmarkGet.*|BenchmarkSet.*)$"
|
||||
|
||||
go test -benchmem -run=^$ -benchtime=10000x -bench "^(BenchmarkParser_.*)$"
|
||||
|
||||
unset SONIC_NO_ASYNC_GC
|
||||
cd $pwd
|
||||
10
vendor/github.com/bytedance/sonic/check_branch_name.sh
generated
vendored
Normal file
10
vendor/github.com/bytedance/sonic/check_branch_name.sh
generated
vendored
Normal file
@@ -0,0 +1,10 @@
|
||||
#!/usr/bin/env bash
|
||||
|
||||
current=$(git status | head -n1 | sed 's/On branch //')
|
||||
name=${1:-$current}
|
||||
if [[ ! $name =~ ^(((opt(imize)?|feat(ure)?|doc|(bug|hot)?fix|test|refact(or)?|ci)/.+)|(main|develop)|(release/.+)|(release-v[0-9]+\.[0-9]+)|(release/v[0-9]+\.[0-9]+\.[0-9]+(-[a-z0-9.]+(\+[a-z0-9.]+)?)?)|revert-[a-z0-9]+)$ ]]; then
|
||||
echo "branch name '$name' is invalid"
|
||||
exit 1
|
||||
else
|
||||
echo "branch name '$name' is valid"
|
||||
fi
|
||||
131
vendor/github.com/bytedance/sonic/compat.go
generated
vendored
Normal file
131
vendor/github.com/bytedance/sonic/compat.go
generated
vendored
Normal file
@@ -0,0 +1,131 @@
|
||||
// +build !amd64 go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package sonic
|
||||
|
||||
import (
|
||||
`bytes`
|
||||
`encoding/json`
|
||||
`io`
|
||||
`reflect`
|
||||
|
||||
`github.com/bytedance/sonic/option`
|
||||
)
|
||||
|
||||
type frozenConfig struct {
|
||||
Config
|
||||
}
|
||||
|
||||
// Froze convert the Config to API
|
||||
func (cfg Config) Froze() API {
|
||||
api := &frozenConfig{Config: cfg}
|
||||
return api
|
||||
}
|
||||
|
||||
func (cfg frozenConfig) marshalOptions(val interface{}, prefix, indent string) ([]byte, error) {
|
||||
w := bytes.NewBuffer([]byte{})
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetEscapeHTML(cfg.EscapeHTML)
|
||||
enc.SetIndent(prefix, indent)
|
||||
err := enc.Encode(val)
|
||||
out := w.Bytes()
|
||||
|
||||
// json.Encoder always appends '\n' after encoding,
|
||||
// which is not same with json.Marshal()
|
||||
if len(out) > 0 && out[len(out)-1] == '\n' {
|
||||
out = out[:len(out)-1]
|
||||
}
|
||||
return out, err
|
||||
}
|
||||
|
||||
// Marshal is implemented by sonic
|
||||
func (cfg frozenConfig) Marshal(val interface{}) ([]byte, error) {
|
||||
if !cfg.EscapeHTML {
|
||||
return cfg.marshalOptions(val, "", "")
|
||||
}
|
||||
return json.Marshal(val)
|
||||
}
|
||||
|
||||
// MarshalToString is implemented by sonic
|
||||
func (cfg frozenConfig) MarshalToString(val interface{}) (string, error) {
|
||||
out, err := cfg.Marshal(val)
|
||||
return string(out), err
|
||||
}
|
||||
|
||||
// MarshalIndent is implemented by sonic
|
||||
func (cfg frozenConfig) MarshalIndent(val interface{}, prefix, indent string) ([]byte, error) {
|
||||
if !cfg.EscapeHTML {
|
||||
return cfg.marshalOptions(val, prefix, indent)
|
||||
}
|
||||
return json.MarshalIndent(val, prefix, indent)
|
||||
}
|
||||
|
||||
// UnmarshalFromString is implemented by sonic
|
||||
func (cfg frozenConfig) UnmarshalFromString(buf string, val interface{}) error {
|
||||
r := bytes.NewBufferString(buf)
|
||||
dec := json.NewDecoder(r)
|
||||
if cfg.UseNumber {
|
||||
dec.UseNumber()
|
||||
}
|
||||
if cfg.DisallowUnknownFields {
|
||||
dec.DisallowUnknownFields()
|
||||
}
|
||||
return dec.Decode(val)
|
||||
}
|
||||
|
||||
// Unmarshal is implemented by sonic
|
||||
func (cfg frozenConfig) Unmarshal(buf []byte, val interface{}) error {
|
||||
return cfg.UnmarshalFromString(string(buf), val)
|
||||
}
|
||||
|
||||
// NewEncoder is implemented by sonic
|
||||
func (cfg frozenConfig) NewEncoder(writer io.Writer) Encoder {
|
||||
enc := json.NewEncoder(writer)
|
||||
if !cfg.EscapeHTML {
|
||||
enc.SetEscapeHTML(cfg.EscapeHTML)
|
||||
}
|
||||
return enc
|
||||
}
|
||||
|
||||
// NewDecoder is implemented by sonic
|
||||
func (cfg frozenConfig) NewDecoder(reader io.Reader) Decoder {
|
||||
dec := json.NewDecoder(reader)
|
||||
if cfg.UseNumber {
|
||||
dec.UseNumber()
|
||||
}
|
||||
if cfg.DisallowUnknownFields {
|
||||
dec.DisallowUnknownFields()
|
||||
}
|
||||
return dec
|
||||
}
|
||||
|
||||
// Valid is implemented by sonic
|
||||
func (cfg frozenConfig) Valid(data []byte) bool {
|
||||
return json.Valid(data)
|
||||
}
|
||||
|
||||
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
|
||||
// order to reduce the first-hit latency at **amd64** Arch.
|
||||
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
|
||||
// a compile option to set the depth of recursive compile for the nested struct type.
|
||||
// * This is the none implement for !amd64.
|
||||
// It will be useful for someone who develop with !amd64 arch,like Mac M1.
|
||||
func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
66
vendor/github.com/bytedance/sonic/decoder/decoder_amd64.go
generated
vendored
Normal file
66
vendor/github.com/bytedance/sonic/decoder/decoder_amd64.go
generated
vendored
Normal file
@@ -0,0 +1,66 @@
|
||||
// +build amd64,go1.15,!go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2023 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`github.com/bytedance/sonic/internal/decoder`
|
||||
)
|
||||
|
||||
// Decoder is the decoder context object
|
||||
type Decoder = decoder.Decoder
|
||||
|
||||
type MismatchTypeError = decoder.MismatchTypeError
|
||||
|
||||
// Options for decode.
|
||||
type Options = decoder.Options
|
||||
|
||||
const (
|
||||
OptionUseInt64 Options = decoder.OptionUseInt64
|
||||
OptionUseNumber Options = decoder.OptionUseNumber
|
||||
OptionUseUnicodeErrors Options = decoder.OptionUseUnicodeErrors
|
||||
OptionDisableUnknown Options = decoder.OptionDisableUnknown
|
||||
OptionCopyString Options = decoder.OptionCopyString
|
||||
OptionValidateString Options = decoder.OptionValidateString
|
||||
)
|
||||
|
||||
// StreamDecoder is the decoder context object for streaming input.
|
||||
type StreamDecoder = decoder.StreamDecoder
|
||||
|
||||
type SyntaxError = decoder.SyntaxError
|
||||
|
||||
var (
|
||||
// NewDecoder creates a new decoder instance.
|
||||
NewDecoder = decoder.NewDecoder
|
||||
|
||||
// NewStreamDecoder adapts to encoding/json.NewDecoder API.
|
||||
//
|
||||
// NewStreamDecoder returns a new decoder that reads from r.
|
||||
NewStreamDecoder = decoder.NewStreamDecoder
|
||||
|
||||
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
|
||||
// order to reduce the first-hit latency.
|
||||
//
|
||||
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
|
||||
// a compile option to set the depth of recursive compile for the nested struct type.
|
||||
Pretouch = decoder.Pretouch
|
||||
|
||||
// Skip skips only one json value, and returns first non-blank character position and its ending position if it is valid.
|
||||
// Otherwise, returns negative error code using start and invalid character position using end
|
||||
Skip = decoder.Skip
|
||||
)
|
||||
196
vendor/github.com/bytedance/sonic/decoder/decoder_compat.go
generated
vendored
Normal file
196
vendor/github.com/bytedance/sonic/decoder/decoder_compat.go
generated
vendored
Normal file
@@ -0,0 +1,196 @@
|
||||
// +build !amd64 go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2023 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`encoding/json`
|
||||
`bytes`
|
||||
`reflect`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/option`
|
||||
`io`
|
||||
)
|
||||
|
||||
const (
|
||||
_F_use_int64 = iota
|
||||
_F_use_number
|
||||
_F_disable_urc
|
||||
_F_disable_unknown
|
||||
_F_copy_string
|
||||
_F_validate_string
|
||||
|
||||
_F_allow_control = 31
|
||||
)
|
||||
|
||||
type Options uint64
|
||||
|
||||
const (
|
||||
OptionUseInt64 Options = 1 << _F_use_int64
|
||||
OptionUseNumber Options = 1 << _F_use_number
|
||||
OptionUseUnicodeErrors Options = 1 << _F_disable_urc
|
||||
OptionDisableUnknown Options = 1 << _F_disable_unknown
|
||||
OptionCopyString Options = 1 << _F_copy_string
|
||||
OptionValidateString Options = 1 << _F_validate_string
|
||||
)
|
||||
|
||||
func (self *Decoder) SetOptions(opts Options) {
|
||||
if (opts & OptionUseNumber != 0) && (opts & OptionUseInt64 != 0) {
|
||||
panic("can't set OptionUseInt64 and OptionUseNumber both!")
|
||||
}
|
||||
self.f = uint64(opts)
|
||||
}
|
||||
|
||||
|
||||
// Decoder is the decoder context object
|
||||
type Decoder struct {
|
||||
i int
|
||||
f uint64
|
||||
s string
|
||||
}
|
||||
|
||||
// NewDecoder creates a new decoder instance.
|
||||
func NewDecoder(s string) *Decoder {
|
||||
return &Decoder{s: s}
|
||||
}
|
||||
|
||||
// Pos returns the current decoding position.
|
||||
func (self *Decoder) Pos() int {
|
||||
return self.i
|
||||
}
|
||||
|
||||
func (self *Decoder) Reset(s string) {
|
||||
self.s = s
|
||||
self.i = 0
|
||||
// self.f = 0
|
||||
}
|
||||
|
||||
// NOTE: api fallback do nothing
|
||||
func (self *Decoder) CheckTrailings() error {
|
||||
pos := self.i
|
||||
buf := self.s
|
||||
/* skip all the trailing spaces */
|
||||
if pos != len(buf) {
|
||||
for pos < len(buf) && (types.SPACE_MASK & (1 << buf[pos])) != 0 {
|
||||
pos++
|
||||
}
|
||||
}
|
||||
|
||||
/* then it must be at EOF */
|
||||
if pos == len(buf) {
|
||||
return nil
|
||||
}
|
||||
|
||||
/* junk after JSON value */
|
||||
return nil
|
||||
}
|
||||
|
||||
|
||||
// Decode parses the JSON-encoded data from current position and stores the result
|
||||
// in the value pointed to by val.
|
||||
func (self *Decoder) Decode(val interface{}) error {
|
||||
r := bytes.NewBufferString(self.s)
|
||||
dec := json.NewDecoder(r)
|
||||
if (self.f | uint64(OptionUseNumber)) != 0 {
|
||||
dec.UseNumber()
|
||||
}
|
||||
if (self.f | uint64(OptionDisableUnknown)) != 0 {
|
||||
dec.DisallowUnknownFields()
|
||||
}
|
||||
return dec.Decode(val)
|
||||
}
|
||||
|
||||
// UseInt64 indicates the Decoder to unmarshal an integer into an interface{} as an
|
||||
// int64 instead of as a float64.
|
||||
func (self *Decoder) UseInt64() {
|
||||
self.f |= 1 << _F_use_int64
|
||||
self.f &^= 1 << _F_use_number
|
||||
}
|
||||
|
||||
// UseNumber indicates the Decoder to unmarshal a number into an interface{} as a
|
||||
// json.Number instead of as a float64.
|
||||
func (self *Decoder) UseNumber() {
|
||||
self.f &^= 1 << _F_use_int64
|
||||
self.f |= 1 << _F_use_number
|
||||
}
|
||||
|
||||
// UseUnicodeErrors indicates the Decoder to return an error when encounter invalid
|
||||
// UTF-8 escape sequences.
|
||||
func (self *Decoder) UseUnicodeErrors() {
|
||||
self.f |= 1 << _F_disable_urc
|
||||
}
|
||||
|
||||
// DisallowUnknownFields indicates the Decoder to return an error when the destination
|
||||
// is a struct and the input contains object keys which do not match any
|
||||
// non-ignored, exported fields in the destination.
|
||||
func (self *Decoder) DisallowUnknownFields() {
|
||||
self.f |= 1 << _F_disable_unknown
|
||||
}
|
||||
|
||||
// CopyString indicates the Decoder to decode string values by copying instead of referring.
|
||||
func (self *Decoder) CopyString() {
|
||||
self.f |= 1 << _F_copy_string
|
||||
}
|
||||
|
||||
// ValidateString causes the Decoder to validate string values when decoding string value
|
||||
// in JSON. Validation is that, returning error when unescaped control chars(0x00-0x1f) or
|
||||
// invalid UTF-8 chars in the string value of JSON.
|
||||
func (self *Decoder) ValidateString() {
|
||||
self.f |= 1 << _F_validate_string
|
||||
}
|
||||
|
||||
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
|
||||
// order to reduce the first-hit latency.
|
||||
//
|
||||
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
|
||||
// a compile option to set the depth of recursive compile for the nested struct type.
|
||||
func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
type StreamDecoder struct {
|
||||
r io.Reader
|
||||
buf []byte
|
||||
scanp int
|
||||
scanned int64
|
||||
err error
|
||||
Decoder
|
||||
}
|
||||
|
||||
// NewStreamDecoder adapts to encoding/json.NewDecoder API.
|
||||
//
|
||||
// NewStreamDecoder returns a new decoder that reads from r.
|
||||
func NewStreamDecoder(r io.Reader) *StreamDecoder {
|
||||
return &StreamDecoder{r : r}
|
||||
}
|
||||
|
||||
// Decode decodes input stream into val with corresponding data.
|
||||
// Redundantly bytes may be read and left in its buffer, and can be used at next call.
|
||||
// Either io error from underlying io.Reader (except io.EOF)
|
||||
// or syntax error from data will be recorded and stop subsequently decoding.
|
||||
func (self *StreamDecoder) Decode(val interface{}) (err error) {
|
||||
dec := json.NewDecoder(self.r)
|
||||
if (self.f | uint64(OptionUseNumber)) != 0 {
|
||||
dec.UseNumber()
|
||||
}
|
||||
if (self.f | uint64(OptionDisableUnknown)) != 0 {
|
||||
dec.DisallowUnknownFields()
|
||||
}
|
||||
return dec.Decode(val)
|
||||
}
|
||||
|
||||
108
vendor/github.com/bytedance/sonic/encoder/encoder_amd64.go
generated
vendored
Normal file
108
vendor/github.com/bytedance/sonic/encoder/encoder_amd64.go
generated
vendored
Normal file
@@ -0,0 +1,108 @@
|
||||
// +build amd64,go1.15,!go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2023 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`github.com/bytedance/sonic/internal/encoder`
|
||||
)
|
||||
|
||||
|
||||
// Encoder represents a specific set of encoder configurations.
|
||||
type Encoder = encoder.Encoder
|
||||
|
||||
// StreamEncoder uses io.Writer as input.
|
||||
type StreamEncoder = encoder.StreamEncoder
|
||||
|
||||
// Options is a set of encoding options.
|
||||
type Options = encoder.Options
|
||||
|
||||
const (
|
||||
// SortMapKeys indicates that the keys of a map needs to be sorted
|
||||
// before serializing into JSON.
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
SortMapKeys Options = encoder.SortMapKeys
|
||||
|
||||
// EscapeHTML indicates encoder to escape all HTML characters
|
||||
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
EscapeHTML Options = encoder.EscapeHTML
|
||||
|
||||
// CompactMarshaler indicates that the output JSON from json.Marshaler
|
||||
// is always compact and needs no validation
|
||||
CompactMarshaler Options = encoder.CompactMarshaler
|
||||
|
||||
// NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler
|
||||
// is always escaped string and needs no quoting
|
||||
NoQuoteTextMarshaler Options = encoder.NoQuoteTextMarshaler
|
||||
|
||||
// NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}',
|
||||
// instead of 'null'
|
||||
NoNullSliceOrMap Options = encoder.NoNullSliceOrMap
|
||||
|
||||
// ValidateString indicates that encoder should validate the input string
|
||||
// before encoding it into JSON.
|
||||
ValidateString Options = encoder.ValidateString
|
||||
|
||||
// CompatibleWithStd is used to be compatible with std encoder.
|
||||
CompatibleWithStd Options = encoder.CompatibleWithStd
|
||||
)
|
||||
|
||||
|
||||
var (
|
||||
// Encode returns the JSON encoding of val, encoded with opts.
|
||||
Encode = encoder.Encode
|
||||
|
||||
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating a new one.
|
||||
EncodeIndented = encoder.EncodeIndented
|
||||
|
||||
// EncodeIndented is like Encode but applies Indent to format the output.
|
||||
// Each JSON element in the output will begin on a new line beginning with prefix
|
||||
// followed by one or more copies of indent according to the indentation nesting.
|
||||
EncodeInto = encoder.EncodeInto
|
||||
|
||||
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029
|
||||
// characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029
|
||||
// so that the JSON will be safe to embed inside HTML <script> tags.
|
||||
// For historical reasons, web browsers don't honor standard HTML
|
||||
// escaping within <script> tags, so an alternative JSON encoding must
|
||||
// be used.
|
||||
HTMLEscape = encoder.HTMLEscape
|
||||
|
||||
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
|
||||
// order to reduce the first-hit latency.
|
||||
//
|
||||
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
|
||||
// a compile option to set the depth of recursive compile for the nested struct type.
|
||||
Pretouch = encoder.Pretouch
|
||||
|
||||
// Quote returns the JSON-quoted version of s.
|
||||
Quote = encoder.Quote
|
||||
|
||||
// Valid validates json and returns first non-blank character position,
|
||||
// if it is only one valid json value.
|
||||
// Otherwise returns invalid character position using start.
|
||||
//
|
||||
// Note: it does not check for the invalid UTF-8 characters.
|
||||
Valid = encoder.Valid
|
||||
|
||||
// NewStreamEncoder adapts to encoding/json.NewDecoder API.
|
||||
//
|
||||
// NewStreamEncoder returns a new encoder that write to w.
|
||||
NewStreamEncoder = encoder.NewStreamEncoder
|
||||
)
|
||||
234
vendor/github.com/bytedance/sonic/encoder/encoder_compat.go
generated
vendored
Normal file
234
vendor/github.com/bytedance/sonic/encoder/encoder_compat.go
generated
vendored
Normal file
@@ -0,0 +1,234 @@
|
||||
// +build !amd64 go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2023 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`io`
|
||||
`bytes`
|
||||
`encoding/json`
|
||||
`reflect`
|
||||
|
||||
`github.com/bytedance/sonic/option`
|
||||
)
|
||||
|
||||
// Options is a set of encoding options.
|
||||
type Options uint64
|
||||
|
||||
const (
|
||||
bitSortMapKeys = iota
|
||||
bitEscapeHTML
|
||||
bitCompactMarshaler
|
||||
bitNoQuoteTextMarshaler
|
||||
bitNoNullSliceOrMap
|
||||
bitValidateString
|
||||
|
||||
// used for recursive compile
|
||||
bitPointerValue = 63
|
||||
)
|
||||
|
||||
const (
|
||||
// SortMapKeys indicates that the keys of a map needs to be sorted
|
||||
// before serializing into JSON.
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
SortMapKeys Options = 1 << bitSortMapKeys
|
||||
|
||||
// EscapeHTML indicates encoder to escape all HTML characters
|
||||
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
EscapeHTML Options = 1 << bitEscapeHTML
|
||||
|
||||
// CompactMarshaler indicates that the output JSON from json.Marshaler
|
||||
// is always compact and needs no validation
|
||||
CompactMarshaler Options = 1 << bitCompactMarshaler
|
||||
|
||||
// NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler
|
||||
// is always escaped string and needs no quoting
|
||||
NoQuoteTextMarshaler Options = 1 << bitNoQuoteTextMarshaler
|
||||
|
||||
// NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}',
|
||||
// instead of 'null'
|
||||
NoNullSliceOrMap Options = 1 << bitNoNullSliceOrMap
|
||||
|
||||
// ValidateString indicates that encoder should validate the input string
|
||||
// before encoding it into JSON.
|
||||
ValidateString Options = 1 << bitValidateString
|
||||
|
||||
// CompatibleWithStd is used to be compatible with std encoder.
|
||||
CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler
|
||||
)
|
||||
|
||||
// Encoder represents a specific set of encoder configurations.
|
||||
type Encoder struct {
|
||||
Opts Options
|
||||
prefix string
|
||||
indent string
|
||||
}
|
||||
|
||||
// Encode returns the JSON encoding of v.
|
||||
func (self *Encoder) Encode(v interface{}) ([]byte, error) {
|
||||
if self.indent != "" || self.prefix != "" {
|
||||
return EncodeIndented(v, self.prefix, self.indent, self.Opts)
|
||||
}
|
||||
return Encode(v, self.Opts)
|
||||
}
|
||||
|
||||
// SortKeys enables the SortMapKeys option.
|
||||
func (self *Encoder) SortKeys() *Encoder {
|
||||
self.Opts |= SortMapKeys
|
||||
return self
|
||||
}
|
||||
|
||||
// SetEscapeHTML specifies if option EscapeHTML opens
|
||||
func (self *Encoder) SetEscapeHTML(f bool) {
|
||||
if f {
|
||||
self.Opts |= EscapeHTML
|
||||
} else {
|
||||
self.Opts &= ^EscapeHTML
|
||||
}
|
||||
}
|
||||
|
||||
// SetValidateString specifies if option ValidateString opens
|
||||
func (self *Encoder) SetValidateString(f bool) {
|
||||
if f {
|
||||
self.Opts |= ValidateString
|
||||
} else {
|
||||
self.Opts &= ^ValidateString
|
||||
}
|
||||
}
|
||||
|
||||
// SetCompactMarshaler specifies if option CompactMarshaler opens
|
||||
func (self *Encoder) SetCompactMarshaler(f bool) {
|
||||
if f {
|
||||
self.Opts |= CompactMarshaler
|
||||
} else {
|
||||
self.Opts &= ^CompactMarshaler
|
||||
}
|
||||
}
|
||||
|
||||
// SetNoQuoteTextMarshaler specifies if option NoQuoteTextMarshaler opens
|
||||
func (self *Encoder) SetNoQuoteTextMarshaler(f bool) {
|
||||
if f {
|
||||
self.Opts |= NoQuoteTextMarshaler
|
||||
} else {
|
||||
self.Opts &= ^NoQuoteTextMarshaler
|
||||
}
|
||||
}
|
||||
|
||||
// SetIndent instructs the encoder to format each subsequent encoded
|
||||
// value as if indented by the package-level function EncodeIndent().
|
||||
// Calling SetIndent("", "") disables indentation.
|
||||
func (enc *Encoder) SetIndent(prefix, indent string) {
|
||||
enc.prefix = prefix
|
||||
enc.indent = indent
|
||||
}
|
||||
|
||||
// Quote returns the JSON-quoted version of s.
|
||||
func Quote(s string) string {
|
||||
/* check for empty string */
|
||||
if s == "" {
|
||||
return `""`
|
||||
}
|
||||
|
||||
out, _ := json.Marshal(s)
|
||||
return string(out)
|
||||
}
|
||||
|
||||
// Encode returns the JSON encoding of val, encoded with opts.
|
||||
func Encode(val interface{}, opts Options) ([]byte, error) {
|
||||
return json.Marshal(val)
|
||||
}
|
||||
|
||||
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating
|
||||
// a new one.
|
||||
func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
|
||||
if buf == nil {
|
||||
panic("user-supplied buffer buf is nil")
|
||||
}
|
||||
w := bytes.NewBuffer(*buf)
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetEscapeHTML((opts & EscapeHTML) != 0)
|
||||
err := enc.Encode(val)
|
||||
*buf = w.Bytes()
|
||||
return err
|
||||
}
|
||||
|
||||
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029
|
||||
// characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029
|
||||
// so that the JSON will be safe to embed inside HTML <script> tags.
|
||||
// For historical reasons, web browsers don't honor standard HTML
|
||||
// escaping within <script> tags, so an alternative JSON encoding must
|
||||
// be used.
|
||||
func HTMLEscape(dst []byte, src []byte) []byte {
|
||||
d := bytes.NewBuffer(dst)
|
||||
json.HTMLEscape(d, src)
|
||||
return d.Bytes()
|
||||
}
|
||||
|
||||
// EncodeIndented is like Encode but applies Indent to format the output.
|
||||
// Each JSON element in the output will begin on a new line beginning with prefix
|
||||
// followed by one or more copies of indent according to the indentation nesting.
|
||||
func EncodeIndented(val interface{}, prefix string, indent string, opts Options) ([]byte, error) {
|
||||
w := bytes.NewBuffer([]byte{})
|
||||
enc := json.NewEncoder(w)
|
||||
enc.SetEscapeHTML((opts & EscapeHTML) != 0)
|
||||
enc.SetIndent(prefix, indent)
|
||||
err := enc.Encode(val)
|
||||
out := w.Bytes()
|
||||
return out, err
|
||||
}
|
||||
|
||||
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
|
||||
// order to reduce the first-hit latency.
|
||||
//
|
||||
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
|
||||
// a compile option to set the depth of recursive compile for the nested struct type.
|
||||
func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Valid validates json and returns first non-blank character position,
|
||||
// if it is only one valid json value.
|
||||
// Otherwise returns invalid character position using start.
|
||||
//
|
||||
// Note: it does not check for the invalid UTF-8 characters.
|
||||
func Valid(data []byte) (ok bool, start int) {
|
||||
return json.Valid(data), 0
|
||||
}
|
||||
|
||||
// StreamEncoder uses io.Writer as
|
||||
type StreamEncoder struct {
|
||||
w io.Writer
|
||||
Encoder
|
||||
}
|
||||
|
||||
// NewStreamEncoder adapts to encoding/json.NewDecoder API.
|
||||
//
|
||||
// NewStreamEncoder returns a new encoder that write to w.
|
||||
func NewStreamEncoder(w io.Writer) *StreamEncoder {
|
||||
return &StreamEncoder{w: w}
|
||||
}
|
||||
|
||||
// Encode encodes interface{} as JSON to io.Writer
|
||||
func (enc *StreamEncoder) Encode(val interface{}) (err error) {
|
||||
jenc := json.NewEncoder(enc.w)
|
||||
jenc.SetEscapeHTML((enc.Opts & EscapeHTML) != 0)
|
||||
jenc.SetIndent(enc.prefix, enc.indent)
|
||||
err = jenc.Encode(val)
|
||||
return err
|
||||
}
|
||||
8
vendor/github.com/bytedance/sonic/go.work
generated
vendored
Normal file
8
vendor/github.com/bytedance/sonic/go.work
generated
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
go 1.18
|
||||
|
||||
use (
|
||||
.
|
||||
./generic_test
|
||||
./fuzz
|
||||
./external_jsonlib_test
|
||||
)
|
||||
0
vendor/github.com/bytedance/sonic/internal/caching/asm.s
generated
vendored
Normal file
0
vendor/github.com/bytedance/sonic/internal/caching/asm.s
generated
vendored
Normal file
115
vendor/github.com/bytedance/sonic/internal/caching/fcache.go
generated
vendored
Normal file
115
vendor/github.com/bytedance/sonic/internal/caching/fcache.go
generated
vendored
Normal file
@@ -0,0 +1,115 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package caching
|
||||
|
||||
import (
|
||||
`strings`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
type FieldMap struct {
|
||||
N uint64
|
||||
b unsafe.Pointer
|
||||
m map[string]int
|
||||
}
|
||||
|
||||
type FieldEntry struct {
|
||||
ID int
|
||||
Name string
|
||||
Hash uint64
|
||||
}
|
||||
|
||||
const (
|
||||
FieldMap_N = int64(unsafe.Offsetof(FieldMap{}.N))
|
||||
FieldMap_b = int64(unsafe.Offsetof(FieldMap{}.b))
|
||||
FieldEntrySize = int64(unsafe.Sizeof(FieldEntry{}))
|
||||
)
|
||||
|
||||
func newBucket(n int) unsafe.Pointer {
|
||||
v := make([]FieldEntry, n)
|
||||
return (*rt.GoSlice)(unsafe.Pointer(&v)).Ptr
|
||||
}
|
||||
|
||||
func CreateFieldMap(n int) *FieldMap {
|
||||
return &FieldMap {
|
||||
N: uint64(n * 2),
|
||||
b: newBucket(n * 2), // LoadFactor = 0.5
|
||||
m: make(map[string]int, n * 2),
|
||||
}
|
||||
}
|
||||
|
||||
func (self *FieldMap) At(p uint64) *FieldEntry {
|
||||
off := uintptr(p) * uintptr(FieldEntrySize)
|
||||
return (*FieldEntry)(unsafe.Pointer(uintptr(self.b) + off))
|
||||
}
|
||||
|
||||
// Get searches FieldMap by name. JIT generated assembly does NOT call this
|
||||
// function, rather it implements its own version directly in assembly. So
|
||||
// we must ensure this function stays in sync with the JIT generated one.
|
||||
func (self *FieldMap) Get(name string) int {
|
||||
h := StrHash(name)
|
||||
p := h % self.N
|
||||
s := self.At(p)
|
||||
|
||||
/* find the element;
|
||||
* the hash map is never full, so the loop will always terminate */
|
||||
for s.Hash != 0 {
|
||||
if s.Hash == h && s.Name == name {
|
||||
return s.ID
|
||||
} else {
|
||||
p = (p + 1) % self.N
|
||||
s = self.At(p)
|
||||
}
|
||||
}
|
||||
|
||||
/* not found */
|
||||
return -1
|
||||
}
|
||||
|
||||
func (self *FieldMap) Set(name string, i int) {
|
||||
h := StrHash(name)
|
||||
p := h % self.N
|
||||
s := self.At(p)
|
||||
|
||||
/* searching for an empty slot;
|
||||
* the hash map is never full, so the loop will always terminate */
|
||||
for s.Hash != 0 {
|
||||
p = (p + 1) % self.N
|
||||
s = self.At(p)
|
||||
}
|
||||
|
||||
/* set the value */
|
||||
s.ID = i
|
||||
s.Hash = h
|
||||
s.Name = name
|
||||
|
||||
/* add the case-insensitive version, prefer the one with smaller field ID */
|
||||
key := strings.ToLower(name)
|
||||
if v, ok := self.m[key]; !ok || i < v {
|
||||
self.m[key] = i
|
||||
}
|
||||
}
|
||||
|
||||
func (self *FieldMap) GetCaseInsensitive(name string) int {
|
||||
if i, ok := self.m[strings.ToLower(name)]; ok {
|
||||
return i
|
||||
} else {
|
||||
return -1
|
||||
}
|
||||
}
|
||||
40
vendor/github.com/bytedance/sonic/internal/caching/hashing.go
generated
vendored
Normal file
40
vendor/github.com/bytedance/sonic/internal/caching/hashing.go
generated
vendored
Normal file
@@ -0,0 +1,40 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package caching
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
var (
|
||||
V_strhash = rt.UnpackEface(strhash)
|
||||
S_strhash = *(*uintptr)(V_strhash.Value)
|
||||
)
|
||||
|
||||
//go:noescape
|
||||
//go:linkname strhash runtime.strhash
|
||||
func strhash(_ unsafe.Pointer, _ uintptr) uintptr
|
||||
|
||||
func StrHash(s string) uint64 {
|
||||
if v := strhash(unsafe.Pointer(&s), 0); v == 0 {
|
||||
return 1
|
||||
} else {
|
||||
return uint64(v)
|
||||
}
|
||||
}
|
||||
173
vendor/github.com/bytedance/sonic/internal/caching/pcache.go
generated
vendored
Normal file
173
vendor/github.com/bytedance/sonic/internal/caching/pcache.go
generated
vendored
Normal file
@@ -0,0 +1,173 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package caching
|
||||
|
||||
import (
|
||||
`sync`
|
||||
`sync/atomic`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
/** Program Map **/
|
||||
|
||||
const (
|
||||
_LoadFactor = 0.5
|
||||
_InitCapacity = 4096 // must be a power of 2
|
||||
)
|
||||
|
||||
type _ProgramMap struct {
|
||||
n uint64
|
||||
m uint32
|
||||
b []_ProgramEntry
|
||||
}
|
||||
|
||||
type _ProgramEntry struct {
|
||||
vt *rt.GoType
|
||||
fn interface{}
|
||||
}
|
||||
|
||||
func newProgramMap() *_ProgramMap {
|
||||
return &_ProgramMap {
|
||||
n: 0,
|
||||
m: _InitCapacity - 1,
|
||||
b: make([]_ProgramEntry, _InitCapacity),
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_ProgramMap) copy() *_ProgramMap {
|
||||
fork := &_ProgramMap{
|
||||
n: self.n,
|
||||
m: self.m,
|
||||
b: make([]_ProgramEntry, len(self.b)),
|
||||
}
|
||||
for i, f := range self.b {
|
||||
fork.b[i] = f
|
||||
}
|
||||
return fork
|
||||
}
|
||||
|
||||
func (self *_ProgramMap) get(vt *rt.GoType) interface{} {
|
||||
i := self.m + 1
|
||||
p := vt.Hash & self.m
|
||||
|
||||
/* linear probing */
|
||||
for ; i > 0; i-- {
|
||||
if b := self.b[p]; b.vt == vt {
|
||||
return b.fn
|
||||
} else if b.vt == nil {
|
||||
break
|
||||
} else {
|
||||
p = (p + 1) & self.m
|
||||
}
|
||||
}
|
||||
|
||||
/* not found */
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *_ProgramMap) add(vt *rt.GoType, fn interface{}) *_ProgramMap {
|
||||
p := self.copy()
|
||||
f := float64(atomic.LoadUint64(&p.n) + 1) / float64(p.m + 1)
|
||||
|
||||
/* check for load factor */
|
||||
if f > _LoadFactor {
|
||||
p = p.rehash()
|
||||
}
|
||||
|
||||
/* insert the value */
|
||||
p.insert(vt, fn)
|
||||
return p
|
||||
}
|
||||
|
||||
func (self *_ProgramMap) rehash() *_ProgramMap {
|
||||
c := (self.m + 1) << 1
|
||||
r := &_ProgramMap{m: c - 1, b: make([]_ProgramEntry, int(c))}
|
||||
|
||||
/* rehash every entry */
|
||||
for i := uint32(0); i <= self.m; i++ {
|
||||
if b := self.b[i]; b.vt != nil {
|
||||
r.insert(b.vt, b.fn)
|
||||
}
|
||||
}
|
||||
|
||||
/* rebuild successful */
|
||||
return r
|
||||
}
|
||||
|
||||
func (self *_ProgramMap) insert(vt *rt.GoType, fn interface{}) {
|
||||
h := vt.Hash
|
||||
p := h & self.m
|
||||
|
||||
/* linear probing */
|
||||
for i := uint32(0); i <= self.m; i++ {
|
||||
if b := &self.b[p]; b.vt != nil {
|
||||
p += 1
|
||||
p &= self.m
|
||||
} else {
|
||||
b.vt = vt
|
||||
b.fn = fn
|
||||
atomic.AddUint64(&self.n, 1)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
/* should never happens */
|
||||
panic("no available slots")
|
||||
}
|
||||
|
||||
/** RCU Program Cache **/
|
||||
|
||||
type ProgramCache struct {
|
||||
m sync.Mutex
|
||||
p unsafe.Pointer
|
||||
}
|
||||
|
||||
func CreateProgramCache() *ProgramCache {
|
||||
return &ProgramCache {
|
||||
m: sync.Mutex{},
|
||||
p: unsafe.Pointer(newProgramMap()),
|
||||
}
|
||||
}
|
||||
|
||||
func (self *ProgramCache) Get(vt *rt.GoType) interface{} {
|
||||
return (*_ProgramMap)(atomic.LoadPointer(&self.p)).get(vt)
|
||||
}
|
||||
|
||||
func (self *ProgramCache) Compute(vt *rt.GoType, compute func(*rt.GoType, ... interface{}) (interface{}, error), ex ...interface{}) (interface{}, error) {
|
||||
var err error
|
||||
var val interface{}
|
||||
|
||||
/* use defer to prevent inlining of this function */
|
||||
self.m.Lock()
|
||||
defer self.m.Unlock()
|
||||
|
||||
/* double check with write lock held */
|
||||
if val = self.Get(vt); val != nil {
|
||||
return val, nil
|
||||
}
|
||||
|
||||
/* compute the value */
|
||||
if val, err = compute(vt, ex...); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
/* update the RCU cache */
|
||||
atomic.StorePointer(&self.p, unsafe.Pointer((*_ProgramMap)(atomic.LoadPointer(&self.p)).add(vt, val)))
|
||||
return val, nil
|
||||
}
|
||||
40
vendor/github.com/bytedance/sonic/internal/cpu/features.go
generated
vendored
Normal file
40
vendor/github.com/bytedance/sonic/internal/cpu/features.go
generated
vendored
Normal file
@@ -0,0 +1,40 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package cpu
|
||||
|
||||
import (
|
||||
`fmt`
|
||||
`os`
|
||||
|
||||
`github.com/klauspost/cpuid/v2`
|
||||
)
|
||||
|
||||
var (
|
||||
HasAVX = cpuid.CPU.Has(cpuid.AVX)
|
||||
HasAVX2 = cpuid.CPU.Has(cpuid.AVX2)
|
||||
HasSSE = cpuid.CPU.Has(cpuid.SSE)
|
||||
)
|
||||
|
||||
func init() {
|
||||
switch v := os.Getenv("SONIC_MODE"); v {
|
||||
case "" : break
|
||||
case "auto" : break
|
||||
case "noavx" : HasAVX = false; fallthrough
|
||||
case "noavx2" : HasAVX2 = false
|
||||
default : panic(fmt.Sprintf("invalid mode: '%s', should be one of 'auto', 'noavx', 'noavx2'", v))
|
||||
}
|
||||
}
|
||||
0
vendor/github.com/bytedance/sonic/internal/decoder/asm.s
generated
vendored
Normal file
0
vendor/github.com/bytedance/sonic/internal/decoder/asm.s
generated
vendored
Normal file
2013
vendor/github.com/bytedance/sonic/internal/decoder/assembler_amd64_go116.go
generated
vendored
Normal file
2013
vendor/github.com/bytedance/sonic/internal/decoder/assembler_amd64_go116.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
1992
vendor/github.com/bytedance/sonic/internal/decoder/assembler_amd64_go117.go
generated
vendored
Normal file
1992
vendor/github.com/bytedance/sonic/internal/decoder/assembler_amd64_go117.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
1155
vendor/github.com/bytedance/sonic/internal/decoder/compiler.go
generated
vendored
Normal file
1155
vendor/github.com/bytedance/sonic/internal/decoder/compiler.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
70
vendor/github.com/bytedance/sonic/internal/decoder/debug.go
generated
vendored
Normal file
70
vendor/github.com/bytedance/sonic/internal/decoder/debug.go
generated
vendored
Normal file
@@ -0,0 +1,70 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`os`
|
||||
`runtime`
|
||||
`runtime/debug`
|
||||
`strings`
|
||||
|
||||
`github.com/bytedance/sonic/internal/jit`
|
||||
)
|
||||
|
||||
|
||||
var (
|
||||
debugSyncGC = os.Getenv("SONIC_SYNC_GC") != ""
|
||||
debugAsyncGC = os.Getenv("SONIC_NO_ASYNC_GC") == ""
|
||||
)
|
||||
|
||||
var (
|
||||
_Instr_End _Instr = newInsOp(_OP_nil_1)
|
||||
|
||||
_F_gc = jit.Func(runtime.GC)
|
||||
_F_force_gc = jit.Func(debug.FreeOSMemory)
|
||||
_F_println = jit.Func(println_wrapper)
|
||||
_F_print = jit.Func(print)
|
||||
)
|
||||
|
||||
func println_wrapper(i int, op1 int, op2 int){
|
||||
println(i, " Intrs ", op1, _OpNames[op1], "next: ", op2, _OpNames[op2])
|
||||
}
|
||||
|
||||
func print(i int){
|
||||
println(i)
|
||||
}
|
||||
|
||||
func (self *_Assembler) force_gc() {
|
||||
self.call_go(_F_gc)
|
||||
self.call_go(_F_force_gc)
|
||||
}
|
||||
|
||||
func (self *_Assembler) debug_instr(i int, v *_Instr) {
|
||||
if debugSyncGC {
|
||||
if (i+1 == len(self.p)) {
|
||||
self.print_gc(i, v, &_Instr_End)
|
||||
} else {
|
||||
next := &(self.p[i+1])
|
||||
self.print_gc(i, v, next)
|
||||
name := _OpNames[next.op()]
|
||||
if strings.Contains(name, "save") {
|
||||
return
|
||||
}
|
||||
}
|
||||
self.force_gc()
|
||||
}
|
||||
}
|
||||
255
vendor/github.com/bytedance/sonic/internal/decoder/decoder.go
generated
vendored
Normal file
255
vendor/github.com/bytedance/sonic/internal/decoder/decoder.go
generated
vendored
Normal file
@@ -0,0 +1,255 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
`encoding/json`
|
||||
`reflect`
|
||||
`runtime`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
`github.com/bytedance/sonic/option`
|
||||
`github.com/bytedance/sonic/utf8`
|
||||
)
|
||||
|
||||
const (
|
||||
_F_use_int64 = iota
|
||||
_F_use_number
|
||||
_F_disable_urc
|
||||
_F_disable_unknown
|
||||
_F_copy_string
|
||||
_F_validate_string
|
||||
|
||||
_F_allow_control = 31
|
||||
)
|
||||
|
||||
type Options uint64
|
||||
|
||||
const (
|
||||
OptionUseInt64 Options = 1 << _F_use_int64
|
||||
OptionUseNumber Options = 1 << _F_use_number
|
||||
OptionUseUnicodeErrors Options = 1 << _F_disable_urc
|
||||
OptionDisableUnknown Options = 1 << _F_disable_unknown
|
||||
OptionCopyString Options = 1 << _F_copy_string
|
||||
OptionValidateString Options = 1 << _F_validate_string
|
||||
)
|
||||
|
||||
func (self *Decoder) SetOptions(opts Options) {
|
||||
if (opts & OptionUseNumber != 0) && (opts & OptionUseInt64 != 0) {
|
||||
panic("can't set OptionUseInt64 and OptionUseNumber both!")
|
||||
}
|
||||
self.f = uint64(opts)
|
||||
}
|
||||
|
||||
|
||||
// Decoder is the decoder context object
|
||||
type Decoder struct {
|
||||
i int
|
||||
f uint64
|
||||
s string
|
||||
}
|
||||
|
||||
// NewDecoder creates a new decoder instance.
|
||||
func NewDecoder(s string) *Decoder {
|
||||
return &Decoder{s: s}
|
||||
}
|
||||
|
||||
// Pos returns the current decoding position.
|
||||
func (self *Decoder) Pos() int {
|
||||
return self.i
|
||||
}
|
||||
|
||||
func (self *Decoder) Reset(s string) {
|
||||
self.s = s
|
||||
self.i = 0
|
||||
// self.f = 0
|
||||
}
|
||||
|
||||
func (self *Decoder) CheckTrailings() error {
|
||||
pos := self.i
|
||||
buf := self.s
|
||||
/* skip all the trailing spaces */
|
||||
if pos != len(buf) {
|
||||
for pos < len(buf) && (types.SPACE_MASK & (1 << buf[pos])) != 0 {
|
||||
pos++
|
||||
}
|
||||
}
|
||||
|
||||
/* then it must be at EOF */
|
||||
if pos == len(buf) {
|
||||
return nil
|
||||
}
|
||||
|
||||
/* junk after JSON value */
|
||||
return SyntaxError {
|
||||
Src : buf,
|
||||
Pos : pos,
|
||||
Code : types.ERR_INVALID_CHAR,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
// Decode parses the JSON-encoded data from current position and stores the result
|
||||
// in the value pointed to by val.
|
||||
func (self *Decoder) Decode(val interface{}) error {
|
||||
/* validate json if needed */
|
||||
if (self.f & (1 << _F_validate_string)) != 0 && !utf8.ValidateString(self.s){
|
||||
dbuf := utf8.CorrectWith(nil, rt.Str2Mem(self.s), "\ufffd")
|
||||
self.s = rt.Mem2Str(dbuf)
|
||||
}
|
||||
|
||||
vv := rt.UnpackEface(val)
|
||||
vp := vv.Value
|
||||
|
||||
/* check for nil type */
|
||||
if vv.Type == nil {
|
||||
return &json.InvalidUnmarshalError{}
|
||||
}
|
||||
|
||||
/* must be a non-nil pointer */
|
||||
if vp == nil || vv.Type.Kind() != reflect.Ptr {
|
||||
return &json.InvalidUnmarshalError{Type: vv.Type.Pack()}
|
||||
}
|
||||
|
||||
etp := rt.PtrElem(vv.Type)
|
||||
|
||||
/* check the defined pointer type for issue 379 */
|
||||
if vv.Type.IsNamed() {
|
||||
newp := vp
|
||||
etp = vv.Type
|
||||
vp = unsafe.Pointer(&newp)
|
||||
}
|
||||
|
||||
/* create a new stack, and call the decoder */
|
||||
sb := newStack()
|
||||
nb, err := decodeTypedPointer(self.s, self.i, etp, vp, sb, self.f)
|
||||
/* return the stack back */
|
||||
self.i = nb
|
||||
freeStack(sb)
|
||||
|
||||
/* avoid GC ahead */
|
||||
runtime.KeepAlive(vv)
|
||||
return err
|
||||
}
|
||||
|
||||
// UseInt64 indicates the Decoder to unmarshal an integer into an interface{} as an
|
||||
// int64 instead of as a float64.
|
||||
func (self *Decoder) UseInt64() {
|
||||
self.f |= 1 << _F_use_int64
|
||||
self.f &^= 1 << _F_use_number
|
||||
}
|
||||
|
||||
// UseNumber indicates the Decoder to unmarshal a number into an interface{} as a
|
||||
// json.Number instead of as a float64.
|
||||
func (self *Decoder) UseNumber() {
|
||||
self.f &^= 1 << _F_use_int64
|
||||
self.f |= 1 << _F_use_number
|
||||
}
|
||||
|
||||
// UseUnicodeErrors indicates the Decoder to return an error when encounter invalid
|
||||
// UTF-8 escape sequences.
|
||||
func (self *Decoder) UseUnicodeErrors() {
|
||||
self.f |= 1 << _F_disable_urc
|
||||
}
|
||||
|
||||
// DisallowUnknownFields indicates the Decoder to return an error when the destination
|
||||
// is a struct and the input contains object keys which do not match any
|
||||
// non-ignored, exported fields in the destination.
|
||||
func (self *Decoder) DisallowUnknownFields() {
|
||||
self.f |= 1 << _F_disable_unknown
|
||||
}
|
||||
|
||||
// CopyString indicates the Decoder to decode string values by copying instead of referring.
|
||||
func (self *Decoder) CopyString() {
|
||||
self.f |= 1 << _F_copy_string
|
||||
}
|
||||
|
||||
// ValidateString causes the Decoder to validate string values when decoding string value
|
||||
// in JSON. Validation is that, returning error when unescaped control chars(0x00-0x1f) or
|
||||
// invalid UTF-8 chars in the string value of JSON.
|
||||
func (self *Decoder) ValidateString() {
|
||||
self.f |= 1 << _F_validate_string
|
||||
}
|
||||
|
||||
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
|
||||
// order to reduce the first-hit latency.
|
||||
//
|
||||
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
|
||||
// a compile option to set the depth of recursive compile for the nested struct type.
|
||||
func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
|
||||
cfg := option.DefaultCompileOptions()
|
||||
for _, opt := range opts {
|
||||
opt(&cfg)
|
||||
}
|
||||
return pretouchRec(map[reflect.Type]bool{vt:true}, cfg)
|
||||
}
|
||||
|
||||
func pretouchType(_vt reflect.Type, opts option.CompileOptions) (map[reflect.Type]bool, error) {
|
||||
/* compile function */
|
||||
compiler := newCompiler().apply(opts)
|
||||
decoder := func(vt *rt.GoType, _ ...interface{}) (interface{}, error) {
|
||||
if pp, err := compiler.compile(_vt); err != nil {
|
||||
return nil, err
|
||||
} else {
|
||||
as := newAssembler(pp)
|
||||
as.name = _vt.String()
|
||||
return as.Load(), nil
|
||||
}
|
||||
}
|
||||
|
||||
/* find or compile */
|
||||
vt := rt.UnpackType(_vt)
|
||||
if val := programCache.Get(vt); val != nil {
|
||||
return nil, nil
|
||||
} else if _, err := programCache.Compute(vt, decoder); err == nil {
|
||||
return compiler.rec, nil
|
||||
} else {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
func pretouchRec(vtm map[reflect.Type]bool, opts option.CompileOptions) error {
|
||||
if opts.RecursiveDepth < 0 || len(vtm) == 0 {
|
||||
return nil
|
||||
}
|
||||
next := make(map[reflect.Type]bool)
|
||||
for vt := range(vtm) {
|
||||
sub, err := pretouchType(vt, opts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for svt := range(sub) {
|
||||
next[svt] = true
|
||||
}
|
||||
}
|
||||
opts.RecursiveDepth -= 1
|
||||
return pretouchRec(next, opts)
|
||||
}
|
||||
|
||||
// Skip skips only one json value, and returns first non-blank character position and its ending position if it is valid.
|
||||
// Otherwise, returns negative error code using start and invalid character position using end
|
||||
func Skip(data []byte) (start int, end int) {
|
||||
s := rt.Mem2Str(data)
|
||||
p := 0
|
||||
m := types.NewStateMachine()
|
||||
ret := native.SkipOne(&s, &p, m, uint64(0))
|
||||
types.FreeStateMachine(m)
|
||||
return ret, p
|
||||
}
|
||||
181
vendor/github.com/bytedance/sonic/internal/decoder/errors.go
generated
vendored
Normal file
181
vendor/github.com/bytedance/sonic/internal/decoder/errors.go
generated
vendored
Normal file
@@ -0,0 +1,181 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`encoding/json`
|
||||
`errors`
|
||||
`fmt`
|
||||
`reflect`
|
||||
`strconv`
|
||||
`strings`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
type SyntaxError struct {
|
||||
Pos int
|
||||
Src string
|
||||
Code types.ParsingError
|
||||
Msg string
|
||||
}
|
||||
|
||||
func (self SyntaxError) Error() string {
|
||||
return fmt.Sprintf("%q", self.Description())
|
||||
}
|
||||
|
||||
func (self SyntaxError) Description() string {
|
||||
return "Syntax error " + self.description()
|
||||
}
|
||||
|
||||
func (self SyntaxError) description() string {
|
||||
i := 16
|
||||
p := self.Pos - i
|
||||
q := self.Pos + i
|
||||
|
||||
/* check for empty source */
|
||||
if self.Src == "" {
|
||||
return fmt.Sprintf("no sources available: %#v", self)
|
||||
}
|
||||
|
||||
/* prevent slicing before the beginning */
|
||||
if p < 0 {
|
||||
p, q, i = 0, q - p, i + p
|
||||
}
|
||||
|
||||
/* prevent slicing beyond the end */
|
||||
if n := len(self.Src); q > n {
|
||||
n = q - n
|
||||
q = len(self.Src)
|
||||
|
||||
/* move the left bound if possible */
|
||||
if p > n {
|
||||
i += n
|
||||
p -= n
|
||||
}
|
||||
}
|
||||
|
||||
/* left and right length */
|
||||
x := clamp_zero(i)
|
||||
y := clamp_zero(q - p - i - 1)
|
||||
|
||||
/* compose the error description */
|
||||
return fmt.Sprintf(
|
||||
"at index %d: %s\n\n\t%s\n\t%s^%s\n",
|
||||
self.Pos,
|
||||
self.Message(),
|
||||
self.Src[p:q],
|
||||
strings.Repeat(".", x),
|
||||
strings.Repeat(".", y),
|
||||
)
|
||||
}
|
||||
|
||||
func (self SyntaxError) Message() string {
|
||||
if self.Msg == "" {
|
||||
return self.Code.Message()
|
||||
}
|
||||
return self.Msg
|
||||
}
|
||||
|
||||
func clamp_zero(v int) int {
|
||||
if v < 0 {
|
||||
return 0
|
||||
} else {
|
||||
return v
|
||||
}
|
||||
}
|
||||
|
||||
/** JIT Error Helpers **/
|
||||
|
||||
var stackOverflow = &json.UnsupportedValueError {
|
||||
Str : "Value nesting too deep",
|
||||
Value : reflect.ValueOf("..."),
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
func error_wrap(src string, pos int, code types.ParsingError) error {
|
||||
return SyntaxError {
|
||||
Pos : pos,
|
||||
Src : src,
|
||||
Code : code,
|
||||
}
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
func error_type(vt *rt.GoType) error {
|
||||
return &json.UnmarshalTypeError{Type: vt.Pack()}
|
||||
}
|
||||
|
||||
type MismatchTypeError struct {
|
||||
Pos int
|
||||
Src string
|
||||
Type reflect.Type
|
||||
}
|
||||
|
||||
func swithchJSONType (src string, pos int) string {
|
||||
var val string
|
||||
switch src[pos] {
|
||||
case 'f': fallthrough
|
||||
case 't': val = "bool"
|
||||
case '"': val = "string"
|
||||
case '{': val = "object"
|
||||
case '[': val = "array"
|
||||
case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9': val = "number"
|
||||
}
|
||||
return val
|
||||
}
|
||||
|
||||
func (self MismatchTypeError) Error() string {
|
||||
se := SyntaxError {
|
||||
Pos : self.Pos,
|
||||
Src : self.Src,
|
||||
Code : types.ERR_MISMATCH,
|
||||
}
|
||||
return fmt.Sprintf("Mismatch type %s with value %s %q", self.Type.String(), swithchJSONType(self.Src, self.Pos), se.description())
|
||||
}
|
||||
|
||||
func (self MismatchTypeError) Description() string {
|
||||
se := SyntaxError {
|
||||
Pos : self.Pos,
|
||||
Src : self.Src,
|
||||
Code : types.ERR_MISMATCH,
|
||||
}
|
||||
return fmt.Sprintf("Mismatch type %s with value %s %s", self.Type.String(), swithchJSONType(self.Src, self.Pos), se.description())
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
func error_mismatch(src string, pos int, vt *rt.GoType) error {
|
||||
return &MismatchTypeError {
|
||||
Pos : pos,
|
||||
Src : src,
|
||||
Type : vt.Pack(),
|
||||
}
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
func error_field(name string) error {
|
||||
return errors.New("json: unknown field " + strconv.Quote(name))
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
func error_value(value string, vtype reflect.Type) error {
|
||||
return &json.UnmarshalTypeError {
|
||||
Type : vtype,
|
||||
Value : value,
|
||||
}
|
||||
}
|
||||
776
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_go116.go
generated
vendored
Normal file
776
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_go116.go
generated
vendored
Normal file
@@ -0,0 +1,776 @@
|
||||
// +build go1.15,!go1.17
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`encoding/json`
|
||||
`fmt`
|
||||
`reflect`
|
||||
`strconv`
|
||||
|
||||
`github.com/bytedance/sonic/internal/jit`
|
||||
`github.com/bytedance/sonic/internal/native`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/twitchyliquid64/golang-asm/obj`
|
||||
`github.com/twitchyliquid64/golang-asm/obj/x86`
|
||||
)
|
||||
|
||||
/** Crucial Registers:
|
||||
*
|
||||
* ST(BX) : ro, decoder stack
|
||||
* DF(R10) : ro, decoder flags
|
||||
* EP(R11) : wo, error pointer
|
||||
* IP(R12) : ro, input pointer
|
||||
* IL(R13) : ro, input length
|
||||
* IC(R14) : rw, input cursor
|
||||
* VP(R15) : ro, value pointer (to an interface{})
|
||||
*/
|
||||
|
||||
const (
|
||||
_VD_args = 8 // 8 bytes for passing arguments to this functions
|
||||
_VD_fargs = 64 // 64 bytes for passing arguments to other Go functions
|
||||
_VD_saves = 40 // 40 bytes for saving the registers before CALL instructions
|
||||
_VD_locals = 88 // 88 bytes for local variables
|
||||
)
|
||||
|
||||
const (
|
||||
_VD_offs = _VD_fargs + _VD_saves + _VD_locals
|
||||
_VD_size = _VD_offs + 8 // 8 bytes for the parent frame pointer
|
||||
)
|
||||
|
||||
var (
|
||||
_VAR_ss = _VAR_ss_Vt
|
||||
_VAR_df = jit.Ptr(_SP, _VD_fargs + _VD_saves)
|
||||
)
|
||||
|
||||
var (
|
||||
_VAR_ss_Vt = jit.Ptr(_SP, _VD_fargs + _VD_saves + 8)
|
||||
_VAR_ss_Dv = jit.Ptr(_SP, _VD_fargs + _VD_saves + 16)
|
||||
_VAR_ss_Iv = jit.Ptr(_SP, _VD_fargs + _VD_saves + 24)
|
||||
_VAR_ss_Ep = jit.Ptr(_SP, _VD_fargs + _VD_saves + 32)
|
||||
_VAR_ss_Db = jit.Ptr(_SP, _VD_fargs + _VD_saves + 40)
|
||||
_VAR_ss_Dc = jit.Ptr(_SP, _VD_fargs + _VD_saves + 48)
|
||||
)
|
||||
|
||||
var (
|
||||
_VAR_cs_LR = jit.Ptr(_SP, _VD_fargs + _VD_saves + 56)
|
||||
_VAR_cs_p = jit.Ptr(_SP, _VD_fargs + _VD_saves + 64)
|
||||
_VAR_cs_n = jit.Ptr(_SP, _VD_fargs + _VD_saves + 72)
|
||||
_VAR_cs_d = jit.Ptr(_SP, _VD_fargs + _VD_saves + 80)
|
||||
)
|
||||
|
||||
type _ValueDecoder struct {
|
||||
jit.BaseAssembler
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) build() uintptr {
|
||||
self.Init(self.compile)
|
||||
return *(*uintptr)(self.Load("decode_value", _VD_size, _VD_args, argPtrs_generic, localPtrs_generic))
|
||||
}
|
||||
|
||||
/** Function Calling Helpers **/
|
||||
|
||||
func (self *_ValueDecoder) save(r ...obj.Addr) {
|
||||
for i, v := range r {
|
||||
if i > _VD_saves / 8 - 1 {
|
||||
panic("too many registers to save")
|
||||
} else {
|
||||
self.Emit("MOVQ", v, jit.Ptr(_SP, _VD_fargs + int64(i) * 8))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) load(r ...obj.Addr) {
|
||||
for i, v := range r {
|
||||
if i > _VD_saves / 8 - 1 {
|
||||
panic("too many registers to load")
|
||||
} else {
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, _VD_fargs + int64(i) * 8), v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) call(fn obj.Addr) {
|
||||
self.Emit("MOVQ", fn, _AX) // MOVQ ${fn}, AX
|
||||
self.Rjmp("CALL", _AX) // CALL AX
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) call_go(fn obj.Addr) {
|
||||
self.save(_REG_go...) // SAVE $REG_go
|
||||
self.call(fn) // CALL ${fn}
|
||||
self.load(_REG_go...) // LOAD $REG_go
|
||||
}
|
||||
|
||||
/** Decoder Assembler **/
|
||||
|
||||
const (
|
||||
_S_val = iota + 1
|
||||
_S_arr
|
||||
_S_arr_0
|
||||
_S_obj
|
||||
_S_obj_0
|
||||
_S_obj_delim
|
||||
_S_obj_sep
|
||||
)
|
||||
|
||||
const (
|
||||
_S_omask_key = (1 << _S_obj_0) | (1 << _S_obj_sep)
|
||||
_S_omask_end = (1 << _S_obj_0) | (1 << _S_obj)
|
||||
_S_vmask = (1 << _S_val) | (1 << _S_arr_0)
|
||||
)
|
||||
|
||||
const (
|
||||
_A_init_len = 1
|
||||
_A_init_cap = 16
|
||||
)
|
||||
|
||||
const (
|
||||
_ST_Sp = 0
|
||||
_ST_Vt = _PtrBytes
|
||||
_ST_Vp = _PtrBytes * (types.MAX_RECURSE + 1)
|
||||
)
|
||||
|
||||
var (
|
||||
_V_true = jit.Imm(int64(pbool(true)))
|
||||
_V_false = jit.Imm(int64(pbool(false)))
|
||||
_F_value = jit.Imm(int64(native.S_value))
|
||||
)
|
||||
|
||||
var (
|
||||
_V_max = jit.Imm(int64(types.V_MAX))
|
||||
_E_eof = jit.Imm(int64(types.ERR_EOF))
|
||||
_E_invalid = jit.Imm(int64(types.ERR_INVALID_CHAR))
|
||||
_E_recurse = jit.Imm(int64(types.ERR_RECURSE_EXCEED_MAX))
|
||||
)
|
||||
|
||||
var (
|
||||
_F_convTslice = jit.Func(convTslice)
|
||||
_F_convTstring = jit.Func(convTstring)
|
||||
_F_invalid_vtype = jit.Func(invalid_vtype)
|
||||
)
|
||||
|
||||
var (
|
||||
_T_map = jit.Type(reflect.TypeOf((map[string]interface{})(nil)))
|
||||
_T_bool = jit.Type(reflect.TypeOf(false))
|
||||
_T_int64 = jit.Type(reflect.TypeOf(int64(0)))
|
||||
_T_eface = jit.Type(reflect.TypeOf((*interface{})(nil)).Elem())
|
||||
_T_slice = jit.Type(reflect.TypeOf(([]interface{})(nil)))
|
||||
_T_string = jit.Type(reflect.TypeOf(""))
|
||||
_T_number = jit.Type(reflect.TypeOf(json.Number("")))
|
||||
_T_float64 = jit.Type(reflect.TypeOf(float64(0)))
|
||||
)
|
||||
|
||||
var _R_tab = map[int]string {
|
||||
'[': "_decode_V_ARRAY",
|
||||
'{': "_decode_V_OBJECT",
|
||||
':': "_decode_V_KEY_SEP",
|
||||
',': "_decode_V_ELEM_SEP",
|
||||
']': "_decode_V_ARRAY_END",
|
||||
'}': "_decode_V_OBJECT_END",
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) compile() {
|
||||
self.Emit("SUBQ", jit.Imm(_VD_size), _SP) // SUBQ $_VD_size, SP
|
||||
self.Emit("MOVQ", _BP, jit.Ptr(_SP, _VD_offs)) // MOVQ BP, _VD_offs(SP)
|
||||
self.Emit("LEAQ", jit.Ptr(_SP, _VD_offs), _BP) // LEAQ _VD_offs(SP), BP
|
||||
|
||||
/* initialize the state machine */
|
||||
self.Emit("XORL", _CX, _CX) // XORL CX, CX
|
||||
self.Emit("MOVQ", _DF, _VAR_df) // MOVQ DF, df
|
||||
/* initialize digital buffer first */
|
||||
self.Emit("MOVQ", jit.Imm(_MaxDigitNums), _VAR_ss_Dc) // MOVQ $_MaxDigitNums, ss.Dcap
|
||||
self.Emit("LEAQ", jit.Ptr(_ST, _DbufOffset), _AX) // LEAQ _DbufOffset(ST), AX
|
||||
self.Emit("MOVQ", _AX, _VAR_ss_Db) // MOVQ AX, ss.Dbuf
|
||||
/* add ST offset */
|
||||
self.Emit("ADDQ", jit.Imm(_FsmOffset), _ST) // ADDQ _FsmOffset, _ST
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.WriteRecNotAX(0, _VP, jit.Ptr(_ST, _ST_Vp), false) // MOVQ VP, ST.Vp[0]
|
||||
self.Emit("MOVQ", jit.Imm(_S_val), jit.Ptr(_ST, _ST_Vt)) // MOVQ _S_val, ST.Vt[0]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* set the value from previous round */
|
||||
self.Link("_set_value") // _set_value:
|
||||
self.Emit("MOVL" , jit.Imm(_S_vmask), _DX) // MOVL _S_vmask, DX
|
||||
self.Emit("MOVQ" , jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ" , jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ AX, DX
|
||||
self.Sjmp("JNC" , "_vtype_error") // JNC _vtype_error
|
||||
self.Emit("XORL" , _SI, _SI) // XORL SI, SI
|
||||
self.Emit("SUBQ" , jit.Imm(1), jit.Ptr(_ST, _ST_Sp)) // SUBQ $1, ST.Sp
|
||||
self.Emit("XCHGQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // XCHGQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ" , _R8, jit.Ptr(_SI, 0)) // MOVQ R8, (SI)
|
||||
self.WriteRecNotAX(1, _R9, jit.Ptr(_SI, 8), false) // MOVQ R9, 8(SI)
|
||||
|
||||
/* check for value stack */
|
||||
self.Link("_next") // _next:
|
||||
self.Emit("MOVQ" , jit.Ptr(_ST, _ST_Sp), _AX) // MOVQ ST.Sp, AX
|
||||
self.Emit("TESTQ", _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JS" , "_return") // JS _return
|
||||
|
||||
/* fast path: test up to 4 characters manually */
|
||||
self.Emit("CMPQ" , _IC, _IL) // CMPQ IC, IL
|
||||
self.Sjmp("JAE" , "_decode_V_EOF") // JAE _decode_V_EOF
|
||||
self.Emit("MOVBQZX", jit.Sib(_IP, _IC, 1, 0), _AX) // MOVBQZX (IP)(IC), AX
|
||||
self.Emit("MOVQ" , jit.Imm(_BM_space), _DX) // MOVQ _BM_space, DX
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(' ')) // CMPQ AX, $' '
|
||||
self.Sjmp("JA" , "_decode_fast") // JA _decode_fast
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ _AX, _DX
|
||||
self.Sjmp("JNC" , "_decode_fast") // JNC _decode_fast
|
||||
self.Emit("ADDQ" , jit.Imm(1), _IC) // ADDQ $1, IC
|
||||
|
||||
/* at least 1 to 3 spaces */
|
||||
for i := 0; i < 3; i++ {
|
||||
self.Emit("CMPQ" , _IC, _IL) // CMPQ IC, IL
|
||||
self.Sjmp("JAE" , "_decode_V_EOF") // JAE _decode_V_EOF
|
||||
self.Emit("MOVBQZX", jit.Sib(_IP, _IC, 1, 0), _AX) // MOVBQZX (IP)(IC), AX
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(' ')) // CMPQ AX, $' '
|
||||
self.Sjmp("JA" , "_decode_fast") // JA _decode_fast
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ _AX, _DX
|
||||
self.Sjmp("JNC" , "_decode_fast") // JNC _decode_fast
|
||||
self.Emit("ADDQ" , jit.Imm(1), _IC) // ADDQ $1, IC
|
||||
}
|
||||
|
||||
/* at least 4 spaces */
|
||||
self.Emit("CMPQ" , _IC, _IL) // CMPQ IC, IL
|
||||
self.Sjmp("JAE" , "_decode_V_EOF") // JAE _decode_V_EOF
|
||||
self.Emit("MOVBQZX", jit.Sib(_IP, _IC, 1, 0), _AX) // MOVBQZX (IP)(IC), AX
|
||||
|
||||
/* fast path: use lookup table to select decoder */
|
||||
self.Link("_decode_fast") // _decode_fast:
|
||||
self.Byte(0x48, 0x8d, 0x3d) // LEAQ ?(PC), DI
|
||||
self.Sref("_decode_tab", 4) // .... &_decode_tab
|
||||
self.Emit("MOVLQSX", jit.Sib(_DI, _AX, 4, 0), _AX) // MOVLQSX (DI)(AX*4), AX
|
||||
self.Emit("TESTQ" , _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JZ" , "_decode_native") // JZ _decode_native
|
||||
self.Emit("ADDQ" , jit.Imm(1), _IC) // ADDQ $1, IC
|
||||
self.Emit("ADDQ" , _DI, _AX) // ADDQ DI, AX
|
||||
self.Rjmp("JMP" , _AX) // JMP AX
|
||||
|
||||
/* decode with native decoder */
|
||||
self.Link("_decode_native") // _decode_native:
|
||||
self.Emit("MOVQ", _IP, _DI) // MOVQ IP, DI
|
||||
self.Emit("MOVQ", _IL, _SI) // MOVQ IL, SI
|
||||
self.Emit("MOVQ", _IC, _DX) // MOVQ IC, DX
|
||||
self.Emit("LEAQ", _VAR_ss, _CX) // LEAQ ss, CX
|
||||
self.Emit("MOVQ", _VAR_df, _R8) // MOVQ $df, R8
|
||||
self.Emit("BTSQ", jit.Imm(_F_allow_control), _R8) // ANDQ $1<<_F_allow_control, R8
|
||||
self.call(_F_value) // CALL value
|
||||
self.Emit("MOVQ", _AX, _IC) // MOVQ AX, IC
|
||||
|
||||
/* check for errors */
|
||||
self.Emit("MOVQ" , _VAR_ss_Vt, _AX) // MOVQ ss.Vt, AX
|
||||
self.Emit("TESTQ", _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JS" , "_parsing_error")
|
||||
self.Sjmp("JZ" , "_invalid_vtype") // JZ _invalid_vtype
|
||||
self.Emit("CMPQ" , _AX, _V_max) // CMPQ AX, _V_max
|
||||
self.Sjmp("JA" , "_invalid_vtype") // JA _invalid_vtype
|
||||
|
||||
/* jump table selector */
|
||||
self.Byte(0x48, 0x8d, 0x3d) // LEAQ ?(PC), DI
|
||||
self.Sref("_switch_table", 4) // .... &_switch_table
|
||||
self.Emit("MOVLQSX", jit.Sib(_DI, _AX, 4, -4), _AX) // MOVLQSX -4(DI)(AX*4), AX
|
||||
self.Emit("ADDQ" , _DI, _AX) // ADDQ DI, AX
|
||||
self.Rjmp("JMP" , _AX) // JMP AX
|
||||
|
||||
/** V_EOF **/
|
||||
self.Link("_decode_V_EOF") // _decode_V_EOF:
|
||||
self.Emit("MOVL", _E_eof, _EP) // MOVL _E_eof, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
|
||||
/** V_NULL **/
|
||||
self.Link("_decode_V_NULL") // _decode_V_NULL:
|
||||
self.Emit("XORL", _R8, _R8) // XORL R8, R8
|
||||
self.Emit("XORL", _R9, _R9) // XORL R9, R9
|
||||
self.Emit("LEAQ", jit.Ptr(_IC, -4), _DI) // LEAQ -4(IC), DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_TRUE **/
|
||||
self.Link("_decode_V_TRUE") // _decode_V_TRUE:
|
||||
self.Emit("MOVQ", _T_bool, _R8) // MOVQ _T_bool, R8
|
||||
// TODO: maybe modified by users?
|
||||
self.Emit("MOVQ", _V_true, _R9) // MOVQ _V_true, R9
|
||||
self.Emit("LEAQ", jit.Ptr(_IC, -4), _DI) // LEAQ -4(IC), DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_FALSE **/
|
||||
self.Link("_decode_V_FALSE") // _decode_V_FALSE:
|
||||
self.Emit("MOVQ", _T_bool, _R8) // MOVQ _T_bool, R8
|
||||
self.Emit("MOVQ", _V_false, _R9) // MOVQ _V_false, R9
|
||||
self.Emit("LEAQ", jit.Ptr(_IC, -5), _DI) // LEAQ -5(IC), DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_ARRAY **/
|
||||
self.Link("_decode_V_ARRAY") // _decode_V_ARRAY
|
||||
self.Emit("MOVL", jit.Imm(_S_vmask), _DX) // MOVL _S_vmask, DX
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ AX, DX
|
||||
self.Sjmp("JNC" , "_invalid_char") // JNC _invalid_char
|
||||
|
||||
/* create a new array */
|
||||
self.Emit("MOVQ", _T_eface, _AX) // MOVQ _T_eface, AX
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 0)) // MOVQ AX, (SP)
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_len), jit.Ptr(_SP, 8)) // MOVQ _A_init_len, 8(SP)
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_cap), jit.Ptr(_SP, 16)) // MOVQ _A_init_cap, 16(SP)
|
||||
self.call_go(_F_makeslice) // CALL_GO runtime.makeslice
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 24), _DX) // MOVQ 24(SP), DX
|
||||
|
||||
/* pack into an interface */
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SP, 0)) // MOVQ DX, (SP)
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_len), jit.Ptr(_SP, 8)) // MOVQ _A_init_len, 8(SP)
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_cap), jit.Ptr(_SP, 16)) // MOVQ _A_init_cap, 16(SP)
|
||||
self.call_go(_F_convTslice) // CALL_GO runtime.convTslice
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 24), _R8) // MOVQ 24(SP), R8
|
||||
|
||||
/* replace current state with an array */
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Imm(_S_arr), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_arr, ST.Vt[CX]
|
||||
self.Emit("MOVQ", _T_slice, _AX) // MOVQ _T_slice, AX
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SI, 0)) // MOVQ AX, (SI)
|
||||
self.WriteRecNotAX(2, _R8, jit.Ptr(_SI, 8), false) // MOVQ R8, 8(SI)
|
||||
|
||||
/* add a new slot for the first element */
|
||||
self.Emit("ADDQ", jit.Imm(1), _CX) // ADDQ $1, CX
|
||||
self.Emit("CMPQ", _CX, jit.Imm(types.MAX_RECURSE)) // CMPQ CX, ${types.MAX_RECURSE}
|
||||
self.Sjmp("JAE" , "_stack_overflow") // JA _stack_overflow
|
||||
self.Emit("MOVQ", jit.Ptr(_R8, 0), _AX) // MOVQ (R8), AX
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.WritePtrAX(3, jit.Sib(_ST, _CX, 8, _ST_Vp), false) // MOVQ AX, ST.Vp[CX]
|
||||
self.Emit("MOVQ", jit.Imm(_S_arr_0), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_arr_0, ST.Vt[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_OBJECT **/
|
||||
self.Link("_decode_V_OBJECT") // _decode_V_OBJECT:
|
||||
self.Emit("MOVL", jit.Imm(_S_vmask), _DX) // MOVL _S_vmask, DX
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ AX, DX
|
||||
self.Sjmp("JNC" , "_invalid_char") // JNC _invalid_char
|
||||
self.call_go(_F_makemap_small) // CALL_GO runtime.makemap_small
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 0), _AX) // MOVQ (SP), AX
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Imm(_S_obj_0), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_obj, ST.Vt[CX]
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", _T_map, _DX) // MOVQ _T_map, DX
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SI, 0)) // MOVQ DX, (SI)
|
||||
self.WritePtrAX(4, jit.Ptr(_SI, 8), false) // MOVQ AX, 8(SI)
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_STRING **/
|
||||
self.Link("_decode_V_STRING") // _decode_V_STRING:
|
||||
self.Emit("MOVQ", _VAR_ss_Iv, _CX) // MOVQ ss.Iv, CX
|
||||
self.Emit("MOVQ", _IC, _AX) // MOVQ IC, AX
|
||||
self.Emit("SUBQ", _CX, _AX) // SUBQ CX, AX
|
||||
|
||||
/* check for escapes */
|
||||
self.Emit("CMPQ", _VAR_ss_Ep, jit.Imm(-1)) // CMPQ ss.Ep, $-1
|
||||
self.Sjmp("JNE" , "_unquote") // JNE _unquote
|
||||
self.Emit("SUBQ", jit.Imm(1), _AX) // SUBQ $1, AX
|
||||
self.Emit("LEAQ", jit.Sib(_IP, _CX, 1, 0), _R8) // LEAQ (IP)(CX), R8
|
||||
self.Byte(0x48, 0x8d, 0x3d) // LEAQ (PC), DI
|
||||
self.Sref("_copy_string_end", 4)
|
||||
self.Emit("BTQ", jit.Imm(_F_copy_string), _VAR_df)
|
||||
self.Sjmp("JC", "copy_string")
|
||||
self.Link("_copy_string_end")
|
||||
self.Emit("XORL", _DX, _DX) // XORL DX, DX
|
||||
/* strings with no escape sequences */
|
||||
self.Link("_noescape") // _noescape:
|
||||
self.Emit("MOVL", jit.Imm(_S_omask_key), _DI) // MOVL _S_omask, DI
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _SI) // MOVQ ST.Vt[CX], SI
|
||||
self.Emit("BTQ" , _SI, _DI) // BTQ SI, DI
|
||||
self.Sjmp("JC" , "_object_key") // JC _object_key
|
||||
|
||||
/* check for pre-packed strings, avoid 1 allocation */
|
||||
self.Emit("TESTQ", _DX, _DX) // TESTQ DX, DX
|
||||
self.Sjmp("JNZ" , "_packed_str") // JNZ _packed_str
|
||||
self.Emit("MOVQ" , _R8, jit.Ptr(_SP, 0)) // MOVQ R8, (SP)
|
||||
self.Emit("MOVQ" , _AX, jit.Ptr(_SP, 8)) // MOVQ AX, 8(SP)
|
||||
self.call_go(_F_convTstring) // CALL_GO runtime.convTstring
|
||||
self.Emit("MOVQ" , jit.Ptr(_SP, 16), _R9) // MOVQ 16(SP), R9
|
||||
|
||||
/* packed string already in R9 */
|
||||
self.Link("_packed_str") // _packed_str:
|
||||
self.Emit("MOVQ", _T_string, _R8) // MOVQ _T_string, R8
|
||||
self.Emit("MOVQ", _VAR_ss_Iv, _DI) // MOVQ ss.Iv, DI
|
||||
self.Emit("SUBQ", jit.Imm(1), _DI) // SUBQ $1, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/* the string is an object key, get the map */
|
||||
self.Link("_object_key")
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
|
||||
/* add a new delimiter */
|
||||
self.Emit("ADDQ", jit.Imm(1), _CX) // ADDQ $1, CX
|
||||
self.Emit("CMPQ", _CX, jit.Imm(types.MAX_RECURSE)) // CMPQ CX, ${types.MAX_RECURSE}
|
||||
self.Sjmp("JAE" , "_stack_overflow") // JA _stack_overflow
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.Emit("MOVQ", jit.Imm(_S_obj_delim), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_obj_delim, ST.Vt[CX]
|
||||
|
||||
/* add a new slot int the map */
|
||||
self.Emit("MOVQ", _T_map, _DX) // MOVQ _T_map, DX
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SP, 0)) // MOVQ DX, (SP)
|
||||
self.Emit("MOVQ", _SI, jit.Ptr(_SP, 8)) // MOVQ SI, 8(SP)
|
||||
self.Emit("MOVQ", _R8, jit.Ptr(_SP, 16)) // MOVQ R9, 16(SP)
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 24)) // MOVQ AX, 24(SP)
|
||||
self.call_go(_F_mapassign_faststr) // CALL_GO runtime.mapassign_faststr
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 32), _AX) // MOVQ 32(SP), AX
|
||||
|
||||
/* add to the pointer stack */
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.WritePtrAX(6, jit.Sib(_ST, _CX, 8, _ST_Vp), false) // MOVQ AX, ST.Vp[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* allocate memory to store the string header and unquoted result */
|
||||
self.Link("_unquote") // _unquote:
|
||||
self.Emit("ADDQ", jit.Imm(15), _AX) // ADDQ $15, AX
|
||||
self.Emit("MOVQ", _T_byte, _CX) // MOVQ _T_byte, CX
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 0)) // MOVQ AX, (SP)
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_SP, 8)) // MOVQ CX, 8(SP)
|
||||
self.Emit("MOVB", jit.Imm(0), jit.Ptr(_SP, 16)) // MOVB $0, 16(SP)
|
||||
self.call_go(_F_mallocgc) // CALL_GO runtime.mallocgc
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 24), _R9) // MOVQ 24(SP), R9
|
||||
|
||||
/* prepare the unquoting parameters */
|
||||
self.Emit("MOVQ" , _VAR_ss_Iv, _CX) // MOVQ ss.Iv, CX
|
||||
self.Emit("LEAQ" , jit.Sib(_IP, _CX, 1, 0), _DI) // LEAQ (IP)(CX), DI
|
||||
self.Emit("NEGQ" , _CX) // NEGQ CX
|
||||
self.Emit("LEAQ" , jit.Sib(_IC, _CX, 1, -1), _SI) // LEAQ -1(IC)(CX), SI
|
||||
self.Emit("LEAQ" , jit.Ptr(_R9, 16), _DX) // LEAQ 16(R8), DX
|
||||
self.Emit("LEAQ" , _VAR_ss_Ep, _CX) // LEAQ ss.Ep, CX
|
||||
self.Emit("XORL" , _R8, _R8) // XORL R8, R8
|
||||
self.Emit("BTQ" , jit.Imm(_F_disable_urc), _VAR_df) // BTQ ${_F_disable_urc}, fv
|
||||
self.Emit("SETCC", _R8) // SETCC R8
|
||||
self.Emit("SHLQ" , jit.Imm(types.B_UNICODE_REPLACE), _R8) // SHLQ ${types.B_UNICODE_REPLACE}, R8
|
||||
|
||||
/* unquote the string, with R9 been preserved */
|
||||
self.save(_R9) // SAVE R9
|
||||
self.call(_F_unquote) // CALL unquote
|
||||
self.load(_R9) // LOAD R9
|
||||
|
||||
/* check for errors */
|
||||
self.Emit("TESTQ", _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JS" , "_unquote_error") // JS _unquote_error
|
||||
self.Emit("MOVL" , jit.Imm(1), _DX) // MOVL $1, DX
|
||||
self.Emit("LEAQ" , jit.Ptr(_R9, 16), _R8) // ADDQ $16, R8
|
||||
self.Emit("MOVQ" , _R8, jit.Ptr(_R9, 0)) // MOVQ R8, (R9)
|
||||
self.Emit("MOVQ" , _AX, jit.Ptr(_R9, 8)) // MOVQ AX, 8(R9)
|
||||
self.Sjmp("JMP" , "_noescape") // JMP _noescape
|
||||
|
||||
/** V_DOUBLE **/
|
||||
self.Link("_decode_V_DOUBLE") // _decode_V_DOUBLE:
|
||||
self.Emit("BTQ" , jit.Imm(_F_use_number), _VAR_df) // BTQ _F_use_number, df
|
||||
self.Sjmp("JC" , "_use_number") // JC _use_number
|
||||
self.Emit("MOVSD", _VAR_ss_Dv, _X0) // MOVSD ss.Dv, X0
|
||||
self.Sjmp("JMP" , "_use_float64") // JMP _use_float64
|
||||
|
||||
/** V_INTEGER **/
|
||||
self.Link("_decode_V_INTEGER") // _decode_V_INTEGER:
|
||||
self.Emit("BTQ" , jit.Imm(_F_use_number), _VAR_df) // BTQ _F_use_number, df
|
||||
self.Sjmp("JC" , "_use_number") // JC _use_number
|
||||
self.Emit("BTQ" , jit.Imm(_F_use_int64), _VAR_df) // BTQ _F_use_int64, df
|
||||
self.Sjmp("JC" , "_use_int64") // JC _use_int64
|
||||
self.Emit("MOVQ" , _VAR_ss_Iv, _AX) // MOVQ ss.Iv, AX
|
||||
self.Emit("CVTSQ2SD", _AX, _X0) // CVTSQ2SD AX, X0
|
||||
|
||||
/* represent numbers as `float64` */
|
||||
self.Link("_use_float64") // _use_float64:
|
||||
self.Emit("MOVSD", _X0, jit.Ptr(_SP, 0)) // MOVSD X0, (SP)
|
||||
self.call_go(_F_convT64) // CALL_GO runtime.convT64
|
||||
self.Emit("MOVQ" , _T_float64, _R8) // MOVQ _T_float64, R8
|
||||
self.Emit("MOVQ" , jit.Ptr(_SP, 8), _R9) // MOVQ 8(SP), R9
|
||||
self.Emit("MOVQ" , _VAR_ss_Ep, _DI) // MOVQ ss.Ep, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/* represent numbers as `json.Number` */
|
||||
self.Link("_use_number") // _use_number
|
||||
self.Emit("MOVQ", _VAR_ss_Ep, _AX) // MOVQ ss.Ep, AX
|
||||
self.Emit("LEAQ", jit.Sib(_IP, _AX, 1, 0), _SI) // LEAQ (IP)(AX), SI
|
||||
self.Emit("MOVQ", _IC, _CX) // MOVQ IC, CX
|
||||
self.Emit("SUBQ", _AX, _CX) // SUBQ AX, CX
|
||||
self.Emit("MOVQ", _SI, jit.Ptr(_SP, 0)) // MOVQ SI, (SP)
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_SP, 8)) // MOVQ CX, 8(SP)
|
||||
self.call_go(_F_convTstring) // CALL_GO runtime.convTstring
|
||||
self.Emit("MOVQ", _T_number, _R8) // MOVQ _T_number, R8
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 16), _R9) // MOVQ 16(SP), R9
|
||||
self.Emit("MOVQ", _VAR_ss_Ep, _DI) // MOVQ ss.Ep, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/* represent numbers as `int64` */
|
||||
self.Link("_use_int64") // _use_int64:
|
||||
self.Emit("MOVQ", _VAR_ss_Iv, _AX) // MOVQ ss.Iv, AX
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 0)) // MOVQ AX, (SP)
|
||||
self.call_go(_F_convT64) // CALL_GO runtime.convT64
|
||||
self.Emit("MOVQ", _T_int64, _R8) // MOVQ _T_int64, R8
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 8), _R9) // MOVQ 8(SP), R9
|
||||
self.Emit("MOVQ", _VAR_ss_Ep, _DI) // MOVQ ss.Ep, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_KEY_SEP **/
|
||||
self.Link("_decode_V_KEY_SEP") // _decode_V_KEY_SEP:
|
||||
// self.Byte(0xcc)
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("CMPQ", _AX, jit.Imm(_S_obj_delim)) // CMPQ AX, _S_obj_delim
|
||||
self.Sjmp("JNE" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("MOVQ", jit.Imm(_S_val), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_val, ST.Vt[CX]
|
||||
self.Emit("MOVQ", jit.Imm(_S_obj), jit.Sib(_ST, _CX, 8, _ST_Vt - 8)) // MOVQ _S_obj, ST.Vt[CX - 1]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_ELEM_SEP **/
|
||||
self.Link("_decode_V_ELEM_SEP") // _decode_V_ELEM_SEP:
|
||||
self.Emit("MOVQ" , jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ" , jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(_S_arr)) // CMPQ _AX, _S_arr
|
||||
self.Sjmp("JE" , "_array_sep") // JZ _next
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(_S_obj)) // CMPQ _AX, _S_arr
|
||||
self.Sjmp("JNE" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("MOVQ" , jit.Imm(_S_obj_sep), jit.Sib(_ST, _CX, 8, _ST_Vt))
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* arrays */
|
||||
self.Link("_array_sep")
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _DX) // MOVQ 8(SI), DX
|
||||
self.Emit("CMPQ", _DX, jit.Ptr(_SI, 16)) // CMPQ DX, 16(SI)
|
||||
self.Sjmp("JAE" , "_array_more") // JAE _array_more
|
||||
|
||||
/* add a slot for the new element */
|
||||
self.Link("_array_append") // _array_append:
|
||||
self.Emit("ADDQ", jit.Imm(1), jit.Ptr(_SI, 8)) // ADDQ $1, 8(SI)
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 0), _SI) // MOVQ (SI), SI
|
||||
self.Emit("ADDQ", jit.Imm(1), _CX) // ADDQ $1, CX
|
||||
self.Emit("CMPQ", _CX, jit.Imm(types.MAX_RECURSE)) // CMPQ CX, ${types.MAX_RECURSE}
|
||||
self.Sjmp("JAE" , "_stack_overflow")
|
||||
self.Emit("SHLQ", jit.Imm(1), _DX) // SHLQ $1, DX
|
||||
self.Emit("LEAQ", jit.Sib(_SI, _DX, 8, 0), _SI) // LEAQ (SI)(DX*8), SI
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.WriteRecNotAX(7 , _SI, jit.Sib(_ST, _CX, 8, _ST_Vp), false) // MOVQ SI, ST.Vp[CX]
|
||||
self.Emit("MOVQ", jit.Imm(_S_val), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_val, ST.Vt[CX}
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_ARRAY_END **/
|
||||
self.Link("_decode_V_ARRAY_END") // _decode_V_ARRAY_END:
|
||||
self.Emit("XORL", _DX, _DX) // XORL DX, DX
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("CMPQ", _AX, jit.Imm(_S_arr_0)) // CMPQ AX, _S_arr_0
|
||||
self.Sjmp("JE" , "_first_item") // JE _first_item
|
||||
self.Emit("CMPQ", _AX, jit.Imm(_S_arr)) // CMPQ AX, _S_arr
|
||||
self.Sjmp("JNE" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("SUBQ", jit.Imm(1), jit.Ptr(_ST, _ST_Sp)) // SUBQ $1, ST.Sp
|
||||
self.Emit("MOVQ", _DX, jit.Sib(_ST, _CX, 8, _ST_Vp)) // MOVQ DX, ST.Vp[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* first element of an array */
|
||||
self.Link("_first_item") // _first_item:
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("SUBQ", jit.Imm(2), jit.Ptr(_ST, _ST_Sp)) // SUBQ $2, ST.Sp
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp - 8), _SI) // MOVQ ST.Vp[CX - 1], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
self.Emit("MOVQ", _DX, jit.Sib(_ST, _CX, 8, _ST_Vp - 8)) // MOVQ DX, ST.Vp[CX - 1]
|
||||
self.Emit("MOVQ", _DX, jit.Sib(_ST, _CX, 8, _ST_Vp)) // MOVQ DX, ST.Vp[CX]
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SI, 8)) // MOVQ DX, 8(SI)
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_OBJECT_END **/
|
||||
self.Link("_decode_V_OBJECT_END") // _decode_V_OBJECT_END:
|
||||
self.Emit("MOVL", jit.Imm(_S_omask_end), _DX) // MOVL _S_omask, DI
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DX)
|
||||
self.Sjmp("JNC" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("XORL", _AX, _AX) // XORL AX, AX
|
||||
self.Emit("SUBQ", jit.Imm(1), jit.Ptr(_ST, _ST_Sp)) // SUBQ $1, ST.Sp
|
||||
self.Emit("MOVQ", _AX, jit.Sib(_ST, _CX, 8, _ST_Vp)) // MOVQ AX, ST.Vp[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* return from decoder */
|
||||
self.Link("_return") // _return:
|
||||
self.Emit("XORL", _EP, _EP) // XORL EP, EP
|
||||
self.Emit("MOVQ", _EP, jit.Ptr(_ST, _ST_Vp)) // MOVQ EP, ST.Vp[0]
|
||||
self.Link("_epilogue") // _epilogue:
|
||||
self.Emit("SUBQ", jit.Imm(_FsmOffset), _ST) // SUBQ _FsmOffset, _ST
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, _VD_offs), _BP) // MOVQ _VD_offs(SP), BP
|
||||
self.Emit("ADDQ", jit.Imm(_VD_size), _SP) // ADDQ $_VD_size, SP
|
||||
self.Emit("RET") // RET
|
||||
|
||||
/* array expand */
|
||||
self.Link("_array_more") // _array_more:
|
||||
self.Emit("MOVQ" , _T_eface, _AX) // MOVQ _T_eface, AX
|
||||
self.Emit("MOVOU", jit.Ptr(_SI, 0), _X0) // MOVOU (SI), X0
|
||||
self.Emit("MOVQ" , jit.Ptr(_SI, 16), _DX) // MOVQ 16(SI), DX
|
||||
self.Emit("MOVQ" , _AX, jit.Ptr(_SP, 0)) // MOVQ AX, (SP)
|
||||
self.Emit("MOVOU", _X0, jit.Ptr(_SP, 8)) // MOVOU X0, 8(SP)
|
||||
self.Emit("MOVQ" , _DX, jit.Ptr(_SP, 24)) // MOVQ DX, 24(SP)
|
||||
self.Emit("SHLQ" , jit.Imm(1), _DX) // SHLQ $1, DX
|
||||
self.Emit("MOVQ" , _DX, jit.Ptr(_SP, 32)) // MOVQ DX, 32(SP)
|
||||
self.call_go(_F_growslice) // CALL_GO runtime.growslice
|
||||
self.Emit("MOVQ" , jit.Ptr(_SP, 40), _DI) // MOVOU 40(SP), DI
|
||||
self.Emit("MOVQ" , jit.Ptr(_SP, 48), _DX) // MOVOU 48(SP), DX
|
||||
self.Emit("MOVQ" , jit.Ptr(_SP, 56), _AX) // MOVQ 56(SP), AX
|
||||
|
||||
/* update the slice */
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SI, 8)) // MOVQ DX, 8(SI)
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SI, 16)) // MOVQ AX, 16(AX)
|
||||
self.WriteRecNotAX(8 , _DI, jit.Ptr(_SI, 0), false) // MOVQ R10, (SI)
|
||||
self.Sjmp("JMP" , "_array_append") // JMP _array_append
|
||||
|
||||
/* copy string */
|
||||
self.Link("copy_string") // pointer: R8, length: AX, return addr: DI
|
||||
// self.Byte(0xcc)
|
||||
self.Emit("MOVQ", _R8, _VAR_cs_p)
|
||||
self.Emit("MOVQ", _AX, _VAR_cs_n)
|
||||
self.Emit("MOVQ", _DI, _VAR_cs_LR)
|
||||
self.Emit("MOVQ", _T_byte, jit.Ptr(_SP, 0))
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 8))
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 16))
|
||||
self.call_go(_F_makeslice)
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, 24), _R8)
|
||||
self.Emit("MOVQ", _R8, _VAR_cs_d)
|
||||
self.Emit("MOVQ", _R8, jit.Ptr(_SP, 0))
|
||||
self.Emit("MOVQ", _VAR_cs_p, _R8)
|
||||
self.Emit("MOVQ", _R8, jit.Ptr(_SP, 8))
|
||||
self.Emit("MOVQ", _VAR_cs_n, _AX)
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 16))
|
||||
self.call_go(_F_memmove)
|
||||
self.Emit("MOVQ", _VAR_cs_d, _R8)
|
||||
self.Emit("MOVQ", _VAR_cs_n, _AX)
|
||||
self.Emit("MOVQ", _VAR_cs_LR, _DI)
|
||||
// self.Byte(0xcc)
|
||||
self.Rjmp("JMP", _DI)
|
||||
|
||||
/* error handlers */
|
||||
self.Link("_stack_overflow")
|
||||
self.Emit("MOVL" , _E_recurse, _EP) // MOVQ _E_recurse, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
self.Link("_vtype_error") // _vtype_error:
|
||||
self.Emit("MOVQ" , _DI, _IC) // MOVQ DI, IC
|
||||
self.Emit("MOVL" , _E_invalid, _EP) // MOVL _E_invalid, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
self.Link("_invalid_char") // _invalid_char:
|
||||
self.Emit("SUBQ" , jit.Imm(1), _IC) // SUBQ $1, IC
|
||||
self.Emit("MOVL" , _E_invalid, _EP) // MOVL _E_invalid, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
self.Link("_unquote_error") // _unquote_error:
|
||||
self.Emit("MOVQ" , _VAR_ss_Iv, _IC) // MOVQ ss.Iv, IC
|
||||
self.Emit("SUBQ" , jit.Imm(1), _IC) // SUBQ $1, IC
|
||||
self.Link("_parsing_error") // _parsing_error:
|
||||
self.Emit("NEGQ" , _AX) // NEGQ AX
|
||||
self.Emit("MOVQ" , _AX, _EP) // MOVQ AX, EP
|
||||
self.Link("_error") // _error:
|
||||
self.Emit("PXOR" , _X0, _X0) // PXOR X0, X0
|
||||
self.Emit("MOVOU", _X0, jit.Ptr(_VP, 0)) // MOVOU X0, (VP)
|
||||
self.Sjmp("JMP" , "_epilogue") // JMP _epilogue
|
||||
|
||||
/* invalid value type, never returns */
|
||||
self.Link("_invalid_vtype")
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SP, 0)) // MOVQ AX, (SP)
|
||||
self.call(_F_invalid_vtype) // CALL invalid_type
|
||||
self.Emit("UD2") // UD2
|
||||
|
||||
/* switch jump table */
|
||||
self.Link("_switch_table") // _switch_table:
|
||||
self.Sref("_decode_V_EOF", 0) // SREF &_decode_V_EOF, $0
|
||||
self.Sref("_decode_V_NULL", -4) // SREF &_decode_V_NULL, $-4
|
||||
self.Sref("_decode_V_TRUE", -8) // SREF &_decode_V_TRUE, $-8
|
||||
self.Sref("_decode_V_FALSE", -12) // SREF &_decode_V_FALSE, $-12
|
||||
self.Sref("_decode_V_ARRAY", -16) // SREF &_decode_V_ARRAY, $-16
|
||||
self.Sref("_decode_V_OBJECT", -20) // SREF &_decode_V_OBJECT, $-20
|
||||
self.Sref("_decode_V_STRING", -24) // SREF &_decode_V_STRING, $-24
|
||||
self.Sref("_decode_V_DOUBLE", -28) // SREF &_decode_V_DOUBLE, $-28
|
||||
self.Sref("_decode_V_INTEGER", -32) // SREF &_decode_V_INTEGER, $-32
|
||||
self.Sref("_decode_V_KEY_SEP", -36) // SREF &_decode_V_KEY_SEP, $-36
|
||||
self.Sref("_decode_V_ELEM_SEP", -40) // SREF &_decode_V_ELEM_SEP, $-40
|
||||
self.Sref("_decode_V_ARRAY_END", -44) // SREF &_decode_V_ARRAY_END, $-44
|
||||
self.Sref("_decode_V_OBJECT_END", -48) // SREF &_decode_V_OBJECT_END, $-48
|
||||
|
||||
/* fast character lookup table */
|
||||
self.Link("_decode_tab") // _decode_tab:
|
||||
self.Sref("_decode_V_EOF", 0) // SREF &_decode_V_EOF, $0
|
||||
|
||||
/* generate rest of the tabs */
|
||||
for i := 1; i < 256; i++ {
|
||||
if to, ok := _R_tab[i]; ok {
|
||||
self.Sref(to, -int64(i) * 4)
|
||||
} else {
|
||||
self.Byte(0x00, 0x00, 0x00, 0x00)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) WritePtrAX(i int, rec obj.Addr, saveDI bool) {
|
||||
self.Emit("MOVQ", _V_writeBarrier, _R10)
|
||||
self.Emit("CMPL", jit.Ptr(_R10, 0), jit.Imm(0))
|
||||
self.Sjmp("JE", "_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
if saveDI {
|
||||
self.save(_DI)
|
||||
}
|
||||
self.Emit("LEAQ", rec, _DI)
|
||||
self.Emit("MOVQ", _F_gcWriteBarrierAX, _R10) // MOVQ ${fn}, AX
|
||||
self.Rjmp("CALL", _R10)
|
||||
if saveDI {
|
||||
self.load(_DI)
|
||||
}
|
||||
self.Sjmp("JMP", "_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Link("_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Emit("MOVQ", _AX, rec)
|
||||
self.Link("_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) WriteRecNotAX(i int, ptr obj.Addr, rec obj.Addr, saveDI bool) {
|
||||
if rec.Reg == x86.REG_AX || rec.Index == x86.REG_AX {
|
||||
panic("rec contains AX!")
|
||||
}
|
||||
self.Emit("MOVQ", _V_writeBarrier, _R10)
|
||||
self.Emit("CMPL", jit.Ptr(_R10, 0), jit.Imm(0))
|
||||
self.Sjmp("JE", "_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Emit("MOVQ", ptr, _AX)
|
||||
if saveDI {
|
||||
self.save(_DI)
|
||||
}
|
||||
self.Emit("LEAQ", rec, _DI)
|
||||
self.Emit("MOVQ", _F_gcWriteBarrierAX, _R10) // MOVQ ${fn}, AX
|
||||
self.Rjmp("CALL", _R10)
|
||||
if saveDI {
|
||||
self.load(_DI)
|
||||
}
|
||||
self.Sjmp("JMP", "_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Link("_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Emit("MOVQ", ptr, rec)
|
||||
self.Link("_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
}
|
||||
|
||||
/** Generic Decoder **/
|
||||
|
||||
var (
|
||||
_subr_decode_value = new(_ValueDecoder).build()
|
||||
)
|
||||
|
||||
//go:nosplit
|
||||
func invalid_vtype(vt types.ValueType) {
|
||||
throw(fmt.Sprintf("invalid value type: %d", vt))
|
||||
}
|
||||
772
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_go117.go
generated
vendored
Normal file
772
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_go117.go
generated
vendored
Normal file
@@ -0,0 +1,772 @@
|
||||
//go:build go1.17 && !go1.21
|
||||
// +build go1.17,!go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`encoding/json`
|
||||
`fmt`
|
||||
`reflect`
|
||||
`strconv`
|
||||
|
||||
`github.com/bytedance/sonic/internal/jit`
|
||||
`github.com/bytedance/sonic/internal/native`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/twitchyliquid64/golang-asm/obj`
|
||||
`github.com/twitchyliquid64/golang-asm/obj/x86`
|
||||
)
|
||||
|
||||
/** Crucial Registers:
|
||||
*
|
||||
* ST(R13) && 0(SP) : ro, decoder stack
|
||||
* DF(AX) : ro, decoder flags
|
||||
* EP(BX) : wo, error pointer
|
||||
* IP(R10) : ro, input pointer
|
||||
* IL(R12) : ro, input length
|
||||
* IC(R11) : rw, input cursor
|
||||
* VP(R15) : ro, value pointer (to an interface{})
|
||||
*/
|
||||
|
||||
const (
|
||||
_VD_args = 8 // 8 bytes for passing arguments to this functions
|
||||
_VD_fargs = 64 // 64 bytes for passing arguments to other Go functions
|
||||
_VD_saves = 48 // 48 bytes for saving the registers before CALL instructions
|
||||
_VD_locals = 96 // 96 bytes for local variables
|
||||
)
|
||||
|
||||
const (
|
||||
_VD_offs = _VD_fargs + _VD_saves + _VD_locals
|
||||
_VD_size = _VD_offs + 8 // 8 bytes for the parent frame pointer
|
||||
)
|
||||
|
||||
var (
|
||||
_VAR_ss = _VAR_ss_Vt
|
||||
_VAR_df = jit.Ptr(_SP, _VD_fargs + _VD_saves)
|
||||
)
|
||||
|
||||
var (
|
||||
_VAR_ss_Vt = jit.Ptr(_SP, _VD_fargs + _VD_saves + 8)
|
||||
_VAR_ss_Dv = jit.Ptr(_SP, _VD_fargs + _VD_saves + 16)
|
||||
_VAR_ss_Iv = jit.Ptr(_SP, _VD_fargs + _VD_saves + 24)
|
||||
_VAR_ss_Ep = jit.Ptr(_SP, _VD_fargs + _VD_saves + 32)
|
||||
_VAR_ss_Db = jit.Ptr(_SP, _VD_fargs + _VD_saves + 40)
|
||||
_VAR_ss_Dc = jit.Ptr(_SP, _VD_fargs + _VD_saves + 48)
|
||||
)
|
||||
|
||||
var (
|
||||
_VAR_R9 = jit.Ptr(_SP, _VD_fargs + _VD_saves + 56)
|
||||
)
|
||||
type _ValueDecoder struct {
|
||||
jit.BaseAssembler
|
||||
}
|
||||
|
||||
var (
|
||||
_VAR_cs_LR = jit.Ptr(_SP, _VD_fargs + _VD_saves + 64)
|
||||
_VAR_cs_p = jit.Ptr(_SP, _VD_fargs + _VD_saves + 72)
|
||||
_VAR_cs_n = jit.Ptr(_SP, _VD_fargs + _VD_saves + 80)
|
||||
_VAR_cs_d = jit.Ptr(_SP, _VD_fargs + _VD_saves + 88)
|
||||
)
|
||||
|
||||
func (self *_ValueDecoder) build() uintptr {
|
||||
self.Init(self.compile)
|
||||
return *(*uintptr)(self.Load("decode_value", _VD_size, _VD_args, argPtrs_generic, localPtrs_generic))
|
||||
}
|
||||
|
||||
/** Function Calling Helpers **/
|
||||
|
||||
func (self *_ValueDecoder) save(r ...obj.Addr) {
|
||||
for i, v := range r {
|
||||
if i > _VD_saves / 8 - 1 {
|
||||
panic("too many registers to save")
|
||||
} else {
|
||||
self.Emit("MOVQ", v, jit.Ptr(_SP, _VD_fargs + int64(i) * 8))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) load(r ...obj.Addr) {
|
||||
for i, v := range r {
|
||||
if i > _VD_saves / 8 - 1 {
|
||||
panic("too many registers to load")
|
||||
} else {
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, _VD_fargs + int64(i) * 8), v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) call(fn obj.Addr) {
|
||||
self.Emit("MOVQ", fn, _R9) // MOVQ ${fn}, AX
|
||||
self.Rjmp("CALL", _R9) // CALL AX
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) call_go(fn obj.Addr) {
|
||||
self.save(_REG_go...) // SAVE $REG_go
|
||||
self.call(fn) // CALL ${fn}
|
||||
self.load(_REG_go...) // LOAD $REG_go
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) callc(fn obj.Addr) {
|
||||
self.Emit("XCHGQ", _IP, _BP)
|
||||
self.call(fn)
|
||||
self.Emit("XCHGQ", _IP, _BP)
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) call_c(fn obj.Addr) {
|
||||
self.Emit("XCHGQ", _IC, _BX)
|
||||
self.callc(fn)
|
||||
self.Emit("XCHGQ", _IC, _BX)
|
||||
}
|
||||
|
||||
/** Decoder Assembler **/
|
||||
|
||||
const (
|
||||
_S_val = iota + 1
|
||||
_S_arr
|
||||
_S_arr_0
|
||||
_S_obj
|
||||
_S_obj_0
|
||||
_S_obj_delim
|
||||
_S_obj_sep
|
||||
)
|
||||
|
||||
const (
|
||||
_S_omask_key = (1 << _S_obj_0) | (1 << _S_obj_sep)
|
||||
_S_omask_end = (1 << _S_obj_0) | (1 << _S_obj)
|
||||
_S_vmask = (1 << _S_val) | (1 << _S_arr_0)
|
||||
)
|
||||
|
||||
const (
|
||||
_A_init_len = 1
|
||||
_A_init_cap = 16
|
||||
)
|
||||
|
||||
const (
|
||||
_ST_Sp = 0
|
||||
_ST_Vt = _PtrBytes
|
||||
_ST_Vp = _PtrBytes * (types.MAX_RECURSE + 1)
|
||||
)
|
||||
|
||||
var (
|
||||
_V_true = jit.Imm(int64(pbool(true)))
|
||||
_V_false = jit.Imm(int64(pbool(false)))
|
||||
_F_value = jit.Imm(int64(native.S_value))
|
||||
)
|
||||
|
||||
var (
|
||||
_V_max = jit.Imm(int64(types.V_MAX))
|
||||
_E_eof = jit.Imm(int64(types.ERR_EOF))
|
||||
_E_invalid = jit.Imm(int64(types.ERR_INVALID_CHAR))
|
||||
_E_recurse = jit.Imm(int64(types.ERR_RECURSE_EXCEED_MAX))
|
||||
)
|
||||
|
||||
var (
|
||||
_F_convTslice = jit.Func(convTslice)
|
||||
_F_convTstring = jit.Func(convTstring)
|
||||
_F_invalid_vtype = jit.Func(invalid_vtype)
|
||||
)
|
||||
|
||||
var (
|
||||
_T_map = jit.Type(reflect.TypeOf((map[string]interface{})(nil)))
|
||||
_T_bool = jit.Type(reflect.TypeOf(false))
|
||||
_T_int64 = jit.Type(reflect.TypeOf(int64(0)))
|
||||
_T_eface = jit.Type(reflect.TypeOf((*interface{})(nil)).Elem())
|
||||
_T_slice = jit.Type(reflect.TypeOf(([]interface{})(nil)))
|
||||
_T_string = jit.Type(reflect.TypeOf(""))
|
||||
_T_number = jit.Type(reflect.TypeOf(json.Number("")))
|
||||
_T_float64 = jit.Type(reflect.TypeOf(float64(0)))
|
||||
)
|
||||
|
||||
var _R_tab = map[int]string {
|
||||
'[': "_decode_V_ARRAY",
|
||||
'{': "_decode_V_OBJECT",
|
||||
':': "_decode_V_KEY_SEP",
|
||||
',': "_decode_V_ELEM_SEP",
|
||||
']': "_decode_V_ARRAY_END",
|
||||
'}': "_decode_V_OBJECT_END",
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) compile() {
|
||||
self.Emit("SUBQ", jit.Imm(_VD_size), _SP) // SUBQ $_VD_size, SP
|
||||
self.Emit("MOVQ", _BP, jit.Ptr(_SP, _VD_offs)) // MOVQ BP, _VD_offs(SP)
|
||||
self.Emit("LEAQ", jit.Ptr(_SP, _VD_offs), _BP) // LEAQ _VD_offs(SP), BP
|
||||
|
||||
/* initialize the state machine */
|
||||
self.Emit("XORL", _CX, _CX) // XORL CX, CX
|
||||
self.Emit("MOVQ", _DF, _VAR_df) // MOVQ DF, df
|
||||
/* initialize digital buffer first */
|
||||
self.Emit("MOVQ", jit.Imm(_MaxDigitNums), _VAR_ss_Dc) // MOVQ $_MaxDigitNums, ss.Dcap
|
||||
self.Emit("LEAQ", jit.Ptr(_ST, _DbufOffset), _AX) // LEAQ _DbufOffset(ST), AX
|
||||
self.Emit("MOVQ", _AX, _VAR_ss_Db) // MOVQ AX, ss.Dbuf
|
||||
/* add ST offset */
|
||||
self.Emit("ADDQ", jit.Imm(_FsmOffset), _ST) // ADDQ _FsmOffset, _ST
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.WriteRecNotAX(0, _VP, jit.Ptr(_ST, _ST_Vp), false) // MOVQ VP, ST.Vp[0]
|
||||
self.Emit("MOVQ", jit.Imm(_S_val), jit.Ptr(_ST, _ST_Vt)) // MOVQ _S_val, ST.Vt[0]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* set the value from previous round */
|
||||
self.Link("_set_value") // _set_value:
|
||||
self.Emit("MOVL" , jit.Imm(_S_vmask), _DX) // MOVL _S_vmask, DX
|
||||
self.Emit("MOVQ" , jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ" , jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ AX, DX
|
||||
self.Sjmp("JNC" , "_vtype_error") // JNC _vtype_error
|
||||
self.Emit("XORL" , _SI, _SI) // XORL SI, SI
|
||||
self.Emit("SUBQ" , jit.Imm(1), jit.Ptr(_ST, _ST_Sp)) // SUBQ $1, ST.Sp
|
||||
self.Emit("XCHGQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // XCHGQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ" , _R8, jit.Ptr(_SI, 0)) // MOVQ R8, (SI)
|
||||
self.WriteRecNotAX(1, _R9, jit.Ptr(_SI, 8), false) // MOVQ R9, 8(SI)
|
||||
|
||||
/* check for value stack */
|
||||
self.Link("_next") // _next:
|
||||
self.Emit("MOVQ" , jit.Ptr(_ST, _ST_Sp), _AX) // MOVQ ST.Sp, AX
|
||||
self.Emit("TESTQ", _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JS" , "_return") // JS _return
|
||||
|
||||
/* fast path: test up to 4 characters manually */
|
||||
self.Emit("CMPQ" , _IC, _IL) // CMPQ IC, IL
|
||||
self.Sjmp("JAE" , "_decode_V_EOF") // JAE _decode_V_EOF
|
||||
self.Emit("MOVBQZX", jit.Sib(_IP, _IC, 1, 0), _AX) // MOVBQZX (IP)(IC), AX
|
||||
self.Emit("MOVQ" , jit.Imm(_BM_space), _DX) // MOVQ _BM_space, DX
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(' ')) // CMPQ AX, $' '
|
||||
self.Sjmp("JA" , "_decode_fast") // JA _decode_fast
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ _AX, _DX
|
||||
self.Sjmp("JNC" , "_decode_fast") // JNC _decode_fast
|
||||
self.Emit("ADDQ" , jit.Imm(1), _IC) // ADDQ $1, IC
|
||||
|
||||
/* at least 1 to 3 spaces */
|
||||
for i := 0; i < 3; i++ {
|
||||
self.Emit("CMPQ" , _IC, _IL) // CMPQ IC, IL
|
||||
self.Sjmp("JAE" , "_decode_V_EOF") // JAE _decode_V_EOF
|
||||
self.Emit("MOVBQZX", jit.Sib(_IP, _IC, 1, 0), _AX) // MOVBQZX (IP)(IC), AX
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(' ')) // CMPQ AX, $' '
|
||||
self.Sjmp("JA" , "_decode_fast") // JA _decode_fast
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ _AX, _DX
|
||||
self.Sjmp("JNC" , "_decode_fast") // JNC _decode_fast
|
||||
self.Emit("ADDQ" , jit.Imm(1), _IC) // ADDQ $1, IC
|
||||
}
|
||||
|
||||
/* at least 4 spaces */
|
||||
self.Emit("CMPQ" , _IC, _IL) // CMPQ IC, IL
|
||||
self.Sjmp("JAE" , "_decode_V_EOF") // JAE _decode_V_EOF
|
||||
self.Emit("MOVBQZX", jit.Sib(_IP, _IC, 1, 0), _AX) // MOVBQZX (IP)(IC), AX
|
||||
|
||||
/* fast path: use lookup table to select decoder */
|
||||
self.Link("_decode_fast") // _decode_fast:
|
||||
self.Byte(0x48, 0x8d, 0x3d) // LEAQ ?(PC), DI
|
||||
self.Sref("_decode_tab", 4) // .... &_decode_tab
|
||||
self.Emit("MOVLQSX", jit.Sib(_DI, _AX, 4, 0), _AX) // MOVLQSX (DI)(AX*4), AX
|
||||
self.Emit("TESTQ" , _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JZ" , "_decode_native") // JZ _decode_native
|
||||
self.Emit("ADDQ" , jit.Imm(1), _IC) // ADDQ $1, IC
|
||||
self.Emit("ADDQ" , _DI, _AX) // ADDQ DI, AX
|
||||
self.Rjmp("JMP" , _AX) // JMP AX
|
||||
|
||||
/* decode with native decoder */
|
||||
self.Link("_decode_native") // _decode_native:
|
||||
self.Emit("MOVQ", _IP, _DI) // MOVQ IP, DI
|
||||
self.Emit("MOVQ", _IL, _SI) // MOVQ IL, SI
|
||||
self.Emit("MOVQ", _IC, _DX) // MOVQ IC, DX
|
||||
self.Emit("LEAQ", _VAR_ss, _CX) // LEAQ ss, CX
|
||||
self.Emit("MOVQ", _VAR_df, _R8) // MOVQ $df, R8
|
||||
self.Emit("BTSQ", jit.Imm(_F_allow_control), _R8) // ANDQ $1<<_F_allow_control, R8
|
||||
self.callc(_F_value) // CALL value
|
||||
self.Emit("MOVQ", _AX, _IC) // MOVQ AX, IC
|
||||
|
||||
/* check for errors */
|
||||
self.Emit("MOVQ" , _VAR_ss_Vt, _AX) // MOVQ ss.Vt, AX
|
||||
self.Emit("TESTQ", _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JS" , "_parsing_error")
|
||||
self.Sjmp("JZ" , "_invalid_vtype") // JZ _invalid_vtype
|
||||
self.Emit("CMPQ" , _AX, _V_max) // CMPQ AX, _V_max
|
||||
self.Sjmp("JA" , "_invalid_vtype") // JA _invalid_vtype
|
||||
|
||||
/* jump table selector */
|
||||
self.Byte(0x48, 0x8d, 0x3d) // LEAQ ?(PC), DI
|
||||
self.Sref("_switch_table", 4) // .... &_switch_table
|
||||
self.Emit("MOVLQSX", jit.Sib(_DI, _AX, 4, -4), _AX) // MOVLQSX -4(DI)(AX*4), AX
|
||||
self.Emit("ADDQ" , _DI, _AX) // ADDQ DI, AX
|
||||
self.Rjmp("JMP" , _AX) // JMP AX
|
||||
|
||||
/** V_EOF **/
|
||||
self.Link("_decode_V_EOF") // _decode_V_EOF:
|
||||
self.Emit("MOVL", _E_eof, _EP) // MOVL _E_eof, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
|
||||
/** V_NULL **/
|
||||
self.Link("_decode_V_NULL") // _decode_V_NULL:
|
||||
self.Emit("XORL", _R8, _R8) // XORL R8, R8
|
||||
self.Emit("XORL", _R9, _R9) // XORL R9, R9
|
||||
self.Emit("LEAQ", jit.Ptr(_IC, -4), _DI) // LEAQ -4(IC), DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_TRUE **/
|
||||
self.Link("_decode_V_TRUE") // _decode_V_TRUE:
|
||||
self.Emit("MOVQ", _T_bool, _R8) // MOVQ _T_bool, R8
|
||||
// TODO: maybe modified by users?
|
||||
self.Emit("MOVQ", _V_true, _R9) // MOVQ _V_true, R9
|
||||
self.Emit("LEAQ", jit.Ptr(_IC, -4), _DI) // LEAQ -4(IC), DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_FALSE **/
|
||||
self.Link("_decode_V_FALSE") // _decode_V_FALSE:
|
||||
self.Emit("MOVQ", _T_bool, _R8) // MOVQ _T_bool, R8
|
||||
self.Emit("MOVQ", _V_false, _R9) // MOVQ _V_false, R9
|
||||
self.Emit("LEAQ", jit.Ptr(_IC, -5), _DI) // LEAQ -5(IC), DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_ARRAY **/
|
||||
self.Link("_decode_V_ARRAY") // _decode_V_ARRAY
|
||||
self.Emit("MOVL", jit.Imm(_S_vmask), _DX) // MOVL _S_vmask, DX
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ AX, DX
|
||||
self.Sjmp("JNC" , "_invalid_char") // JNC _invalid_char
|
||||
|
||||
/* create a new array */
|
||||
self.Emit("MOVQ", _T_eface, _AX) // MOVQ _T_eface, AX
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_len), _BX) // MOVQ _A_init_len, BX
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_cap), _CX) // MOVQ _A_init_cap, CX
|
||||
self.call_go(_F_makeslice) // CALL_GO runtime.makeslice
|
||||
|
||||
/* pack into an interface */
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_len), _BX) // MOVQ _A_init_len, BX
|
||||
self.Emit("MOVQ", jit.Imm(_A_init_cap), _CX) // MOVQ _A_init_cap, CX
|
||||
self.call_go(_F_convTslice) // CALL_GO runtime.convTslice
|
||||
self.Emit("MOVQ", _AX, _R8) // MOVQ AX, R8
|
||||
|
||||
/* replace current state with an array */
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Imm(_S_arr), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_arr, ST.Vt[CX]
|
||||
self.Emit("MOVQ", _T_slice, _AX) // MOVQ _T_slice, AX
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SI, 0)) // MOVQ AX, (SI)
|
||||
self.WriteRecNotAX(2, _R8, jit.Ptr(_SI, 8), false) // MOVQ R8, 8(SI)
|
||||
|
||||
/* add a new slot for the first element */
|
||||
self.Emit("ADDQ", jit.Imm(1), _CX) // ADDQ $1, CX
|
||||
self.Emit("CMPQ", _CX, jit.Imm(types.MAX_RECURSE)) // CMPQ CX, ${types.MAX_RECURSE}
|
||||
self.Sjmp("JAE" , "_stack_overflow") // JA _stack_overflow
|
||||
self.Emit("MOVQ", jit.Ptr(_R8, 0), _AX) // MOVQ (R8), AX
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.WritePtrAX(3, jit.Sib(_ST, _CX, 8, _ST_Vp), false) // MOVQ AX, ST.Vp[CX]
|
||||
self.Emit("MOVQ", jit.Imm(_S_arr_0), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_arr_0, ST.Vt[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_OBJECT **/
|
||||
self.Link("_decode_V_OBJECT") // _decode_V_OBJECT:
|
||||
self.Emit("MOVL", jit.Imm(_S_vmask), _DX) // MOVL _S_vmask, DX
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DX) // BTQ AX, DX
|
||||
self.Sjmp("JNC" , "_invalid_char") // JNC _invalid_char
|
||||
self.call_go(_F_makemap_small) // CALL_GO runtime.makemap_small
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Imm(_S_obj_0), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_obj_0, ST.Vt[CX]
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", _T_map, _DX) // MOVQ _T_map, DX
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SI, 0)) // MOVQ DX, (SI)
|
||||
self.WritePtrAX(4, jit.Ptr(_SI, 8), false) // MOVQ AX, 8(SI)
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_STRING **/
|
||||
self.Link("_decode_V_STRING") // _decode_V_STRING:
|
||||
self.Emit("MOVQ", _VAR_ss_Iv, _CX) // MOVQ ss.Iv, CX
|
||||
self.Emit("MOVQ", _IC, _AX) // MOVQ IC, AX
|
||||
self.Emit("SUBQ", _CX, _AX) // SUBQ CX, AX
|
||||
|
||||
/* check for escapes */
|
||||
self.Emit("CMPQ", _VAR_ss_Ep, jit.Imm(-1)) // CMPQ ss.Ep, $-1
|
||||
self.Sjmp("JNE" , "_unquote") // JNE _unquote
|
||||
self.Emit("SUBQ", jit.Imm(1), _AX) // SUBQ $1, AX
|
||||
self.Emit("LEAQ", jit.Sib(_IP, _CX, 1, 0), _R8) // LEAQ (IP)(CX), R8
|
||||
self.Byte(0x48, 0x8d, 0x3d) // LEAQ (PC), DI
|
||||
self.Sref("_copy_string_end", 4)
|
||||
self.Emit("BTQ", jit.Imm(_F_copy_string), _VAR_df)
|
||||
self.Sjmp("JC", "copy_string")
|
||||
self.Link("_copy_string_end")
|
||||
self.Emit("XORL", _DX, _DX)
|
||||
|
||||
/* strings with no escape sequences */
|
||||
self.Link("_noescape") // _noescape:
|
||||
self.Emit("MOVL", jit.Imm(_S_omask_key), _DI) // MOVL _S_omask, DI
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _SI) // MOVQ ST.Vt[CX], SI
|
||||
self.Emit("BTQ" , _SI, _DI) // BTQ SI, DI
|
||||
self.Sjmp("JC" , "_object_key") // JC _object_key
|
||||
|
||||
/* check for pre-packed strings, avoid 1 allocation */
|
||||
self.Emit("TESTQ", _DX, _DX) // TESTQ DX, DX
|
||||
self.Sjmp("JNZ" , "_packed_str") // JNZ _packed_str
|
||||
self.Emit("MOVQ" , _AX, _BX) // MOVQ AX, BX
|
||||
self.Emit("MOVQ" , _R8, _AX) // MOVQ R8, AX
|
||||
self.call_go(_F_convTstring) // CALL_GO runtime.convTstring
|
||||
self.Emit("MOVQ" , _AX, _R9) // MOVQ AX, R9
|
||||
|
||||
/* packed string already in R9 */
|
||||
self.Link("_packed_str") // _packed_str:
|
||||
self.Emit("MOVQ", _T_string, _R8) // MOVQ _T_string, R8
|
||||
self.Emit("MOVQ", _VAR_ss_Iv, _DI) // MOVQ ss.Iv, DI
|
||||
self.Emit("SUBQ", jit.Imm(1), _DI) // SUBQ $1, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/* the string is an object key, get the map */
|
||||
self.Link("_object_key")
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
|
||||
/* add a new delimiter */
|
||||
self.Emit("ADDQ", jit.Imm(1), _CX) // ADDQ $1, CX
|
||||
self.Emit("CMPQ", _CX, jit.Imm(types.MAX_RECURSE)) // CMPQ CX, ${types.MAX_RECURSE}
|
||||
self.Sjmp("JAE" , "_stack_overflow") // JA _stack_overflow
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.Emit("MOVQ", jit.Imm(_S_obj_delim), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_obj_delim, ST.Vt[CX]
|
||||
|
||||
/* add a new slot int the map */
|
||||
self.Emit("MOVQ", _AX, _DI) // MOVQ AX, DI
|
||||
self.Emit("MOVQ", _T_map, _AX) // MOVQ _T_map, AX
|
||||
self.Emit("MOVQ", _SI, _BX) // MOVQ SI, BX
|
||||
self.Emit("MOVQ", _R8, _CX) // MOVQ R9, CX
|
||||
self.call_go(_F_mapassign_faststr) // CALL_GO runtime.mapassign_faststr
|
||||
|
||||
/* add to the pointer stack */
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.WritePtrAX(6, jit.Sib(_ST, _CX, 8, _ST_Vp), false) // MOVQ AX, ST.Vp[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* allocate memory to store the string header and unquoted result */
|
||||
self.Link("_unquote") // _unquote:
|
||||
self.Emit("ADDQ", jit.Imm(15), _AX) // ADDQ $15, AX
|
||||
self.Emit("MOVQ", _T_byte, _BX) // MOVQ _T_byte, BX
|
||||
self.Emit("MOVB", jit.Imm(0), _CX) // MOVB $0, CX
|
||||
self.call_go(_F_mallocgc) // CALL_GO runtime.mallocgc
|
||||
self.Emit("MOVQ", _AX, _R9) // MOVQ AX, R9
|
||||
|
||||
/* prepare the unquoting parameters */
|
||||
self.Emit("MOVQ" , _VAR_ss_Iv, _CX) // MOVQ ss.Iv, CX
|
||||
self.Emit("LEAQ" , jit.Sib(_IP, _CX, 1, 0), _DI) // LEAQ (IP)(CX), DI
|
||||
self.Emit("NEGQ" , _CX) // NEGQ CX
|
||||
self.Emit("LEAQ" , jit.Sib(_IC, _CX, 1, -1), _SI) // LEAQ -1(IC)(CX), SI
|
||||
self.Emit("LEAQ" , jit.Ptr(_R9, 16), _DX) // LEAQ 16(R8), DX
|
||||
self.Emit("LEAQ" , _VAR_ss_Ep, _CX) // LEAQ ss.Ep, CX
|
||||
self.Emit("XORL" , _R8, _R8) // XORL R8, R8
|
||||
self.Emit("BTQ" , jit.Imm(_F_disable_urc), _VAR_df) // BTQ ${_F_disable_urc}, fv
|
||||
self.Emit("SETCC", _R8) // SETCC R8
|
||||
self.Emit("SHLQ" , jit.Imm(types.B_UNICODE_REPLACE), _R8) // SHLQ ${types.B_UNICODE_REPLACE}, R8
|
||||
|
||||
/* unquote the string, with R9 been preserved */
|
||||
self.Emit("MOVQ", _R9, _VAR_R9) // SAVE R9
|
||||
self.call_c(_F_unquote) // CALL unquote
|
||||
self.Emit("MOVQ", _VAR_R9, _R9) // LOAD R9
|
||||
|
||||
/* check for errors */
|
||||
self.Emit("TESTQ", _AX, _AX) // TESTQ AX, AX
|
||||
self.Sjmp("JS" , "_unquote_error") // JS _unquote_error
|
||||
self.Emit("MOVL" , jit.Imm(1), _DX) // MOVL $1, DX
|
||||
self.Emit("LEAQ" , jit.Ptr(_R9, 16), _R8) // ADDQ $16, R8
|
||||
self.Emit("MOVQ" , _R8, jit.Ptr(_R9, 0)) // MOVQ R8, (R9)
|
||||
self.Emit("MOVQ" , _AX, jit.Ptr(_R9, 8)) // MOVQ AX, 8(R9)
|
||||
self.Sjmp("JMP" , "_noescape") // JMP _noescape
|
||||
|
||||
/** V_DOUBLE **/
|
||||
self.Link("_decode_V_DOUBLE") // _decode_V_DOUBLE:
|
||||
self.Emit("BTQ" , jit.Imm(_F_use_number), _VAR_df) // BTQ _F_use_number, df
|
||||
self.Sjmp("JC" , "_use_number") // JC _use_number
|
||||
self.Emit("MOVSD", _VAR_ss_Dv, _X0) // MOVSD ss.Dv, X0
|
||||
self.Sjmp("JMP" , "_use_float64") // JMP _use_float64
|
||||
|
||||
/** V_INTEGER **/
|
||||
self.Link("_decode_V_INTEGER") // _decode_V_INTEGER:
|
||||
self.Emit("BTQ" , jit.Imm(_F_use_number), _VAR_df) // BTQ _F_use_number, df
|
||||
self.Sjmp("JC" , "_use_number") // JC _use_number
|
||||
self.Emit("BTQ" , jit.Imm(_F_use_int64), _VAR_df) // BTQ _F_use_int64, df
|
||||
self.Sjmp("JC" , "_use_int64") // JC _use_int64
|
||||
//TODO: use ss.Dv directly
|
||||
self.Emit("MOVSD", _VAR_ss_Dv, _X0) // MOVSD ss.Dv, X0
|
||||
|
||||
/* represent numbers as `float64` */
|
||||
self.Link("_use_float64") // _use_float64:
|
||||
self.Emit("MOVQ" , _X0, _AX) // MOVQ X0, AX
|
||||
self.call_go(_F_convT64) // CALL_GO runtime.convT64
|
||||
self.Emit("MOVQ" , _T_float64, _R8) // MOVQ _T_float64, R8
|
||||
self.Emit("MOVQ" , _AX, _R9) // MOVQ AX, R9
|
||||
self.Emit("MOVQ" , _VAR_ss_Ep, _DI) // MOVQ ss.Ep, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/* represent numbers as `json.Number` */
|
||||
self.Link("_use_number") // _use_number
|
||||
self.Emit("MOVQ", _VAR_ss_Ep, _AX) // MOVQ ss.Ep, AX
|
||||
self.Emit("LEAQ", jit.Sib(_IP, _AX, 1, 0), _SI) // LEAQ (IP)(AX), SI
|
||||
self.Emit("MOVQ", _IC, _CX) // MOVQ IC, CX
|
||||
self.Emit("SUBQ", _AX, _CX) // SUBQ AX, CX
|
||||
self.Emit("MOVQ", _SI, _AX) // MOVQ SI, AX
|
||||
self.Emit("MOVQ", _CX, _BX) // MOVQ CX, BX
|
||||
self.call_go(_F_convTstring) // CALL_GO runtime.convTstring
|
||||
self.Emit("MOVQ", _T_number, _R8) // MOVQ _T_number, R8
|
||||
self.Emit("MOVQ", _AX, _R9) // MOVQ AX, R9
|
||||
self.Emit("MOVQ", _VAR_ss_Ep, _DI) // MOVQ ss.Ep, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/* represent numbers as `int64` */
|
||||
self.Link("_use_int64") // _use_int64:
|
||||
self.Emit("MOVQ", _VAR_ss_Iv, _AX) // MOVQ ss.Iv, AX
|
||||
self.call_go(_F_convT64) // CALL_GO runtime.convT64
|
||||
self.Emit("MOVQ", _T_int64, _R8) // MOVQ _T_int64, R8
|
||||
self.Emit("MOVQ", _AX, _R9) // MOVQ AX, R9
|
||||
self.Emit("MOVQ", _VAR_ss_Ep, _DI) // MOVQ ss.Ep, DI
|
||||
self.Sjmp("JMP" , "_set_value") // JMP _set_value
|
||||
|
||||
/** V_KEY_SEP **/
|
||||
self.Link("_decode_V_KEY_SEP") // _decode_V_KEY_SEP:
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("CMPQ", _AX, jit.Imm(_S_obj_delim)) // CMPQ AX, _S_obj_delim
|
||||
self.Sjmp("JNE" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("MOVQ", jit.Imm(_S_val), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_val, ST.Vt[CX]
|
||||
self.Emit("MOVQ", jit.Imm(_S_obj), jit.Sib(_ST, _CX, 8, _ST_Vt - 8)) // MOVQ _S_obj, ST.Vt[CX - 1]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_ELEM_SEP **/
|
||||
self.Link("_decode_V_ELEM_SEP") // _decode_V_ELEM_SEP:
|
||||
self.Emit("MOVQ" , jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ" , jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(_S_arr))
|
||||
self.Sjmp("JE" , "_array_sep") // JZ _next
|
||||
self.Emit("CMPQ" , _AX, jit.Imm(_S_obj)) // CMPQ _AX, _S_arr
|
||||
self.Sjmp("JNE" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("MOVQ" , jit.Imm(_S_obj_sep), jit.Sib(_ST, _CX, 8, _ST_Vt))
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* arrays */
|
||||
self.Link("_array_sep")
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _DX) // MOVQ 8(SI), DX
|
||||
self.Emit("CMPQ", _DX, jit.Ptr(_SI, 16)) // CMPQ DX, 16(SI)
|
||||
self.Sjmp("JAE" , "_array_more") // JAE _array_more
|
||||
|
||||
/* add a slot for the new element */
|
||||
self.Link("_array_append") // _array_append:
|
||||
self.Emit("ADDQ", jit.Imm(1), jit.Ptr(_SI, 8)) // ADDQ $1, 8(SI)
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 0), _SI) // MOVQ (SI), SI
|
||||
self.Emit("ADDQ", jit.Imm(1), _CX) // ADDQ $1, CX
|
||||
self.Emit("CMPQ", _CX, jit.Imm(types.MAX_RECURSE)) // CMPQ CX, ${types.MAX_RECURSE}
|
||||
self.Sjmp("JAE" , "_stack_overflow") // JA _stack_overflow
|
||||
self.Emit("SHLQ", jit.Imm(1), _DX) // SHLQ $1, DX
|
||||
self.Emit("LEAQ", jit.Sib(_SI, _DX, 8, 0), _SI) // LEAQ (SI)(DX*8), SI
|
||||
self.Emit("MOVQ", _CX, jit.Ptr(_ST, _ST_Sp)) // MOVQ CX, ST.Sp
|
||||
self.WriteRecNotAX(7 , _SI, jit.Sib(_ST, _CX, 8, _ST_Vp), false) // MOVQ SI, ST.Vp[CX]
|
||||
self.Emit("MOVQ", jit.Imm(_S_val), jit.Sib(_ST, _CX, 8, _ST_Vt)) // MOVQ _S_val, ST.Vt[CX}
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_ARRAY_END **/
|
||||
self.Link("_decode_V_ARRAY_END") // _decode_V_ARRAY_END:
|
||||
self.Emit("XORL", _DX, _DX) // XORL DX, DX
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("CMPQ", _AX, jit.Imm(_S_arr_0)) // CMPQ AX, _S_arr_0
|
||||
self.Sjmp("JE" , "_first_item") // JE _first_item
|
||||
self.Emit("CMPQ", _AX, jit.Imm(_S_arr)) // CMPQ AX, _S_arr
|
||||
self.Sjmp("JNE" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("SUBQ", jit.Imm(1), jit.Ptr(_ST, _ST_Sp)) // SUBQ $1, ST.Sp
|
||||
self.Emit("MOVQ", _DX, jit.Sib(_ST, _CX, 8, _ST_Vp)) // MOVQ DX, ST.Vp[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* first element of an array */
|
||||
self.Link("_first_item") // _first_item:
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("SUBQ", jit.Imm(2), jit.Ptr(_ST, _ST_Sp)) // SUBQ $2, ST.Sp
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp - 8), _SI) // MOVQ ST.Vp[CX - 1], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
self.Emit("MOVQ", _DX, jit.Sib(_ST, _CX, 8, _ST_Vp - 8)) // MOVQ DX, ST.Vp[CX - 1]
|
||||
self.Emit("MOVQ", _DX, jit.Sib(_ST, _CX, 8, _ST_Vp)) // MOVQ DX, ST.Vp[CX]
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SI, 8)) // MOVQ DX, 8(SI)
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/** V_OBJECT_END **/
|
||||
self.Link("_decode_V_OBJECT_END") // _decode_V_OBJECT_END:
|
||||
self.Emit("MOVL", jit.Imm(_S_omask_end), _DI) // MOVL _S_omask, DI
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vt), _AX) // MOVQ ST.Vt[CX], AX
|
||||
self.Emit("BTQ" , _AX, _DI)
|
||||
self.Sjmp("JNC" , "_invalid_char") // JNE _invalid_char
|
||||
self.Emit("XORL", _AX, _AX) // XORL AX, AX
|
||||
self.Emit("SUBQ", jit.Imm(1), jit.Ptr(_ST, _ST_Sp)) // SUBQ $1, ST.Sp
|
||||
self.Emit("MOVQ", _AX, jit.Sib(_ST, _CX, 8, _ST_Vp)) // MOVQ AX, ST.Vp[CX]
|
||||
self.Sjmp("JMP" , "_next") // JMP _next
|
||||
|
||||
/* return from decoder */
|
||||
self.Link("_return") // _return:
|
||||
self.Emit("XORL", _EP, _EP) // XORL EP, EP
|
||||
self.Emit("MOVQ", _EP, jit.Ptr(_ST, _ST_Vp)) // MOVQ EP, ST.Vp[0]
|
||||
self.Link("_epilogue") // _epilogue:
|
||||
self.Emit("SUBQ", jit.Imm(_FsmOffset), _ST) // SUBQ _FsmOffset, _ST
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, _VD_offs), _BP) // MOVQ _VD_offs(SP), BP
|
||||
self.Emit("ADDQ", jit.Imm(_VD_size), _SP) // ADDQ $_VD_size, SP
|
||||
self.Emit("RET") // RET
|
||||
|
||||
/* array expand */
|
||||
self.Link("_array_more") // _array_more:
|
||||
self.Emit("MOVQ" , _T_eface, _AX) // MOVQ _T_eface, AX
|
||||
self.Emit("MOVQ" , jit.Ptr(_SI, 0), _BX) // MOVQ (SI), BX
|
||||
self.Emit("MOVQ" , jit.Ptr(_SI, 8), _CX) // MOVQ 8(SI), CX
|
||||
self.Emit("MOVQ" , jit.Ptr(_SI, 16), _DI) // MOVQ 16(SI), DI
|
||||
self.Emit("MOVQ" , _DI, _SI) // MOVQ DI, 24(SP)
|
||||
self.Emit("SHLQ" , jit.Imm(1), _SI) // SHLQ $1, SI
|
||||
self.call_go(_F_growslice) // CALL_GO runtime.growslice
|
||||
self.Emit("MOVQ" , _AX, _DI) // MOVQ AX, DI
|
||||
self.Emit("MOVQ" , _BX, _DX) // MOVQ BX, DX
|
||||
self.Emit("MOVQ" , _CX, _AX) // MOVQ CX, AX
|
||||
|
||||
/* update the slice */
|
||||
self.Emit("MOVQ", jit.Ptr(_ST, _ST_Sp), _CX) // MOVQ ST.Sp, CX
|
||||
self.Emit("MOVQ", jit.Sib(_ST, _CX, 8, _ST_Vp), _SI) // MOVQ ST.Vp[CX], SI
|
||||
self.Emit("MOVQ", jit.Ptr(_SI, 8), _SI) // MOVQ 8(SI), SI
|
||||
self.Emit("MOVQ", _DX, jit.Ptr(_SI, 8)) // MOVQ DX, 8(SI)
|
||||
self.Emit("MOVQ", _AX, jit.Ptr(_SI, 16)) // MOVQ AX, 16(AX)
|
||||
self.WriteRecNotAX(8 , _DI, jit.Ptr(_SI, 0), false) // MOVQ R10, (SI)
|
||||
self.Sjmp("JMP" , "_array_append") // JMP _array_append
|
||||
|
||||
/* copy string */
|
||||
self.Link("copy_string") // pointer: R8, length: AX, return addr: DI
|
||||
self.Emit("MOVQ", _R8, _VAR_cs_p)
|
||||
self.Emit("MOVQ", _AX, _VAR_cs_n)
|
||||
self.Emit("MOVQ", _DI, _VAR_cs_LR)
|
||||
self.Emit("MOVQ", _AX, _BX)
|
||||
self.Emit("MOVQ", _AX, _CX)
|
||||
self.Emit("MOVQ", _T_byte, _AX)
|
||||
self.call_go(_F_makeslice)
|
||||
self.Emit("MOVQ", _AX, _VAR_cs_d)
|
||||
self.Emit("MOVQ", _VAR_cs_p, _BX)
|
||||
self.Emit("MOVQ", _VAR_cs_n, _CX)
|
||||
self.call_go(_F_memmove)
|
||||
self.Emit("MOVQ", _VAR_cs_d, _R8)
|
||||
self.Emit("MOVQ", _VAR_cs_n, _AX)
|
||||
self.Emit("MOVQ", _VAR_cs_LR, _DI)
|
||||
self.Rjmp("JMP", _DI)
|
||||
|
||||
/* error handlers */
|
||||
self.Link("_stack_overflow")
|
||||
self.Emit("MOVL" , _E_recurse, _EP) // MOVQ _E_recurse, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
self.Link("_vtype_error") // _vtype_error:
|
||||
self.Emit("MOVQ" , _DI, _IC) // MOVQ DI, IC
|
||||
self.Emit("MOVL" , _E_invalid, _EP) // MOVL _E_invalid, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
self.Link("_invalid_char") // _invalid_char:
|
||||
self.Emit("SUBQ" , jit.Imm(1), _IC) // SUBQ $1, IC
|
||||
self.Emit("MOVL" , _E_invalid, _EP) // MOVL _E_invalid, EP
|
||||
self.Sjmp("JMP" , "_error") // JMP _error
|
||||
self.Link("_unquote_error") // _unquote_error:
|
||||
self.Emit("MOVQ" , _VAR_ss_Iv, _IC) // MOVQ ss.Iv, IC
|
||||
self.Emit("SUBQ" , jit.Imm(1), _IC) // SUBQ $1, IC
|
||||
self.Link("_parsing_error") // _parsing_error:
|
||||
self.Emit("NEGQ" , _AX) // NEGQ AX
|
||||
self.Emit("MOVQ" , _AX, _EP) // MOVQ AX, EP
|
||||
self.Link("_error") // _error:
|
||||
self.Emit("PXOR" , _X0, _X0) // PXOR X0, X0
|
||||
self.Emit("MOVOU", _X0, jit.Ptr(_VP, 0)) // MOVOU X0, (VP)
|
||||
self.Sjmp("JMP" , "_epilogue") // JMP _epilogue
|
||||
|
||||
/* invalid value type, never returns */
|
||||
self.Link("_invalid_vtype")
|
||||
self.call_go(_F_invalid_vtype) // CALL invalid_type
|
||||
self.Emit("UD2") // UD2
|
||||
|
||||
/* switch jump table */
|
||||
self.Link("_switch_table") // _switch_table:
|
||||
self.Sref("_decode_V_EOF", 0) // SREF &_decode_V_EOF, $0
|
||||
self.Sref("_decode_V_NULL", -4) // SREF &_decode_V_NULL, $-4
|
||||
self.Sref("_decode_V_TRUE", -8) // SREF &_decode_V_TRUE, $-8
|
||||
self.Sref("_decode_V_FALSE", -12) // SREF &_decode_V_FALSE, $-12
|
||||
self.Sref("_decode_V_ARRAY", -16) // SREF &_decode_V_ARRAY, $-16
|
||||
self.Sref("_decode_V_OBJECT", -20) // SREF &_decode_V_OBJECT, $-20
|
||||
self.Sref("_decode_V_STRING", -24) // SREF &_decode_V_STRING, $-24
|
||||
self.Sref("_decode_V_DOUBLE", -28) // SREF &_decode_V_DOUBLE, $-28
|
||||
self.Sref("_decode_V_INTEGER", -32) // SREF &_decode_V_INTEGER, $-32
|
||||
self.Sref("_decode_V_KEY_SEP", -36) // SREF &_decode_V_KEY_SEP, $-36
|
||||
self.Sref("_decode_V_ELEM_SEP", -40) // SREF &_decode_V_ELEM_SEP, $-40
|
||||
self.Sref("_decode_V_ARRAY_END", -44) // SREF &_decode_V_ARRAY_END, $-44
|
||||
self.Sref("_decode_V_OBJECT_END", -48) // SREF &_decode_V_OBJECT_END, $-48
|
||||
|
||||
/* fast character lookup table */
|
||||
self.Link("_decode_tab") // _decode_tab:
|
||||
self.Sref("_decode_V_EOF", 0) // SREF &_decode_V_EOF, $0
|
||||
|
||||
/* generate rest of the tabs */
|
||||
for i := 1; i < 256; i++ {
|
||||
if to, ok := _R_tab[i]; ok {
|
||||
self.Sref(to, -int64(i) * 4)
|
||||
} else {
|
||||
self.Byte(0x00, 0x00, 0x00, 0x00)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) WritePtrAX(i int, rec obj.Addr, saveDI bool) {
|
||||
self.Emit("MOVQ", _V_writeBarrier, _R9)
|
||||
self.Emit("CMPL", jit.Ptr(_R9, 0), jit.Imm(0))
|
||||
self.Sjmp("JE", "_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
if saveDI {
|
||||
self.save(_DI)
|
||||
}
|
||||
self.Emit("LEAQ", rec, _DI)
|
||||
self.call(_F_gcWriteBarrierAX)
|
||||
if saveDI {
|
||||
self.load(_DI)
|
||||
}
|
||||
self.Sjmp("JMP", "_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Link("_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Emit("MOVQ", _AX, rec)
|
||||
self.Link("_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
}
|
||||
|
||||
func (self *_ValueDecoder) WriteRecNotAX(i int, ptr obj.Addr, rec obj.Addr, saveDI bool) {
|
||||
if rec.Reg == x86.REG_AX || rec.Index == x86.REG_AX {
|
||||
panic("rec contains AX!")
|
||||
}
|
||||
self.Emit("MOVQ", _V_writeBarrier, _AX)
|
||||
self.Emit("CMPL", jit.Ptr(_AX, 0), jit.Imm(0))
|
||||
self.Sjmp("JE", "_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Emit("MOVQ", ptr, _AX)
|
||||
if saveDI {
|
||||
self.save(_DI)
|
||||
}
|
||||
self.Emit("LEAQ", rec, _DI)
|
||||
self.call(_F_gcWriteBarrierAX)
|
||||
if saveDI {
|
||||
self.load(_DI)
|
||||
}
|
||||
self.Sjmp("JMP", "_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Link("_no_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
self.Emit("MOVQ", ptr, rec)
|
||||
self.Link("_end_writeBarrier" + strconv.Itoa(i) + "_{n}")
|
||||
}
|
||||
|
||||
/** Generic Decoder **/
|
||||
|
||||
var (
|
||||
_subr_decode_value = new(_ValueDecoder).build()
|
||||
)
|
||||
|
||||
//go:nosplit
|
||||
func invalid_vtype(vt types.ValueType) {
|
||||
throw(fmt.Sprintf("invalid value type: %d", vt))
|
||||
}
|
||||
37
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_go117_test.s
generated
vendored
Normal file
37
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_go117_test.s
generated
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
// +build go1.17,!go1.21
|
||||
|
||||
//
|
||||
// Copyright 2021 ByteDance Inc.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
#include "go_asm.h"
|
||||
#include "funcdata.h"
|
||||
#include "textflag.h"
|
||||
|
||||
TEXT ·decodeValueStub(SB), NOSPLIT, $0 - 72
|
||||
NO_LOCAL_POINTERS
|
||||
PXOR X0, X0
|
||||
MOVOU X0, rv+48(FP)
|
||||
MOVQ st+0(FP) , R13
|
||||
MOVQ sp+8(FP) , R10
|
||||
MOVQ sn+16(FP), R12
|
||||
MOVQ ic+24(FP), R11
|
||||
MOVQ vp+32(FP), R15
|
||||
MOVQ df+40(FP), AX
|
||||
MOVQ ·_subr_decode_value(SB), BX
|
||||
CALL BX
|
||||
MOVQ R11, rp+48(FP)
|
||||
MOVQ BX, ex+56(FP)
|
||||
RET
|
||||
37
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_test.s
generated
vendored
Normal file
37
vendor/github.com/bytedance/sonic/internal/decoder/generic_amd64_test.s
generated
vendored
Normal file
@@ -0,0 +1,37 @@
|
||||
// +build go1.15,!go1.17
|
||||
|
||||
//
|
||||
// Copyright 2021 ByteDance Inc.
|
||||
//
|
||||
// Licensed under the Apache License, Version 2.0 (the "License");
|
||||
// you may not use this file except in compliance with the License.
|
||||
// You may obtain a copy of the License at
|
||||
//
|
||||
// http://www.apache.org/licenses/LICENSE-2.0
|
||||
//
|
||||
// Unless required by applicable law or agreed to in writing, software
|
||||
// distributed under the License is distributed on an "AS IS" BASIS,
|
||||
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
// See the License for the specific language governing permissions and
|
||||
// limitations under the License.
|
||||
//
|
||||
|
||||
#include "go_asm.h"
|
||||
#include "funcdata.h"
|
||||
#include "textflag.h"
|
||||
|
||||
TEXT ·decodeValueStub(SB), NOSPLIT, $0 - 72
|
||||
NO_LOCAL_POINTERS
|
||||
PXOR X0, X0
|
||||
MOVOU X0, rv+48(FP)
|
||||
MOVQ st+0(FP), BX
|
||||
MOVQ sp+8(FP), R12
|
||||
MOVQ sn+16(FP), R13
|
||||
MOVQ ic+24(FP), R14
|
||||
MOVQ vp+32(FP), R15
|
||||
MOVQ df+40(FP), R10
|
||||
MOVQ ·_subr_decode_value(SB), AX
|
||||
CALL AX
|
||||
MOVQ R14, rp+48(FP)
|
||||
MOVQ R11, ex+56(FP)
|
||||
RET
|
||||
143
vendor/github.com/bytedance/sonic/internal/decoder/pools.go
generated
vendored
Normal file
143
vendor/github.com/bytedance/sonic/internal/decoder/pools.go
generated
vendored
Normal file
@@ -0,0 +1,143 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`sync`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/caching`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
const (
|
||||
_MinSlice = 2
|
||||
_MaxStack = 4096 // 4k slots
|
||||
_MaxStackBytes = _MaxStack * _PtrBytes
|
||||
_MaxDigitNums = 800 // used in atof fallback algorithm
|
||||
)
|
||||
|
||||
const (
|
||||
_PtrBytes = _PTR_SIZE / 8
|
||||
_FsmOffset = (_MaxStack + 1) * _PtrBytes
|
||||
_DbufOffset = _FsmOffset + int64(unsafe.Sizeof(types.StateMachine{})) + types.MAX_RECURSE * _PtrBytes
|
||||
_StackSize = unsafe.Sizeof(_Stack{})
|
||||
)
|
||||
|
||||
var (
|
||||
stackPool = sync.Pool{}
|
||||
valueCache = []unsafe.Pointer(nil)
|
||||
fieldCache = []*caching.FieldMap(nil)
|
||||
fieldCacheMux = sync.Mutex{}
|
||||
programCache = caching.CreateProgramCache()
|
||||
)
|
||||
|
||||
type _Stack struct {
|
||||
sp uintptr
|
||||
sb [_MaxStack]unsafe.Pointer
|
||||
mm types.StateMachine
|
||||
vp [types.MAX_RECURSE]unsafe.Pointer
|
||||
dp [_MaxDigitNums]byte
|
||||
}
|
||||
|
||||
type _Decoder func(
|
||||
s string,
|
||||
i int,
|
||||
vp unsafe.Pointer,
|
||||
sb *_Stack,
|
||||
fv uint64,
|
||||
sv string, // DO NOT pass value to this arguement, since it is only used for local _VAR_sv
|
||||
vk unsafe.Pointer, // DO NOT pass value to this arguement, since it is only used for local _VAR_vk
|
||||
) (int, error)
|
||||
|
||||
var _KeepAlive struct {
|
||||
s string
|
||||
i int
|
||||
vp unsafe.Pointer
|
||||
sb *_Stack
|
||||
fv uint64
|
||||
sv string
|
||||
vk unsafe.Pointer
|
||||
|
||||
ret int
|
||||
err error
|
||||
|
||||
frame_decoder [_FP_offs]byte
|
||||
frame_generic [_VD_offs]byte
|
||||
}
|
||||
|
||||
var (
|
||||
argPtrs = []bool{true, false, false, true, true, false, true, false, true}
|
||||
localPtrs = []bool{}
|
||||
)
|
||||
|
||||
var (
|
||||
argPtrs_generic = []bool{true}
|
||||
localPtrs_generic = []bool{}
|
||||
)
|
||||
|
||||
func newStack() *_Stack {
|
||||
if ret := stackPool.Get(); ret == nil {
|
||||
return new(_Stack)
|
||||
} else {
|
||||
return ret.(*_Stack)
|
||||
}
|
||||
}
|
||||
|
||||
func resetStack(p *_Stack) {
|
||||
memclrNoHeapPointers(unsafe.Pointer(p), _StackSize)
|
||||
}
|
||||
|
||||
func freeStack(p *_Stack) {
|
||||
p.sp = 0
|
||||
stackPool.Put(p)
|
||||
}
|
||||
|
||||
func freezeValue(v unsafe.Pointer) uintptr {
|
||||
valueCache = append(valueCache, v)
|
||||
return uintptr(v)
|
||||
}
|
||||
|
||||
func freezeFields(v *caching.FieldMap) int64 {
|
||||
fieldCacheMux.Lock()
|
||||
fieldCache = append(fieldCache, v)
|
||||
fieldCacheMux.Unlock()
|
||||
return referenceFields(v)
|
||||
}
|
||||
|
||||
func referenceFields(v *caching.FieldMap) int64 {
|
||||
return int64(uintptr(unsafe.Pointer(v)))
|
||||
}
|
||||
|
||||
func makeDecoder(vt *rt.GoType, _ ...interface{}) (interface{}, error) {
|
||||
if pp, err := newCompiler().compile(vt.Pack()); err != nil {
|
||||
return nil, err
|
||||
} else {
|
||||
return newAssembler(pp).Load(), nil
|
||||
}
|
||||
}
|
||||
|
||||
func findOrCompile(vt *rt.GoType) (_Decoder, error) {
|
||||
if val := programCache.Get(vt); val != nil {
|
||||
return val.(_Decoder), nil
|
||||
} else if ret, err := programCache.Compute(vt, makeDecoder); err == nil {
|
||||
return ret.(_Decoder), nil
|
||||
} else {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
46
vendor/github.com/bytedance/sonic/internal/decoder/primitives.go
generated
vendored
Normal file
46
vendor/github.com/bytedance/sonic/internal/decoder/primitives.go
generated
vendored
Normal file
@@ -0,0 +1,46 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`encoding`
|
||||
`encoding/json`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
func decodeTypedPointer(s string, i int, vt *rt.GoType, vp unsafe.Pointer, sb *_Stack, fv uint64) (int, error) {
|
||||
if fn, err := findOrCompile(vt); err != nil {
|
||||
return 0, err
|
||||
} else {
|
||||
rt.MoreStack(_FP_size + _VD_size + native.MaxFrameSize)
|
||||
rt.StopProf()
|
||||
ret, err := fn(s, i, vp, sb, fv, "", nil)
|
||||
rt.StartProf()
|
||||
return ret, err
|
||||
}
|
||||
}
|
||||
|
||||
func decodeJsonUnmarshaler(vv interface{}, s string) error {
|
||||
return vv.(json.Unmarshaler).UnmarshalJSON(rt.Str2Mem(s))
|
||||
}
|
||||
|
||||
func decodeTextUnmarshaler(vv interface{}, s string) error {
|
||||
return vv.(encoding.TextUnmarshaler).UnmarshalText(rt.Str2Mem(s))
|
||||
}
|
||||
217
vendor/github.com/bytedance/sonic/internal/decoder/stream.go
generated
vendored
Normal file
217
vendor/github.com/bytedance/sonic/internal/decoder/stream.go
generated
vendored
Normal file
@@ -0,0 +1,217 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`bytes`
|
||||
`io`
|
||||
`sync`
|
||||
|
||||
`github.com/bytedance/sonic/option`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
)
|
||||
|
||||
var (
|
||||
minLeftBufferShift uint = 1
|
||||
)
|
||||
|
||||
// StreamDecoder is the decoder context object for streaming input.
|
||||
type StreamDecoder struct {
|
||||
r io.Reader
|
||||
buf []byte
|
||||
scanp int
|
||||
scanned int64
|
||||
err error
|
||||
Decoder
|
||||
}
|
||||
|
||||
var bufPool = sync.Pool{
|
||||
New: func () interface{} {
|
||||
return make([]byte, 0, option.DefaultDecoderBufferSize)
|
||||
},
|
||||
}
|
||||
|
||||
// NewStreamDecoder adapts to encoding/json.NewDecoder API.
|
||||
//
|
||||
// NewStreamDecoder returns a new decoder that reads from r.
|
||||
func NewStreamDecoder(r io.Reader) *StreamDecoder {
|
||||
return &StreamDecoder{r : r}
|
||||
}
|
||||
|
||||
// Decode decodes input stream into val with corresponding data.
|
||||
// Redundantly bytes may be read and left in its buffer, and can be used at next call.
|
||||
// Either io error from underlying io.Reader (except io.EOF)
|
||||
// or syntax error from data will be recorded and stop subsequently decoding.
|
||||
func (self *StreamDecoder) Decode(val interface{}) (err error) {
|
||||
if self.err != nil {
|
||||
return self.err
|
||||
}
|
||||
|
||||
var buf = self.buf[self.scanp:]
|
||||
var p = 0
|
||||
var recycle bool
|
||||
if cap(buf) == 0 {
|
||||
buf = bufPool.Get().([]byte)
|
||||
recycle = true
|
||||
}
|
||||
|
||||
var first = true
|
||||
var repeat = true
|
||||
read_more:
|
||||
for {
|
||||
l := len(buf)
|
||||
realloc(&buf)
|
||||
n, err := self.r.Read(buf[l:cap(buf)])
|
||||
buf = buf[:l+n]
|
||||
if err != nil {
|
||||
repeat = false
|
||||
if err == io.EOF {
|
||||
if len(buf) == 0 {
|
||||
return err
|
||||
}
|
||||
break
|
||||
}
|
||||
self.err = err
|
||||
return err
|
||||
}
|
||||
if n > 0 || first {
|
||||
break
|
||||
}
|
||||
}
|
||||
first = false
|
||||
|
||||
l := len(buf)
|
||||
if l > 0 {
|
||||
self.Decoder.Reset(string(buf))
|
||||
err = self.Decoder.Decode(val)
|
||||
if err != nil {
|
||||
if repeat && self.repeatable(err) {
|
||||
goto read_more
|
||||
}
|
||||
self.err = err
|
||||
}
|
||||
|
||||
p = self.Decoder.Pos()
|
||||
self.scanned += int64(p)
|
||||
self.scanp = 0
|
||||
}
|
||||
|
||||
if l > p {
|
||||
// remain undecoded bytes, so copy them into self.buf
|
||||
self.buf = append(self.buf[:0], buf[p:]...)
|
||||
} else {
|
||||
self.buf = nil
|
||||
recycle = true
|
||||
}
|
||||
|
||||
if recycle {
|
||||
buf = buf[:0]
|
||||
bufPool.Put(buf)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (self StreamDecoder) repeatable(err error) bool {
|
||||
if ee, ok := err.(SyntaxError); ok &&
|
||||
(ee.Code == types.ERR_EOF || (ee.Code == types.ERR_INVALID_CHAR && self.i >= len(self.s)-1)) {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// InputOffset returns the input stream byte offset of the current decoder position.
|
||||
// The offset gives the location of the end of the most recently returned token and the beginning of the next token.
|
||||
func (self *StreamDecoder) InputOffset() int64 {
|
||||
return self.scanned + int64(self.scanp)
|
||||
}
|
||||
|
||||
// Buffered returns a reader of the data remaining in the Decoder's buffer.
|
||||
// The reader is valid until the next call to Decode.
|
||||
func (self *StreamDecoder) Buffered() io.Reader {
|
||||
return bytes.NewReader(self.buf[self.scanp:])
|
||||
}
|
||||
|
||||
// More reports whether there is another element in the
|
||||
// current array or object being parsed.
|
||||
func (self *StreamDecoder) More() bool {
|
||||
if self.err != nil {
|
||||
return false
|
||||
}
|
||||
c, err := self.peek()
|
||||
return err == nil && c != ']' && c != '}'
|
||||
}
|
||||
|
||||
func (self *StreamDecoder) peek() (byte, error) {
|
||||
var err error
|
||||
for {
|
||||
for i := self.scanp; i < len(self.buf); i++ {
|
||||
c := self.buf[i]
|
||||
if isSpace(c) {
|
||||
continue
|
||||
}
|
||||
self.scanp = i
|
||||
return c, nil
|
||||
}
|
||||
// buffer has been scanned, now report any error
|
||||
if err != nil {
|
||||
if err != io.EOF {
|
||||
self.err = err
|
||||
}
|
||||
return 0, err
|
||||
}
|
||||
err = self.refill()
|
||||
}
|
||||
}
|
||||
|
||||
func isSpace(c byte) bool {
|
||||
return types.SPACE_MASK & (1 << c) != 0
|
||||
}
|
||||
|
||||
func (self *StreamDecoder) refill() error {
|
||||
// Make room to read more into the buffer.
|
||||
// First slide down data already consumed.
|
||||
if self.scanp > 0 {
|
||||
self.scanned += int64(self.scanp)
|
||||
n := copy(self.buf, self.buf[self.scanp:])
|
||||
self.buf = self.buf[:n]
|
||||
self.scanp = 0
|
||||
}
|
||||
|
||||
// Grow buffer if not large enough.
|
||||
realloc(&self.buf)
|
||||
|
||||
// Read. Delay error for next iteration (after scan).
|
||||
n, err := self.r.Read(self.buf[len(self.buf):cap(self.buf)])
|
||||
self.buf = self.buf[0 : len(self.buf)+n]
|
||||
|
||||
return err
|
||||
}
|
||||
|
||||
func realloc(buf *[]byte) {
|
||||
l := uint(len(*buf))
|
||||
c := uint(cap(*buf))
|
||||
if c - l <= c >> minLeftBufferShift {
|
||||
e := l+(l>>minLeftBufferShift)
|
||||
if e < option.DefaultDecoderBufferSize {
|
||||
e = option.DefaultDecoderBufferSize
|
||||
}
|
||||
tmp := make([]byte, l, e)
|
||||
copy(tmp, *buf)
|
||||
*buf = tmp
|
||||
}
|
||||
}
|
||||
|
||||
111
vendor/github.com/bytedance/sonic/internal/decoder/stubs_go115.go
generated
vendored
Normal file
111
vendor/github.com/bytedance/sonic/internal/decoder/stubs_go115.go
generated
vendored
Normal file
@@ -0,0 +1,111 @@
|
||||
// +build go1.15,!go1.20
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
`reflect`
|
||||
|
||||
_ `github.com/chenzhuoyu/base64x`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
//go:linkname _subr__b64decode github.com/chenzhuoyu/base64x._subr__b64decode
|
||||
var _subr__b64decode uintptr
|
||||
|
||||
// runtime.maxElementSize
|
||||
const _max_map_element_size uintptr = 128
|
||||
|
||||
func mapfast(vt reflect.Type) bool {
|
||||
return vt.Elem().Size() <= _max_map_element_size
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
//go:linkname throw runtime.throw
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func throw(s string)
|
||||
|
||||
//go:linkname convT64 runtime.convT64
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func convT64(v uint64) unsafe.Pointer
|
||||
|
||||
//go:linkname convTslice runtime.convTslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func convTslice(v []byte) unsafe.Pointer
|
||||
|
||||
//go:linkname convTstring runtime.convTstring
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func convTstring(v string) unsafe.Pointer
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memequal runtime.memequal
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memequal(a unsafe.Pointer, b unsafe.Pointer, size uintptr) bool
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memmove runtime.memmove
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname mallocgc runtime.mallocgc
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mallocgc(size uintptr, typ *rt.GoType, needzero bool) unsafe.Pointer
|
||||
|
||||
//go:linkname makeslice runtime.makeslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func makeslice(et *rt.GoType, len int, cap int) unsafe.Pointer
|
||||
|
||||
//go:noescape
|
||||
//go:linkname growslice runtime.growslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
|
||||
|
||||
//go:linkname makemap_small runtime.makemap_small
|
||||
func makemap_small() unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign runtime.mapassign
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign(t *rt.GoType, h unsafe.Pointer, k unsafe.Pointer) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_fast32 runtime.mapassign_fast32
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_fast32(t *rt.GoType, h unsafe.Pointer, k uint32) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_fast64 runtime.mapassign_fast64
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_fast64(t *rt.GoType, h unsafe.Pointer, k uint64) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_fast64ptr runtime.mapassign_fast64ptr
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_fast64ptr(t *rt.GoType, h unsafe.Pointer, k unsafe.Pointer) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_faststr runtime.mapassign_faststr
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_faststr(t *rt.GoType, h unsafe.Pointer, s string) unsafe.Pointer
|
||||
|
||||
//go:nosplit
|
||||
//go:linkname memclrHasPointers runtime.memclrHasPointers
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memclrHasPointers(ptr unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
|
||||
111
vendor/github.com/bytedance/sonic/internal/decoder/stubs_go120.go
generated
vendored
Normal file
111
vendor/github.com/bytedance/sonic/internal/decoder/stubs_go120.go
generated
vendored
Normal file
@@ -0,0 +1,111 @@
|
||||
// +build go1.20
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
`reflect`
|
||||
|
||||
_ `github.com/chenzhuoyu/base64x`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
//go:linkname _subr__b64decode github.com/chenzhuoyu/base64x._subr__b64decode
|
||||
var _subr__b64decode uintptr
|
||||
|
||||
// runtime.maxElementSize
|
||||
const _max_map_element_size uintptr = 128
|
||||
|
||||
func mapfast(vt reflect.Type) bool {
|
||||
return vt.Elem().Size() <= _max_map_element_size
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
//go:linkname throw runtime.throw
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func throw(s string)
|
||||
|
||||
//go:linkname convT64 runtime.convT64
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func convT64(v uint64) unsafe.Pointer
|
||||
|
||||
//go:linkname convTslice runtime.convTslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func convTslice(v []byte) unsafe.Pointer
|
||||
|
||||
//go:linkname convTstring runtime.convTstring
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func convTstring(v string) unsafe.Pointer
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memequal runtime.memequal
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memequal(a unsafe.Pointer, b unsafe.Pointer, size uintptr) bool
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memmove runtime.memmove
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname mallocgc runtime.mallocgc
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mallocgc(size uintptr, typ *rt.GoType, needzero bool) unsafe.Pointer
|
||||
|
||||
//go:linkname makeslice runtime.makeslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func makeslice(et *rt.GoType, len int, cap int) unsafe.Pointer
|
||||
|
||||
//go:noescape
|
||||
//go:linkname growslice reflect.growslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
|
||||
|
||||
//go:linkname makemap_small runtime.makemap_small
|
||||
func makemap_small() unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign runtime.mapassign
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign(t *rt.GoType, h unsafe.Pointer, k unsafe.Pointer) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_fast32 runtime.mapassign_fast32
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_fast32(t *rt.GoType, h unsafe.Pointer, k uint32) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_fast64 runtime.mapassign_fast64
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_fast64(t *rt.GoType, h unsafe.Pointer, k uint64) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_fast64ptr runtime.mapassign_fast64ptr
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_fast64ptr(t *rt.GoType, h unsafe.Pointer, k unsafe.Pointer) unsafe.Pointer
|
||||
|
||||
//go:linkname mapassign_faststr runtime.mapassign_faststr
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapassign_faststr(t *rt.GoType, h unsafe.Pointer, s string) unsafe.Pointer
|
||||
|
||||
//go:nosplit
|
||||
//go:linkname memclrHasPointers runtime.memclrHasPointers
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memclrHasPointers(ptr unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
|
||||
58
vendor/github.com/bytedance/sonic/internal/decoder/types.go
generated
vendored
Normal file
58
vendor/github.com/bytedance/sonic/internal/decoder/types.go
generated
vendored
Normal file
@@ -0,0 +1,58 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`encoding`
|
||||
`encoding/base64`
|
||||
`encoding/json`
|
||||
`reflect`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
var (
|
||||
byteType = reflect.TypeOf(byte(0))
|
||||
intType = reflect.TypeOf(int(0))
|
||||
int8Type = reflect.TypeOf(int8(0))
|
||||
int16Type = reflect.TypeOf(int16(0))
|
||||
int32Type = reflect.TypeOf(int32(0))
|
||||
int64Type = reflect.TypeOf(int64(0))
|
||||
uintType = reflect.TypeOf(uint(0))
|
||||
uint8Type = reflect.TypeOf(uint8(0))
|
||||
uint16Type = reflect.TypeOf(uint16(0))
|
||||
uint32Type = reflect.TypeOf(uint32(0))
|
||||
uint64Type = reflect.TypeOf(uint64(0))
|
||||
float32Type = reflect.TypeOf(float32(0))
|
||||
float64Type = reflect.TypeOf(float64(0))
|
||||
stringType = reflect.TypeOf("")
|
||||
bytesType = reflect.TypeOf([]byte(nil))
|
||||
jsonNumberType = reflect.TypeOf(json.Number(""))
|
||||
base64CorruptInputError = reflect.TypeOf(base64.CorruptInputError(0))
|
||||
)
|
||||
|
||||
var (
|
||||
errorType = reflect.TypeOf((*error)(nil)).Elem()
|
||||
jsonUnmarshalerType = reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
|
||||
encodingTextUnmarshalerType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
|
||||
)
|
||||
|
||||
func rtype(t reflect.Type) (*rt.GoItab, *rt.GoType) {
|
||||
p := (*rt.GoIface)(unsafe.Pointer(&t))
|
||||
return p.Itab, (*rt.GoType)(p.Value)
|
||||
}
|
||||
39
vendor/github.com/bytedance/sonic/internal/decoder/utils.go
generated
vendored
Normal file
39
vendor/github.com/bytedance/sonic/internal/decoder/utils.go
generated
vendored
Normal file
@@ -0,0 +1,39 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package decoder
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/loader`
|
||||
)
|
||||
|
||||
//go:nosplit
|
||||
func pbool(v bool) uintptr {
|
||||
return freezeValue(unsafe.Pointer(&v))
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
func ptodec(p loader.Function) _Decoder {
|
||||
return *(*_Decoder)(unsafe.Pointer(&p))
|
||||
}
|
||||
|
||||
func assert_eq(v int64, exp int64, msg string) {
|
||||
if v != exp {
|
||||
panic(msg)
|
||||
}
|
||||
}
|
||||
0
vendor/github.com/bytedance/sonic/internal/encoder/asm.s
generated
vendored
Normal file
0
vendor/github.com/bytedance/sonic/internal/encoder/asm.s
generated
vendored
Normal file
1199
vendor/github.com/bytedance/sonic/internal/encoder/assembler_amd64_go116.go
generated
vendored
Normal file
1199
vendor/github.com/bytedance/sonic/internal/encoder/assembler_amd64_go116.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
1202
vendor/github.com/bytedance/sonic/internal/encoder/assembler_amd64_go117.go
generated
vendored
Normal file
1202
vendor/github.com/bytedance/sonic/internal/encoder/assembler_amd64_go117.go
generated
vendored
Normal file
File diff suppressed because it is too large
Load Diff
885
vendor/github.com/bytedance/sonic/internal/encoder/compiler.go
generated
vendored
Normal file
885
vendor/github.com/bytedance/sonic/internal/encoder/compiler.go
generated
vendored
Normal file
@@ -0,0 +1,885 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`fmt`
|
||||
`reflect`
|
||||
`strconv`
|
||||
`strings`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/resolver`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
`github.com/bytedance/sonic/option`
|
||||
)
|
||||
|
||||
type _Op uint8
|
||||
|
||||
const (
|
||||
_OP_null _Op = iota + 1
|
||||
_OP_empty_arr
|
||||
_OP_empty_obj
|
||||
_OP_bool
|
||||
_OP_i8
|
||||
_OP_i16
|
||||
_OP_i32
|
||||
_OP_i64
|
||||
_OP_u8
|
||||
_OP_u16
|
||||
_OP_u32
|
||||
_OP_u64
|
||||
_OP_f32
|
||||
_OP_f64
|
||||
_OP_str
|
||||
_OP_bin
|
||||
_OP_quote
|
||||
_OP_number
|
||||
_OP_eface
|
||||
_OP_iface
|
||||
_OP_byte
|
||||
_OP_text
|
||||
_OP_deref
|
||||
_OP_index
|
||||
_OP_load
|
||||
_OP_save
|
||||
_OP_drop
|
||||
_OP_drop_2
|
||||
_OP_recurse
|
||||
_OP_is_nil
|
||||
_OP_is_nil_p1
|
||||
_OP_is_zero_1
|
||||
_OP_is_zero_2
|
||||
_OP_is_zero_4
|
||||
_OP_is_zero_8
|
||||
_OP_is_zero_map
|
||||
_OP_goto
|
||||
_OP_map_iter
|
||||
_OP_map_stop
|
||||
_OP_map_check_key
|
||||
_OP_map_write_key
|
||||
_OP_map_value_next
|
||||
_OP_slice_len
|
||||
_OP_slice_next
|
||||
_OP_marshal
|
||||
_OP_marshal_p
|
||||
_OP_marshal_text
|
||||
_OP_marshal_text_p
|
||||
_OP_cond_set
|
||||
_OP_cond_testc
|
||||
)
|
||||
|
||||
const (
|
||||
_INT_SIZE = 32 << (^uint(0) >> 63)
|
||||
_PTR_SIZE = 32 << (^uintptr(0) >> 63)
|
||||
_PTR_BYTE = unsafe.Sizeof(uintptr(0))
|
||||
)
|
||||
|
||||
const (
|
||||
_MAX_ILBUF = 100000 // cutoff at 100k of IL instructions
|
||||
_MAX_FIELDS = 50 // cutoff at 50 fields struct
|
||||
)
|
||||
|
||||
var _OpNames = [256]string {
|
||||
_OP_null : "null",
|
||||
_OP_empty_arr : "empty_arr",
|
||||
_OP_empty_obj : "empty_obj",
|
||||
_OP_bool : "bool",
|
||||
_OP_i8 : "i8",
|
||||
_OP_i16 : "i16",
|
||||
_OP_i32 : "i32",
|
||||
_OP_i64 : "i64",
|
||||
_OP_u8 : "u8",
|
||||
_OP_u16 : "u16",
|
||||
_OP_u32 : "u32",
|
||||
_OP_u64 : "u64",
|
||||
_OP_f32 : "f32",
|
||||
_OP_f64 : "f64",
|
||||
_OP_str : "str",
|
||||
_OP_bin : "bin",
|
||||
_OP_quote : "quote",
|
||||
_OP_number : "number",
|
||||
_OP_eface : "eface",
|
||||
_OP_iface : "iface",
|
||||
_OP_byte : "byte",
|
||||
_OP_text : "text",
|
||||
_OP_deref : "deref",
|
||||
_OP_index : "index",
|
||||
_OP_load : "load",
|
||||
_OP_save : "save",
|
||||
_OP_drop : "drop",
|
||||
_OP_drop_2 : "drop_2",
|
||||
_OP_recurse : "recurse",
|
||||
_OP_is_nil : "is_nil",
|
||||
_OP_is_nil_p1 : "is_nil_p1",
|
||||
_OP_is_zero_1 : "is_zero_1",
|
||||
_OP_is_zero_2 : "is_zero_2",
|
||||
_OP_is_zero_4 : "is_zero_4",
|
||||
_OP_is_zero_8 : "is_zero_8",
|
||||
_OP_is_zero_map : "is_zero_map",
|
||||
_OP_goto : "goto",
|
||||
_OP_map_iter : "map_iter",
|
||||
_OP_map_stop : "map_stop",
|
||||
_OP_map_check_key : "map_check_key",
|
||||
_OP_map_write_key : "map_write_key",
|
||||
_OP_map_value_next : "map_value_next",
|
||||
_OP_slice_len : "slice_len",
|
||||
_OP_slice_next : "slice_next",
|
||||
_OP_marshal : "marshal",
|
||||
_OP_marshal_p : "marshal_p",
|
||||
_OP_marshal_text : "marshal_text",
|
||||
_OP_marshal_text_p : "marshal_text_p",
|
||||
_OP_cond_set : "cond_set",
|
||||
_OP_cond_testc : "cond_testc",
|
||||
}
|
||||
|
||||
func (self _Op) String() string {
|
||||
if ret := _OpNames[self]; ret != "" {
|
||||
return ret
|
||||
} else {
|
||||
return "<invalid>"
|
||||
}
|
||||
}
|
||||
|
||||
func _OP_int() _Op {
|
||||
switch _INT_SIZE {
|
||||
case 32: return _OP_i32
|
||||
case 64: return _OP_i64
|
||||
default: panic("unsupported int size")
|
||||
}
|
||||
}
|
||||
|
||||
func _OP_uint() _Op {
|
||||
switch _INT_SIZE {
|
||||
case 32: return _OP_u32
|
||||
case 64: return _OP_u64
|
||||
default: panic("unsupported uint size")
|
||||
}
|
||||
}
|
||||
|
||||
func _OP_uintptr() _Op {
|
||||
switch _PTR_SIZE {
|
||||
case 32: return _OP_u32
|
||||
case 64: return _OP_u64
|
||||
default: panic("unsupported pointer size")
|
||||
}
|
||||
}
|
||||
|
||||
func _OP_is_zero_ints() _Op {
|
||||
switch _INT_SIZE {
|
||||
case 32: return _OP_is_zero_4
|
||||
case 64: return _OP_is_zero_8
|
||||
default: panic("unsupported integer size")
|
||||
}
|
||||
}
|
||||
|
||||
type _Instr struct {
|
||||
u uint64 // union {op: 8, _: 8, vi: 48}, vi maybe int or len(str)
|
||||
p unsafe.Pointer // maybe GoString.Ptr, or *GoType
|
||||
}
|
||||
|
||||
func packOp(op _Op) uint64 {
|
||||
return uint64(op) << 56
|
||||
}
|
||||
|
||||
func newInsOp(op _Op) _Instr {
|
||||
return _Instr{u: packOp(op)}
|
||||
}
|
||||
|
||||
func newInsVi(op _Op, vi int) _Instr {
|
||||
return _Instr{u: packOp(op) | rt.PackInt(vi)}
|
||||
}
|
||||
|
||||
func newInsVs(op _Op, vs string) _Instr {
|
||||
return _Instr {
|
||||
u: packOp(op) | rt.PackInt(len(vs)),
|
||||
p: (*rt.GoString)(unsafe.Pointer(&vs)).Ptr,
|
||||
}
|
||||
}
|
||||
|
||||
func newInsVt(op _Op, vt reflect.Type) _Instr {
|
||||
return _Instr {
|
||||
u: packOp(op),
|
||||
p: unsafe.Pointer(rt.UnpackType(vt)),
|
||||
}
|
||||
}
|
||||
|
||||
func newInsVp(op _Op, vt reflect.Type, pv bool) _Instr {
|
||||
i := 0
|
||||
if pv {
|
||||
i = 1
|
||||
}
|
||||
return _Instr {
|
||||
u: packOp(op) | rt.PackInt(i),
|
||||
p: unsafe.Pointer(rt.UnpackType(vt)),
|
||||
}
|
||||
}
|
||||
|
||||
func (self _Instr) op() _Op {
|
||||
return _Op(self.u >> 56)
|
||||
}
|
||||
|
||||
func (self _Instr) vi() int {
|
||||
return rt.UnpackInt(self.u)
|
||||
}
|
||||
|
||||
func (self _Instr) vf() uint8 {
|
||||
return (*rt.GoType)(self.p).KindFlags
|
||||
}
|
||||
|
||||
func (self _Instr) vs() (v string) {
|
||||
(*rt.GoString)(unsafe.Pointer(&v)).Ptr = self.p
|
||||
(*rt.GoString)(unsafe.Pointer(&v)).Len = self.vi()
|
||||
return
|
||||
}
|
||||
|
||||
func (self _Instr) vk() reflect.Kind {
|
||||
return (*rt.GoType)(self.p).Kind()
|
||||
}
|
||||
|
||||
func (self _Instr) vt() reflect.Type {
|
||||
return (*rt.GoType)(self.p).Pack()
|
||||
}
|
||||
|
||||
func (self _Instr) vp() (vt reflect.Type, pv bool) {
|
||||
return (*rt.GoType)(self.p).Pack(), rt.UnpackInt(self.u) == 1
|
||||
}
|
||||
|
||||
func (self _Instr) i64() int64 {
|
||||
return int64(self.vi())
|
||||
}
|
||||
|
||||
func (self _Instr) vlen() int {
|
||||
return int((*rt.GoType)(self.p).Size)
|
||||
}
|
||||
|
||||
func (self _Instr) isBranch() bool {
|
||||
switch self.op() {
|
||||
case _OP_goto : fallthrough
|
||||
case _OP_is_nil : fallthrough
|
||||
case _OP_is_nil_p1 : fallthrough
|
||||
case _OP_is_zero_1 : fallthrough
|
||||
case _OP_is_zero_2 : fallthrough
|
||||
case _OP_is_zero_4 : fallthrough
|
||||
case _OP_is_zero_8 : fallthrough
|
||||
case _OP_map_check_key : fallthrough
|
||||
case _OP_map_write_key : fallthrough
|
||||
case _OP_slice_next : fallthrough
|
||||
case _OP_cond_testc : return true
|
||||
default : return false
|
||||
}
|
||||
}
|
||||
|
||||
func (self _Instr) disassemble() string {
|
||||
switch self.op() {
|
||||
case _OP_byte : return fmt.Sprintf("%-18s%s", self.op().String(), strconv.QuoteRune(rune(self.vi())))
|
||||
case _OP_text : return fmt.Sprintf("%-18s%s", self.op().String(), strconv.Quote(self.vs()))
|
||||
case _OP_index : return fmt.Sprintf("%-18s%d", self.op().String(), self.vi())
|
||||
case _OP_recurse : fallthrough
|
||||
case _OP_map_iter : fallthrough
|
||||
case _OP_marshal : fallthrough
|
||||
case _OP_marshal_p : fallthrough
|
||||
case _OP_marshal_text : fallthrough
|
||||
case _OP_marshal_text_p : return fmt.Sprintf("%-18s%s", self.op().String(), self.vt())
|
||||
case _OP_goto : fallthrough
|
||||
case _OP_is_nil : fallthrough
|
||||
case _OP_is_nil_p1 : fallthrough
|
||||
case _OP_is_zero_1 : fallthrough
|
||||
case _OP_is_zero_2 : fallthrough
|
||||
case _OP_is_zero_4 : fallthrough
|
||||
case _OP_is_zero_8 : fallthrough
|
||||
case _OP_is_zero_map : fallthrough
|
||||
case _OP_cond_testc : fallthrough
|
||||
case _OP_map_check_key : fallthrough
|
||||
case _OP_map_write_key : return fmt.Sprintf("%-18sL_%d", self.op().String(), self.vi())
|
||||
case _OP_slice_next : return fmt.Sprintf("%-18sL_%d, %s", self.op().String(), self.vi(), self.vt())
|
||||
default : return self.op().String()
|
||||
}
|
||||
}
|
||||
|
||||
type (
|
||||
_Program []_Instr
|
||||
)
|
||||
|
||||
func (self _Program) pc() int {
|
||||
return len(self)
|
||||
}
|
||||
|
||||
func (self _Program) tag(n int) {
|
||||
if n >= _MaxStack {
|
||||
panic("type nesting too deep")
|
||||
}
|
||||
}
|
||||
|
||||
func (self _Program) pin(i int) {
|
||||
v := &self[i]
|
||||
v.u &= 0xffff000000000000
|
||||
v.u |= rt.PackInt(self.pc())
|
||||
}
|
||||
|
||||
func (self _Program) rel(v []int) {
|
||||
for _, i := range v {
|
||||
self.pin(i)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Program) add(op _Op) {
|
||||
*self = append(*self, newInsOp(op))
|
||||
}
|
||||
|
||||
func (self *_Program) key(op _Op) {
|
||||
*self = append(*self,
|
||||
newInsVi(_OP_byte, '"'),
|
||||
newInsOp(op),
|
||||
newInsVi(_OP_byte, '"'),
|
||||
)
|
||||
}
|
||||
|
||||
func (self *_Program) int(op _Op, vi int) {
|
||||
*self = append(*self, newInsVi(op, vi))
|
||||
}
|
||||
|
||||
func (self *_Program) str(op _Op, vs string) {
|
||||
*self = append(*self, newInsVs(op, vs))
|
||||
}
|
||||
|
||||
func (self *_Program) rtt(op _Op, vt reflect.Type) {
|
||||
*self = append(*self, newInsVt(op, vt))
|
||||
}
|
||||
|
||||
func (self *_Program) vp(op _Op, vt reflect.Type, pv bool) {
|
||||
*self = append(*self, newInsVp(op, vt, pv))
|
||||
}
|
||||
|
||||
func (self _Program) disassemble() string {
|
||||
nb := len(self)
|
||||
tab := make([]bool, nb + 1)
|
||||
ret := make([]string, 0, nb + 1)
|
||||
|
||||
/* prescan to get all the labels */
|
||||
for _, ins := range self {
|
||||
if ins.isBranch() {
|
||||
tab[ins.vi()] = true
|
||||
}
|
||||
}
|
||||
|
||||
/* disassemble each instruction */
|
||||
for i, ins := range self {
|
||||
if !tab[i] {
|
||||
ret = append(ret, "\t" + ins.disassemble())
|
||||
} else {
|
||||
ret = append(ret, fmt.Sprintf("L_%d:\n\t%s", i, ins.disassemble()))
|
||||
}
|
||||
}
|
||||
|
||||
/* add the last label, if needed */
|
||||
if tab[nb] {
|
||||
ret = append(ret, fmt.Sprintf("L_%d:", nb))
|
||||
}
|
||||
|
||||
/* add an "end" indicator, and join all the strings */
|
||||
return strings.Join(append(ret, "\tend"), "\n")
|
||||
}
|
||||
|
||||
type _Compiler struct {
|
||||
opts option.CompileOptions
|
||||
pv bool
|
||||
tab map[reflect.Type]bool
|
||||
rec map[reflect.Type]uint8
|
||||
}
|
||||
|
||||
func newCompiler() *_Compiler {
|
||||
return &_Compiler {
|
||||
opts: option.DefaultCompileOptions(),
|
||||
tab: map[reflect.Type]bool{},
|
||||
rec: map[reflect.Type]uint8{},
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) apply(opts option.CompileOptions) *_Compiler {
|
||||
self.opts = opts
|
||||
if self.opts.RecursiveDepth > 0 {
|
||||
self.rec = map[reflect.Type]uint8{}
|
||||
}
|
||||
return self
|
||||
}
|
||||
|
||||
func (self *_Compiler) rescue(ep *error) {
|
||||
if val := recover(); val != nil {
|
||||
if err, ok := val.(error); ok {
|
||||
*ep = err
|
||||
} else {
|
||||
panic(val)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compile(vt reflect.Type, pv bool) (ret _Program, err error) {
|
||||
defer self.rescue(&err)
|
||||
self.compileOne(&ret, 0, vt, pv)
|
||||
return
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileOne(p *_Program, sp int, vt reflect.Type, pv bool) {
|
||||
if self.tab[vt] {
|
||||
p.vp(_OP_recurse, vt, pv)
|
||||
} else {
|
||||
self.compileRec(p, sp, vt, pv)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileRec(p *_Program, sp int, vt reflect.Type, pv bool) {
|
||||
pr := self.pv
|
||||
pt := reflect.PtrTo(vt)
|
||||
|
||||
/* check for addressable `json.Marshaler` with pointer receiver */
|
||||
if pv && pt.Implements(jsonMarshalerType) {
|
||||
p.rtt(_OP_marshal_p, pt)
|
||||
return
|
||||
}
|
||||
|
||||
/* check for `json.Marshaler` */
|
||||
if vt.Implements(jsonMarshalerType) {
|
||||
self.compileMarshaler(p, _OP_marshal, vt, jsonMarshalerType)
|
||||
return
|
||||
}
|
||||
|
||||
/* check for addressable `encoding.TextMarshaler` with pointer receiver */
|
||||
if pv && pt.Implements(encodingTextMarshalerType) {
|
||||
p.rtt(_OP_marshal_text_p, pt)
|
||||
return
|
||||
}
|
||||
|
||||
/* check for `encoding.TextMarshaler` */
|
||||
if vt.Implements(encodingTextMarshalerType) {
|
||||
self.compileMarshaler(p, _OP_marshal_text, vt, encodingTextMarshalerType)
|
||||
return
|
||||
}
|
||||
|
||||
/* enter the recursion, and compile the type */
|
||||
self.pv = pv
|
||||
self.tab[vt] = true
|
||||
self.compileOps(p, sp, vt)
|
||||
|
||||
/* exit the recursion */
|
||||
self.pv = pr
|
||||
delete(self.tab, vt)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileOps(p *_Program, sp int, vt reflect.Type) {
|
||||
switch vt.Kind() {
|
||||
case reflect.Bool : p.add(_OP_bool)
|
||||
case reflect.Int : p.add(_OP_int())
|
||||
case reflect.Int8 : p.add(_OP_i8)
|
||||
case reflect.Int16 : p.add(_OP_i16)
|
||||
case reflect.Int32 : p.add(_OP_i32)
|
||||
case reflect.Int64 : p.add(_OP_i64)
|
||||
case reflect.Uint : p.add(_OP_uint())
|
||||
case reflect.Uint8 : p.add(_OP_u8)
|
||||
case reflect.Uint16 : p.add(_OP_u16)
|
||||
case reflect.Uint32 : p.add(_OP_u32)
|
||||
case reflect.Uint64 : p.add(_OP_u64)
|
||||
case reflect.Uintptr : p.add(_OP_uintptr())
|
||||
case reflect.Float32 : p.add(_OP_f32)
|
||||
case reflect.Float64 : p.add(_OP_f64)
|
||||
case reflect.String : self.compileString (p, vt)
|
||||
case reflect.Array : self.compileArray (p, sp, vt.Elem(), vt.Len())
|
||||
case reflect.Interface : self.compileInterface (p, vt)
|
||||
case reflect.Map : self.compileMap (p, sp, vt)
|
||||
case reflect.Ptr : self.compilePtr (p, sp, vt.Elem())
|
||||
case reflect.Slice : self.compileSlice (p, sp, vt.Elem())
|
||||
case reflect.Struct : self.compileStruct (p, sp, vt)
|
||||
default : panic (error_type(vt))
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileNil(p *_Program, sp int, vt reflect.Type, nil_op _Op, fn func(*_Program, int, reflect.Type)) {
|
||||
x := p.pc()
|
||||
p.add(_OP_is_nil)
|
||||
fn(p, sp, vt)
|
||||
e := p.pc()
|
||||
p.add(_OP_goto)
|
||||
p.pin(x)
|
||||
p.add(nil_op)
|
||||
p.pin(e)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compilePtr(p *_Program, sp int, vt reflect.Type) {
|
||||
self.compileNil(p, sp, vt, _OP_null, self.compilePtrBody)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compilePtrBody(p *_Program, sp int, vt reflect.Type) {
|
||||
p.tag(sp)
|
||||
p.add(_OP_save)
|
||||
p.add(_OP_deref)
|
||||
self.compileOne(p, sp + 1, vt, true)
|
||||
p.add(_OP_drop)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileMap(p *_Program, sp int, vt reflect.Type) {
|
||||
self.compileNil(p, sp, vt, _OP_empty_obj, self.compileMapBody)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileMapBody(p *_Program, sp int, vt reflect.Type) {
|
||||
p.tag(sp + 1)
|
||||
p.int(_OP_byte, '{')
|
||||
p.add(_OP_save)
|
||||
p.rtt(_OP_map_iter, vt)
|
||||
p.add(_OP_save)
|
||||
i := p.pc()
|
||||
p.add(_OP_map_check_key)
|
||||
u := p.pc()
|
||||
p.add(_OP_map_write_key)
|
||||
self.compileMapBodyKey(p, vt.Key())
|
||||
p.pin(u)
|
||||
p.int(_OP_byte, ':')
|
||||
p.add(_OP_map_value_next)
|
||||
self.compileOne(p, sp + 2, vt.Elem(), false)
|
||||
j := p.pc()
|
||||
p.add(_OP_map_check_key)
|
||||
p.int(_OP_byte, ',')
|
||||
v := p.pc()
|
||||
p.add(_OP_map_write_key)
|
||||
self.compileMapBodyKey(p, vt.Key())
|
||||
p.pin(v)
|
||||
p.int(_OP_byte, ':')
|
||||
p.add(_OP_map_value_next)
|
||||
self.compileOne(p, sp + 2, vt.Elem(), false)
|
||||
p.int(_OP_goto, j)
|
||||
p.pin(i)
|
||||
p.pin(j)
|
||||
p.add(_OP_map_stop)
|
||||
p.add(_OP_drop_2)
|
||||
p.int(_OP_byte, '}')
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileMapBodyKey(p *_Program, vk reflect.Type) {
|
||||
if !vk.Implements(encodingTextMarshalerType) {
|
||||
self.compileMapBodyTextKey(p, vk)
|
||||
} else {
|
||||
self.compileMapBodyUtextKey(p, vk)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileMapBodyTextKey(p *_Program, vk reflect.Type) {
|
||||
switch vk.Kind() {
|
||||
case reflect.Invalid : panic("map key is nil")
|
||||
case reflect.Bool : p.key(_OP_bool)
|
||||
case reflect.Int : p.key(_OP_int())
|
||||
case reflect.Int8 : p.key(_OP_i8)
|
||||
case reflect.Int16 : p.key(_OP_i16)
|
||||
case reflect.Int32 : p.key(_OP_i32)
|
||||
case reflect.Int64 : p.key(_OP_i64)
|
||||
case reflect.Uint : p.key(_OP_uint())
|
||||
case reflect.Uint8 : p.key(_OP_u8)
|
||||
case reflect.Uint16 : p.key(_OP_u16)
|
||||
case reflect.Uint32 : p.key(_OP_u32)
|
||||
case reflect.Uint64 : p.key(_OP_u64)
|
||||
case reflect.Uintptr : p.key(_OP_uintptr())
|
||||
case reflect.Float32 : p.key(_OP_f32)
|
||||
case reflect.Float64 : p.key(_OP_f64)
|
||||
case reflect.String : self.compileString(p, vk)
|
||||
default : panic(error_type(vk))
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileMapBodyUtextKey(p *_Program, vk reflect.Type) {
|
||||
if vk.Kind() != reflect.Ptr {
|
||||
p.rtt(_OP_marshal_text, vk)
|
||||
} else {
|
||||
self.compileMapBodyUtextPtr(p, vk)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileMapBodyUtextPtr(p *_Program, vk reflect.Type) {
|
||||
i := p.pc()
|
||||
p.add(_OP_is_nil)
|
||||
p.rtt(_OP_marshal_text, vk)
|
||||
j := p.pc()
|
||||
p.add(_OP_goto)
|
||||
p.pin(i)
|
||||
p.str(_OP_text, "\"\"")
|
||||
p.pin(j)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileSlice(p *_Program, sp int, vt reflect.Type) {
|
||||
self.compileNil(p, sp, vt, _OP_empty_arr, self.compileSliceBody)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileSliceBody(p *_Program, sp int, vt reflect.Type) {
|
||||
if isSimpleByte(vt) {
|
||||
p.add(_OP_bin)
|
||||
} else {
|
||||
self.compileSliceArray(p, sp, vt)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileSliceArray(p *_Program, sp int, vt reflect.Type) {
|
||||
p.tag(sp)
|
||||
p.int(_OP_byte, '[')
|
||||
p.add(_OP_save)
|
||||
p.add(_OP_slice_len)
|
||||
i := p.pc()
|
||||
p.rtt(_OP_slice_next, vt)
|
||||
self.compileOne(p, sp + 1, vt, true)
|
||||
j := p.pc()
|
||||
p.rtt(_OP_slice_next, vt)
|
||||
p.int(_OP_byte, ',')
|
||||
self.compileOne(p, sp + 1, vt, true)
|
||||
p.int(_OP_goto, j)
|
||||
p.pin(i)
|
||||
p.pin(j)
|
||||
p.add(_OP_drop)
|
||||
p.int(_OP_byte, ']')
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileArray(p *_Program, sp int, vt reflect.Type, nb int) {
|
||||
p.tag(sp)
|
||||
p.int(_OP_byte, '[')
|
||||
p.add(_OP_save)
|
||||
|
||||
/* first item */
|
||||
if nb != 0 {
|
||||
self.compileOne(p, sp + 1, vt, self.pv)
|
||||
p.add(_OP_load)
|
||||
}
|
||||
|
||||
/* remaining items */
|
||||
for i := 1; i < nb; i++ {
|
||||
p.int(_OP_byte, ',')
|
||||
p.int(_OP_index, i * int(vt.Size()))
|
||||
self.compileOne(p, sp + 1, vt, self.pv)
|
||||
p.add(_OP_load)
|
||||
}
|
||||
|
||||
/* end of array */
|
||||
p.add(_OP_drop)
|
||||
p.int(_OP_byte, ']')
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileString(p *_Program, vt reflect.Type) {
|
||||
if vt != jsonNumberType {
|
||||
p.add(_OP_str)
|
||||
} else {
|
||||
p.add(_OP_number)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileStruct(p *_Program, sp int, vt reflect.Type) {
|
||||
if sp >= self.opts.MaxInlineDepth || p.pc() >= _MAX_ILBUF || (sp > 0 && vt.NumField() >= _MAX_FIELDS) {
|
||||
p.vp(_OP_recurse, vt, self.pv)
|
||||
if self.opts.RecursiveDepth > 0 {
|
||||
if self.pv {
|
||||
self.rec[vt] = 1
|
||||
} else {
|
||||
self.rec[vt] = 0
|
||||
}
|
||||
}
|
||||
} else {
|
||||
self.compileStructBody(p, sp, vt)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileStructBody(p *_Program, sp int, vt reflect.Type) {
|
||||
p.tag(sp)
|
||||
p.int(_OP_byte, '{')
|
||||
p.add(_OP_save)
|
||||
p.add(_OP_cond_set)
|
||||
|
||||
/* compile each field */
|
||||
for _, fv := range resolver.ResolveStruct(vt) {
|
||||
var s []int
|
||||
var o resolver.Offset
|
||||
|
||||
/* "omitempty" for arrays */
|
||||
if fv.Type.Kind() == reflect.Array {
|
||||
if fv.Type.Len() == 0 && (fv.Opts & resolver.F_omitempty) != 0 {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
/* index to the field */
|
||||
for _, o = range fv.Path {
|
||||
if p.int(_OP_index, int(o.Size)); o.Kind == resolver.F_deref {
|
||||
s = append(s, p.pc())
|
||||
p.add(_OP_is_nil)
|
||||
p.add(_OP_deref)
|
||||
}
|
||||
}
|
||||
|
||||
/* check for "omitempty" option */
|
||||
if fv.Type.Kind() != reflect.Struct && fv.Type.Kind() != reflect.Array && (fv.Opts & resolver.F_omitempty) != 0 {
|
||||
s = append(s, p.pc())
|
||||
self.compileStructFieldZero(p, fv.Type)
|
||||
}
|
||||
|
||||
/* add the comma if not the first element */
|
||||
i := p.pc()
|
||||
p.add(_OP_cond_testc)
|
||||
p.int(_OP_byte, ',')
|
||||
p.pin(i)
|
||||
|
||||
/* compile the key and value */
|
||||
ft := fv.Type
|
||||
p.str(_OP_text, Quote(fv.Name) + ":")
|
||||
|
||||
/* check for "stringnize" option */
|
||||
if (fv.Opts & resolver.F_stringize) == 0 {
|
||||
self.compileOne(p, sp + 1, ft, self.pv)
|
||||
} else {
|
||||
self.compileStructFieldStr(p, sp + 1, ft)
|
||||
}
|
||||
|
||||
/* patch the skipping jumps and reload the struct pointer */
|
||||
p.rel(s)
|
||||
p.add(_OP_load)
|
||||
}
|
||||
|
||||
/* end of object */
|
||||
p.add(_OP_drop)
|
||||
p.int(_OP_byte, '}')
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileStructFieldStr(p *_Program, sp int, vt reflect.Type) {
|
||||
pc := -1
|
||||
ft := vt
|
||||
sv := false
|
||||
|
||||
/* dereference the pointer if needed */
|
||||
if ft.Kind() == reflect.Ptr {
|
||||
ft = ft.Elem()
|
||||
}
|
||||
|
||||
/* check if it can be stringized */
|
||||
switch ft.Kind() {
|
||||
case reflect.Bool : sv = true
|
||||
case reflect.Int : sv = true
|
||||
case reflect.Int8 : sv = true
|
||||
case reflect.Int16 : sv = true
|
||||
case reflect.Int32 : sv = true
|
||||
case reflect.Int64 : sv = true
|
||||
case reflect.Uint : sv = true
|
||||
case reflect.Uint8 : sv = true
|
||||
case reflect.Uint16 : sv = true
|
||||
case reflect.Uint32 : sv = true
|
||||
case reflect.Uint64 : sv = true
|
||||
case reflect.Uintptr : sv = true
|
||||
case reflect.Float32 : sv = true
|
||||
case reflect.Float64 : sv = true
|
||||
case reflect.String : sv = true
|
||||
}
|
||||
|
||||
/* if it's not, ignore the "string" and follow the regular path */
|
||||
if !sv {
|
||||
self.compileOne(p, sp, vt, self.pv)
|
||||
return
|
||||
}
|
||||
|
||||
/* dereference the pointer */
|
||||
if vt.Kind() == reflect.Ptr {
|
||||
pc = p.pc()
|
||||
vt = vt.Elem()
|
||||
p.add(_OP_is_nil)
|
||||
p.add(_OP_deref)
|
||||
}
|
||||
|
||||
/* special case of a double-quoted string */
|
||||
if ft != jsonNumberType && ft.Kind() == reflect.String {
|
||||
p.add(_OP_quote)
|
||||
} else {
|
||||
self.compileStructFieldQuoted(p, sp, vt)
|
||||
}
|
||||
|
||||
/* the "null" case of the pointer */
|
||||
if pc != -1 {
|
||||
e := p.pc()
|
||||
p.add(_OP_goto)
|
||||
p.pin(pc)
|
||||
p.add(_OP_null)
|
||||
p.pin(e)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileStructFieldZero(p *_Program, vt reflect.Type) {
|
||||
switch vt.Kind() {
|
||||
case reflect.Bool : p.add(_OP_is_zero_1)
|
||||
case reflect.Int : p.add(_OP_is_zero_ints())
|
||||
case reflect.Int8 : p.add(_OP_is_zero_1)
|
||||
case reflect.Int16 : p.add(_OP_is_zero_2)
|
||||
case reflect.Int32 : p.add(_OP_is_zero_4)
|
||||
case reflect.Int64 : p.add(_OP_is_zero_8)
|
||||
case reflect.Uint : p.add(_OP_is_zero_ints())
|
||||
case reflect.Uint8 : p.add(_OP_is_zero_1)
|
||||
case reflect.Uint16 : p.add(_OP_is_zero_2)
|
||||
case reflect.Uint32 : p.add(_OP_is_zero_4)
|
||||
case reflect.Uint64 : p.add(_OP_is_zero_8)
|
||||
case reflect.Uintptr : p.add(_OP_is_nil)
|
||||
case reflect.Float32 : p.add(_OP_is_zero_4)
|
||||
case reflect.Float64 : p.add(_OP_is_zero_8)
|
||||
case reflect.String : p.add(_OP_is_nil_p1)
|
||||
case reflect.Interface : p.add(_OP_is_nil_p1)
|
||||
case reflect.Map : p.add(_OP_is_zero_map)
|
||||
case reflect.Ptr : p.add(_OP_is_nil)
|
||||
case reflect.Slice : p.add(_OP_is_nil_p1)
|
||||
default : panic(error_type(vt))
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileStructFieldQuoted(p *_Program, sp int, vt reflect.Type) {
|
||||
p.int(_OP_byte, '"')
|
||||
self.compileOne(p, sp, vt, self.pv)
|
||||
p.int(_OP_byte, '"')
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileInterface(p *_Program, vt reflect.Type) {
|
||||
x := p.pc()
|
||||
p.add(_OP_is_nil_p1)
|
||||
|
||||
/* iface and efaces are different */
|
||||
if vt.NumMethod() == 0 {
|
||||
p.add(_OP_eface)
|
||||
} else {
|
||||
p.add(_OP_iface)
|
||||
}
|
||||
|
||||
/* the "null" value */
|
||||
e := p.pc()
|
||||
p.add(_OP_goto)
|
||||
p.pin(x)
|
||||
p.add(_OP_null)
|
||||
p.pin(e)
|
||||
}
|
||||
|
||||
func (self *_Compiler) compileMarshaler(p *_Program, op _Op, vt reflect.Type, mt reflect.Type) {
|
||||
pc := p.pc()
|
||||
vk := vt.Kind()
|
||||
|
||||
/* direct receiver */
|
||||
if vk != reflect.Ptr {
|
||||
p.rtt(op, vt)
|
||||
return
|
||||
}
|
||||
|
||||
/* value receiver with a pointer type, check for nil before calling the marshaler */
|
||||
p.add(_OP_is_nil)
|
||||
p.rtt(op, vt)
|
||||
i := p.pc()
|
||||
p.add(_OP_goto)
|
||||
p.pin(pc)
|
||||
p.add(_OP_null)
|
||||
p.pin(i)
|
||||
}
|
||||
66
vendor/github.com/bytedance/sonic/internal/encoder/debug_go116.go
generated
vendored
Normal file
66
vendor/github.com/bytedance/sonic/internal/encoder/debug_go116.go
generated
vendored
Normal file
@@ -0,0 +1,66 @@
|
||||
// +build go1.15,!go1.17
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`os`
|
||||
`strings`
|
||||
`runtime`
|
||||
`runtime/debug`
|
||||
|
||||
`github.com/bytedance/sonic/internal/jit`
|
||||
)
|
||||
|
||||
var (
|
||||
debugSyncGC = os.Getenv("SONIC_SYNC_GC") != ""
|
||||
debugAsyncGC = os.Getenv("SONIC_NO_ASYNC_GC") == ""
|
||||
)
|
||||
|
||||
var (
|
||||
_Instr_End _Instr = newInsOp(_OP_null)
|
||||
|
||||
_F_gc = jit.Func(runtime.GC)
|
||||
_F_force_gc = jit.Func(debug.FreeOSMemory)
|
||||
_F_println = jit.Func(println_wrapper)
|
||||
)
|
||||
|
||||
func println_wrapper(i int, op1 int, op2 int){
|
||||
println(i, " Intrs ", op1, _OpNames[op1], "next: ", op2, _OpNames[op2])
|
||||
}
|
||||
|
||||
func (self *_Assembler) force_gc() {
|
||||
self.call_go(_F_gc)
|
||||
self.call_go(_F_force_gc)
|
||||
}
|
||||
|
||||
func (self *_Assembler) debug_instr(i int, v *_Instr) {
|
||||
if debugSyncGC {
|
||||
if (i+1 == len(self.p)) {
|
||||
self.print_gc(i, v, &_Instr_End)
|
||||
} else {
|
||||
next := &(self.p[i+1])
|
||||
self.print_gc(i, v, next)
|
||||
name := _OpNames[next.op()]
|
||||
if strings.Contains(name, "save") {
|
||||
return
|
||||
}
|
||||
}
|
||||
self.force_gc()
|
||||
}
|
||||
}
|
||||
205
vendor/github.com/bytedance/sonic/internal/encoder/debug_go117.go
generated
vendored
Normal file
205
vendor/github.com/bytedance/sonic/internal/encoder/debug_go117.go
generated
vendored
Normal file
@@ -0,0 +1,205 @@
|
||||
// +build go1.17,!go1.21
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`fmt`
|
||||
`os`
|
||||
`runtime`
|
||||
`strings`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/jit`
|
||||
`github.com/twitchyliquid64/golang-asm/obj`
|
||||
)
|
||||
|
||||
const _FP_debug = 128
|
||||
|
||||
var (
|
||||
debugSyncGC = os.Getenv("SONIC_SYNC_GC") != ""
|
||||
debugAsyncGC = os.Getenv("SONIC_NO_ASYNC_GC") == ""
|
||||
debugCheckPtr = os.Getenv("SONIC_CHECK_POINTER") != ""
|
||||
)
|
||||
|
||||
var (
|
||||
_Instr_End = newInsOp(_OP_is_nil)
|
||||
|
||||
_F_gc = jit.Func(gc)
|
||||
_F_println = jit.Func(println_wrapper)
|
||||
_F_print = jit.Func(print)
|
||||
)
|
||||
|
||||
func (self *_Assembler) dsave(r ...obj.Addr) {
|
||||
for i, v := range r {
|
||||
if i > _FP_debug / 8 - 1 {
|
||||
panic("too many registers to save")
|
||||
} else {
|
||||
self.Emit("MOVQ", v, jit.Ptr(_SP, _FP_fargs + _FP_saves + _FP_locals + int64(i) * 8))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_Assembler) dload(r ...obj.Addr) {
|
||||
for i, v := range r {
|
||||
if i > _FP_debug / 8 - 1 {
|
||||
panic("too many registers to load")
|
||||
} else {
|
||||
self.Emit("MOVQ", jit.Ptr(_SP, _FP_fargs + _FP_saves + _FP_locals + int64(i) * 8), v)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func println_wrapper(i int, op1 int, op2 int){
|
||||
println(i, " Intrs ", op1, _OpNames[op1], "next: ", op2, _OpNames[op2])
|
||||
}
|
||||
|
||||
func print(i int){
|
||||
println(i)
|
||||
}
|
||||
|
||||
func gc() {
|
||||
if !debugSyncGC {
|
||||
return
|
||||
}
|
||||
runtime.GC()
|
||||
// debug.FreeOSMemory()
|
||||
}
|
||||
|
||||
func (self *_Assembler) dcall(fn obj.Addr) {
|
||||
self.Emit("MOVQ", fn, _R10) // MOVQ ${fn}, R10
|
||||
self.Rjmp("CALL", _R10) // CALL R10
|
||||
}
|
||||
|
||||
func (self *_Assembler) debug_gc() {
|
||||
if !debugSyncGC {
|
||||
return
|
||||
}
|
||||
self.dsave(_REG_debug...)
|
||||
self.dcall(_F_gc)
|
||||
self.dload(_REG_debug...)
|
||||
}
|
||||
|
||||
func (self *_Assembler) debug_instr(i int, v *_Instr) {
|
||||
if debugSyncGC {
|
||||
if i+1 == len(self.p) {
|
||||
self.print_gc(i, v, &_Instr_End)
|
||||
} else {
|
||||
next := &(self.p[i+1])
|
||||
self.print_gc(i, v, next)
|
||||
name := _OpNames[next.op()]
|
||||
if strings.Contains(name, "save") {
|
||||
return
|
||||
}
|
||||
}
|
||||
// self.debug_gc()
|
||||
}
|
||||
}
|
||||
|
||||
//go:noescape
|
||||
//go:linkname checkptrBase runtime.checkptrBase
|
||||
func checkptrBase(p unsafe.Pointer) uintptr
|
||||
|
||||
//go:noescape
|
||||
//go:linkname findObject runtime.findObject
|
||||
func findObject(p, refBase, refOff uintptr) (base uintptr, s unsafe.Pointer, objIndex uintptr)
|
||||
|
||||
var (
|
||||
_F_checkptr = jit.Func(checkptr)
|
||||
_F_printptr = jit.Func(printptr)
|
||||
)
|
||||
|
||||
var (
|
||||
_R10 = jit.Reg("R10")
|
||||
)
|
||||
var _REG_debug = []obj.Addr {
|
||||
jit.Reg("AX"),
|
||||
jit.Reg("BX"),
|
||||
jit.Reg("CX"),
|
||||
jit.Reg("DX"),
|
||||
jit.Reg("DI"),
|
||||
jit.Reg("SI"),
|
||||
jit.Reg("BP"),
|
||||
jit.Reg("SP"),
|
||||
jit.Reg("R8"),
|
||||
jit.Reg("R9"),
|
||||
jit.Reg("R10"),
|
||||
jit.Reg("R11"),
|
||||
jit.Reg("R12"),
|
||||
jit.Reg("R13"),
|
||||
jit.Reg("R14"),
|
||||
jit.Reg("R15"),
|
||||
}
|
||||
|
||||
func checkptr(ptr uintptr) {
|
||||
if ptr == 0 {
|
||||
return
|
||||
}
|
||||
fmt.Printf("pointer: %x\n", ptr)
|
||||
f := checkptrBase(unsafe.Pointer(uintptr(ptr)))
|
||||
if f == 0 {
|
||||
fmt.Printf("! unknown-based pointer: %x\n", ptr)
|
||||
} else if f == 1 {
|
||||
fmt.Printf("! stack pointer: %x\n", ptr)
|
||||
} else {
|
||||
fmt.Printf("base: %x\n", f)
|
||||
}
|
||||
findobj(ptr)
|
||||
}
|
||||
|
||||
func findobj(ptr uintptr) {
|
||||
base, s, objIndex := findObject(ptr, 0, 0)
|
||||
if s != nil && base == 0 {
|
||||
fmt.Printf("! invalid pointer: %x\n", ptr)
|
||||
}
|
||||
fmt.Printf("objIndex: %d\n", objIndex)
|
||||
}
|
||||
|
||||
func (self *_Assembler) check_ptr(ptr obj.Addr, lea bool) {
|
||||
if !debugCheckPtr {
|
||||
return
|
||||
}
|
||||
|
||||
self.dsave(_REG_debug...)
|
||||
if lea {
|
||||
self.Emit("LEAQ", ptr, _R10)
|
||||
} else {
|
||||
self.Emit("MOVQ", ptr, _R10)
|
||||
}
|
||||
self.Emit("MOVQ", _R10, jit.Ptr(_SP, 0))
|
||||
self.dcall(_F_checkptr)
|
||||
self.dload(_REG_debug...)
|
||||
}
|
||||
|
||||
func printptr(i int, ptr uintptr) {
|
||||
fmt.Printf("[%d] ptr: %x\n", i, ptr)
|
||||
}
|
||||
|
||||
func (self *_Assembler) print_ptr(i int, ptr obj.Addr, lea bool) {
|
||||
self.dsave(_REG_debug...)
|
||||
if lea {
|
||||
self.Emit("LEAQ", ptr, _R10)
|
||||
} else {
|
||||
self.Emit("MOVQ", ptr, _R10)
|
||||
}
|
||||
|
||||
self.Emit("MOVQ", jit.Imm(int64(i)), _AX)
|
||||
self.Emit("MOVQ", _R10, _BX)
|
||||
self.dcall(_F_printptr)
|
||||
self.dload(_REG_debug...)
|
||||
}
|
||||
328
vendor/github.com/bytedance/sonic/internal/encoder/encoder.go
generated
vendored
Normal file
328
vendor/github.com/bytedance/sonic/internal/encoder/encoder.go
generated
vendored
Normal file
@@ -0,0 +1,328 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`bytes`
|
||||
`encoding/json`
|
||||
`reflect`
|
||||
`runtime`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/native`
|
||||
`github.com/bytedance/sonic/internal/native/types`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
`github.com/bytedance/sonic/utf8`
|
||||
`github.com/bytedance/sonic/option`
|
||||
)
|
||||
|
||||
// Options is a set of encoding options.
|
||||
type Options uint64
|
||||
|
||||
const (
|
||||
bitSortMapKeys = iota
|
||||
bitEscapeHTML
|
||||
bitCompactMarshaler
|
||||
bitNoQuoteTextMarshaler
|
||||
bitNoNullSliceOrMap
|
||||
bitValidateString
|
||||
|
||||
// used for recursive compile
|
||||
bitPointerValue = 63
|
||||
)
|
||||
|
||||
const (
|
||||
// SortMapKeys indicates that the keys of a map needs to be sorted
|
||||
// before serializing into JSON.
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
SortMapKeys Options = 1 << bitSortMapKeys
|
||||
|
||||
// EscapeHTML indicates encoder to escape all HTML characters
|
||||
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
|
||||
// WARNING: This hurts performance A LOT, USE WITH CARE.
|
||||
EscapeHTML Options = 1 << bitEscapeHTML
|
||||
|
||||
// CompactMarshaler indicates that the output JSON from json.Marshaler
|
||||
// is always compact and needs no validation
|
||||
CompactMarshaler Options = 1 << bitCompactMarshaler
|
||||
|
||||
// NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler
|
||||
// is always escaped string and needs no quoting
|
||||
NoQuoteTextMarshaler Options = 1 << bitNoQuoteTextMarshaler
|
||||
|
||||
// NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}',
|
||||
// instead of 'null'
|
||||
NoNullSliceOrMap Options = 1 << bitNoNullSliceOrMap
|
||||
|
||||
// ValidateString indicates that encoder should validate the input string
|
||||
// before encoding it into JSON.
|
||||
ValidateString Options = 1 << bitValidateString
|
||||
|
||||
// CompatibleWithStd is used to be compatible with std encoder.
|
||||
CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler
|
||||
)
|
||||
|
||||
// Encoder represents a specific set of encoder configurations.
|
||||
type Encoder struct {
|
||||
Opts Options
|
||||
prefix string
|
||||
indent string
|
||||
}
|
||||
|
||||
// Encode returns the JSON encoding of v.
|
||||
func (self *Encoder) Encode(v interface{}) ([]byte, error) {
|
||||
if self.indent != "" || self.prefix != "" {
|
||||
return EncodeIndented(v, self.prefix, self.indent, self.Opts)
|
||||
}
|
||||
return Encode(v, self.Opts)
|
||||
}
|
||||
|
||||
// SortKeys enables the SortMapKeys option.
|
||||
func (self *Encoder) SortKeys() *Encoder {
|
||||
self.Opts |= SortMapKeys
|
||||
return self
|
||||
}
|
||||
|
||||
// SetEscapeHTML specifies if option EscapeHTML opens
|
||||
func (self *Encoder) SetEscapeHTML(f bool) {
|
||||
if f {
|
||||
self.Opts |= EscapeHTML
|
||||
} else {
|
||||
self.Opts &= ^EscapeHTML
|
||||
}
|
||||
}
|
||||
|
||||
// SetValidateString specifies if option ValidateString opens
|
||||
func (self *Encoder) SetValidateString(f bool) {
|
||||
if f {
|
||||
self.Opts |= ValidateString
|
||||
} else {
|
||||
self.Opts &= ^ValidateString
|
||||
}
|
||||
}
|
||||
|
||||
// SetCompactMarshaler specifies if option CompactMarshaler opens
|
||||
func (self *Encoder) SetCompactMarshaler(f bool) {
|
||||
if f {
|
||||
self.Opts |= CompactMarshaler
|
||||
} else {
|
||||
self.Opts &= ^CompactMarshaler
|
||||
}
|
||||
}
|
||||
|
||||
// SetNoQuoteTextMarshaler specifies if option NoQuoteTextMarshaler opens
|
||||
func (self *Encoder) SetNoQuoteTextMarshaler(f bool) {
|
||||
if f {
|
||||
self.Opts |= NoQuoteTextMarshaler
|
||||
} else {
|
||||
self.Opts &= ^NoQuoteTextMarshaler
|
||||
}
|
||||
}
|
||||
|
||||
// SetIndent instructs the encoder to format each subsequent encoded
|
||||
// value as if indented by the package-level function EncodeIndent().
|
||||
// Calling SetIndent("", "") disables indentation.
|
||||
func (enc *Encoder) SetIndent(prefix, indent string) {
|
||||
enc.prefix = prefix
|
||||
enc.indent = indent
|
||||
}
|
||||
|
||||
// Quote returns the JSON-quoted version of s.
|
||||
func Quote(s string) string {
|
||||
var n int
|
||||
var p []byte
|
||||
|
||||
/* check for empty string */
|
||||
if s == "" {
|
||||
return `""`
|
||||
}
|
||||
|
||||
/* allocate space for result */
|
||||
n = len(s) + 2
|
||||
p = make([]byte, 0, n)
|
||||
|
||||
/* call the encoder */
|
||||
_ = encodeString(&p, s)
|
||||
return rt.Mem2Str(p)
|
||||
}
|
||||
|
||||
// Encode returns the JSON encoding of val, encoded with opts.
|
||||
func Encode(val interface{}, opts Options) ([]byte, error) {
|
||||
var ret []byte
|
||||
|
||||
buf := newBytes()
|
||||
err := encodeInto(&buf, val, opts)
|
||||
|
||||
/* check for errors */
|
||||
if err != nil {
|
||||
freeBytes(buf)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
/* htmlescape or correct UTF-8 if opts enable */
|
||||
old := buf
|
||||
buf = encodeFinish(old, opts)
|
||||
pbuf := ((*rt.GoSlice)(unsafe.Pointer(&buf))).Ptr
|
||||
pold := ((*rt.GoSlice)(unsafe.Pointer(&old))).Ptr
|
||||
|
||||
/* return when allocated a new buffer */
|
||||
if pbuf != pold {
|
||||
freeBytes(old)
|
||||
return buf, nil
|
||||
}
|
||||
|
||||
/* make a copy of the result */
|
||||
ret = make([]byte, len(buf))
|
||||
copy(ret, buf)
|
||||
|
||||
freeBytes(buf)
|
||||
/* return the buffer into pool */
|
||||
return ret, nil
|
||||
}
|
||||
|
||||
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating
|
||||
// a new one.
|
||||
func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
|
||||
err := encodeInto(buf, val, opts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
*buf = encodeFinish(*buf, opts)
|
||||
return err
|
||||
}
|
||||
|
||||
func encodeInto(buf *[]byte, val interface{}, opts Options) error {
|
||||
stk := newStack()
|
||||
efv := rt.UnpackEface(val)
|
||||
err := encodeTypedPointer(buf, efv.Type, &efv.Value, stk, uint64(opts))
|
||||
|
||||
/* return the stack into pool */
|
||||
if err != nil {
|
||||
resetStack(stk)
|
||||
}
|
||||
freeStack(stk)
|
||||
|
||||
/* avoid GC ahead */
|
||||
runtime.KeepAlive(buf)
|
||||
runtime.KeepAlive(efv)
|
||||
return err
|
||||
}
|
||||
|
||||
func encodeFinish(buf []byte, opts Options) []byte {
|
||||
if opts & EscapeHTML != 0 {
|
||||
buf = HTMLEscape(nil, buf)
|
||||
}
|
||||
if opts & ValidateString != 0 && !utf8.Validate(buf) {
|
||||
buf = utf8.CorrectWith(nil, buf, `\ufffd`)
|
||||
}
|
||||
return buf
|
||||
}
|
||||
|
||||
var typeByte = rt.UnpackType(reflect.TypeOf(byte(0)))
|
||||
|
||||
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029
|
||||
// characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029
|
||||
// so that the JSON will be safe to embed inside HTML <script> tags.
|
||||
// For historical reasons, web browsers don't honor standard HTML
|
||||
// escaping within <script> tags, so an alternative JSON encoding must
|
||||
// be used.
|
||||
func HTMLEscape(dst []byte, src []byte) []byte {
|
||||
return htmlEscape(dst, src)
|
||||
}
|
||||
|
||||
// EncodeIndented is like Encode but applies Indent to format the output.
|
||||
// Each JSON element in the output will begin on a new line beginning with prefix
|
||||
// followed by one or more copies of indent according to the indentation nesting.
|
||||
func EncodeIndented(val interface{}, prefix string, indent string, opts Options) ([]byte, error) {
|
||||
var err error
|
||||
var out []byte
|
||||
var buf *bytes.Buffer
|
||||
|
||||
/* encode into the buffer */
|
||||
out = newBytes()
|
||||
err = EncodeInto(&out, val, opts)
|
||||
|
||||
/* check for errors */
|
||||
if err != nil {
|
||||
freeBytes(out)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
/* indent the JSON */
|
||||
buf = newBuffer()
|
||||
err = json.Indent(buf, out, prefix, indent)
|
||||
|
||||
/* check for errors */
|
||||
if err != nil {
|
||||
freeBytes(out)
|
||||
freeBuffer(buf)
|
||||
return nil, err
|
||||
}
|
||||
|
||||
/* copy to the result buffer */
|
||||
ret := make([]byte, buf.Len())
|
||||
copy(ret, buf.Bytes())
|
||||
|
||||
/* return the buffers into pool */
|
||||
freeBytes(out)
|
||||
freeBuffer(buf)
|
||||
return ret, nil
|
||||
}
|
||||
|
||||
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
|
||||
// order to reduce the first-hit latency.
|
||||
//
|
||||
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
|
||||
// a compile option to set the depth of recursive compile for the nested struct type.
|
||||
func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
|
||||
cfg := option.DefaultCompileOptions()
|
||||
for _, opt := range opts {
|
||||
opt(&cfg)
|
||||
break
|
||||
}
|
||||
return pretouchRec(map[reflect.Type]uint8{vt: 0}, cfg)
|
||||
}
|
||||
|
||||
// Valid validates json and returns first non-blank character position,
|
||||
// if it is only one valid json value.
|
||||
// Otherwise returns invalid character position using start.
|
||||
//
|
||||
// Note: it does not check for the invalid UTF-8 characters.
|
||||
func Valid(data []byte) (ok bool, start int) {
|
||||
n := len(data)
|
||||
if n == 0 {
|
||||
return false, -1
|
||||
}
|
||||
s := rt.Mem2Str(data)
|
||||
p := 0
|
||||
m := types.NewStateMachine()
|
||||
ret := native.ValidateOne(&s, &p, m)
|
||||
types.FreeStateMachine(m)
|
||||
|
||||
if ret < 0 {
|
||||
return false, p-1
|
||||
}
|
||||
|
||||
/* check for trailing spaces */
|
||||
for ;p < n; p++ {
|
||||
if (types.SPACE_MASK & (1 << data[p])) == 0 {
|
||||
return false, p
|
||||
}
|
||||
}
|
||||
|
||||
return true, ret
|
||||
}
|
||||
65
vendor/github.com/bytedance/sonic/internal/encoder/errors.go
generated
vendored
Normal file
65
vendor/github.com/bytedance/sonic/internal/encoder/errors.go
generated
vendored
Normal file
@@ -0,0 +1,65 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`encoding/json`
|
||||
`fmt`
|
||||
`reflect`
|
||||
`strconv`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
var _ERR_too_deep = &json.UnsupportedValueError {
|
||||
Str : "Value nesting too deep",
|
||||
Value : reflect.ValueOf("..."),
|
||||
}
|
||||
|
||||
var _ERR_nan_or_infinite = &json.UnsupportedValueError {
|
||||
Str : "NaN or ±Infinite",
|
||||
Value : reflect.ValueOf("NaN or ±Infinite"),
|
||||
}
|
||||
|
||||
func error_type(vtype reflect.Type) error {
|
||||
return &json.UnsupportedTypeError{Type: vtype}
|
||||
}
|
||||
|
||||
func error_number(number json.Number) error {
|
||||
return &json.UnsupportedValueError {
|
||||
Str : "invalid number literal: " + strconv.Quote(string(number)),
|
||||
Value : reflect.ValueOf(number),
|
||||
}
|
||||
}
|
||||
|
||||
func error_marshaler(ret []byte, pos int) error {
|
||||
return fmt.Errorf("invalid Marshaler output json syntax at %d: %q", pos, ret)
|
||||
}
|
||||
|
||||
const (
|
||||
panicNilPointerOfNonEmptyString int = 1 + iota
|
||||
)
|
||||
|
||||
func goPanic(code int, val unsafe.Pointer) {
|
||||
switch(code){
|
||||
case panicNilPointerOfNonEmptyString:
|
||||
panic(fmt.Sprintf("val: %#v has nil pointer while its length is not zero!", (*rt.GoString)(val)))
|
||||
default:
|
||||
panic("encoder error!")
|
||||
}
|
||||
}
|
||||
199
vendor/github.com/bytedance/sonic/internal/encoder/mapiter.go
generated
vendored
Normal file
199
vendor/github.com/bytedance/sonic/internal/encoder/mapiter.go
generated
vendored
Normal file
@@ -0,0 +1,199 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
"encoding"
|
||||
"reflect"
|
||||
"sync"
|
||||
"unsafe"
|
||||
|
||||
"github.com/bytedance/sonic/internal/native"
|
||||
"github.com/bytedance/sonic/internal/rt"
|
||||
)
|
||||
|
||||
type _MapPair struct {
|
||||
k string // when the map key is integer, k is pointed to m
|
||||
v unsafe.Pointer
|
||||
m [32]byte
|
||||
}
|
||||
|
||||
type _MapIterator struct {
|
||||
it rt.GoMapIterator // must be the first field
|
||||
kv rt.GoSlice // slice of _MapPair
|
||||
ki int
|
||||
}
|
||||
|
||||
var (
|
||||
iteratorPool = sync.Pool{}
|
||||
iteratorPair = rt.UnpackType(reflect.TypeOf(_MapPair{}))
|
||||
)
|
||||
|
||||
func init() {
|
||||
if unsafe.Offsetof(_MapIterator{}.it) != 0 {
|
||||
panic("_MapIterator.it is not the first field")
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
func newIterator() *_MapIterator {
|
||||
if v := iteratorPool.Get(); v == nil {
|
||||
return new(_MapIterator)
|
||||
} else {
|
||||
return resetIterator(v.(*_MapIterator))
|
||||
}
|
||||
}
|
||||
|
||||
func resetIterator(p *_MapIterator) *_MapIterator {
|
||||
p.ki = 0
|
||||
p.it = rt.GoMapIterator{}
|
||||
p.kv.Len = 0
|
||||
return p
|
||||
}
|
||||
|
||||
func (self *_MapIterator) at(i int) *_MapPair {
|
||||
return (*_MapPair)(unsafe.Pointer(uintptr(self.kv.Ptr) + uintptr(i) * unsafe.Sizeof(_MapPair{})))
|
||||
}
|
||||
|
||||
func (self *_MapIterator) add() (p *_MapPair) {
|
||||
p = self.at(self.kv.Len)
|
||||
self.kv.Len++
|
||||
return
|
||||
}
|
||||
|
||||
func (self *_MapIterator) data() (p []_MapPair) {
|
||||
*(*rt.GoSlice)(unsafe.Pointer(&p)) = self.kv
|
||||
return
|
||||
}
|
||||
|
||||
func (self *_MapIterator) append(t *rt.GoType, k unsafe.Pointer, v unsafe.Pointer) (err error) {
|
||||
p := self.add()
|
||||
p.v = v
|
||||
|
||||
/* check for strings */
|
||||
if tk := t.Kind(); tk != reflect.String {
|
||||
return self.appendGeneric(p, t, tk, k)
|
||||
}
|
||||
|
||||
/* fast path for strings */
|
||||
p.k = *(*string)(k)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (self *_MapIterator) appendGeneric(p *_MapPair, t *rt.GoType, v reflect.Kind, k unsafe.Pointer) error {
|
||||
switch v {
|
||||
case reflect.Int : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int)(k)))]) ; return nil
|
||||
case reflect.Int8 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int8)(k)))]) ; return nil
|
||||
case reflect.Int16 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int16)(k)))]) ; return nil
|
||||
case reflect.Int32 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int32)(k)))]) ; return nil
|
||||
case reflect.Int64 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], *(*int64)(k))]) ; return nil
|
||||
case reflect.Uint : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint)(k)))]) ; return nil
|
||||
case reflect.Uint8 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint8)(k)))]) ; return nil
|
||||
case reflect.Uint16 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint16)(k)))]) ; return nil
|
||||
case reflect.Uint32 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint32)(k)))]) ; return nil
|
||||
case reflect.Uint64 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], *(*uint64)(k))]) ; return nil
|
||||
case reflect.Uintptr : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uintptr)(k)))]) ; return nil
|
||||
case reflect.Interface : return self.appendInterface(p, t, k)
|
||||
case reflect.Struct, reflect.Ptr : return self.appendConcrete(p, t, k)
|
||||
default : panic("unexpected map key type")
|
||||
}
|
||||
}
|
||||
|
||||
func (self *_MapIterator) appendConcrete(p *_MapPair, t *rt.GoType, k unsafe.Pointer) (err error) {
|
||||
// compiler has already checked that the type implements the encoding.MarshalText interface
|
||||
if !t.Indirect() {
|
||||
k = *(*unsafe.Pointer)(k)
|
||||
}
|
||||
eface := rt.GoEface{Value: k, Type: t}.Pack()
|
||||
out, err := eface.(encoding.TextMarshaler).MarshalText()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.k = rt.Mem2Str(out)
|
||||
return
|
||||
}
|
||||
|
||||
func (self *_MapIterator) appendInterface(p *_MapPair, t *rt.GoType, k unsafe.Pointer) (err error) {
|
||||
if len(rt.IfaceType(t).Methods) == 0 {
|
||||
panic("unexpected map key type")
|
||||
} else if p.k, err = asText(k); err == nil {
|
||||
return nil
|
||||
} else {
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
func iteratorStop(p *_MapIterator) {
|
||||
iteratorPool.Put(p)
|
||||
}
|
||||
|
||||
func iteratorNext(p *_MapIterator) {
|
||||
i := p.ki
|
||||
t := &p.it
|
||||
|
||||
/* check for unordered iteration */
|
||||
if i < 0 {
|
||||
mapiternext(t)
|
||||
return
|
||||
}
|
||||
|
||||
/* check for end of iteration */
|
||||
if p.ki >= p.kv.Len {
|
||||
t.K = nil
|
||||
t.V = nil
|
||||
return
|
||||
}
|
||||
|
||||
/* update the key-value pair, and increase the pointer */
|
||||
t.K = unsafe.Pointer(&p.at(p.ki).k)
|
||||
t.V = p.at(p.ki).v
|
||||
p.ki++
|
||||
}
|
||||
|
||||
func iteratorStart(t *rt.GoMapType, m *rt.GoMap, fv uint64) (*_MapIterator, error) {
|
||||
it := newIterator()
|
||||
mapiterinit(t, m, &it.it)
|
||||
|
||||
/* check for key-sorting, empty map don't need sorting */
|
||||
if m.Count == 0 || (fv & uint64(SortMapKeys)) == 0 {
|
||||
it.ki = -1
|
||||
return it, nil
|
||||
}
|
||||
|
||||
/* pre-allocate space if needed */
|
||||
if m.Count > it.kv.Cap {
|
||||
it.kv = growslice(iteratorPair, it.kv, m.Count)
|
||||
}
|
||||
|
||||
/* dump all the key-value pairs */
|
||||
for ; it.it.K != nil; mapiternext(&it.it) {
|
||||
if err := it.append(t.Key, it.it.K, it.it.V); err != nil {
|
||||
iteratorStop(it)
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
/* sort the keys, map with only 1 item don't need sorting */
|
||||
if it.ki = 1; m.Count > 1 {
|
||||
radixQsort(it.data(), 0, maxDepth(it.kv.Len))
|
||||
}
|
||||
|
||||
/* load the first pair into iterator */
|
||||
it.it.V = it.at(0).v
|
||||
it.it.K = unsafe.Pointer(&it.at(0).k)
|
||||
return it, nil
|
||||
}
|
||||
193
vendor/github.com/bytedance/sonic/internal/encoder/pools.go
generated
vendored
Normal file
193
vendor/github.com/bytedance/sonic/internal/encoder/pools.go
generated
vendored
Normal file
@@ -0,0 +1,193 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`bytes`
|
||||
`sync`
|
||||
`unsafe`
|
||||
`errors`
|
||||
`reflect`
|
||||
|
||||
`github.com/bytedance/sonic/internal/caching`
|
||||
`github.com/bytedance/sonic/option`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
const (
|
||||
_MaxStack = 4096 // 4k states
|
||||
|
||||
_StackSize = unsafe.Sizeof(_Stack{})
|
||||
)
|
||||
|
||||
var (
|
||||
bytesPool = sync.Pool{}
|
||||
stackPool = sync.Pool{}
|
||||
bufferPool = sync.Pool{}
|
||||
programCache = caching.CreateProgramCache()
|
||||
)
|
||||
|
||||
type _State struct {
|
||||
x int
|
||||
f uint64
|
||||
p unsafe.Pointer
|
||||
q unsafe.Pointer
|
||||
}
|
||||
|
||||
type _Stack struct {
|
||||
sp uint64
|
||||
sb [_MaxStack]_State
|
||||
}
|
||||
|
||||
type _Encoder func(
|
||||
rb *[]byte,
|
||||
vp unsafe.Pointer,
|
||||
sb *_Stack,
|
||||
fv uint64,
|
||||
) error
|
||||
|
||||
var _KeepAlive struct {
|
||||
rb *[]byte
|
||||
vp unsafe.Pointer
|
||||
sb *_Stack
|
||||
fv uint64
|
||||
err error
|
||||
frame [_FP_offs]byte
|
||||
}
|
||||
|
||||
var errCallShadow = errors.New("DON'T CALL THIS!")
|
||||
|
||||
// Faker func of _Encoder, used to export its stackmap as _Encoder's
|
||||
func _Encoder_Shadow(rb *[]byte, vp unsafe.Pointer, sb *_Stack, fv uint64) (err error) {
|
||||
// align to assembler_amd64.go: _FP_offs
|
||||
var frame [_FP_offs]byte
|
||||
|
||||
// must keep all args and frames noticeable to GC
|
||||
_KeepAlive.rb = rb
|
||||
_KeepAlive.vp = vp
|
||||
_KeepAlive.sb = sb
|
||||
_KeepAlive.fv = fv
|
||||
_KeepAlive.err = err
|
||||
_KeepAlive.frame = frame
|
||||
|
||||
return errCallShadow
|
||||
}
|
||||
|
||||
func newBytes() []byte {
|
||||
if ret := bytesPool.Get(); ret != nil {
|
||||
return ret.([]byte)
|
||||
} else {
|
||||
return make([]byte, 0, option.DefaultEncoderBufferSize)
|
||||
}
|
||||
}
|
||||
|
||||
func newStack() *_Stack {
|
||||
if ret := stackPool.Get(); ret == nil {
|
||||
return new(_Stack)
|
||||
} else {
|
||||
return ret.(*_Stack)
|
||||
}
|
||||
}
|
||||
|
||||
func resetStack(p *_Stack) {
|
||||
memclrNoHeapPointers(unsafe.Pointer(p), _StackSize)
|
||||
}
|
||||
|
||||
func newBuffer() *bytes.Buffer {
|
||||
if ret := bufferPool.Get(); ret != nil {
|
||||
return ret.(*bytes.Buffer)
|
||||
} else {
|
||||
return bytes.NewBuffer(make([]byte, 0, option.DefaultEncoderBufferSize))
|
||||
}
|
||||
}
|
||||
|
||||
func freeBytes(p []byte) {
|
||||
p = p[:0]
|
||||
bytesPool.Put(p)
|
||||
}
|
||||
|
||||
func freeStack(p *_Stack) {
|
||||
p.sp = 0
|
||||
stackPool.Put(p)
|
||||
}
|
||||
|
||||
func freeBuffer(p *bytes.Buffer) {
|
||||
p.Reset()
|
||||
bufferPool.Put(p)
|
||||
}
|
||||
|
||||
func makeEncoder(vt *rt.GoType, ex ...interface{}) (interface{}, error) {
|
||||
if pp, err := newCompiler().compile(vt.Pack(), ex[0].(bool)); err != nil {
|
||||
return nil, err
|
||||
} else {
|
||||
as := newAssembler(pp)
|
||||
as.name = vt.String()
|
||||
return as.Load(), nil
|
||||
}
|
||||
}
|
||||
|
||||
func findOrCompile(vt *rt.GoType, pv bool) (_Encoder, error) {
|
||||
if val := programCache.Get(vt); val != nil {
|
||||
return val.(_Encoder), nil
|
||||
} else if ret, err := programCache.Compute(vt, makeEncoder, pv); err == nil {
|
||||
return ret.(_Encoder), nil
|
||||
} else {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
func pretouchType(_vt reflect.Type, opts option.CompileOptions, v uint8) (map[reflect.Type]uint8, error) {
|
||||
/* compile function */
|
||||
compiler := newCompiler().apply(opts)
|
||||
encoder := func(vt *rt.GoType, ex ...interface{}) (interface{}, error) {
|
||||
if pp, err := compiler.compile(_vt, ex[0].(bool)); err != nil {
|
||||
return nil, err
|
||||
} else {
|
||||
as := newAssembler(pp)
|
||||
as.name = vt.String()
|
||||
return as.Load(), nil
|
||||
}
|
||||
}
|
||||
|
||||
/* find or compile */
|
||||
vt := rt.UnpackType(_vt)
|
||||
if val := programCache.Get(vt); val != nil {
|
||||
return nil, nil
|
||||
} else if _, err := programCache.Compute(vt, encoder, v == 1); err == nil {
|
||||
return compiler.rec, nil
|
||||
} else {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
|
||||
func pretouchRec(vtm map[reflect.Type]uint8, opts option.CompileOptions) error {
|
||||
if opts.RecursiveDepth < 0 || len(vtm) == 0 {
|
||||
return nil
|
||||
}
|
||||
next := make(map[reflect.Type]uint8)
|
||||
for vt, v := range vtm {
|
||||
sub, err := pretouchType(vt, opts, v)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for svt, v := range sub {
|
||||
next[svt] = v
|
||||
}
|
||||
}
|
||||
opts.RecursiveDepth -= 1
|
||||
return pretouchRec(next, opts)
|
||||
}
|
||||
168
vendor/github.com/bytedance/sonic/internal/encoder/primitives.go
generated
vendored
Normal file
168
vendor/github.com/bytedance/sonic/internal/encoder/primitives.go
generated
vendored
Normal file
@@ -0,0 +1,168 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`encoding`
|
||||
`encoding/json`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/internal/jit`
|
||||
`github.com/bytedance/sonic/internal/native`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
/** Encoder Primitives **/
|
||||
|
||||
func encodeNil(rb *[]byte) error {
|
||||
*rb = append(*rb, 'n', 'u', 'l', 'l')
|
||||
return nil
|
||||
}
|
||||
|
||||
func encodeString(buf *[]byte, val string) error {
|
||||
var sidx int
|
||||
var pbuf *rt.GoSlice
|
||||
var pstr *rt.GoString
|
||||
|
||||
/* opening quote */
|
||||
*buf = append(*buf, '"')
|
||||
pbuf = (*rt.GoSlice)(unsafe.Pointer(buf))
|
||||
pstr = (*rt.GoString)(unsafe.Pointer(&val))
|
||||
|
||||
/* encode with native library */
|
||||
for sidx < pstr.Len {
|
||||
sn := pstr.Len - sidx
|
||||
dn := pbuf.Cap - pbuf.Len
|
||||
sp := padd(pstr.Ptr, sidx)
|
||||
dp := padd(pbuf.Ptr, pbuf.Len)
|
||||
nb := native.Quote(sp, sn, dp, &dn, 0)
|
||||
|
||||
/* check for errors */
|
||||
if pbuf.Len += dn; nb >= 0 {
|
||||
break
|
||||
}
|
||||
|
||||
/* not enough space, grow the slice and try again */
|
||||
sidx += ^nb
|
||||
*pbuf = growslice(rt.UnpackType(byteType), *pbuf, pbuf.Cap * 2)
|
||||
}
|
||||
|
||||
/* closing quote */
|
||||
*buf = append(*buf, '"')
|
||||
return nil
|
||||
}
|
||||
|
||||
func encodeTypedPointer(buf *[]byte, vt *rt.GoType, vp *unsafe.Pointer, sb *_Stack, fv uint64) error {
|
||||
if vt == nil {
|
||||
return encodeNil(buf)
|
||||
} else if fn, err := findOrCompile(vt, (fv&(1<<bitPointerValue)) != 0); err != nil {
|
||||
return err
|
||||
} else if vt.Indirect() {
|
||||
rt.MoreStack(_FP_size + native.MaxFrameSize)
|
||||
rt.StopProf()
|
||||
err := fn(buf, *vp, sb, fv)
|
||||
rt.StartProf()
|
||||
return err
|
||||
} else {
|
||||
rt.MoreStack(_FP_size + native.MaxFrameSize)
|
||||
rt.StopProf()
|
||||
err := fn(buf, unsafe.Pointer(vp), sb, fv)
|
||||
rt.StartProf()
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
func encodeJsonMarshaler(buf *[]byte, val json.Marshaler, opt Options) error {
|
||||
if ret, err := val.MarshalJSON(); err != nil {
|
||||
return err
|
||||
} else {
|
||||
if opt & CompactMarshaler != 0 {
|
||||
return compact(buf, ret)
|
||||
}
|
||||
if ok, s := Valid(ret); !ok {
|
||||
return error_marshaler(ret, s)
|
||||
}
|
||||
*buf = append(*buf, ret...)
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func encodeTextMarshaler(buf *[]byte, val encoding.TextMarshaler, opt Options) error {
|
||||
if ret, err := val.MarshalText(); err != nil {
|
||||
return err
|
||||
} else {
|
||||
if opt & NoQuoteTextMarshaler != 0 {
|
||||
*buf = append(*buf, ret...)
|
||||
return nil
|
||||
}
|
||||
return encodeString(buf, rt.Mem2Str(ret) )
|
||||
}
|
||||
}
|
||||
|
||||
func htmlEscape(dst []byte, src []byte) []byte {
|
||||
var sidx int
|
||||
|
||||
dst = append(dst, src[:0]...) // avoid check nil dst
|
||||
sbuf := (*rt.GoSlice)(unsafe.Pointer(&src))
|
||||
dbuf := (*rt.GoSlice)(unsafe.Pointer(&dst))
|
||||
|
||||
/* grow dst if it is shorter */
|
||||
if cap(dst) - len(dst) < len(src) + native.BufPaddingSize {
|
||||
cap := len(src) * 3 / 2 + native.BufPaddingSize
|
||||
*dbuf = growslice(typeByte, *dbuf, cap)
|
||||
}
|
||||
|
||||
for sidx < sbuf.Len {
|
||||
sp := padd(sbuf.Ptr, sidx)
|
||||
dp := padd(dbuf.Ptr, dbuf.Len)
|
||||
|
||||
sn := sbuf.Len - sidx
|
||||
dn := dbuf.Cap - dbuf.Len
|
||||
nb := native.HTMLEscape(sp, sn, dp, &dn)
|
||||
|
||||
/* check for errors */
|
||||
if dbuf.Len += dn; nb >= 0 {
|
||||
break
|
||||
}
|
||||
|
||||
/* not enough space, grow the slice and try again */
|
||||
sidx += ^nb
|
||||
*dbuf = growslice(typeByte, *dbuf, dbuf.Cap * 2)
|
||||
}
|
||||
return dst
|
||||
}
|
||||
|
||||
var (
|
||||
argPtrs = []bool { true, true, true, false }
|
||||
localPtrs = []bool{}
|
||||
)
|
||||
|
||||
var (
|
||||
_F_assertI2I = jit.Func(assertI2I)
|
||||
)
|
||||
|
||||
func asText(v unsafe.Pointer) (string, error) {
|
||||
text := assertI2I(_T_encoding_TextMarshaler, *(*rt.GoIface)(v))
|
||||
r, e := (*(*encoding.TextMarshaler)(unsafe.Pointer(&text))).MarshalText()
|
||||
return rt.Mem2Str(r), e
|
||||
}
|
||||
|
||||
func asJson(v unsafe.Pointer) (string, error) {
|
||||
text := assertI2I(_T_json_Marshaler, *(*rt.GoIface)(v))
|
||||
r, e := (*(*json.Marshaler)(unsafe.Pointer(&text))).MarshalJSON()
|
||||
return rt.Mem2Str(r), e
|
||||
}
|
||||
206
vendor/github.com/bytedance/sonic/internal/encoder/sort.go
generated
vendored
Normal file
206
vendor/github.com/bytedance/sonic/internal/encoder/sort.go
generated
vendored
Normal file
@@ -0,0 +1,206 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
// Algorithm 3-way Radix Quicksort, d means the radix.
|
||||
// Reference: https://algs4.cs.princeton.edu/51radix/Quick3string.java.html
|
||||
func radixQsort(kvs []_MapPair, d, maxDepth int) {
|
||||
for len(kvs) > 11 {
|
||||
// To avoid the worst case of quickSort (time: O(n^2)), use introsort here.
|
||||
// Reference: https://en.wikipedia.org/wiki/Introsort and
|
||||
// https://github.com/golang/go/issues/467
|
||||
if maxDepth == 0 {
|
||||
heapSort(kvs, 0, len(kvs))
|
||||
return
|
||||
}
|
||||
maxDepth--
|
||||
|
||||
p := pivot(kvs, d)
|
||||
lt, i, gt := 0, 0, len(kvs)
|
||||
for i < gt {
|
||||
c := byteAt(kvs[i].k, d)
|
||||
if c < p {
|
||||
swap(kvs, lt, i)
|
||||
i++
|
||||
lt++
|
||||
} else if c > p {
|
||||
gt--
|
||||
swap(kvs, i, gt)
|
||||
} else {
|
||||
i++
|
||||
}
|
||||
}
|
||||
|
||||
// kvs[0:lt] < v = kvs[lt:gt] < kvs[gt:len(kvs)]
|
||||
// Native implemention:
|
||||
// radixQsort(kvs[:lt], d, maxDepth)
|
||||
// if p > -1 {
|
||||
// radixQsort(kvs[lt:gt], d+1, maxDepth)
|
||||
// }
|
||||
// radixQsort(kvs[gt:], d, maxDepth)
|
||||
// Optimize as follows: make recursive calls only for the smaller parts.
|
||||
// Reference: https://www.geeksforgeeks.org/quicksort-tail-call-optimization-reducing-worst-case-space-log-n/
|
||||
if p == -1 {
|
||||
if lt > len(kvs) - gt {
|
||||
radixQsort(kvs[gt:], d, maxDepth)
|
||||
kvs = kvs[:lt]
|
||||
} else {
|
||||
radixQsort(kvs[:lt], d, maxDepth)
|
||||
kvs = kvs[gt:]
|
||||
}
|
||||
} else {
|
||||
ml := maxThree(lt, gt-lt, len(kvs)-gt)
|
||||
if ml == lt {
|
||||
radixQsort(kvs[lt:gt], d+1, maxDepth)
|
||||
radixQsort(kvs[gt:], d, maxDepth)
|
||||
kvs = kvs[:lt]
|
||||
} else if ml == gt-lt {
|
||||
radixQsort(kvs[:lt], d, maxDepth)
|
||||
radixQsort(kvs[gt:], d, maxDepth)
|
||||
kvs = kvs[lt:gt]
|
||||
d += 1
|
||||
} else {
|
||||
radixQsort(kvs[:lt], d, maxDepth)
|
||||
radixQsort(kvs[lt:gt], d+1, maxDepth)
|
||||
kvs = kvs[gt:]
|
||||
}
|
||||
}
|
||||
}
|
||||
insertRadixSort(kvs, d)
|
||||
}
|
||||
|
||||
func insertRadixSort(kvs []_MapPair, d int) {
|
||||
for i := 1; i < len(kvs); i++ {
|
||||
for j := i; j > 0 && lessFrom(kvs[j].k, kvs[j-1].k, d); j-- {
|
||||
swap(kvs, j, j-1)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func pivot(kvs []_MapPair, d int) int {
|
||||
m := len(kvs) >> 1
|
||||
if len(kvs) > 40 {
|
||||
// Tukey's ``Ninther,'' median of three mediankvs of three.
|
||||
t := len(kvs) / 8
|
||||
return medianThree(
|
||||
medianThree(byteAt(kvs[0].k, d), byteAt(kvs[t].k, d), byteAt(kvs[2*t].k, d)),
|
||||
medianThree(byteAt(kvs[m].k, d), byteAt(kvs[m-t].k, d), byteAt(kvs[m+t].k, d)),
|
||||
medianThree(byteAt(kvs[len(kvs)-1].k, d),
|
||||
byteAt(kvs[len(kvs)-1-t].k, d),
|
||||
byteAt(kvs[len(kvs)-1-2*t].k, d)))
|
||||
}
|
||||
return medianThree(byteAt(kvs[0].k, d), byteAt(kvs[m].k, d), byteAt(kvs[len(kvs)-1].k, d))
|
||||
}
|
||||
|
||||
func medianThree(i, j, k int) int {
|
||||
if i > j {
|
||||
i, j = j, i
|
||||
} // i < j
|
||||
if k < i {
|
||||
return i
|
||||
}
|
||||
if k > j {
|
||||
return j
|
||||
}
|
||||
return k
|
||||
}
|
||||
|
||||
func maxThree(i, j, k int) int {
|
||||
max := i
|
||||
if max < j {
|
||||
max = j
|
||||
}
|
||||
if max < k {
|
||||
max = k
|
||||
}
|
||||
return max
|
||||
}
|
||||
|
||||
// maxDepth returns a threshold at which quicksort should switch
|
||||
// to heapsort. It returnkvs 2*ceil(lg(n+1)).
|
||||
func maxDepth(n int) int {
|
||||
var depth int
|
||||
for i := n; i > 0; i >>= 1 {
|
||||
depth++
|
||||
}
|
||||
return depth * 2
|
||||
}
|
||||
|
||||
// siftDown implements the heap property on kvs[lo:hi].
|
||||
// first is an offset into the array where the root of the heap lies.
|
||||
func siftDown(kvs []_MapPair, lo, hi, first int) {
|
||||
root := lo
|
||||
for {
|
||||
child := 2*root + 1
|
||||
if child >= hi {
|
||||
break
|
||||
}
|
||||
if child+1 < hi && kvs[first+child].k < kvs[first+child+1].k {
|
||||
child++
|
||||
}
|
||||
if kvs[first+root].k >= kvs[first+child].k {
|
||||
return
|
||||
}
|
||||
swap(kvs, first+root, first+child)
|
||||
root = child
|
||||
}
|
||||
}
|
||||
|
||||
func heapSort(kvs []_MapPair, a, b int) {
|
||||
first := a
|
||||
lo := 0
|
||||
hi := b - a
|
||||
|
||||
// Build heap with the greatest element at top.
|
||||
for i := (hi - 1) / 2; i >= 0; i-- {
|
||||
siftDown(kvs, i, hi, first)
|
||||
}
|
||||
|
||||
// Pop elements, the largest first, into end of kvs.
|
||||
for i := hi - 1; i >= 0; i-- {
|
||||
swap(kvs, first, first+i)
|
||||
siftDown(kvs, lo, i, first)
|
||||
}
|
||||
}
|
||||
|
||||
// Note that _MapPair.k is NOT pointed to _MapPair.m when map key is integer after swap
|
||||
func swap(kvs []_MapPair, a, b int) {
|
||||
kvs[a].k, kvs[b].k = kvs[b].k, kvs[a].k
|
||||
kvs[a].v, kvs[b].v = kvs[b].v, kvs[a].v
|
||||
}
|
||||
|
||||
// Compare two strings from the pos d.
|
||||
func lessFrom(a, b string, d int) bool {
|
||||
l := len(a)
|
||||
if l > len(b) {
|
||||
l = len(b)
|
||||
}
|
||||
for i := d; i < l; i++ {
|
||||
if a[i] == b[i] {
|
||||
continue
|
||||
}
|
||||
return a[i] < b[i]
|
||||
}
|
||||
return len(a) < len(b)
|
||||
}
|
||||
|
||||
func byteAt(b string, p int) int {
|
||||
if p < len(b) {
|
||||
return int(b[p])
|
||||
}
|
||||
return -1
|
||||
}
|
||||
84
vendor/github.com/bytedance/sonic/internal/encoder/stream.go
generated
vendored
Normal file
84
vendor/github.com/bytedance/sonic/internal/encoder/stream.go
generated
vendored
Normal file
@@ -0,0 +1,84 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`encoding/json`
|
||||
`io`
|
||||
)
|
||||
|
||||
// StreamEncoder uses io.Writer as input.
|
||||
type StreamEncoder struct {
|
||||
w io.Writer
|
||||
Encoder
|
||||
}
|
||||
|
||||
// NewStreamEncoder adapts to encoding/json.NewDecoder API.
|
||||
//
|
||||
// NewStreamEncoder returns a new encoder that write to w.
|
||||
func NewStreamEncoder(w io.Writer) *StreamEncoder {
|
||||
return &StreamEncoder{w: w}
|
||||
}
|
||||
|
||||
// Encode encodes interface{} as JSON to io.Writer
|
||||
func (enc *StreamEncoder) Encode(val interface{}) (err error) {
|
||||
out := newBytes()
|
||||
|
||||
/* encode into the buffer */
|
||||
err = EncodeInto(&out, val, enc.Opts)
|
||||
if err != nil {
|
||||
goto free_bytes
|
||||
}
|
||||
|
||||
if enc.indent != "" || enc.prefix != "" {
|
||||
/* indent the JSON */
|
||||
buf := newBuffer()
|
||||
err = json.Indent(buf, out, enc.prefix, enc.indent)
|
||||
if err != nil {
|
||||
freeBuffer(buf)
|
||||
goto free_bytes
|
||||
}
|
||||
|
||||
// according to standard library, terminate each value with a newline...
|
||||
buf.WriteByte('\n')
|
||||
|
||||
/* copy into io.Writer */
|
||||
_, err = io.Copy(enc.w, buf)
|
||||
if err != nil {
|
||||
freeBuffer(buf)
|
||||
goto free_bytes
|
||||
}
|
||||
|
||||
} else {
|
||||
/* copy into io.Writer */
|
||||
var n int
|
||||
for len(out) > 0 {
|
||||
n, err = enc.w.Write(out)
|
||||
out = out[n:]
|
||||
if err != nil {
|
||||
goto free_bytes
|
||||
}
|
||||
}
|
||||
|
||||
// according to standard library, terminate each value with a newline...
|
||||
enc.w.Write([]byte{'\n'})
|
||||
}
|
||||
|
||||
free_bytes:
|
||||
freeBytes(out)
|
||||
return err
|
||||
}
|
||||
65
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go116.go
generated
vendored
Normal file
65
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go116.go
generated
vendored
Normal file
@@ -0,0 +1,65 @@
|
||||
// +build go1.15,!go1.17
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
|
||||
_ `github.com/chenzhuoyu/base64x`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
//go:linkname _subr__b64encode github.com/chenzhuoyu/base64x._subr__b64encode
|
||||
var _subr__b64encode uintptr
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memmove runtime.memmove
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname growslice runtime.growslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
|
||||
|
||||
//go:linkname assertI2I runtime.assertI2I
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func assertI2I(inter *rt.GoType, i rt.GoIface) rt.GoIface
|
||||
|
||||
//go:linkname mapiternext runtime.mapiternext
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapiternext(it *rt.GoMapIterator)
|
||||
|
||||
//go:linkname mapiterinit runtime.mapiterinit
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapiterinit(t *rt.GoMapType, m *rt.GoMap, it *rt.GoMapIterator)
|
||||
|
||||
//go:linkname isValidNumber encoding/json.isValidNumber
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func isValidNumber(s string) bool
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
|
||||
|
||||
var _runtime_writeBarrier uintptr = rt.GcwbAddr()
|
||||
|
||||
//go:linkname gcWriteBarrierAX runtime.gcWriteBarrier
|
||||
func gcWriteBarrierAX()
|
||||
66
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go117.go
generated
vendored
Normal file
66
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go117.go
generated
vendored
Normal file
@@ -0,0 +1,66 @@
|
||||
// +build go1.17,!go1.20
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
|
||||
_ `github.com/chenzhuoyu/base64x`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
//go:linkname _subr__b64encode github.com/chenzhuoyu/base64x._subr__b64encode
|
||||
var _subr__b64encode uintptr
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memmove runtime.memmove
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname growslice runtime.growslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
|
||||
|
||||
//go:linkname assertI2I runtime.assertI2I2
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func assertI2I(inter *rt.GoType, i rt.GoIface) rt.GoIface
|
||||
|
||||
//go:linkname mapiternext runtime.mapiternext
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapiternext(it *rt.GoMapIterator)
|
||||
|
||||
//go:linkname mapiterinit runtime.mapiterinit
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapiterinit(t *rt.GoMapType, m *rt.GoMap, it *rt.GoMapIterator)
|
||||
|
||||
//go:linkname isValidNumber encoding/json.isValidNumber
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func isValidNumber(s string) bool
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname _runtime_writeBarrier runtime.writeBarrier
|
||||
var _runtime_writeBarrier uintptr
|
||||
|
||||
//go:linkname gcWriteBarrierAX runtime.gcWriteBarrier
|
||||
func gcWriteBarrierAX()
|
||||
66
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go120.go
generated
vendored
Normal file
66
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go120.go
generated
vendored
Normal file
@@ -0,0 +1,66 @@
|
||||
// +build go1.20
|
||||
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`unsafe`
|
||||
|
||||
_ `github.com/chenzhuoyu/base64x`
|
||||
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
)
|
||||
|
||||
//go:linkname _subr__b64encode github.com/chenzhuoyu/base64x._subr__b64encode
|
||||
var _subr__b64encode uintptr
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memmove runtime.memmove
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname growslice reflect.growslice
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
|
||||
|
||||
//go:linkname assertI2I runtime.assertI2I2
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func assertI2I(inter *rt.GoType, i rt.GoIface) rt.GoIface
|
||||
|
||||
//go:linkname mapiternext runtime.mapiternext
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapiternext(it *rt.GoMapIterator)
|
||||
|
||||
//go:linkname mapiterinit runtime.mapiterinit
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func mapiterinit(t *rt.GoMapType, m *rt.GoMap, it *rt.GoMapIterator)
|
||||
|
||||
//go:linkname isValidNumber encoding/json.isValidNumber
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func isValidNumber(s string) bool
|
||||
|
||||
//go:noescape
|
||||
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
|
||||
//goland:noinspection GoUnusedParameter
|
||||
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
|
||||
|
||||
//go:linkname _runtime_writeBarrier runtime.writeBarrier
|
||||
var _runtime_writeBarrier uintptr
|
||||
|
||||
//go:linkname gcWriteBarrierAX runtime.gcWriteBarrier
|
||||
func gcWriteBarrierAX()
|
||||
47
vendor/github.com/bytedance/sonic/internal/encoder/types.go
generated
vendored
Normal file
47
vendor/github.com/bytedance/sonic/internal/encoder/types.go
generated
vendored
Normal file
@@ -0,0 +1,47 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`encoding`
|
||||
`encoding/json`
|
||||
`reflect`
|
||||
)
|
||||
|
||||
var (
|
||||
byteType = reflect.TypeOf(byte(0))
|
||||
jsonNumberType = reflect.TypeOf(json.Number(""))
|
||||
jsonUnsupportedValueType = reflect.TypeOf(new(json.UnsupportedValueError))
|
||||
)
|
||||
|
||||
var (
|
||||
errorType = reflect.TypeOf((*error)(nil)).Elem()
|
||||
jsonMarshalerType = reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||
encodingTextMarshalerType = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
|
||||
)
|
||||
|
||||
func isSimpleByte(vt reflect.Type) bool {
|
||||
if vt.Kind() != byteType.Kind() {
|
||||
return false
|
||||
} else {
|
||||
return !isEitherMarshaler(vt) && !isEitherMarshaler(reflect.PtrTo(vt))
|
||||
}
|
||||
}
|
||||
|
||||
func isEitherMarshaler(vt reflect.Type) bool {
|
||||
return vt.Implements(jsonMarshalerType) || vt.Implements(encodingTextMarshalerType)
|
||||
}
|
||||
52
vendor/github.com/bytedance/sonic/internal/encoder/utils.go
generated
vendored
Normal file
52
vendor/github.com/bytedance/sonic/internal/encoder/utils.go
generated
vendored
Normal file
@@ -0,0 +1,52 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package encoder
|
||||
|
||||
import (
|
||||
`encoding/json`
|
||||
`unsafe`
|
||||
|
||||
`github.com/bytedance/sonic/loader`
|
||||
)
|
||||
|
||||
//go:nosplit
|
||||
func padd(p unsafe.Pointer, v int) unsafe.Pointer {
|
||||
return unsafe.Pointer(uintptr(p) + uintptr(v))
|
||||
}
|
||||
|
||||
//go:nosplit
|
||||
func ptoenc(p loader.Function) _Encoder {
|
||||
return *(*_Encoder)(unsafe.Pointer(&p))
|
||||
}
|
||||
|
||||
func compact(p *[]byte, v []byte) error {
|
||||
buf := newBuffer()
|
||||
err := json.Compact(buf, v)
|
||||
|
||||
/* check for errors */
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
/* add to result */
|
||||
v = buf.Bytes()
|
||||
*p = append(*p, v...)
|
||||
|
||||
/* return the buffer into pool */
|
||||
freeBuffer(buf)
|
||||
return nil
|
||||
}
|
||||
67
vendor/github.com/bytedance/sonic/internal/jit/arch_amd64.go
generated
vendored
Normal file
67
vendor/github.com/bytedance/sonic/internal/jit/arch_amd64.go
generated
vendored
Normal file
@@ -0,0 +1,67 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package jit
|
||||
|
||||
import (
|
||||
`github.com/twitchyliquid64/golang-asm/asm/arch`
|
||||
`github.com/twitchyliquid64/golang-asm/obj`
|
||||
)
|
||||
|
||||
var (
|
||||
_AC = arch.Set("amd64")
|
||||
)
|
||||
|
||||
func As(op string) obj.As {
|
||||
if ret, ok := _AC.Instructions[op]; ok {
|
||||
return ret
|
||||
} else {
|
||||
panic("invalid instruction: " + op)
|
||||
}
|
||||
}
|
||||
|
||||
func Imm(imm int64) obj.Addr {
|
||||
return obj.Addr {
|
||||
Type : obj.TYPE_CONST,
|
||||
Offset : imm,
|
||||
}
|
||||
}
|
||||
|
||||
func Reg(reg string) obj.Addr {
|
||||
if ret, ok := _AC.Register[reg]; ok {
|
||||
return obj.Addr{Reg: ret, Type: obj.TYPE_REG}
|
||||
} else {
|
||||
panic("invalid register name: " + reg)
|
||||
}
|
||||
}
|
||||
|
||||
func Ptr(reg obj.Addr, offs int64) obj.Addr {
|
||||
return obj.Addr {
|
||||
Reg : reg.Reg,
|
||||
Type : obj.TYPE_MEM,
|
||||
Offset : offs,
|
||||
}
|
||||
}
|
||||
|
||||
func Sib(reg obj.Addr, idx obj.Addr, scale int16, offs int64) obj.Addr {
|
||||
return obj.Addr {
|
||||
Reg : reg.Reg,
|
||||
Index : idx.Reg,
|
||||
Scale : scale,
|
||||
Type : obj.TYPE_MEM,
|
||||
Offset : offs,
|
||||
}
|
||||
}
|
||||
0
vendor/github.com/bytedance/sonic/internal/jit/asm.s
generated
vendored
Normal file
0
vendor/github.com/bytedance/sonic/internal/jit/asm.s
generated
vendored
Normal file
281
vendor/github.com/bytedance/sonic/internal/jit/assembler_amd64.go
generated
vendored
Normal file
281
vendor/github.com/bytedance/sonic/internal/jit/assembler_amd64.go
generated
vendored
Normal file
@@ -0,0 +1,281 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package jit
|
||||
|
||||
import (
|
||||
`encoding/binary`
|
||||
`strconv`
|
||||
`strings`
|
||||
`sync`
|
||||
|
||||
`github.com/bytedance/sonic/loader`
|
||||
`github.com/bytedance/sonic/internal/rt`
|
||||
`github.com/twitchyliquid64/golang-asm/obj`
|
||||
`github.com/twitchyliquid64/golang-asm/obj/x86`
|
||||
)
|
||||
|
||||
const (
|
||||
_LB_jump_pc = "_jump_pc_"
|
||||
)
|
||||
|
||||
type BaseAssembler struct {
|
||||
i int
|
||||
f func()
|
||||
c []byte
|
||||
o sync.Once
|
||||
pb *Backend
|
||||
xrefs map[string][]*obj.Prog
|
||||
labels map[string]*obj.Prog
|
||||
pendings map[string][]*obj.Prog
|
||||
}
|
||||
|
||||
/** Instruction Encoders **/
|
||||
|
||||
var _NOPS = [][16]byte {
|
||||
{0x90}, // NOP
|
||||
{0x66, 0x90}, // 66 NOP
|
||||
{0x0f, 0x1f, 0x00}, // NOP DWORD ptr [EAX]
|
||||
{0x0f, 0x1f, 0x40, 0x00}, // NOP DWORD ptr [EAX + 00H]
|
||||
{0x0f, 0x1f, 0x44, 0x00, 0x00}, // NOP DWORD ptr [EAX + EAX*1 + 00H]
|
||||
{0x66, 0x0f, 0x1f, 0x44, 0x00, 0x00}, // 66 NOP DWORD ptr [EAX + EAX*1 + 00H]
|
||||
{0x0f, 0x1f, 0x80, 0x00, 0x00, 0x00, 0x00}, // NOP DWORD ptr [EAX + 00000000H]
|
||||
{0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00}, // NOP DWORD ptr [EAX + EAX*1 + 00000000H]
|
||||
{0x66, 0x0f, 0x1f, 0x84, 0x00, 0x00, 0x00, 0x00, 0x00}, // 66 NOP DWORD ptr [EAX + EAX*1 + 00000000H]
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) NOP() *obj.Prog {
|
||||
p := self.pb.New()
|
||||
p.As = obj.ANOP
|
||||
self.pb.Append(p)
|
||||
return p
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) NOPn(n int) {
|
||||
for i := len(_NOPS); i > 0 && n > 0; i-- {
|
||||
for ; n >= i; n -= i {
|
||||
self.Byte(_NOPS[i - 1][:i]...)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) StorePtr(ptr int64, to obj.Addr, tmp obj.Addr) {
|
||||
if (to.Type != obj.TYPE_MEM) || (tmp.Type != obj.TYPE_REG) {
|
||||
panic("must store imm to memory, tmp must be register")
|
||||
}
|
||||
if (ptr >> 32) != 0 {
|
||||
self.Emit("MOVQ", Imm(ptr), tmp)
|
||||
self.Emit("MOVQ", tmp, to)
|
||||
} else {
|
||||
self.Emit("MOVQ", Imm(ptr), to);
|
||||
}
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Byte(v ...byte) {
|
||||
for ; len(v) >= 8; v = v[8:] { self.From("QUAD", Imm(rt.Get64(v))) }
|
||||
for ; len(v) >= 4; v = v[4:] { self.From("LONG", Imm(int64(rt.Get32(v)))) }
|
||||
for ; len(v) >= 2; v = v[2:] { self.From("WORD", Imm(int64(rt.Get16(v)))) }
|
||||
for ; len(v) >= 1; v = v[1:] { self.From("BYTE", Imm(int64(v[0]))) }
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Mark(pc int) {
|
||||
self.i++
|
||||
self.Link(_LB_jump_pc + strconv.Itoa(pc))
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Link(to string) {
|
||||
var p *obj.Prog
|
||||
var v []*obj.Prog
|
||||
|
||||
/* placeholder substitution */
|
||||
if strings.Contains(to, "{n}") {
|
||||
to = strings.ReplaceAll(to, "{n}", strconv.Itoa(self.i))
|
||||
}
|
||||
|
||||
/* check for duplications */
|
||||
if _, ok := self.labels[to]; ok {
|
||||
panic("label " + to + " has already been linked")
|
||||
}
|
||||
|
||||
/* get the pending links */
|
||||
p = self.NOP()
|
||||
v = self.pendings[to]
|
||||
|
||||
/* patch all the pending jumps */
|
||||
for _, q := range v {
|
||||
q.To.Val = p
|
||||
}
|
||||
|
||||
/* mark the label as resolved */
|
||||
self.labels[to] = p
|
||||
delete(self.pendings, to)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Xref(pc int, d int64) {
|
||||
self.Sref(_LB_jump_pc + strconv.Itoa(pc), d)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Sref(to string, d int64) {
|
||||
p := self.pb.New()
|
||||
p.As = x86.ALONG
|
||||
p.From = Imm(-d)
|
||||
|
||||
/* placeholder substitution */
|
||||
if strings.Contains(to, "{n}") {
|
||||
to = strings.ReplaceAll(to, "{n}", strconv.Itoa(self.i))
|
||||
}
|
||||
|
||||
/* record the patch point */
|
||||
self.pb.Append(p)
|
||||
self.xrefs[to] = append(self.xrefs[to], p)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Xjmp(op string, to int) {
|
||||
self.Sjmp(op, _LB_jump_pc + strconv.Itoa(to))
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Sjmp(op string, to string) {
|
||||
p := self.pb.New()
|
||||
p.As = As(op)
|
||||
|
||||
/* placeholder substitution */
|
||||
if strings.Contains(to, "{n}") {
|
||||
to = strings.ReplaceAll(to, "{n}", strconv.Itoa(self.i))
|
||||
}
|
||||
|
||||
/* check for backward jumps */
|
||||
if v, ok := self.labels[to]; ok {
|
||||
p.To.Val = v
|
||||
} else {
|
||||
self.pendings[to] = append(self.pendings[to], p)
|
||||
}
|
||||
|
||||
/* mark as a branch, and add to instruction buffer */
|
||||
p.To.Type = obj.TYPE_BRANCH
|
||||
self.pb.Append(p)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Rjmp(op string, to obj.Addr) {
|
||||
p := self.pb.New()
|
||||
p.To = to
|
||||
p.As = As(op)
|
||||
self.pb.Append(p)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) From(op string, val obj.Addr) {
|
||||
p := self.pb.New()
|
||||
p.As = As(op)
|
||||
p.From = val
|
||||
self.pb.Append(p)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Emit(op string, args ...obj.Addr) {
|
||||
p := self.pb.New()
|
||||
p.As = As(op)
|
||||
self.assignOperands(p, args)
|
||||
self.pb.Append(p)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) assignOperands(p *obj.Prog, args []obj.Addr) {
|
||||
switch len(args) {
|
||||
case 0 :
|
||||
case 1 : p.To = args[0]
|
||||
case 2 : p.To, p.From = args[1], args[0]
|
||||
case 3 : p.To, p.From, p.RestArgs = args[2], args[0], args[1:2]
|
||||
case 4 : p.To, p.From, p.RestArgs = args[2], args[3], args[:2]
|
||||
default : panic("invalid operands")
|
||||
}
|
||||
}
|
||||
|
||||
/** Assembler Helpers **/
|
||||
|
||||
func (self *BaseAssembler) Size() int {
|
||||
self.build()
|
||||
return len(self.c)
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Init(f func()) {
|
||||
self.i = 0
|
||||
self.f = f
|
||||
self.c = nil
|
||||
self.o = sync.Once{}
|
||||
}
|
||||
|
||||
var jitLoader = loader.Loader{
|
||||
Name: "sonic.jit.",
|
||||
File: "github.com/bytedance/sonic/jit.go",
|
||||
Options: loader.Options{
|
||||
NoPreempt: true,
|
||||
},
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) Load(name string, frameSize int, argSize int, argStackmap []bool, localStackmap []bool) loader.Function {
|
||||
self.build()
|
||||
return jitLoader.LoadOne(self.c, name, frameSize, argSize, argStackmap, localStackmap)
|
||||
}
|
||||
|
||||
/** Assembler Stages **/
|
||||
|
||||
func (self *BaseAssembler) init() {
|
||||
self.pb = newBackend("amd64")
|
||||
self.xrefs = map[string][]*obj.Prog{}
|
||||
self.labels = map[string]*obj.Prog{}
|
||||
self.pendings = map[string][]*obj.Prog{}
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) build() {
|
||||
self.o.Do(func() {
|
||||
self.init()
|
||||
self.f()
|
||||
self.validate()
|
||||
self.assemble()
|
||||
self.resolve()
|
||||
self.release()
|
||||
})
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) release() {
|
||||
self.pb.Release()
|
||||
self.pb = nil
|
||||
self.xrefs = nil
|
||||
self.labels = nil
|
||||
self.pendings = nil
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) resolve() {
|
||||
for s, v := range self.xrefs {
|
||||
for _, prog := range v {
|
||||
if prog.As != x86.ALONG {
|
||||
panic("invalid RIP relative reference")
|
||||
} else if p, ok := self.labels[s]; !ok {
|
||||
panic("links are not fully resolved: " + s)
|
||||
} else {
|
||||
off := prog.From.Offset + p.Pc - prog.Pc
|
||||
binary.LittleEndian.PutUint32(self.c[prog.Pc:], uint32(off))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) validate() {
|
||||
for key := range self.pendings {
|
||||
panic("links are not fully resolved: " + key)
|
||||
}
|
||||
}
|
||||
|
||||
func (self *BaseAssembler) assemble() {
|
||||
self.c = self.pb.Assemble()
|
||||
}
|
||||
120
vendor/github.com/bytedance/sonic/internal/jit/backend.go
generated
vendored
Normal file
120
vendor/github.com/bytedance/sonic/internal/jit/backend.go
generated
vendored
Normal file
@@ -0,0 +1,120 @@
|
||||
/*
|
||||
* Copyright 2021 ByteDance Inc.
|
||||
*
|
||||
* Licensed under the Apache License, Version 2.0 (the "License");
|
||||
* you may not use this file except in compliance with the License.
|
||||
* You may obtain a copy of the License at
|
||||
*
|
||||
* http://www.apache.org/licenses/LICENSE-2.0
|
||||
*
|
||||
* Unless required by applicable law or agreed to in writing, software
|
||||
* distributed under the License is distributed on an "AS IS" BASIS,
|
||||
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
* See the License for the specific language governing permissions and
|
||||
* limitations under the License.
|
||||
*/
|
||||
|
||||
package jit
|
||||
|
||||
import (
|
||||
`fmt`
|
||||
`sync`
|
||||
_ `unsafe`
|
||||
|
||||
`github.com/twitchyliquid64/golang-asm/asm/arch`
|
||||
`github.com/twitchyliquid64/golang-asm/obj`
|
||||
`github.com/twitchyliquid64/golang-asm/objabi`
|
||||
)
|
||||
|
||||
type Backend struct {
|
||||
Ctxt *obj.Link
|
||||
Arch *arch.Arch
|
||||
Head *obj.Prog
|
||||
Tail *obj.Prog
|
||||
Prog []*obj.Prog
|
||||
}
|
||||
|
||||
var (
|
||||
_progPool sync.Pool
|
||||
)
|
||||
|
||||
//go:nosplit
|
||||
//go:linkname throw runtime.throw
|
||||
func throw(_ string)
|
||||
|
||||
func newProg() *obj.Prog {
|
||||
if val := _progPool.Get(); val == nil {
|
||||
return new(obj.Prog)
|
||||
} else {
|
||||
return remProg(val.(*obj.Prog))
|
||||
}
|
||||
}
|
||||
|
||||
func remProg(p *obj.Prog) *obj.Prog {
|
||||
*p = obj.Prog{}
|
||||
return p
|
||||
}
|
||||
|
||||
func newBackend(name string) (ret *Backend) {
|
||||
ret = new(Backend)
|
||||
ret.Arch = arch.Set(name)
|
||||
ret.Ctxt = newLinkContext(ret.Arch.LinkArch)
|
||||
ret.Arch.Init(ret.Ctxt)
|
||||
return
|
||||
}
|
||||
|
||||
func newLinkContext(arch *obj.LinkArch) (ret *obj.Link) {
|
||||
ret = obj.Linknew(arch)
|
||||
ret.Headtype = objabi.Hlinux
|
||||
ret.DiagFunc = diagLinkContext
|
||||
return
|
||||
}
|
||||
|
||||
func diagLinkContext(str string, args ...interface{}) {
|
||||
throw(fmt.Sprintf(str, args...))
|
||||
}
|
||||
|
||||
func (self *Backend) New() (ret *obj.Prog) {
|
||||
ret = newProg()
|
||||
ret.Ctxt = self.Ctxt
|
||||
self.Prog = append(self.Prog, ret)
|
||||
return
|
||||
}
|
||||
|
||||
func (self *Backend) Append(p *obj.Prog) {
|
||||
if self.Head == nil {
|
||||
self.Head = p
|
||||
self.Tail = p
|
||||
} else {
|
||||
self.Tail.Link = p
|
||||
self.Tail = p
|
||||
}
|
||||
}
|
||||
|
||||
func (self *Backend) Release() {
|
||||
self.Arch = nil
|
||||
self.Ctxt = nil
|
||||
|
||||
/* return all the progs into pool */
|
||||
for _, p := range self.Prog {
|
||||
_progPool.Put(p)
|
||||
}
|
||||
|
||||
/* clear all the references */
|
||||
self.Head = nil
|
||||
self.Tail = nil
|
||||
self.Prog = nil
|
||||
}
|
||||
|
||||
func (self *Backend) Assemble() []byte {
|
||||
var sym obj.LSym
|
||||
var fnv obj.FuncInfo
|
||||
|
||||
/* construct the function */
|
||||
sym.Func = &fnv
|
||||
fnv.Text = self.Head
|
||||
|
||||
/* call the assembler */
|
||||
self.Arch.Assemble(self.Ctxt, &sym, self.New)
|
||||
return sym.P
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user