添加全局验证器，优化白名单创建请求的参数验证
This commit is contained in:
104
README.md
104
README.md
@@ -1,79 +1,41 @@
|
||||
## todo
|
||||
|
||||
核心流程：
|
||||
|
||||
完成账户总览
|
||||
- 统一简化包导入别名
|
||||
- 修改 postgres 默认事务级别到 RR
|
||||
- 撤销令牌需要改成用户权限，只有本人可以撤销
|
||||
- 移动端适配
|
||||
- 扩展 device 权限验证方式，提供一种方法区分内部和外部服务
|
||||
- 检查数据库枚举字段，0 值只作为空值使用
|
||||
- 更新数据库填充
|
||||
- 错误处理类型转换失败问题
|
||||
- channel 接口
|
||||
- 重新梳理逻辑流程，简化循环
|
||||
- 端口分配时加锁
|
||||
- 用对称加密处理密钥
|
||||
- 环境变量配置默认会话配置
|
||||
- 微信支付
|
||||
- 支付回调处理
|
||||
- 页面 账户总览
|
||||
- 保存 session 到数据库
|
||||
- 中间件 Limiter
|
||||
- 中间件 Compress
|
||||
- 页面 提取记录
|
||||
- 页面 使用记录
|
||||
- 废弃 password 授权模式，迁移到 authorization code 授权模式
|
||||
- oauth token 验证授权范围
|
||||
- 实现白银节点的 warp 服务，用来去重与端口分配，保证测试与生产环境不会产生端口竞争
|
||||
- 增加 domain 层，缓解同包字段过长的问题
|
||||
- callback 结果直接由 api 端提供，不通过前端转发
|
||||
- debug：白银节点提供一些工具接口，方便快速操作
|
||||
- 批量下线端口
|
||||
- 统一使用 validator 进行参数验证
|
||||
|
||||
表格页筛选日期，范围筛选需要联动
|
||||
### 长期
|
||||
|
||||
购买套餐页的冗余组件
|
||||
|
||||
确认各个页面操作列的内容
|
||||
|
||||
- [ ] 提取记录
|
||||
- [ ] 使用记录
|
||||
|
||||
中间件：
|
||||
|
||||
- [ ] Limiter
|
||||
- [ ] Compress
|
||||
|
||||
考虑统计接口调用频率并通过接口展示
|
||||
|
||||
考虑登录时曾经输入过验证码的用户，登录成功后允许一段时间内免输验证码
|
||||
|
||||
修改 postgres 默认事务级别到 RR
|
||||
|
||||
撤销令牌需要改成用户权限，只有本人可以撤销
|
||||
|
||||
扩展 device 权限验证方式，提供一种方法区分内部和外部服务
|
||||
|
||||
白名单限制可输入网段
|
||||
|
||||
废弃 password 授权模式，迁移到 authorization code 授权模式
|
||||
|
||||
使用 fiber 自带 validator 进行参数验证
|
||||
|
||||
增加 domain 层，缓解同包字段过长的问题
|
||||
|
||||
移动端适配
|
||||
|
||||
授权过期跳转登录，成功后返回原链接
|
||||
|
||||
错误处理类型转换失败问题
|
||||
|
||||
callback 结果直接由 api 端提供，不通过前端转发
|
||||
|
||||
统一简化包导入别名
|
||||
|
||||
更新数据库填充
|
||||
|
||||
检查数据库枚举字段，0 值只作为空值使用
|
||||
|
||||
实现开发时迁移脚本：
|
||||
- 检查 sql 注释
|
||||
- 执行迁移
|
||||
- 如果有必要则填充数据
|
||||
|
||||
查端口需要通过外部接口实现，防止不同环境下的端口覆盖。提供一个额外的简便方法用来实现端口覆盖
|
||||
|
||||
业务代码和测试代码共用的控制变量可以优化为环境变量
|
||||
|
||||
channel 优化：
|
||||
|
||||
- 重新梳理逻辑流程，简化循环
|
||||
- 端口分配时加锁
|
||||
- 数据存入顺序，数据库 > 缓存 > 外部接口
|
||||
|
||||
用对称加密处理密钥
|
||||
|
||||
考虑将鉴权逻辑放到 handler 里，统一动静态鉴权以及解耦服务层
|
||||
|
||||
环境变量配置默认会话配置
|
||||
|
||||
oauth token 验证授权范围
|
||||
|
||||
保存 session 到数据库
|
||||
- 业务代码和测试代码共用的控制变量可以优化为环境变量
|
||||
- 考虑统计接口调用频率并通过接口展示
|
||||
- 考虑登录时曾经输入过验证码的用户，登录成功后允许一段时间内免输验证码
|
||||
|
||||
## 环境变量和脚本
|
||||
|
||||
|
||||
7
go.mod
7
go.mod
@@ -39,7 +39,11 @@ require (
|
||||
github.com/clbanning/mxj/v2 v2.7.0 // indirect
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/gabriel-vasile/mimetype v1.4.8 // indirect
|
||||
github.com/glebarez/go-sqlite v1.21.2 // indirect
|
||||
github.com/go-playground/locales v0.14.1 // indirect
|
||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||
github.com/go-playground/validator/v10 v10.26.0 // indirect
|
||||
github.com/go-sql-driver/mysql v1.9.1 // indirect
|
||||
github.com/gofrs/uuid v4.4.0+incompatible // indirect
|
||||
github.com/jackc/pgpassfile v1.0.0 // indirect
|
||||
@@ -50,6 +54,7 @@ require (
|
||||
github.com/jinzhu/now v1.1.5 // indirect
|
||||
github.com/json-iterator/go v1.1.12 // indirect
|
||||
github.com/klauspost/compress v1.18.0 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
github.com/mattn/go-colorable v0.1.14 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/mattn/go-runewidth v0.0.16 // indirect
|
||||
@@ -61,7 +66,7 @@ require (
|
||||
github.com/smartwalle/ncrypto v1.0.4 // indirect
|
||||
github.com/smartwalle/ngx v1.0.9 // indirect
|
||||
github.com/smartwalle/nsign v1.0.9 // indirect
|
||||
github.com/stretchr/testify v1.8.2 // indirect
|
||||
github.com/stretchr/testify v1.8.4 // indirect
|
||||
github.com/tjfoc/gmsm v1.4.1 // indirect
|
||||
github.com/valyala/bytebufferpool v1.0.0 // indirect
|
||||
github.com/yuin/gopher-lua v1.1.1 // indirect
|
||||
|
||||
11
go.sum
11
go.sum
@@ -81,10 +81,18 @@ github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+m
|
||||
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
|
||||
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
|
||||
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
|
||||
github.com/gabriel-vasile/mimetype v1.4.8 h1:FfZ3gj38NjllZIeJAmMhr+qKL8Wu+nOoI3GqacKw1NM=
|
||||
github.com/gabriel-vasile/mimetype v1.4.8/go.mod h1:ByKUIKGjh1ODkGM1asKUbQZOLGrPjydw3hYPU2YU9t8=
|
||||
github.com/glebarez/go-sqlite v1.21.2 h1:3a6LFC4sKahUunAmynQKLZceZCOzUthkRkEAl9gAXWo=
|
||||
github.com/glebarez/go-sqlite v1.21.2/go.mod h1:sfxdZyhQjTM2Wry3gVYWaW072Ri1WMdWJi0k6+3382k=
|
||||
github.com/glebarez/sqlite v1.11.0 h1:wSG0irqzP6VurnMEpFGer5Li19RpIRi2qvQz++w0GMw=
|
||||
github.com/glebarez/sqlite v1.11.0/go.mod h1:h8/o8j5wiAsqSPoWELDUdJXhjAhsVliSn7bWZjOhrgQ=
|
||||
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
|
||||
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
|
||||
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
||||
github.com/go-playground/validator/v10 v10.26.0 h1:SP05Nqhjcvz81uJaRfEV0YBSSSGMc/iMaVtFbr3Sw2k=
|
||||
github.com/go-playground/validator/v10 v10.26.0/go.mod h1:I5QpIEbmr8On7W0TktmJAumgzX4CA1XNl4ZmDuVHKKo=
|
||||
github.com/go-sql-driver/mysql v1.7.0/go.mod h1:OXbVy3sEdcQ2Doequ6Z5BW6fXNQTmx+9S1MCJN5yJMI=
|
||||
github.com/go-sql-driver/mysql v1.9.1 h1:FrjNGn/BsJQjVRuSa8CBrM5BWA9BWoXXat3KrtSb/iI=
|
||||
github.com/go-sql-driver/mysql v1.9.1/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
|
||||
@@ -146,6 +154,8 @@ github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zt
|
||||
github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
|
||||
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
|
||||
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
|
||||
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||
github.com/lmittmann/tint v1.0.7 h1:D/0OqWZ0YOGZ6AyC+5Y2kD8PBEzBk6rFHVSfOqCkF9Y=
|
||||
github.com/lmittmann/tint v1.0.7/go.mod h1:HIS3gSy7qNwGCj+5oRjAutErFBl4BzdQP6cJZ0NfMwE=
|
||||
github.com/mattn/go-colorable v0.1.14 h1:9A9LHSqF/7dyVVX6g0U9cwm9pG3kP9gSzcuIPHPsaIE=
|
||||
@@ -202,6 +212,7 @@ github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO
|
||||
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||
github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
|
||||
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/tjfoc/gmsm v1.3.2/go.mod h1:HaUcFuY0auTiaHB9MHFGCPx5IaLhTUd2atbCFBQXn9w=
|
||||
github.com/tjfoc/gmsm v1.4.1 h1:aMe1GlZb+0bLjn+cKTPEvvn9oUEBlJitaZiiBwsbgho=
|
||||
github.com/tjfoc/gmsm v1.4.1/go.mod h1:j4INPkHWMrhJb38G+J6W4Tw0AbuN8Thu3PbdVYhVcTE=
|
||||
|
||||
55
web/globals/validator.go
Normal file
55
web/globals/validator.go
Normal file
@@ -0,0 +1,55 @@
|
||||
package globals
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strings"
|
||||
|
||||
"github.com/go-playground/locales/zh"
|
||||
ut "github.com/go-playground/universal-translator"
|
||||
"github.com/go-playground/validator/v10"
|
||||
zhtrans "github.com/go-playground/validator/v10/translations/zh"
|
||||
"github.com/gofiber/fiber/v2"
|
||||
)
|
||||
|
||||
var Validator *ValidatorHolder
|
||||
|
||||
type ValidatorHolder struct {
|
||||
validator *validator.Validate
|
||||
translator ut.Translator
|
||||
}
|
||||
|
||||
func (v *ValidatorHolder) Validate(c *fiber.Ctx, data any) error {
|
||||
|
||||
if err := c.BodyParser(data); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if errs := v.validator.Struct(data); errs != nil {
|
||||
var sb = strings.Builder{}
|
||||
var typeErrs validator.ValidationErrors
|
||||
errors.As(errs, &typeErrs)
|
||||
for i, err := range typeErrs {
|
||||
sb.WriteString(err.Translate(v.translator))
|
||||
if i < len(typeErrs)-1 {
|
||||
sb.WriteString("\n")
|
||||
}
|
||||
}
|
||||
return fiber.NewError(fiber.StatusBadRequest, sb.String())
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func InitValidator() {
|
||||
var validate = validator.New(validator.WithRequiredStructEnabled())
|
||||
|
||||
var translator = ut.New(zh.New()).GetFallback()
|
||||
err := zhtrans.RegisterDefaultTranslations(validate, translator)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
Validator = &ValidatorHolder{
|
||||
validator: validate,
|
||||
translator: translator,
|
||||
}
|
||||
}
|
||||
@@ -1,8 +1,10 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"net"
|
||||
"platform/web/auth"
|
||||
"platform/web/common"
|
||||
g "platform/web/globals"
|
||||
m "platform/web/models"
|
||||
q "platform/web/queries"
|
||||
"platform/web/services"
|
||||
@@ -69,8 +71,8 @@ func ListWhitelist(c *fiber.Ctx) error {
|
||||
}
|
||||
|
||||
type CreateWhitelistReq struct {
|
||||
Host string `json:"host" validate:"required"`
|
||||
Remark string `json:"remark"`
|
||||
Host string `json:"host" validate:"required,ip"`
|
||||
Remark string `json:"remark,omitempty"`
|
||||
}
|
||||
|
||||
func CreateWhitelist(c *fiber.Ctx) error {
|
||||
@@ -83,21 +85,22 @@ func CreateWhitelist(c *fiber.Ctx) error {
|
||||
|
||||
// 解析请求参数
|
||||
req := new(CreateWhitelistReq)
|
||||
if err := c.BodyParser(req); err != nil {
|
||||
err = g.Validator.Validate(c, req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if req.Host == "" {
|
||||
return fiber.NewError(fiber.StatusBadRequest, "host is required")
|
||||
|
||||
err = secureAddr(req.Host)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// 创建白名单
|
||||
whitelist := &m.Whitelist{
|
||||
err = q.Whitelist.Create(&m.Whitelist{
|
||||
UserID: authContext.Payload.Id,
|
||||
Host: req.Host,
|
||||
Remark: req.Remark,
|
||||
}
|
||||
|
||||
err = q.Whitelist.Create(whitelist)
|
||||
})
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -183,3 +186,15 @@ func RemoveWhitelist(c *fiber.Ctx) error {
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func secureAddr(str string) error {
|
||||
var addr = net.ParseIP(str)
|
||||
if addr == nil {
|
||||
return fiber.NewError(fiber.StatusBadRequest, "IP 解析失败")
|
||||
}
|
||||
|
||||
if addr.IsGlobalUnicast() {
|
||||
return nil
|
||||
}
|
||||
return fiber.NewError(fiber.StatusBadRequest, "IP 地址不可用")
|
||||
}
|
||||
|
||||
127
web/handlers/whitelist_test.go
Normal file
127
web/handlers/whitelist_test.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package handlers
|
||||
|
||||
import "testing"
|
||||
|
||||
func Test_secureAddr(t *testing.T) {
|
||||
type args struct {
|
||||
str string
|
||||
}
|
||||
tests := []struct {
|
||||
name string
|
||||
args args
|
||||
wantErr bool
|
||||
}{
|
||||
// 有效的公网 IP 地址
|
||||
{
|
||||
name: "有效公网IPv4地址",
|
||||
args: args{str: "203.0.113.1"},
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "有效公网IPv6地址",
|
||||
args: args{str: "2001:db8::1"},
|
||||
wantErr: false,
|
||||
},
|
||||
|
||||
// 私有地址
|
||||
{
|
||||
name: "IPv4私有地址(10.x.x.x)",
|
||||
args: args{str: "10.0.0.1"},
|
||||
wantErr: false, // 取决于需求，通常私有地址是被允许的全局单播地址
|
||||
},
|
||||
{
|
||||
name: "IPv4私有地址(172.16.x.x)",
|
||||
args: args{str: "172.16.0.1"},
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "IPv4私有地址(192.168.x.x)",
|
||||
args: args{str: "192.168.0.1"},
|
||||
wantErr: false,
|
||||
},
|
||||
{
|
||||
name: "IPv6私有地址(ULA)",
|
||||
args: args{str: "fd00::1"},
|
||||
wantErr: false,
|
||||
},
|
||||
|
||||
// 广播地址
|
||||
{
|
||||
name: "IPv4本地广播地址",
|
||||
args: args{str: "255.255.255.255"},
|
||||
wantErr: true,
|
||||
},
|
||||
|
||||
// 未指定地址
|
||||
{
|
||||
name: "IPv4未指定地址",
|
||||
args: args{str: "0.0.0.0"},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "IPv6未指定地址",
|
||||
args: args{str: "::"},
|
||||
wantErr: true,
|
||||
},
|
||||
|
||||
// 回环地址
|
||||
{
|
||||
name: "IPv4回环地址",
|
||||
args: args{str: "127.0.0.1"},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "IPv6回环地址",
|
||||
args: args{str: "::1"},
|
||||
wantErr: true,
|
||||
},
|
||||
|
||||
// 组播地址
|
||||
{
|
||||
name: "IPv4组播地址",
|
||||
args: args{str: "224.0.0.1"},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "IPv6组播地址",
|
||||
args: args{str: "ff00::1"},
|
||||
wantErr: true,
|
||||
},
|
||||
|
||||
// 链路本地地址
|
||||
{
|
||||
name: "IPv4链路本地地址",
|
||||
args: args{str: "169.254.0.1"},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "IPv6链路本地地址",
|
||||
args: args{str: "fe80::1"},
|
||||
wantErr: true,
|
||||
},
|
||||
|
||||
// 格式错误的地址
|
||||
{
|
||||
name: "格式错误的IP地址",
|
||||
args: args{str: "not-an-ip"},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "不完整的IP地址",
|
||||
args: args{str: "192.168.0"},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "超出范围的IP地址",
|
||||
args: args{str: "256.256.256.256"},
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
for _, tt := range tests {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
if err := secureAddr(tt.args.str); (err != nil) != tt.wantErr {
|
||||
t.Errorf("secureAddr() error = %v, wantErr %v", err, tt.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -43,10 +43,11 @@ func (s *Server) Run() error {
|
||||
g.InitAlipay()
|
||||
// g.InitWechatPay()
|
||||
g.InitAliyun()
|
||||
g.InitValidator()
|
||||
|
||||
// config
|
||||
s.fiber = fiber.New(fiber.Config{
|
||||
ProxyHeader: fiber.HeaderXForwardedFor,
|
||||
ProxyHeader: fiber.HeaderXForwardedFor,
|
||||
ErrorHandler: ErrorHandler,
|
||||
})
|
||||
|
||||
|
||||
Reference in New Issue
Block a user