April 22, 2022
Go 1.18 预定义接口类型
#
先看一个提案:
proposal: spec: permit values to have type “comparable” – 允许值拥有comparable
类型,我的理解是,现在的comparable
只能用作泛型里的类型参数的约束,不能像普通类型那样使用,如下:
type Set[E comparable] []E // 可以用做类型参数的约束
// 使用go1.18编译,报错:interface is (or embeds) comparable
var A comparable // 变量不可以使用`comparable`类型
那么,结合例子就能更好地理解这个提案了。
这个提案的主要目的就是让例子里的var A comparable
成立,也就是允许comparable
作为变量的类型,跟其它普通的接口类型(var E error
)一样。
// proposal: spec: permit values to have type "comparable"
// As part of adding generics, Go 1.18 introduces a new predeclared interface type comparable. That interface type is implemented by any non-interface type that is comparable, and by any interface type that is or embeds comparable. Comparable non-interface types are numeric, string, boolean, pointer, and channel types, structs all of whose field types are comparable, and arrays whose element types are comparable. Slice, map, and function types are not comparable.
// -- 作为泛型的一部分,Go1.18引入了一个新的预定义接口类型:`comparable`。这个接口类型由任何可比较的非接口类型实现,和任何是`comparable`或内嵌了`comparable`的接口类型实现。可比较的非接口类型有:数值、字符串、布尔、指针、字段类型均是可比较的管道或结构体类型、元素是可比较的数组类型。切片、映射、函数类型均是不可比较的。
// In Go, interface types are comparable in the sense that they can be compared with the == and != operators. However, interface types do not in general implement the predeclared interface type comparable. An interface type only implements comparable if it is or embeds comparable.
// 在Go里面,接口类型是可比较的意味着它们可以用`==`和`!=`操作符进行比较。但是,接口类型一般来说没有实现预定义接口类型`comparable`。一个接口类型只有它是或内嵌了`comparable`时才实现了`comparable`。
// Developing this distinction between the predeclared type comparable and the general language notion of comparable has been confusing; see #50646. The distinction makes it hard to write certain kinds of generic code; see #51257.
// 出现的两个问题:[怎么在文档里说明哪些接口实现它了呢?](https://github.com/golang/go/issues/50646), [any是任意类型的意思,那必然比comparable大吧](https://github.com/golang/go/issues/51257)
// 突然想到:如果我要把一个变量表示为不可比较的,怎么样可以用`comparable`来表示呢,`!comparable`?
// For a specific example, you can today write a generic Set type of some specific (comparable) element type and write functions that work on sets of any element type:
//
// type Set[E comparable] map[E]bool
// func Union[E comparable](s1, s2 Set[E]) Set[E] { ... }
//
// But there is no way today to instantiate this Set type to create a general set that works for any (comparable) value. That is, you can't write Set[any], because any does not satisfy the constraint comparable. You can get a very similar effect by writing map[any]bool, but then all the functions like Union have to be written anew for this new version.
// We can reduce this kind of problem by permitting comparable to be an ordinary type. It then becomes possible to write Set[comparable].
// As an ordinary type, comparable would be an interface type that is implemented by any comparable type.
// 作为一个普通类型,`comparable`是一个可以被任意`comparable`类型实现的接口类型。
// Any comparable non-interface type could be assigned to a variable of type comparable.
// -- 任何可比较的非接口类型可以被分配到类型为`comparable`的变量。
// A value of an interface type that is or embeds comparable could be assigned to a variable of type comparable.
// -- 接口类型是或内嵌了`comparable`的值可以被分配到类型为`comparable`的变量。
// A type assertion to comparable, as in x.(comparable), would succeed if the dynamic type of x is a comparable type.
// 类型断言,如`x.(comparable)`,当x的动态类型是一个`comparable`类型时可以成功。
// Similarly for a type switch case comparable.
// 对`type switch`来说类似。
type C interface {
comparable
}
var c C
func main() {
var A comparable
var a int
if v, ok := a.(comparable); ok {
}
switch a.(type) {
case comparable:
}
}
反射的Comparable
#
func ReflectComparable(v interface{}) bool {
typ := reflect.TypeOf(v)
// Comparable reports whether values of this type are comparable.
// Even if Comparable returns true, the comparison may still panic.
// For example, values of interface type are comparable,
// but the comparison will panic if their dynamic type is not comparable.
// -- 即使返回true,也有可能panic。
// 比如:接口类型的值是可比较的,但如果它们的动态类型是不可比较的,就会panic
return typ.Comparable()
}
go/types的Comparable
#
func TypesComparable() bool {
t := types.NewChan(types.SendOnly, &types.Basic{})
return types.Comparable(t)
}
更新
#
使comparable仅在类型集里没有任何一个不可比较类型时正确,否则依然在编译时可通过,但运行时panic
...March 22, 2022
KMP字符串匹配算法
精确匹配
状态机
给定一个pattern,查找其在另一字符串s出现的最早位置。(找不到则返回-1)
func index(s string, pattern string) int {
return -1
}
状态推移
func index(s string, pattern string) int {
n := len(s)
m := len(pattern)
// 根据pattern构造dp
var dp [n][m]int
// 在s上应用dp,判断pattern位置
return -1
}
February 18, 2022
当有人说要在屋里开个窗,一定惹得大伙不开心,无人同意;若要在屋里凿个洞,就有人来协调,愿意开窗了。
看到了吗?这里面有提议的人,有反对的人,有开始反对后面协调的人。看似只有这几种人,实则还有一种人,哪边人多站哪边。恶则落井下石,善则“好言相劝”。
一盆散沙,就算反对,也难以“碍事”。聪明人早就明白这个道理。只要能裹挟着一群人,与自己利益捆绑,那么就能为己所用。至于“所用”是何物,自然无关紧要,只要“为己”即可。
同样地,要击溃捆绑,自然需要强大的力量,也就是另一群人。
February 16, 2022
Reddit讨论贴
Go uses a different strategy for blocking systemcalls. It does not run them on a threadpool - it moves all the other goroutines that are queued to run on the current thread to a new worker thread, then runs the blocking systemcall on the current thread. This minimizes context switching.
You can do this in tokio as well, using task::block_in_place. If I change your code to use that instead of tokio::fs, it gets a lot closer to the go numbers. Note that using block_in_place is not without caveats, and it only works on the multi-threaded runtime, not the single-threaded one. That’s why it’s not used in the implementation of tokio::fs.
...
February 11, 2022
src/runtime/chan.go
:
// Invariants:
// At least one of c.sendq and c.recvq is empty,
// except for the case of an unbuffered channel with a single goroutine
// blocked on it for both sending and receiving using a select statement,
// in which case the length of c.sendq and c.recvq is limited only by the
// size of the select statement.
//
// For buffered channels, also:
// c.qcount > 0 implies that c.recvq is empty.
// c.qcount < c.dataqsiz implies that c.sendq is empty.
// 在文件开头,说明了几个不变量:
// c.sendq和c.recvq中至少有一个是空的,
// 除非,一个无缓冲管道在一个goroutine里阻塞了,这个管道的发送和接收都使用了一个select语句,这时
// c.sendq和c.recvq的长度被select语句限制。
//
// 对于缓冲管道,同样地:
// c.qcount > 0 表明c.recvq是空的。
// c.qcount < c.dataqsiz 表明c.sendq是空的。
// 实际的chan类型
type hchan struct {
qcount uint // total data in the queue - 队列里的数据总数量
dataqsiz uint // size of the circular queue - 循环队列的大小,make时传进来的值
buf unsafe.Pointer // points to an array of dataqsiz elements - dataqsiz元素组成的数组的指针
elemsize uint16 // 元素大小
closed uint32 // 是否关闭
elemtype *_type // element type - 元素类型
sendx uint // send index - 发送索引
recvx uint // receive index - 接收索引
recvq waitq // list of recv waiters - 等待接收者列表,表明这个管道的接收者;一个链表,里面的每个元素代表一个g;
sendq waitq // list of send waiters - 等待发送者列表,编码这个管道的发送者
// lock protects all fields in hchan, as well as several
// fields in sudogs blocked on this channel.
//
// Do not change another G's status while holding this lock
// (in particular, do not ready a G), as this can deadlock
// with stack shrinking.
lock mutex // 保护chan里的所有字段,以及阻塞在本管道里的sudog;当持有这个锁时,不要改变其它G的状态,因为在栈收缩时可能引起死锁。
}
type waitq struct {
first *sudog
last *sudog
}
// sudog represents a g in a wait list, such as for sending/receiving
// on a channel. - 代表了一个在等待列表的g
//
// sudog is necessary because the g ↔ synchronization object relation
// is many-to-many. A g can be on many wait lists, so there may be
// many sudogs for one g; and many gs may be waiting on the same
// synchronization object, so there may be many sudogs for one object.
// - sudog是必须的,因为g和同步对象关系是多对多。一个g可以在多个等待列表里,因此一个g对应有多个sudog;
// 多个g可以等待同一个同步对象,因此一个对象会对应多个sudog。
//
// sudogs are allocated from a special pool. Use acquireSudog and
// releaseSudog to allocate and free them.
// - sudog从一个特殊池子里分配,使用acquireSudog分配和releaseSudog释放它们。
type sudog struct {
// The following fields are protected by the hchan.lock of the
// channel this sudog is blocking on. shrinkstack depends on
// this for sudogs involved in channel ops.
// - 以下字段由hchan.lock来保护。
g *g // 代表的g
next *sudog // 链表中的下一个
prev *sudog // 链表中的上一个
elem unsafe.Pointer // data element (may point to stack) - 数据元素,可能是指向栈的指针
// The following fields are never accessed concurrently.
// For channels, waitlink is only accessed by g.
// For semaphores, all fields (including the ones above)
// are only accessed when holding a semaRoot lock.
// - 以下字段永远不会被并发访问。
// 对于管道,waitlink只会被g访问。
// 对于信号量,所有字段(包括上面的)只有在持有semaRoot锁时才能被访问
acquiretime int64 // 获取时间
releasetime int64 // 释放时间
ticket uint32 // 票据
// isSelect indicates g is participating in a select, so
// g.selectDone must be CAS'd to win the wake-up race.
isSelect bool // 表明g是否参与到了一个select里,从而使得g.selectDone必须CAS地去赢得唤醒竞赛
// success indicates whether communication over channel c
// succeeded. It is true if the goroutine was awoken because a
// value was delivered over channel c, and false if awoken
// because c was closed.
success bool // 表明管道的通信是否成功了,如果goroutine因为一个值被管道传送到来而唤醒即为成功
parent *sudog // semaRoot binary tree - 根信号量二叉树
waitlink *sudog // g.waiting list or semaRoot - g的等待列表或semaRoot
waittail *sudog // semaRoot
c *hchan // channel - 所属管道
}
// 新建
func makechan(t *chantype, size int) *hchan {
elem := t.elem
// compiler checks this but be safe.
if elem.size >= 1<<16 { // 管道的元素大小不能太大
throw("makechan: invalid channel element type")
}
// const hchanSize uintptr = 96
if hchanSize%maxAlign != 0 || elem.align > maxAlign { // 对齐检查
throw("makechan: bad alignment")
}
// 元素大小乘以管道大小,计算出来所需内存大小
mem, overflow := math.MulUintptr(elem.size, uintptr(size))
if overflow || mem > maxAlloc-hchanSize || size < 0 {
panic(plainError("makechan: size out of range"))
}
// Hchan does not contain pointers interesting for GC when elements stored in buf do not contain pointers.
// buf points into the same allocation, elemtype is persistent.
// SudoG's are referenced from their owning thread so they can't be collected.
// TODO(dvyukov,rlh): Rethink when collector can move allocated objects.
var c *hchan
switch {
case mem == 0:
// Queue or element size is zero.
c = (*hchan)(mallocgc(hchanSize, nil, true))
// Race detector uses this location for synchronization.
c.buf = c.raceaddr()
case elem.ptrdata == 0:
// Elements do not contain pointers. -- 元素没有包含指针
// Allocate hchan and buf in one call.
c = (*hchan)(mallocgc(hchanSize+mem, nil, true))
c.buf = add(unsafe.Pointer(c), hchanSize)
default:
// Elements contain pointers. -- 元素包含指针
c = new(hchan)
c.buf = mallocgc(mem, elem, true)
}
c.elemsize = uint16(elem.size)
c.elemtype = elem
c.dataqsiz = uint(size)
lockInit(&c.lock, lockRankHchan) // 初始化锁
if debugChan {
print("makechan: chan=", c, "; elemsize=", elem.size, "; dataqsiz=", size, "\n")
}
return c
}
// 发送
func sendDirect(t *_type, sg *sudog, src unsafe.Pointer) {
// src is on our stack, dst is a slot on another stack.
// - src是在我们的栈上,dst是另一个栈上的槽
// Once we read sg.elem out of sg, it will no longer
// be updated if the destination's stack gets copied (shrunk).
// So make sure that no preemption points can happen between read & use.
dst := sg.elem
typeBitsBulkBarrier(t, uintptr(dst), uintptr(src), t.size)
// No need for cgo write barrier checks because dst is always
// Go memory.
memmove(dst, src, t.size) // 移动src到dst
}
// 接收 -- 请看源码
// 关闭
func closechan(c *hchan) {
if c == nil {
panic(plainError("close of nil channel"))
}
lock(&c.lock)
if c.closed != 0 { // 已关闭的chan,如果再次关闭会panic
unlock(&c.lock)
panic(plainError("close of closed channel"))
}
if raceenabled {
callerpc := getcallerpc()
racewritepc(c.raceaddr(), callerpc, abi.FuncPCABIInternal(closechan))
racerelease(c.raceaddr())
}
c.closed = 1 // 设为关闭
var glist gList
// 先释放接收者,再释放发送者
// release all readers
for {
sg := c.recvq.dequeue() // 逐个出队sudog
if sg == nil {
break
}
if sg.elem != nil {
typedmemclr(c.elemtype, sg.elem) // 清理元素
sg.elem = nil
}
if sg.releasetime != 0 {
sg.releasetime = cputicks()
}
gp := sg.g
gp.param = unsafe.Pointer(sg)
sg.success = false
if raceenabled {
raceacquireg(gp, c.raceaddr())
}
glist.push(gp) // 把关联的g存到glist里
}
// release all writers (they will panic)
for {
sg := c.sendq.dequeue()
if sg == nil {
break
}
sg.elem = nil
if sg.releasetime != 0 {
sg.releasetime = cputicks()
}
gp := sg.g
gp.param = unsafe.Pointer(sg)
sg.success = false
if raceenabled {
raceacquireg(gp, c.raceaddr())
}
glist.push(gp)
}
unlock(&c.lock)
// Ready all Gs now that we've dropped the channel lock.
for !glist.empty() {
gp := glist.pop() // 逐个处理g
gp.schedlink = 0
goready(gp, 3) // 因为我们已经释放了这些g所关联的chan,所以让这些g进入ready状态,准备运行 -- Mark gp ready to run.
}
}
src/runtime/type.go
:
...February 10, 2022
go1.18将要推出workspace
模式,此举是为了方便在本地开发多个不同module
时的依赖管理。
命令说明:
$ go help work
Go workspace provides access to operations on workspaces.
Note that support for workspaces is built into many other commands, not
just 'go work'.
See 'go help modules' for information about Go\'s module system of which
workspaces are a part.
A workspace is specified by a go.work file that specifies a set of
module directories with the "use" directive. These modules are used as
root modules by the go command for builds and related operations. A
workspace that does not specify modules to be used cannot be used to do
builds from local modules.
go.work files are line-oriented. Each line holds a single directive,
made up of a keyword followed by arguments. For example:
go 1.18
use ../foo/bar
use ./baz
replace example.com/foo v1.2.3 => example.com/bar v1.4.5
The leading keyword can be factored out of adjacent lines to create a block,
like in Go imports.
use (
../foo/bar
./baz
)
The use directive specifies a module to be included in the workspace\'s
set of main modules. The argument to the use directive is the directory
containing the module\'s go.mod file.
The go directive specifies the version of Go the file was written at. It
is possible there may be future changes in the semantics of workspaces
that could be controlled by this version, but for now the version
specified has no effect.
The replace directive has the same syntax as the replace directive in a
go.mod file and takes precedence over replaces in go.mod files. It is
primarily intended to override conflicting replaces in different workspace
modules.
To determine whether the go command is operating in workspace mode, use
the "go env GOWORK" command. This will specify the workspace file being
used.
Usage:
go work <command> [arguments]
The commands are:
edit edit go.work from tools or scripts
init initialize workspace file
sync sync workspace build list to modules
use add modules to workspace file
Use "go help work <command>" for more information about a command.
使用use
指令指定包含在workspace
里的module
集。use
指令后紧接着的是包含了模块的go.mod
文件的目录–相对go.work
的目录。
...February 10, 2022
情绪绑定
#
先把一样好(好看、好听、好闻)的东西抛出来,收集大众的积极情绪,进而把大家的情绪控制。
当这样东西喜,你就跟着喜;当这样东西悲,你就跟着悲;当这样东西静,你就跟着静。当这样东西动,你却看不到了。
信仰缺失的年代,把自己交付给这样的东西,只为换到一丝“慰挤”。
精神上的追求太难找到共鸣了,不如转而追求物质上的欢喜。每天吃吃喝喝,打打闹闹,不以物喜,不以己悲,不是挺好。
歌好听,那就听,何必在意歌手爱天怼地。如果真的这么较真,最好听歌前做好背景调查,如果不慎那歌手竟信息不详,那只好叫耳朵过滤掉了。若实在忍不住,也不妨在确实之前先恩施一番,以传我宽大之名。当然,如有丝言片语,只要未到石锤之境,自然轻松忽略。毕竟,真假难辨,不如不辩。
万一真的发现了黑历史,这时就要斟酌一番了,继续爱如谦,抑或恨如龙,搞不好就被别人发现你居然喜欢“黑”歌手,承担巨大的社会压力,就得不偿失了。
但是,害怕很难成事,只会坏事。如果只因为怕,那如何能算英雄,或者竟连个孤勇者都算不上,实在于心难安。那何不拉拢一批共同情绪的人,把那位别人先打黑。
嘿,只需证明别人是错的,何苦花费心思证明自己是对呢!
嗯,情绪输出总算有着落了,难受的只要是别人,自己就永远开心了,谁管别人是亲是疏,是喜是恶呢。只要不管不顾,虎牛之力也拿我没辙。对别人施暴,哪怕隔着个屏幕,也能爽到嗨。
别人这时就难受了:我好心劝你们远离毒瘤,居然不识抬举,还要拉帮结派来搞我。真的是越想越气,越气越想。奈何对方人多势众,单拳难敌四手。
自诩孤高者,自然不屑于群斗,但被逼到墙角了,也不得不群起。但标准越高,规模自然越小,苦费心思,依然难以匹敌,最后只好在猪圈方圈里丢三骂四了。
聪明人居然不懂不聪明人的想法,为什么敢自认聪明呢?
懂的话,大抵不会自称聪明人,而要转称愚人了吧。不聪明人也不真的不聪明,只是知道往身上贴上聪明人标签,更多时候只捞得个劳苦功低、得不偿失,活得还不如马屁精。
说到马屁精,我就猜到马屁是香的,或至少在喜爱之人闻来别有一番风味。
马屁精自然是冤枉的,不过说了几句某人爱听的话,或者不小心成了习惯–见某人说某话,别人就来指责他,并打上马屁精标签,在圈里不断丢三骂四。只是不对你这样说话,你就这么生气,别人真的是坏。
谁怪你不是某人呢?你若竟是某人,想听几句某话,那还不难。只怕你成了某人,你还嫌少呢!
马屁精也不全是敌人,是非精、八卦精等“朋友”是大大的有。而且,精的本事也不能小,至少要在亦敌亦友的关系转变中拿捏得准确无误。不然闹出“人门前弄是非,精面前摆事实”的笑话来,就颜面无存了。
精,未成人之前,或竟不做人,选择做精,自然是如老鼠过街,人人喊打。
既然是精,那就必须没有情绪,笑脸迎臭脸自然是家常便饭。但只要熬出头,拥有一星半点某人之像,好生活自然而至,竟也开始享受到了某话。
路漫漫兮,修就是了。怎么修的,你就别管了。好好的丢三骂四还不够,还敢来管修的事,怕不是吃饱了思起淫欲来。
精在那里,你不骂,你敢往这边看,你怕不是想吃大过年不想吃的饭了。就不怕,我饭都不给你吃,把你饿成精。
原来精是饿出来的!
January 28, 2022
有一些东西,做了一些事情。
有什么东西,做了什么呢?
有文件、结构体、特征、类型,调用了函数、方法,读了文件/读了body,算了结果,写了文件/答了请求。
IO or 计算。
或者说,更强调IO,还是计算。
内存安全
#
并发安全
#
January 28, 2022
源码
#
# 下载
git clone git@github.com:bytecodealliance/wasmtime.git
# 子模块
git submodule update --init --recursive
# 安装
cargo build
如果忘了拉子模块,vscode
的rust-analyzer
会报错,导致智能提示等功能失效。
不过整个初始化过程还是有点长,等了好久才能正常使用。
阅读
#
从build.rs
开始,首先映入眼帘的是use anyhow::Context;
:
/// Provides the `context` method for `Result`.
///
/// This trait is sealed and cannot be implemented for types outside of
/// `anyhow`.
这是一个为其它类型(anyhow::Result
)引入context
方法的特征啊,多么伟大,在anyhow
包外面的类型就不要想着去实现它了,你们高攀不起的。
再看anyhow::Context
的定义:
// lib.rs:598
pub trait Context<T, E>: context::private::Sealed { // 继承了Sealed,那它又是怎么样的、做什么的呢?
/// Wrap the error value with additional context. -- 给error值包装上下文信息
fn context<C>(self, context: C) -> Result<T, Error>
where
C: Display + Send + Sync + 'static; // 能展示,并发安全,全局可见的类型值
/// Wrap the error value with additional context that is evaluated lazily
/// only once an error does occur. -- 通过传入一个FnOnce的函数来延迟获取上下文信息
fn with_context<C, F>(self, f: F) -> Result<T, Error>
where
C: Display + Send + Sync + 'static,
F: FnOnce() -> C;
}
// context.rs:170
pub(crate) mod private { // pub(crate)表明这个mod只能在本crate里被使用
use super::*; // 使用父mod的东西
pub trait Sealed {} // 特征里没有方法
impl<T, E> Sealed for Result<T, E> where E: ext::StdError {} // 为Result实现Sealed
impl<T> Sealed for Option<T> {} // 为Option实现Sealed
}
主要逻辑
#
做了哪些事情呢?
...January 26, 2022
如果我在创建镜像时把源码也打包了进去,要怎么防止别人通过这个镜像把源码给窃取了呢?
加密
镜像加密
源码加密:在COPY源码进去之前先加密;这种适合服务器不是自己的,并且在局域网里的(接过医院系统的应该都懂吧);留这样一份加密源码也只是在方便有bug时可以快速修复的同时,还可以稍微保护一下源码;
先使用zip压缩源码:`zip -q -r code.zip ./code`;
再使用gpg加密:`gpg --yes --batch --passphrase=123 -c ebpf.zip`; -- 通过`--yes --batch --passphrase`三个选项避免键盘交互,最后生成`ebpf.zip.gpg`。
后续进入容器后,使用gpg解密:`gpg -o ebpf2.zip -d ebpf.zip.gpg`;
再使用unzip解压:`unzip -d ebpf2 ebpf2.zip`。
在镜像构建后,还要防止docker history -H cb0b42c0cb03 --no-trunc=true
查看镜像构建历史时,泄露秘钥等信息。– 可使用多阶段构建:在前一阶段使用密钥加密源码,后一阶段复制加密源码,从而避免密钥泄露。因为一般只需要把后一阶段构建出来的镜像分发出去就好了,而查看后一阶段构建出来的镜像的构建历史,是看不到密钥信息的(查看前一阶段的构建历史才会看到)。
dockerfile COPY before mkdir will get a no such file or directory
error
#
error:
```dockerfile
# …
RUN mkdir -p /abc
COPY –from=builder /opt/efg /abc/efg
```
没有指定创建/abc/efg
目录,会导致后续想读取该目录内容时报错:no such file or directory
success:
...