Original address:How did goslip get its maximum capacity?
Preface
In “Understanding Go Slice in Depth”, we mentioned the processing logic of “obtaining the maximum capacity size that can be applied according to its type size”. Today, we will go deeper into what the bottom layer has done and what knowledge points are involved.
The corresponding code for gosice is as follows:
func makeslice(et *_type, len, cap int) slice {
maxElements := maxSliceCap(et.size)
if len < 0 || uintptr(len) > maxElements {
...
}
if cap < len || uintptr(cap) > maxElements {
...
}
p := mallocgc(et.size*uintptr(cap), et, true)
return slice{p, len, cap}
}
According to the logic you want to pursue, you have locatedmaxSliceCap
Method, it will be based onThe size of the current type gets the maximum allowed capacity size.To make threshold judgment, that is, security check. This is a shallow understanding. Let’s keep on chasing it and see what else has been done.
maxSliceCap
func maxSliceCap(elemsize uintptr) uintptr {
if elemsize < uintptr(len(maxElems)) {
return maxElems[elemsize]
}
return maxAlloc / elemsize
}
maxElems
var maxElems = [...]uintptr{
^uintptr(0),
maxAlloc / 1, maxAlloc / 2, maxAlloc / 3, maxAlloc / 4,
maxAlloc / 5, maxAlloc / 6, maxAlloc / 7, maxAlloc / 8,
maxAlloc / 9, maxAlloc / 10, maxAlloc / 11, maxAlloc / 12,
maxAlloc / 13, maxAlloc / 14, maxAlloc / 15, maxAlloc / 16,
maxAlloc / 17, maxAlloc / 18, maxAlloc / 19, maxAlloc / 20,
maxAlloc / 21, maxAlloc / 22, maxAlloc / 23, maxAlloc / 24,
maxAlloc / 25, maxAlloc / 26, maxAlloc / 27, maxAlloc / 28,
maxAlloc / 29, maxAlloc / 30, maxAlloc / 31, maxAlloc / 32,
}
maxElems
Is a lookup table that contains some predefined slice maximum capacity values, and the index is the type size of slice elements. But the value looks “strange” not familiar, what are they? The three core points are as follows:
- ^uintptr(0)
- maxAlloc
- maxAlloc / typeSize
^uintptr(0)
func main() {
log.Printf("uintptr: %v\n", uintptr(0))
log.Printf("^uintptr: %v\n", ^uintptr(0))
}
Output results:
2019/01/05 17:51:52 uintptr: 0
2019/01/05 17:51:52 ^uintptr: 18446744073709551615
Let’s take a look at the output. It’s amazing. Why is it 18446744073709551615 after inversion?
What is uintptr
Before analyzing, we need to know the essence (true face) of uintptr, that is, what is its type, as follows:
type uintptr uintptr
Uintptr’s type is custom type, then find its true face, as follows:
#ifdef _64BIT
typedef uint64 uintptr;
#else
typedef uint32 uintptr;
#endif
Through the analysis of the above codes, the following conclusions can be drawn:
- In a 32-bit system, uintptr is uint32 type and occupies 4 bytes in size
- Under 64-bit system, uintptr is uint64 type, occupying 8 bytes
What did ^uintptr do
The bitwise operator is used toBitwise XOR, as follows:
func main() {
log.Println(^1)
log.Println(^uint64(0))
}
Output results:
2019/01/05 20:44:49 -2
2019/01/05 20:44:49 18446744073709551615
Next, let’s analyze what these two pieces of code have done.
^1
Binary: 0001
Reversal by Bit: 1110
The number is a signed integer, and the most significant bit is the sign bit. The lower three digits represent the numerical value. According to the previous explanation, the highest bit is 1, so it is expressed as-. After inversion, 110 corresponds to decimal -2
^uint64(0)
Binary: 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000
Bitwise inversion: 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111
The number is an unsigned integer, and the decimal value obtained by inverting this bit is 18446744073709551615
Does this value look familiar? Yes, it is^uintptr(0)
The value of. It also confirms the fact that its underlying data type is uint64 (64 bits in this machine). At the same time, it also represents the following:
- math.MaxUint64
- 2 to the 64th power minus 1
maxAlloc
const GoarchMips = 0
const GoarchMipsle = 0
const GoarchWasm = 0
...
_64bit = 1 << (^uintptr(0) >> 63) / 2
heapAddrBits = (_64bit*(1-sys.GoarchWasm))*48 + (1-_64bit+sys.GoarchWasm)*(32-(sys.GoarchMips+sys.GoarchMipsle))
maxAlloc = (1 << heapAddrBits) - (1-_64bit)*1
maxAlloc
YesMaximum virtual memory space allowed to be allocated by users. At 64 bits, the maximum allocation is theoretically possible1 << heapAddrBits
Bytes. At 32 bits, the maximum allocatable is less than1 << 32
Byte
In this article, it is only necessary to know what it carries. The specific memory management in the future will be described in the article
Note: this variable is at go 10.1_MaxMem
, go 11.4 has been changed tomaxAlloc
. correlativeheapAddrBits
The calculation method has also changed
maxAlloc / typeSize
We recall once againmaxSliceCap
This time, the focus is on control logic, as follows:
// func makeslice
maxElements := maxSliceCap(et.size)
...
// func maxSliceCap
if elemsize < uintptr(len(maxElems)) {
return maxElems[elemsize]
}
return maxAlloc / elemsize
Through this code and Slice context logic, we can know when we want to get the maximum capacity of this type. According to the corresponding type size, the index will be looked up in the lookup table (the index is of type size, and the order of placement is considered). It will only be calculated manually “in case of necessity”. The memory byte size finally calculated is an integer multiple of this type of size.
The setting of lookup table is more like an optimization logic. Reduce common computational overhead:)
Summary
Through the analysis of this article, we can get the maximum capacity allowed by Slice, and the currentValue typeAnd the currentNumber of platform bitsHave a direct relationship
Last
This article and“Go unsafe.Pointer, A Little Unsafe but Bright”Belong together“a deeper understanding of gosice”The relevant chapters of the. If you have doubts about these fragments when reading the source code. Remember to do everything possible to dig deeper and understand it
In fact, a short sentence contains many knowledge points. I hope this article is good enough to help you solve your doubts.
Note: This Go code is based on version 11.4