Original address:How did goslip get its maximum capacity?

Preface

In “Understanding Go Slice in Depth”, we mentioned the processing logic of “obtaining the maximum capacity size that can be applied according to its type size”. Today, we will go deeper into what the bottom layer has done and what knowledge points are involved.

The corresponding code for gosice is as follows:

func makeslice(et *_type, len, cap int) slice {
    maxElements := maxSliceCap(et.size)
    if len < 0 || uintptr(len) > maxElements {
        ...
    }

    if cap < len || uintptr(cap) > maxElements {
        ...
    }

    p := mallocgc(et.size*uintptr(cap), et, true)
    return slice{p, len, cap}
}

According to the logic you want to pursue, you have locatedmaxSliceCapMethod, it will be based onThe size of the current type gets the maximum allowed capacity size.To make threshold judgment, that is, security check. This is a shallow understanding. Let’s keep on chasing it and see what else has been done.

maxSliceCap

func maxSliceCap(elemsize uintptr) uintptr {
    if elemsize < uintptr(len(maxElems)) {
        return maxElems[elemsize]
    }
    return maxAlloc / elemsize
}

maxElems

var maxElems = [...]uintptr{
    ^uintptr(0),
    maxAlloc / 1, maxAlloc / 2, maxAlloc / 3, maxAlloc / 4,
    maxAlloc / 5, maxAlloc / 6, maxAlloc / 7, maxAlloc / 8,
    maxAlloc / 9, maxAlloc / 10, maxAlloc / 11, maxAlloc / 12,
    maxAlloc / 13, maxAlloc / 14, maxAlloc / 15, maxAlloc / 16,
    maxAlloc / 17, maxAlloc / 18, maxAlloc / 19, maxAlloc / 20,
    maxAlloc / 21, maxAlloc / 22, maxAlloc / 23, maxAlloc / 24,
    maxAlloc / 25, maxAlloc / 26, maxAlloc / 27, maxAlloc / 28,
    maxAlloc / 29, maxAlloc / 30, maxAlloc / 31, maxAlloc / 32,
}

maxElemsIs a lookup table that contains some predefined slice maximum capacity values, and the index is the type size of slice elements. But the value looks “strange” not familiar, what are they? The three core points are as follows:

• ^uintptr(0)
• maxAlloc
• maxAlloc / typeSize

^uintptr(0)

func main() {
    log.Printf("uintptr: %v\n", uintptr(0))
    log.Printf("^uintptr: %v\n", ^uintptr(0))
}

Output results:

2019/01/05 17:51:52 uintptr: 0
2019/01/05 17:51:52 ^uintptr: 18446744073709551615

Let’s take a look at the output. It’s amazing. Why is it 18446744073709551615 after inversion?

What is uintptr

Before analyzing, we need to know the essence (true face) of uintptr, that is, what is its type, as follows:

type uintptr uintptr

Uintptr’s type is custom type, then find its true face, as follows:

#ifdef _64BIT
typedef    uint64        uintptr;
#else
typedef    uint32        uintptr;
#endif

Through the analysis of the above codes, the following conclusions can be drawn:

• In a 32-bit system, uintptr is uint32 type and occupies 4 bytes in size
• Under 64-bit system, uintptr is uint64 type, occupying 8 bytes

What did ^uintptr do

The bitwise operator is used toBitwise XOR, as follows:

func main() {
    log.Println(^1)
    log.Println(^uint64(0))
}

Output results:

2019/01/05 20:44:49 -2
2019/01/05 20:44:49 18446744073709551615

Next, let’s analyze what these two pieces of code have done.

^1

Binary: 0001

Reversal by Bit: 1110

The number is a signed integer, and the most significant bit is the sign bit. The lower three digits represent the numerical value. According to the previous explanation, the highest bit is 1, so it is expressed as-. After inversion, 110 corresponds to decimal -2

^uint64(0)

Binary: 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000

Bitwise inversion: 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111 1111

The number is an unsigned integer, and the decimal value obtained by inverting this bit is 18446744073709551615

Does this value look familiar? Yes, it is^uintptr(0)The value of. It also confirms the fact that its underlying data type is uint64 (64 bits in this machine). At the same time, it also represents the following:

• math.MaxUint64
• 2 to the 64th power minus 1

maxAlloc

const GoarchMips = 0
const GoarchMipsle = 0
const GoarchWasm = 0

...

_64bit = 1 << (^uintptr(0) >> 63) / 2

heapAddrBits = (_64bit*(1-sys.GoarchWasm))*48 + (1-_64bit+sys.GoarchWasm)*(32-(sys.GoarchMips+sys.GoarchMipsle))

maxAlloc = (1 << heapAddrBits) - (1-_64bit)*1

maxAllocYesMaximum virtual memory space allowed to be allocated by users. At 64 bits, the maximum allocation is theoretically possible1 << heapAddrBitsBytes. At 32 bits, the maximum allocatable is less than1 << 32Byte

In this article, it is only necessary to know what it carries. The specific memory management in the future will be described in the article

Note: this variable is at go 10.1_MaxMem, go 11.4 has been changed tomaxAlloc. correlativeheapAddrBitsThe calculation method has also changed

maxAlloc / typeSize

We recall once againmaxSliceCapThis time, the focus is on control logic, as follows:

// func makeslice
maxElements := maxSliceCap(et.size)

...

// func maxSliceCap
if elemsize < uintptr(len(maxElems)) {
    return maxElems[elemsize]
}
return maxAlloc / elemsize

Through this code and Slice context logic, we can know when we want to get the maximum capacity of this type. According to the corresponding type size, the index will be looked up in the lookup table (the index is of type size, and the order of placement is considered). It will only be calculated manually “in case of necessity”. The memory byte size finally calculated is an integer multiple of this type of size.

The setting of lookup table is more like an optimization logic. Reduce common computational overhead:)

Summary

Through the analysis of this article, we can get the maximum capacity allowed by Slice, and the currentValue typeAnd the currentNumber of platform bitsHave a direct relationship

Last

This article and“Go unsafe.Pointer, A Little Unsafe but Bright”Belong together“a deeper understanding of gosice”The relevant chapters of the. If you have doubts about these fragments when reading the source code. Remember to do everything possible to dig deeper and understand it

In fact, a short sentence contains many knowledge points. I hope this article is good enough to help you solve your doubts.

Note: This Go code is based on version 11.4