Python Multiprocess, Global Variable Not Updated, Resolved


Python Multiprocess, Global Variable Not Updated

Problem: Sub-process global variables are not updated as expected, resulting in data not being written to disk. The process pool must be in theif __name__ == '__main__'The following is used
Reproduction: see the experimental code.
Environment: centos7, python3.7
Reason: of the subprocess__name__Attribute value is not'__main__'
Solution: The resources needed should be inif __name__ == '__main__':Before the statement. The value after this statement is not passed to the subprocess.


from multiprocessing import Pool

date = '初始日期'
def foo():
​    print(f'date的值: {date}')

if __name__ == '__main__':
​    date = '覆盖初始日期'

​    # 多进程,全部不能覆盖
​    with Pool(2) as mypool:
​        for i in range(2):
​            print(f'进程{i}:')
​            mypool.apply(foo,)


date的值: 初始日期
date的值: 初始日期


Python’s multi-process implementation is different under UNIX-like OS and Windows OS. The former can use fork, while the latter is a python interpreter that starts a new process and then transfers it to that process using the resources needed for serialization. At this time, it prints under those modules of the subprocess.locals()It will show inside__name__Field is not__main__(What I’m showing is_ua_main__)。
In addition, if you are running python scripts, multiple processes must be in theif __name__ == '__main__':Statement, otherwise an error will be reported.
In addition, if you want to pass between processes, please use the official queue method or pipeline, because the values of global variables between the parent process and the child process are independent, changing one of them will not synchronize to the other process.

The link to the official website is as follows, which clearly explains some points of attention and pits of multi-process: highlight=multiprocessing#module-multiprocessing