How To Share Array Of Objects In Python

I have a function in which I create a pool of processes. More over I use multiprocessing.Value() and multiprocessing.Lock() in order to manage some shared values between processes.

Solution 1:

The easy way of providing shared static data is simply to make it a global variable accessible to the function you want to call. If you're using an operating system which supports "fork", it is very straightforward to use global variables in child processes as long as they're constant (if you modify them, changes won't be reflected in the other processes)

import multiprocessing as mp
from random import randint

shared = ['some', 'shared', 'data', f'{randint(0,1e6)}']

deffoo():
    print(' '.join(shared))

if __name__ == "__main__":
    mp.set_start_method("fork")
    #defining "shared" here would be valid also
    p = mp.Process(target=foo)
    p.start()
    p.join()
    
    print(' '.join(shared)) #same random number means "shared" is same object

This won't work when using "spawn" as the start method (the only one available on windows), because the memory of the parent is not shared in any way with the child, so the child must "import" the main file to gain access to whatever the target function is (this is also why you can run into problems with decorators.) If you define your data outside the if __name__ == "__main__": block, it will kinda work, but you will have made separate copies of the data, which can be undesirable if it's big, slow to create, or can change each time it's created.

import multiprocessing as mp
from random import randint

shared = ['some', 'shared', 'data', f'{randint(0,1e6)}']

deffoo():
    print(' '.join(shared))

if __name__ == "__main__":
    mp.set_start_method("spawn")

    p = mp.Process(target=foo)
    p.start()
    p.join()
    
    print(' '.join(shared)) #different number means different copy of "shared" (1 a million chance of being same i guess...)