mantidimaging.core.parallel.shared module

mantidimaging.core.parallel.shared.arithmetic(func, i)[source]
mantidimaging.core.parallel.shared.create_partial(func, fwd_function, **kwargs)[source]

Create a partial using functools.partial, to forward the kwargs to the parallel execution of imap.

If you seem to be getting nans, check if the correct fwd_function is set!

Parameters:
  • func – Function that will be executed

  • fwd_function – The function will be forwarded through function.

  • kwargs – kwargs to forward to the function func that will be executed

Returns:

The decorated forwarded function, ready for further execution

mantidimaging.core.parallel.shared.execute(partial_func: partial, num_operations: int, progress=None, msg: str = '', cores=None) None[source]

Executes a function in parallel with shared memory between the processes.

The array must have been created using parallel.utility.create_shared_array(shape, dtype).

If the input array IS NOT a shared array, the data will NOT BE CHANGED!

The reason for that is that the processes don’t work on the data, but on a copy.

When they process it and return the result, THE RESULT IS NOT ASSIGNED BACK TO REPLACE THE ORIGINAL, it is discarded.

  • imap_unordered gives the images back in random order

  • map and map_async do not improve speed performance

  • imap seems to be the best choice

Using _ in the for _ enumerate is slightly faster, because the tuple from enumerate isn’t unpacked, and thus some time is saved.

From performance tests, the chunksize doesn’t seem to make much of a difference, but having larger chunks usually led to slower performance:

Shape: (50,512,512) 1 chunk 3.06s 2 chunks 3.05s 3 chunks 3.07s 4 chunks 3.06s 5 chunks 3.16s 6 chunks 3.06s 7 chunks 3.058s 8 chunks 3.25s 9 chunks 3.45s

Parameters:
  • partial_func – A function constructed using create_partial

  • num_operations – The expected number of operations - should match the number of images being processed Also used to set the number of progress steps

  • cores – number of cores that the processing will use

  • progress – Progress instance to use for progress reporting (optional)

  • msg – Message to be shown on the progress bar

Returns:

mantidimaging.core.parallel.shared.inplace1(func, i, **kwargs)[source]
mantidimaging.core.parallel.shared.inplace2(func, i, **kwargs)[source]
mantidimaging.core.parallel.shared.inplace3(func, i, **kwargs)[source]
mantidimaging.core.parallel.shared.inplace_second_2d(func, i, **kwargs)[source]
mantidimaging.core.parallel.shared.return_to_second(func, i, **kwargs)[source]
mantidimaging.core.parallel.shared.return_to_second_at_i(func, i, **kwargs)[source]
mantidimaging.core.parallel.shared.return_to_self(func, i, **kwargs)[source]