Parallelize
✖
Parallelize
Details and Options
- Parallelize[expr] automatically distributes different parts of the evaluation of expr among different available kernels and processors.
- Parallelize[expr] normally gives the same result as evaluating expr, except for side effects during the computation.
- Parallelize has attribute HoldFirst, so that expressions are not evaluated before parallelization.
- The following options can be given:
-
Method Automatic granularity of parallelization DistributedContexts $DistributedContexts contexts used to distribute symbols to parallel computations ProgressReporting $ProgressReporting whether to report the progress of the computation - The Method option specifies the parallelization method to use. Possible settings include:
-
"CoarsestGrained" break the computation into as many pieces as there are available kernels "FinestGrained" break the computation into the smallest possible subunits "EvaluationsPerKernel"->e break the computation into at most e pieces per kernel "ItemsPerEvaluation"->m break the computation into evaluations of at most m subunits each Automatic compromise between overhead and load balancing - Method->"CoarsestGrained" is suitable for computations involving many subunits, all of which take the same amount of time. It minimizes overhead but does not provide any load balancing.
- Method->"FinestGrained" is suitable for computations involving few subunits whose evaluations take different amounts of time. It leads to higher overhead but maximizes load balancing.
- The DistributedContexts option specifies which symbols appearing in expr have their definitions automatically distributed to all available kernels before the computation.
- The default value is DistributedContexts:>$DistributedContexts with $DistributedContexts:=$Context, which distributes definitions of all symbols in the current context, but does not distribute definitions of symbols from packages.
- The ProgressReporting option specifies whether to report the progress of the parallel computation.
- The default value is ProgressReporting:>$ProgressReporting.
- Parallelize[f[…]] parallelizes these functions that operate on a list element by element: Apply, AssociationMap, Cases, Count, FreeQ, KeyMap, KeySelect, KeyValueMap, Map, MapApply, MapIndexed, MapThread, Comap, ComapApply, MemberQ, Pick, Scan, Select and Through.
- Parallelize[iter] parallelizes the iterators Array, Do, Product, Sum, Table.
- Parallelize[list] evaluates the elements of list in parallel.
- Parallelize[f[…]] can parallelize listable and associative functions and inner and outer products. »
- Parallelize[cmd1;cmd2;…] wraps Parallelize around each cmdi and evaluates these in sequence. »
- Parallelize[s=expr] is converted to s=Parallelize[expr].
- Parallelize[expr] evaluates expr sequentially if expr is not one of the cases recognized by Parallelize.
Parallelize Options
Parallelize Scope
Examples
open allclose allBasic Examples (4)Summary of the most common use cases
https://wolfram.com/xid/01yyzzmytcu-jnad22
https://wolfram.com/xid/01yyzzmytcu-bw6ame
Functions defined interactively can immediately be used in parallel:
https://wolfram.com/xid/01yyzzmytcu-kv8i0c
https://wolfram.com/xid/01yyzzmytcu-f2ig2p
Longer computations display information about their progress and estimated time to completion:
https://wolfram.com/xid/01yyzzmytcu-o6k425
Scope (23)Survey of the scope of standard use cases
Listable Functions (1)
Structure-Preserving Functions (8)
Many functional programming constructs that preserve list structure parallelize:
https://wolfram.com/xid/01yyzzmytcu-lc1lu
https://wolfram.com/xid/01yyzzmytcu-ebrc8w
https://wolfram.com/xid/01yyzzmytcu-ci4udx
https://wolfram.com/xid/01yyzzmytcu-6udj5
https://wolfram.com/xid/01yyzzmytcu-dnsia2
https://wolfram.com/xid/01yyzzmytcu-syzda
https://wolfram.com/xid/01yyzzmytcu-xvgl0n
f@@@list is equivalent to MapApply[f,list]:
https://wolfram.com/xid/01yyzzmytcu-3l9m3k
https://wolfram.com/xid/01yyzzmytcu-cm2yym
https://wolfram.com/xid/01yyzzmytcu-jjasnc
https://wolfram.com/xid/01yyzzmytcu-dny9d8
The result need not have the same length as the input:
https://wolfram.com/xid/01yyzzmytcu-gkpw84
https://wolfram.com/xid/01yyzzmytcu-buu3i5
https://wolfram.com/xid/01yyzzmytcu-ctrdfo
Without a function, Parallelize simply evaluates the elements in parallel:
https://wolfram.com/xid/01yyzzmytcu-kq29c7
Reductions (4)
Count the number of primes up to one million:
https://wolfram.com/xid/01yyzzmytcu-k6j9nd
Check whether 93 occurs in a list of the first 100 primes:
https://wolfram.com/xid/01yyzzmytcu-xcvi9
Check whether a list is free of 5:
https://wolfram.com/xid/01yyzzmytcu-bjhz1v
The argument does not have to be an explicit List:
https://wolfram.com/xid/01yyzzmytcu-d65xgs
Inner and Outer Products (2)
Iterators (3)
Evaluate a table in parallel, with or without an iterator variable:
https://wolfram.com/xid/01yyzzmytcu-d2892b
https://wolfram.com/xid/01yyzzmytcu-tvpnl
Generate an array in parallel:
https://wolfram.com/xid/01yyzzmytcu-ezqvw
Evaluate sums and products in parallel:
https://wolfram.com/xid/01yyzzmytcu-gpobym
https://wolfram.com/xid/01yyzzmytcu-evk3is
The evaluation of the function happens in parallel:
https://wolfram.com/xid/01yyzzmytcu-c6z511
https://wolfram.com/xid/01yyzzmytcu-4x39r
The list of file names is expanded locally on the subkernels:
https://wolfram.com/xid/01yyzzmytcu-3lk4ul
Associative Functions (1)
Functions with the attribute Flat automatically parallelize:
https://wolfram.com/xid/01yyzzmytcu-e0oul5
https://wolfram.com/xid/01yyzzmytcu-g432u7
https://wolfram.com/xid/01yyzzmytcu-cyfhs6
Functions for Associations (4)
Parallelize AssociationMap:
https://wolfram.com/xid/01yyzzmytcu-w7eg90
Parallelize KeyMap:
https://wolfram.com/xid/01yyzzmytcu-oztu3l
Parallelize KeySelect:
https://wolfram.com/xid/01yyzzmytcu-8hbu8x
Parallelize KeyValueMap:
https://wolfram.com/xid/01yyzzmytcu-1jpicz
Generalizations & Extensions (4)Generalized and extended use cases
Listable functions of several arguments:
https://wolfram.com/xid/01yyzzmytcu-g7fu5a
Only the right side of an assignment is parallelized:
https://wolfram.com/xid/01yyzzmytcu-tfacv
Elements of a compound expression are parallelized one after the other:
https://wolfram.com/xid/01yyzzmytcu-z3uj2k
Parallelize the generation of video frames:
https://wolfram.com/xid/01yyzzmytcu-oplta4
Options (13)Common values & functionality for each option
DistributedContexts (5)
By default, definitions in the current context are distributed automatically:
https://wolfram.com/xid/01yyzzmytcu-g9gh34
https://wolfram.com/xid/01yyzzmytcu-clbr3t
Do not distribute any definitions of functions:
https://wolfram.com/xid/01yyzzmytcu-6xao0
https://wolfram.com/xid/01yyzzmytcu-mbrc4e
Distribute definitions for all symbols in all contexts appearing in a parallel computation:
https://wolfram.com/xid/01yyzzmytcu-hu6d1y
https://wolfram.com/xid/01yyzzmytcu-s80e6
Distribute only definitions in the given contexts:
https://wolfram.com/xid/01yyzzmytcu-yht7o
https://wolfram.com/xid/01yyzzmytcu-n7ym7a
Restore the value of the DistributedContexts option to its default:
https://wolfram.com/xid/01yyzzmytcu-qkhgjy
Method (6)
Break the computation into the smallest possible subunits:
https://wolfram.com/xid/01yyzzmytcu-beyh5x
Break the computation into as many pieces as there are available kernels:
https://wolfram.com/xid/01yyzzmytcu-bsicn2
Break the computation into at most 2 evaluations per kernel for the entire job:
https://wolfram.com/xid/01yyzzmytcu-b4y3d3
Break the computation into evaluations of at most 5 elements each:
https://wolfram.com/xid/01yyzzmytcu-4yvup
The default option setting balances evaluation size and number of evaluations:
https://wolfram.com/xid/01yyzzmytcu-d7os7j
Calculations with vastly differing runtimes should be parallelized as finely as possible:
https://wolfram.com/xid/01yyzzmytcu-b82tnj
A large number of simple calculations should be distributed into as few batches as possible:
https://wolfram.com/xid/01yyzzmytcu-bq3f3v
ProgressReporting (2)
Do not show a temporary progress report:
https://wolfram.com/xid/01yyzzmytcu-o979po
Use Method"FinestGrained" for the most accurate progress report:
https://wolfram.com/xid/01yyzzmytcu-qudjj1
Applications (4)Sample problems that can be solved with this function
https://wolfram.com/xid/01yyzzmytcu-cjyqrj
Watch the results appear as they are found:
https://wolfram.com/xid/01yyzzmytcu-csbyjm
https://wolfram.com/xid/01yyzzmytcu-ewil70
Compute a whole table of visualizations:
https://wolfram.com/xid/01yyzzmytcu-j6d2dp
https://wolfram.com/xid/01yyzzmytcu-z5brn
https://wolfram.com/xid/01yyzzmytcu-i34ds
Search a range in parallel for local minima:
https://wolfram.com/xid/01yyzzmytcu-h8sk3t
https://wolfram.com/xid/01yyzzmytcu-cwfpcj
Use a shared function to record timing results as they are generated:
https://wolfram.com/xid/01yyzzmytcu-bghqh2
Set up a dynamic bar chart with the timing results:
https://wolfram.com/xid/01yyzzmytcu-bmfdby
Run a series of calculations with vastly varying runtimes:
https://wolfram.com/xid/01yyzzmytcu-d3zx6r
Properties & Relations (7)Properties of the function, and connections to other functions
For data parallel functions, Parallelize is implemented in terms of ParallelCombine:
https://wolfram.com/xid/01yyzzmytcu-iregv
https://wolfram.com/xid/01yyzzmytcu-ekxb2
https://wolfram.com/xid/01yyzzmytcu-gmyxp9
https://wolfram.com/xid/01yyzzmytcu-c2le3q
Parallel speedup can be measured with a calculation that takes a known amount of time:
https://wolfram.com/xid/01yyzzmytcu-c1edr3
Define a number of tasks with known runtimes:
https://wolfram.com/xid/01yyzzmytcu-hh54z9
The time for a sequential execution is the sum of the individual times:
https://wolfram.com/xid/01yyzzmytcu-ekyx71
https://wolfram.com/xid/01yyzzmytcu-ieg85b
Measure the speedup for parallel execution:
https://wolfram.com/xid/01yyzzmytcu-mzffh5
Finest-grained scheduling gives better load balancing and higher speedup:
https://wolfram.com/xid/01yyzzmytcu-ffvffo
Scheduling large tasks first gives even better results:
https://wolfram.com/xid/01yyzzmytcu-j2q9n7
Form the arithmetic expression 1⊗2⊗3⊗4⊗5⊗6⊗7⊗8⊗9 for ⊗ chosen from +, –, *, /:
https://wolfram.com/xid/01yyzzmytcu-iojp1g
Each list of arithmetic operations gives a simple calculation:
https://wolfram.com/xid/01yyzzmytcu-ed8xbm
https://wolfram.com/xid/01yyzzmytcu-f4vzui
Find all sequences of arithmetic operations that give 0:
https://wolfram.com/xid/01yyzzmytcu-ifym74
Display the corresponding expressions:
https://wolfram.com/xid/01yyzzmytcu-bj2c2u
Functions defined interactively are automatically distributed to all kernels when needed:
https://wolfram.com/xid/01yyzzmytcu-ysr612
https://wolfram.com/xid/01yyzzmytcu-9aj3zp
Distribute definitions manually and disable automatic distribution:
https://wolfram.com/xid/01yyzzmytcu-4luuf0
https://wolfram.com/xid/01yyzzmytcu-x3m4oa
https://wolfram.com/xid/01yyzzmytcu-psl1j3
For functions from a package, use ParallelNeeds rather than DistributeDefinitions:
https://wolfram.com/xid/01yyzzmytcu-164rkx
https://wolfram.com/xid/01yyzzmytcu-37jd03
https://wolfram.com/xid/01yyzzmytcu-tm6e7u
https://wolfram.com/xid/01yyzzmytcu-kd2hrg
Set up a random number generator that is suitable for parallel use and initialize each kernel:
https://wolfram.com/xid/01yyzzmytcu-8thusp
https://wolfram.com/xid/01yyzzmytcu-ncm1rp
Possible Issues (8)Common pitfalls and unexpected behavior
Expressions that cannot be parallelized are evaluated normally:
https://wolfram.com/xid/01yyzzmytcu-nb5lp5
https://wolfram.com/xid/01yyzzmytcu-c7p9a1
Side effects cannot be used in the function mapped in parallel:
https://wolfram.com/xid/01yyzzmytcu-kobxkj
Use a shared variable to support side effects:
https://wolfram.com/xid/01yyzzmytcu-g8q98c
https://wolfram.com/xid/01yyzzmytcu-7jqnj
If no subkernels are available, the result is computed on the master kernel:
https://wolfram.com/xid/01yyzzmytcu-hc279q
https://wolfram.com/xid/01yyzzmytcu-itu32h
If a function used is not distributed first, the result may still appear to be correct:
https://wolfram.com/xid/01yyzzmytcu-dodxy4
https://wolfram.com/xid/01yyzzmytcu-juzxie
Only if the function is distributed is the result actually calculated on the available kernels:
https://wolfram.com/xid/01yyzzmytcu-l3d40q
https://wolfram.com/xid/01yyzzmytcu-6kf0o
Definitions of functions in the current context are distributed automatically:
https://wolfram.com/xid/01yyzzmytcu-buqs89
https://wolfram.com/xid/01yyzzmytcu-djmh91
Definitions from contexts other than the default context are not distributed automatically:
https://wolfram.com/xid/01yyzzmytcu-x4grox
https://wolfram.com/xid/01yyzzmytcu-elkup8
Use DistributeDefinitions to distribute such definitions:
https://wolfram.com/xid/01yyzzmytcu-p6tavy
https://wolfram.com/xid/01yyzzmytcu-3mtydl
Alternatively, set the DistributedContexts option to include all contexts:
https://wolfram.com/xid/01yyzzmytcu-4fcrwb
https://wolfram.com/xid/01yyzzmytcu-zu57r0
Explicitly distribute the definition of a function:
https://wolfram.com/xid/01yyzzmytcu-o49jku
https://wolfram.com/xid/01yyzzmytcu-075yth
The modified definition is automatically distributed:
https://wolfram.com/xid/01yyzzmytcu-0ourt9
Suppress the automatic distribution of definitions:
https://wolfram.com/xid/01yyzzmytcu-m9tayr
https://wolfram.com/xid/01yyzzmytcu-btegrb
https://wolfram.com/xid/01yyzzmytcu-i4ig5d
Symbols defined only on the subkernels are not distributed automatically:
https://wolfram.com/xid/01yyzzmytcu-84gy2v
https://wolfram.com/xid/01yyzzmytcu-079c11
The value of $DistributedContexts is not used in Parallelize:
https://wolfram.com/xid/01yyzzmytcu-tnpbhh
https://wolfram.com/xid/01yyzzmytcu-y607it
https://wolfram.com/xid/01yyzzmytcu-uqc5ie
Set the value of the DistributedContexts option of Parallelize:
https://wolfram.com/xid/01yyzzmytcu-bvwdyb
https://wolfram.com/xid/01yyzzmytcu-oheeji
https://wolfram.com/xid/01yyzzmytcu-ce87q1
Restore all settings to their default values:
https://wolfram.com/xid/01yyzzmytcu-zd138q
https://wolfram.com/xid/01yyzzmytcu-538pew
Trivial operations may take longer when parallelized:
https://wolfram.com/xid/01yyzzmytcu-g3n24y
https://wolfram.com/xid/01yyzzmytcu-cwkgcx
Wolfram Research (2008), Parallelize, Wolfram Language function, https://reference.wolfram.com/language/ref/Parallelize.html (updated 2021).
Text
Wolfram Research (2008), Parallelize, Wolfram Language function, https://reference.wolfram.com/language/ref/Parallelize.html (updated 2021).
Wolfram Research (2008), Parallelize, Wolfram Language function, https://reference.wolfram.com/language/ref/Parallelize.html (updated 2021).
CMS
Wolfram Language. 2008. "Parallelize." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2021. https://reference.wolfram.com/language/ref/Parallelize.html.
Wolfram Language. 2008. "Parallelize." Wolfram Language & System Documentation Center. Wolfram Research. Last Modified 2021. https://reference.wolfram.com/language/ref/Parallelize.html.
APA
Wolfram Language. (2008). Parallelize. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/Parallelize.html
Wolfram Language. (2008). Parallelize. Wolfram Language & System Documentation Center. Retrieved from https://reference.wolfram.com/language/ref/Parallelize.html
BibTeX
@misc{reference.wolfram_2024_parallelize, author="Wolfram Research", title="{Parallelize}", year="2021", howpublished="\url{https://reference.wolfram.com/language/ref/Parallelize.html}", note=[Accessed: 09-January-2025
]}
BibLaTeX
@online{reference.wolfram_2024_parallelize, organization={Wolfram Research}, title={Parallelize}, year={2021}, url={https://reference.wolfram.com/language/ref/Parallelize.html}, note=[Accessed: 09-January-2025
]}