org.openimaj.util.parallel

Class Parallel

java.lang.Object

org.openimaj.util.parallel.Parallel

public class Parallel
extends Object

Parallel processing utilities for looping.

Inspired by the .NET Task Parallel Library. Allows control over the way data is partitioned using inspiration from Reed Copsey's blog.

Author:

Jonathon Hare (jsh2@ecs.soton.ac.uk)

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Parallel.IntRange An integer range with a step size.

Constructor Summary

Constructors

Constructor and Description
Parallel()

Method Summary

Modifier and Type	Method and Description
static <T> void	forEach(Iterable<T> objects, Operation<T> op) Parallel ForEach loop over Iterable data.
static <T> void	forEach(Iterable<T> objects, Operation<T> op, ThreadPoolExecutor pool) Parallel ForEach loop over Iterable data.
static <T> void	forEach(Partitioner<T> partitioner, Operation<T> op) Parallel ForEach loop over partitioned data.
static <T> void	forEach(Partitioner<T> partitioner, Operation<T> op, ThreadPoolExecutor pool) Parallel ForEach loop over partitioned data.
static <T> void	forEachPartitioned(Partitioner<T> partitioner, Operation<Iterator<T>> op) Parallel ForEach loop over batched partitioned data.
static <T> void	forEachPartitioned(Partitioner<T> partitioner, Operation<Iterator<T>> op, ThreadPoolExecutor pool) Parallel ForEach loop over partitioned data with batches of data.
static <T> void	forEachUnpartitioned(Iterator<T> data, Operation<T> op) Parallel ForEach loop over unpartitioned data.
static <T> void	forEachUnpartitioned(Iterator<T> data, Operation<T> op, ThreadPoolExecutor pool) Parallel ForEach loop over unpartitioned data.
static void	forIndex(int start, int stop, int incr, Operation<Integer> op) Parallel integer for loop.
static void	forIndex(int start, int stop, int incr, Operation<Integer> op, ThreadPoolExecutor pool) Parallel integer for loop.
static void	forRange(int start, int stop, int incr, Operation<Parallel.IntRange> op) Parallel integer for loop.
static void	forRange(int start, int stop, int incr, Operation<Parallel.IntRange> op, ThreadPoolExecutor pool) Parallel integer for loop.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Parallel

public Parallel()

Method Detail

forIndex

public static void forIndex(int start,
                            int stop,
                            int incr,
                            Operation<Integer> op,
                            ThreadPoolExecutor pool)

Parallel integer for loop.

Parameters:

start - starting value

stop - stopping value

incr - increment amount

op - operation to perform

pool - the thread pool.

forIndex

public static void forIndex(int start,
                            int stop,
                            int incr,
                            Operation<Integer> op)

Parallel integer for loop. Uses the default global thread pool.

Parameters:

start - starting value

stop - stopping value

incr - increment amount

op - operation to perform

See Also:

GlobalExecutorPool.getPool()

forRange

public static void forRange(int start,
                            int stop,
                            int incr,
                            Operation<Parallel.IntRange> op)

Parallel integer for loop. Fundamentally this is the same as forIndex(int, int, int, Operation), but potentially slightly faster as it avoids auto-boxing/unboxing and results in fewer method calls. The downside is that users have to write an extra loop to iterate over the Parallel.IntRange object. Uses the default global thread pool.

Parameters:

start - starting value

stop - stopping value

incr - increment amount

op - operation to perform

forRange

public static void forRange(int start,
                            int stop,
                            int incr,
                            Operation<Parallel.IntRange> op,
                            ThreadPoolExecutor pool)

Parallel integer for loop. Fundamentally this is the same as forIndex(int, int, int, Operation, ThreadPoolExecutor), but potentially slightly faster as it avoids auto-boxing/unboxing and results in fewer method calls. The downside is that users have to write an extra loop to iterate over the Parallel.IntRange object.

Parameters:

start - starting value

stop - stopping value

incr - increment amount

op - operation to perform

pool - the thread pool.

forEach

public static <T> void forEach(Iterable<T> objects,
                               Operation<T> op,
                               ThreadPoolExecutor pool)

Parallel ForEach loop over Iterable data. The data is automatically partitioned; if the data is a List, then a RangePartitioner is used, otherwise a GrowingChunkPartitioner is used.

Type Parameters:

T - type of the data items

Parameters:

objects - the data

op - the operation to apply

pool - the thread pool.

See Also:

GlobalExecutorPool.getPool()

forEach

public static <T> void forEach(Iterable<T> objects,
                               Operation<T> op)

Parallel ForEach loop over Iterable data. Uses the default global thread pool. The data is automatically partitioned; if the data is a List, then a RangePartitioner is used, otherwise a GrowingChunkPartitioner is used.

Type Parameters:

T - type of the data items

Parameters:

objects - the data

op - the operation to apply

See Also:

GlobalExecutorPool.getPool()

forEach

public static <T> void forEach(Partitioner<T> partitioner,
                               Operation<T> op)

Parallel ForEach loop over partitioned data. Uses the default global thread pool.

Type Parameters:

T - type of the data items

Parameters:

partitioner - the partitioner applied to the data

op - the operation to apply

See Also:

GlobalExecutorPool.getPool()

forEach

public static <T> void forEach(Partitioner<T> partitioner,
                               Operation<T> op,
                               ThreadPoolExecutor pool)

Parallel ForEach loop over partitioned data.

Implementation details: 1.) create partitions enumerator 2.) schedule nprocs partitions 3.) while there are still partitions to process 3.1) on completion of a partition schedule the next one 4.) wait for completion of remaining partitions

Type Parameters:

T - type of the data items

Parameters:

partitioner - the partitioner applied to the data

op - the operation to apply

pool - the thread pool.

forEachUnpartitioned

public static <T> void forEachUnpartitioned(Iterator<T> data,
                                            Operation<T> op)

Parallel ForEach loop over unpartitioned data. This is effectively the same as using a FixedSizeChunkPartitioner with a chunk size of 1, but with slightly less overhead. The unpartitioned for-each loop has slightly less throughput than a partitioned for-each loop, but exhibits much less delay in scheduling an item for processing as a partition does not have to first be populated. The unpartitioned for-each loop is particularly useful for processing temporal Streams of data.

Implementation details: 1.) create partitions enumerator 2.) schedule nprocs partitions 3.) while there are still partitions to process 3.1) on completion of a partition schedule the next one 4.) wait for completion of remaining partitions

Type Parameters:

T - type of the data items

Parameters:

data - the iterator of data items

op - the operation to apply

forEachUnpartitioned

public static <T> void forEachUnpartitioned(Iterator<T> data,
                                            Operation<T> op,
                                            ThreadPoolExecutor pool)

Parallel ForEach loop over unpartitioned data. This is effectively the same as using a FixedSizeChunkPartitioner with a chunk size of 1, but with slightly less overhead. The unpartitioned for-each loop has slightly less throughput than a partitioned for-each loop, but exhibits much less delay in scheduling an item for processing as a partition does not have to first be populated. The unpartitioned for-each loop is particularly useful for processing temporal Streams of data.

Implementation details: 1.) create partitions enumerator 2.) schedule nprocs partitions 3.) while there are still partitions to process 3.1) on completion of a partition schedule the next one 4.) wait for completion of remaining partitions

Type Parameters:

T - type of the data items

Parameters:

data - the iterator of data items

op - the operation to apply

pool - the thread pool.

forEachPartitioned

public static <T> void forEachPartitioned(Partitioner<T> partitioner,
                                          Operation<Iterator<T>> op,
                                          ThreadPoolExecutor pool)

Parallel ForEach loop over partitioned data with batches of data.

Implementation details: 1.) create partitions enumerator 2.) schedule nprocs partitions 3.) while there are still partitions to process 3.1) on completion of a partition schedule the next one 4.) wait for completion of remaining partitions

Type Parameters:

T - type of the data items

Parameters:

partitioner - the partitioner applied to the data

op - the operation to apply

pool - the thread pool.

forEachPartitioned

public static <T> void forEachPartitioned(Partitioner<T> partitioner,
                                          Operation<Iterator<T>> op)

Parallel ForEach loop over batched partitioned data. Uses the default global thread pool.

Type Parameters:

T - type of the data items

Parameters:

partitioner - the partitioner applied to the data

op - the operation to apply

See Also:

GlobalExecutorPool.getPool()