Class ProcessingService


  • public final class ProcessingService
    extends java.lang.Object
    A simple wrapper around an ExecutorService that makes it easier to process collections of items in parallel. The work items are processed by a Consumer, Function or TwoStepMapper. In particular the TwoStepMapper can be used to aggregate/reduce data from the work items and then combine the collected data into a final result.
    • Constructor Summary

      Constructors 
      Constructor Description
      ProcessingService​(java.util.concurrent.ExecutorService executor)  
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods 
      Modifier and Type Method Description
      <W,​R>
      java.util.Map<W,​R>
      compute​(java.util.Collection<W> work, int parallelism, java.util.function.Function<W,​R> computer)
      Compute an output item for each (unique) input item, and return the results as a Map.
      <W,​R>
      java.util.Map<W,​R>
      compute​(java.util.Collection<W> work, java.util.function.Function<W,​R> computer)
      Using parallelism Parallelism.CORES.
      <W,​R>
      java.util.Map<W,​R>
      compute​(java.util.Collection<W> work, java.util.function.IntSupplier parallelism, java.util.function.Function<W,​R> computer)  
      DivideAndConquer.Divider divider()  
      java.util.concurrent.ExecutorService getExecutor()  
      <W,​R>
      java.util.Collection<R>
      map​(java.util.Collection<W> work, int parallelism, java.util.function.Function<W,​R> mapper)
      Simply map each (unique) input item to an output item - a Collection of input results in a Collection of output.
      <W,​R>
      java.util.Collection<R>
      map​(java.util.Collection<W> work, java.util.function.Function<W,​R> mapper)
      Using parallelism Parallelism.CORES.
      <W,​R>
      java.util.Collection<R>
      map​(java.util.Collection<W> work, java.util.function.IntSupplier parallelism, java.util.function.Function<W,​R> mapper)  
      static ProcessingService newInstance​(java.lang.String name)  
      <W> void process​(java.util.Collection<? extends W> work, int parallelism, java.util.function.Consumer<W> processor)
      Will create at most parallelism tasks to work through the work items, processing them with processor.
      <W> void process​(java.util.Collection<? extends W> work, java.util.function.Consumer<W> processor)
      Using parallelism Parallelism.CORES.
      <W> void process​(java.util.Collection<? extends W> work, java.util.function.IntSupplier parallelism, java.util.function.Consumer<W> processor)  
      <W> void processPair​(W work1, W work2, java.util.function.Consumer<W> processor)
      Just 2 work items.
      <W> void processTriplet​(W work1, W work2, W work3, java.util.function.Consumer<W> processor)
      Just 3 work items.
      <W,​R>
      R
      reduce​(java.util.Collection<W> work, int parallelism, java.util.function.Supplier<? extends TwoStepMapper.Mergeable<W,​R>> reducer)
      Deprecated.
      v54 Use #reduceMergeable(Collection,int,Supplier>) instead
      <W,​R>
      R
      reduce​(java.util.Collection<W> work, java.util.function.IntSupplier parallelism, java.util.function.Supplier<? extends TwoStepMapper.Mergeable<W,​R>> reducer)
      Deprecated.
      v54 Use #reduceMergeable(Collection,IntSupplier,Supplier>) instead
      <W,​R>
      R
      reduce​(java.util.Collection<W> work, java.util.function.Supplier<? extends TwoStepMapper.Mergeable<W,​R>> reducer)
      Deprecated.
      v54 Use #reduceMergeable(Collection,Supplier>) instead
      <W,​R,​A extends TwoStepMapper.Combineable<W,​R,​A>>
      R
      reduceCombineable​(java.util.Collection<W> work, int parallelism, java.util.function.Supplier<A> reducer)
      Will create at most parallelism tasks to work through the work items, processing them with reducer.
      <W,​R,​A extends TwoStepMapper.Combineable<W,​R,​A>>
      R
      reduceCombineable​(java.util.Collection<W> work, java.util.function.IntSupplier parallelism, java.util.function.Supplier<A> reducer)  
      <W,​R,​A extends TwoStepMapper.Combineable<W,​R,​A>>
      R
      reduceCombineable​(java.util.Collection<W> work, java.util.function.Supplier<A> reducer)
      Using parallelism Parallelism.CORES.
      <W,​R,​A extends TwoStepMapper.Mergeable<W,​R>>
      R
      reduceMergeable​(java.util.Collection<W> work, int parallelism, java.util.function.Supplier<A> reducer)
      Will create at most parallelism tasks to work through the work items, processing them with reducer.
      <W,​R,​A extends TwoStepMapper.Mergeable<W,​R>>
      R
      reduceMergeable​(java.util.Collection<W> work, java.util.function.IntSupplier parallelism, java.util.function.Supplier<A> reducer)  
      <W,​R,​A extends TwoStepMapper.Mergeable<W,​R>>
      R
      reduceMergeable​(java.util.Collection<W> work, java.util.function.Supplier<A> reducer)
      Using parallelism Parallelism.CORES.
      void run​(int parallelism, java.lang.Runnable processor)
      Will create precisely parallelism tasks that each execute the processor.
      void run​(java.util.function.IntSupplier parallelism, java.lang.Runnable processor)  
      <T> java.util.concurrent.atomic.AtomicBoolean take​(java.util.concurrent.BlockingQueue<T> queue, int parallelism, java.util.function.Consumer<T> processor)
      Will submit precisely parallelism tasks that each take from the queue feeding the items to the processor.
      <T> java.util.concurrent.atomic.AtomicBoolean take​(java.util.concurrent.BlockingQueue<T> queue, java.util.function.IntSupplier parallelism, java.util.function.Consumer<T> processor)  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • myExecutor

        private final java.util.concurrent.ExecutorService myExecutor
    • Constructor Detail

      • ProcessingService

        public ProcessingService​(java.util.concurrent.ExecutorService executor)
    • Method Detail

      • newInstance

        public static ProcessingService newInstance​(java.lang.String name)
      • compute

        public <W,​R> java.util.Map<W,​R> compute​(java.util.Collection<W> work,
                                                            int parallelism,
                                                            java.util.function.Function<W,​R> computer)
        Compute an output item for each (unique) input item, and return the results as a Map. If the input contains duplicates, the output will have fewer items. It is therefore vital that the input type implements Object.hashCode() and Object.equals(Object) properly.

        Will create at most parallelism tasks to work through the work items, processing them with computer and collectiing the results in a Map.

        Type Parameters:
        W - The work item type
        R - The function return type
        Parameters:
        work - The collection of work items
        parallelism - The maximum number of concurrent workers that will process the work items
        computer - The processing code
        Returns:
        A map of function input to output
      • compute

        public <W,​R> java.util.Map<W,​R> compute​(java.util.Collection<W> work,
                                                            java.util.function.IntSupplier parallelism,
                                                            java.util.function.Function<W,​R> computer)
        See Also:
        compute(Collection, int, Function)
      • getExecutor

        public java.util.concurrent.ExecutorService getExecutor()
        Returns:
        The underlying ExecutorService
      • map

        public <W,​R> java.util.Collection<R> map​(java.util.Collection<W> work,
                                                       int parallelism,
                                                       java.util.function.Function<W,​R> mapper)
        Simply map each (unique) input item to an output item - a Collection of input results in a Collection of output. If the input contains duplicates, the output will have fewer items. It is therefore vital that the input type implements Object.hashCode() and Object.equals(Object) properly.
        Type Parameters:
        W - The input item type
        R - The output item type
        Parameters:
        work - The collection of work items
        parallelism - The maximum number of concurrent workers that will process the work items
        mapper - The mapper functiom
        Returns:
        The mapped results
      • map

        public <W,​R> java.util.Collection<R> map​(java.util.Collection<W> work,
                                                       java.util.function.IntSupplier parallelism,
                                                       java.util.function.Function<W,​R> mapper)
        See Also:
        map(Collection, int, Function)
      • process

        public <W> void process​(java.util.Collection<? extends W> work,
                                int parallelism,
                                java.util.function.Consumer<W> processor)
        Will create at most parallelism tasks to work through the work items, processing them with processor.
        Type Parameters:
        W - The work item type
        Parameters:
        work - The collection of work items
        parallelism - The maximum number of concurrent workers that will process the work items
        processor - The processing code
      • process

        public <W> void process​(java.util.Collection<? extends W> work,
                                java.util.function.IntSupplier parallelism,
                                java.util.function.Consumer<W> processor)
        See Also:
        process(Collection, int, Consumer)
      • processPair

        public <W> void processPair​(W work1,
                                    W work2,
                                    java.util.function.Consumer<W> processor)
        Just 2 work items.
        See Also:
        process(Collection, Consumer)
      • processTriplet

        public <W> void processTriplet​(W work1,
                                       W work2,
                                       W work3,
                                       java.util.function.Consumer<W> processor)
        Just 3 work items.
        See Also:
        process(Collection, Consumer)
      • reduce

        @Deprecated
        public <W,​R> R reduce​(java.util.Collection<W> work,
                                    int parallelism,
                                    java.util.function.Supplier<? extends TwoStepMapper.Mergeable<W,​R>> reducer)
        Deprecated.
        v54 Use #reduceMergeable(Collection,int,Supplier>) instead
      • reduce

        @Deprecated
        public <W,​R> R reduce​(java.util.Collection<W> work,
                                    java.util.function.IntSupplier parallelism,
                                    java.util.function.Supplier<? extends TwoStepMapper.Mergeable<W,​R>> reducer)
        Deprecated.
        v54 Use #reduceMergeable(Collection,IntSupplier,Supplier>) instead
      • reduce

        @Deprecated
        public <W,​R> R reduce​(java.util.Collection<W> work,
                                    java.util.function.Supplier<? extends TwoStepMapper.Mergeable<W,​R>> reducer)
        Deprecated.
        v54 Use #reduceMergeable(Collection,Supplier>) instead
      • reduceCombineable

        public <W,​R,​A extends TwoStepMapper.Combineable<W,​R,​A>> R reduceCombineable​(java.util.Collection<W> work,
                                                                                                            int parallelism,
                                                                                                            java.util.function.Supplier<A> reducer)
        Will create at most parallelism tasks to work through the work items, processing them with reducer. The state of each task's reducer will be combined into a single instance, and the results of that instance will be returned.

        Each TwoStepMapper.Combineable is only worked on by a single thread, and the results are combined into a single instance. The instances are not reused.

        Parameters:
        work - The collection of work items
        parallelism - The maximum number of concurrent workers that will process the work items
        reducer - A TwoStepMapper.Combineable implementation that does what you want.
        Returns:
        The results...
      • reduceMergeable

        public <W,​R,​A extends TwoStepMapper.Mergeable<W,​R>> R reduceMergeable​(java.util.Collection<W> work,
                                                                                                int parallelism,
                                                                                                java.util.function.Supplier<A> reducer)
        Will create at most parallelism tasks to work through the work items, processing them with reducer. The results of each task's reducer will be merged into a single instance, and the results of that instance will be returned.

        Each TwoStepMapper.Mergeable is only worked on by a single thread, and the results are combined into a single instance. The instances are not reused.

        Parameters:
        work - The collection of work items
        parallelism - The maximum number of concurrent workers that will process the work items
        reducer - A TwoStepMapper.Mergeable implementation that does what you want.
        Returns:
        The results...
      • run

        public void run​(int parallelism,
                        java.lang.Runnable processor)
        Will create precisely parallelism tasks that each execute the processor.
        Parameters:
        parallelism - The number of concurrent workers/threads that will run
        processor - The processing code
      • run

        public void run​(java.util.function.IntSupplier parallelism,
                        java.lang.Runnable processor)
        See Also:
        run(int, Runnable)
      • take

        public <T> java.util.concurrent.atomic.AtomicBoolean take​(java.util.concurrent.BlockingQueue<T> queue,
                                                                  int parallelism,
                                                                  java.util.function.Consumer<T> processor)
        Will submit precisely parallelism tasks that each take from the queue feeding the items to the processor. The tasks will continue to run until the returned AtomicBoolean is set to false (or the thread is interrupted).

        If the threads of the underlying ExecutorService are daemon threads, the JVM will not wait for them to finish before it exits. The default behaviour, using INSTANCE or newInstance(String), is to make use of ojAlgo's DaemonPoolExecutor.

        Type Parameters:
        T - The work item type
        Parameters:
        queue - The queue to take from
        parallelism - How many parallel workers to create
        processor - What to do with each of the work items
        Returns:
        A flag that can be used to signal the tasks to stop
      • take

        public <T> java.util.concurrent.atomic.AtomicBoolean take​(java.util.concurrent.BlockingQueue<T> queue,
                                                                  java.util.function.IntSupplier parallelism,
                                                                  java.util.function.Consumer<T> processor)
        See Also:
        take(BlockingQueue, int, Consumer)