Java Concurrency and Thread Pool

A thread is an execution context that can run a set of instructions within a process – aka a running program. Multithreaded programming refers to using threads to execute multiple tasks concurrently in a program. On this chapter lets find out some advanced concept in java thread

Java Concurrency

All modern operating systems support concurrency both via processes and threads. Processes are instances of programs which typically run independent to each other, e.g. if you start a java program the operating system spawns a new process which runs in parallel to other programs. Inside those processes we can utilize threads to execute code concurrently, so we can make the most out of the available cores of the CPU.

Working with the Thread class can be very tedious and error-prone. Due to that reason the Concurrency API has been introduced back in 2004 with the release of Java 5. The API is located in package java.util.concurrent and contains many useful classes for handling concurrent programming. Since that time the Concurrency API has been enhanced with every new Java release and even Java 8 provides new classes and methods for dealing with concurrency.

Thread Pool

The Thread brings several advantages, primarily regarding the performance of a program, the multithreaded programming can also have disadvantages – such as increased complexity of the code, concurrency issues, unexpected results and adding the overhead of thread creation.

Thread pools can resolve above mentioned issues upto some extent.

Creating and starting a thread can be an expensive process. By repeating this process every time we need to execute a task, we’re incurring a significant performance cost – which is exactly what we were attempting to improve by using threads.

For a better understanding of the cost of creating and starting a thread, let’s see what the JVM actually does behind the scenes:

• it allocates memory for a thread stack that holds a frame for every thread method invocation
• each frame consists of a local variable array, return value, operand stack and constant pool
• some JVMs that support native methods also allocate a native stack
• each thread gets a program counter that tells it what the current instruction executed by the processor is
• the system creates a native thread corresponding to the Java thread
• descriptors relating to the thread are added to the JVM internal data structures
• the threads share the heap and method area

A thread pool helps mitigate the issue of performance by reducing the number of threads needed and managing their lifecycle.

Java Thread Pools

Java provides its own implementations of the thread pool pattern, through objects called executors. These can be used through executor interfaces or directly through thread pool implementations – which does allow for finer-grained control.

The java.util.concurrent package contains the following interfaces:

• Executor – a simple interface for executing tasks
• ExecutorService – a more complex interface which contains additional methods for managing the tasks and the executor itself
• ScheduledExecutorService – extends ExecutorService with methods for scheduling the execution of a task

Alongside these interfaces, the package also provides the Executors helper class for obtaining executor instances, as well as implementations for these interfaces.

Generally, a Java thread pool is composed of:

• the pool of worker threads, responsible for managing the threads
• a thread factory that is responsible for creating new threads
• a queue of tasks waiting to be executed

The Executors class and Executor interface

The Executors class contains factory methods for creating different types of thread pools, while Executor is the simplest thread pool interface, with a single execute() method.

Let’s use these two classes in conjunction with an example that creates a single-thread pool, then uses it to execute a simple statement:

Executor executor = Executors.newSingleThreadExecutor();

executor.execute(() -> System.out.println("Single thread pool test"));

The factory methods in the Executors class can create several types of thread pools:

• newSingleThreadExecutor() – a thread pool with only one thread with an unbounded queue, which only executes one task at a time
• newFixedThreadPool() – a thread pool with a fixed number of threads which share an unbounded queue; if all threads are active when a new task is submitted, they will wait in queue until a thread becomes available
• newCachedThreadPool() – a thread pool that creates new threads as they are needed
• newWorkStealingThreadPool() – a thread pool based on a “work-stealing” algorithm which will be detailed more in a later section

ExecutorService

The Java ExecutorService interface, java.util.concurrent.ExecutorService, represents an asynchronous execution mechanism which is capable of executing tasks concurrently in the background

Before we get too deep into the ExecutorService, let us look at a simple example. Here is a simple Java ExecutorService example:

ExecutorService executorService = Executors.newFixedThreadPool(10);

executorService.execute(new Runnable() {
    public void run() {
        System.out.println("Asynchronous task");
    }

});
executorService.shutdown();

First an ExecutorService is created using the Executors newFixedThreadPool() factory method. This creates a thread pool with 10 threads executing tasks.

Second, an anonymous implementation of the Runnable interface is passed to the execute() method. This causes the Runnable to be executed by one of the threads in the ExecutorService.

Creating an ExecutorService

How you create an ExecutorService depends on the implementation you use. However, you can use the Executors factory class to create ExecutorService instances too. Here are a few examples of creating an ExecutorService:

ExecutorService executorService1 = Executors.newSingleThreadExecutor();

ExecutorService executorService2 = Executors.newFixedThreadPool(10);

ExecutorService executorService3 = Executors.newScheduledThreadPool(10);

ExecutorService Usage

There are a few different ways to delegate tasks for execution to an ExecutorService:

• execute(Runnable)
• submit(Runnable)
• submit(Callable)
• invokeAny(...)
• invokeAll(...)

Lets look at each one of those

execute(Runnable)

The Java ExecutorService execute(Runnable) method takes a java.lang.Runnable object, and executes it asynchronously. Here is an example of executing a Runnable with an ExecutorService:

ExecutorService executorService = Executors.newSingleThreadExecutor();

executorService.execute(new Runnable() {
    public void run() {
        System.out.println("Asynchronous task");
    }
});

executorService.shutdown();

There is no way of obtaining the result of the executed Runnable, if necessary. You will have to use a Callable for that (explained in the following sections).

submit(Runnable)

The Java ExecutorService submit(Runnable) method also takes a Runnable implementation, but returns a Future object. This Future object can be used to check if the Runnable has finished executing.

Here is a Java ExecutorService submit() example:

Future future = executorService.submit(new Runnable() {
    public void run() {
        System.out.println("Asynchronous task");
    }
});
future.get();  //returns null if the task has finished correctly.

The submit() method returns a Java Future object which can be used to check when the Runnable has completed.

submit(Callable)

The Java ExecutorService submit(Callable) method is similar to the submit(Runnable) method except it takes a Java Callable instead of a Runnable. The precise difference between a Callable and a Runnable is explained a bit later.

The Callable's result can be obtained via the Java Future object returned by the submit(Callable) method. Here is an ExecutorService Callable example:

invokeAny()

The invokeAny() method takes a collection of Callable objects, or subinterfaces of Callable. Invoking this method does not return a Future, but returns the result of one of the Callable objects. You have no guarantee about which of the Callable's results you get. Just one of the ones that finish.

If one of the tasks complete (or throws an exception), the rest of the Callable's are cancelled.

Here is a code example:

This code example will print out the object returned by one of the Callable's in the given collection. I have tried running it a few times, and the result changes. Sometimes it is "Task 1", sometimes "Task 2" etc.

invokeAll()

The invokeAll() method invokes all of the Callable objects you pass to it in the collection passed as parameter. The invokeAll() returns a list of Future objects via which you can obtain the results of the executions of each Callable.

Keep in mind that a task might finish due to an exception, so it may not have "succeeded". There is no way on a Future to tell the difference.

Here is a code example:

Runnable vs. Callable

The Runnable interface is very similar to the Callable interface. Both interfaces represents a task that can be executed concurrently by a thread or an ExecutorService. Both interfaces only has a single method. There is one small difference between the Callable and Runnable interface though. The difference between the Runnable and Callable interface is more easily visible when you see the interface declarations.

Here is first the Runnable interface declaration:

public interface Runnable {

    public void run();

}

And here is the Callable interface declaration:

public interface Callable{

    public Object call() throws Exception;

}

The main difference between the Runnable run() method and the Callable call() method is that the call() method can return an Object from the method call. Another difference between call() and run() is that call() can throw an exception, whereas run() cannot (except for unchecked exceptions - subclasses of RuntimeException).

If you need to submit a task to a Java ExecutorService and you need a result from the task, then you need to make your task implement the Callable interface. Otherwise your task can just implement the Runnable interface.

Cancel Task

You can cancel a task (Runnable or Callable) submitted to a Java ExecutorService by calling the cancel() method on the Future returned when the task is submitted. Cancelling the task is only possible if the task has not yet started executing. Here is an example of cancelling a task by calling the Future.cancel() method:

future.cancel();

ExecutorService Shutdown

When you are done using the Java ExecutorService you should shut it down, so the threads do not keep running. If your application is started via a main() method and your main thread exits your application, the application will keep running if you have an active ExexutorService in your application. The active threads inside this ExecutorService prevents the JVM from shutting down.

shutdown()

To terminate the threads inside the ExecutorService you call its shutdown() method. The ExecutorService will not shut down immediately, but it will no longer accept new tasks, and once all threads have finished current tasks, the ExecutorService shuts down. All tasks submitted to the ExecutorService before shutdown() is called, are executed. Here is an example of performing a Java ExecutorService shutdown:

executorService.shutdown();

shutdownNow()

If you want to shut down the ExecutorService immediately, you can call the shutdownNow() method. This will attempt to stop all executing tasks right away, and skips all submitted but non-processed tasks. There are no guarantees given about the executing tasks. Perhaps they stop, perhaps the execute until the end. It is a best effort attempt. Here is an example of calling ExecutorService shutdownNow:

executorService.shutdownNow();

awaitTermination()

The ExecutorService awaitTermination() method will block the thread calling it until either the ExecutorService has shutdown completely, or until a given time out occurs. The awaitTermination() method is typically called after calling shutdown() or shutdownNow(). Here is an example of calling ExecutorService awaitTermination():

executorService.shutdown();

executorService.awaitTermination();

ScheduledExecutorService

The java.util.concurrent.ScheduledExecutorService is an ExecutorService which can schedule tasks to run after a delay, or to execute repeatedly with a fixed interval of time in between each execution.

Once you have created a ScheduledExecutorService you use it by calling one of its methods:

• schedule (Callable task, long delay, TimeUnit timeunit)
• schedule (Runnable task, long delay, TimeUnit timeunit)
• scheduleAtFixedRate (Runnable, long initialDelay, long period, TimeUnit timeunit)
• scheduleWithFixedDelay (Runnable, long initialDelay, long period, TimeUnit timeunit)

Here is an example: