Hello World

public class HelloWorld {
  public static void main(String[] args) {
    [
      || System.out.println("Hello");
      || System.out.println("World");
    ]
  }
}

Each || symbol introduces a parallel branch. Running this program will print either

Hello
World

or

World
Hello

depending on how the parallel branches happen to be scheduled.

Data parallelism

[
  || (int i : array.length) array[i]++;
]

The quantification (int i : N) creates one parallel branch for each value of i. The effect of this code is to increment all elements of array in parallel. This code is equivalent to

[
  || array[0]++;
  || array[1]++;
  ...
  || array[array.length-1]++;
]

More complex quantifications are possible. The following example quantifies over the upper left triangle of a square matrix:

[
|| (int i:N, int j:N, if i+j<N) matrix[i][j]++;
]

Code that performs a similar and typically small operation on a large collection of elements is called data parallel, and appears often in high-performance scientific applications. A typical representative of data-parallel languages for the C/C++ or Fortran ecosystems is OpenMP.

Data parallelism features can also be implemented by libraries using dedicated data structures, such as parallel arrays.

Task parallelism

The term task parallelism is used when work can conceptually be decomposed into a number of logical tasks. In this example, tasks are created recursively:

int fib(int n) {
  if (n <= 1) return 1;
  int fib1, fib2;
  // recursively create parallel branches
  [
    || fib1 = fib(n-1);
    || fib2 = fib(n-2);
  ]
  return fib1 + fib2;
}

Task parallelism is implemented in languages such as Cilk, and in libraries similar to the fork/join pair of Unix system calls.

Message-passing

Parallel branches have two ways of communicating; either by concurrently reading and writing shared variables, or by sending explicit messages. The operators ! and ? respectively send and receive a message on a channel.

In this example, two parallel branches communicate via explicit message passing:

Chan<String> chan = new Chan<String>();
[
  // branch 1 sends a value over the channel
  || chan ! "Hello";

  // branch 2 receives a value from the channel and prints it
  || chan ? s; System.out.println(s);
]

Data-flow

A program is said to be data-flow when computation is initiated and synchronized by the availability of data in a flow. A typical example is an adder: it has two inputs, one output, and whenever the two inputs are ready, it sends their sum on the output.

void adder(Chan<Integer> in1, Chan<Integer> in2, Chan<Integer> out) {
  for (;;) {
    int value1, value2;
    [ in1 ? value1; || in2 ? value2; ];
    out ! value1 + value2;
}}

Note the parallel read [ in1 ? value1; || in2 ? value2; ]. It means that the two input values can come in any order. Without it, the code may deadlock if values were coming in the wrong order. This shows that parallel primitives in a programming language are not only about performance, but also about the behavior of programs.

The adder by itself doesn't do anything, since it reacts on input data. It needs to be put in a context where other parts feed input values and read output values. The way to express this is to compose all pieces in a large parallel block:

[
  || source(c1); // generates values on c1
  || source(c2); // generates values on c2
  || adder(c1, c2, c3);
  || sink(c3); // read values from c3
]

Anything that can be thought of as a combination of logical gates or electrical circuits can readily be expressed in this way as a data-flow program.