Ranges and Looping with IntStream

In the previous posts we looked at taking a container, getting it to stream its contents into a pipeline and looked at a few different operations on data in that pipeline (forEach, map, filter, peek).

Java 8 supports several specialist streams where the pipeline contains a specific type of object. Today we’ll look at IntStream which passes Integers along its pipeline.

public class IntStreamExample
{
	public static void main(String[] args)
	{
		System.out.println("[1,5]");
		IntStream.rangeClosed(1, 5).forEach(System.out::println);

		System.out.println("[1,5)");
		IntStream.range(1, 5).forEach(System.out::println);

		System.out.println("Just 3");
		IntStream.of(3).forEach(System.out::println);

		System.out.println("Specific values");
		IntStream.of(1, 3, 5, 6).forEach
                (System.out::println);

		System.out.println("[1,3] and [4,6] joined");
		IntStream.concat(IntStream.rangeClosed(1, 3),
		IntStream.rangeClosed(4, 6)).forEach(System.out::println);
	}
}

To build our stream we take the IntStream class and use one of its static methods. A selection of such methods are demonstrated in the example above.

For those who missed the previous articles, the :: operator means pass the function on the right, calling with the object on the left.

Let’s describe each of the methods:

  • The range and rangeClosed methods produce a stream which has an ordered pipeline of integers starting at the first number and ending at the second. The difference is that rangeClosed has an inclusive endpoint where are range does not. There is no version yet with a step or descending values – the pipeline is initialised as empty if the start is beyond the last element. If we wanted a step, we use transformations with map:
  • IntStream.rangeClosed(1, 5).map(x -> 6-x)
                               .forEach(System.out::println);
    

    This is a bit clumsy (and hopefully a step version will be added soon) but it does the trick. If we need a step often, we could make our own based on the IntStream class.

  • The of method puts a one or more values in the pipeline. The multiple value version takes a variable number of ints (which means we can also pass an array of int).
  • Finally the concat method can be used to put two or more IntStreams together into a single IntStream.

Note that there also is an empty() method which produces an empty stream.

A use of range is a functional-style for-loop. It’s functional-style because there is no mutable loop variable. The examples above have already demonstrate this – the ‘body’ of the loop just printed the loop counter.

What if we want to nest two loops? That’s easy:

public class Multiplication
{
	public static void main(String[] args)
	{
		IntStream.rangeClosed(1, 10)
		         .forEach(i -> IntStream.rangeClosed(1, 10)
                         .forEach(
            j -> System.out.println(i + " * " + j + " = " + i * j)));
	}
}

By using the forEach operation we can map each element onto a another stream. We don’t have to use the element there and then, we can use it later in the pipeline. In the example we use both streams to produce a multiplication table.

We could also save the output of an IntStream to an array like this:

	int[] a = IntStream.rangeClosed(1, 10).toArray();

This provides us with a very handy way to initialise an array to a sequence of integers.

The pipeline operation toArray returns an int[]. What if we want to create an array from two or more nested loops? The problem we run into in the nested version is that we can’t use toArray() in the inner loop since the inner loop is part of a map function which is expecting an int not an int[]. This means we have to use another trick:

	int[] a = IntStream.rangeClosed(1, 10)
			.flatMap(i -> IntStream.rangeClosed(1, 10)
                                          .map(j -> i * j))
            .toArray();

Here we use flatMap. flatMap flattens a number of IntStreams (10 of them here) into pipeline elements (ints). These are then passed on to the next operation which is toArray().

The lambda expression passed to flatMap is converted to an IntFunction and its apply function takes an int and returns an IntStream. Here is an inner-class with it implemented explicitly:

	private static class MultiplicationTable implements
			IntFunction<IntStream>
	{
		@Override
		public IntStream apply(int value)
		{
			return IntStream.rangeClosed(1, 10).map(j -> value * j);
		}
	}

using the call:

		int[] a = IntStream.rangeClosed(1, 10)
			               .flatMap(new MultiplicationTable())
			               .toArray();

Finally here is a version using a local function:

public class Multiplication
{
	private IntStream getTable(int i)
	{
		return IntStream.rangeClosed(1, 10).map(j -> i * j);
	}

	public void test()
	{
		int[] a = IntStream.rangeClosed(1, 10).flatMap
                                               (this::getTable)
					       .toArray();

		Arrays.stream(a).forEach(System.out::println);
	}

	public static void main(String[] args)
	{
		new Multiplication().test();
	}
}

Note we also used the stream function on the Arrays helper class to print out the array. To do this we need to pass in the array into the stream function and then we can use forEach to print.

That should give you a good start for using IntStreams. Note there are also special streams for Long and Double which you might want to take a look at.

Introduction to Functional Programming in Java 8 – Part Two

Welcome to Part Two! To begin this session, let’s start with a return to our HelloWorld example from the last article.

public class HelloWorldConcise
{
        private void doPrint(String str)
        {
                System.out.println(str);
        }

        private String greet(String country)
        {
                return "Hello " + country + "!";
        }

        public void greetCountries()
        {
                List<String> countries = Arrays.asList("France", "India", 
                "China", "USA", "Germany");

                countries.stream().map(this::greet)
                                  .forEach(this::doPrint);
        }

        public static void main(String[] args)
        {
                new HelloWorldConcise().greetCountries();
        }
}

Let’s make one more change to the pipeline – how about we get rid of countries that contain a G:

                countries.stream().filter(country -> country.indexOf('G') == -1)
                                  .map(this::greet).forEach(this::doPrint);

Now it only greets 4 countries. The filter operation keeps items where the filter’s expression evaluates to true, and discards the others. In this case we have another lambda expression which taking the item to be called country checks that it doesn’t contain a G.

This is very useful as we don’t have to alter the original list, or stop work half way to manually remove some items. Note that it’s also very easy to add another element to our pipeline, and still quite readable what’s going on.

As we’ve done before, let’s look at the under the hood at filter. In this case the lambda expression is actually a Predicate. Predicate has a function ‘test’ which we override to perform our check. Here is an inner-class with the filter’s test explicitly written out:

private static class DoesntContainG implements Predicate<String>
{
	@Override
	public boolean test(String str)
	{
		return str.indexOf('G') == -1;
	}
}

We can change the pipeline as follows to use this class:

		countries.stream().filter(new DoesntContainG())
				  .map(this::greet).forEach(this::doPrint);

Instead of the inner class we could add a new function and pass that as we’ve done before:

        private boolean doesntContainG(String str)
        {
	        return str.indexOf('G') == -1;
        }

and then change the pipeline to the very concise:

		countries.stream().filter(this::doesntContainG)
				  .map(this::greet).forEach(this::doPrint);

How readable is that? Take countries, stream them, filter leaving those that do not contain G, map them to a greeting and print them.

Now suppose we need to use a value part of the way along the pipeline. One obvious application is debugging. If we wanted to print the values so far, we couldn’t use a forEach. forEach is a terminal operation, it consumes all the items and so no operation can follow it. Instead the operation peek is what we need.

Let’s see Germany being removed. We’ll add the following function:

private void check(String country)
{
	System.out.println("Found " + country);
}

and change the pipeline as follows:

countries.stream().peek(this::check).filter(this::doesntContainG)
		  .peek(this::check).map(this::greet)
  		  .forEach(this::doPrint);

This will print the countries before and after the filter. Note that Germany is only found once (before the filter), where as all the others are found after as well, so we can conclude that the filter is indeed removing it.

For a preview of what will be covered in a future article, try changing stream() to parallelStream() in the example and see what happens.

So that’s the basic operations we’ve covered so far in the blog. We can take a container make a stream, transform items, filter items, peek at them and do something with them.

An Introduction to Functional Programming with Java 8

Java 8 is perhaps one of the most exciting editions of the Java language in recent times. One of the headline features is support for functional programming which is the focus of this blog. The support comes mostly in three features:

  • Support for [work pipeline] streams. Streams allow us to process data through a number of stages in a pipeline in a functional way. We can create such streams from container instances such as List, or create them using specific stream classes such as IntStream.
  • Support for lambda functions. In simple terms this is an anonymous function (we don’t need to give it a name) which takes a number of parameters and returns a value. A returned value will be passed down the pipeline to the next operation.
  • Support for passing functions into other functions.

Given functional programming has been around since the 50s, and until recently mostly disregarded by the mainstream, why has it become such a hot topic? My opinion is that it’s because of its ability to easily process work in parallel taking advantage of multi-core processors, lazy (on-demand) evaluation, and ease of integration with other languages such as Java. Certainly the JVM has provided a good base for Scala which can even be embedded in a Java program, giving the best of both worlds plus an easier migration path for developers. Scala, however, has quite a steep learning curve, and there are already many existing Java programmers out there, so eventually there was pressure for Java itself to catch up to avoid having to learn a whole new language to do some cool and useful things.

Enough words, let’s get down to business and consider a hello world example:

public class HelloWorld
{
	public static void main(String[] args)
	{
		List<String> countries = Arrays.asList("France", "India", 
			"China", "USA", "Germany");

		for (String country : countries)
		{
			System.out.println("Hello " + country + "!");
		}
	}
}

This should be familiar to every Java programmer; we take a list of countries and print a greeting using an enhanced for-loop.

So, is there anything wrong just doing it like this? For a simple example, not a lot, but let’s imagine we were doing something more complicated.

We want to write some tests to check the code works, but testing the body of the loop might be tricky. We could extract the body into its own function of course, but if we did that everywhere we’d end up with lots of little functions for the bodies of loops.

What if we want to process items in parallel? We’d need to hand over a country to a thread that implements the greeting function. Should one thread process one country, or a batch of them? How are we going to pass the work in batches and check when it’s done? We have to also handle communication between the threads.

Another problem is that the loop body’s code is disjoint from the source of the loop variable. To see how a country gets created we must find the for statement. We’d prefer to tell the list of countries to do the message printing. The problem is that lists don’t understand being told to do things, they are just containers.

However, since Java 8, containers can pass their contents to an entity which does handle instructions – a Stream. I’ve changed the example to use a stream:

	public static void main(String[] args)
	{
		List<String> countries = Arrays.asList("France", "India", "China",
				"USA", "Germany");

		countries.stream().forEach(
				(String country) -> System.out
						.println("Hello " + country + "!"));
	}

The list of countries now creates a Stream and passes it a ‘spliterator’ from the container. This is an iterator which is capable of splitting the contents into batches of work. We’ll take a look at this batching (which then can be parallelised) in a future post.

Stream itself is an interface which is implemented by the abstract class ReferencePipeline. Jobs can then be chained to the returned Stream. Each job does some work, and perhaps calls a downstream job when it’s done. In the example the list creates a Stream and a forEach job is added to its pipeline. Like our loop in the first example, the forEach job does some work on each item. The funny -> indicates a lambda expression (an anonymous function), here taking a String named country and printing it in a message.

Since in this case the compiler can work out that country is a String, we can dispense with the type, and as there is just one parameter we can also dispense with the () around the lefthand side leaving the more concise:

countries.stream()
   .forEach(country -> System.out.println("Hello " + country + "!"));

Note it’s a lot clearer that we are doing work on countries rather than just referencing a loop variable which happened to be a country. It’s also easier to read as there is less boilerplate obscuring the actual business logic. Clearer and easier to read code is easier to understand, also it’s easier to spot mistakes. We don’t also have to worry about making mistakes in the boilerplate which cuts down our tests. Overall, coding is more fun and the development time is shortened.

Let’s experiment with this example a bit to illustrate what is going on under the hood. The lambda that foreach is taking is actually an implementation of Consumer<T>.

* Reminder for those a bit rusty or coming to generics for the first time that this means the Consumer type uses an unknown type T which is decided at compile time.

We can explicitly show use of Consumer by creating an inner-class:

private static class Greeter<T> implements Consumer<T>
{
	@Override
	public void accept(T t)
	{
		System.out.println("Hello " + t + "!");
	}
}

and changing the pipeline:

countries.stream().forEach(new Greeter<String>());

Consumer has an accept method which is called every time by forEach with each item in the pipeline.

Let’s separate the forming of the message and the printing into two jobs. To form the greeting message, we can use the map function. This applies a lambda function to an input transforming the input. The printing job is still carried out by a foreach:

public static void main(String[] args)
{
	List<String> countries = Arrays.asList("France", "India",
			"China", "USA", "Germany");

	countries.stream().map(country -> "Hello " + country + "!")
			.forEach(System.out::println);
}

Note the strange syntax System.out::println. We’re actually passing a function into forEach, the println method of the System.out instance. The :: syntax simply means pass the function on the right, calling with the object on the left. The left hand side can either be a class name for a static call, an object, or an alias (this or super) to an object. Note we cannot pass only the name of the function, since this would be taken to mean a variable whose type is-a Consumer.

Instead of a static main method, if we ran this inside an instance of an object which also had a doPrint method (taking a String and returning void), we could write:

       countries.stream().map(country -> "Hello " + country + "!")
                        .forEach(this::doPrint);

Let’s look at how map works under the hood. Map takes an instance of a class which is-a Function and calls its apply method. Apply takes type T and returns a type U. We’ll create a Greeter inner-class to demonstrate this:

private static class Greeter<T> implements Function<T, String>
{
	@Override
	public String apply(T t)
	{
		return "Hello " + t + "!";
	}
}

...

countries.stream().map(new Greeter<String>())
                  .forEach(System.out::println);

When we used a lambda function here, it was just taken to be an anonymous instance of Function.

To finish, let’s run the pipeline inside an object has both a doPrint and a makeGreeting method (taking a String and returning a String), and see what it looks like:

public class HelloWorldConcise
{
	private void doPrint(String str)
	{
		System.out.println(str);
	}

	private String greet(String country)
	{
		return "Hello " + country + "!";
	}
		
	public void greetCountries()
	{
		List<String> countries = Arrays.asList("France", "India", 
			"China", "USA", "Germany");

		countries.stream().map(this::greet).forEach(this::doPrint);
	}

	public static void main(String[] args)
	{
		new HelloWorldConcise().greetCountries();
	}
}

The greetCountries function is very concise. It’s clear that we take each country, make a greeting from it and then print it. Another great thing is it’s really easy to test. If we extend the HelloWorldConcise class to make a HelloWorldConciseTest, we can implement our own versions of makeGreeting and doPrint. Our own makeGreeting can be used to check the greeting is being formed correctly after calling its parent version, and the doPrint method can be a stub.

* Okay, printing to stdout during a test is not a big deal, but imagine if doPrint sent the greeting to a web page, we wouldn’t want to have to set all that up in order to test it.

So that’s our first taste of Java 8 functional programming. I hope you found it useful. In the next article I will look at some other operations which can be included in the pipeline.