Tag Archives: Ranges

Generators with Java 8

Today we’ll look at creating generators. In simple terms, a generator is a function which returns the next value in a sequence. Unlike an iterator, it generates the next value when needed, rather than returning the next item of a pre-generated collection. Some languages such as Python support generators natively via keywords such as yield. When a generator’s next value is requested in Python, the generator function continues to run until the next yield statement, where a value is returned. The generator function is able to continue where it left off which can be quite confusing for the uninitiated. So how to do something similar in Java?

We saw in the last article that we can use an IntStream to generate a simple set of numbers, but we had to generate them all up front. That’s fine if we know how many we’re going to need. What if we don’t, and we want to be able to get the next whenever we like? This is where a generator comes in.

Let’s choose a simple infinite sequence, the square numbers. In a standard Java implementation we’d end up with something like the following:

public class Squares
{
        private int i = 1;

        public int next()
        {
                int thisOne = i++;
                return thisOne * thisOne;
        }

        public static void main(String args[])
        {
                Squares squareGenerator = new Squares();

                System.out.println(squareGenerator.next());
                System.out.println(squareGenerator.next());
                System.out.println(squareGenerator.next());
        }
}

This prints the first three square numbers. Note we could have gone further and implemented this as an iterator.

What we have here is an example of lazy evaluation in a non-functional style. Wikipedia defines lazy evaluation as: ‘In programming language theory, lazy evaluation, or call-by-need is an evaluation strategy which delays the evaluation of an expression until its value is needed’. Lazy evaluation is useful because we don’t need to worry about infinite sequences, performing computationally expensive operations up-front, and about storage.

Let’s expand on the example to allow getting a batch of results. This is easy – create a nextN function which calls next() a number of times and returns the results in say a List:

public class Squares2
{
        private int i = 1;

        public int next()
        {
                int thisOne = i++;
                return thisOne * thisOne;
        }

        public List<Integer> nextN(int n)
        {
                List<Integer> l = new ArrayList<>();

                for (int i = 0; i < n; i++)
                {
                        l.add(next());
                }

                return l;
        }

        public static void main(String args[])
        {
                Squares2 squareGenerator = new Squares2();

                squareGenerator.nextN(10).forEach(System.out::println);
        }
}

A few points:

  • Notice in the nextN function there is the empty diamond in the new ArrayList statement. This was added in Java 7 to save having to state the type both on the left and the right hand side; the compiler now works it out.
  • List is an Iterable, and Iterable now has a forEach() method which was added in Java 8. We could use stream() as before to create a stream, but if all we want to do is pass the contents to a function forEach() does nicely.

Now, to save having to write nextN for every sequence we make, we could create a new type which extends Iterator providing the nextN function.

The only problem we face here is that we have to save the batch in a list before we can operate on it. Java 8 provides another way. Let’s go back and start again with the following code:

public class Squares3
{
        public static void main(String args[])
        {
                IntStream.rangeClosed(1, 10).map(i -> i * i)
                         .forEach(System.out::println);
        }
}

This uses IntStream to get the indexes of the sequence in a stream and calls map to convert them into their squares. The problem is that to get more squares than the tenth we need to duplicate the pipeline and start it off from the right place. Let’s look at another way without using a range:


        public static void main(String args[])
        {
                IntStream myStream = IntStream.iterate(1, i -> i + 1);

                myStream.limit(10).map(i -> i * i)
                                  .forEach(System.out::println);
        }

This also generates the first 10 square numbers. This time it uses the iterate function. This takes two parameters, the first is our initial value, and the second is a function defining how to get to the next value from the previous. It’s a good place to use a lambda function. We can even dispense of the map function since we can undo squaring easily in iterate to get what the last index was:

        public static void main(String args[])
        {
                IntStream myStream = IntStream.iterate(1,
                        i -> ((int) Math.pow(Math.sqrt(i) + 1, 2)));

                myStream.limit(10).forEach(System.out::println);
        }

This solves one of the problems of having to buffer beforehand. However, we need to use the limit operator on the stream to limit it to 10 items, otherwise it would keep on going. Unfortunately this is a problem, since once we’ve got the 10 the stream is ‘operated on’ and we can’t use it again to generate more. If we try, we get an IllegalStateException. We’d have to create another stream to get more.

So how do we get around the problem of the stream being used up? Instead of using IntStream’s iterate function, we can use generate instead. IntStream’s generate function takes an instance of an IntSupplier. IntSupplier has a getAsInt() function which returns the next int in the sequence which is very much like our next() function. Here is an example that prints the first 20 square numbers in two batches:

public class SquaresGenerator
{
        private static class SqSupplier implements IntSupplier
        {
                int i = 0;

                @Override
                public int getAsInt()
                {
                        i++;
                        return i * i;
                }
        }

        public static void main(String args[])
        {
                SqSupplier sqSupplier = new SqSupplier();
                IntStream myStream = IntStream.generate(sqSupplier);
                IntStream myStream2 = IntStream.generate(sqSupplier);

                myStream.limit(10).forEach(System.out::println);
                myStream2.limit(10).forEach(System.out::println);
        }
}

Again we’re using limit to stop the stream continuing indefinitely. However unlike last time, although the stream is used up, the generator still survives and can be used again. No buffering needed either, just keeping hold of the supplier. The only downside vs the old Java way is that we have to use Streams to get sequence members, although this comes with other benefits such as parallelism which we’ll see in a later article.

Overall, there are several ways to generate a sequence and which we chose may depend on our needs. Using an IntSupplier is a good way to integrate with the rest of the Java 8 functional programming support.

Ranges and Looping with IntStream

In the previous posts we looked at taking a container, getting it to stream its contents into a pipeline and looked at a few different operations on data in that pipeline (forEach, map, filter, peek).

Java 8 supports several specialist streams where the pipeline contains a specific type of object. Today we’ll look at IntStream which passes Integers along its pipeline.

public class IntStreamExample
{
	public static void main(String[] args)
	{
		System.out.println("[1,5]");
		IntStream.rangeClosed(1, 5).forEach(System.out::println);

		System.out.println("[1,5)");
		IntStream.range(1, 5).forEach(System.out::println);

		System.out.println("Just 3");
		IntStream.of(3).forEach(System.out::println);

		System.out.println("Specific values");
		IntStream.of(1, 3, 5, 6).forEach
                (System.out::println);

		System.out.println("[1,3] and [4,6] joined");
		IntStream.concat(IntStream.rangeClosed(1, 3),
		IntStream.rangeClosed(4, 6)).forEach(System.out::println);
	}
}

To build our stream we take the IntStream class and use one of its static methods. A selection of such methods are demonstrated in the example above.

For those who missed the previous articles, the :: operator means pass the function on the right, calling with the object on the left.

Let’s describe each of the methods:

  • The range and rangeClosed methods produce a stream which has an ordered pipeline of integers starting at the first number and ending at the second. The difference is that rangeClosed has an inclusive endpoint where are range does not. There is no version yet with a step or descending values – the pipeline is initialised as empty if the start is beyond the last element. If we wanted a step, we use transformations with map:
  • IntStream.rangeClosed(1, 5).map(x -> 6-x)
                               .forEach(System.out::println);
    

    This is a bit clumsy (and hopefully a step version will be added soon) but it does the trick. If we need a step often, we could make our own based on the IntStream class.

  • The of method puts a one or more values in the pipeline. The multiple value version takes a variable number of ints (which means we can also pass an array of int).
  • Finally the concat method can be used to put two or more IntStreams together into a single IntStream.

Note that there also is an empty() method which produces an empty stream.

A use of range is a functional-style for-loop. It’s functional-style because there is no mutable loop variable. The examples above have already demonstrate this – the ‘body’ of the loop just printed the loop counter.

What if we want to nest two loops? That’s easy:

public class Multiplication
{
	public static void main(String[] args)
	{
		IntStream.rangeClosed(1, 10)
		         .forEach(i -> IntStream.rangeClosed(1, 10)
                         .forEach(
            j -> System.out.println(i + " * " + j + " = " + i * j)));
	}
}

By using the forEach operation we can map each element onto a another stream. We don’t have to use the element there and then, we can use it later in the pipeline. In the example we use both streams to produce a multiplication table.

We could also save the output of an IntStream to an array like this:

	int[] a = IntStream.rangeClosed(1, 10).toArray();

This provides us with a very handy way to initialise an array to a sequence of integers.

The pipeline operation toArray returns an int[]. What if we want to create an array from two or more nested loops? The problem we run into in the nested version is that we can’t use toArray() in the inner loop since the inner loop is part of a map function which is expecting an int not an int[]. This means we have to use another trick:

	int[] a = IntStream.rangeClosed(1, 10)
			.flatMap(i -> IntStream.rangeClosed(1, 10)
                                          .map(j -> i * j))
            .toArray();

Here we use flatMap. flatMap flattens a number of IntStreams (10 of them here) into pipeline elements (ints). These are then passed on to the next operation which is toArray().

The lambda expression passed to flatMap is converted to an IntFunction and its apply function takes an int and returns an IntStream. Here is an inner-class with it implemented explicitly:

	private static class MultiplicationTable implements
			IntFunction<IntStream>
	{
		@Override
		public IntStream apply(int value)
		{
			return IntStream.rangeClosed(1, 10).map(j -> value * j);
		}
	}

using the call:

		int[] a = IntStream.rangeClosed(1, 10)
			               .flatMap(new MultiplicationTable())
			               .toArray();

Finally here is a version using a local function:

public class Multiplication
{
	private IntStream getTable(int i)
	{
		return IntStream.rangeClosed(1, 10).map(j -> i * j);
	}

	public void test()
	{
		int[] a = IntStream.rangeClosed(1, 10).flatMap
                                               (this::getTable)
					       .toArray();

		Arrays.stream(a).forEach(System.out::println);
	}

	public static void main(String[] args)
	{
		new Multiplication().test();
	}
}

Note we also used the stream function on the Arrays helper class to print out the array. To do this we need to pass in the array into the stream function and then we can use forEach to print.

That should give you a good start for using IntStreams. Note there are also special streams for Long and Double which you might want to take a look at.