Interface RequestSource

Functional Interface:
This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
@FunctionalInterface public interface RequestSource
A lazy source of requests for a RequestExecutionEngine, passed to RequestExecutionEngine.sendAll(RequestSource). The engine pulls the next request on demand - only when it has capacity to send it - so requests are generated lazily and only a bounded number are ever held in memory at once, regardless of the total count. This is what lets a run cover an arbitrarily large (even unbounded) number of requests without materialising them all up front.

next() is called serially by the engine (never concurrently), so a stateful source needs no synchronisation. Pair this with a ResponseHandler that returns Retention.DROP for the uninteresting responses to keep the result set bounded too. 

 Iterator<String> words = hugeWordlist.iterator();
 engine.sendAll(
     () -> words.hasNext() ? SourcedRequest.sourcedRequest(base.withParameter(q(words.next()))) : null,
     (result, execution) -> interesting(result) ? Retention.KEEP : Retention.DROP);

  • Method Details

    • next

      SourcedRequest next()
      Returns:
      The next request to send (with its optional label), or null when the source is exhausted. Called serially, on demand. Throwing aborts the run with that error.

    • from

      static RequestSource from(Iterator<HttpRequest> iterator)
      Adapt an Iterator of requests into a source (no correlation labels).
      Parameters:
      iterator - The requests to send, pulled lazily.
      Returns:
      A request source over the iterator.

    • from

      static RequestSource from(Stream<HttpRequest> stream)
      Adapt a Stream of requests into a source (no correlation labels). The stream is consumed lazily, so an infinite or generated stream is fine.
      Parameters:
      stream - The requests to send, pulled lazily.
      Returns:
      A request source over the stream.