Class Enumerator supports:
An Enumerator may be created by the following methods:
In addition, certain Ruby methods return Enumerator objects: a Ruby iterator method that accepts a block may return an Enumerator if no block is given. There are many such methods, for example, in classes Array and Hash. (In the documentation for those classes, search for ‘new_enumerator`.)
Internal Iteration
In _internal iteration_, an iterator method drives the iteration and the caller’s block handles the processing; this example uses method each_with_index:
words = %w[foo bar baz] # => ["foo", "bar", "baz"]
enumerator = words.each # => #<Enumerator: ...>
enumerator.each_with_index {|word, i| puts "#{i}: #{word}" }
0: foo
1: bar
2: baz
Iterator methods in class Enumerator include:
-
each: passes each item to the block. -
each_with_index: passes each item and its index to the block. -
each_with_object(aliased aswith_object): passes each item and a given object to the block. -
with_index: likeeach_with_index, but starting at a given offset (instead of zero).
Class Enumerator includes module Enumerable, which provides many more iterator methods.
External Iteration
In _external iteration_, the user’s program both drives the iteration and handles the processing in stream-like fashion; this example uses method next:
words = %w[foo bar baz] enumerator = words.each enumerator.next # => "foo" enumerator.next # => "bar" enumerator.next # => "baz" enumerator.next # Raises StopIteration: iteration reached an end
External iteration methods in class Enumerator include:
-
feed: sets the value that is next to be returned. -
next: returns the next value and increments the position. -
next_values: returns the next value in a 1-element array and increments the position. -
peek: returns the next value but does not increment the position. -
peek_values: returns the next value in a 1-element array but does not increment the position. -
rewind: sets the position to zero.
Each of these methods raises FrozenError if called from a frozen Enumerator.
External Iteration and Fiber
External iteration that uses Fiber differs significantly from internal iteration:
-
Using Fiber adds some overhead compared to internal enumeration.
-
The stacktrace will only include the stack from the Enumerator, not above.
-
Fiber-local variables are not inherited inside the Enumerator Fiber, which instead starts with no Fiber-local variables.
-
Fiber storage variables are inherited and are designed to handle Enumerator Fibers. Assigning to a Fiber storage variable only affects the current Fiber, so if you want to change state in the caller Fiber of the Enumerator Fiber, you need to use an extra indirection (e.g., use some object in the Fiber storage variable and mutate some ivar of it).
Concretely:
Thread.current[:fiber_local] = 1 Fiber[:storage_var] = 1 e = Enumerator.new do |y| p Thread.current[:fiber_local] # for external iteration: nil, for internal iteration: 1 p Fiber[:storage_var] # => 1, inherited Fiber[:storage_var] += 1 y << 42 end p e.next # => 42 p Fiber[:storage_var] # => 1 (it ran in a different Fiber) e.each { p _1 } p Fiber[:storage_var] # => 2 (it ran in the same Fiber/"stack" as the current Fiber)
Converting External Iteration to Internal Iteration
You can use an external iterator to implement an internal iterator as follows:
def ext_each(e) while true begin vs = e.next_values rescue StopIteration return $!.result end y = yield(*vs) e.feed y end end o = Object.new def o.each puts yield puts yield(1) puts yield(1, 2) 3 end # use o.each as an internal iterator directly. puts o.each {|*x| puts x; [:b, *x] } # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3 # convert o.each to an external iterator for # implementing an internal iterator. puts ext_each(o.to_enum) {|*x| puts x; [:b, *x] } # => [], [:b], [1], [:b, 1], [1, 2], [:b, 1, 2], 3
static VALUE
enumerator_initialize(int argc, VALUE *argv, VALUE obj)
{
VALUE iter = rb_block_proc();
VALUE recv = generator_init(generator_allocate(rb_cGenerator), iter);
VALUE arg0 = rb_check_arity(argc, 0, 1) ? argv[0] : Qnil;
VALUE size = convert_to_feasible_size_value(arg0);
return enumerator_init(obj, recv, sym_each, 0, 0, 0, size, false);
}
Returns a new Enumerator object that can be used for iteration.
The given block defines the iteration; it is called with a “yielder” object that can yield an object via a call to method yielder.yield:
fib = Enumerator.new do |yielder| n = next_n = 1 while true do yielder.yield(n) n, next_n = next_n, n + next_n end end fib.take(10) # => [1, 1, 2, 3, 5, 8, 13, 21, 34, 55]
Parameter size specifies how the size is to be calculated (see size); it can either be a value or a callable object:
Enumerator.new{}.size # => nil Enumerator.new(42){}.size # => 42 Enumerator.new(-> {42}){}.size # => 42
static VALUE
enumerator_s_produce(int argc, VALUE *argv, VALUE klass)
{
VALUE init, producer, opts, size;
ID keyword_ids[1];
if (!rb_block_given_p()) rb_raise(rb_eArgError, "no block given");
keyword_ids[0] = rb_intern("size");
rb_scan_args_kw(RB_SCAN_ARGS_LAST_HASH_KEYWORDS, argc, argv, "01:", &init, &opts);
rb_get_kwargs(opts, keyword_ids, 0, 1, &size);
size = UNDEF_P(size) ? DBL2NUM(HUGE_VAL) : convert_to_feasible_size_value(size);
if (argc == 0 || (argc == 1 && !NIL_P(opts))) {
init = Qundef;
}
producer = producer_init(producer_allocate(rb_cEnumProducer), init, rb_block_proc(), size);
return rb_enumeratorize_with_size_kw(producer, sym_each, 0, 0, producer_size, RB_NO_KEYWORDS);
}
Creates an infinite enumerator from any block, just called over and over. The result of the previous iteration is passed to the next one. If initial is provided, it is passed to the first iteration, and becomes the first element of the enumerator; if it is not provided, the first iteration receives nil, and its result becomes the first element of the iterator.
Raising StopIteration from the block stops an iteration.
Enumerator.produce(1, &:succ) # => enumerator of 1, 2, 3, 4, .... Enumerator.produce { rand(10) } # => infinite random number sequence ancestors = Enumerator.produce(node) { |prev| node = prev.parent or raise StopIteration } enclosing_section = ancestors.find { |n| n.type == :section }
Using ::produce together with Enumerable methods like Enumerable#detect, Enumerable#slice_after, Enumerable#take_while can provide Enumerator-based alternatives for while and until cycles:
# Find next Tuesday require "date" Enumerator.produce(Date.today, &:succ).detect(&:tuesday?) # Simple lexer: require "strscan" scanner = StringScanner.new("7+38/6") PATTERN = %r{\d+|[-/+*]} Enumerator.produce { scanner.scan(PATTERN) }.slice_after { scanner.eos? }.first # => ["7", "+", "38", "/", "6"]
The optional size keyword argument specifies the size of the enumerator, which can be retrieved by Enumerator#size. It can be an integer, Float::INFINITY, a callable object (such as a lambda), or nil to indicate unknown size. When not specified, the size defaults to Float::INFINITY.
# Infinite enumerator enum = Enumerator.produce(1, size: Float::INFINITY, &:succ) enum.size # => Float::INFINITY # Finite enumerator with known/computable size abs_dir = File.expand_path("./baz") # => "/foo/bar/baz" traverser = Enumerator.produce(abs_dir, size: -> { abs_dir.count("/") + 1 }) { raise StopIteration if it == "/" File.dirname(it) } traverser.size # => 4 # Finite enumerator with unknown size calendar = Enumerator.produce(Date.today, size: nil) { it.monday? ? raise(StopIteration) : it + 1 } calendar.size # => nil
static VALUE
enumerator_s_product(int argc, VALUE *argv, VALUE klass)
{
VALUE enums = Qnil, options = Qnil, block = Qnil;
rb_scan_args(argc, argv, "*:&", &enums, &options, &block);
if (!NIL_P(options) && !RHASH_EMPTY_P(options)) {
rb_exc_raise(rb_keyword_error_new("unknown", rb_hash_keys(options)));
}
VALUE obj = enum_product_initialize(argc, argv, enum_product_allocate(rb_cEnumProduct));
if (!NIL_P(block)) {
enum_product_run(obj, block);
return Qnil;
}
return obj;
}
Generates a new enumerator object that generates a Cartesian product of given enumerable objects. This is equivalent to Enumerator::Product.new.
e = Enumerator.product(1..3, [4, 5]) e.to_a #=> [[1, 4], [1, 5], [2, 4], [2, 5], [3, 4], [3, 5]] e.size #=> 6
When a block is given, calls the block with each N-element array generated and returns nil.
static VALUE
enumerator_plus(VALUE obj, VALUE eobj)
{
return new_enum_chain(rb_ary_new_from_args(2, obj, eobj));
}
Returns an enumerator object generated from this enumerator and a given enumerable.
e = (1..3).each + [4, 5] e.to_a #=> [1, 2, 3, 4, 5]
static VALUE
enumerator_each(int argc, VALUE *argv, VALUE obj)
{
struct enumerator *e = enumerator_ptr(obj);
if (argc > 0) {
VALUE args = (e = enumerator_ptr(obj = rb_obj_dup(obj)))->args;
if (args) {
#if SIZEOF_INT < SIZEOF_LONG
/* check int range overflow */
rb_long2int(RARRAY_LEN(args) + argc);
#endif
args = rb_ary_dup(args);
rb_ary_cat(args, argv, argc);
}
else {
args = rb_ary_new4(argc, argv);
}
RB_OBJ_WRITE(obj, &e->args, args);
e->size = Qnil;
e->size_fn = 0;
}
if (!rb_block_given_p()) return obj;
if (!lazy_precheck(e->procs)) return Qnil;
return enumerator_block_call(obj, 0, obj);
}
Iterates over the block according to how this Enumerator was constructed. If no block and no arguments are given, returns self.
Examples
"Hello, world!".scan(/\w+/) #=> ["Hello", "world"] "Hello, world!".to_enum(:scan, /\w+/).to_a #=> ["Hello", "world"] "Hello, world!".to_enum(:scan).each(/\w+/).to_a #=> ["Hello", "world"] obj = Object.new def obj.each_arg(a, b=:b, *rest) yield a yield b yield rest :method_returned end enum = obj.to_enum :each_arg, :a, :x enum.each.to_a #=> [:a, :x, []] enum.each.equal?(enum) #=> true enum.each { |elm| elm } #=> :method_returned enum.each(:y, :z).to_a #=> [:a, :x, [:y, :z]] enum.each(:y, :z).equal?(enum) #=> false enum.each(:y, :z) { |elm| elm } #=> :method_returned
static VALUE
enumerator_each_with_index(VALUE obj)
{
return enumerator_with_index(0, NULL, obj);
}
Same as Enumerator#with_index(0), i.e. there is no starting offset.
If no block is given, a new Enumerator is returned that includes the index.
static VALUE
enumerator_with_object(VALUE obj, VALUE memo)
{
RETURN_SIZED_ENUMERATOR(obj, 1, &memo, enumerator_enum_size);
enumerator_block_call(obj, enumerator_with_object_i, memo);
return memo;
}
Iterates the given block for each element with an arbitrary object, obj, and returns obj
If no block is given, returns a new Enumerator.
Example
to_three = Enumerator.new do |y| 3.times do |x| y << x end end to_three_with_string = to_three.with_object("foo") to_three_with_string.each do |x,string| puts "#{string}: #{x}" end # => foo: 0 # => foo: 1 # => foo: 2
static VALUE
enumerator_feed(VALUE obj, VALUE v)
{
struct enumerator *e = enumerator_ptr(obj);
rb_check_frozen(obj);
if (!UNDEF_P(e->feedvalue)) {
rb_raise(rb_eTypeError, "feed value already set");
}
RB_OBJ_WRITE(obj, &e->feedvalue, v);
return Qnil;
}
Sets the value to be returned by the next yield inside e.
If the value is not set, the yield returns nil.
This value is cleared after being yielded.
# Array#map passes the array's elements to "yield" and collects the # results of "yield" as an array. # Following example shows that "next" returns the passed elements and # values passed to "feed" are collected as an array which can be # obtained by StopIteration#result. e = [1,2,3].map p e.next #=> 1 e.feed "a" p e.next #=> 2 e.feed "b" p e.next #=> 3 e.feed "c" begin e.next rescue StopIteration p $!.result #=> ["a", "b", "c"] end o = Object.new def o.each x = yield # (2) blocks p x # (5) => "foo" x = yield # (6) blocks p x # (8) => nil x = yield # (9) blocks p x # not reached w/o another e.next end e = o.to_enum e.next # (1) e.feed "foo" # (3) e.next # (4) e.next # (7) # (10)
static VALUE
enumerator_inspect(VALUE obj)
{
return rb_exec_recursive(inspect_enumerator, obj, 0);
}
Creates a printable version of e.
static VALUE
enumerator_next(VALUE obj)
{
VALUE vs = enumerator_next_values(obj);
return ary2sv(vs, 0);
}
Returns the next object in the enumerator, and move the internal position forward. When the position reached at the end, StopIteration is raised.
Example
a = [1,2,3] e = a.to_enum p e.next #=> 1 p e.next #=> 2 p e.next #=> 3 p e.next #raises StopIteration
See class-level notes about external iterators.
static VALUE
enumerator_next_values(VALUE obj)
{
struct enumerator *e = enumerator_ptr(obj);
VALUE vs;
rb_check_frozen(obj);
if (!UNDEF_P(e->lookahead)) {
vs = e->lookahead;
e->lookahead = Qundef;
return vs;
}
return get_next_values(obj, e);
}
Returns the next object as an array in the enumerator, and move the internal position forward. When the position reached at the end, StopIteration is raised.
See class-level notes about external iterators.
This method can be used to distinguish yield and yield nil.
Example
o = Object.new def o.each yield yield 1 yield 1, 2 yield nil yield [1, 2] end e = o.to_enum p e.next_values p e.next_values p e.next_values p e.next_values p e.next_values e = o.to_enum p e.next p e.next p e.next p e.next p e.next ## yield args next_values next # yield [] nil # yield 1 [1] 1 # yield 1, 2 [1, 2] [1, 2] # yield nil [nil] nil # yield [1, 2] [[1, 2]] [1, 2]
static VALUE
enumerator_peek(VALUE obj)
{
VALUE vs = enumerator_peek_values(obj);
return ary2sv(vs, 1);
}
Returns the next object in the enumerator, but doesn’t move the internal position forward. If the position is already at the end, StopIteration is raised.
See class-level notes about external iterators.
Example
a = [1,2,3] e = a.to_enum p e.next #=> 1 p e.peek #=> 2 p e.peek #=> 2 p e.peek #=> 2 p e.next #=> 2 p e.next #=> 3 p e.peek #raises StopIteration
static VALUE
enumerator_peek_values_m(VALUE obj)
{
return rb_ary_dup(enumerator_peek_values(obj));
}
Returns the next object as an array, similar to Enumerator#next_values, but doesn’t move the internal position forward. If the position is already at the end, StopIteration is raised.
See class-level notes about external iterators.
Example
o = Object.new def o.each yield yield 1 yield 1, 2 end e = o.to_enum p e.peek_values #=> [] e.next p e.peek_values #=> [1] p e.peek_values #=> [1] e.next p e.peek_values #=> [1, 2] e.next p e.peek_values # raises StopIteration
static VALUE
enumerator_rewind(VALUE obj)
{
struct enumerator *e = enumerator_ptr(obj);
rb_check_frozen(obj);
rb_check_funcall(e->obj, id_rewind, 0, 0);
e->fib = 0;
e->dst = Qnil;
e->lookahead = Qundef;
e->feedvalue = Qundef;
e->stop_exc = Qfalse;
return obj;
}
Rewinds the enumeration sequence to the beginning.
If the enclosed object responds to a “rewind” method, it is called.
static VALUE
enumerator_size(VALUE obj)
{
struct enumerator *e = enumerator_ptr(obj);
int argc = 0;
const VALUE *argv = NULL;
VALUE size;
if (e->procs) {
struct generator *g = generator_ptr(e->obj);
VALUE receiver = rb_check_funcall(g->obj, id_size, 0, 0);
long i = 0;
for (i = 0; i < RARRAY_LEN(e->procs); i++) {
VALUE proc = RARRAY_AREF(e->procs, i);
struct proc_entry *entry = proc_entry_ptr(proc);
lazyenum_size_func *size_fn = entry->fn->size;
if (!size_fn) {
return Qnil;
}
receiver = (*size_fn)(proc, receiver);
}
return receiver;
}
if (e->size_fn) {
return (*e->size_fn)(e->obj, e->args, obj);
}
if (e->args) {
argc = (int)RARRAY_LEN(e->args);
argv = RARRAY_CONST_PTR(e->args);
}
size = rb_check_funcall_kw(e->size, id_call, argc, argv, e->kw_splat);
if (!UNDEF_P(size)) return size;
return e->size;
}
Returns the size of the enumerator, or nil if it can’t be calculated lazily.
(1..100).to_a.permutation(4).size # => 94109400 loop.size # => Float::INFINITY (1..100).drop_while.size # => nil
Note that enumerator size might be inaccurate, and should be rather treated as a hint. For example, there is no check that the size provided to ::new is accurate:
e = Enumerator.new(5) { |y| 2.times { y << it} } e.size # => 5 e.to_a.size # => 2
Another example is an enumerator created by ::produce without a size argument. Such enumerators return Infinity for size, but this is inaccurate if the passed block raises StopIteration:
e = Enumerator.produce(1) { it + 1 } e.size # => Infinity e = Enumerator.produce(1) { it > 3 ? raise(StopIteration) : it + 1 } e.size # => Infinity e.to_a.size # => 4
static VALUE
enumerator_with_index(int argc, VALUE *argv, VALUE obj)
{
VALUE memo;
rb_check_arity(argc, 0, 1);
RETURN_SIZED_ENUMERATOR(obj, argc, argv, enumerator_enum_size);
memo = (!argc || NIL_P(memo = argv[0])) ? INT2FIX(0) : rb_to_int(memo);
return enumerator_block_call(obj, enumerator_with_index_i, (VALUE)rb_imemo_memo_new(memo, 0, 0));
}
Iterates the given block for each element with an index, which starts from offset. If no block is given, returns a new Enumerator that includes the index, starting from offset
offset-
the starting index to use