Error `@yield macro outside a @resumable function!` when using @yield inside a macro

[wmanning-cra]

I wanted a macro that takes a sub-generator and yields its elements. I wrote this:

macro yieldmany(resumable::Expr)
      return quote
          for x in $(esc(resumable))
            @yield x
          end
      end
    end

However, when I use it in a function, I get an error:

@resumable function F2()
        @yield 2
        @yieldmany F1()
        @yield 4
     end
  ERROR: LoadError: LoadError: @yield macro outside a @resumable function!

I tried using macroexpand to see what it's making, but I get the same error:

@macroexpand @yieldmany F1()
ERROR: LoadError: @yield macro outside a @resumable function!

@macroexpand @yield x
ERROR: LoadError: @yield macro outside a @resumable function!

[wmanning-cra]

When i write it out manually, things work as expected:

@resumable function F2()
          @yield 2
          for i in F1()
             @yield i
           end
          @yield 4
end
julia> collect(F2())
4-element Array{Any,1}:
 2
 1
 3
 4

[hdavid16]

What is F1? I've seen weird things happen when trying to call macros inside a quote block (https://discourse.julialang.org/t/error-handling-in-quote-block-using-jump/79362/3?u=hdperez). I would discourage doing this.

[cscherrer]

There are a couple of issues here involving using macros within @resumable. Usually, a quick fix would be to have @resumable call macroexpand on the expression before rewriting things. The problem with that is that there's not actually a working @yield anywhere. In a way, it's not really a macro, more of a syntactic indicator the rewrite engine looks at. So macroexpand will see a @yield and get very confused.

But I think a PR adding something like this could work:

function expand(M, ex::Expr)
    head = ex.head
    args = ex.args

    if head == :macrocall && args[1] != Symbol("@yield")
        return expand(M, macroexpand(M, ex; recursive=false))
    else
        newargs = map(Base.Fix1(expand, M), ex.args)
        return Expr(ex.head, newargs...)
    end
end
        
expand(M, s) = s

With your example, you'd then get

julia> macro yieldmany(resumable::Expr)
           return quote
               for x in $(esc(resumable))
                 @yield x
               end
           end
       end
@yieldmany

julia> ex = quote
           function F2()
               @yield 2
               @yieldmany F1()
               @yield 4
           end
       end;

julia> expand(Main, ex)
quote
    #= REPL[174]:2 =#
    function F2()
        #= REPL[174]:2 =#
        #= REPL[174]:3 =#
        #= REPL[174]:3 =# @yield 2
        #= REPL[174]:4 =#
        begin
            #= REPL[173]:3 =#
            for var"#889#x" = F1()
                #= REPL[173]:4 =#
                #= REPL[173]:4 =# @yield x
                #= REPL[173]:5 =#
            end
        end
        #= REPL[174]:5 =#
        #= REPL[174]:5 =# @yield 4
    end
end

It's not quite right - the variables in the for loop got mixed up - but I think something along these lines could be made to work.