Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Redesign Proposal #22

Closed
wants to merge 13 commits into from
Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
name = "TermInterface"
uuid = "8ea1fca8-c5ef-4a55-8b96-4e9afe9c9a3c"
authors = ["Shashi Gowda <[email protected]>", "Alessandro Cheli <[email protected]>"]
version = "0.3.3"
version = "0.4"

[compat]
julia = "1"
Expand Down
102 changes: 91 additions & 11 deletions src/TermInterface.jl
Original file line number Diff line number Diff line change
Expand Up @@ -45,12 +45,41 @@ and pattern matching features.
function exprhead end
export exprhead

"""
head(x)

If `x` is a term as defined by `istree(x)`, `head(x)` returns the head of the
term if `x`. The `head` type has to be provided by the package.
"""
function head end
export head

"""
head_symbol(x::HeadType)

If `x` is a head object, `head_symbol(T, x)` returns a `Symbol` object that
corresponds to `y.head` if `y` was the representation of the corresponding term
as a Julia Expression. This is useful to define interoperability between
symbolic term types defined in different packages and should be used when
calling `maketerm`.
"""
function head_symbol end
export head_symbol
0x0f0f0f marked this conversation as resolved.
Show resolved Hide resolved

"""
tail(x)

Get the arguments of `x`, must be defined if `istree(x)` is `true`.
"""
function tail end
export tail

0x0f0f0f marked this conversation as resolved.
Show resolved Hide resolved

"""
operation(x)

If `x` is a term as defined by `istree(x)`, `operation(x)` returns the
head of the term if `x` represents a function call, for example, the head
operation of the term if `x` represents a function call, for example, the head
is the function being called.
"""
function operation end
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again should not be in this package.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I believe this one should stay in this package.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This package is used to define an AST interface so that pattern matchers can match against the AST and traverse it. I think it's a little bit of a mistake to introduce @rule with the example of +(~a, ~b), because in many IRs we would want to write this as :call(+, ~a, ~b). Not all ASTs will have something analogous to a function call, and it's unclear what benefit would be derived from standardizing this notion here. operation and arguments should live in packages that define ASTs that can give these functions meaning.

TL;DR: I don't think we can define what the new meaning of operation is supposed to do in a way that captures all of its possible use cases in downstream packages, so I think it should not be in this package.

Copy link
Member Author

@0x0f0f0f 0x0f0f0f Dec 9, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not all ASTs will have something analogous to a function call

I guess most of what the (old/current) dependents of this package do:

  • Metatheory.jl AbstractPat AST for patterns
  • Julia Exprs
  • SymbolicUtils IIRC terms have function calls and array indexing
  • Everything depending on Symbolics
  • Most other symbolic mathematics packages.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can use trait like abstractrees? Will make a larger comment below

Copy link
Contributor

@willow-ahrens willow-ahrens Jan 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shashi, do we agree now that function operation end is fine as-is, just adjust the documentation to say that this one is optional?

Expand Down Expand Up @@ -108,23 +137,74 @@ end


"""
similarterm(x, head, args, symtype=nothing; metadata=nothing, exprhead=:call)
maketerm(head::H, tail; type=Any, metadata=nothing)

Has to be implemented by the provider of H.
Returns a term that is in the same closure of types as `typeof(x)`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x is no longer defined here.

with `head` as the head and `args` as the arguments, `type` as the symtype
and `metadata` as the metadata. By default this will execute `head(args...)`.
`x` parameter can also be a `Type`. The `exprhead` keyword argument is useful
when manipulating `Expr`s.
with `head` as the head and `tail` as the arguments, `type` as the symtype
and `metadata` as the metadata.
"""
function similarterm(x, head, args, symtype = nothing; metadata = nothing, exprhead = nothing)
head(args...)
end
function maketerm(head, tail; type=Any, metadata=nothing) end
0x0f0f0f marked this conversation as resolved.
Show resolved Hide resolved
export maketerm

export similarterm
"""
is_operation(f)

Returns a single argument anonymous function predicate, that returns `true` if and only if
the argument to the predicate satisfies `istree` and `operation(x) == f`
"""
is_operation(f) = @nospecialize(x) -> istree(x) && (operation(x) == f)
export is_operation

include("utils.jl")

"""
node_count(t)
Count the nodes in a symbolic expression tree satisfying `istree` and `arguments`.
"""
node_count(t) = istree(t) ? reduce(+, node_count(x) for x in arguments(t), init = 0) + 1 : 1
export node_count

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine, though I wonder if AbstractTrees could help here.

include("expr.jl")

"""
@matchable struct Foo fields... end [HeadType]

Take a struct definition and automatically define `TermInterface` methods. This
will automatically define a head type. If `HeadType` is given then it will be
used as `head(::Foo)`. If it is omitted, and the struct is called `Foo`, then
the head type will be called `FooHead`. The `head_symbol` of such head types
will default to `:call`.
"""
macro matchable(expr, head_name=nothing)
@assert expr.head == :struct
name = expr.args[2]
if name isa Expr
name.head === :(<:) && (name = name.args[1])
name isa Expr && name.head === :curly && (name = name.args[1])
end
fields = filter(x -> x isa Symbol || (x isa Expr && x.head == :(::)), expr.args[3].args)
get_name(s::Symbol) = s
get_name(e::Expr) = (@assert(e.head == :(::)); e.args[1])
fields = map(get_name, fields)
head_name = isnothing(head_name) ? Symbol(name, :Head) : head_name

quote
$expr
struct $head_name
head
end
TermInterface.head_symbol(x::$head_name) = x.head
# TODO default to call?
TermInterface.head(::$name) = $head_name(:call)
TermInterface.istree(::$name) = true
TermInterface.operation(::$name) = $name
TermInterface.arguments(x::$name) = getfield.((x,), ($(QuoteNode.(fields)...),))
TermInterface.tail(x::$name) = [operation(x); arguments(x)...]
TermInterface.arity(x::$name) = $(length(fields))
Base.length(x::$name) = $(length(fields) + 1)
end |> esc
end
export @matchable

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This approach to building ASTs results in huge slowdowns in compilers, because compilers specialize on each combination of node types. I appreciate the idea of adding an optional AST easy implementation feature, could we add it in a separate PR?

A few ideas:

  • enforce symbols or enums to differentiate heads in the AST (runtime not inference time)
  • make it clear that head_symbol is not part of the TermInterface interface, it's just part of the easy-mode ast builder.
  • add a simplified approach towards types with dynamic field information. Because we would emit types that have the same type for every head, we would need to use dynamic field information.

If we feel like an optimized AST implementation is out of scope, we shouldn't include programming patterns that will inconvenience users later on with performance issues.

end # module

50 changes: 33 additions & 17 deletions src/expr.jl
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file seems like the right approach to me

Original file line number Diff line number Diff line change
@@ -1,26 +1,42 @@
# This file contains default definitions for TermInterface methods on Julia
# Builtin Expr type.

istree(x::Expr) = true
exprhead(e::Expr) = e.head

operation(e::Expr) = expr_operation(e, Val{exprhead(e)}())
arguments(e::Expr) = expr_arguments(e, Val{exprhead(e)}())

# See https://docs.julialang.org/en/v1/devdocs/ast/
expr_operation(e::Expr, ::Union{Val{:call},Val{:macrocall}}) = e.args[1]
expr_operation(e::Expr, ::Union{Val{:ref}}) = getindex
expr_operation(e::Expr, ::Val{T}) where {T} = T
struct ExprHead
head
end
export ExprHead

expr_arguments(e::Expr, ::Union{Val{:call},Val{:macrocall}}) = e.args[2:end]
expr_arguments(e::Expr, _) = e.args
head_symbol(eh::ExprHead) = eh.head

istree(x::Expr) = true
head(e::Expr) = ExprHead(e.head)
tail(e::Expr) = e.args

function similarterm(x::Expr, head, args, symtype = nothing; metadata = nothing, exprhead = exprhead(x))
expr_similarterm(head, args, Val{exprhead}())
# See https://docs.julialang.org/en/v1/devdocs/ast/
function operation(e::Expr)
h = head(e)
hh = h.head
if hh in (:call, :macrocall)
e.args[1]
else
hh
end
end

function arguments(e::Expr)
h = head(e)
hh = h.head
if hh in (:call, :macrocall)
e.args[2:end]
else
e.args
end
end

expr_similarterm(head, args, ::Val{:call}) = Expr(:call, head, args...)
expr_similarterm(head, args, ::Val{:macrocall}) = Expr(:macrocall, head, args...) # discard linenumbernodes?
expr_similarterm(head, args, ::Val{eh}) where {eh} = Expr(eh, args...)
function maketerm(head::ExprHead, tail; type=Any, metadata=nothing)
if !isempty(tail) && first(tail) isa Union{Function,DataType}
Expr(head.head, nameof(first(tail)), @view(tail[2:end])...)
else
Expr(head.head, tail...)
end
end
16 changes: 0 additions & 16 deletions src/utils.jl

This file was deleted.

69 changes: 61 additions & 8 deletions test/runtests.jl
Original file line number Diff line number Diff line change
@@ -1,17 +1,70 @@
using TermInterface
using Test
using TermInterface, Test

@testset "Expr" begin
ex = :(f(a, b))
@test head(ex) == ExprHead(:call)
@test tail(ex) == [:f, :a, :b]
@test operation(ex) == :f
@test arguments(ex) == [:a, :b]
@test exprhead(ex) == :call
@test ex == similarterm(ex, :f, [:a, :b])
@test ex == maketerm(ExprHead(:call), [:f, :a, :b])

ex = :(arr[i, j])
@test operation(ex) == getindex
@test head(ex) == ExprHead(:ref)
@test operation(ex) == :ref
@test arguments(ex) == [:arr, :i, :j]
@test exprhead(ex) == :ref
@test ex == similarterm(ex, :ref, [:arr, :i, :j]; exprhead = :ref)
@test ex == similarterm(ex, :ref, [:arr, :i, :j])
@test ex == maketerm(ExprHead(:ref), [:arr, :i, :j])


ex = :(i, j)
@test head(ex) == ExprHead(:tuple)
@test operation(ex) == :tuple
@test arguments(ex) == [:i, :j]
@test tail(ex) == [:i, :j]
@test ex == maketerm(ExprHead(:tuple), [:i, :j])


ex = Expr(:block, :a, :b, :c)
@test head(ex) == ExprHead(:block)
@test operation(ex) == :block
@test tail(ex) == arguments(ex) == [:a, :b, :c]
@test ex == maketerm(ExprHead(:block), [:a, :b, :c])
end

@testset "Custom Struct" begin
struct Foo
args
Foo(args...) = new(args)
end
struct FooHead
head
end
TermInterface.head(::Foo) = FooHead(:call)
TermInterface.head_symbol(q::FooHead) = q.head
TermInterface.operation(::Foo) = Foo
TermInterface.istree(::Foo) = true
TermInterface.arguments(x::Foo) = [x.args...]
TermInterface.tail(x::Foo) = [operation(x); x.args...]

t = Foo(1, 2)
@test head(t) == FooHead(:call)
@test head_symbol(head(t)) == :call
@test operation(t) == Foo
@test istree(t) == true
@test arguments(t) == [1, 2]
@test tail(t) == [Foo, 1, 2]
end

@testset "Automatically Generated Methods" begin
@matchable struct Bar
a
b::Int
end

t = Bar(1, 2)
@test head(t) == BarHead(:call)
@test head_symbol(head(t)) == :call
@test operation(t) == Bar
@test istree(t) == true
@test arguments(t) == (1, 2)
@test tail(t) == [Bar, 1, 2]
end
Loading