前置知识：KDA算法公式、triton、

1 TVM算子融合介绍

1.1 图级优化

TVM图级优化按照优化范围，可分为局部优化和全局优化

局部优化是TVM图级优化的重点，其中算子融合是AI编译器必不可少的优化方法。 算子融合核心思想就是**将多个算子合并成一个内核，因而无需将中间结果写回全局内存，减少了中间变量的分配，也减少了片上缓存和片外存储之间的数据传输，**从而节省计算时间。
全局优化：计算图提供了全局视图，TVM可以在整个计算图中搜索特定特征，并针对这些特征执行特定全局优化操作。例如：计算图中的死代码通常是由其他图优化造成的。因此，在其他图优化之后还应执行死代码清除(Deal Code Elimination,DCE) 和公共子表达式消除(Common Subexpression Elimination,CSE)操作。

1.2 TVM中，算子融合的种类

enum OpPatternKind {
// Elementwise operation
kElemWise = 0,
// Broadcasting operator, can always map output axis to the input in order.
// for example :code:`out[i, ax1, j, ax2] = input[i, j]`.
// Note that the axis need to be in order so transpose is not a bcast operator.
kBroadcast = 1,
// Injective operator, can always injectively map output axis to a single input axis.
// All injective operator can still be safely fused to injective and reduction.
kInjective = 2,
// Communicative reduction operator.
kCommReduce = 3,
// Complex operation, can still fuse elemwise operations into its output.
// but cannot chain another complex op
kOutEWiseFusable = 4,
// The pattern for tuple nodes. Can fuse into subsequent injective ops,
// but treated specially
kTuple = 7,
// Opaque operation, cannot fuse anything.
kOpaque = 8
};

kElemWise：两个tensor之间按照元素逐个操作的算子，实际上所有的四则运算都是这种类型
kBroadcast：带有广播操作的算子
kInjective：输入和输出之间具有一对一映射关系的算子，如add/sqrt/exp等操作算子(operator)
kCommReduce：多到少的映射，输入到输出就有降维性质，如sum/max/min等操作算子
kOutEWiseFusable：这是计算比较复杂的算子，输出可与kElemWise进行fuse的算子，如conv2d/bn/relu等算子
kTuple：操作元祖的算子，如TupleNode，TupleGetItemNode等；
kOpaque：无法进行融合的算子，如sort

为了制定通用的融合规则，分为了四类：

Injective（单射/一对一映射算子）

定义：输入和输出是一一对应的关系。
例子：Add（加法）、ReLU、Sigmoid 等。
特点：并行度高，通常只涉及简单的逐元素（Element-wise）操作。

Reduction（归约算子）

定义：将多个输入元素组合计算成较少的输出元素（通常会降低张量的维度）。
例子：Sum（求和）、Max（最大值）、Mean（平均值）。
特点：需要对输入数据进行累加或比较，涉及跨元素的数据依赖。

Complex-out-fusable（复杂输出可融合算子）

定义：计算逻辑复杂，但允许将后续的逐元素操作融合到其输出阶段的算子。
例子：Conv2d（二维卷积）、MatMul（矩阵乘法）。
特点：这些通常是计算密集型算子。虽然它们内部很复杂，但我们可以把跟在它后面的简单操作（如激活函数）直接在生成输出时顺便做掉，而不需要把中间结果写回内存。

Opaque（不透明算子）

定义：无法被简单融合的算子，或者编译器不知道其内部实现细节的算子。
例子：Sort（排序）。
特点：这类算子通常作为融合的边界，既不能融合别人，也很难被别人融合。

1.3 算子融合的规则 (Generic Rules)

基于上述分类，文本提出了通用的融合策略，目的是将多个小算子合并成一个大算子（Kernel）：

规则一：Injective + Injective（单射融合）
描述：多个连续的单射算子可以合并成一个单射算子。
例子：先做（Add），再做。可以融合在一个循环里完成，不需要中间存取。
规则二：Injective + Reduction（输入融合）
描述：归约算子可以将其输入端的单射算子融合进来。
例子：Scale（缩放，属于Injective）后接 Sum（求和）。可以在读取数据进行求和的过程中，顺便把缩放算好。这被称为“融合到归约算子的输入”。
规则三：Complex + Injective（输出融合）
描述：像 Conv2d 这样的复杂算子，可以将后续的逐元素（Element-wise）算子融合到其输出阶段。
例子：Conv2d 后接 ReLU。在卷积计算出结果写入内存之前，直接对寄存器里的值应用 ReLU，然后再写入内存。

2 TVM算子融合代码解析

pass之前的IR如下

@R.function
    def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
        cls = Module
        with R.dataflow():
            lv = R.call_tir(cls.add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv1 = R.call_tir(cls.exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv2 = R.call_tir(cls.tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv3 = R.call_tir(cls.add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            gv: R.Tensor((10, 20), dtype="float32") = lv3
            R.output(gv)
        return gv
IRModule FuseOps(IRModule mod, int opt_level, size_t max_fuse_depth) {
  support::Arena arena;

  // Step 1. Create the indexed-forward graph according to the input IRModule.
  IndexedForwardGraph graph = GraphCreator::Create(mod, &arena);

  // Step 2. Partition the graph by applying the fusion algorithm.
  std::vector<GraphPartitioner::Group*> groups =
      GraphPartitioner(&arena, opt_level, max_fuse_depth, /*max_function_args=*/0).Partition(graph);

  // Step 3. Transform the IRModule by fusing the operators in accordance with the graph partition
  // results.
  return OperatorFusor(mod, graph, groups, /*lift_constants*/ true).Transform();
}

Pass FuseOps(int fuse_opt_level) {
  auto pass_func =  //
      [=](IRModule m, PassContext pc) {
        int opt_level = fuse_opt_level == -1 ? pc->opt_level : fuse_opt_level;
        auto max_fuse_depth = pc->GetConfig("relax.FuseOps.max_depth", Integer(kMaxFusedOps));
        return relax::FuseOps(m, opt_level, max_fuse_depth.value().IntValue());
      };
  return CreateModulePass(/*pass_function=*/pass_func,  //
                          /*opt_level=*/0,              //
                          /*name=*/"FuseOps",           //
                          /*required=*/{});
}

TVM_FFI_STATIC_INIT_BLOCK() {
  namespace refl = tvm::ffi::reflection;
  refl::GlobalDef().def("relax.transform.FuseOps", FuseOps);
}

总体分为三步：

遍历relax树结构，采用indexed-forward graph来存储DAG
构建后支配树，能够快速求取任意节点的后支配点
根据当前节点的后支配点信息，在两节点路径之间进行融合算法

2.1 Create the indexed-forward graph

2.1.1 图DS解析

class IndexedForwardGraph {
 public:
  struct Node;
  /*!
   * The forward edge in the dataflow graph.
   */
  struct Edge {
    /*! \brief The corresponding node */
    Node* node{nullptr};
    /*! \brief The respective pattern of this op */
    OpPatternKind pattern{kOpaque};
  };
  /*! \brief A node in the graph. */
  struct Node {
    /*! \brief weak reference to the corresponding edge. */
    const tvm::Object* ref{nullptr};
    /*! \brief The index of the node in topological order. */
    size_t index{0};
    /*! \brief Whether this node is referenced by external source */
    bool extern_ref{false};
    /*! \brief The general pattern in the node */
    OpPatternKind pattern{kOpaque};
    /*! \brief The outputs of the node. */
    LinkedList<Edge> outputs;
  };
  /*! \brief The node map that maps node to graph */
  std::unordered_map<const tvm::Object*, Node*> node_map;
  /*! \brief All the nodes in post DFS order */
  std::vector<Node*> post_dfs_order;
};

Node：表示节点，存储了引用对象reg, 拓扑序index, 是否被引用extern_ref, 算子类型pattern以及节点输出边outputs这些信息
Edge：表示边，存储管理的node节点以及算子的pattern
node_map：存储了对象和节点的映射关系
post_dfs_order：保存了所有节点的后序遍历节点

该类主要通过IndexedForwardGraphCreator creator对 Relay IR转换为 Graph node 的 IR 数据结构的转换。

IndexedForwardGraphCreator 继承 ExprVisitor，主要对 FunctionNode、CallNode、ConstantNode等节点的遍历进行重写

2.1.2 图工具类

创建图节点

IndexedForwardGraph::Node* CreateNode(const Object* key) {
    ICHECK(graph_.node_map.find(key) == graph_.node_map.end())
        << "The object " << ffi::GetRef<ObjectRef>(key) << " appears at multiple definition sites.";
    auto* node = arena_->make<IndexedForwardGraph::Node>();
    graph_.node_map[key] = node;
    return node;
  }

把节点加入DFS序列

void AddToPostDFSOrder(IndexedForwardGraph::Node* node, const Object* key) {
    auto it = graph_.node_map.find(key);
    ICHECK(it != graph_.node_map.end() && it->second == node)
        << "Cannot add node " << ffi::GetRef<ObjectRef>(key) << " to the post-DFS order, "
        << "because the node for this object has not yet been created.";

    // We only set the reference of the node when adding it to the post-dfs order. Thus, if the
    // reference of a node is already set, it must have been appended to the post-dfs order.
    ICHECK(node->ref == nullptr) << "Cannot add node " << ffi::GetRef<ObjectRef>(key)
                                 << " to the post-DFS order, "
                                 << "because it has already been added.";

    node->ref = key;
    node->index = graph_.post_dfs_order.size();
    graph_.post_dfs_order.push_back(node);
  }

添加边

void AddEdge(IndexedForwardGraph::Node* start, IndexedForwardGraph::Node* end,
               OpPatternKind pattern) {
    auto* link = arena_->make<LinkNode<IndexedForwardGraph::Edge>>();
    link->value.node = end;
    link->value.pattern = pattern;
    start->outputs.Push(link);
  }

设置节点为外部引用

1	void MarkAsExternRef(IndexedForwardGraph::Node* node) { node->extern_ref = true; }

设置节点融合模式

void SetNodePattern(IndexedForwardGraph::Node* node, OpPatternKind pattern) {
    ICHECK(initialized_nodes_.find(node) == initialized_nodes_.end())
        << "The input node " << ffi::GetRef<ObjectRef>(node->ref)
        << " cannot have have its OpPatternKind set more than once.";
    initialized_nodes_.insert(node);
    node->pattern = pattern;
  }

2.1.3 图构建流程

static IndexedForwardGraph Create(IRModule mod, support::Arena* arena) {
    GraphCreator creator(mod, arena);
    for (const auto& it : mod->functions) {
      // Only visit Relax functions with neither attr::kPrimitive nor
      // attr::kCodegen.  Relax functions with `attr::kPrimitive` are
      // previously fused functions, potentially from a previous use
      // of `FuseOps` or `FuseOpsByPattern`.  Relax functions with
      // `attr::kCodegen` are previously fused functions from
      // `FuseOpsByPattern`, when the `annotate_codegen` option is
      // true.
      const auto* func = it.second.as<FunctionNode>();
      if (func == nullptr || func->HasNonzeroAttr(attr::kPrimitive) ||
          func->GetAttr<ffi::String>(attr::kCodegen).has_value()) {
        continue;
      }
      creator(ffi::GetRef<Function>(func));
    }

    // The algorithm of the graph creator ensures that each created node will be added to the
    // post-dfs order and will be set its op pattern. Thus we check whether all these containers
    // have the same size.
    size_t n_nodes = creator.graph_.node_map.size();
    ICHECK_EQ(n_nodes, creator.graph_.post_dfs_order.size());
    ICHECK_EQ(n_nodes, creator.initialized_nodes_.size());

    return creator.graph_;
  }

遍历IRModule的function（对应样例的main）

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    cls = Module
    with R.dataflow():
        lv = R.call_tir(cls.add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv1 = R.call_tir(cls.exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv2 = R.call_tir(cls.tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv3 = R.call_tir(cls.add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        gv: R.Tensor((10, 20), dtype="float32") = lv3
        R.output(gv)
    return gv

GraphCreator继承ExprVisitor，ExprVisitor继承ExprFunctor

R operator()(const Expr& n, Args... args) { return VisitExpr(n, std::forward<Args>(args)...); }
 
virtual R VisitExpr(const Expr& n, Args... args) {
   ICHECK(n.defined()) << "Found null pointer node while traversing AST. The previous pass may "
                          "have generated invalid data.";
   static FType vtable = InitVTable();
   return vtable(n, this, std::forward<Args>(args)...);
 }

重载+虚函数+函数指针实现不同的VisitExpr的分发机制

这里会调用

void VisitExpr_(const FunctionNode* func) final {
    for (const Var& param : func->params) {
      IndexedForwardGraph::Node* param_node = CreateNode(param.get());
      // The parameter is passed in from the outside, and thus it's marked as an external reference,
      // and it's pattern is `kOpaque`.
      MarkAsExternRef(param_node);
      SetNodePattern(param_node, OpPatternKind::kOpaque);
      AddToPostDFSOrder(param_node, param.get());
    }
    if (auto opt_num_input = func->GetAttr<Integer>(attr::kNumInput)) {
      for (int i = static_cast<int>(opt_num_input.value()->value);
           i < static_cast<int>(func->params.size()); ++i) {
        input_params_.insert(func->params[i].get());
      }
    }
    ExprVisitor::VisitExpr_(func);
  }

逻辑分析：

func是上面的main函数
对函数的入参（这里是x、w）构建图节点，
标记为外部引用
节点模式设置为禁止融合
加入dfs序列
入参存进input_params_
进入父类的VisitExpr_方法进行递归遍历。

在父类中走如下函数调用

void ExprVisitor::VisitExpr_(const FunctionNode* op) {
  this->VisitSpan(op->span);
  for (Var param : op->params) {
    this->VisitVarDef(param);
  }

  this->VisitExpr(op->body);
  // FuncStructInfo does not depend on Expr.
}
void ExprVisitor::VisitExpr_(const SeqExprNode* op) {
  this->VisitSpan(op->span);
  for (BindingBlock block : op->blocks) {
    this->VisitBindingBlock(block);
  }
  this->VisitExpr(op->body);

  if (auto* sinfo = op->struct_info_.as<StructInfoNode>()) {
    this->VisitExprDepStructInfoField(ffi::GetRef<StructInfo>(sinfo));
  }
}
void ExprVisitor::VisitBindingBlock(const BindingBlock& block) {
  if (const auto* node = block.as<DataflowBlockNode>()) {
    VisitBindingBlock_(node);
  } else if (const auto* node = block.as<BindingBlockNode>()) {
    VisitBindingBlock_(node);
  } else {
    LOG(FATAL) << "TypeError: Invalid type: " << block->GetTypeKey();
  }
}
void ExprVisitor::VisitBindingBlock_(const DataflowBlockNode* block) {
  for (Binding binding : block->bindings) {
    this->VisitBinding(binding);
  }
}
void ExprVisitor::VisitBinding(const Binding& binding) {
  if (const auto* node = binding.as<VarBindingNode>()) {
    VisitBinding_(node);
  } else if (const auto* node = binding.as<MatchCastNode>()) {
    VisitBinding_(node);
  } else {
    LOG(FATAL) << "TypeError: Invalid type: " << binding->GetTypeKey();
  }
}

最终执行如下函数

void VisitBinding_(const VarBindingNode* binding) final {
    IndexedForwardGraph::Node* node = CreateNode(binding->var.get());

    // If the variable is not a dataflow variable, it must be the output variable of this dataflow
    // block
    if (!binding->var->IsInstance<DataflowVarNode>()) {
      this->MarkAsExternRef(node);
    }
    if (const auto* call = binding->value.as<CallNode>()) {
      // Case 1. The expression is a CallNode
      VisitCall(call, node);
    } else if (const auto* tuple_get_item = binding->value.as<TupleGetItemNode>()) {
      // Case 2. The expression is a TupleGetItemNode
      VisitTupleGetItem(tuple_get_item, node);
    } else {
      VisitUnsupportedNode(binding->value, node);
      // Case 3. The type of the expression is not fusion-supported.
      // In this case, we skip adding edges, adding an empty node into graph.
    }
    AddToPostDFSOrder(node, binding->var.get());
  }

binding内容如下：

源代码：
    lv = R.call_tir(cls.add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
｜        
VarBindingNode* binding：
    Visit VarBinding(var=lv, value=R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32")))

逻辑：

构建lv的图节点
因为这里是函数调用：R.call_tir，所以走 if 分支

void VisitCall(const CallNode* call, IndexedForwardGraph::Node* binding_var_node) {
    ICHECK_NOTNULL(binding_var_node);

    static const Op& call_tir_op_ = Op::Get("relax.call_tir");
    static const Op& call_tir_inplace_op_ = Op::Get("relax.call_tir_inplace");

    OpPatternKind pattern = OpPatternKind::kOpaque;
    ffi::Array<Expr> args = call->args;

    // - If the op being called is a TIR PrimFunc, we get the function op pattern directly from the
    // function attribute and visit the arguments one by one.
    // - Otherwise, the pattern of the current binding variable node is set to `kOpaque`, and we
    // recurse into the call expression.
    const auto* op = call->op.as<OpNode>();
    if (op == call_tir_op_.get() || op == call_tir_inplace_op_.get()) {
      const GlobalVar& global_var = Downcast<GlobalVar>(call->args[0]);
      tir::PrimFunc func = Downcast<tir::PrimFunc>(mod_->Lookup(global_var));

      // Override args for call_tir
      args = Downcast<Tuple>(call->args[1])->fields;

      ffi::Optional<Integer> opt_pattern = func->GetAttr<Integer>("op_pattern");
      if (opt_pattern.defined()) {
        pattern = static_cast<OpPatternKind>(Downcast<IntImm>(opt_pattern)->value);
      } else {
        pattern = OpPatternKind::kOpaque;
      }
    }
    // The pattern of the current binding variable node is set to the pattern of this operator.
    SetNodePattern(binding_var_node, pattern);
    // Visit all call args
    for (const Expr& arg : args) {
      ICHECK(IsLeafOrTuple(arg))
          << "FuseOps expects all relax::Call nodes to have non-nested arguments, "
          << "but " << ffi::GetRef<Expr>(call) << " has argument " << arg
          << ", which is neither a leaf node nor a relax::Tuple";
      VisitLeaf(arg, binding_var_node, pattern);
    }
  }

call内容如下

1	VisitCall(op=relax.call_tir, nargs=2)

逻辑：

这里的方法是call_tir，我们直接从函数属性中获取函数的算子模式，并逐一访问各个参数。
否则，当前绑定变量节点的模式被设置为 kOpaque（不透明），然后我们递归进入该调用表达式
最后调用VisitLeaf

void VisitLeaf(const Expr& leaf_expr, IndexedForwardGraph::Node* binding_var_node,
const OpPatternKind& pattern) {
    ICHECK_NOTNULL(binding_var_node);

    // Recursive visit if it's Tuple
    if (const auto* tuple = leaf_expr.as<TupleNode>()) {
        for (const Expr& expr : tuple->fields) {
            VisitLeaf(expr, binding_var_node, pattern);
        }
        return;
    }

    if (!leaf_expr->IsInstance<LeafExprNode>()) {
        // Skip GlobalVar, ExternFunc, OpNode.
        return;
    }

    auto it = graph_.node_map.find(leaf_expr.get());
    IndexedForwardGraph::Node* leaf_node = nullptr;
    if (it != graph_.node_map.end()) {
        leaf_node = it->second;
    } else {
        leaf_node = CreateNode(leaf_expr.get());
        // Since we never fuse constants, the pattern of the constant is set to `kOpaque`.
        SetNodePattern(leaf_node, OpPatternKind::kOpaque);
        AddToPostDFSOrder(leaf_node, leaf_expr.get());
    }
    AddEdge(leaf_node, binding_var_node, pattern);
}

这里的leaf_expr内容如下

1 2	VisitLeaf(edge_pattern=kElemWise, leaf=x) VisitLeaf(edge_pattern=kElemWise, leaf=w)

逻辑：

创建图节点（叶节点）
设置节点模式
把节点加入DFS序列
加入边（x—>lv w---->lv）

一轮执行完毕

下一轮的BindingBlock内容如下

源代码：
    lv1 = R.call_tir(cls.exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
｜        
VarBindingNode* binding：
    Visit VarBinding(var=lv1, value=R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32")))

接着依次走

1	VisitBinding_-->VisitCall-->VisitLeaf

完整的流程

Create graph
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:177: [Relax.FuseOps][GraphCreator] Visit GlobalVar=main
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:198: [Relax.FuseOps][GraphCreator] Visit Function in main (params=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv, value=R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=x)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=w)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv1, value=R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv2, value=R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv3, value=R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv1)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=gv, value=lv3)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:332: [Relax.FuseOps][GraphCreator] VisitUnsupportedNode(expr=lv3)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kOpaque, leaf=lv3)

最终的图结构如下

flowchart LR
  x["node0: x"] --> lv["node2: lv = add(x,w)"]
  w["node1: w"] --> lv

  lv --> lv1["node3: lv1 = exp(lv)"]
  lv --> lv2["node4: lv2 = tanh(lv)"]

  lv1 --> lv3["node5: lv3 = add(lv1,lv2)"]
  lv2 --> lv3

  lv3 --> gv["node6: gv (output)"]

2.2 Partition the graph by applying the fusion algorithm

std::vector<GraphPartitioner::Group*> groups =
      GraphPartitioner(&arena, opt_level, max_fuse_depth, /*max_function_args=*/0).Partition(graph);

std::vector<GraphPartitioner::Group*> GraphPartitioner::Partition(
    const IndexedForwardGraph& graph) {
  this->InitGroups(graph);
  if (opt_level_ == 0) return std::move(groups_);
  // get post dominator tree
  auto post_dom_tree = DominatorTree::PostDom(arena_, graph);
  // run fusion algorithm.
  for (int phase = 0; phase < 3; ++phase) {
    this->RunFuse(graph, post_dom_tree, phase);
  }
  return std::move(groups_);
}

总体分为三步：

初始化组
构建支配树
分三阶段进行融合

2.2.1 树DS解析

/*!
 * \brief Dominator tree that represent domination or
 *  post domination relation of the node.
 */
class DominatorTree {
 public:
  /*!
   * \brief A node in the dominator tree.
   */
  struct Node {
    /*! \brief The node in the tree */
    IndexedForwardGraph::Node* gnode{nullptr};
    /*! \brief parent of the tree */
    Node* parent{nullptr};
    /*! \brief current depth*/
    int depth{0};
    /*! \brief aggregated pattern to parent */
    OpPatternKind pattern{kOpaque};
  };
  // index -> node.
  std::vector<Node*> nodes;
 	...
};

此处定义的支配树包括了index到节点的映射，节点包括以下字段，填充这些数据结构即完成了Graph -> DominatorTree数据结构的转换

gnode：相对Graph的节点引用
parent：父节点
depth：深度，方便计算LCA
pattern：算子类型

2.2.2 支配树介绍

什么是支配树？对于一张有向图(可以有环)我们规定一个起点r, 从r点到图上另一个点w可能存在很多条路径(下面将r到w简写为r→w). 如果对于r→w的任意一条路径中都存在一个点p, 那么我们称点p为w的支配点(也可以称作是r→w的必经点), 注意r点不讨论支配点. 下面用idom[u]表示离点u最近的支配点. 对于原图上除r外每一个点u, 从idom[u]向u建一条边, 最后我们可以得到一个以r为根的树. 这个树我们就叫它"支配树".

如何构建？考虑下面两种DS

树对于一棵树, 我们用r表示根节点, u表示树上的某个非根节点. 很容易发现从r→u路径上的所有点都是支配点, 而idom[u]就是u的父节点. 这个可以在O(n)的时间内实现.
DAG(有向无环图) 因为是有向无环图, 所以我们可以按照拓扑序构建支配树. 假设当前我们构造到拓扑序中第x个节点编号为u, 那么拓扑序中第1 ~ X−1个节点已经处理好了, 考虑所有能够直接到达点u的节点, 对于这些节点我们求出它们在支配树上的最近公共祖先v, 这个点v就是点u在支配树上的父亲. 如果使用倍增求LCA, 这个问题可以在O((n+m)logn)的时间内实现.

2.2.3 支配树构建流程

auto post_dom_tree = DominatorTree::PostDom(arena_, graph);
DominatorTree DominatorTree::PostDom(support::Arena* arena, const IndexedForwardGraph& graph) {
  DominatorTree tree;
  tree.nodes.resize(graph.post_dfs_order.size(), nullptr);
  // reverse topo order
  for (size_t i = graph.post_dfs_order.size(); i != 0; --i) {
    size_t index = i - 1;
    tree.nodes[index] = tree.GetNode(arena, graph.post_dfs_order[index]);
  }
  return tree;
}

根据逆向拓扑序依次处理graph中的节点。

DominatorTree::Node* DominatorTree::GetNode(support::Arena* arena,
                                            IndexedForwardGraph::Node* gnode) {
  Node* tnode = arena->make<Node>();
  tnode->gnode = gnode;
  if (gnode->extern_ref) {
    tnode->depth = 1;
    tnode->parent = nullptr;
    tnode->pattern = kOpaque;
  } else {
    // find the LCAs of all outputs.
    OpPatternKind pattern = kElemWise;
    Node* parent = LeastCommonAncestor(gnode->outputs, &pattern);
    tnode->depth = parent ? parent->depth + 1 : 1;
    tnode->parent = parent;
    tnode->pattern = pattern;
  }
  return tnode;
}

逻辑

如果是节点是外部引用，则为根结节点
否则，会进入LeastCommonAncestor逻辑，其中，CombinePattern的处理逻辑：返回两个算子类型中更不容易融合的类型

DominatorTree::Node* DominatorTree::LeastCommonAncestor(Node* lhs, Node* rhs,
                                                        OpPatternKind* edge_pattern) {
  while (lhs != rhs) {
    if (lhs == nullptr) return nullptr;
    if (rhs == nullptr) return nullptr;
    if (lhs->depth < rhs->depth) {
      edge_pattern[0] = CombinePattern(edge_pattern[0], rhs->pattern);
      rhs = rhs->parent;
    } else if (rhs->depth < lhs->depth) {
      edge_pattern[0] = CombinePattern(edge_pattern[0], lhs->pattern);
      lhs = lhs->parent;
    } else {
      edge_pattern[0] = CombinePattern(edge_pattern[0], lhs->pattern);
      edge_pattern[0] = CombinePattern(edge_pattern[0], rhs->pattern);
      lhs = lhs->parent;
      rhs = rhs->parent;
    }
  }
  return lhs;
}

DominatorTree::Node* DominatorTree::LeastCommonAncestor(
    const LinkedList<IndexedForwardGraph::Edge>& input_nodes, OpPatternKind* edge_pattern) {
  auto link = input_nodes.head;
  if (link == nullptr) {
    return nullptr;
  }
  auto get_node = [&](const IndexedForwardGraph::Edge& edge) {
    size_t oindex = edge.node->index;
    ICHECK_LT(oindex, nodes.size());
    Node* onode = nodes[oindex];
    ICHECK(onode != nullptr);
    return onode;
  };
  Node* parent = get_node(link->value);
  *edge_pattern = CombinePattern(*edge_pattern, link->value.pattern);
  link = link->next;
  for (; link != nullptr; link = link->next) {
    parent = LeastCommonAncestor(parent, get_node(link->value), edge_pattern);
    *edge_pattern = CombinePattern(*edge_pattern, link->value.pattern);
  }
  return parent;
}

LCA逻辑：

获取该图节点的输出节点链表的头节点（第一个输出节点），并将对应树节点的父亲指向该输出图节点对应的树节点
get_node是根据输出边来获取对应的树节点
接着依次访问输出链表的剩下输出图节点，求当前的树节点father和当前输出图节点对应的树节点的LCA，并且更新father
LCA常见方法：朴素版、倍增、重链剖分等，这里采用的是朴素版本，两个节点按深度向上爬，直至相遇

最终构建的Dominator Tree如下：

flowchart TB
  gv["node6: gv (root)"] --> lv3["node5: lv3"]

  lv3 --> lv["node2: lv"]
  lv3 --> lv1["node3: lv1"]
  lv3 --> lv2["node4: lv2"]

  x["node0: x (root)"]
  w["node1: w (root)"]

2.2.4 图分区器DS解析

/*!
 * \brief A partition of the graph marked by union find data structure.
 */
class GraphPartitioner {
 public:
  explicit GraphPartitioner(support::Arena* arena, int opt_level, size_t max_fuse_depth,
                            size_t max_function_args)
      : arena_(arena),
        opt_level_(opt_level),
        max_fuse_depth_(max_fuse_depth),
        max_function_args_(max_function_args) {}
  /*!
   * \brief Group as a union find data structure.
   */
  struct Group {
    /*! \brief The parent in the union find data structure. */
    Group* parent{nullptr};
    /*! \brief The pattern of the group */
    OpPatternKind pattern;
    /*! \brief reference to the root node. */
    const tvm::Object* root_ref{nullptr};
    /*!
     * \brief Reference to the anchor node,
     * this field is not nullptr only if pattern is kOutEWiseFusable.
     */
    const tvm::Object* anchor_ref{nullptr};
    /*!
     * \brief The number of nodes belonging to this group
     */
    uint32_t num_nodes{1};
    /*!
     * \brief The number of function arguments belonging to this group
     */
    size_t args_num{0};

    /*! \brief Optional attributes to annotate the grouped function. */
    runtime::Map<runtime::String, ObjectRef> attrs;
    /*!
     * \brief Find the group root, perform path compression
     * \return The root type node.
     */
    Group* FindRoot();
  };
  /*!
   * \brief Partition a graph.
   * \return group assignments of each node.
   */
  std::vector<Group*> Partition(const IndexedForwardGraph& graph);

 private:
  /*! \brief The internal arena for temporary space. */
  support::Arena* arena_;
  /*! \brief optimization level for fuse operation. */
  int opt_level_;
  /*! \brief The maximum number of operations in one fused function */
  size_t max_fuse_depth_;
  /*! \brief The maximum number of arguments in one fused function */
  size_t max_function_args_;
  /*! \brief The internal groups. */
  std::vector<Group*> groups_;
  /*! \brief internal field used for deduplication */
  std::unordered_set<IndexedForwardGraph::Node*> visited_;
  /*! \brief The map with nodes which were postponed for fusing. */
  std::unordered_multimap<const IndexedForwardGraph::Node*, IndexedForwardGraph::Node*>
      postponed_fusing_map_;
	...

很明显，Group是一个并查集。

初始化：每个节点对应一个group，并根据Group信息填充其字段

void GraphPartitioner::InitGroups(const IndexedForwardGraph& graph) {
  auto args_counter = [this](const tvm::Object* obj) {
    size_t args_num = 0;
    if (auto call_node = GetRef<ObjectRef>(obj).as<CallNode>()) {
      for (auto& it : call_node->args) {
        if (it.as<VarNode>() || it.as<TupleGetItemNode>()) {
          args_num++;
          if (const auto* ttype = it.as<ExprNode>()->checked_type().as<TensorTypeNode>()) {
            args_num += CountAdditionalArgs_(ttype);
          }
        }
      }
    } else if (auto tuple_node = GetRef<ObjectRef>(obj).as<TupleNode>()) {
      for (auto& it : tuple_node->fields) {
        if (it.as<VarNode>() || it.as<TupleGetItemNode>()) {
          args_num++;
          if (const auto* ttype = it.as<ExprNode>()->checked_type().as<TensorTypeNode>()) {
            args_num += CountAdditionalArgs_(ttype);
          }
        }
      }
    } else if (GetRef<ObjectRef>(obj).as<VarNode>()) {
      args_num++;
      if (const auto* ttype =
              GetRef<ObjectRef>(obj).as<ExprNode>()->checked_type().as<TensorTypeNode>()) {
        args_num += CountAdditionalArgs_(ttype);
      }
    }
    return args_num;
  };

  groups_.resize(graph.post_dfs_order.size());
  for (size_t nid = 0; nid < groups_.size(); ++nid) {
    const auto* graph_node = graph.post_dfs_order[nid];
    auto* group_node = arena_->make<Group>();
    group_node->pattern = graph_node->pattern;
    group_node->root_ref = graph_node->ref;
    // set anchor ref if necessary.
    if (group_node->pattern == relay::kOutEWiseFusable) {
      group_node->anchor_ref = graph_node->ref;
    }
    group_node->args_num = args_counter(graph_node->ref);
    groups_[nid] = group_node;
  }
}

2.2.5 多阶段融合

std::vector<GraphPartitioner::Group*> GraphPartitioner::Partition(
    const IndexedForwardGraph& graph) {
  this->InitGroups(graph);
  if (opt_level_ == 0) return std::move(groups_);
  // get post dominator tree
  auto post_dom_tree = DominatorTree::PostDom(arena_, graph);
  // run fusion algorithm.
  for (int phase = 0; phase < 3; ++phase) {
    this->RunFuse(graph, post_dom_tree, phase);
  }
  return std::move(groups_);
}
void GraphPartitioner::RunFuse(const IndexedForwardGraph& graph,    //
                               const DominatorTree& post_dom_tree,  //
                               int phase) {
  for (size_t nid = 0; nid < groups_.size(); ++nid) {
    // the group of current node has been specified already.
    // 取graph_node, dom_node和group_node；
    auto* graph_node = graph.post_dfs_order[nid];
    auto* dom_node = post_dom_tree.nodes[nid];
    Group* group_node = groups_[nid];
    ICHECK(group_node != nullptr);
    postpone_node_ = nullptr;
    // Check if the fusing of some inputs was postponed
    if (postponed_fusing_map_.count(graph_node)) {
      auto range = postponed_fusing_map_.equal_range(graph_node);
      for (auto it = range.first; it != range.second; ++it) {
        // If the number of arguments is less than the limit then the input can be fused
        if (CountArgs_(graph_node, graph, false) <= CountArgsLimit_(graph_node)) {
          auto* src = it->second;
          auto* snode = post_dom_tree.nodes[src->index]->parent->gnode;
          if (groups_[snode->index]->anchor_ref != nullptr) continue;
          CommitFuse(src, snode);
        }
      }
      postponed_fusing_map_.erase(graph_node);
    }
    // no actions for opaque nodes
    // 遇到不可融合算子kOpaque，直接返回
    if (group_node->pattern == kOpaque) continue;
    // no actions needed if the current node have no dominator
    // 没有支配点信息的算子直接返回
    if (dom_node->parent == nullptr) continue;
    ICHECK(!graph_node->extern_ref);
    // 获取该节点后支配树节点的对应的图索引
    size_t dom_parent_gindex = dom_node->parent->gnode->index;

    // refuse the fusion if too many ops are going to be fused together
    if (CountFusedNodesWithNewChild(graph_node, dom_node->parent->gnode) > max_fuse_depth_)
      continue;
    // Refuse the fusion if too many arguments are going to be in the fused function
    if (max_function_args_ > 0) {
      auto limit = CountArgsLimit_(graph_node);
      if (limit > 0) {
        if (CountFusedArgs(graph, graph_node) > limit) {
          continue;
        }
      }
    }
    // 第三阶段处理逻辑
    if (phase == 2) {
      // Fuse injective ops into intermediate tuples, if any
      if (group_node->pattern > kInjective) continue;
      Group* dom_parent_group = groups_[dom_parent_gindex];
      Group* dom_root_group = dom_parent_group->FindRoot();
      // If dom node group has a tuple as its root, we do not fuse tuple fields into it
      if (dom_root_group->pattern == kTuple) continue;
      if (dom_parent_group->pattern == kTuple && dom_root_group->pattern <= kInjective) {
        // Now we know the tuple has been fused into subsequent injective ops
        auto fcond = [](OpPatternKind kind, bool is_sink) { return kind <= kInjective; };
        // dom_root_group can also be tuple, as in inception layers
        // CheckPath is needed to avoid fusing two intermediate tuples
        if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
          CommitFuse(graph_node, dom_node->parent->gnode);
        }
      }
      continue;
    }
    // 当前节点已和其后支配点融合，则跳过
    // Skip if current node is already fused to the parent.
    if (groups_[dom_parent_gindex] != nullptr &&
        group_node->FindRoot() == groups_[dom_parent_gindex]->FindRoot()) {
      continue;
    }
    // 跳过tuple相关操作
    // Do not fuse into tuple for now
    if (groups_[dom_parent_gindex]->pattern == kTuple) continue;
    // Try to fuse current node to its post-dominator.
    // 第一阶段处理kOutEltwiseFusable
    if (group_node->pattern == kOutEWiseFusable) {
      if (phase != 0) continue;
      // Path for OutEWiseFusable: conv2d
      // Check if the dominator relation is elemwise.
      if (dom_node->parent != nullptr && dom_node->pattern == kElemWise) {
        ICHECK(dom_node->parent->gnode != nullptr);
        // The fuse can be executed if all the intermediate ops are still broadcast.
        auto fcond = [](OpPatternKind kind, bool is_sink) { return kind <= kBroadcast; };
        if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
          CommitFuse(graph_node, dom_node->parent->gnode);
        }
      }
    }
    // 每一阶段都会对 kEltwise 或 kBroadcast 处理
    else if (group_node->pattern <= kBroadcast) {
      // Pre-condition: can only be fused to parent which is injective or reduction.
      if (dom_node->parent != nullptr &&
          (dom_node->pattern <= kInjective || dom_node->pattern == kCommReduce)) {
        // Check if all the intermediate ops are still broadcast.
        // The final terminal node can already be fused to a OutEWiseFusable group.
        auto fcond = [](OpPatternKind kind, bool is_sink) {
          if (!is_sink) {
            // Elemwise, broadcast, and injective ops on the parallel branches
            // are allowed be fused to the elemwise/broadcast anchor.
            return kind <= kInjective;
          } else {
            return (kind <= kBroadcast || kind == kCommReduce || kind == kInjective ||
                    kind == kOutEWiseFusable);
          }
        };
        if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
          CommitFuse(graph_node, dom_node->parent->gnode);
        }
      }
    } 
    // 第二阶段处理 kInjective 或 kTuple
    else if (group_node->pattern == kInjective || group_node->pattern == kTuple) {
      // defer injective fusion to second phase.
      // so conv2d always finishes fusing.
      if (phase != 1) continue;
      // Check if all path are injective.
      auto fcond = [](OpPatternKind kind, bool is_sink) { return kind <= kInjective; };
      if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
        CommitFuse(graph_node, dom_node->parent->gnode);
      }
    }
    // kCommReduce相关逻辑
    else {
      // do nothing.
      ICHECK(group_node->pattern == kCommReduce);
    }
  }
}

先看一些工具函数

CheckPath函数，主要用于遍历当前节点和其后支配节点之间的所有节点，并判断其是否满足给定的fcond：

template <typename F>
bool GraphPartitioner::CheckPath(IndexedForwardGraph::Node* src, IndexedForwardGraph::Node* sink,
                                 F fcond) {
  ICHECK(!src->extern_ref);
  visited_.clear();
  ICHECK(src != sink);
  for (auto link = src->outputs.head; link != nullptr; link = link->next) {
    if (!CheckPath_(link->value.node, sink, fcond)) return false;
  }
  return true;
}

如果CheckPath返回结果为True，一般会进行CommitFuse过程，其逻辑如下，CommitFuse_主要用于遍历过程，MergeFromTo用于更新Group的parent、pattern、num_nodes等字段；并查集的merge过程

void GraphPartitioner::MergeFromTo(Group* child, Group* parent) {
  child = child->FindRoot();
  parent = parent->FindRoot();
  if (child == parent) return;
  // update the number of nodes of the parent group
  parent->num_nodes += child->num_nodes;
  parent->args_num += child->args_num;
  child->parent = parent;
  // update anchor ref and pattern
  if (child->anchor_ref != nullptr) {
    ICHECK(parent->anchor_ref == nullptr);
    parent->anchor_ref = child->anchor_ref;
    parent->pattern = CombinePattern(child->pattern, parent->pattern);
  }
}

void GraphPartitioner::CommitFuse_(IndexedForwardGraph::Node* src, IndexedForwardGraph::Node* sink,
                                   Group* target) {
  if (postpone_node_ != nullptr) {
    postponed_fusing_map_.insert({postpone_node_, src});
    return;
  }
  if (src == sink) return;
  if (visited_.count(src)) return;
  visited_.insert(src);
  Group* gnode = groups_[src->index];
  ICHECK(gnode != nullptr);
  // merge the current group to the parent if possible.
  MergeFromTo(gnode, target);
  for (auto link = src->outputs.head; link != nullptr; link = link->next) {
    CommitFuse_(link->value.node, sink, target);
  }
}

void GraphPartitioner::CommitFuse(IndexedForwardGraph::Node* src, IndexedForwardGraph::Node* sink) {
  Group* target = groups_[sink->index];
  visited_.clear();
  ICHECK(src != sink);
  CommitFuse_(src, sink, target);
}

分阶段来看Runfuse

每一阶段都会处理kElemWise和kBroadcast：当前节点与其后支配点中的任意节点都满足patten<=kInjective且后支配点满足patten<=kOutEWiseFusable则可以融合；

else if (group_node->pattern <= kBroadcast) {
      // Pre-condition: can only be fused to parent which is injective or reduction.
      if (dom_node->parent != nullptr &&
          (dom_node->pattern <= kInjective || dom_node->pattern == kCommReduce)) {
        // Check if all the intermediate ops are still broadcast.
        // The final terminal node can already be fused to a OutEWiseFusable group.
        auto fcond = [](OpPatternKind kind, bool is_sink) {
          if (!is_sink) {
            // Elemwise, broadcast, and injective ops on the parallel branches
            // are allowed be fused to the elemwise/broadcast anchor.
            return kind <= kInjective;
          } else {
            return (kind <= kBroadcast || kind == kCommReduce || kind == kInjective ||
                    kind == kOutEWiseFusable);
          }
        };
        if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
          CommitFuse(graph_node, dom_node->parent->gnode);
        }
      }
    }

第一阶段处理了kOutEWiseFusable：

// Try to fuse current node to its post-dominator.
    if (group_node->pattern == kOutEWiseFusable) {
      if (phase != 0) continue;
      // Path for OutEWiseFusable: conv2d
      // Check if the dominator relation is elemwise.
      if (dom_node->parent != nullptr && dom_node->pattern == kElemWise) {
        ICHECK(dom_node->parent->gnode != nullptr);
        // The fuse can be executed if all the intermediate ops are still broadcast.
        auto fcond = [](OpPatternKind kind, bool is_sink) { return kind <= kBroadcast; };
        if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
          CommitFuse(graph_node, dom_node->parent->gnode);
        }
      }
    }

当前节点为kOutEWiseFusable，后支配点为kElemWise，且两节点的路径中所有算子均满足patten<=kBroadcast则可以融合；

第二阶段处理了kInjective和kTuple：

else if (group_node->pattern == kInjective || group_node->pattern == kTuple) {
      // defer injective fusion to second phase.
      // so conv2d always finishes fusing.
      if (phase != 1) continue;
      // Check if all path are injective.
      auto fcond = [](OpPatternKind kind, bool is_sink) { return kind <= kInjective; };
      if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
        CommitFuse(graph_node, dom_node->parent->gnode);
      }
    }

当前节点为kInjective或kTuple且所有到后支配点路径的所有节点均满足patten <= kInjective，则可以融合；第三阶段尝试将patten<=kInjective的算子融入kTuple中：

if (phase == 2) {
    // Fuse injective ops into intermediate tuples, if any
    if (group_node->pattern > relay::kInjective) continue;
    Group* dom_parent_group = groups_[dom_parent_gindex];
    Group* dom_root_group = dom_parent_group->FindRoot();
    // If dom node group has a tuple as its root, we do not fuse tuple fields into it
    if (dom_root_group->pattern == relay::kTuple) continue;
    if (dom_parent_group->pattern == kTuple && dom_root_group->pattern <= relay::kInjective) {
      // Now we know the tuple has been fused into subsequent injective ops
      auto fcond = [](OpPatternKind kind, bool is_sink) { return kind <= kInjective; };
      // dom_root_group can also be tuple, as in inception layers
      // CheckPath is needed to avoid fusing two intermediate tuples
      if (CheckPath(graph_node, dom_node->parent->gnode, fcond)) {
        CommitFuse(graph_node, dom_node->parent->gnode);
      }
    }
    continue;
  }

当前节点满足pattern<=kInjective，后支配点满足pattern=kTuple，且后支配点所属组的父节点满足pattern<=kInjective，则可以融合其实经过第一阶段的处理，我们的示例已经被完全融合了。

融合后的Group如下

flowchart TB
  lv3["root: lv3"] --> lv["lv"]
  lv3 --> lv1["lv1"]
  lv3 --> lv2["lv2"]

  x["root: x"]
  w["root: w"]
  gv["root: gv"]

到目前为止，所有的fuse信息均已存在Group节点中，接下来只需根据这些信息更新IR表示即可。

2.3 Transform the IRModule by fusing the operators in accordance with the graph partition results

构造函数初始化obj2group

OperatorFusor(IRModule mod, const IndexedForwardGraph& graph, const std::vector<Group*>& groups,
          bool lift_constant = true, bool debug_dump = false)
      : OperatorFusor(mod, CreateGroupMap(graph, groups), lift_constant, debug_dump) {}

static GroupMap CreateGroupMap(const IndexedForwardGraph& graph,
                                 const std::vector<Group*>& groups) {
    GroupMap obj2group;
    for (int nid = 0; nid < static_cast<int>(graph.post_dfs_order.size()); ++nid) {
      Group* group_root = groups[nid]->FindRoot();
      ICHECK(group_root != nullptr);
      ICHECK(graph.post_dfs_order[nid]->ref != nullptr);
      obj2group[graph.post_dfs_order[nid]->ref] = group_root;
    }
    return obj2group;
  }

先看DS

class OperatorFusor : public ExprMutator {
public:
  using Group = GraphPartitioner::Group;
  using GroupMap = std::unordered_map<const Object*, Group*>;
private:
  /*! \brief The IRModule. */
  IRModule mod_;
  /*! \brief Internal arena. */
  support::Arena arena_;
  /*! \brief The group assignment map. */
  GroupMap obj2group_;
  /*! \brief Internal function information map. */
  std::unordered_map<Group*, FunctionCreator> group2func_;
  /*! \brief Record the index for TupleGetItem if the variable needs to be remapped to an output
   * tuple element after fusion. */
  std::unordered_map<const VarNode*, int> tuple_get_indices_;
  /*!
   * \brief A map from a group to its dependent groups, used to detect cyclic dependencies.
   * \note Use vector so we can be deterministic, there won't be a lot of dep groups so
   *       linear search is OK.
   */
  std::unordered_map<Group*, std::vector<Group*>> group_deps_;
  /*! \brief Whether or not to lift bound constants to parameters of the grouped function. */
  bool lift_constants_{true};
};

**IRModule mod_;**
含义: 保存当前的 IR 模块（包含所有的函数定义和全局变量）。
作用: 这是变异（Mutation）发生的上下文环境，融合后的新函数会被添加到这个模块中。
**support::Arena arena_;**
含义: 一个内存池（Arena Allocator）。
作用: 用于高效地分配内部使用的临时对象（如 Group 对象等）。使用 Arena 可以避免频繁的 **new**/**delete** 开销，并在处理结束后统一释放内存。
**GroupMap obj2group_;**
含义: 对象到组的映射表。
作用: 这是融合的“蓝图”。在遍历 AST 时，Mutator 会查询当前访问的算子节点是否存在于此表中。如果存在，说明它需要被融合。这个 Map 通常是由前置步骤（如 **GraphPartitioner**）生成的。
**std::unordered_map<Group\*, FunctionCreator> group2func_;**
含义: 组到函数创建器的映射。
作用: 每一个 **Group** 最终都会变成一个新的 **Function**。这个 Map 记录了每个组对应的函数构建状态或构建器（**FunctionCreator** 是一个辅助类，用于收集组内的节点并构造函数体）。
**std::unordered_map<const VarNode\*, int> tuple_get_indices_;**
含义: 变量节点到元组索引的映射。
作用: 当一个融合函数有多个输出时，它会返回一个 Tuple（元组）。
外部的原有变量（VarNode）需要被重映射为 **TupleGetItem(call_node, index)**。这个 Map 记录了原来的变量对应新函数返回值的第几个元素。
**std::unordered_map<Group\*, std::vector<Group\*>> group_deps_;**
含义: 组之间的依赖关系图。
作用: 用于检测循环依赖（Cyclic Dependencies）。
如果 Group A 的输出是 Group B 的输入，而 Group B 的输出又是 Group A 的输入，这不仅无法执行，还会导致编译器死循环。这个结构用于在融合前或融合中确保拓扑顺序的正确性。
**bool lift_constants_{true};**
含义: 配置项，是否提升常量。
作用:
如果为 **true**：被融合子图中的常量（如卷积的权重）会被提取出来，作为新生成的融合函数的参数传入。这样做有利于后端代码生成时的去重和管理。
如果为 **false**：常量会保留在函数体内部。

class FunctionCreator : public ExprMutator {
public:
/*! \brief The original bindings of the function */
  ffi::Array<Binding> bindings_;
  /*! \brief The parameters of the function */
  ffi::Array<Var> params_;
  /*! \brief The arguments to call the function on the caller side */
  ffi::Array<Expr> arguments_;
  /*! \brief The name for the fused function */
  ffi::String name_hint_ = "fused";
  /*! \brief The constructed Relax function */
  ffi::Optional<Function> function_ = std::nullopt;
private:
  /*! \brief The variables defined in this function */
  std::unordered_set<const VarNode*> defined_vars_;
  /*! \brief The number of parameters reserved for constants */
  int n_param_for_const_ = 0;
  /*! \brief The output vars */
  std::vector<const VarNode*> output_vars_;
  /*! \brief Whether or not to lift bound constants to parameters */
  bool lift_constant_;
  /*! \brief Whether to emit verbose visit logs */
  bool debug_dump_{false};
  /*! \brief Mapping from tuple parameter of the function to its position index */
  std::unordered_map<const ExprNode*, int> tuple_param_idx_;
  /*!
   * \brief Mapping from partially referenced tuple parameter to the list of
   * indices that the parameter is referred by TupleGetItem
   */
  std::unordered_map<const ExprNode*, std::vector<int>> partially_used_tuple_params_;
};

**bindings_**：保存函数体内的中间变量绑定（如 **let %x = ...**）。
**params_**：函数定义的形参，类型为 **Var**。
**arguments_**：调用该函数时实际传入的参数表达式（用于生成 Call 节点）。
**name_hint_**：给优化或调试用的函数名，默认叫 “fused” 很可能是用于算子融合（operator fusion）后的函数命名。
**function_**：最终构建出来的 **relax::Function** 节点，初始为空（**nullopt**），需由构建逻辑填充。
defined_vars_：记录当前正在构建的函数内部定义的变量。
n_param_for_const_：计数器。记录了有多少个常量被提升为了参数。这有助于在生成函数参数列表时，知道前 N 个参数其实是权重/常量。
output_vars_：记录函数的输出变量。
lift_constant_：一个开关。如果为 **true**，编译器会将原本在代码中硬编码的常量（如卷积的权重 Weights）提取出来，作为函数的参数传入。
tuple_param_idx_：记录作为参数传入的 Tuple 节点在参数列表中的索引位置。
partially_used_tuple_params_：记录那些“只被部分使用”的 Tuple 参数及其被访问的索引。

/*!
   * \brief The main transformation on the IRModule
   * \return The new IRModule after transformation
   */
  IRModule Transform(const ffi::Array<ffi::String>& entry_function_names = {}) {
    ffi::Array<GlobalVar> entry_functions;
    if (entry_function_names.empty()) {
      entry_functions = mod_->GetGlobalVars();
    } else {
      for (const auto& name : entry_function_names) {
        entry_functions.push_back(mod_->GetGlobalVar(name));
      }
    }
    for (const auto& gv : entry_functions) {
      const auto& func = mod_->Lookup(gv);
      // Only visit Relax functions with neither attr::kPrimitive nor
      // attr::kCodegen.
      if (func->IsInstance<relax::FunctionNode>() && !func->HasNonzeroAttr(attr::kPrimitive) &&
          !func->GetAttr<ffi::String>(attr::kCodegen).has_value()) {
        auto updated_func = Downcast<Function>(VisitExpr(func));
        builder_->UpdateFunction(gv, updated_func);
      }
    }
    return builder_->GetContextIRModule();
  }

逻辑：

如果调用时没有指定函数名（**entry_function_names** 为空），则默认处理模块 **mod_** 中的所有全局函数。
如果指定了列表，则只查找并处理列表中指定的函数。
排除了三类不应该被处理的函数：

非 Relax 函数 (**func->IsInstance<relax::FunctionNode>()**)：

只处理 Relax IR 的函数节点。

Primitive 函数 (**!func->HasNonzeroAttr(attr::kPrimitive)**)：

**kPrimitive** 属性通常标记该函数已经是一个被融合过的底层算子（Primitive Function）。
原因：我们不希望对已经是底层原语的函数再次进行高层融合，避免重复操作或破坏已生成的内核结构。

外部 Codegen 函数 (**!func->GetAttr... (attr::kCodegen).has_value()**)：

**kCodegen** 属性表示该函数已经被标记为要发送给外部编译器/后端（如 TensorRT, CUDA, CUTLASS 等）处理。
原因：既然已经决定交给外部专用编译器处理，TVM 通用的 **OperatorFusor** 就不应该再插手修改它。
**VisitExpr(func)**:
这是调用父类 **ExprMutator** 的核心方法。它会深入遍历函数体内部的表达式。
正是在这个过程中，**OperatorFusor** 会利用之前定义的 **obj2group_**（分组映射），将散落的算子重写为对新融合函数的调用。
**builder_->UpdateFunction(gv, updated_func)**:
使用 **BlockBuilder**（TVM Relax 构建 IR 的工具）将旧的函数替换为处理后的新函数（**updated_func**）。

ffi::Array<GlobalVar> entry_functions;
    if (entry_function_names.empty()) {
      entry_functions = mod_->GetGlobalVars();
    } else {
      for (const auto& name : entry_function_names) {
        entry_functions.push_back(mod_->GetGlobalVar(name));
      }
    }

日志输出如下，在处理main函数

1 2	Transform begin entry_function_names=0 Transform: VisitExpr on GlobalVar=main

下面这行代码很关键

1	auto updated_func = Downcast<Function>(VisitExpr(func));

这行代码执行前main函数的ir

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    with R.dataflow():
        lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv3 = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        gv: R.Tensor((10, 20), dtype="float32") = lv3
        R.output(gv)
    return gv

这行代码执行后main函数的ir

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    with R.dataflow():
        lv: R.Tensor((10, 20), dtype="float32") = fused_add_exp_tanh_add(x, w)
        gv: R.Tensor((10, 20), dtype="float32") = lv
        R.output(gv)
    return gv

父类调用

Expr ExprMutator::VisitExpr(const Expr& expr) {
  return builder_->Normalize(ExprFunctor::VisitExpr(expr));
}

virtual R VisitExpr(const Expr& n, Args... args) {
    ICHECK(n.defined()) << "Found null pointer node while traversing AST. The previous pass may "
                           "have generated invalid data.";
    static FType vtable = InitVTable();
    return vtable(n, this, std::forward<Args>(args)...);
  }
/**
会按运行时节点类型（TypeKey）分发到对应的 VisitExpr_(TNode*)。
这里传入的expr是func
*/

Expr ExprMutator::VisitExpr_(const FunctionNode* op) {
  tvm::ffi::Array<Var> params;
  bool all_params_unchanged = true;
  for (Var param : op->params) {
    Var new_param = this->VisitVarDef(param);
    params.push_back(new_param);
    if (!param.same_as(new_param)) {
      var_remap_[param->vid] = new_param;
      all_params_unchanged = false;
    }
  }

  Expr body = this->VisitWithNewScope(op->body, params);

  if (all_params_unchanged && body.same_as(op->body)) {
    // No changes to the function, return the original object
    return ffi::GetRef<Expr>(op);
  } else if (IsBaseOf(GetStructInfo(body), op->ret_struct_info)) {
    // If the function was mutated into a form that can no longer
    // propagate shape information all the way to the return value, we
    // may keep the return struct info.  This is only allowed when the
    // body produces a return value that is the same as, or more
    // specific than, the pre-mutation struct info.  For example, if
    // the previous return value was `TensorStructInfo(shape=[16,16])`
    // but the body only produced `TensorStructInfo(ndim=2)`, we can
    // keep the more specific information.
    return Function(params, body, op->ret_struct_info, op->is_pure, op->attrs);
  } else {
    // If the function was mutated such that the body produces an
    // output that is incompatible with the original return struct
    // info, the original return struct info should not be used.  For
    // example, if the previous return value was
    // `TensorStructInfo(shape=[16,16])`, but the new return value is
    // `TensorStructInfo(shape=[8,8])`.
    return Function(params, body, std::nullopt, op->is_pure, op->attrs);
  }
}


Expr ExprMutator::VisitWithNewScope(const Expr& expr, ffi::Optional<ffi::Array<Var>> params) {
  ICHECK(expr->IsInstance<SeqExprNode>())
      << "Normal form requires all new scope is stored as SeqExpr";

  PrimExpr constraint = Bool(true);
  if (params.defined()) {
    auto non_negative_expressions =
        CollectNonNegativeExpressions(TupleStructInfo(params.value().Map(GetStructInfo)));
    for (const auto& expr : non_negative_expressions) {
      constraint = constraint && (expr >= 0);
    }
  }

  builder_->BeginScope(params);
  // Outer scope only includes TIR variables that can be inferred from
  // the function parameters.
  With<arith::ConstraintContext> context(builder_->GetAnalyzer(), constraint);
  builder_->BeginInnerScope();
  // Inner scope also includes any TIR variables that are defined by
  // MatchCast nodes, and are internal to the scope.
  Expr ret = this->VisitExpr(expr);

  builder_->EndScope();

  // Normalization (and the resulting StructInfo inference) of the
  // expr occurs outside of the body's parameters, but inside the
  // function signature's scope.  This keeps variables that are
  // inferable based on the function signature, to allow callers to
  // propagate StructInfo across the function.
  ret = builder_->Normalize(ret);
  builder_->EndScope();
  return ret;
}
/**
先处理 params（VisitVarDef 等），然后关键是：
Expr body = this->VisitWithNewScope(op->body, params);
Relax 的 normal form 里 Function.body 通常是一个 SeqExpr（里面装着 dataflow blocks + 最后的返回值 expr）。
所以接下来会在新 scope 下访问 SeqExpr。
*/

Expr ExprMutator::VisitExpr_(const SeqExprNode* op) {
  bool all_blocks_unchanged = true;
  ffi::Array<BindingBlock> blocks;
  for (auto block : op->blocks) {
    BindingBlock new_block = this->VisitBindingBlock(block);
    if (!new_block->bindings.empty()) {
      blocks.push_back(new_block);
    }
    all_blocks_unchanged &= block.same_as(new_block);
  }

  builder_->BeginBindingBlock();
  Expr body = this->VisitExpr(op->body);
  BindingBlock prologue = builder_->EndBlock();
  if (!prologue->bindings.empty()) {
    blocks.push_back(prologue);
    all_blocks_unchanged = false;
  }

  if (all_blocks_unchanged && body.same_as(op->body) &&
      VisitAndCheckStructInfoFieldUnchanged(op->struct_info_)) {
    return ffi::GetRef<Expr>(op);
  } else {
    return SeqExpr(blocks, body);
  }
}
/**
核心循环：
for (auto block : op->blocks) { BindingBlock new_block = this->VisitBindingBlock(block); ... }
也就是说：SeqExpr 会逐个访问其中的 block。
*/

最终调用

BindingBlock VisitBindingBlock_(const DataflowBlockNode* block) final {
    group2func_.clear();

    // Step 1. Collect the bindings for each grouped function.
    CollectFuncBindings(block->bindings);

    // Step 2. Collect all group's boundary (i.e. the output vars for each group)
    CollectFuncBoundary(block->bindings);

    // Step 3. Create the grouped function for each group.
    for (auto& [g, creator] : group2func_) {
      creator.CreateFunction(g->attrs);
    }

    // Step 4. Start generating the new binding block.
    //  - For groups with single binding, we directly recurse into the binding and emit the new one.
    //  - For groups with multiple bindings, we emit the call to the grouped function only when
    //  visiting the last binding of the group, because only by doing this we don't break the
    //  dependencies among the bindings of different groups. And therefore, we will skip all but the
    //  last binding of the group.
    builder_->BeginDataflowBlock();

    // For each group, record which variables need to be remapped to the output of TupleGetItem.
    // Only relevant when the output of the grouped function is a tuple.
    std::unordered_map<Group*, std::vector<Var>> pending_tuple_get;

    // A grouped function which returns a tuple requires attaching TupleGetItem to each element and
    // remapping variables in earlier bindings appropriately. Thus, a binding whose value depends on
    // some elements of a tuple from other group's function must be emitted after a call to the
    // tuple-producing function is emitted and remapping is done.
    // To guarantee this, we process bindings in the order of the topological sort of the group
    // dependency relations.
    for (const auto& binding : TopoSortByGroupDep(block->bindings)) {
      // Case 1. If the binding is the only binding in its group, recurse into it and emit the
      // transformed binding as usual.
      Group* group = GetGroupFromBinding(binding);
      if (group->num_nodes == 1 && group->attrs.empty()) {
        VisitBinding(binding);
        continue;
      }

      const auto& it_creator = group2func_.find(group);
      ICHECK(it_creator != group2func_.end());
      const FunctionCreator& func_info = it_creator->second;

      if (!func_info.function_.defined()) {
        // The function is not created yet, so we skip the binding.
        continue;
      }
      const Function& func = func_info.function_.value();

      // If this binding belongs to a group whose output is a tuple, the original bound variable
      // needs to be remapped to the output of TupleGetItem after the corresponding tuple is
      // emitted.
      if (IsTupleOutput(func) && tuple_get_indices_.count(binding->var.get())) {
        if (!GetStructInfo(binding->var)->IsInstance<TupleStructInfoNode>() ||
            IsNestedTupleOutput(func)) {
          // When binding->var itself is a tuple, we do not need to remap this variable to the
          // output of TupleGetItem unless the output is a nested tuple.
          pending_tuple_get[group].push_back(binding->var);
        }
      }

      // Case 2. If the binding is not the last binding of the group, we skip it.
      if (!func_info.bindings_.back().same_as(binding)) {
        continue;
      }

      // Case 3. The binding is the last binding of the group.
      const auto* var_binding = binding.as<VarBindingNode>();
      ICHECK(var_binding != nullptr) << "The last binding of a group whose size is larger than 1 "
                                        "is supposed to be a variable binding";

      // Step a. Add the grouped function to the IRModule
      GlobalVar gv = builder_->AddFunction(func, func_info.name_hint_);

      // Step b. Create the call to the deduplicated function, and then emit the call.
      //  - If this binding is an output binding, emit an output variable.
      //  - Otherwise, emit a dataflow variable.
      Var new_var;
      Call call_to_emit = Call(gv, UpdateArgs(func_info.arguments_));

      if (var_binding->var->IsInstance<DataflowVarNode>()) {
        new_var = builder_->Emit(call_to_emit);
      } else {
        new_var = builder_->EmitOutput(call_to_emit);
      }

      // Step c. Update the mapping used for the remapping of the binding variables.
      if (IsTupleOutput(func) && !pending_tuple_get.empty()) {
        // If the output is a tuple, attach TupleGetItem to all tuple elements, and
        // remap variables approriately.
        // The variables that need to be remapped and the corresponding tuple indices are
        // available in pending_tuple_get and tuple_get_indices_ respectively.
        for (const auto& var : pending_tuple_get[group]) {
          auto tuple_get = TupleGetItem(new_var, tuple_get_indices_[var.get()]);
          var_remap_[var->vid] = builder_->Emit(tuple_get);
        }
      } else {
        var_remap_[var_binding->var->vid] = new_var;
      }
    }
    // Step 5. Finish the binding block generation.
    return builder_->EndBlock();
  }

逻辑：

group2func_.clear(); 每个 dataflow block 都独立处理，所以进 block 先清掉。
给每个group收集它的bindings
推导每个group的“边界输出/依赖关系”
真正把一个group变成relax::function
生成新的binding block

入参日志

1 2	VisitBindingBlock(bindings=5) lv lv1 lv2 lv3 gv

2.3.1 Collect the bindings for each grouped function.

void CollectFuncBindings(const ffi::Array<Binding>& bindings) {
    for (const Binding& binding : bindings) {
      // If the binding is the only binding in its group, there is no need to create a new function.
      Group* group = GetGroupFromBinding(binding);
      if (group->num_nodes == 1 && group->attrs.empty()) {
        continue;
      }
      // Add the binding to the grouped function it's in, and update the function information
      // accordingly.
      if (!group2func_.count(group)) {
        group2func_.emplace(group, lift_constants_);
      }
      group2func_.find(group)->second.AppendBinding(binding);
    }
  }
CollectFuncBindings bindings=5

逻辑：

遍历 block 的每个 Binding（一般是 VarBinding），对每个 binding：

通过 GetGroupFromBinding(binding) 找这个 binding（准确说是它 LHS var）属于哪个 group；
如果 group 只有 1 个节点且没 attrs（group->num_nodes == 1 && group->attrs.empty()），说明不需要融合成新函数，跳过；
否则为这个 group 创建/获取一个 FunctionCreator，然后 AppendBinding(binding) 把 binding 加入该 group 的 “待生成 fused function” 中。

2.3.1.1 binding查找对应的group

Group* GetGroupFromBinding(const Binding& binding) {
    Var var = binding->var;
    if (debug_dump_) {
      LOG(INFO) << "[Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=" << var->name_hint();
    }
    return GetGroupFromVar(var);
  }
Group* GetGroupFromVar(const Var& var) {
    const auto& it_group = obj2group_.find(var.get());
    ICHECK(it_group != obj2group_.end())
        << "Variable " << var << " could not be found in any group";
    Group* group = it_group->second;
    if (debug_dump_) {
      LOG(INFO) << "[Relax.FuseOps][OperatorFusor] GetGroupFromVar var=" << var->name_hint()
                << " group=" << group << " root=" << group->FindRoot();
    }
    return group->FindRoot();
  }

日志输出

1 2	GetGroupFromBinding var=lv GetGroupFromVar var=lv group=0x5579309d6398 root=0x5579309d6398

2.3.1.2 将同group的其他bindings加入group2func

void AppendBinding(const Binding& binding) {
    ICHECK(!function_.defined())
        << "The `function_` is supposed to be uncreated when adding bindings";

    if (debug_dump_) {
      if (const auto* var_binding = binding.as<VarBindingNode>()) {
        std::string rhs;
        if (const auto* call = var_binding->value.as<CallNode>()) {
          if (const auto* op_node = call->op.as<OpNode>()) {
            rhs = ffi::GetRef<Op>(op_node)->name;
          } else if (const auto* gv_node = call->op.as<GlobalVarNode>()) {
            rhs = std::string("GlobalVar(") + std::string(gv_node->name_hint) + ")";
          } else {
            rhs = call->op->GetTypeKey();
          }
          LOG(INFO) << "[Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var="
                    << var_binding->var->name_hint() << ", rhs=Call(" << rhs
                    << ") nargs=" << call->args.size() << ")";
        } else if (const auto* tgi = var_binding->value.as<TupleGetItemNode>()) {
          LOG(INFO) << "[Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var="
                    << var_binding->var->name_hint() << ", rhs=TupleGetItem(index="
                    << tgi->index << "))";
        } else {
          LOG(INFO) << "[Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var="
                    << var_binding->var->name_hint() << ", rhs="
                    << var_binding->value->GetTypeKey() << ")";
        }
      } else {
        LOG(INFO) << "[Relax.FuseOps][FunctionCreator] AppendBinding binding_type="
                  << binding->GetTypeKey();
      }
    }

    if (const auto* var_binding = binding.as<VarBindingNode>()) {
      if (const auto* call = var_binding->value.as<CallNode>()) {
        if (call->op == Op::Get("relax.call_tir") ||
            call->op == Op::Get("relax.call_tir_inplace")) {
          // Update the name of the function.
          name_hint_ = name_hint_ + "_" + Downcast<GlobalVar>(call->args[0])->name_hint;

          const Tuple& args = Downcast<Tuple>(call->args[1]);
          for (const Expr& arg : args->fields) {
            CheckDefAndUpdateParam(arg);
            ICHECK(GetStructInfoAs<TupleStructInfoNode>(arg) == nullptr);
          }
          // TODO(tvm-team): handle shape expr
        } else {
          if (call->op->IsInstance<OpNode>()) {
            name_hint_ = name_hint_ + "_" + Downcast<Op>(call->op)->name;
          } else if (call->op->IsInstance<GlobalVarNode>()) {
            std::string gvar_name = Downcast<GlobalVar>(call->op)->name_hint;
            if (auto pos = gvar_name.find("fused_"); pos == 0) {
              name_hint_ = name_hint_ + "_" + gvar_name.substr(std::string("fused_").size());
            } else {
              name_hint_ = name_hint_ + "_" + gvar_name;
            }
          }

          for (const Expr& arg : call->args) {
            if (auto tuple = arg.as<TupleNode>()) {
              for (const Expr& tup_arg : tuple->fields) {
                CheckDefAndUpdateParam(tup_arg);
                ICHECK(GetStructInfoAs<TupleStructInfoNode>(tup_arg) == nullptr);
              }
            } else {
              CheckDefAndUpdateParam(arg);
            }
            if (GetStructInfoAs<TupleStructInfoNode>(arg) != nullptr) {
              // The argument is fully referenced. Thus we remove it from the mapping.
              partially_used_tuple_params_.erase(arg.get());
            }
          }
        }
      } else if (var_binding->value.as<TupleGetItemNode>()) {
        const auto* tuple_item = var_binding->value.as<TupleGetItemNode>();
        CheckDefAndUpdateParam(tuple_item->tuple);

        if (partially_used_tuple_params_.find(tuple_item->tuple.get()) !=
            partially_used_tuple_params_.end()) {
          // Appending get-item index to the mapping.
          partially_used_tuple_params_[tuple_item->tuple.get()].push_back(tuple_item->index);
        }
      }

      // Mark the binding variable as defined.
      defined_vars_.insert(var_binding->var.get());
      // Set var as output true if the binding is not a dataflow variable
      if (!var_binding->var->IsInstance<DataflowVarNode>()) {
        AppendOutput(var_binding->var);
      }
    } else {
      // TODO(tvm-team): handle match_cast
    }
    bindings_.push_back(binding);
  }

职责：把“属于同一个 fusion group 的一个 binding”追加到正在构建的 fused 函数里，同时把这个 binding 依赖到的外部输入（本 group 未定义的 Var/常量等）收集成 fused 函数的 params_ / arguments_；并记录哪些 group 内定义的变量要作为 fused 函数输出。

这里指分析RHS是**relax.call_tir** / **relax.call_tir_inplace**

逻辑：

一旦 fused 函数已经 CreateFunction() 生成了，就不允许再 Append 新 binding（否则参数/输出分析都可能失效）
VarBinding 且 RHS 是 Call ，且是call_tir。我们的样例是lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype=“float32”))，RHS符合。
更新 fused 函数名：把被调用的 PrimFunc（用 GlobalVar 表示）拼到 name_hint_ 里，保证生成的 fused_xxx_yyy 名字更可读/更稳定。
收集参数：call_tir 的“数据输入 args”在 call->args[1]，而且是一个 Tuple，所以这里取 Tuple(fields)，对每个 field 调 CheckDefAndUpdateParam(arg)。这一步的关键效果：如果 arg 是“本 group 外部来的 Var/常量”，它会被加入 fused 函数的 arguments_（caller 侧实参）以及 params_（callee 侧形参）。CheckDefAndUpdateParam 的具体规则在后面 CreateFunction 阶段才会真正把 binding RHS 替换成新参数。具体分析在下面。
ICHECK(GetStructInfoAs(arg) == nullptr)：FuseOps 期望 call_tir 的每个输入是“非 tuple 的叶子”，避免嵌套 tuple 搞乱参数展开/模式判定。
收尾：记录定义域与输出
defined_vars_.insert(var_binding->var.get())：标记这个 LHS 变量是“本 group 内定义的”。这会影响后续 CheckDefAndUpdateParam 判断一个 Var 是否需要提升为参数。
如果 LHS 不是 DataflowVar（也就是块外可见的变量，通常是 dataflow block 的输出变量），就把它记为 fused 函数输出：AppendOutput(var_binding->var)。
最后：把 binding 原样放入 bindings_
bindings_.push_back(binding)：只是“记录顺序”，真正的 IR 重写（把 RHS 中的外部 Var/常量替换成新参数、emit output 等）是在 CreateFunction() 里做。

void CheckDefAndUpdateParam(const Expr& expr) {
    // If the expression has already served as an argument, no need to create another one for it.
    if (std::find(arguments_.begin(), arguments_.end(), expr) != arguments_.end()) {
      return;
    }

    // If the expression is not a variable or is a undefined variable, it should be populated as a
    // parameter of the relax function.
    const auto* var = expr.as<VarNode>();
    if ((var == nullptr || defined_vars_.count(var) == 0) &&
        (lift_constant_ || !expr->IsInstance<ConstantNode>())) {
      ffi::String name = var != nullptr
                             ? var->name_hint()
                             : ffi::String("param_" + std::to_string(n_param_for_const_++));
      StructInfo param_sinfo = GetStructInfo(expr);
      if (!IsInlinableConstants(expr)) {
        Var param(std::move(name), GetStructInfo(expr));
        arguments_.push_back(expr);
        params_.push_back(param);
      }

      // Mark the tuple parameter is partially referenced in the beginning.
      // We will remove it from the mapping once we find it is fully referenced.
      if (param_sinfo->IsInstance<TupleStructInfoNode>()) {
        partially_used_tuple_params_[expr.get()] = {};
        tuple_param_idx_[expr.get()] = static_cast<int>(arguments_.size()) - 1;
      }
    }
  }

作用：判断某个表达式 expr 是否“在当前 fused group 内已定义”，若不是，就把它提升为 fused 函数的一个参数（params_）以及对应的调用实参（arguments_），并顺带记录 tuple 参数是否“只被部分取用”。

逻辑：

如果 expr 已经在 arguments_ 里出现过，直接 return，避免重复创建参数。
决定“要不要提升为参数”（核心 if 条件）
先判断 expr 是否是 Var：const auto* var = expr.as<VarNode>();
触发提升的条件是：

expr 不是 Var（var == nullptr），例如 Constant / PrimValue / ShapeExpr 等；或
expr 是 Var，但它不在 defined_vars_ 中（defined_vars_.count(var) == 0），也就是“这个 Var 不是在本 fused group 里通过前面 Append 的 bindings 定义出来的”，属于外部输入；并且还要满足：
常量是否允许提升：lift_constant_ || !expr->IsInstance<ConstantNode>()。也就是说：如果 lift_constant_==false，那么 Constant 不会被提升为参数（可能直接内联到 fused 函数里）。

生成参数名、struct info，并决定是否真的 push 到 params_/arguments_
参数名规则：
如果是 Var：用原本的 var->name_hint()
如果不是 Var：用 param_0/param_1/...（n_param_for_const_++）
取 struct info：StructInfo param_sinfo = GetStructInfo(expr);
关键点：IsInlinableConstants(expr) 会阻止创建参数如果 expr 是可内联常量（例如无 free var 的 PrimValue/ShapeExpr 或它们的 tuple），就不会创建 param/arg；否则才会：
params_.push_back(param)（callee 形参）
arguments_.push_back(expr)（caller 实参）
tuple 参数“部分使用”跟踪

日志输出

CheckDefAndUpdateParam add param=x for expr_type=relax.expr.Var args=1 params=1
CheckDefAndUpdateParam add param=w for expr_type=relax.expr.Var args=2 params=2
size_t AppendOutput(const Var& var) {
    ICHECK(defined_vars_.count(var.get()));
    auto output_idx = GetOutputIndex(var);
    if (output_idx) {
      if (debug_dump_) {
        LOG(INFO) << "[Relax.FuseOps][FunctionCreator] AppendOutput var=" << var->name_hint()
                  << " already output, idx=" << *output_idx;
      }
      return *output_idx;
    }
    output_vars_.push_back(var.get());
    if (debug_dump_) {
      LOG(INFO) << "[Relax.FuseOps][FunctionCreator] AppendOutput var=" << var->name_hint()
                << " new idx=" << (output_vars_.size() - 1);
    }
    return output_vars_.size() - 1;
  }

整个阶段执行的日子输出如下

[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1126: [Relax.FuseOps][OperatorFusor] CollectFuncBindings bindings=5
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1138: [Relax.FuseOps][OperatorFusor]   new FunctionCreator for group=0x5579309d6398 num_nodes=4 attrs=0
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv, rhs=Call(relax.call_tir) nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:770: [Relax.FuseOps][FunctionCreator] CheckDefAndUpdateParam add param=x for expr_type=relax.expr.Var args=1 params=1
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:770: [Relax.FuseOps][FunctionCreator] CheckDefAndUpdateParam add param=w for expr_type=relax.expr.Var args=2 params=2
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv1
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv1, rhs=Call(relax.call_tir) nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv2
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv2, rhs=Call(relax.call_tir) nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv3
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv3, rhs=Call(relax.call_tir) nargs=2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=gv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=gv group=0x5579309d63d0 root=0x5579309d63d0
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:980: [Relax.FuseOps][OperatorFusor] After CollectFuncBindings: groups_to_fuse=1

2.3.2 Collect all group’s boundary (i.e. the output vars for each group)

void CollectFuncBoundary(const ffi::Array<Binding>& bindings) {
    if (debug_dump_) {
      LOG(INFO) << "[Relax.FuseOps][OperatorFusor] CollectFuncBoundary bindings=" << bindings.size();
    }
    for (const Binding& binding : bindings) {
      // Step 1. Get current binding's group
      Group* cur_group = GetGroupFromBinding(binding);

      // Step 2. Collect all used vars in the binding value and update bondary.
      // - If the var's group is same as the binding's, the var is defined in the same group
      // - If the var's group is different with the binding's, the var must be the output from
      //   another group. Mark it to be the group output.
      auto update_boundary = [this, binding, &cur_group](const Expr& e) {
        if (e->IsInstance<VarNode>()) {
          const Var& used_var = Downcast<Var>(e);
          Group* producer_group = GetGroupFromVar(used_var);
          // Only check those group defined before.
          // Skip the vars from input or groups with single binding.
          if (producer_group != cur_group) {
            for (Group* depgroup : group_deps_[producer_group]) {
              ICHECK(depgroup != cur_group)
                  << "A cyclic dependency detected between the groups " << binding->var->name_hint()
                  << " and " << used_var->name_hint() << " are in.";
            }
            group_deps_[cur_group].push_back(producer_group);

            if (debug_dump_) {
              LOG(INFO) << "[Relax.FuseOps][OperatorFusor]   boundary: consumer_var="
                        << binding->var->name_hint() << " depends_on producer_group="
                        << producer_group << " via used_var=" << used_var->name_hint();
            }
          }

          if (auto producer = group2func_.find(producer_group);
              producer_group != cur_group && producer != group2func_.end()) {
            auto output_index = producer->second.AppendOutput(used_var);
            tuple_get_indices_[used_var.get()] = output_index;

            if (debug_dump_) {
              LOG(INFO) << "[Relax.FuseOps][OperatorFusor]   boundary: mark tuple_get used_var="
                        << used_var->name_hint() << " output_index=" << output_index;
            }
          }
        }
      };

      if (const auto* var_binding = binding.as<VarBindingNode>()) {
        PostOrderVisit(var_binding->value, update_boundary);
      } else {
        const auto* match_cast = binding.as<MatchCastNode>();
        ICHECK_NOTNULL(match_cast);
        PostOrderVisit(match_cast->value, update_boundary);
      }
    }
  }

可以拆成两件事：

建 group 之间的依赖图 group_deps_（用于后面 TopoSortByGroupDep 保证“先算 producer 组，再算 consumer 组”）
决定“哪些 Var 必须作为 producer 组 fused 函数的输出”，并给这些输出分配 tuple 下标 tuple_get_indices_（用于后面把 call(fused) 的返回值用 TupleGetItem 拆开并 remap）

整体结构：遍历本 DataflowBlock 的每个 binding
for 循环逐个处理 binding。
cur_group = GetGroupFromBinding(binding)：拿到“当前 binding 的 LHS var 所属的 group”，
核心：update_boundary lambda（对 RHS 表达式里用到的每个 Var 做处理）
它只关心 VarNode：
used_var 是 RHS 里用到的某个 Var，producer_group = GetGroupFromVar(used_var) 是“定义 used_var 的 group”
Part A：建立 group 依赖（consumer 组依赖 producer 组）
条件：producer_group != cur_group，说明 RHS 用到了“别的组”产出的值，因此 consumer 组必须依赖 producer 组。
循环 ICHECK 用来检测环：它在把 producer_group 加入 cur_group 依赖前，会检查 producer_group 的依赖列表里不能已经出现 cur_group，否则形成 group 层面的环（后面 topo sort 会炸）。
记录依赖：group_deps_[cur_group].push_back(producer_group)
日志：
GetGroupFromVar var=x group=0x5579309d6280 root=0x5579309d6280
GetGroupFromVar var=lv group=0x5579309d6398 root=0x5579309d6398
boundary: consumer_var=lv depends_on producer_group=0x5579309d6280 via used_var=x
Part B：把跨组被用到的 used_var 标为 producer 组 fused 函数的输出，并分配 tuple 下标
if (producer_group != cur_group && producer != group2func_.end())：只对“确实要生成 fused 函数的 producer 组”做输出标记。原因是 group2func_ 里只存了那些“需要生成 grouped function 的组”（多节点或有 attrs），单节点 passthrough 组不会在 group2func_，也就不需要“给它的 fused 函数追加输出”。
output_index = producer->second.AppendOutput(used_var)：把 used_var 加入 producer 组 fused 函数的 output 列表，返回它在 outputs tuple 里的位置 index。见 fuse_ops.cc:1188。
tuple_get_indices_[used_var.get()] = output_index：记下“将来谁用到了 used_var，就应该从 producer 组 call 返回值里用 TupleGetItem(index) 取出来对应它”。见 fuse_ops.cc:1189。
日志：
boundary: mark tuple_get used_var=lv3 output_index=0
Part C：用 PostOrderVisit 扫 RHS，把所有用到的 Var 都喂给 update_boundary
VarBinding：对 var_binding->value 做 PostOrderVisit；MatchCast：对 match_cast->value 做 PostOrderVisit。见 fuse_ops.cc:1198-1206。
它和后面重写阶段怎么对上
group_deps_ 会被 TopoSortByGroupDep 用来按依赖顺序处理 bindings，见 fuse_ops.cc:1254-1302。
tuple_get_indices_ 会在 VisitBindingBlock_ 里影响 pending_tuple_get，并在 emit call 后用 TupleGetItem 做 var_remap_，见 fuse_ops.cc:1031-1040 和 fuse_ops.cc:1109-1120。

结束的日志

1	After CollectFuncBoundary: group_deps=4 tuple_get_indices=1

2.3.3 Create the grouped function for each group.

需要构造函数的组信息如下

CreateFunction for group=0x5579309d6398 num_nodes=4 attrs=0
void CreateFunction(ffi::Map<ffi::String, Any> group_attrs) {
    if (debug_dump_) {
      LOG(INFO) << "[Relax.FuseOps][FunctionCreator] CreateFunction begin name_hint="
                << name_hint_ << " bindings=" << bindings_.size() << " params=" << params_.size()
                << " outputs=" << output_vars_.size() << " group_attrs=" << group_attrs.size();
    }
    // Step 1. Start constructing a new dataflow block.
    builder_->BeginDataflowBlock();

    // Step 2. Handing partially used tuple parameters: replacing entire tuple
    // parameters with the parameters of its fields that are accessed in the
    // function.
    std::unordered_map<const ExprNode*, std::unordered_map<int, Var>> tuple_get_item_remap;
    for (auto& [tuple_arg, item_indices] : partially_used_tuple_params_) {
      ICHECK(!item_indices.empty());
      int param_idx = tuple_param_idx_[tuple_arg];
      Var param = params_[param_idx];
      ffi::String param_name = params_[param_idx]->name_hint();
      TupleStructInfo param_sinfo = Downcast<TupleStructInfo>(tuple_arg->struct_info_);

      ffi::Array<Expr> item_args;
      ffi::Array<Var> item_params;
      item_args.reserve(item_indices.size());
      item_params.reserve(item_indices.size());
      for (int item_idx : item_indices) {
        Var item_param(param_name + "_" + std::to_string(item_idx), param_sinfo->fields[item_idx]);
        item_args.push_back(TupleGetItem(ffi::GetRef<Expr>(tuple_arg), item_idx));
        item_params.push_back(item_param);
        tuple_get_item_remap[tuple_arg][item_idx] = item_param;
      }
      arguments_.erase(arguments_.begin() + param_idx);
      arguments_.insert(arguments_.begin() + param_idx, item_args.begin(), item_args.end());
      params_.erase(params_.begin() + param_idx);
      params_.insert(params_.begin() + param_idx, item_params.begin(), item_params.end());
    }

    // Step 3. Visit each binding and collect outputs one by one.
    ffi::Array<Expr> outputs(output_vars_.size(), Expr());
    for (const Binding& binding : bindings_) {
      // Special handing for TupleGetItem.
      if (const auto* var_binding = binding.as<VarBindingNode>()) {
        if (const auto* tuple_get_item = var_binding->value.as<TupleGetItemNode>()) {
          auto it = tuple_get_item_remap.find(tuple_get_item->tuple.get());
          if (it != tuple_get_item_remap.end()) {
            ICHECK(it->second.find(tuple_get_item->index) != it->second.end());
            var_remap_[var_binding->var->vid] = it->second[tuple_get_item->index];
            if (auto output_idx = GetOutputIndex(binding->var)) {
              outputs.Set(*output_idx, it->second[tuple_get_item->index]);
            }
            continue;
          }
        }
      }

      if (auto output_idx = GetOutputIndex(binding->var)) {
        // Case 1. It is an output binding
        // We only allow VarBinding as output.
        const auto* var_binding = binding.as<VarBindingNode>();
        ICHECK_NOTNULL(var_binding);
        Var output_var = builder_->EmitOutput(VisitExpr(var_binding->value));
        var_remap_[var_binding->var->vid] = output_var;
        outputs.Set(*output_idx, output_var);
      } else {
        // Case 2. It is an internal binding, add it to the binding list.
        VisitBinding(binding);
      }
    }

    // Step 4. Finish constructing the new block.
    BindingBlock new_block = builder_->EndBlock();
    if (outputs.empty()) {
      // If the result is not used outside
      LOG(WARNING) << "There are dead codes in the current IRModule, please run the "
                      "DeadCodeElimination Pass before FuseOps";
      function_ = std::nullopt;
    } else {
      Expr body = outputs.size() == 1 ? outputs[0] : Tuple(outputs);
      body = builder_->Normalize(body);
      body = builder_->Normalize(SeqExpr({new_block}, body));
      group_attrs.Set(tvm::relax::attr::kPrimitive, true);
      Function function = Function(/*params=*/params_,                //
                                   /*body=*/body,                     //
                                   /*ret_struct_info=*/std::nullopt,  //
                                   /*is_pure=*/true,                  //
                                   /*attrs=*/DictAttrs(group_attrs));
      ffi::Array<PrimExpr> free_vars =
          FreeSymbolicVars(function).Map([](const tir::Var& var) -> PrimExpr { return var; });
      if (!free_vars.empty()) {
        params_.push_back(Var("tir_vars", ShapeStructInfo(free_vars)));
        arguments_.push_back(ShapeExpr(free_vars));
        function = Function(/*params=*/params_,                //
                            /*body=*/body,                     //
                            /*ret_struct_info=*/std::nullopt,  //
                            /*is_pure=*/true,                  //
                            /*attrs=*/DictAttrs(group_attrs));
      }
      function_ = SymbolicVarRenewMutator::Renew(function);
    }

    if (debug_dump_) {
      LOG(INFO) << "[Relax.FuseOps][FunctionCreator] CreateFunction end name_hint=" << name_hint_
                << " function_defined=" << (function_.defined() ? 1 : 0)
                << " final_params=" << params_.size() << " final_args=" << arguments_.size();
    }
  }

CreateFunction 是把前面通过 AppendBinding(...) 累积起来的 bindings_ / params_ / arguments_ / output_vars_ 真正“落地”为一个新的 fused Relax Function 的地方；它同时还会做一次“参数/绑定重写”，把 fused 函数体里的外部输入替换成新形参，并把需要的输出用 EmitOutput 收集起来。

逻辑：

输入/前置状态（来自 AppendBinding 阶段）
bindings_：本 group 内所有被收集的 binding，按原顺序保存。
params_ / arguments_：fused 函数的形参列表 + caller 侧调用这个函数时要传的实参列表（两者位置一一对应）。
output_vars_：哪些“本 group 内定义的 Var”需要作为 fused 函数返回值（通常是跨组被用到的 var，或 dataflow block 输出 var）。
partially_used_tuple_params_ / tuple_param_idx_：记录某些 tuple 参数是否只被 TupleGetItem 用了部分字段，供后面拆参优化。
日志：
CreateFunction begin name_hint=fused_add_exp_tanh_add bindings=4 params=2 outputs=1 group_attrs=0
Step 1：开始构造新的 DataflowBlock
builder_->BeginDataflowBlock()。
之后对 fused 函数体里每个 binding 的 emit 都会发生在这个新 block 里。
Step 2：处理“tuple 参数只部分使用”的拆参（关键优化点）
遍历 partially_used_tuple_params_，对每个 tuple 参数 tuple_arg：
找到它在 params_ / arguments_ 里的位置 param_idx（用 tuple_param_idx_）。
把“整个 tuple 参数”替换成“若干个字段参数”：
新的 caller 实参是 TupleGetItem(tuple_arg, item_idx)；
新的 callee 形参是对应 struct info 的 Var item_param；
同时在 tuple_get_item_remap[tuple_arg][item_idx] = item_param 里记一份映射，后面重写 TupleGetItem binding 会用到。
最后用 erase/insert 在原位置把 arguments_ / params_ 替换成展开后的列表。
直观效果：如果 fused 函数只用到了 tuple 的第 0、2 个字段，它就只会生成两个形参，而不是把整个 tuple 传进去再在函数里取字段。
Step 3：逐个处理 bindings_，构造新的 block，并收集 outputs
先创建 outputs 数组，大小等于 output_vars_，见 fuse_ops.cc:686-687。
这一段有 3 个重要分支：

特殊处理：如果某个 binding 自身是 TupleGetItem，并且它的 tuple 参数被拆参了

当 var_binding->value 是 TupleGetItem(tuple, index)，且 tuple 在 tuple_get_item_remap 里：
说明这个 TupleGetItem 的结果其实就是“某个新形参”（字段形参），不需要再 emit 任何东西；
于是直接设置 var_remap_：把原 var remap 到对应 item_param；
如果这个 var 还是输出之一，则 outputs[output_idx] 直接填 item_param；
continue 跳过后续 emit。见 fuse_ops.cc:689-705。

输出 binding：需要把 RHS 变成输出 var

如果 binding->var 在 output_vars_ 里（GetOutputIndex(binding->var) 有值）：
计算 RHS：VisitExpr(var_binding->value)；
builder_->EmitOutput(…) 生成一个输出 var；
更新 var_remap_：原 var → 新输出 var；
outputs[output_idx] = output_var。见 fuse_ops.cc:710-718。

这里 VisitExpr 的语义很关键：FunctionCreator 覆盖了 VisitExpr，如果 RHS 某个子表达式等于 arguments_ 里的某项，就直接返回对应 params_[i]，从而把“外部输入”替换成“fused 函数参数”。见 fuse_ops.cc:791-811。

内部 binding：照常 emit 到 dataflow block

VisitBinding(binding)：会递归调用 VisitExpr 重写 RHS，并用 builder emit 新 binding。
Step 4：结束 block，并组装 Function
new_block = builder_->EndBlock() 结束 dataflow block。
如果 outputs 为空：认为这个 fused group 的结果没人用（死代码），function_ = std::nullopt 并提示应先跑 DCE。
否则：
body = 单输出则直接用该 Expr，否则 Tuple(outputs)；
Normalize + 包一层 SeqExpr({new_block}, body) 再 Normalize；
给 attrs 塞 attr::kPrimitive=true，表示这是 fused primitive 函数；
构造 Function(params_, body, …, attrs)

Expr VisitExpr(const Expr& expr) final {
    // If the expression serves as an argument, return its correspondng parameter.
    auto it = std::find(arguments_.begin(), arguments_.end(), expr);
    if (it != arguments_.end()) {
      if (debug_dump_) {
        LOG(INFO) << "[Relax.FuseOps][FunctionCreator] VisitExpr hit argument, arg=" << expr;
      }
      return params_[it - arguments_.begin()];
    }
    if (debug_dump_) {
      LOG(INFO) << "[Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=" << expr;
    }
    // Otherwise, recurse into this expression.
    return ExprMutator::VisitExpr(expr);
  }

在 CreateFunction() 阶段重写 RHS 时，把“caller 侧实参里出现过的外部输入 Expr”替换成“callee 侧对应形参 Var”，从而完成外部输入 → fused 函数参数的绑定。

在 AppendBinding/CheckDefAndUpdateParam 阶段，你把 group 外部的输入记录成了：

arguments_: Expr（调用 fused 函数时要传的实参，来自原 IR 的 Var/Constant/ShapeExpr…）
params_: Var（fused 函数的形参）

但是此时 bindings_ 里的 RHS 还是“原来的 Expr”。到了 CreateFunction() 的 Step 3，你要 emit 新的 binding（VisitBinding(binding)）或者 emit 输出（EmitOutput(VisitExpr(value))），这一步必须把 RHS 里对外部 Var/常量的引用换成对新形参的引用，否则 fused 函数体会引用到“函数外部的变量”，语义就错了。

这就是 FunctionCreator::VisitExpr 的使命。

逻辑：

分支A：命中“它就是一个参数实参”
如果当前正在访问的 expr，刚好等于 arguments_ 里的某个元素（注意：这里是按 Expr 结构/引用相等来匹配），那么说明它被提升为 fused 函数参数了。
于是返回对应位置的 params_[i]（一个 Var），完成替换。

这一步就是把：

RHS 里的 x（原函数里的 Var）替换成：
fused 函数形参 x（一个新的 Var，通常同名但不同 vid）
分支 B：没命中，就递归重写子表达式
对更复杂的表达式（比如 Call、Tuple、TupleGetItem 等），逐层往下走。
一旦走到叶子（Var/Constant/ShapeExpr 等），如果它在 arguments_ 里，就会触发分支 A，被替换成形参 Var。

CreateFunction() 里两处会触发这个 VisitExpr：

输出 binding：builder_->EmitOutput(VisitExpr(var_binding->value)) 见 fuse_ops.cc:715
内部 binding：VisitBinding(binding) 内部会访问 RHS，最终也会走到 VisitExpr 见 fuse_ops.cc:721。

日志里里的：

[FunctionCreator] VisitExpr hit argument, arg=… 就是某个 leaf expr 被替换为参数的那一刻。

整个VisitExpr日志如下

[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:620: [Relax.FuseOps][FunctionCreator] CreateFunction begin name_hint=fused_add_exp_tanh_add bindings=4 params=2 outputs=1 group_attrs=0
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("add")
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(x, w)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:790: [Relax.FuseOps][FunctionCreator] VisitExpr hit argument, arg=x
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:790: [Relax.FuseOps][FunctionCreator] VisitExpr hit argument, arg=w
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("exp")
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(lv,)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("tanh")
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(lv,)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("add")
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(lv1, lv2)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv1
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv2
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])

结束日志如下

1	CreateFunction end name_hint=fused_add_exp_tanh_add function_defined=1 final_params=2 final_args=2

2.3.4 Start generating the new binding block.

for (const auto& binding : TopoSortByGroupDep(block->bindings)) {
      if (debug_dump_) {
        LOG(INFO) << "[Relax.FuseOps][OperatorFusor] Process binding var="
                  << binding->var->name_hint();
      }
      // Case 1. If the binding is the only binding in its group, recurse into it and emit the
      // transformed binding as usual.
      Group* group = GetGroupFromBinding(binding);
      if (group->num_nodes == 1 && group->attrs.empty()) {
        if (debug_dump_) {
          LOG(INFO) << "[Relax.FuseOps][OperatorFusor]   single-node group, passthrough var="
                    << binding->var->name_hint();
        }
        VisitBinding(binding);
        continue;
      }

      const auto& it_creator = group2func_.find(group);
      ICHECK(it_creator != group2func_.end());
      const FunctionCreator& func_info = it_creator->second;

      if (!func_info.function_.defined()) {
        // The function is not created yet, so we skip the binding.
        if (debug_dump_) {
          LOG(INFO) << "[Relax.FuseOps][OperatorFusor]   skip: grouped function not created yet";
        }
        continue;
      }
      const Function& func = func_info.function_.value();

      // If this binding belongs to a group whose output is a tuple, the original bound variable
      // needs to be remapped to the output of TupleGetItem after the corresponding tuple is
      // emitted.
      if (IsTupleOutput(func) && tuple_get_indices_.count(binding->var.get())) {
        if (!GetStructInfo(binding->var)->IsInstance<TupleStructInfoNode>() ||
            IsNestedTupleOutput(func)) {
          // When binding->var itself is a tuple, we do not need to remap this variable to the
          // output of TupleGetItem unless the output is a nested tuple.
          pending_tuple_get[group].push_back(binding->var);
        }
      }

      // Case 2. If the binding is not the last binding of the group, we skip it.
      if (!func_info.bindings_.back().same_as(binding)) {
        if (debug_dump_) {
          LOG(INFO) << "[Relax.FuseOps][OperatorFusor]   skip: not last binding of group";
        }
        continue;
      }

      // Case 3. The binding is the last binding of the group.
      const auto* var_binding = binding.as<VarBindingNode>();
      ICHECK(var_binding != nullptr) << "The last binding of a group whose size is larger than 1 "
                                        "is supposed to be a variable binding";

      // Step a. Add the grouped function to the IRModule
      GlobalVar gv = builder_->AddFunction(func, func_info.name_hint_);

      if (debug_dump_) {
        LOG(INFO) << "[Relax.FuseOps][OperatorFusor]   AddFunction fused GlobalVar="
                  << gv->name_hint << " (name_hint=" << func_info.name_hint_ << ")";
      }

      // Step b. Create the call to the deduplicated function, and then emit the call.
      //  - If this binding is an output binding, emit an output variable.
      //  - Otherwise, emit a dataflow variable.
      Var new_var;
      Call call_to_emit = Call(gv, UpdateArgs(func_info.arguments_));

      if (var_binding->var->IsInstance<DataflowVarNode>()) {
        new_var = builder_->Emit(call_to_emit);
      } else {
        new_var = builder_->EmitOutput(call_to_emit);
      }

      if (debug_dump_) {
        LOG(INFO) << "[Relax.FuseOps][OperatorFusor]   Emit call result var=" << new_var->name_hint()
                  << " (dataflow=" << (var_binding->var->IsInstance<DataflowVarNode>() ? 1 : 0)
                  << ")";
      }

      // Step c. Update the mapping used for the remapping of the binding variables.
      if (IsTupleOutput(func) && !pending_tuple_get.empty()) {
        // If the output is a tuple, attach TupleGetItem to all tuple elements, and
        // remap variables approriately.
        // The variables that need to be remapped and the corresponding tuple indices are
        // available in pending_tuple_get and tuple_get_indices_ respectively.
        for (const auto& var : pending_tuple_get[group]) {
          auto tuple_get = TupleGetItem(new_var, tuple_get_indices_[var.get()]);
          var_remap_[var->vid] = builder_->Emit(tuple_get);
        }
      } else {
        var_remap_[var_binding->var->vid] = new_var;
      }
    }

这段逻辑在做 “把一个 group（融合子图）里的一串 bindings，用一次对 fused function 的调用替换掉”，并且处理“fused function 返回 tuple 时，老变量要用 TupleGetItem 去接回来的 remap”。

代码位置： fuse_ops.cc:1019-1113

整体循环（按 group 依赖拓扑序处理 bindings）

for (binding : TopoSortByGroupDep(block->bindings))：不是按原始 binding 顺序，而是按 group_deps_ 的拓扑序。目的：如果某个 group 的 fused call 会返回 tuple，并且后续别的 group 需要用其中某个元素，那么必须先“emit fused call + emit TupleGetItem + 做 remap”，再处理消费者 group，否则消费者在 VisitExpr/UpdateArgs 时会找不到对应的 remap。

Case 1：单节点 group 直接透传

if (group->num_nodes == 1 && group->attrs.empty()) { VisitBinding(binding); continue; }
含义：没融合价值（只有一个节点且无 attrs），就走普通 ExprMutator 的逻辑，维持原 binding，只做递归改写（比如把用到的 var 替换成已经 remap 的新 var）。

取出该 group 的 FunctionCreator，并跳过“函数尚未创建”的情况

const FunctionCreator& func_info = group2func_[group];
if (!func_info.function_.defined()) continue;
背景：前面 Step 3 会对 group2func_ 逐个 CreateFunction，正常来说这里一般都应该 defined；但如果 CreateFunction 因为 dead code（outputs.empty()）把 function_ 置空，这里会跳过该 group 的 bindings（相当于不替换，依赖后续 DCE 等处理）。

tuple 输出的“延迟 remap”登记（pending_tuple_get）

条件：if (IsTupleOutput(func) && tuple_get_indices_.count(binding->var.get()))
IsTupleOutput(func)：这个 fused function 的返回 StructInfo 是 tuple。
tuple_get_indices_：在 CollectFuncBoundary 里建立的映射：某个“会被跨 group 使用的 producer var”在 producer 的返回 tuple 里的 index。
内层判断：
if (!GetStructInfo(binding->var)->IsInstance<TupleStructInfoNode>() || IsNestedTupleOutput(func))
含义：如果 binding->var 自身不是 tuple（通常是 tuple 的某个元素），那必须 remap 成 TupleGetItem(new_tuple, idx)；如果 binding->var 自身就是 tuple，那么只有在“nested tuple 输出”时才需要再 TupleGetItem（否则直接把它 remap 成整个 tuple 值即可）。
动作：pending_tuple_get[group].push_back(binding->var);
注意这里即便后面会 continue（跳过非 last binding），也先把“这个 var 将来要靠 TupleGetItem 接回来”登记起来，等到遇到该 group 的 last binding（也就是 emit fused call 的地方）再统一生成这些 TupleGetItem。

Case 2：不是 group 的最后一个 binding 就跳过

if (!func_info.bindings_.back().same_as(binding)) continue;
核心点：对一个 group，只在最后一个 binding 处真正 Emit(call fused_fn) 一次；其他 bindings 都被“融合掉了”，不再逐条 emit。这样做的原因是要保持跨 group 的依赖顺序：你提前 emit 这个 group 的 call，可能会打乱原来的 dataflow 依赖/定义顺序。

Case 3：遇到该 group 的最后一个 binding，真正执行替换

Step a：GlobalVar gv = builder_->AddFunction(func, func_info.name_hint_)
把刚构造好的 fused function 放进 IRModule，拿到一个 GlobalVar 名字（用于后续 call）。
Step b：构造并 emit 调用
Call call_to_emit = Call(gv, UpdateArgs(func_info.arguments_));
func_info.arguments_ 是 FunctionCreator 记录的“caller-side 实参列表”（来自原始 expr 的叶子输入、跨组输入等）。
UpdateArgs(...) 会对每个 arg 再做一次 VisitExpr，把已经存在的 var_remap_ 应用上（确保实参用的是“当前 block 里已 emit 的新 var”）。
根据原 var 是否 dataflow var 决定 Emit vs EmitOutput：保持 block 的 output 语义不变。
Step c：更新 var_remap_（这是后续 VisitExpr 重写变量引用的关键表）
非 tuple 返回：var_remap_[old_var_vid] = new_var;
tuple 返回：对 pending_tuple_get[group] 里的每个旧 var，生成
tuple_get = TupleGetItem(new_var, tuple_get_indices_[old_var])
var_remap_[old_var_vid] = builder_->Emit(tuple_get)
直观效果：原来用到 old_var 的地方，之后都会被改写成用 TupleGetItem 拿到的那个新 SSA var。

一个值得注意的细节（可能是潜在坑）

这里的判断是 if (IsTupleOutput(func) && !pending_tuple_get.empty())，并且在该分支里只给 pending_tuple_get[group] 里的 vars 建 remap；否则才会给 var_binding->var->vid 建 remap。
如果某个 group 的输出是 tuple，但“当前这个 group 并没有需要 TupleGetItem 的元素要 remap”（例如只需要整个 tuple，或只 remap 发生在别的 group），那么这段逻辑可能不会给 var_binding->var->vid 设置 var_remap_（取决于 pending_tuple_get 的全局是否为空以及当前 group 的 vector 是否为空），后续引用可能会缺映射。更稳妥的写法通常是：先默认 var_remap_[last_binding_var] = new_var，再额外为 tuple 元素补充 TupleGetItem 的 remap。

日志如下：

[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1255: [Relax.FuseOps][OperatorFusor] TopoSortByGroupDep bindings=5
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv1
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv2
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv3
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=gv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=gv group=0x5579309d63d0 root=0x5579309d63d0
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1290: [Relax.FuseOps][OperatorFusor] TopoSortByGroupDep sorted=5
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1064: [Relax.FuseOps][OperatorFusor]   skip: not last binding of group
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv1
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv1
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1064: [Relax.FuseOps][OperatorFusor]   skip: not last binding of group
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv2
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv2
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1064: [Relax.FuseOps][OperatorFusor]   skip: not last binding of group
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv3
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv3
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x5579309d6398 root=0x5579309d6398
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1078: [Relax.FuseOps][OperatorFusor]   AddFunction fused GlobalVar=fused_add_exp_tanh_add (name_hint=fused_add_exp_tanh_add)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1241: [Relax.FuseOps][OperatorFusor] UpdateArgs nargs=2
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1095: [Relax.FuseOps][OperatorFusor]   Emit call result var=lv (dataflow=1)
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=gv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=gv
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=gv group=0x5579309d63d0 root=0x5579309d63d0
[16:52:12] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1029: [Relax.FuseOps][OperatorFusor]   single-node group, passthrough var=gv

最后回到transfomr函数里，执行完毕。

附录

[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1327: [Relax.FuseOps] ===== IR Before FuseOps =====
# from tvm.script import ir as I
# from tvm.script import tir as T
# from tvm.script import relax as R

@I.ir_module
class Module:
    @T.prim_func(private=True)
    def add(x: T.Buffer((T.int64(10), T.int64(20)), "float32"), w: T.Buffer((T.int64(10), T.int64(20)), "float32"), T_add: T.Buffer((T.int64(10), T.int64(20)), "float32")):
        T.func_attr({"op_pattern": 0, "tir.noalias": True})
        # with T.block("root"):
        for ax0, ax1 in T.grid(T.int64(10), T.int64(20)):
            with T.block("T_add"):
                v_ax0, v_ax1 = T.axis.remap("SS", [ax0, ax1])
                T.reads(x[v_ax0, v_ax1], w[v_ax0, v_ax1])
                T.writes(T_add[v_ax0, v_ax1])
                T_add[v_ax0, v_ax1] = x[v_ax0, v_ax1] + w[v_ax0, v_ax1]

    @T.prim_func(private=True)
    def exp(lv: T.Buffer((T.int64(10), T.int64(20)), "float32"), compute: T.Buffer((T.int64(10), T.int64(20)), "float32")):
        T.func_attr({"op_pattern": 0, "tir.noalias": True})
        # with T.block("root"):
        for i0, i1 in T.grid(T.int64(10), T.int64(20)):
            with T.block("compute"):
                v_i0, v_i1 = T.axis.remap("SS", [i0, i1])
                T.reads(lv[v_i0, v_i1])
                T.writes(compute[v_i0, v_i1])
                compute[v_i0, v_i1] = T.exp(lv[v_i0, v_i1])

    @T.prim_func(private=True)
    def tanh(lv: T.Buffer((T.int64(10), T.int64(20)), "float32"), compute: T.Buffer((T.int64(10), T.int64(20)), "float32")):
        T.func_attr({"op_pattern": 0, "tir.noalias": True})
        # with T.block("root"):
        for i0, i1 in T.grid(T.int64(10), T.int64(20)):
            with T.block("compute"):
                v_i0, v_i1 = T.axis.remap("SS", [i0, i1])
                T.reads(lv[v_i0, v_i1])
                T.writes(compute[v_i0, v_i1])
                compute[v_i0, v_i1] = T.tanh(lv[v_i0, v_i1])

    @R.function
    def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
        cls = Module
        with R.dataflow():
            lv = R.call_tir(cls.add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv1 = R.call_tir(cls.exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv2 = R.call_tir(cls.tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv3 = R.call_tir(cls.add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            gv: R.Tensor((10, 20), dtype="float32") = lv3
            R.output(gv)
        return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1332: [Relax.FuseOps] Step 1: Create graph
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:177: [Relax.FuseOps][GraphCreator] Visit GlobalVar=main
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Function
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:198: [Relax.FuseOps][GraphCreator] Visit Function in main (params=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:157: [ExprVisitor::VisitExpr_(FunctionNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:158: [ExprVisitor::VisitExpr_(FunctionNode)]     params=2, has_attrs=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:162: [ExprVisitor::VisitExpr_(FunctionNode)]     visiting param[0]: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: x, type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:162: [ExprVisitor::VisitExpr_(FunctionNode)]     visiting param[1]: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: w, type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:167: [ExprVisitor::VisitExpr_(FunctionNode)]     visiting body...
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.SeqExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:235: [ExprVisitor::VisitExpr_(SeqExprNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:236: [ExprVisitor::VisitExpr_(SeqExprNode)]     num_blocks=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:237: [ExprVisitor::VisitExpr_(SeqExprNode)]     body: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:241: [ExprVisitor::VisitExpr_(SeqExprNode)]     visiting block[0] type: relax.expr.DataflowBlock, num_bindings=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:352: [ExprVisitor::VisitBindingBlock] >>> Enter, type: relax.expr.DataflowBlock
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:354: [ExprVisitor::VisitBindingBlock]     dispatching to VisitBindingBlock_(DataflowBlockNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:314: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:315: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     num_bindings=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[0] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv, value=R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=x)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[1] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv1, value=R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[2] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv2, value=R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[3] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=lv3, value=R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32")))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:268: [Relax.FuseOps][GraphCreator] VisitCall(op=relax.call_tir, nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv1)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kElemWise, leaf=lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[4] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:230: [Relax.FuseOps][GraphCreator] Visit VarBinding(var=gv, value=lv3)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:332: [Relax.FuseOps][GraphCreator] VisitUnsupportedNode(expr=lv3)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:351: [Relax.FuseOps][GraphCreator] VisitLeaf(edge_pattern=kOpaque, leaf=lv3)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:327: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:362: [ExprVisitor::VisitBindingBlock] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:245: [ExprVisitor::VisitExpr_(SeqExprNode)]     visiting body...
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:251: [ExprVisitor::VisitExpr_(SeqExprNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:170: [ExprVisitor::VisitExpr_(FunctionNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1336: [Relax.FuseOps] Graph DebugDump
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/../analysis/graph_partitioner.h:92: node[0], x outputs=[2, ]
node[1], w outputs=[2, ]
node[2], lv outputs=[3, 4, ]
node[3], lv1 outputs=[5, ]
node[4], lv2 outputs=[5, ]
node[5], lv3 outputs=[6, ]
node[6], gv outputs=[]

[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:148: [Relax.FuseOps] PostDominatorTree nodes=7
  tnode[0] gnode.index=0 extern_ref=1 gnode.pattern=kOpaque parent.gnode.index=<null> depth=1 agg_pattern_to_parent=kOpaque ref=x
  tnode[1] gnode.index=1 extern_ref=1 gnode.pattern=kOpaque parent.gnode.index=<null> depth=1 agg_pattern_to_parent=kOpaque ref=w
  tnode[2] gnode.index=2 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=5 depth=3 agg_pattern_to_parent=kElemWise ref=lv
  tnode[3] gnode.index=3 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=5 depth=3 agg_pattern_to_parent=kElemWise ref=lv1
  tnode[4] gnode.index=4 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=5 depth=3 agg_pattern_to_parent=kElemWise ref=lv2
  tnode[5] gnode.index=5 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=6 depth=2 agg_pattern_to_parent=kOpaque ref=lv3
  tnode[6] gnode.index=6 extern_ref=1 gnode.pattern=kOpaque parent.gnode.index=<null> depth=1 agg_pattern_to_parent=kOpaque ref=gv

[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1344: [Relax.FuseOps] Step 2: Partition graph
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:158: [Relax.FuseOps][Partition] Begin
  opt_level=3 max_fuse_depth=256 max_function_args=0
  nodes=7
[Relax.FuseOps][Partition] PostDominatorTree (tnode == post_dfs_order index)
  tnode[0] gnode.index=0 extern_ref=1 gnode.pattern=kOpaque parent.gnode.index=<null> depth=1 agg_pattern_to_parent=kOpaque ref=x
  tnode[1] gnode.index=1 extern_ref=1 gnode.pattern=kOpaque parent.gnode.index=<null> depth=1 agg_pattern_to_parent=kOpaque ref=w
  tnode[2] gnode.index=2 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=5 depth=3 agg_pattern_to_parent=kElemWise ref=lv
  tnode[3] gnode.index=3 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=5 depth=3 agg_pattern_to_parent=kElemWise ref=lv1
  tnode[4] gnode.index=4 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=5 depth=3 agg_pattern_to_parent=kElemWise ref=lv2
  tnode[5] gnode.index=5 extern_ref=0 gnode.pattern=kElemWise parent.gnode.index=6 depth=2 agg_pattern_to_parent=kOpaque ref=lv3
  tnode[6] gnode.index=6 extern_ref=1 gnode.pattern=kOpaque parent.gnode.index=<null> depth=1 agg_pattern_to_parent=kOpaque ref=gv

[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:164: [Relax.FuseOps][Partition] RunFuse phase=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=0 nid=0 gnode.index=0 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=0 nid=1 gnode.index=1 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=0 nid=2 gnode.index=2 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:644: [Relax.FuseOps][Fuse] CheckPath(broadcast path rules) gnode.index=2 ipdom.gnode.index=5 ok=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:300: [Relax.FuseOps][Fuse] CommitFuse src.gnode.index=2 sink.gnode.index=5 src.pattern=kElemWise sink.pattern=kElemWise src.ref=lv sink.ref=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:242: [Relax.FuseOps][Fuse] MergeFromTo child.pattern=kElemWise parent.pattern=kElemWise child.root_ref=lv parent.root_ref=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:242: [Relax.FuseOps][Fuse] MergeFromTo child.pattern=kElemWise parent.pattern=kElemWise child.root_ref=lv1 parent.root_ref=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:242: [Relax.FuseOps][Fuse] MergeFromTo child.pattern=kElemWise parent.pattern=kElemWise child.root_ref=lv2 parent.root_ref=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=0 nid=3 gnode.index=3 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:581: [Relax.FuseOps][Fuse] Skip: already fused to ipdom gnode.index=3 ipdom.gnode.index=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=0 nid=4 gnode.index=4 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:581: [Relax.FuseOps][Fuse] Skip: already fused to ipdom gnode.index=4 ipdom.gnode.index=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=0 nid=5 gnode.index=5 extern_ref=0 pattern=kElemWise ipdom.gnode.index=6 edge_pattern=kOpaque ref=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:653: [Relax.FuseOps][Fuse] Skip: broadcast node but ipdom edge_pattern not allowed gnode.index=5 edge_pattern=kOpaque
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=0 nid=6 gnode.index=6 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=6
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:164: [Relax.FuseOps][Partition] RunFuse phase=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=1 nid=0 gnode.index=0 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=1 nid=1 gnode.index=1 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=1 nid=2 gnode.index=2 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:581: [Relax.FuseOps][Fuse] Skip: already fused to ipdom gnode.index=2 ipdom.gnode.index=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=1 nid=3 gnode.index=3 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:581: [Relax.FuseOps][Fuse] Skip: already fused to ipdom gnode.index=3 ipdom.gnode.index=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=1 nid=4 gnode.index=4 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:581: [Relax.FuseOps][Fuse] Skip: already fused to ipdom gnode.index=4 ipdom.gnode.index=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=1 nid=5 gnode.index=5 extern_ref=0 pattern=kElemWise ipdom.gnode.index=6 edge_pattern=kOpaque ref=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:653: [Relax.FuseOps][Fuse] Skip: broadcast node but ipdom edge_pattern not allowed gnode.index=5 edge_pattern=kOpaque
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=1 nid=6 gnode.index=6 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=6
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:164: [Relax.FuseOps][Partition] RunFuse phase=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=2 nid=0 gnode.index=0 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=2 nid=1 gnode.index=1 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=2 nid=2 gnode.index=2 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=2 nid=3 gnode.index=3 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=2 nid=4 gnode.index=4 extern_ref=0 pattern=kElemWise ipdom.gnode.index=5 edge_pattern=kElemWise ref=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=2 nid=5 gnode.index=5 extern_ref=0 pattern=kElemWise ipdom.gnode.index=6 edge_pattern=kOpaque ref=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:449: [Relax.FuseOps][Fuse] Consider phase=2 nid=6 gnode.index=6 extern_ref=1 pattern=kOpaque ipdom.gnode.index=<null> ref=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:496: [Relax.FuseOps][Fuse] Skip: opaque gnode.index=6
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/analysis/graph_partitioner.cc:170: [Relax.FuseOps][Partition] End
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1353: [Relax.FuseOps] Step 3: Rewrite IRModule
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:886: [Relax.FuseOps][OperatorFusor] Transform begin entry_function_names=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:904: [Relax.FuseOps][OperatorFusor] Transform: VisitExpr on GlobalVar=main
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:913: [Relax.FuseOps][OperatorFusor] ===== Function Before Rewrite (main) =====
# from tvm.script import relax as R

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    with R.dataflow():
        lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv3 = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        gv: R.Tensor((10, 20), dtype="float32") = lv3
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Function
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: # from tvm.script import relax as R

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    with R.dataflow():
        lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv3 = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        gv: R.Tensor((10, 20), dtype="float32") = lv3
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.SeqExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: x: R.Tensor((10, 20), dtype="float32")
w: R.Tensor((10, 20), dtype="float32")
with R.dataflow():
    lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv3 = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    gv: R.Tensor((10, 20), dtype="float32") = lv3
    R.output(gv)
gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:701: [ExprMutator::VisitExpr_(SeqExprNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:702: [ExprMutator::VisitExpr_(SeqExprNode)]     num_blocks=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:703: [ExprMutator::VisitExpr_(SeqExprNode)]     body: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:709: [ExprMutator::VisitExpr_(SeqExprNode)]     visiting block[0] type: relax.expr.DataflowBlock, num_bindings=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:919: [ExprMutator::VisitBindingBlock] >>> Enter, type: relax.expr.DataflowBlock
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:922: [ExprMutator::VisitBindingBlock]     dispatching to VisitBindingBlock_(DataflowBlockNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:971: [Relax.FuseOps][OperatorFusor] VisitBindingBlock(bindings=5)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1126: [Relax.FuseOps][OperatorFusor] CollectFuncBindings bindings=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1138: [Relax.FuseOps][OperatorFusor]   new FunctionCreator for group=0x56264fc9df98 num_nodes=4 attrs=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv, rhs=Call(relax.call_tir) nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:770: [Relax.FuseOps][FunctionCreator] CheckDefAndUpdateParam add param=x for expr_type=relax.expr.Var args=1 params=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:770: [Relax.FuseOps][FunctionCreator] CheckDefAndUpdateParam add param=w for expr_type=relax.expr.Var args=2 params=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv1, rhs=Call(relax.call_tir) nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv2, rhs=Call(relax.call_tir) nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:512: [Relax.FuseOps][FunctionCreator] AppendBinding VarBinding(var=lv3, rhs=Call(relax.call_tir) nargs=2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=gv group=0x56264fc9dfd0 root=0x56264fc9dfd0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:980: [Relax.FuseOps][OperatorFusor] After CollectFuncBindings: groups_to_fuse=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1149: [Relax.FuseOps][OperatorFusor] CollectFuncBoundary bindings=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=x group=0x56264fc9de80 root=0x56264fc9de80
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1174: [Relax.FuseOps][OperatorFusor]   boundary: consumer_var=lv depends_on producer_group=0x56264fc9de80 via used_var=x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=w group=0x56264fc9deb8 root=0x56264fc9deb8
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1174: [Relax.FuseOps][OperatorFusor]   boundary: consumer_var=lv depends_on producer_group=0x56264fc9deb8 via used_var=w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=gv group=0x56264fc9dfd0 root=0x56264fc9dfd0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1174: [Relax.FuseOps][OperatorFusor]   boundary: consumer_var=gv depends_on producer_group=0x56264fc9df98 via used_var=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:606: [Relax.FuseOps][FunctionCreator] AppendOutput var=lv3 new idx=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1186: [Relax.FuseOps][OperatorFusor]   boundary: mark tuple_get used_var=lv3 output_index=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:988: [Relax.FuseOps][OperatorFusor] After CollectFuncBoundary: group_deps=4 tuple_get_indices=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:995: [Relax.FuseOps][OperatorFusor] CreateFunction for group=0x56264fc9df98 num_nodes=4 attrs=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:620: [Relax.FuseOps][FunctionCreator] CreateFunction begin name_hint=fused_add_exp_tanh_add bindings=4 params=2 outputs=1 group_attrs=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:790: [Relax.FuseOps][FunctionCreator] VisitExpr hit argument, arg=x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:790: [Relax.FuseOps][FunctionCreator] VisitExpr hit argument, arg=w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: lv, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:803: [ExprMutator::ReEmitBinding]     var_remap updated: lv -> lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:806: [ExprMutator::ReEmitBinding] <<< Exit (EmitNormalized new binding)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: lv1, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:783: [ExprMutator::ReEmitBinding] <<< Exit (fast path, unchanged)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: lv2, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:783: [ExprMutator::ReEmitBinding] <<< Exit (fast path, unchanged)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=(lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:795: [Relax.FuseOps][FunctionCreator] VisitExpr recurse, expr=R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Function
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:157: [ExprVisitor::VisitExpr_(FunctionNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:158: [ExprVisitor::VisitExpr_(FunctionNode)]     params=2, has_attrs=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:162: [ExprVisitor::VisitExpr_(FunctionNode)]     visiting param[0]: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: x, type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:162: [ExprVisitor::VisitExpr_(FunctionNode)]     visiting param[1]: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: w, type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:167: [ExprVisitor::VisitExpr_(FunctionNode)]     visiting body...
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.SeqExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:235: [ExprVisitor::VisitExpr_(SeqExprNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:236: [ExprVisitor::VisitExpr_(SeqExprNode)]     num_blocks=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:237: [ExprVisitor::VisitExpr_(SeqExprNode)]     body: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:241: [ExprVisitor::VisitExpr_(SeqExprNode)]     visiting block[0] type: relax.expr.DataflowBlock, num_bindings=4
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:352: [ExprVisitor::VisitBindingBlock] >>> Enter, type: relax.expr.DataflowBlock
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:354: [ExprVisitor::VisitBindingBlock]     dispatching to VisitBindingBlock_(DataflowBlockNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:314: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:315: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     num_bindings=4
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[0] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: lv, type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[1] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: lv1, type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[2] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: lv2, type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:319: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[3] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:322: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)]       var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:337: [ExprVisitor::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:339: [ExprVisitor::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:340: [ExprVisitor::VisitBinding]     var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:174: [ExprVisitor::VisitExpr_(CallNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:175: [ExprVisitor::VisitExpr_(CallNode)]     op: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:176: [ExprVisitor::VisitExpr_(CallNode)]     num_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[0]: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:186: [ExprVisitor::VisitExpr_(CallNode)]     visiting arg[1]: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:193: [ExprVisitor::VisitExpr_(CallNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:366: [ExprVisitor::VisitVarDef] >>> Enter, var: gv, type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:375: [ExprVisitor::VisitVarDef] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:348: [ExprVisitor::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:327: [ExprVisitor::VisitBindingBlock_(DataflowBlockNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:362: [ExprVisitor::VisitBindingBlock] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:245: [ExprVisitor::VisitExpr_(SeqExprNode)]     visiting body...
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:115: [Relax.ExprVisitor] VisitExpr relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:251: [ExprVisitor::VisitExpr_(SeqExprNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:170: [ExprVisitor::VisitExpr_(FunctionNode)] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Function
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: # from tvm.script import relax as R

@R.function(private=True)
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    R.func_attr({"Primitive": True})
    with R.dataflow():
        lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        gv = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.SeqExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: x: R.Tensor((10, 20), dtype="float32")
w: R.Tensor((10, 20), dtype="float32")
with R.dataflow():
    lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    gv = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    R.output(gv)
gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:701: [ExprMutator::VisitExpr_(SeqExprNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:702: [ExprMutator::VisitExpr_(SeqExprNode)]     num_blocks=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:703: [ExprMutator::VisitExpr_(SeqExprNode)]     body: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:709: [ExprMutator::VisitExpr_(SeqExprNode)]     visiting block[0] type: relax.expr.DataflowBlock, num_bindings=4
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:919: [ExprMutator::VisitBindingBlock] >>> Enter, type: relax.expr.DataflowBlock
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:922: [ExprMutator::VisitBindingBlock]     dispatching to VisitBindingBlock_(DataflowBlockNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:859: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:860: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]     num_bindings=4
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:865: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[0] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:868: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]       var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (x, w)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: lv, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:783: [ExprMutator::ReEmitBinding] <<< Exit (fast path, unchanged)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:865: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[1] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:868: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]       var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("exp")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: lv1, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:783: [ExprMutator::ReEmitBinding] <<< Exit (fast path, unchanged)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:865: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[2] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:868: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]       var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("tanh")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (lv,)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: lv2, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:783: [ExprMutator::ReEmitBinding] <<< Exit (fast path, unchanged)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:865: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]     visiting binding[3] type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:868: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)]       var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.Op
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: Op(relax.call_tir)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: ir.GlobalVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: I.GlobalVar("add")
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Tuple
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: (lv1, lv2)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Call
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: gv, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:783: [ExprMutator::ReEmitBinding] <<< Exit (fast path, unchanged)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:874: [ExprMutator::VisitBindingBlock_(DataflowBlockNode)] <<< Exit, result bindings=4
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:930: [ExprMutator::VisitBindingBlock] <<< Exit, result type: relax.expr.DataflowBlock, bindings=4
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:712: [ExprMutator::VisitExpr_(SeqExprNode)]     block[0] after visit: num_bindings=4, unchanged=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:721: [ExprMutator::VisitExpr_(SeqExprNode)]     BeginBindingBlock for prologue
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:723: [ExprMutator::VisitExpr_(SeqExprNode)]     visiting body...
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:725: [ExprMutator::VisitExpr_(SeqExprNode)]     body after visit: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:727: [ExprMutator::VisitExpr_(SeqExprNode)]     EndBlock, prologue bindings=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:739: [ExprMutator::VisitExpr_(SeqExprNode)]     all_blocks_unchanged=0, body_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:747: [ExprMutator::VisitExpr_(SeqExprNode)] <<< Exit (changed, new SeqExpr with 1 blocks)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: x: R.Tensor((10, 20), dtype="float32")
w: R.Tensor((10, 20), dtype="float32")
with R.dataflow():
    lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    gv = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    R.output(gv)
gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.SeqExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: x: R.Tensor((10, 20), dtype="float32")
w: R.Tensor((10, 20), dtype="float32")
with R.dataflow():
    lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    gv = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
    R.output(gv)
gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: # from tvm.script import relax as R

@R.function(private=True)
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    R.func_attr({"Primitive": True})
    with R.dataflow():
        lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        gv = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Function
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: # from tvm.script import relax as R

@R.function(private=True)
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    R.func_attr({"Primitive": True})
    with R.dataflow():
        lv = R.call_tir(add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv1 = R.call_tir(exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        lv2 = R.call_tir(tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        gv = R.call_tir(add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:718: [Relax.FuseOps][FunctionCreator] CreateFunction end name_hint=fused_add_exp_tanh_add function_defined=1 final_params=2 final_args=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1255: [Relax.FuseOps][OperatorFusor] TopoSortByGroupDep bindings=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=gv group=0x56264fc9dfd0 root=0x56264fc9dfd0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1290: [Relax.FuseOps][OperatorFusor] TopoSortByGroupDep sorted=5
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1064: [Relax.FuseOps][OperatorFusor]   skip: not last binding of group
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv1 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1064: [Relax.FuseOps][OperatorFusor]   skip: not last binding of group
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv2 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1064: [Relax.FuseOps][OperatorFusor]   skip: not last binding of group
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=lv3 group=0x56264fc9df98 root=0x56264fc9df98
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1078: [Relax.FuseOps][OperatorFusor]   AddFunction fused GlobalVar=fused_add_exp_tanh_add (name_hint=fused_add_exp_tanh_add)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1241: [Relax.FuseOps][OperatorFusor] UpdateArgs nargs=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: x
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: w
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1095: [Relax.FuseOps][OperatorFusor]   Emit call result var=lv (dataflow=1)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1021: [Relax.FuseOps][OperatorFusor] Process binding var=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1211: [Relax.FuseOps][OperatorFusor] GetGroupFromBinding var=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1227: [Relax.FuseOps][OperatorFusor] GetGroupFromVar var=gv group=0x56264fc9dfd0 root=0x56264fc9dfd0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1029: [Relax.FuseOps][OperatorFusor]   single-node group, passthrough var=gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:904: [ExprMutator::VisitBinding] >>> Enter, type: relax.expr.VarBinding
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:906: [ExprMutator::VisitBinding]     dispatching to VisitBinding_(VarBindingNode)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:907: [ExprMutator::VisitBinding]     var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.DataflowVar
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:772: [ExprMutator::ReEmitBinding] >>> Enter
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:773: [ExprMutator::ReEmitBinding]     var: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:774: [ExprMutator::ReEmitBinding]     old_value: lv3
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:775: [ExprMutator::ReEmitBinding]     new_value: lv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.ShapeExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: R.shape([10, 20])
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:778: [ExprMutator::ReEmitBinding]     new_var: gv, var_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:803: [ExprMutator::ReEmitBinding]     var_remap updated: gv -> gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:806: [ExprMutator::ReEmitBinding] <<< Exit (EmitNormalized new binding)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:915: [ExprMutator::VisitBinding] <<< Exit
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:930: [ExprMutator::VisitBindingBlock] <<< Exit, result type: relax.expr.DataflowBlock, bindings=2
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:712: [ExprMutator::VisitExpr_(SeqExprNode)]     block[0] after visit: num_bindings=2, unchanged=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:721: [ExprMutator::VisitExpr_(SeqExprNode)]     BeginBindingBlock for prologue
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:723: [ExprMutator::VisitExpr_(SeqExprNode)]     visiting body...
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:619: [ExprMutator::VisitExpr] >>> Enter, expr type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:622: [ExprMutator::VisitExpr]     expr: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Var
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:725: [ExprMutator::VisitExpr_(SeqExprNode)]     body after visit: gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:727: [ExprMutator::VisitExpr_(SeqExprNode)]     EndBlock, prologue bindings=0
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:739: [ExprMutator::VisitExpr_(SeqExprNode)]     all_blocks_unchanged=0, body_unchanged=1
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:747: [ExprMutator::VisitExpr_(SeqExprNode)] <<< Exit (changed, new SeqExpr with 1 blocks)
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: x: R.Tensor((10, 20), dtype="float32")
w: R.Tensor((10, 20), dtype="float32")
with R.dataflow():
    lv: R.Tensor((10, 20), dtype="float32") = fused_add_exp_tanh_add(x, w)
    gv: R.Tensor((10, 20), dtype="float32") = lv
    R.output(gv)
gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.SeqExpr
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: x: R.Tensor((10, 20), dtype="float32")
w: R.Tensor((10, 20), dtype="float32")
with R.dataflow():
    lv: R.Tensor((10, 20), dtype="float32") = fused_add_exp_tanh_add(x, w)
    gv: R.Tensor((10, 20), dtype="float32") = lv
    R.output(gv)
gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:625: [ExprMutator::VisitExpr]     after ExprFunctor::VisitExpr: # from tvm.script import relax as R

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    with R.dataflow():
        lv: R.Tensor((10, 20), dtype="float32") = fused_add_exp_tanh_add(x, w)
        gv: R.Tensor((10, 20), dtype="float32") = lv
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:627: [ExprMutator::VisitExpr] <<< Exit, result type: relax.expr.Function
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/ir/expr_functor.cc:629: [ExprMutator::VisitExpr]     result: # from tvm.script import relax as R

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    with R.dataflow():
        lv: R.Tensor((10, 20), dtype="float32") = fused_add_exp_tanh_add(x, w)
        gv: R.Tensor((10, 20), dtype="float32") = lv
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:923: [Relax.FuseOps][OperatorFusor] ===== Function After Rewrite (main) =====
# from tvm.script import relax as R

@R.function
def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
    with R.dataflow():
        lv: R.Tensor((10, 20), dtype="float32") = fused_add_exp_tanh_add(x, w)
        gv: R.Tensor((10, 20), dtype="float32") = lv
        R.output(gv)
    return gv
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:929: [Relax.FuseOps][OperatorFusor] Transform: UpdateFunction done for GlobalVar=main
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:935: [Relax.FuseOps][OperatorFusor] Transform end
[19:45:02] /workspace/redmatx/3rdparty/tvm/src/relax/transform/fuse_ops.cc:1361: [Relax.FuseOps] ===== IR After FuseOps =====
# from tvm.script import ir as I
# from tvm.script import tir as T
# from tvm.script import relax as R

@I.ir_module
class Module:
    @T.prim_func(private=True)
    def add(x: T.Buffer((T.int64(10), T.int64(20)), "float32"), w: T.Buffer((T.int64(10), T.int64(20)), "float32"), T_add: T.Buffer((T.int64(10), T.int64(20)), "float32")):
        T.func_attr({"op_pattern": 0, "tir.noalias": True})
        # with T.block("root"):
        for ax0, ax1 in T.grid(T.int64(10), T.int64(20)):
            with T.block("T_add"):
                v_ax0, v_ax1 = T.axis.remap("SS", [ax0, ax1])
                T.reads(x[v_ax0, v_ax1], w[v_ax0, v_ax1])
                T.writes(T_add[v_ax0, v_ax1])
                T_add[v_ax0, v_ax1] = x[v_ax0, v_ax1] + w[v_ax0, v_ax1]

    @T.prim_func(private=True)
    def exp(lv: T.Buffer((T.int64(10), T.int64(20)), "float32"), compute: T.Buffer((T.int64(10), T.int64(20)), "float32")):
        T.func_attr({"op_pattern": 0, "tir.noalias": True})
        # with T.block("root"):
        for i0, i1 in T.grid(T.int64(10), T.int64(20)):
            with T.block("compute"):
                v_i0, v_i1 = T.axis.remap("SS", [i0, i1])
                T.reads(lv[v_i0, v_i1])
                T.writes(compute[v_i0, v_i1])
                compute[v_i0, v_i1] = T.exp(lv[v_i0, v_i1])

    @T.prim_func(private=True)
    def tanh(lv: T.Buffer((T.int64(10), T.int64(20)), "float32"), compute: T.Buffer((T.int64(10), T.int64(20)), "float32")):
        T.func_attr({"op_pattern": 0, "tir.noalias": True})
        # with T.block("root"):
        for i0, i1 in T.grid(T.int64(10), T.int64(20)):
            with T.block("compute"):
                v_i0, v_i1 = T.axis.remap("SS", [i0, i1])
                T.reads(lv[v_i0, v_i1])
                T.writes(compute[v_i0, v_i1])
                compute[v_i0, v_i1] = T.tanh(lv[v_i0, v_i1])

    @R.function(private=True)
    def fused_add_exp_tanh_add(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
        R.func_attr({"Primitive": True})
        cls = Module
        with R.dataflow():
            lv = R.call_tir(cls.add, (x, w), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv1 = R.call_tir(cls.exp, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            lv2 = R.call_tir(cls.tanh, (lv,), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            gv = R.call_tir(cls.add, (lv1, lv2), out_sinfo=R.Tensor((10, 20), dtype="float32"))
            R.output(gv)
        return gv

    @R.function
    def main(x: R.Tensor((10, 20), dtype="float32"), w: R.Tensor((10, 20), dtype="float32")) -> R.Tensor((10, 20), dtype="float32"):
        cls = Module
        with R.dataflow():
            lv: R.Tensor((10, 20), dtype="float32") = cls.fused_add_exp_tanh_add(x, w)
            gv: R.Tensor((10, 20), dtype="float32") = lv
            R.output(gv)
        return gv
OK

TVM Pass解读之fuse pass