Short background: I have two inputs x₁ and x₂ and a label y = f₁(x₁) + f₂(x₂). The functions f₁ and f₂ are unknown and approximated by f1nn and f2nn. So far so good, and I can train a big network ynn containing the structure for f1nn and f2nn. However, the predicted value y is not of interest, but instead I want f₁ and f₂.

In the current form of the Graph module, I couldn't figure out how to do this in a convenient way. At first I extracted these networks by copying neurons, but then moved on to constructing networks for f₁ and f₂ and then create ynn using the neurons from f1nn and f2nn (without copying), which, by mutability, will be trained in the training of y. Prediction is then accomplished by Graph.model f1nn x.

In the applications of interest to me, this is a quite common problem, so some sort of support for this would be great, unless my use case is very niche.

I see three ways to accomplish this.

1. Like in the crude code below, just take existing neurons from a given network and chain them onto another network.
2. Create a Network or Subnetwork node which does essentially the same thing. This could be more pure, as one does not have to rely on mutability like in 1. Instead, when chaining, everything could be copied, provided the Network node has an interface for extracting a copy to use for prediction later. This approach also makes the network printout a bit cleaner, as there are fewer nodes.
3. For any given node in a network, make a network copy with all nodes it depends on, i.e., all prev nodes.

In my opinion, the second is the cleanest, whereas the third is the most flexible. If anything is of interest, I would gladly work on a PR.

open Graph
let chain_network_ nn in_node =
  let in_nn = get_network in_node in
  let add_node parents child =
    in_nn.size <- in_nn.size + 1;
    connect_to_parents parents child;
    in_nn.topo <- Array.append in_nn.topo [| child |]
  in
  let find_node name topo =
    let x = Owl_utils_array.filter (fun n -> n.name = name) topo in
    x.(0)
  in
  let new_nodes = Array.map
    (fun node : node ->
      let neuron = node.neuron in
      make_node ~name:(nn.nnid ^ node.name) ~train:node.train [||] [||] neuron None in_nn)
    nn.topo
  in
  Array.iter2
    (fun node node' ->
      match node.neuron with
      | Input _ -> ()
      | _ ->
        node'.prev <- Array.map
          (fun n -> match n.neuron with
          | Input _ -> in_node
          | _ -> find_node (nn.nnid ^ n.name) new_nodes)
          node.prev;
        node'.next <- Array.map (fun n -> find_node (nn.nnid ^ n.name) new_nodes) node.next;
        add_node node'.prev node')
    nn.topo
    new_nodes;
  let output_name = nn.nnid ^ (Array.get (get_outputs nn) 0).name in
  find_node output_name new_nodes