I'm working on porting a neural network library to Go. I want to be able to save and restore a trained network, so I'm attempting to serialize it directly. The problem is, the network struct contains cycles in its field (Neuron A has a connection to Neuron B, which has a connection to Neuron A). Whenever I try to serialize the entire network with encoding/gob, it fails with a stackoverflow.
Here's a very simple example of code that breaks in the same way:
package main
import (
"bytes"
"encoding/gob"
"fmt"
"log"
)
type P struct {
Name string
Q *Q
}
type Q struct {
Name string
P *P
}
func main() {
var network bytes.Buffer // Stand-in for a network connection
enc := gob.NewEncoder(&network) // Will write to network.
dec := gob.NewDecoder(&network) // Will read from network.
p := &P{ "P", nil }
q := &Q{ "Q", p }
p.Q = q
err := enc.Encode(p)
if err != nil {
log.Fatal("encode error:", err)
}
// Decode (receive) the value.
var p2 *P
err = dec.Decode(&p2)
if err != nil {
log.Fatal("decode error:", err)
}
fmt.Printf("%#v", p2)
}
http://play.golang.org/p/LrO0VlLnX4
Barring rewriting the entire structure of the library to avoid cycles, is there a straightforward way to get around this problem?
Thanks
You can't use gob directly, but fear not brave citizen of the world!
You can implement the BinaryMarshaler
/BinaryUnmarshaler
interfaces on your type as a work around and gob will happily use them instead when it's encoding/decoding your type.
func (p *P) MarshalBinary() (_ []byte, err error) {
var buf bytes.Buffer
enc := gob.NewEncoder(&buf)
enc.Encode(p.Name)
if p.Q == nil {
return buf.Bytes(), nil
}
isCyclic := p.Q != nil && p.Q.P == p
enc.Encode(isCyclic)
if isCyclic {
p.Q.P = nil
err = enc.Encode(p.Q)
p.Q.P = p
} else {
err = enc.Encode(p.Q)
}
//buf.Encode
return buf.Bytes(), err
}
func (p *P) UnmarshalBinary(data []byte) (err error) {
dec := gob.NewDecoder(bytes.NewReader(data))
if err = dec.Decode(&p.Name); err != nil {
return
}
var isCyclic bool
if err = dec.Decode(&isCyclic); err != nil {
return
}
err = dec.Decode(&p.Q)
if isCyclic {
p.Q.P = p
}
return
}
warning creating a new decoder/encoder every time is extremely inefficient, you might want to look into using binary.*
.