mongodbgomarshallingbsonmongoimport

Marshal Go Struct to BSON for mongoimport


I have a slice of structs that I want to write to a BSON file for doing a mongoimport.

This a rough idea of what I'm doing (using gopkg.in/mgo.v2/bson):

type Item struct {
    ID   string `bson:"_id"`
    Text string `bson:"text"`
}

items := []Item{
    {
        ID: "abc",
        Text: "def",
    },
    {
        ID: "uvw",
        Text: "xyz",
    },
}

file, err := bson.Marshal(items)
if err != nil {
    fmt.Printf("Failed to marshal BSON file: '%s'", err.Error())
}

if err := ioutil.WriteFile("test.bson", file, 0644); err != nil {
    fmt.Printf("Failed to write BSON file: '%s'", err.Error())
}

This runs and generates the file, but it's not formatted correctly - instead it looks like this (using bsondump --pretty test.bson):

{
    "1": {
        "_id": "abc",
        "text": "def"
    },
    "2": {
        "_id": "abc",
        "text": "def"
    }
}

When I think it should look more like:

{
    "_id": "abc",
    "text": "def"
{
}
    "_id": "abc",
    "text": "def"
}

Is this possible to do in Go? I just want to generate a .bson file that you would expect a mongodump command to produce, so that I can run mongoimport and populate a colection.


Solution

  • You want independent BSON documents, so marshal the items individually:

    buf := &bytes.Buffer{}
    for _, item := range items {
        data, err := bson.Marshal(item)
        if err != nil {
            fmt.Printf("Failed to marshal BSON item: '%v'", err)
        }
        buf.Write(data)
    }
    
    if err := ioutil.WriteFile("test.bson", buf.Bytes(), 0644); err != nil {
        fmt.Printf("Failed to write BSON file: '%v'", err)
    }
    

    Running bsondump --pretty test.bson, the output will be:

    {
            "_id": "abc",
            "text": "def"
    }
    {
            "_id": "uvw",
            "text": "xyz"
    }
    2022-02-09T10:23:44.886+0100    2 objects found
    

    Note that the buffer is not needed if you write directly to the file:

    f, err := os.Create("test.bson")
    if err != nil {
        log.Panicf("os.Create failed: %v", err)
    }
    defer f.Close()
    
    for _, item := range items {
        data, err := bson.Marshal(item)
        if err != nil {
            log.Panicf("bson.Marshal failed: %v", err)
        }
        if _, err := f.Write(data); err != nil {
            log.Panicf("f.Write failed: %v", err)
        }
    }