postgresqlgogrpcgo-gormproto3

Integrating gorm.Model fields into protobuf definitions


I am trying to figure out how to integrate the gorm.Model fields (deleted_at, create_at, id, etc) into my proto3 definitions. However, I can't a datetime type for proto3. I tried looking for documentation on how to serialize the gorm fields to strings (since proto3 handles strings) but I have not found anything.

Has anyone been able to successfully use the gorm model fields in their proto definitions? I'm using go-micro's plugin to generate *pb.go files.

Here's my current message definition which doesn't work. It seems like empty strings are being stored in the database for deleted_at since when querying for deleted_at is null the postgres database returns nothing.

message DatabaseConfig {
    string address = 1;
    int32 port = 2;
    string databaseName = 3;
    string username = 4;
    string password = 5;
    string databaseType = 6;
    string quertStatement = 7;
    int32 id = 8;
    string createdAt = 9;
    string updatedAt = 10;
    string deletedAt = 11;
}

UPDATE: I've updated my proto def to the following but gorm still isn't properly using the Id, CreatedAt, UpdatedAt, and DeletedAt fields

syntax = "proto3";

package go.micro.srv.importer;

import "google/protobuf/timestamp.proto";
import "github.com/gogo/protobuf/gogoproto/gogo.proto";

service ImporterService {
    rpc CreateDatabaseConfig(DatabaseConfig) returns (Response) {}
    rpc RetrieveDatabaseConfig(GetRequest) returns (Response) {}
    rpc UpdateDatabaseConfig(DatabaseConfig) returns (Response) {}
    rpc DeleteDatabaseConfig(DatabaseConfig) returns (Response) {}
}


message GetRequest {}


message DatabaseConfig {
    string address = 1;
    int32 port = 2;
    string databaseName = 3;
    string username = 4;
    string password = 5;
    string databaseType = 6;
    string quertStatement = 7;
    int32 id = 8;
    google.protobuf.Timestamp createdAt = 9 [(gogoproto.stdtime) = true];
    google.protobuf.Timestamp updatedAt = 10 [(gogoproto.stdtime) = true];
    google.protobuf.Timestamp deletedAt = 11 [(gogoproto.stdtime) = true];
}


message Response {
    bool created = 1;
    DatabaseConfig database_config = 2;

    repeated DatabaseConfig databaseConfigs = 3;
}

Solution

  • The protoc-gen-gorm project did not work for me. It looks like there is some blending of proto2 and proto3 happening, and ultimately I was unable to get it to work.

    My solution was to create a script to do post processing after I generate the go files from protobuf.

    If this was my proto profile/profile.proto:

    message Profile {
      uint64 id = 1;
      string name = 2;
      bool active = 3;
      // ...
    }
    

    Which created profile/profile.pb.go with standard protoc command:

    // ...
    type Profile struct {
        state         protoimpl.MessageState
        sizeCache     protoimpl.SizeCache
        unknownFields protoimpl.UnknownFields
    
        Id     uint64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
        Name   string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty"`
        Active bool   `protobuf:"varint,3,opt,name=active,proto3" json:"active,omitempty"`
    }
    // ...
    

    I use this script gorm.sh:

    #!/bin/bash
    
    g () {
      sed "s/json:\"$1,omitempty\"/json:\"$1,omitempty\" gorm:\"type:$2\"/"
    }
    
    cat $1 \
    | g "id" "primary_key" \
    | g "name" "varchar(100)" \
    > $1.tmp && mv $1{.tmp,}
    

    Which I invoke on my go file after it's generated with ./gorm.sh profile/profile.pb.go and the result of profile/profile.pb.go is:

    // ...
    type Profile struct {
        state         protoimpl.MessageState
        sizeCache     protoimpl.SizeCache
        unknownFields protoimpl.UnknownFields
    
        Id     uint64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty" gorm:"type:primary_key"`
        Name   string `protobuf:"bytes,2,opt,name=name,proto3" json:"name,omitempty" gorm:"type:varchar(100)"`
        Active bool   `protobuf:"varint,3,opt,name=active,proto3" json:"active,omitempty"`
    }
    // ...