goretry-logicexponential-backoff

Keep retrying a function in Golang


I am trying to make a functionality which would work in the following manner:

  1. As soon as the service function is called, it uses the Fetch function to get records from a service (which come in the form of byte array), JSON unmarshal the byte array, populate the struct and then send the struct to a DB function to save to database.
  2. Now, since this needs to be a continuous job, I have added two if conditions such that, if the records received are of length 0, then we use the retry function to retry pulling the records, else we just write to the database.

I have been trying to debug the retry function for a while now, but it is just not working, and basically stops after the first retry (even though I specify the attempts as 100). What can I do to make sure, it keeps retrying pulling the records ?

The code is as Follows:

// RETRY FUNCTION
func retry(attempts int, sleep time.Duration, f func() error) (err error) {
for i := 0; ; i++ {
    err = f()
    if err == nil {
        return
    }

    if i >= (attempts - 1) {
        break
    }

    time.Sleep(sleep)
    sleep *= 2

    log.Println("retrying after error:", err)
}
return fmt.Errorf("after %d attempts, last error: %s", attempts, err) }


//Save Data function 

type Records struct {
Messages [][]byte
}

func (s *Service) SaveData(records Records, lastSentPlace uint) error {

//lastSentPlace is sent as 0 to begin with.
for i := lastSentPlace; i <= records.Place-1; i++ {

    var msg Records
    msg.Unmarshal(records.Messages[i])

    order := MyStruct{
        Fruit:    msg.Fruit,
        Burger:   msg.Burger,
        Fries:    msg.Fries,
     }

    err := s.db.UpdateOrder(context.TODO(), nil , order)
    if err != nil {
        logging.Error("Error occured...")
    }
    
}return nil}



//Service function (This runs as a batch, which is why we need retrying)

func (s *Service) MyServiceFunction(ctx context.Context, place uint, length uint) (err error) {

var lastSentPlace = place

records, err := s.Poll(context.Background(), place, length)
if err != nil {
    logging.Info(err)
}

// if no records found then retry.
if len(records.Messages) == 0 {
    
    err = retry(100, 2*time.Minute, func() (err error) {
        records, err := s.Poll(context.Background(), place, length)
        
        // if data received, write to DB
        if len(records.Messages) != 0 {
            err = s.SaveData(records, lastSentPlace)
        }
        return
    })
    // if data is not received, or if err is not null, retry
    if err != nil || len(records.Messages) == 0 {
        log.Println(err)
        return
    }
// if data received on first try, then no need to retry, write to db 
} else if len(records.Messages) >0 {
    err = s.SaveData(records, lastSentPlace)
    if err != nil {
        return err
    }
}

return nil }

I think, the issue is with the way I am trying to implement the retry function, I have been trying to debug this for a while, but being new to the language, I am really stuck. What I wanted to do was, implement a backoff if no records are found. Any help is greatly appreciated.

Thanks !!!


Solution

  • I make a simpler retry.

    Here's the code:

    func retry(attempts int, sleep time.Duration, f func() error) (err error) {
        for i := 0; i < attempts; i++ {
            if i > 0 {
                log.Println("retrying after error:", err)
                time.Sleep(sleep)
                sleep *= 2
            }
            err = f()
            if err == nil {
                return nil
            }
        }
        return fmt.Errorf("after %d attempts, last error: %s", attempts, err)
    }