apigoserverecho-server

Optimise multiple network request


I made a model serving server with Python Tornado library and its sole purpose is to accept http request with payload and return result in json. The request can be made with either application/json or multipart/form-data.

To authenticate and authorise users, I made another server with Golang echo library. So all user requests should reach here before reaching my resource server.

Here I have a problem, because my program requires images as input, so users will dispatch their request with FormData. When it first hit my Golang server, I need to do the following steps

  1. Read the form file.
  2. Save it in local disk.
  3. Load the file and save it in a byte buffer.
  4. Initialise a multipart writer
  5. Make a request to my resource server
  6. Got result, return to user

I feel like this is redundant as I imagine there is a way to propagate those request directly to my resource server (after auth is done), without having to go through the I/O parts.

My code currently looks like this, at this point authentication is done through middleware. Is there a way to optimise this flow?

func (h Handler) ProcessFormData(c echo.Context) error {

    // some validation

    file, err := c.FormFile("file")
    if err != nil {
        return c.JSON(http.StatusBadRequest, response.Exception{
            Code:   errcode.InvalidRequest,
            Detail: "Invalid file uploaded",
            Error:  err,
        })
    }

    filePath, err := fileUtil.SaveNetworkFile(file)
    if err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:   errcode.SystemError,
            Detail: "Error when processing file",
            Error:  err,
        })
    }

    f, err := os.Open(filePath)
    if err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:   errcode.SystemError,
            Detail: "Error when processing file",
            Error:  err,
        })
    }
    defer f.Close()

    fi, err := f.Stat()
    if err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:   errcode.SystemError,
            Detail: "Error when processing file",
            Error:  err,
        })
    }

    var body bytes.Buffer
    writer := multipart.NewWriter(&body)
    part, err := writer.CreateFormFile("file", fi.Name())
    if err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:   errcode.SystemError,
            Detail: "Error when processing file",
            Error:  err,
        })
    }

    if _, err := io.Copy(part, f); err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:   errcode.SystemError,
            Detail: "Error when processing file",
            Error:  err,
        })
    }
    writer.Close()

    req, err := http.NewRequest("POST", fmt.Sprintf("%s", env.ResourceServer), &body)
    req.Header.Set("Content-Type", writer.FormDataContentType())
    if err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:  errcode.APIRequestError,
            Error: err,
        })
    }

    client := &http.Client{}
    res, err := client.Do(req)
    if err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:   errcode.APIRequestError,
            Detail: "Error when posting request to resource server",
            Error:  err,
        })
    }

    defer res.Body.Close()

    data, _ := ioutil.ReadAll(res.Body)

    if res.StatusCode != 200 {
        errorData := &model.PanicResponse{}
        err := json.Unmarshal(data, errorData)
        if err != nil {
            return c.JSON(http.StatusInternalServerError, response.Exception{
                Code:  errcode.UnmarshalError,
                Error: err,
            })
        }

        return c.JSON(res.StatusCode, errorData)
    }

    result := &model.SuccessResponse{}
    err = json.Unmarshal(data, result)
    if err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:  errcode.UnmarshalError,
            Error: err,
        })
    }

    if fileUtil.IsFileExists(filePath) {
        fileUtil.DeleteFile(filePath)
    }

    // track and update usage
    userData := c.Get("USER")
    user := userData.(model.User)
    db.UpdateUsage(h.Db, &user.ID)

    return c.JSON(200, result)
}

Solution

  • Found a solution thanks to the comment from @cerise-limón

    Essentially, I need just 2 lines

    f, err := file.Open()
    if _, err := io.Copy(part, f); err != nil {
        return c.JSON(http.StatusInternalServerError, response.Exception{
            Code:   errcode.SystemError,
            Detail: "Error when processing file",
        })
    }