machine-learningdeep-learningautoencodergenerative-adversarial-networkimage-generation

Generate or predict image based on a vector input


I want to generate a state field (e.g. temperature field in x-y plane) in form of an mxn image, based on a given input (a vector or a scalar).

To keep it simple, let me put it in a simple example: if input is 0, the output should be a black cat. For input = 0.53, the output is a brown cat, ...., for input = 3, it's a black dog and so on and so forth.

It can be thought as an inverse of a classification, but I am not sure about it.

Based on my search so far, I think this is an image generation problem where GANs or Autoencoders can be used on a labeled dataset (I have images for different input vectors). My questions:

  1. Is this the right way to solve this problem? If so, can you recommend good examples?

  2. If this is not the correct way to solve it, would you please share your opinion on how to solve it?


Solution

  • This is a typical problem for GANs. Especially conditional GANs can be used to solve this problem. Check the following link for more info: https://phillipi.github.io/pix2pix/