Karthik Yearning Deep Learning

U-GAT-IT Architecture


Paper: U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation



Architecture:


Generator:


Discriminator:


Loss Function:

Adversarial Loss: An adversarial loss is employed to match the distribution of the translated images to the target image distribution.

Cycle Loss: To alleviate the mode collapse problem, we apply a cycle consistency constraint to the generator. Given an image x ∈ Xs , after the sequential translations of x from Xs to Xt and from Xt to Xs, the image should be successfully translated back to the original domain.

Identity Loss: To ensure that the color distributions of the input image and output image are similar, we apply an identity consistency constraint to the generator. Given an image x ∈ Xt, after the translation of x using Gs→t, the image should not change.

CAM Loss: By exploiting the information from the auxiliary classifiers ηs and η_Dt, given an image x ∈ {Xs, Xt}. Gs→t and Dt get to where they need to improve or what makes the most difference between two domains in the current state.


Full Objective: Finally, we jointly train the encoders, de-coders, discriminators, and auxiliary classifiers to optimize the final objective:


Resource

comments powered by Disqus