Fun with Snapchat’s Gender Swapping Filter
Initial Observations
The center picture is a standard bathroom selfie. To the left is the "male" filter, and on the right the "female" filter.
The first thing most users probably notice is that the app works in real time, works with a few different face angles, and does not require an internet connection to run. Hair behaves very naturally when wearing a beanie.
Occlusion Tests
Ok, it works pretty well. Can we get it to fail? The app detects when the face is in the wrong pose, but what if there are things occluding the face? Do those occluding objects get "transformed" too?
The answer is yes. Below is a test where I slide an object across my face. The app works when half the face is occluded, but it seems like if too much of the face is blocked, the "should I face swap" bit is set to False.
How does it work? A guess
At first glance, my mind jumped to some sort of CycleGAN architecture that maps the distribution of male faces to female faces, and vice versa. The dataset would be the billions of selfies Snap has, er, not deleted in the last 8 years.
This does raise a lot of questions though:
- Are they training truly unpaired image translation? That would be incredibly impressive, ...
Will gender fluidity & drag culture will become more normalized in society as our daily social media normalize gender-bending?
See the full story here: https://blog.evjang.com/2019/05/fun-with-snapchats-gender-swapping.html?fbclid=IwAR2zMkl9X_3-1lvOPj7KBKLCHCEouEORizuOi1JP_YEEVaLm1MxK6xnQBmI