Image "Cloaking" for Personal Privacy

Original Cloaked Original Cloaked

Shawn Shan, PhD Student
Emily Wenger, PhD Student
Jiayun Zhang, Visiting Student
Huiying Li, PhD Student
Haitao Zheng, Professor
Ben Y. Zhao, Professor
Project co-leaders
and co-first authors



Introduction Technical Paper Downloads Media/Press Limitations FAQ

NEWS
  • 5-12-22: v1.01 release for Macbook M1!
  • 5-1-22: Fawkes hits 840,000 downloads!
  • 4-23-21: v1.0 release for Windows/MacOS apps and Win/Mac/Linux binaries!
  • 4-2-21: Fawkes hits 500,000 downloads!
  • 1-28-21: Adversarial training against Fawkes detected in Microsoft Azure (see below)
  • 8-23-20: Email us to join the Fawkes-announce mailing list for updates/news on Fawkes
  • 8-13-20: Fawkes paper presented at USENIX Security 2020

News: May 12, 2022. Sorry for the delays. The Fawkes team has been busy with much research, but finally produced an initial prototype of Fawkes for the Macbook M1 chip. Please see below for the download link. It is significantly faster than the Intel version (both due to just hardware speedup and the fact we can now leverage some of the GPU capacity on the M1).

News: Jan 28, 2021. It has recently come to our attention that there was a significant change made to the Microsoft Azure facial recognition platform in their backend model. Along with general improvements, our experiments seem to indicate that Azure has been trained to lower the efficacy of the specific version of Fawkes that has been released in the wild. We are unclear as to why this was done (since Microsoft, to the best of our knowledge, does not build unauthorized models from public facial images), nor have we received any communication from Microsoft on this. However, we feel it is important for our users to know of this development. We have made a major update (v1.0) to the tool to circumvent this change (and others like it). Please download the newest version of Fawkes below.


2020 is a watershed year for machine learning. It has seen the true arrival of commodized machine learning, where deep learning models and algorithms are readily available to Internet users. GPUs are cheaper and more readily available than ever, and new training methods like transfer learning have made it possible to train powerful deep learning models using smaller sets of data.

But accessible machine learning also has its downsides. A recent New York Times article by Kashmir Hill profiled clearview.ai, an unregulated facial recognition service that has downloaded over 3 billion photos of people from the Internet and social media and used them to build facial recognition models for millions of citizens without their knowledge or permission. Clearview.ai demonstrates just how easy it is to build invasive tools for monitoring and tracking using deep learning.

So how do we protect ourselves against unauthorized third parties building facial recognition models that recognize us wherever we may go? Regulations can and will help restrict the use of machine learning by public companies but will have negligible impact on private organizations, individuals, or even other nation states with similar goals.

The SAND Lab at University of Chicago has developed Fawkes1, an algorithm and software tool (running locally on your computer) that gives individuals the ability to limit how unknown third parties can track them by building facial recognition models out of their publicly available photos. At a high level, Fawkes "poisons" models that try to learn what you look like, by putting hidden changes into your photos, and using them as Trojan horses to deliver that poison to any facial recognition models of you. Fawkes takes your personal images and makes tiny, pixel-level changes that are invisible to the human eye, in a process we call image cloaking. You can then use these "cloaked" photos as you normally would, sharing them on social media, sending them to friends, printing them or displaying them on digital devices, the same way you would any other photo. The difference, however, is that if and when someone tries to use these photos to build a facial recognition model, "cloaked" images will teach the model an highly distorted version of what makes you look like you. The cloak effect is not easily detectable by humans or machines and will not cause errors in model training. However, when someone tries to identify you by presenting an unaltered, "uncloaked" image of you (e.g. a photo taken in public) to the model, the model will fail to recognize you.

Fawkes has been tested extensively and proven effective in a variety of environments and is 100% effective against state-of-the-art facial recognition models (Microsoft Azure Face API, Amazon Rekognition, and Face++). We are in the process of adding more material here to explain how and why Fawkes works. For now, please see the link below to our technical paper, which was presented at USENIX Security Symposium, held on August 12-14, 2000.

The Fawkes project is led by two PhD students at SAND Lab, Emily Wenger and Shawn Shan, with important contributions from Jiayun Zhang (SAND Lab visitor and current PhD student at UC San Diego) and Huiying Li, also a SAND Lab PhD student. The faculty advisors are SAND Lab co-directors and Neubauer Professors Ben Zhao and Heather Zheng.

1The Guy Fawkes mask, a la V for Vendetta.

In addition to the photos of the team cloaked above, here are a couple more examples of cloaked images and their originals. Can you tell which is the original? (Cloaked image of the Queen courtesy of TheVerge).


Publication & Presentation

Fawkes: Protecting Personal Privacy against Unauthorized Deep Learning Models.
Shawn Shan, Emily Wenger, Jiayun Zhang, Huiying Li, Haitao Zheng, and Ben Y. Zhao.
In Proceedings of USENIX Security Symposium 2020. ( Download PDF here )


Downloads and Source Code - v1.0 Release!

  • NEW! Fawkes v1.01 for Macbook M1 is here! This version achieves a significant speedup due to the new M1 hardware!
  • Fawkes v1.0 is a major update. We made the following updates to significantly improve the protection and software reliability.
    • We updated the backend feature extractor to the-state-of-art ArcFace models.
    • We injected additional randomness to the cloak generation process through randomized model selection.
    • We migrated the code base from TF 1 to TF 2, which resulted in a significant speedup and better compatibility.
    • Other minor tweaks to improve protection and minimize image perturbations.
  • Download the Fawkes Software:

    (new) DMG for M1 Mac (v1.01)
    Compatibility: Macbook M1
    EXE for Windows (v1.0)
    Compatibility: Windows 10

    For Intel chip Macbooks (before Nov 2020) download DMG here.

    Setup Instructions: For MacOS, download the .dmg file and double click to install. If your Mac refuses to open because the APP is from an unidentified developer, please go to System Preference>Security & Privacy>General and click Open Anyway.

  • Download the Fawkes Executable Binary:
    Fawkes binary offers additional options on selecting different parameters. Check here for more information on how to select the best parameters for your use case.
    Download Mac Binary (v1.0)
    Download Windows Binary (v1.0)
    Download Linux Binary (v1.0)
    For binary, simply run "./protection -d imgs/"
  • Fawkes Source Code on Github, for development.


If you have any issues running Fawkes, please feel free to ask us by email or raising an issue in our Github repo. Check back often for new releases, or subscribe to our (very) low-volume mailing list for Fawkes announcements and news.


Media and Press Coverage

We stopped trying to keep this up to date. If you see an interesting article talking about Fawkes that's not listed here (especially if in non-English format), email us to let us know. Thanks.

Limitations

To our users, please note that Fawkes is not a "silver bullet" for protecting yourself against intrusive facial recognition systems, as our paper clearly states. To be more clear:

Fawkes cannot:

  • Protect you already-existent facial recognition models. Instead, Fawkes is designed to poison future facial recognition training datasets.
  • Be future-proof. We do not claim that Fawkes provides you with protection against facial recognition for all time. In fact, the changes we observed in Azure after the release of Fawkes0.3 demonstrate that it cannot.

However, Fawkes can:

  • Empower individuals to make their personal data less accessible/usable by big tech companies. While Fawkes is not perfect, it provides individuals with recourse they didn’t have before and can provide some protection against unwanted facial recognition.
  • Catalyze future research which understands and improves upon Fawkes’ strengths and weaknesses. The fight against unwanted facial recognition is far from over. Fawkes is one battlefield in a much-larger war.

All in all, Fawkes is meant to shift the power dynamic in favor of internet users over intrusive tech companies. We thank you for supporting our tool.


Frequently Asked Questions

  • I just cloaked an image, and when I uploaded it to (specific facial recognition model) of me, it still recognized me! So Fawkes doesn't work?
    This is one of the most common misunderstandings of what Fawkes does. Fawkes is NOT designed to protect the specific images that you apply the cloak to. After all, facial recognition tools that recognize you from photos taken by third parties are not going to give you the chance to cloak them before trying to determine your identity. Instead, Fawkes is designed to prevent a 3rd party from training a recognition model of you, based on photos of you. If someone already has a model of what you look like, e.g. Facebook or Pimeyes or another model you trained using your real images, they are quite likely to recognize you in photos, even cloaked ones. However, if a 3rd party does NOT have a facial recognition model built to recognize you, cloaking your photo ensures that they cannot build an accurate model of your face using your (cloaked) photos. Cloaking your photos means that someone trying to build such a model will produce a wrong idea of what your face looks like, and subsequently, would incorrectly classify a real photo of you as someone else.

    To properly test the functionality that Fawkes was designed for, what you should do is: choose a ML as a service platform, or download an existing feature extractor for facial recognition (if you want to train the model yourself). Now train a model of "you" using a set of "cloaked" images of you until the model converges. Then test that model by giving it a non-cloaked, i.e. original, photo of your face to identify. If your model is trained to recognize a non-trivial number of faces, then it should mis-identify your real photo as someone else. Then you will know that Fawkes has been successful.

  • How effective is Fawkes against 3rd party facial recognition models like ClearView.ai?
    We have extensive experiments and results in the technical paper (linked above). The short version is that we provide strong protection against unauthorized models. Our tests against state of the art facial recognition models from Microsoft Azure, Amazon Rekognition, and Face++ are at or near 100%. Protection level will vary depending on your willingness to tolerate small tweaks to your photos. Please do remember that this is a research effort first and foremost, and while we are trying hard to produce something useful for privacy-aware Internet users at large, there are likely issues in configuration, usability in the tool itself, and it may not work against all models for all images.

  • How could this possibly work against DNNs? Aren't they supposed to be smart?
    This is a popular reaction to Fawkes, and quite reasonable. We hear often in popular press how amazingly powerful DNNs are and the impressive things they can do with large datasets, often detecting patterns where human cannot. Yet the achilles heel for DNNs has been this phenomenon called adversarial examples, small tweaks in inputs that can produce massive differences in how DNNs classify the input. These adversarial examples have been recognized since 2014 (here's one of the first papers on the topic), and numerous defenses have been proposed over the years since (and some of them are from our lab). Turns out they are extremely difficult to remove, and in a way are a fundamental consequence of the imperfect training of DNNs. There have been multiple PhD dissertations written already on the subject, but suffice it to say, this is a fundamentally difficult thing to remove, and many in the research area accept it now as a necessary evil for DNNs.

    The underlying techniques used by Fawkes draw directly from the same properties that give rise to adversarial examples. Is it possible that DNNs evolve significantly to eliminate this property? It's certainly possible, but we expect that will require a significant change in how DNNs are architected and built. Until then, Fawkes works precisely because of fundamental weaknesses in how DNNs are designed today.

  • Can't you just apply some filter, or compression, or blurring algorithm, or add some noise to the image to destroy image cloaks?
    As counterintuitive as this may be, the high level answer is no simple tools work to destroy the perturbation that form image cloaks. To make sense of this, it helps to first understand that Fawkes does not use high-intensity pixels, or rely on bright patterns to distort the classification value of the image in the feature space. It is a precisely computed combination of a number of pixels that do not easily stand out, that produce the distortion in the feature space. If you're interested in seeing some details, we encourage you to take a look at the technical paper (also linked above). In it we present detailed experimental results showing how robust Fawkes is to things like image compression and distortion/noise injection. The quick takeaway is that as you increase the magnitude of these noisy disruptions to the image, protection of image cloaking does fall, but slower than normal image classification accuracy. Translated: Yes, it is possible to add noise and distortions at a high enough level to distort image cloaks. But such distortions will hurt normal classification far more and faster. By the time a distortion is large enough to break cloaking, it has already broken normal image classification and made the image useless for facial recognition.

  • How is Fawkes different from things like the invisibility cloak projects at UMaryland, led by Tom Goldstein, and other similar efforts?
    Fawkes works quite differently from these prior efforts, and we believe it is the first practical tool that the average Internet user can make use of. Prior projects like the invisibility cloak project involve users wearing a specially printed patterned sweater, which then prevents the wearer from being recognized by person-detection models. In other cases, the user is asked to wear a printed placard, or a special patterned hat. One fundamental difference is that these approaches can only protect a user when the user is wearing the sweater/hat/placard. Even if users were comfortable wearing these unusual objects in their daily lives, these mechanisms are model-specific, that is, they are specially encoded to prevent detection against a single specific model (in most cases, it is the YOLO model). Someone trying to track you can either use a different model (there are many), or just target users in settings where they can't wear these conspicuous accessories. In contrast, Fawkes is different because it protects users by targeting the model itself. Once you disrupt the model that's trying to track you, the protection is always on no matter where you go or what you wear, and even extends to attempts to identify you from static photos of you taken, shared or sent digitally.

  • How can Fawkes be useful when there are so many uncloaked, original images of me on social media that I can't take down?
    Fawkes works by training the unauthorized model to learn about a cluster of your cloaked images in its "feature space." If you, like many of us, already have a significant set of public images online, then a model like Clearview.AI has likely already downloaded those images, and used them to learn "what you look like" as a cluster in its feature space. However, these models are always adding more training data in order to improve their accuracy and keep up with changes in your looks over time. The more cloaked images you "release," the larger the cluster of "cloaked features" will be learned by the model. At some point, when your cloaked cluster of images grows bigger than the cluster of uncloaked images, the tracker's model will switch its definition of you to the new cloaked cluster and abandon the original images as outliers.

  • Is Fawkes specifically designed as a response to Clearview.ai?
    It might surprise some to learn that we started the Fawkes project a while before the New York Times article that profiled Clearview.ai in February 2020. Our original goal was to serve as a preventative measure for Internet users to inoculate themselves against the possibility of some third-party, unauthorized model. Imagine our surprise when we learned 3 months into our project that such companies already existed, and had already built up a powerful model trained from massive troves of online photos. It is our belief that Clearview.ai is likely only the (rather large) tip of the iceberg. Fawkes is designed to significantly raise the costs of building and maintaining accurate models for large-scale facial recognition. If we can reduce the accuracy of these models to make them untrustable, or force the model's owners to pay significant per-person costs to maintain accuracy, then we would have largely succeeded. For example, someone carefully examining a large set of photos of a single user might be able to detect that some of them are cloaked. However, that same person is quite possibly capable of identifying the target person in equal or less time using traditional means (without the facial recognition model).

  • Can Fawkes be used to impersonate someone else?
    The goal of Fawkes is to avoid identification by someone with access to an unauthorized facial recognition model. While it is possible for Fawkes to make you "look" like someone else (e.g. "person X") in the eyes of a recognition model, we would not consider it an impersonation attack, since "person X" is highly likely to want to avoid identification by the model themselves. If you cloaked an image of yourself before giving it as training data to a legitimate model, the model trainer can simply detect the cloak by asking you for a real-time image, and testing it against your cloaked images in the feature space. The key to detecting cloaking is the "ground truth" image of you that a legitmate model can obtain, and unauthorized models cannot.

  • How can I distinguish photos that have been cloaked from those that have not?
    A big part of the goal of Fawkes is to make cloaking as subtle and undetectable as possible and minimize impact on your photos. Thus it is intentionally difficult to tell cloaked images from the originals. We are looking into adding small markers into the cloak as a way to help users identify cloaked photos. More information to come.

  • How do I get Fawkes and use it to protect my photos?
    We are working hard to produce user-friendly versions of Fawkes for use on Mac and Windows platforms. We have some initial binaries for the major platforms (see above). Fawkes is also available as source code, and you can compile it on your own computer. Feel free to report bugs and issues on github, but please bear with us if you have issues with the usability of these binaries. Note that we do not have any plans to release any Fawkes mobile apps, because it requires significant computational power that would be challenging for the most powerful mobile devices.

  • We are adding more Q&A soon. If you don't see your question here, please email us and we will add it to the page soon.