Skip to search formSkip to main contentSkip to account menu

Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models

This work establishes a novel image generation test bed—inappropriate image prompts (I2P)—containing dedicated, real-world image-to-text prompts covering concepts such as nudity and violence, and introduces safe latent diffusion (SLD), a method to combat degenerated and biased human behavior.
View PDF on arXiv (opens in a new tab)

How Long Do Vulnerabilities Live in the Code? A Large-Scale Empirical Measurement Study on FOSS Vulnerability Lifetimes

The first large-scale measurement of Free and Open Source Software vulnerability lifetimes is performed, and it is found that the average lifetime of a vulnerability is around 4 years, varying significantly between projects (~2 years for Chromium, ~7 years for OpenSSL).
View Paper (opens in a new tab)

Does CLIP Know My Face?

This work introduces a novel method to assess privacy for multi-modal models, specifically vision-language models like CLIP, and suggests that IDIAs can be used to prove the unauthorized use of data for training and to enforce privacy laws.
View PDF on arXiv (opens in a new tab)

Image Classifiers Leak Sensitive Attributes About Their Classes

This work introduces the first Class Attribute Inference Attack (Caia), which leverages recent advances in text-to-image synthesis to infer sensitive attributes of individual classes in a black-box setting, while remaining competitive with related white-box attacks.
View PDF on arXiv (opens in a new tab)

The Stable Artist: Steering Semantics in Diffusion Latent Space

The Stable Artist is presented, to enable control by allow-ing the artist to steer the diffusion process along a variable number of semantic directions, which allows for subtle edits to images, changes in composition and style, as well as optimization of the overall artistic conception.
View PDF on arXiv (opens in a new tab)

Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness

This work presents a novel strategy, called Fair Diffusion, to attenuate biases after the deployment of generative text-to-image models, and demonstrates shifting a bias, based on human instructions, in any direction yielding arbitrarily new proportions for, e.g., identity groups.
View PDF on arXiv (opens in a new tab)

SEGA: Instructing Diffusion using Semantic Dimensions

Text-to-image diffusion models have recently re-ceived a lot of interest for their astonishing ability to produce high-fidelity images from text only. However, achieving one-shot generation that…
View PDF on arXiv (opens in a new tab)

ILLUME: Rationalizing Vision-Language Models by Interacting with their Jabber

This work proposes an iterative sampling and tuning paradigm, called I LLUME, that executes the following loop: Given an image-question-answer prompt, the VLM samples multiple candidate rationales, and a human critic provides minimal feedback via preference selec- tion, used for fine-tuning.
View via Publisher (opens in a new tab)

Exploiting Cultural Biases via Homoglyphs in Text-to-Image Synthesis

It is shown that by simply inserting single non-Latin characters in a textual description, common models reflect cultural stereotypes and biases in their generated images, and a novel homoglyph unlearning method is proposed to fine-tune a text encoder, making it robust againsthomoglyph manipulations.
View PDF on arXiv (opens in a new tab)

ILLUME: Rationalizing Vision-Language Models through Human Interactions

This work proposes a tuning paradigm based on human interactions with machine generated data that is competitive with standard supervised tuning while using significantly fewer training data and only requiring minimal feedback.
View PDF on arXiv (opens in a new tab)
By clicking accept or continuing to use the site, you agree to the terms outlined in our Privacy Policy (opens in a new tab), Terms of Service (opens in a new tab), and Dataset License (opens in a new tab)