Interventions & Control
For HCI researchers, a generative model is a design material. This table maps intervention levers by pipeline stage, allowing you to gauge the engineering cost versus the structural control gained.
| Intervention | Stage | Modality | Mechanism |
|---|---|---|---|
| Prompt & Seed | Inference | All | Altering semantic conditioning vector. Seeds control pseudo-randomness for a specific hardware stack, but rarely guarantee absolute cross-stack reproducibility. |
| Inpainting | Inference | Image, Video | Masking the latent space during reverse diffusion to preserve specific regions. |
| Attention Editing | Inference | Text | Manipulating KV caches or attention maps directly at runtime to rewrite context without changing weights. |
| Activation Steering | Inference | Text | Adding a bias vector to hidden layer activations during the forward pass to shift tone/concept without fine-tuning. |
| LoRA | Adapters | Text, Image | Low-Rank Adaptation: injecting small trainable matrices to learn specific styles or concepts. |
| ControlNet | Adapters | Image, Video | Adding a trainable spatial-conditioning branch to a frozen diffusion model, using signals such as edges, depth, or pose. |
| RLHF / DPO | Training | Text | Updating base weights using preference data to align language model outputs with human feedback. |
Evaluation and Study Design
Interpretive intuitionWhen designing HCI studies around these systems, remember that generative output variance means a single output cannot characterize a stochastic system. Study designs must incorporate fixed seeds to control pseudo-randomness within the experiment, and systematically sample across the temperature/guidance space to measure the robustness of the human-AI interaction.
References
- Zhang et al. (2023). Adding Conditional Control to Text-to-Image Diffusion Models (ControlNet). arXiv:2302.05543