How image generation works
- ai images
- foundations
A text-to-image model takes your prompt and constructs a picture — it does not look one up. Internally it starts from random noise and refines it step by step toward something that matches your words. The useful part is simpler than the internals: generation creates a new image from scratch, while editing changes an image you already have — and most tools treat these as separate modes.
The pipeline, plainly
How it works, step by step
- Your prompt is read and encoded — the model builds an internal summary of the scene you described.
- It starts from pure noise. There is no draft picture hidden inside waiting to be revealed.
- It refines the noise in repeated passes — the diffusion process — nudging it toward your description.
- Details settle: composition, light and texture are decided during these passes, which is why small wording changes can shift large parts of the image.
- The finished image is decoded and returned. Run it again and you get a new sample, not the same picture twice.
Here is the same pipeline as a picture — five passes from noise to image:
- noise
- pass 2
- pass 3
- pass 4
- your image
The same process drawn out: each pass nudges random noise toward the prompt. There is no draft hidden underneath — every pass refines what the last one produced.
Generation vs editing — two different jobs
- Generation fits “make something new”: concepts, drafts, illustrations, mood boards.
- Editing fits “change this one thing”: keep the photo, swap the lamp. Tools support editing the whole image with a new prompt, or a selected region only — you mark the area (a mask) and describe the change. OpenAI’s image API, for example, separates generations from edits exactly this way.
A bad example
Ask: “Generate the exact photo I saw in that article last month.”
The model has no gallery to search and no memory of your browsing. It will build something plausible instead — and it will not be the photo.
A better example
Ask: “Here is a photo of our product. Keep everything identical; replace only the background with a light grey studio backdrop and a soft shadow under the object.”
An edit of material you supplied — surgical, verifiable, and exactly what edit modes are for.
Why it works
Hold two ideas and most image-tool behaviour follows: generation constructs; editing transforms. Construction tolerates variety — ask twice, get two samples. Transformation should preserve everything you did not ask to change, which is why dedicated edit modes beat re-describing a whole scene: fewer variables, less accidental drift.
Practice
Generation or editing — pick the mode
A bakery wants their product photo kept exactly as it is — same pastry, same plate — but with the background replaced by a warm café table scene. Everything about the pastry itself must stay identical.
Which mode fits the task best?
Hint
Ask: is the goal to construct something new, or to change one thing in an image that already exists?
Why this is the answer
Generation constructs from your words; editing transforms an image you supply. The moment a brief contains “keep everything else the same”, you want an edit — ideally on a selected region. That single choice is the difference between a product photo and a product photo of something that is no longer your product.
Transfer
- Product photos: edit backgrounds and light; never regenerate the product itself.
- Iteration: change one description at a time so you can tell what caused what.
- Concepts and mood boards: generate several samples on purpose — variety is the feature here.
Key takeaways
- Models construct images from prompts — nothing is looked up or retrieved.
- Generation creates from scratch; editing transforms an image you supply.
- When the brief says “keep everything else the same”, you want an edit — ideally a masked one.
Next
Next: Subject, scene, style — the four-part recipe behind every strong image prompt. The diffusion entry and generative AI cover the background.