AI Character Design: Prompt Engineering That Actually Works
You generated a gorgeous elven warrior on your first try. Then you tried to get her in a different pose — and got a completely different person. Sound familiar?
Consistency is the core challenge in AI game art. The same character must stay recognizable across poses, expressions, outfits and scenes — or your animation flickers, your UI looks off, and your art director starts twitching.
This is the 9-step framework we use to get consistent, production-ready game characters out of Midjourney, Stable Diffusion and ComfyUI — not lucky one-offs.
Step 1: Prompt Engineering Fundamentals
AI generation is probabilistic. Your prompt is the "remote control" for that probability distribution — the more precise your description, the closer you get to what you want.
Why game characters need consistency
| Reason | What happens without it |
|---|---|
| Brand identity | Players can't recognize the protagonist between scenes |
| Animation coherence | Sprite frames "flicker" when the character shifts shape |
| Asset reuse | UI icons, portraits and cutscenes all disagree |
| Team collaboration | Multiple artists produce five different versions of the same hero |
Tool overview: pick the right tool per phase
| Tool | Type | Best for |
|---|---|---|
| Midjourney | Paid · Cloud | Highest art quality, --cref character ref, --sref style ref — concept phase |
| Stable Diffusion | Open source · Local | LoRA/ControlNet, scriptable batch generation — production phase |
| DALL-E 3 | Paid · API | Prompt understanding, accurate text rendering — rapid prototyping |
| ComfyUI | Open source · Node | Visual workflows, automated pipelines — engineered mass production |
Tip: Explore styles fast in Midjourney during concept. Then move to Stable Diffusion + ComfyUI for batch production. DALL-E 3 is great for UI elements that need precise text.
Step 2: Prompt Architecture
Great prompts aren't random keyword dumps — they have a clear hierarchical structure. Four layers, from core to detail:
| Layer | What it defines | Example |
|---|---|---|
| 1. Subject | Who — identity, race, gender, body type | "a young elven female warrior, athletic build, silver hair" |
| 2. Style | How it looks — art style, rendering, palette | "anime cel-shading, bold outlines, vibrant colors" |
| 3. Detail | What they wear — equipment, clothing, hair, props | "ornate silver plate armor with blue gem inlays, crystal longsword" |
| 4. Technical | How it's presented — resolution, angle, background | "full body, front view, white background, 2D game sprite" |
Core prompt template
# Layer 1: Subject
a young elven female warrior, athletic build, silver hair, pointed ears,
# Layer 2: Style
anime cel-shading style, bold outlines, vibrant colors, game character art,
# Layer 3: Detail
wearing ornate silver plate armor with blue gem inlays,
holding a crystal longsword, flowing white cape,
# Layer 4: Technical
full body, front view, white background, high detail,
2D game sprite, transparent background
Weight syntax per tool
| Tool | Weight syntax | Notes |
|---|---|---|
| Stable Diffusion | (silver hair:1.3) |
Range 0.5-2.0 |
| Midjourney | silver hair::2 |
Default weight is 1 |
| DALL-E 3 | "prominently silver hair" | Natural-language emphasis, no explicit weights |
| ComfyUI | CLIP Text Encode nodes | Visual weight adjustment |
⚠️ Pitfall: weights above 1.8 cause artifacts and oversaturation. Keep core features at 1.2-1.5, secondary descriptions at default.
Step 3: Style Control & Consistency
Style consistency is the biggest challenge in AI game art. Lock your art direction with style anchors:
- Style keyword lock — repeat the same style descriptors in every prompt
-
Reference image anchoring — MJ
--sref, SD IP-Adapter, ComfyUI StyleModel - LoRA model lock — train a style-specific LoRA, use it everywhere
- Color anchoring — fix palette descriptions or use color reference images
Style-specific prompt differences
# Lineart / Anime
clean lineart, anime cel shading, bold black outlines, flat colors, game UI art
# Cel-shaded / Flat
flat color, cel-shaded, 2-3 tone shading, sharp edges, vibrant palette, sticker art
# Painterly / Semi-realistic
painterly style, soft brush strokes, rich gradients, semi-realistic, concept art quality
# Pixel Art
pixel art, 32x32 sprite, limited palette, no anti-aliasing, retro game style
Consistency secret: build a Style Board — collect 5-10 reference images of your target style, extract the shared keywords, and include them in every single prompt. Midjourney's
--srefaccepts style reference URLs directly.
Step 4: Character Variation & Iteration
Games need variants: different angles, expressions, outfits, body types. The key is changing only what needs to change.
Multi-angle generation — what actually works
| ✅ Recommended | ❌ Avoid |
|---|---|
Midjourney --cref character reference |
Random generation + cherry-picking |
| SD + ControlNet OpenPose | Relying on text descriptions alone |
| IP-Adapter FaceID | Mixing different tools mid-project |
| LoRA character training (15-30 images) | Not locking seed values |
| ComfyUI workflow reuse | Manual post-production stitching |
Expression variation matrix
| Expression | Prompt keywords |
|---|---|
| Happy | smiling, bright eyes, relaxed expression |
| Angry | frowning, furrowed brows, intense gaze |
| Sad | downturned mouth, teary eyes, drooping shoulders |
| Surprised | wide eyes, raised brows, open mouth |
| Determined | clenched jaw, focused eyes, confident stance |
| Fearful | wide scared eyes, tense body, defensive posture |
Outfit / age / body type variations
# Base character (LOCKED — never change this part)
a young elven warrior, silver hair, athletic build,
# Outfit variation - Battle
wearing heavy battle armor, scarred metal, war paint,
# Outfit variation - Casual
wearing casual tavern clothes, linen shirt, leather vest,
# Age variation - Veteran
older version, battle scars, weathered face, grey streaks in hair,
# Body type variation
[slim / athletic / muscular / heavy] build, adjust proportions accordingly
Key rule: only modify the part that changes. Keep every other descriptor byte-identical. Seed lock + inpainting is the most efficient combo.
Step 5: Batch Generation Workflow
A real game needs dozens to hundreds of character assets. Manual one-by-one generation isn't feasible.
Seed control
Seed is the random-number starting point of generation. Fixed seed + fixed prompt = reproducible result.
| Parameter | Recommended |
|---|---|
| Seed | Fixed for reproducibility |
| CFG scale | 7-12 |
| Steps | 20-50 |
| Batch size | 4-8 per batch, pick the best |
Prompt templating with variable substitution
Split prompts into fixed and variable parts, then batch-replace with a script:
# Template with variables
template = """
{base_character}, {action_pose},
{outfit}, {expression},
{style_keywords},
full body, front view, game sprite, transparent background
"""
# Variables
characters = ["elven warrior", "dwarf smith", "human mage"]
actions = ["idle stance", "attack pose", "walk cycle", "cast spell"]
outfits = ["battle armor", "casual clothes", "ceremonial robes"]
expressions = ["neutral", "happy", "angry", "determined"]
# Generate all combinations
total = len(characters) * len(actions) * len(outfits) * len(expressions)
print(f"Total: {total} images")
Automated batch pipeline
Character Definition → Template Assembly → Batch Generation
(JSON/YAML tables) (script substitution) (API or ComfyUI queue)
→ Quality Filtering → Post-Processing → Delivery Export
(auto-score + review) (bg removal, crop, name) (sprite pack, metadata)
Efficiency tip: ComfyUI's batch nodes can chain generate → remove bg → crop → name into one automated pipeline. Feed it a CSV and generate hundreds of assets in one click.
Step 6: Game Asset Spec Compliance
AI output must conform to your engine's technical specs. Doing this at prompt time saves you hours of post-processing.
Character size specs by game type
| Game type | Frame size | Format / notes |
|---|---|---|
| Pixel game | 32x32 / 16x16 | Integer scaling, Point filter, no anti-aliasing |
| Mobile 2D | 256x256 / 512x512 | ASTC/ETC2 compression |
| HD 2D | 512x512+ | Alpha channel, 2x scaling |
| PC HD | 1024x1024 | Mipmaps optional |
| Console / 4K | 2048x2048 | High precision |
Size control in prompts
# Pixel art — exact size in prompt
"pixel art, 32x32 sprite, limited 16-color palette, clean pixels, no anti-aliasing"
# HD 2D — specify resolution
"2D game character, 512x512, high detail, transparent PNG background"
# Midjourney — aspect ratio
"--ar 1:1 --quality 2 --stylize 100"
# Stable Diffusion — explicit resolution
"--width 512 --height 512 --sampler dpm++2m_karras --steps 30"
Sprite-sheet-friendly generation
- Unified canvas — every frame, same canvas size
-
Centered composition — prompt
centered, full body - Bottom alignment — consistent feet position for animation
- Spacing buffer — account for 2-4px to prevent color bleeding
Step 7: Post-Processing & Corrections
AI generation is rarely perfect first try. Here's the fix table we actually use:
| ❌ Issue | ✅ Fix |
|---|---|
| Wrong finger count (6 fingers) | Inpaint hands + add extra fingers to negative prompt |
| Dirty background, leftover objects |
rembg / RMBG model + check edge artifacts |
| Inconsistent character sizing | Unified crop to fixed canvas + ControlNet pose skeleton |
| Blurry or lost equipment detail | Upscale + local inpainting for detail sharpening |
| Color drift from palette | Post-process color grading + fixed-palette LoRA |
| White/black edges on transparency | Matting (MODNet) + 1px inner expansion |
Inpainting workflow
# 1. Mask the problem area (hands, accessories, etc.)
# 2. Set denoising strength: 0.4-0.7 (lower = preserve more original)
# 3. Write a focused prompt for the masked region only
# 4. Use "Only masked" mode for precision
# ControlNet modes for character control:
# - OpenPose: exact body pose
# - Canny/Lineart: preserve outline structure
# - Depth: spatial relationships
# - IP-Adapter: transfer style from reference
# - Reference: maintain character identity
Workflow advice: check in this order after every generation — fingers → edges → symmetry → colors → details. Fix by priority, never redo from scratch.
Step 8: Advanced Prompt Techniques
Negative prompting
Negative prompts tell the model what not to generate — the cheapest quality lever you have:
# Universal negative for game characters
extra fingers, missing fingers, deformed hands, bad anatomy, mutated,
extra limbs, floating limbs, disconnected limbs,
blurry, low quality, low resolution, jpeg artifacts, watermark, text,
signature, cropped, out of frame, duplicate,
# Style conflicts (when doing clean 2D)
3D render, photorealistic, realistic, watercolor, oil painting,
# Game asset specific
background elements, scenery, props, shadow on ground,
multiple characters, frame border, UI elements
Regional prompting
Specify different prompts for different image regions:
- SD Regional Prompter — split image into regions, independent prompts each
- ComfyUI Area Conditioning — node-based region control
-
MJ
--cref+--cw— control which parts of the character reference apply (face vs full body)
LoRA vs Textual Inversion
| LoRA | Textual Inversion | |
|---|---|---|
| Training data | 15-50 images | 3-5 images |
| Quality | High | Medium |
| Flexibility | High | Medium |
Best practice combo: Character LoRA (locks appearance) + Style LoRA (locks art style) + ControlNet (locks pose). Three layers = precise control.
Step 9: QA & Delivery Checklist
Before any asset ships, it passes professional QA. Here's ours:
Consistency QA standards
| Visual | Technical | Naming |
|---|---|---|
| Face consistent across frames | All frames same size | character_action_variant_frame.png |
| Palette matches spec | Clean alpha, no edge artifacts | Clear folder structure |
| Line weight unified | File size within budget | JSON metadata per batch |
| Light direction consistent | Resolution meets platform spec | — |
Naming convention
# Pattern: {character}_{action}_{variant}_{frame}.png
hero_knight_idle_front_001.png
hero_knight_attack_sword_001.png
npc_mage_cast_fire_001.png
enemy_goblin_idle_001.png
# Folder structure:
# /assets/characters/
# /hero_knight/ → /idle/ /walk/ /attack/
# /npc_mage/ → /idle/ /cast/
# /enemy_goblin/ → /idle/
QA item checklist
| Check | Method | Pass criteria |
|---|---|---|
| Face consistency | Frame-by-frame compare in sprite sheet | No visible difference |
| Size spec | Batch detection | 100% match target size |
| Alpha channel | Edge inspection | No white/black edges |
| File size | Batch statistics | Within budget |
| Naming | Regex validation | 100% match rules |
| Animation smoothness | Preview in Unity/Cocos | No jitter or flicker |
Key Takeaways
- Prompt architecture beats prompt length. Four layers: Subject → Style → Detail → Technical.
- Lock what must not change. Style keywords, seeds, and the base-character line stay byte-identical across variants.
- Change only what changes. One variable per generation; everything else frozen.
- Engineer for batch, not for singles. Templates + variable substitution + ComfyUI pipelines turn one good prompt into 500 assets.
- QA is part of the pipeline. Consistency checklist before delivery is what separates "AI slop" from shippable art.
The framework is a loop: build templates → lock style → batch generate → strict QA. Master it, and AI stops being a lucky-dip generator and becomes your art team.
This guide is part of the GameArtForge tutorial library. You'll find the full illustrated version with more examples at gameartforge.com/tutorials/ai-character-design-prompt-engineering.html. We also build free game-art tools that automate parts of this pipeline.
Got a prompt trick that saves you hours? Drop it in the comments — I read every one.
Top comments (0)