DEV Community

Cover image for Why does ChatGPT keep giving me Godot 3 code? We tested 150 answers
Ziva
Ziva

Posted on Originally published at ziva.sh

Why does ChatGPT keep giving me Godot 3 code? We tested 150 answers

ChatGPT gives you Godot 3 code when the model behind it mixes up the two versions. Godot 4.0 shipped on 1 March 2023 and renamed or removed many Godot 3 classes, methods and keywords, so KinematicBody2D, export var and yield no longer work. A current model may not do this anymore: in November 2026 we asked GPT-6 Luna, a current OpenAI model, 10 beginner Godot questions in plain chat, with "Godot 4" in the prompt, without a version, and with web search on. In 150 answers it wrote 0 Godot 3 API errors. What did go wrong was different: a few parse errors, and a few runtime-only bugs that Godot's parser accepts. If you still get Godot 3 code, use a current model, say your Godot version, and check the code in Godot.

Ziva.sh makes Ziva, an AI agent for Godot, and ran this test. We captured every answer on 10 November 2026 (UTC) and checked the code on Godot 4.7.2. We called GPT-6 Luna through OpenAI's Codex endpoint with a ChatGPT account login, not through the ChatGPT app, which can behave differently. We also ran the same prompts through Ziva on the same model; results are below.

Where the complaints come from

Only the first names a specific model version, GPT-4o. We cannot tell which models the others used, and our result says nothing about those models.

How we tested

We wrote 10 beginner prompts from forum threads: a 2D platformer player, a third-person controller, spawning enemies on a timer, saving a high score, a Start button that changes scene, fading out a coin, placing tiles from code, a health bar that follows an exported variable, random numbers, and a trap that disappears after 2 seconds. None names an API that changed between Godot 3 and 4. Example: "I'm making my first 2D platformer in Godot 4. Can you write me a player script so my character can run left and right, jump, and fall with gravity?"

Plain chat got each prompt 3 times per setup, with no project and no system prompt apart from the web-search setup's short instruction to answer from web results. An AI agent read every code block and a script scanned it for a checklist of Godot 3 forms built from the official upgrade guide and Godot 4.7.2's own upgrade-tool rename map; the same agent did both, and the read was not blind to the scan. Every script also went through Godot 4.7.2's --check-only. Every runtime error below was confirmed by running the code headless on Godot 4.7.2.

Results

Setup Godot 3 API errors Other real errors
GPT-6 Luna plain chat, prompt says "Godot 4", default reasoning GPT-6 Luna plain chat: 0 in 30 answers GPT-6 Luna plain chat: 3 answers (2 parse errors, 1 runtime error)
GPT-6 Luna plain chat, prompt says "Godot 4", low reasoning GPT-6 Luna plain chat, low reasoning: 0 in 30 GPT-6 Luna plain chat, low reasoning: none
GPT-6 Luna plain chat, prompt says only "Godot" GPT-6 Luna plain chat, no version: 0 in 30 GPT-6 Luna plain chat, no version: 2 answers (1 runtime error, 1 told to connect a signal StaticBody2D does not have)
GPT-6 Luna with web search, prompt says "Godot 4" GPT-6 Luna with web search: 0 in 30 GPT-6 Luna with web search: 2 answers (1 runtime error, 1 wrong setup instruction)
GPT-6 Luna with web search, prompt says only "Godot" GPT-6 Luna with web search, no version: 0 in 30 GPT-6 Luna with web search, no version: 2 answers (both parse errors)
Ziva on GPT-6 Luna, question prompts as above Ziva on GPT-6 Luna: 0 in 10 chats Ziva on GPT-6 Luna: 2 chats (1 parse error in chat reply code, 1 runtime error)
Ziva on GPT-6 Luna, build-it prompts ("Add the script to my project.") Ziva on GPT-6 Luna: 0 in 10 chats Ziva on GPT-6 Luna: 1 chat (runtime error)
Ziva on GPT-6 Luna, build-it prompts, only "Godot" Ziva on GPT-6 Luna: 0 in 10 chats Ziva on GPT-6 Luna: 2 chats (1 runtime error, 1 instruction that breaks the script)

The code used Godot 4 forms throughout: @export, @onready, CharacterBody2D with the velocity property and move_and_slide() without arguments, instantiate(), pressed.connect(...), create_tween(), change_scene_to_file(), TileMapLayer.set_cell(), randi_range() and await ... .timeout.

Without a version in the prompt and without search, 18 of 30 plain-chat answers said they target Godot 4 and the rest named no version; none targeted Godot 3. With web search on, every search GPT-6 Luna ran started with site:docs.godotengine.org/en/stable, its own choice. Docs pages for Godot 3 or older came back in the search results of 23 of 60 answers, no answer cited one, and none of those answers had Godot 3 code.

Zero is not proof of never. If every answer were independent, 0 in 150 would put the rate below about 2% with 95% confidence; answers to the same prompt are not independent, so the real uncertainty is larger.

What went wrong instead

Seven of the 150 plain-chat answers had code that does not work in Godot 4.7.2: four fail to parse and three parse cleanly and break when they run. Two more gave wrong setup instructions. Three examples.

1. Turning off an Area2D inside its own signal. This line appeared in five places across plain chat and Ziva, in the coin and trap prompts:

func _on_body_entered(_body: Node2D) -> void:
    collected = true
    monitoring = false
Enter fullscreen mode Exit fullscreen mode

--check-only accepts it. When the player touches the coin, Godot 4.7.2 prints this and monitoring stays true:

ERROR: Function blocked during in/out signal. Use set_deferred("monitoring", true/false).
Enter fullscreen mode Exit fullscreen mode

Use set_deferred("monitoring", false), as several other answers did.

2. A setter that touches a node before it exists. One web-search answer wrote this health bar script:

@export var health: float = 100.0:
    set(value):
        health = clampf(value, 0.0, max_health)
        $ProgressBar.max_value = max_health
        $ProgressBar.value = health
Enter fullscreen mode Exit fullscreen mode

It parses. Set Health to 50 in the Inspector and run the scene: the saved value is applied before the ProgressBar child is added, so the setter fails and the bar shows 0 until health changes again in code:

ERROR: Node not found: "ProgressBar" (relative to "Node").
SCRIPT ERROR: Invalid assignment of property or key 'max_value' with value of type 'float' on a base object of type 'null instance'.
Enter fullscreen mode Exit fullscreen mode

Wrap the two $ProgressBar lines in if is_node_ready(): and update the bar once in _ready().

3. A type Godot cannot infer. Four answers failed to parse, three of them on an untyped loop variable in tile-placement code, like this:

for direction in [Vector2i.UP, Vector2i.DOWN, Vector2i.LEFT, Vector2i.RIGHT]:
    var neighbor := cell + direction
Enter fullscreen mode Exit fullscreen mode
Parse Error: Cannot infer the type of "neighbor" variable because the value doesn't have a set type.
Enter fullscreen mode Exit fullscreen mode

for direction: Vector2i in [...] fixes it. Unlike the first two, --check-only catches this one.

What Godot's parser catches, and what it does not

No answer used Godot 3 code, so this list comes from running each Godot 3 form through Godot 4.7.2 ourselves.

Godot 4.7.2's --check-only rejects these, so a parse check finds them:

  • export var, onready var, tool, setget, yield and remote func
  • button.connect("pressed", self, "_on_pressed"): "Invalid argument for "connect()" function: argument 2 should be "Callable" but is ..."
  • File.new(), Directory.new(), OS.get_ticks_msec(), rand_range(), update(), BUTTON_LEFT
  • .empty() on a typed Array
  • move_and_slide(velocity, Vector2.UP) and velocity = move_and_slide()
  • classes such as KinematicBody2D, Spatial, Position2D and PoolStringArray
  • rect_position and translation on the script's own node

These pass Godot 4.7.2's --check-only and fail only when the line runs:

  • scene.instance(): "Invalid call. Nonexistent function 'instance' in base 'PackedScene'."
  • get_tree().change_scene() and change_scene_to()
  • Tween.new() and the Tween node's interpolate_property(): "Tween can't be created directly. Use create_tween() method."
  • yield(get_tree(), "idle_frame") rewritten as await get_tree().idle_frame
  • TileMap set_cellv() and world_to_map()
  • rect_position and .empty() on an untyped variable

Check AI-written GDScript in one line

Parse and analyze one script without opening the editor:

godot --headless --path /path/to/your/project --check-only --script res://player.gd
Enter fullscreen mode Exit fullscreen mode

It prints SCRIPT ERROR: Parse Error: ... with the line and exits with code 1. A clean script exits with 0. Replace godot with the path to your Godot 4 executable.

For the runtime-only list, run the scene. In the editor, press F6 and read the Output and Debugger panels. Headless:

godot --headless --path /path/to/your/project --quit-after 120 res://main.tscn
Enter fullscreen mode Exit fullscreen mode

--quit-after 120 stops after 120 frames. Read the output for SCRIPT ERROR lines: in our run a scene calling change_scene() printed "Invalid call. Nonexistent function 'change_scene' in base 'SceneTree'." and Godot still exited with code 0.

If you still get Godot 3 code

  • Use a current model. The complaint above that names a model is about GPT-4o in 2024; GPT-6 Luna wrote none in our test. We did not test older models side by side.
  • Say your exact version, such as "Godot 4.7". It made no difference for GPT-6 Luna, but a chat model cannot see your project.
  • Check the code in Godot with the two commands above before you build on it.
  • Give your agent a rename table. Our free, MIT-licensed godot-skills pack includes a godot-4-api-pitfalls skill with a Godot 3 to 4 rewrite table and the list of Godot 3 lines that pass --check-only and fail at runtime. It works with agents that read the Agent Skills format; see Godot AI skills.

Where Ziva fits

Ziva is our AI agent inside the Godot editor. It is proprietary and has a free Hobby plan. We ran the same 10 prompts through Ziva on the same model, GPT-6 Luna, one chat per prompt, in three sets of 10. Ziva also wrote 0 Godot 3 API errors.

  • Asked a question, Ziva mostly answered in chat. With the prompts above ("How do I ...? Please write the script."), Ziva wrote a file in 1 of 10 chats and replied in chat in the other 9. Ziva does not check code in a chat reply.
  • Asked to build it, Ziva wrote the file and checked it. With the same prompts reworded as instructions ("Add the script to my project."), Ziva wrote a .gd file in all 20 chats. After each write, Ziva's check returned "No problems in this file or its callers." on all 21 writes in the build-it chats.
  • The check is a parse, not a run. It is Godot's GDScript language server plus the editor's script reload, the same kind of check as --check-only. Two of those files had the monitoring = false bug from example 1 and still got "No problems". In a separate probe, a script with instance(), change_scene(), Tween.new() and set_cellv() also got "No problems".

So this test did not show Ziva's check catching a Godot 3 error, because there were none to catch, and the check cannot see runtime-only errors. Running the scene and reading its output catches those. Ziva can run scenes too, but that was not part of this test.

Limits

  • One model, GPT-6 Luna, on one date. Older models, other providers and the ChatGPT app were not tested.
  • Plain chat used OpenAI's Codex endpoint, and web search used that endpoint's search tool with instructions to answer from web results. The model searched only the Godot docs, so old forum answers never came up.
  • 10 prompts that we wrote, 3 answers per prompt in each plain-chat setup and 1 chat per prompt in each Ziva set.
  • We added the "add the script to my project" Ziva prompts after seeing that Ziva answered the question prompts in chat, so the two Ziva sets are not a planned comparison. The no-version and web-search sets were also added after the first results, each written down before it ran.
  • Ziva ran with low reasoning effort and plain chat with the default; the low-reasoning plain chat set covers that gap. Ziva tells the model the Godot version in its own instructions, so its "only Godot" set still had a version.
  • An AI agent, not a person, read the code, and the same agent ran the checklist scan, so the read was not blind to the scan.
  • For the build-it Ziva runs we moved the second editor's language-server port, because two Godot editors were open on the test machine and Ziva's check otherwise queried the other editor, as it did for the one file written in the question set. We filed that as a Ziva bug.

Related reading

FAQ

Why does ChatGPT keep giving me Godot 3 code when I'm using Godot 4?

Godot 4.0 (March 2023) renamed or removed many Godot 3 classes, methods and keywords, and a model that mixes up the two versions writes code Godot 4 rejects. The forum complaints are real; the one we found that names a specific model version is about GPT-4o, in October 2024. In our November 2026 test, GPT-6 Luna wrote 0 Godot 3 API errors in 150 plain-chat answers to 10 beginner Godot prompts, with Godot 4 in the prompt, without a version, and with web search on. If you still get Godot 3 code, use a current model, say your Godot version, and check the code in Godot.

Does it help to write "Godot 4" in the prompt?

Not measurably for GPT-6 Luna in our test: it wrote 0 Godot 3 API errors in 30 answers with Godot 4 in the prompt and 0 in 30 answers that said only Godot. Without a version, 18 of the 30 answers said they target Godot 4 and none targeted Godot 3. Stating the version still costs nothing, and a plain chat model has no other way to know which Godot you use.

Does web search make ChatGPT write Godot 3 code?

Not in our test. With web search on, GPT-6 Luna wrote 0 Godot 3 API errors in 60 answers. Every search it ran was restricted to the Godot docs site, and docs pages for Godot 3 or older came back in 23 of the 60 answers, but no answer cited one. We used the search tool on OpenAI's Codex endpoint, not the ChatGPT app, and no forum or Reddit page was ever returned, so this does not test old forum answers.

How do I check if AI-written GDScript is valid Godot 4?

Run godot --headless --path /path/to/project --check-only --script res://player.gd with Godot 4. It exits with code 1 and prints Parse Error lines for removed keywords such as export, onready and yield, renamed classes such as KinematicBody2D, and the old connect() form. Some Godot 3 code passes this check and only fails when the line runs, such as instance(), change_scene() and Tween.new(), so also run the scene and read the output.

Which Godot 3 code does Godot 4 only catch at runtime?

On Godot 4.7.2 these pass --check-only and fail when the line runs: PackedScene.instance(), get_tree().change_scene() and change_scene_to(), Tween.new() and the Tween node's interpolate_property, the idle_frame signal, TileMap set_cellv and world_to_map, and rect_position or .empty() on an untyped variable.

What mistakes did GPT-6 Luna make if not Godot 3 code?

Across 150 plain-chat answers, 4 had parse errors (for example var neighbor := cell + direction, where Godot 4.7.2 cannot infer the type) and 3 had runtime-only bugs that pass the parser: setting monitoring = false inside an Area2D body_entered handler, which Godot blocks, and a health setter that touches $ProgressBar before the node exists. Two more gave wrong setup instructions.

Does Ziva stop Godot 3 code?

This test cannot show that. Ziva, running the same GPT-6 Luna model, wrote 0 Godot 3 API errors in 30 chats, so there was nothing for its check to catch. In the build-it chats, its check (Godot's GDScript language server plus the editor's script reload) returned No problems on all 21 writes, including two scripts with a runtime-only bug, because the check parses and analyzes the script without running it.

Top comments (0)