Gitlab Runner errors and solutions

Error 1) open /root/.ssh/known_hosts: no such file or directory

Using SSH executor...

ERROR: Preparation failed: ssh command Connect() error:
getting host key callback: open /root/.ssh/known_hosts: 
no such file or directory

Will be retried in 3s ...

Solution:

Follow the steps to resolve:

1) Login to gitlab instance via SSH
2) Become sudo via:

sudo su

3) Now, you need to connect gitlab instance to the host where runner is try to connect

ssh <host-username>@<host-ip>

<host-username> and <host-ip> should match with the gitlab runner, it will ask for password then it will ask to accept key fingerprint.

Now, try to run the job with the runner. It should be working

---

Error 2) Job failed: prepare environment: Process exited with status 1.

If you are getting following error in your when running gitlab ci/cd job via gitlab-runner:

ERROR: Job failed: prepare environment: Process exited with status 1. 
Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading 
for more information

Solution:

Run following commands on the server where gitlab runner trying to login:

find / -name .bash_logout

and delete following files if exist

sudo rm -r /home/gitlab-runner/.bash_logout
sudo rm -r /home/<username>/.bash_logout

Try to re-run the jobs it should be working.

---

Error 3) handshake failed: knownhosts: key is unknown

ERROR: Preparation failed: ssh command Connect() error: ssh Dial() error: ssh: handshake failed: knownhosts: key is unknown

Solution:

Solution A

Verify your login credentials

Solution B

Verify that SSH port is open

Solution C

Edit your runner and add disable_strict_host_key_checking = true

sudo nano /etc/gitlab-runner/config.toml

[[runners]]
  name = "..."
  url = "..."
  token = "..."
  executor = "ssh"
  [runners.custom_build_dir]
  [runners.cache]
    [runners.cache.s3]
    [runners.cache.gcs]
    [runners.cache.azure]
  [runners.ssh]
    user = "..."
    password = "..."
    host = "..."
    port = "..."
    disable_strict_host_key_checking = true

Then restart the gitlab-runner

sudo gitlab-runner restart

Solution D

If you're using WHM as your hosting control panel, enable following settings:

SSH Password Authorization Tweak

Comments

[Matías Iturburu]

I know this is not a support forum but here it goes anyway.

I'm running a local runner to deploy in my droplet.
Using the root account I registered my runner and I see it creates the config file in /home/gitlab-runner/.gitlab-runner/config.toml.

I registered an ssh executor and as the gitlab-runner user I can log in as the application user "deploy" which matches my configuration.

But I can't get it to work. It fails everytime with:

ERROR: Job failed (system failure): ssh command Connect() error: ssh Dial() error: ssh: handshake failed: knownhosts: key mismatch

I'm lost. I've been using this setup in several projects in the past. Can't understand if it's some recent change or something specific to digitalocean.

Any help is welcome.

[Pradeep Kumar]

@7u7uca did you try Solution C

[Matías Iturburu]

Yes, didn't work. I don't know what else to try. I just enabled the IPv6 support on my hosting as that was blocking the ssl support from let's encrypt. Prolly something along those lines is messing the connection.

I'll be sure to report back once I find what's broken.

Thanks for the response!

[Pradeep Kumar]

Let me give you another simple hack to try, ssh to the machine where you installed gitlab-runner and they try to ssh the target IP on which gitlab-runner trying to deploy the code. When you ssh the target ip, it will prompt you to type yes, and you do that. Once it's done try to run the job, hopefully it will work.

[Matías Iturburu]

Yes, it seems like the issue relies when the runner gets triggered, when trying to connect to gitlab.com.

Ssh from the agent host to the web host works fine.

[Joy Kumar]

Many Many Thanks.