How do you debug a Python script that crashes Python itself?

TL;DR

Use faulthandler.enable(); Python will print the stack trace to stderr before fully crashing.

Backstory

Today I tried to daemonize a Python script that calls request.get on macOS, and it was insta-dying for no apparent reason.
I tested without daemonization, but the problem only popped up when I daemonize it.

I was doing something like this:

from daemon import DaemonContext
from daemon.pidfile import PIDLockFile
import requests
import time


pid = PIDLockFile(path.join(home_dir, 'pidfile_watcher'))
fp  = open("out.log", "a+")
fpe = open("err.log", "a+")

with DaemonContext(
  pidfile=pid,
  stdout=fp,
  stderr=fpe
):
  while True:
    response = requests.get("https://example.com");
    print(response.text)
    time.sleep(60*10)

fp.close()
fpe.close()

Many researches, print()s and one final check on Console.app (the macOS standard logger) later, I found that it was segfaulting at requests.get().

Why did it take so long?

Because daemonized script don't properly log its death even when you pass a file pointer for stderr.

Also, crash report found in Console.app does not provide much information unless you write most of your program (note the word 'program' - this includes not only your script, but also Python packages and/or even Python itself)! So the only realistic bet may be to spam print(), which is already tedious because you need to properly tell DaemonContext to redirect stdout to a file (I didn't give any at first, which caused even more headaches).

The solution

It was faulthandler.enable() that I needed to call.

faulthandler — Dump the Python traceback — Python 3.11.3 documentation

This module contains functions to dump Python tracebacks explicitly, on a fault, after a timeout, or on a user signal. Call faulthandler.enable() to install fault handlers for the SIGSEGV, SIGFPE, ...

docs.python.org

This causes Python to print stack trace to stderr when signal is received (in macOS, segmentation fault causes a signal SIGSEGV; probably all *nix systems also does.) instead of straight-up dying on the spot. This method enabled me to pinpoint exactly which call was causing my script to crash.

What was the problem, by the way?

Turns out, this is a known, (probably) unavoidable issue when using request package in macOS.

To avoid this issue, you must drop proxy support as segfault occurs at process related to polling OS's proxy configuration, which is "not fork safe".

import os
os.environ['no_proxy'] = "*"

Comments

[KT TechLearn]

Hello Everyone ,

I am trying to build a machine learning model for my understanding .I am getting exception when running and trying to build model for training data. I see below exception during execution of programme.

**Exception Trace :

C:\python-workspace\venv\lib\site-packages\sklearn\utils\validation.py:976: FutureWarning: Arrays of bytes/strings is being converted to decimal numbers if dtype='numeric'. This behavior is deprecated in 0.24 and will be removed in 1.1 (renaming of 0.26). Please convert your data to numeric values explicitly instead.
estimator=estimator,
Traceback (most recent call last):
File "C:\python-workspace\venv\lib\site-packages\sklearn\utils\validation.py", line 787, in check_array
array = array.astype(np.float64)
ValueError: could not convert string to float: 'male'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\python-workspace\NTLRApp.py", line 113, in
lr.fit(X_train, y_train)
File "C:\python-workspace\venv\lib\site-packages\sklearn\linear_model_base.py", line 663, in fit
X, y, accept_sparse=accept_sparse, y_numeric=True, multi_output=True
File "C:\python-workspace\venv\lib\site-packages\sklearn\base.py", line 581, in _validate_data
X, y = check_X_y(X, y, *check_params)
File "C:\python-workspace\venv\lib\site-packages\sklearn\utils\validation.py", line 976, in check_X_y
estimator=estimator,
File "C:\python-workspace\venv\lib\site-packages\sklearn\utils\validation.py", line 792, in check_array
) from e
ValueError: Unable to convert array of bytes/strings into decimal numbers with dtype='numeric'
*

Could you please look at above trace and suggest me what wrong , i have done in the piece of code.

[Jean-Michel 🕵🏻‍♂️ Fayard]

Try to Google/Stackoverflow the last line

Unable to convert array of bytes/strings into decimal numbers with dtype='numeric'

[KT TechLearn]

Hi , I tried to understand and facing some doubts about error which getting during execution.

Below is my dtypes :

age int64
sex object
bmi float64
children int64
smoker object
region object
expenses float64
dtype: object

Index(['age', 'bmi', 'children', 'expenses', 'sex_female', 'sex_male',
'smoker_no', 'smoker_yes', 'region_northeast', 'region_northwest',
'region_southeast', 'region_southwest'],
dtype='object')

Error : ValueError: could not convert string to float: 'male'

Question/Query : I am not able to understand why programme is trying to convert object or string into float data type. What is the need to perform conversion here.

[Jean-Michel 🕵🏻‍♂️ Fayard]

The odd that you find a specialist of your problem at hand here is very slim.
This kind of super focused question work better at stackoverflow.com/