DEV Community

Cover image for Install python 3.8.0 via pyenv on BigSur
0xkoji
0xkoji

Posted on • Updated on

Install python 3.8.0 via pyenv on BigSur

Photo by Ganapathy Kumar on Unsplash

I’m using macOS11(beta), BigSur.
I think BigSur is cool but, I have faced many issues that I haven’t had before.

  • Crash so many times
  • Cannot install some software as well as macOS 10 In this article, I will show you how I could install python via pyenv on BigSur. The following was the issue I had.
$ pyenv install 3.7.6
python-build: use openssl@1.1 from homebrew
python-build: use readline from homebrew
Downloading Python-3.7.6.tar.xz...
-> https://www.python.org/ftp/python/3.7.6/Python-3.7.6.tar.xz
Installing Python-3.7.6...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0 using python-build 20180424)

Inspect or clean up the working tree at /var/folders/gj/x6v5vwdx1v7741fdfcxwmr100000gn/T/python-build.20200830033458.15319
Results logged to /var/folders/gj/x6v5vwdx1v7741fdfcxwmr100000gn/T/python-build.20200830033458.15319.log

Last 10 log lines:
    struct sf_hdtr sf;
           ^
./Modules/posixmodule.c:8401:15: error: implicit declaration of function 'sendfile' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
        ret = sendfile(in, out, offset, &sbytes, &sf, flags);
              ^
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include   -I/Applications/Xcode-beta.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include   -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration   -I. -I./Include -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/Users/koji/.pyenv/versions/3.7.6/include  -I/usr/local/opt/readline/include -I/usr/local/opt/readline/include -I/Users/koji/.pyenv/versions/3.7.6/include    -c ./Modules/pwdmodule.c -o Modules/pwdmodule.o
2 errors generated.
make: *** [Modules/posixmodule.o] Error 1
make: *** Waiting for unfinished jobs....
1 warning generated.
Enter fullscreen mode Exit fullscreen mode

According to the wiki on GitHub, basically this issue will be solved by installing zlib.
So, I installed zlib via Homebrew

$ brew install zlib
$ export LDFLAGS="-L/usr/local/opt/zlib/lib" 
$ export CPPFLAGS="-I/usr/local/opt/zlib/include
Enter fullscreen mode Exit fullscreen mode

Maybe some of you will need to install the following.

$ brew install readline xz
Enter fullscreen mode Exit fullscreen mode

However, I got the same error again… I did some research on this issue.
I also tried reinstalling Xcode command-line tools, but it didn’t solve the issue…
Then finally, I figured out the solution.

There are 2 steps to resolve this issue.

Step 1 Align command-line tools

In my case, I use Xcode-beta.app
I needed to check the version of command-line tools matches Xcode.

  1. Open Xcode-beta.app
  2. Go to Preference > Locations
  3. Select the right version of command-line tools (* I use Xcode-beta.app version 5 and command-line tools version 5)

Alt Text

Step 2 Install python

In this case, I installed 3.8.0. If you want to install a different version, you will need to change the version in the following command.

$ CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.8.0 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)
Enter fullscreen mode Exit fullscreen mode
$ pyenv versions
* system (set by /Users/koji/.pyenv/version)
  3.8.0
Enter fullscreen mode Exit fullscreen mode

Hopefully, this will be useful to some of you!!!

Top comments (18)

Collapse
 
ruevaughn profile image
Chasej

Thanks!!

Collapse
 
mj profile image
Michael Jackson

Does pyenv install 3.9.0 work for you, then? I've followed all your steps and still get:

michaeljackson@Michaels-MacBook-Pro ~ % pyenv install 3.9.0                                
Downloading openssl-1.1.0j.tar.gz...
-> https://www.openssl.org/source/old/1.1.0/openssl-1.1.0j.tar.gz
Installing openssl-1.1.0j...
Installed openssl-1.1.0j to /Users/michaeljackson/.pyenv/versions/3.9.0

python-build: use readline from homebrew
Downloading Python-3.9.0.tar.xz...
-> https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tar.xz
Installing Python-3.9.0...
python-build: use readline from homebrew
python-build: use zlib from xcode sdk

BUILD FAILED (OS X 11.0 using python-build 1.2.21)

Inspect or clean up the working tree at /var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241
Results logged to /var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241.log

Last 10 log lines:
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 210, in _main
    return _bootstrap(
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 129, in _bootstrap
    return _run_pip(args + [p[0] for p in _PROJECTS], additional_paths)
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/ensurepip/__init__.py", line 38, in _run_pip
    return subprocess.run([sys.executable, "-c", code], check=True).returncode
  File "/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/Lib/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/private/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/python-build.20201019140810.65241/Python-3.9.0/python.exe', '-c', '\nimport runpy\nimport sys\nsys.path = [\'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t/setuptools-49.2.1-py3-none-any.whl\', \'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t/pip-20.2.3-py2.py3-none-any.whl\'] + sys.path\nsys.argv[1:] = [\'install\', \'--no-cache-dir\', \'--no-index\', \'--find-links\', \'/var/folders/yq/nv901ffx2bx6ynbszjbn46b80000gn/T/tmpzx8xka5t\', \'--root\', \'/\', \'--upgrade\', \'setuptools\', \'pip\']\nrunpy.run_module("pip", run_name="__main__", alter_sys=True)\n']' returned non-zero exit status 1.
make: *** [install] Error 1
michaeljackson@Michaels-MacBook-Pro ~ % 
Enter fullscreen mode Exit fullscreen mode
Collapse
 
craiga profile image
Craig Anderson

For me, the problem was that I needed to have zlib (and possibly openssl@.1.1 and readline) installed from homebrew, and then having the following environment variable set.

export LDFLAGS="-L/usr/local/opt/readline/lib -L/usr/local/opt/openssl@1.1/lib -L/usr/local/opt/zlib/lib"
Enter fullscreen mode Exit fullscreen mode

I also had the following set, though I'm not sure if they're required.

export CPPFLAGS="-I/usr/local/opt/readline/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/zlib/include"
export PKG_CONFIG_PATH="/usr/local/opt/readline/lib/pkgconfig:/usr/local/opt/openssl@1.1/lib/pkgconfig:/usr/local/opt/zlib/lib/pkgconfig"
Enter fullscreen mode Exit fullscreen mode

Hopefully this helps!

Collapse
 
eshack94 profile image
Elijah Shackelford • Edited

I'm having this same issue after re-installing pyenv (no issues prior to re-installing, but this is the first time I've needed to re-install after upgrading to Big Sur, so I think it's related). Trying your solution now.

Edit: Just reporting back — this worked for me. Thank you very much.

Others who come across this might also want to install and link bzip2 prior to compiling.

I.e.,

brew install bzip2
export LDFLAGS="-L/usr/local/opt/bzip2/lib"
export CPPFLAGS="-I/usr/local/opt/bzip2/include"
Enter fullscreen mode Exit fullscreen mode
Thread Thread
 
jaclu profile image
Jacob Lundqvist • Edited

This is a snippet I use on all my machines, if you don't always use brew you could always wrap it in an outer check. And yes the -O2 might not be for everybody, then just skip it :)
This one will adhere to whatever location you might have used for Linux Homebrew, on Darwin you would always use /usr/local, since otherwise a lot of brew stuff fails, and it would be a pointless pain to workaround.

comment: Darwin doesn't set this variable, Linux HomeBrew does

if [ "$HOMEBREW_PREFIX" = "" ]; then
HOMEBREW_PREFIX="/usr/local"
fi

comment: Set the env to ensure brew can build stuff

export LDFLAGS="-L$HOMEBREW_PREFIX/opt/zlib/lib -L$HOMEBREW_PREFIX/opt/bzip2/lib"
export CPPFLAGS="-I$HOMEBREW_PREFIX/opt/zlib/include -I$HOMEBREW_PREFIX/opt/bzip2/include"
export CFLAGS="-O2"

Collapse
 
codebrotha profile image
Tineyi Takawira

Thank you, this worked!

Collapse
 
0xkoji profile image
0xkoji

pyenv install 3.9.0 won't work since there is no 3.9.0.
only 3.9.0b5, 3.9-dev

Collapse
 
ruslanskira profile image
Ruslan Skira

I'm still stuck with the same problem(( M1 mac

Collapse
 
0xkoji profile image
0xkoji

What is your issue?
Installing pyenv or install python via pyenv?

Collapse
 
ruslanskira profile image
Ruslan Skira

awesome You are here).

  1. pyenv install 3.7.6
  2. BUILD FAILED (OS X 11.2.1 using python-build 20180424)
Thread Thread
 
0xkoji profile image
0xkoji

I saw some major versions don't work on M1.
Maybe you can try to install python 3.9.1

Thread Thread
 
ruslanskira profile image
Ruslan Skira

I have 3.9.1
but the project on 3.6.9.
and during pyenv install occurs errors with my python.
and I should use 3.6.9

and couldn't solve this issue.

Thread Thread
 
0xkoji profile image
0xkoji

Did you try the following?

$ CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.9.1 < <(curl -sSL github.com/python/cpython/commit/8... install miniforge3-4.9.2full_index=1)

Thread Thread
 
ruslanskira profile image
Ruslan Skira • Edited

Yes I trying.
answer

pyenv: /Users/ruslanskira/.pyenv/versions/3.9.1 already exists )))

but for 3.6.9
patch unexpectedly ends in middle of line
patch: **** Only garbage was found in the patch input.

BUILD FAILED (OS X 11.2.1 using python-build 20180424)

what I found
github.com/pyenv/pyenv/issues/1643
it is open issue.

Thread Thread
 
0xkoji profile image
0xkoji

Please check this out. Actually I can install 3.6.9 on BigSur.
dev.to/kojikanao/install-python-3-...

Collapse
 
akashayatet profile image
akashayatet

Is brew readline xz supposed to be brew install readline xz? Still having trouble downgrading to 3.8.0 for Tensorflow compatibility. Am stuck on 3.9.1 currently.

Collapse
 
0xkoji profile image
0xkoji

Good catch, thank you!
You mean you haven't installed 3.8.0?
Are you using an M1 Mac?

Collapse
 
colinrhys profile image
Colin Rhys • Edited

Hey thank you for writing and sharing this. This worked for me (installed 3.7.3)