Discussion on: Help on Django Deployment

View post

Hi Dennis, you're not giving much info so it's hard to help by just reading your post. You have to make it easy for people to help you :D

Did you save a backtrace? What kind of error are you having? What version of each library are you using? Which problems did you have with cairocffi? Did you try to check their github (especially the closed issues) to see if anyone had your same error?

You mention you switched from WeasyPrint to xhtml2pdf, where does it show the encoding error? Can you isolate it to a reproducible example? What's "unsupported type" error?

Maybe you just have to install compatible libraries in your container.

Anyhow, start from the beginning: you need an error backtrace (or traceback in Python) and a list of packages with versions :D

Dennis Quesada Cruz • Apr 20 '19

Yes, this is the traceback error when i use xhtml2pdf view:

UnicodeEncodeError at /generatepdf
'latin-1' codec can't encode character '\ufeff' in position 0: ordinal not in range(256)
Request Method: GET
Request URL: 127.0.0.1:8000/generatepdf
Django Version: 2.0.6
Exception Type: UnicodeEncodeError
Exception Value:

'latin-1' codec can't encode character '\ufeff' in position 0: ordinal not in range(256)
Exception Location: /home/dcruz/PycharmProjects/Clientes/misclientes/utils.py in render_to_pdf, line 12
Python Executable: /usr/bin/python3
Python Version: 3.6.5
Python Path:

['/home/dcruz/PycharmProjects/Clientes',
'/usr/lib/python36.zip',
'/usr/lib/python3.6',
'/usr/lib/python3.6/lib-dynload',
'/home/dcruz/.local/lib/python3.6/site-packages',
'/usr/local/lib/python3.6/dist-packages',
'/usr/local/lib/python3.6/dist-packages/Django-2.0.6-py3.6.egg',
'/usr/local/lib/python3.6/dist-packages/django_templated_email-2.3.0-py3.6.egg',
'/usr/local/lib/python3.6/dist-packages/django_render_block-0.5-py3.6.egg',
'/usr/lib/python3/dist-packages']
Server time: Sáb, 20 Abr 2019 14:46:22 -0400
Unicode error hint
The string that could not be encoded/decoded was: <ht

I think there is a rare character at the first position that makes the error.. :S

rhymes • Apr 20 '19 • Edited

So, the error is telling you something. I don't know what you're encoding nor I can see the code you're writing but the error as you guessed means the first character is outside latin-1.

According to the search engine \ufeff is the byte order mark.

You should try to encode the text with UTF-8:

>>> print("\ufeffTEXT".encode("latin-1").decode("latin-1"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeEncodeError: 'latin-1' codec can't encode character '\ufeff' in position 0: ordinal not in range(256)
>>> print("\ufeffTEXT".encode("utf-8").decode("utf-8"))
TEXT

I think you're doing something wrong with encoding/decoding the text before you transform it.

Can you show me how are you opening the source file? Also check what encoding does it have on disk:

➜  ~ cat > foo.txt
€®™
➜  ~ file foo.txt
foo.txt: UTF-8 Unicode text

Dennis Quesada Cruz • Apr 20 '19

Im saving the html file as utf-8 right now and there is another error.

Environment:

Request Method: GET
Request URL: 127.0.0.1:8000/generatepdf

Django Version: 2.0.6
Python Version: 3.6.5
Installed Applications:
['django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django.contrib.sites',
'misclientes.apps.MisclientesConfig']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django.middleware.clickjacking.XFrameOptionsMiddleware']

Traceback:

File "/usr/local/lib/python3.6/dist-packages/Django-2.0.6-py3.6.egg/django/core/handlers/exception.py" in inner

response = get_response(request)

File "/usr/local/lib/python3.6/dist-packages/Django-2.0.6-py3.6.egg/django/core/handlers/base.py" in _get_response

response = self.process_exception_by_middleware(e, request)

File "/usr/local/lib/python3.6/dist-packages/Django-2.0.6-py3.6.egg/django/core/handlers/base.py" in _get_response

response = wrapped_callback(request, *callback_args, **callback_kwargs)

File "/usr/local/lib/python3.6/dist-packages/Django-2.0.6-py3.6.egg/django/views/generic/base.py" in view

return self.dispatch(request, *args, **kwargs)

File "/usr/local/lib/python3.6/dist-packages/Django-2.0.6-py3.6.egg/django/views/generic/base.py" in dispatch

return handler(request, *args, **kwargs)

File "/home/dcruz/PycharmProjects/Clientes/misclientes/views.py" in get

pdf = render_to_pdf('listapdf.html', context)

File "/home/dcruz/PycharmProjects/Clientes/misclientes/utils.py" in render_to_pdf

pdf = pisa.pisaDocument(BytesIO(html.encode("ISO-8859-1")), result)

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/document.py" in pisaDocument

encoding, context=context, xml_output=xml_output)

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/document.py" in pisaStory

pisaParser(src, context, default_css, xhtml, encoding, xml_output)

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/parser.py" in pisaParser

context.parseCSS()

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/context.py" in parseCSS

self.css = self.cssParser.parse(self.cssText)

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/w3c/cssParser.py" in parse

src, stylesheet = self._parseStylesheet(src)

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/w3c/cssParser.py" in _parseStylesheet

src, atResults = self._parseAtKeyword(src)

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/w3c/cssParser.py" in _parseAtKeyword

src, result = self._parseAtPage(src)

File "/home/dcruz/.local/lib/python3.6/site-packages/xhtml2pdf/w3c/cssParser.py" in _parseAtPage

stylesheetElements.extend(atResults)

Exception Type: TypeError at /generatepdf
Exception Value: 'NotImplementedType' object is not iterable

NotImplementedType object is not iterable.. and there is all :(

Here is my utils.py code: (pretty the same as the manual example)

from io import BytesIO
from django.http import HttpResponse
from django.template.loader import get_template

from xhtml2pdf import pisa

def render_to_pdf(template_src, context_dict={}):
template = get_template(template_src)
html = template.render(context_dict)
result = BytesIO()
pdf = pisa.pisaDocument(BytesIO(html.encode("ISO-8859-1")), result)
if not pdf.err:
return HttpResponse(result.getvalue(), content_type='application/pdf')
return None

I don't know how to use markdown sorry for the unformatted thing

kip • Apr 20 '19 • Edited

What version of xhtml2pdf are you using ?

I the docs you can see an example, but in your shared code I dont see nothing like that example.

Maybe you use this guide, in that case you need to set the correct encondig because by default the library take this based from the HTML header data, something like this:

pdf = pisa.pisaDocument(BytesIO(html.encode("ISO-8859-1")), dest=result, encoding="ISO-8859-1")

But if your HTML file is UTF8 you need to encode that and pass the correct encoding argument to pisaDocument.

Dennis Quesada Cruz • Apr 21 '19

I'm using xhtml2pdf version 0.2.3.
Still getting
NotImplementedType object its not iterable
This gonna make me crazy guys..

kip • Apr 21 '19 • Edited

Could be a bug in the xhtml2pdf library.

But if you go to the master branch you can see the correct lines

So just install directly from Github:

pip install https://github.com/xhtml2pdf/xhtml2pdf/archive/master.zip

Dennis Quesada Cruz • Apr 21 '19

I'll try this next mon. Right now i can't download files. Thanks a lot @Kip

Dennis Quesada Cruz • Apr 22 '19

thepracticaldev.s3.amazonaws.com/i...

I've installed it but same error.... :(

I don't know what else i can do... im going to install Deepin 15.9 on my server to try WeasyPrint again. :S but i know that ins't a good deployment practice :S

kip • Apr 22 '19 • Edited

You use pip3 but when you run the Django app what version are you using ? Maybe the xhtml2pdf in your app was taken from the python2 packages folder...

My advice is you need to use virtualenv or maybe the venv built-in module in >= python3.3 and with that you ensure the packages installed via pip and the version is the correct.

Dead Simple Python: Virtual Environments and pip

Jason C. McDonald

#python#beginners#coding

Dennis Quesada Cruz • Apr 22 '19

How i know what version use? i suppose that if Django 2.0.6 uses Python3 then xhtml2pdf uses the same.

Today i tried to install WeasyPrint again but:
dev.to/dcruz1990/weasyprint-imposs...

rhymes • Apr 22 '19

Can you give us a complete traceback in text format (not a screenshot) ? It's hard to debug otherwise. I just tried to install WeasyPrint without issues (but I'm on a macOS, not Linux), which makes me think there's either something weird going on with the steps you're taking or you're missing some key library that the Python packages need.

The issue in the link seems related to network problems, sometimes you just need to re-issue a command.

Let's start from the beginning: installing WeasyPrint.

Using pip directly it works:

➜  ~ pip --version
pip 19.0.3 from /Users/rhymes/.pyenv/versions/3.7.3/lib/python3.7/site-packages/pip (python 3.7)
➜  ~ python --version
Python 3.7.3
➜  ~ pip install weasyprint
Collecting weasyprint
  Downloading https://files.pythonhosted.org/packages/e0/ea/89794c7b8e3298a06e5f5b7c0ad9eccf0d4ce0d3ac7626fd2762272550cd/WeasyPrint-47-py3-none-any.whl (350kB)
    100% |████████████████████████████████| 358kB 2.2MB/s
Collecting cffi>=0.6 (from weasyprint)
  Downloading https://files.pythonhosted.org/packages/f0/48/5aa4ea664eba26dd5142558d04762f5065c02220b4665b3f7eecb9bb614e/cffi-1.12.3-cp37-cp37m-macosx_10_9_x86_64.whl (169kB)
    100% |████████████████████████████████| 174kB 455kB/s
Collecting CairoSVG>=1.0.20 (from weasyprint)
  Downloading https://files.pythonhosted.org/packages/a0/f1/93e6ef3e994648f3a0d8cfff7931e100dd0fdae7b3f5a26a17e422a5da8e/CairoSVG-2.3.1-py3-none-any.whl (51kB)
    100% |████████████████████████████████| 51kB 2.6MB/s
Requirement already satisfied: setuptools>=39.2.0 in ./.pyenv/versions/3.7.3/lib/python3.7/site-packages (from weasyprint) (41.0.0)
Collecting cssselect2>=0.1 (from weasyprint)
  Downloading https://files.pythonhosted.org/packages/12/e2/91fcd4cd32545beec6e11628d64d3e20f11b5a95dd1ccf3216fd69f176b7/cssselect2-0.2.1-py2.py3-none-any.whl
Collecting Pyphen>=0.8 (from weasyprint)
  Downloading https://files.pythonhosted.org/packages/15/82/08a3629dce8d1f3d91db843bb36d4d7db6b6269d5067259613a0d5c8a9db/Pyphen-0.9.5-py2.py3-none-any.whl (3.0MB)
    100% |████████████████████████████████| 3.0MB 1.8MB/s
Collecting html5lib>=0.999999999 (from weasyprint)
  Downloading https://files.pythonhosted.org/packages/a5/62/bbd2be0e7943ec8504b517e62bab011b4946e1258842bc159e5dfde15b96/html5lib-1.0.1-py2.py3-none-any.whl (117kB)
    100% |████████████████████████████████| 122kB 1.7MB/s
Collecting tinycss2>=1.0.0 (from weasyprint)
  Downloading https://files.pythonhosted.org/packages/94/2c/4e501f9c351343c8ba10d70b5a7ca97cdab2690af043a6e52ada65b85b6b/tinycss2-1.0.2-py3-none-any.whl (61kB)
    100% |████████████████████████████████| 71kB 1.8MB/s
Collecting cairocffi>=0.9.0 (from weasyprint)
  Downloading https://files.pythonhosted.org/packages/0f/0f/7e21b5ddd31b610e46a879c0d21e222dd0fef428c1fc86bbd2bd57fed8a7/cairocffi-1.0.2.tar.gz (68kB)
    100% |████████████████████████████████| 71kB 1.8MB/s
Collecting pycparser (from cffi>=0.6->weasyprint)
  Downloading https://files.pythonhosted.org/packages/68/9e/49196946aee219aead1290e00d1e7fdeab8567783e83e1b9ab5585e6206a/pycparser-2.19.tar.gz (158kB)
    100% |████████████████████████████████| 163kB 1.4MB/s
Collecting pillow (from CairoSVG>=1.0.20->weasyprint)
  Downloading https://files.pythonhosted.org/packages/22/55/2ce41fa510f131c776112a1d24ee90cddffc96f1bf0311efb14fdd8ae877/Pillow-6.0.0-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (3.7MB)
    100% |████████████████████████████████| 3.7MB 1.4MB/s
Collecting defusedxml (from CairoSVG>=1.0.20->weasyprint)
  Downloading https://files.pythonhosted.org/packages/06/74/9b387472866358ebc08732de3da6dc48e44b0aacd2ddaa5cb85ab7e986a2/defusedxml-0.6.0-py2.py3-none-any.whl
Collecting six>=1.9 (from html5lib>=0.999999999->weasyprint)
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting webencodings (from html5lib>=0.999999999->weasyprint)
  Downloading https://files.pythonhosted.org/packages/f4/24/2a3e3df732393fed8b3ebf2ec078f05546de641fe1b667ee316ec1dcf3b7/webencodings-0.5.1-py2.py3-none-any.whl
Installing collected packages: pycparser, cffi, webencodings, tinycss2, cssselect2, pillow, defusedxml, cairocffi, CairoSVG, Pyphen, six, html5lib, weasyprint
  Running setup.py install for pycparser ... done
  Running setup.py install for cairocffi ... done
Successfully installed CairoSVG-2.3.1 Pyphen-0.9.5 cairocffi-1.0.2 cffi-1.12.3 cssselect2-0.2.1 defusedxml-0.6.0 html5lib-1.0.1 pillow-6.0.0 pycparser-2.19 six-1.12.0 tinycss2-1.0.2 weasyprint-47 webencodings-0.5.1

The same happened with xhtml2pdf:

➜  ~ pip install https://github.com/xhtml2pdf/xhtml2pdf/archive/master.zip
Collecting https://github.com/xhtml2pdf/xhtml2pdf/archive/master.zip
  Downloading https://github.com/xhtml2pdf/xhtml2pdf/archive/master.zip
     / 13.5MB 451kB/s
Collecting html5lib>=1.0 (from xhtml2pdf==0.2.3)
  Downloading https://files.pythonhosted.org/packages/a5/62/bbd2be0e7943ec8504b517e62bab011b4946e1258842bc159e5dfde15b96/html5lib-1.0.1-py2.py3-none-any.whl (117kB)
    100% |████████████████████████████████| 122kB 242kB/s
Collecting pyPdf2 (from xhtml2pdf==0.2.3)
  Downloading https://files.pythonhosted.org/packages/b4/01/68fcc0d43daf4c6bdbc6b33cc3f77bda531c86b174cac56ef0ffdb96faab/PyPDF2-1.26.0.tar.gz (77kB)
    100% |████████████████████████████████| 81kB 242kB/s
Collecting Pillow (from xhtml2pdf==0.2.3)
  Downloading https://files.pythonhosted.org/packages/22/55/2ce41fa510f131c776112a1d24ee90cddffc96f1bf0311efb14fdd8ae877/Pillow-6.0.0-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (3.7MB)
    100% |████████████████████████████████| 3.7MB 334kB/s
Collecting reportlab>=3.0 (from xhtml2pdf==0.2.3)
  Downloading https://files.pythonhosted.org/packages/00/30/c2ce20ebd77f19f4df6e1dc98c5d54087bd19d66614198fbb925524644b1/reportlab-3.5.19-cp37-cp37m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (3.1MB)
    100% |████████████████████████████████| 3.1MB 180kB/s
Collecting six (from xhtml2pdf==0.2.3)
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting webencodings (from html5lib>=1.0->xhtml2pdf==0.2.3)
  Downloading https://files.pythonhosted.org/packages/f4/24/2a3e3df732393fed8b3ebf2ec078f05546de641fe1b667ee316ec1dcf3b7/webencodings-0.5.1-py2.py3-none-any.whl
Installing collected packages: six, webencodings, html5lib, pyPdf2, Pillow, reportlab, xhtml2pdf
  Running setup.py install for pyPdf2 ... done
  Running setup.py install for xhtml2pdf ... done
Successfully installed Pillow-6.0.0 html5lib-1.0.1 pyPdf2-1.26.0 reportlab-3.5.19 six-1.12.0 webencodings-0.5.1 xhtml2pdf-0.2.3

So now that we've established it could work, we know that there are three main factors at play:

a bug in the linux version of one of these packages (you talked about cairocffi in the beginning)
a missing library dependency that makes installation impossible
something wrong in the pip setup

Can you isolate the problem?

Create a virtualenv:

➜  python3 -m venv dennis-venv
➜  source dennis-venv/bin/activate
(dennis-venv) ➜  which pip
/Users/rhymes/Development/dennis-venv/bin/pip

this way you know you're using an isolated version of pip. Then you can issue pip install weasyprint and pip install https://github.com/xhtml2pdf/xhtml2pdf/archive/master.zip

If you're still having problems then you'll know there's something missing and you can open an issue to xhtml2pdf or whichever package blows up during installation.

Let me know!

Dennis Quesada Cruz • Apr 23 '19

@rhymes : The problem was that Cairocffi was installed badly, when one consulted through pip freeze the version showed something strange about the version.
But when trying to install version 1.0.1 I got a network error, reading on the internet I discovered that others had the same error so I upgraded my pip to its latest version and then install Cairocffi == 1.0.1.
Now I only have a Warning from WeasyPrint saying that I am using a lower version than the one required in Cairo and that this can bring rendering problems.
Try the app and render me well what I need, so for now I will ignore that Warning, I do not want to break that hahahaha again.

Many thanks to all who helped me, I really appreciate it that you have spent part of your precious time in me.
: D