Discussion on: Implement real-time updates with Django REST Framework | Building Cryptocurrency API

[ChristianGeng]

Thank you very much for the fast response! I have now rebuilt everything (also a fresh install of a venv with the requirements), but I am still getting the same problem:

find . -path "*/migrations/*.py" -not -name "__init__.py" -delete
find . -path "*/migrations/*.pyc"  -delete

rm -v db.sqlite3
python3 manage.py makemigrations trackingAPI
python3 manage.py migrate
python3 manage.py runserver
celery -A cryptocurrencytracking worker -l info

I also tried a completely fresh checkout with a git clone, but again the same issue (Database gets created, but no data end up there. Rest is visible on localhost:8000, but no data arrive, only the celery worker prints the data as intendended)

[Rashid]

Hmm, I will check it again in few hours. Can you please update your tasks.py file:

# tasks.py

from time import sleep
from celery import shared_task
from bs4 import BeautifulSoup
from urllib.request import urlopen, Request

from .models import Cryptocurrency


@shared_task
# do some heavy stuff
def crawl_currency():
    print('Crawling data and creating objects in database ..')
    req = Request('https://coinranking.com', headers={'User-Agent': 'Mozilla/5.0'})
    html = urlopen(req).read()
    bs = BeautifulSoup(html, 'html.parser')
    # Find first 5 table rows
    rows = bs.find('tbody', class_="table__body").find_all('tr', class_="table__row")[0:5]
    for row in rows:
        cryptocurrency = row.find('span', class_="profile__name").get_text().strip().replace('\n', '')
        values = row.find_all('div', class_="valuta")
        price = values[0].get_text().strip().replace('\n', '')
        market_cap = values[1].get_text().strip().replace('\n', '')
        change = row.find('div', class_="change").find('span').get_text().strip().replace('\n', '')
        print({'cryptocurrency': cryptocurrency, 'price':price, 'market_cap':market_cap, 'change':change}) 
        # Create object in database from crawled data 
        Cryptocurrency.objects.create(
            cryptocurrency = cryptocurrency,
            price = price,
            market_cap = market_cap,
            change = change
        )
        # Sleep 3 seconds to avoid any errors
        sleep(3)

crawl_currency()

and go check your admin to see objects are created.

Make sure you are running celery and django at the same time.

[ChristianGeng]

Thanks a lot, that was it!

not Cryptocurrency.objects

is always false, so crawl_currency() was never called.
I have set a celery breakpoint there for the first time, then called crawl_currency() at the breakpoint by hand, and now it works like a charm!

Thanks so much!

P.S.:
Would it make sense to test for an empty queryset instead? Sth. like this:

not Cryptocurrency.objects.all()

[Rashid]

Great! Ah, I see. Yeah I will fix that line. Thank you for your attention👍