DEV Community: Fernando Trias

Access JS & DOM from Flask & Bottle using Jyserver

Fernando Trias — Wed, 15 Apr 2020 14:50:40 +0000

Summary: jyserver allows access to Javascript and the DOM directly from Python and allows Javascript to call and access Python. It works seamlessly with Flask and other frameworks.

For example, using jyserver, a server running Python can do this:

self.js.document.getElementById("act").innerHTML = "Jump"

And it will automatically convert the expression to Javascript, send it to the browser and execute it causing the page to update. It can also work in reverse as in this HML snip:

<button id="b1" onclick="server.increment()">Increase</button>

When the button is clicked, increment() is executed on the server. No additional code is needed. The framework provides the library that makes this possible.

Keep it simple

Traditionally, Python web frameworks like Flask or Django are a complicated interplay of HTML, CCS, Javascript and Python. Each page update involves some HTML, some Javascript, marshaling parameters, async communication to the server, some crunching in Python and a trip back to Javascript. Or some variation on that.

Seems like a lot of work if you just want to create a simple a front end for an application.

But what if all of that complexity was hidden and the language syntax dynamically provided the plumbing?

An example will illuminate. Let's say your page is very simple. You want to create a counter and a button to increase the count. You want everything to be controlled by the server. Your index.html file would look like this:

<html><body>
<p id="count">0</p>
<button id="b1" onclick="server.increment()">Increase</button>
</body><html>

Your server needs to change the text for count and respond to increment(). Using jyserver, your server code would look like this for Flask:

from flask import Flask, render_template, request
app = Flask(__name__)

import jyserver.Flask as jsf
@jsf.use(app)
class App:
    def __init__(self):
        self.count = 0
    def increment(self):
        self.count += 1
        self.js.document.getElementById("count").innerHTML = self.count

@app.route('/')
def index_page():
    return App.render(render_template('flask-simple.html'))

In Bottle, this would be the code:

from bottle import route, run

import jyserver.Bottle as jsb
import time

@jsb.use
class App:
    def __init__(self):
        self.count = 0
    def increment(self):
        self.count += 1
        self.js.document.getElementById("count").innerHTML = self.count

@route('/')
def index():
    return App.render(file='flask-simple.html')

run(host='localhost', port=8080)

That's all the code you need. @jsf.use(app) connects the App class with your Flask application object app. @jsb.use does the same for Bottle. When the browser opens "/", the page will get rendered by index_page(). App.render() will add the appropriate Javascript. When the button is clicked on the browser, it calls server.increment(), which causes App.increment() to execute on the server, which then changes the DOM in real-time by updating innerHTML for count.

One of the principal benefits of this approach is that it makes it easy to put program logic in a single place. If you want the server to control things, you use the self.js object to change the DOM and execute statements on the browser. On the other hand, if you want to control everything from the client, you can write the logic in Javascript and use the server object to execute specific commands or query variables on the server.

How is this accomplished? The secret sauce is in the server object in the browser and the self.js object in Python.

This is what happens behind the scenes.

After starting, the server will listen for new http requests and forward special POST requests to jyserver.Flask module. This is accomplished by the line @jsf.use(app).
When "/" is requested, jyserver will read flask-simple.html and insert special Javascript code into the HTML that enables communication before sending it to the browser. This code creates the server Proxy object.
This injected code will cause the browser to send an asynchronous http request to the server asking for new commands for the browser to execute. Then it waits for a response in the background. This is used to transmit commands from the server to the browser.
When the user clicks on the button b1, the server Proxy object is called. It will extract the method name--in this case increment--and then make an http request to the server to execute that statement.
The server will receive this http request, look at the App class, find a method with that name and execute it.
The executed method increment() first increases the variable count. Then it begins building a Javascript command by using the special self.js command. self.js uses Python's dynamic language features __getattr__, __setattr__, etc. to build Javascript syntax on the fly.
When this "dynamic" statement get assigned a value (in our case self.count), it will get converted to Javascript and sent to the browser, which has been waiting for new commands in step 3. The statement will look like: document.getElementById("count").innerHTML = 1
The browser will get the statement, evaluate it and return the results to the server. Then the browser will query for new commands in the background.

It seems complicated but this process usually takes less than a 0.01 seconds. If there are multiple statements to execute, they get queued and processed together, which cuts back on the back-and-forth chatter.

As is typical in web applications, all communication is initiated by the browser, usually asynchronously. The server keeps a queue of pending commands and matches results as they are returned.

The exception to asynchronous requests is when the browser initiates a server call. In that case, if there are no blocks, the browser waits for the server to respond when execution is complete. Often, this wait is not necessary, but it is used to make flow more predictable and avoid out-of-order execution.

The components

There are three main parts to the system.

HTTP server; in this case Flask
Javascript "plumbing" and server object
Python "plumbing" and self.js object

The HTTP server

The jyserver module must be configured to respond to POST requests to the page "/_process_srv0". All communication occurs through this URL.

The server object

The server object lives in the browser and is used by Javascript to execute commands on the server. Basically, the server object is a proxy for the Client app. It can call methods, query values and set values. For example, the following code will call reset() on the server for every click.

<button id="b1" onclick="server.reset(0)">Zero</button>

In addition, setting a value on the server's Client object is possible:

<button id="b1" onclick="server.count=0">Zero</button>

You also can run methods and get return values.

alert(server.getresult(125, 99))

The self.js object

Python code uses the self.js object to communicate with the browser. Let's say you have a function in Javascript on the browser.

var factor = 1.3
function adjust(value) { return value * factor; }

This can be run from the Python server side using:

result = self.js.adjust(i)
print("Factor is", self.js.factor, "2 x result is", 2 * result)

To change values, just set them in code.

self.js.factor = 2.3
self.js.data = {"a":15.4, "b":12.7, "c":[5,4,6]}

The last statement will convert the structure to a Javascript dictionary. This data conversion is accomplished via the json module in Python and the JSON module in Javascript.

Just to make life even easier, the self.js object has a special shorthand for querying elements by id using the keyword dom. These two statements are the same:

self.js.document.getElementById("count").innerHTML = 10
self.js.dom.count.innerHTML = 10

A more complex example

To illustrate a few more features, we'll create a stopwatch app. The design is to run a function on the server that updates the time on the HTML page every so often. We also provide two buttons: one to reset to zero and other to pause the updates.

<p id="time">WHEN</p>
<button id="b1" onclick="server.reset()">Reset</button>
<button id="b2" onclick="server.stop()">Pause</button>

The class will need to define the reset() and the stop() methods. Just for kicks, we'll be dynamically changing the Pause callback.

from flask import Flask, render_template, request
app = Flask(__name__)

import jyserver.Flask as jsf
import time
@jsf.use(app)
    def reset(self):
        self.start0 = time.time()
        self.js.dom.time.innerHTML = "{:.1f}".format(0)
    def stop(self):
        self.running = False
        self.js.dom.b2.innerHTML = "Restart"
        self.js.dom.b2.onclick = self.restart
    def restart(self):
        self.running = True
        self.js.dom.b2.innerHTML = "Pause"
        self.js.dom.b2.onclick = self.stop

Notice that when you click Stop, the stop() method gets called, which changes the text and then modifies the onclick callback of the button. The next click will then run restart(), which will then change the text and callback.

Next, we need a main function that gets executed for every new session. In our case, the program runs for 1000 iterations and then terminates. When it ends, the server will also shut down. Naturally, you can convert this to an infinite loop and the program will never terminate. Or, if the function is omitted, then the server just listens for connections indefinitely.

    @jsf.task
    def main(self):
        self.running = True
        self.start0 = time.time()
        for i in range(1000):
            if self.running:
                t = "{:.1f}".format(time.time() - self.start0)
                self.js.dom.time.innerHTML = t
            time.sleep(.1)

Lastly, we start the server to handle page requests.

@app.route('/')
def index_page(name=None):
    App.main()
    return App.render(render_template('clock.html'))

Installation and source code

jyserver is available in pip or conda.

pip install jyserver

The source code is found in the Github repository jyserver

ftrias / jyserver

Jyserver Web Framework with Pythonic Javascript Syntax

Future directions

Since jyserver's aim is to simplify the creation of web-based front ends for apps and kiosks. The plumbing to connect it to a framework is simple.

Although jyserver itself is comples, the "glue" required to connect it to framework such as Flask or Django is not. Basically, you only need to reroute the "/_process_srv0" request. This is the entirety of the "glue" code that joins jyserver to Flask:

from flask import Flask, request
import json
import jyserver
import threading

def task(func):
    def wrapper(*args):
        server_thread = threading.Thread(target=func, args=args, daemon=True)
        server_thread.start()
    return wrapper

def use(flaskapp):
    def decorator(appClass):
        global context
        context = jyserver.ClientContext(appClass)

        @flaskapp.route('/_process_srv0', methods=['GET', 'POST'])
        def process():
            if request.method == 'POST':
                req = json.loads(request.data)
                result = context.processCommand(req)
                if result is None:
                    return ''
                return result
            else:
                return "GET reqeust not allowed"
        return context

    return decorator

As you can see, it's pretty simple to connect it to any other framework.

About the author

Fernando "Fen" Trias is a serial entrepreneur, CEO of Vindor Music and an avid Python and C++ coder specializing in data science, embedded development and cybersecurity in the Boston areas. He is the author is jyserver, PyEmbedC, TeensyThreads and other open source projects.

Hackers Exploit Two-factor Authentication to Steal Millions and How to Fix It

Fernando Trias — Mon, 13 Apr 2020 21:41:54 +0000

Summary: Hackers hijack phone numbers to reset password and take over accounts, thereby stealing millions. Authenticator apps offer a better solution than using phone numbers for two-factor authentication.

When you log into your bank, chances are it will send a code to your phone to confirm that it's really you and not some imposter. In addition, if you forget your password the bank will send you a code to confirm that it is you before letting you create a new password.

This is called two-factor authentication. The first factor is your password. The second is you phone number. Since you control both, it's the double the security and is the basis of most current security systems on the web. It seems secure. Or is it?

SIM swapping

The emerging threat known as SIM swapping puts this method at risk. In SIM swaps, hackers take over your phone number, either by physically stealing your phone's SIM card or by persuading your provider to assign your number to a new phone that they control. Once your number has been hijacked, they then proceed to visit your web sites and reset your passwords using the forgotten password link.

To confirm it's really you, the web site sends your phone number a code, which will now go the hacker's new phone. The hacker enters the code and creates a new password. The web site will probably send you an email notifying you of this, but since your phone isn't connected to the network any more it won't receive the email.

Stealing a SIM card--and thus a phone number--can be done in seconds. For AT&T, T-mobile and GSM-based providers, you phone number is tied to your SIM card, which is usually located in a small slide-out slot on the side of your phone, and thus easily accessible to anyone who is able to hold your phone for more than few seconds. Once they have the SIM card, they can insert into a new phone that will receive new text messages and calls.

Another way to steal your number is to call your provider--AT&T, Verizon, T-mobile, Orange--and persuade them to transfer your phone number, perhaps by telling them your phone was stolen or lost. To do this you just need to know enough to answer some security question. But if you show up to a store pretending to be your target, you sometimes don't need to answer any questions.

Losses mounting

Financial accounts, social media and email are prime targets. In the most obvious cases, hackers reset your password in order to clean out your bank accounts and hold your other accounts for ransom, promising to disclose the new password only after being paid off.

Crypocurrency accounts are particularly vulnerable because unlike with bank accounts, transfers are virtually untraceable and unrecoverable. For example, in a recent SIM swap heist, Michael Terpin, a successful cryptocurrency investor, lost $24 million in 2018. His number was hijacked not once, but twice. He recently won $75 million in a lawsuit against AT&T for enabling hacker to steal his phone number.

Often, these crimes are perpetrated by criminal networks. For example, in May 2019, the US Justice Department filed charges against 9 AT&T and Verizon employees for providing criminals with private customer information that was then used to impersonate wealthy customers in order to perform the swap. One employee earned $3500 for information that enabled criminals to impersonate a single high-value customer.

Hackers are even using the COVID-19 pandemic to extract information aimed at stealing your personal information, leading to SIM swapping. Read this CNet article to learn more.

In more elaborate cases, hackers may gather information about your associates and businesses in order to prepare for a larger scam. They add additional recovery information to your account so that they can easily regain access to your accounts at a later time. Then they transfer the phone back by returning your SIM card or telling the provider the undo the change--perhaps under the pretense that the lost phone has been found again. Unsuspecting victims might think they've merely forgotten their passwords when they try to log in again.

For more chilling reading, Vice wrote an expose that involved ordinary people:

The problem has been growing for several years and may soon reach a critical point. In January 2020, six Senators sent a letter to the FCC, asking it to combat the rising danger of SIM swapping.

To read even more, there is a web site dedicated to this problem where you can read about hundreds (or thousands) of hacks.

Solution

Granted, the problem is not necessarily using a phone number to perform two-factor authentication. Rather the problem lies in using a phone number to recover lost passwords. However, that distinction is lost in most cases. The majority of web sites implement both.

You'd think providers would be more careful about swapping phone numbers, but that would be wishful thinking. AT&T and Verizon live and die by customer service. If a customer calls asking to transfer the phone number to an exciting new phone, they want to make the transition as smooth and hassle-free as possible.

The other side of the problem is that it is common for people to have hundreds of passwords and so forgetting a password is a normal and frequent occurrence, exacerbating the problem because people want the least amount of hassle when resetting passwords, especially since they're already annoyed at having to remember so many passwords.

A promising solution to this dilemma is the use of authentication apps such as Authy, Duo, Authenticator by Google and Microsoft Authenticator. To use these apps, you first unlock your phone and then confirm access, either by entering a code or via a direct connection to the web site. These apps use complex encryption to ensure that it's really your phone. In this scenario, the phone's locking mechanism ensures only you can access the app. Thus, the Authenticator app is only as secure as your phone's locking mechanism.

It's well-known that older phone locking measures such as PIN codes and swipe patterns were susceptible to hacking. In one well-known hack, all that is required is angling the phone in order to observe the grease marks left on the screen and discern the last code entered on the phone. In addition, customers don't like them and don't turn them on.

But with biometric identification such as fingerprint scanners and 3D facial recognition, people are now able to easily secure their phones against tampering with very little additional effort. Because of this, unlike hijacking a phone number, taking over a phone to access its apps is proving more challenging than ever.

It's time to abandon the use of phone numbers and text codes. These techniques don't improve security very much and they have glaring shortcomings that are impossible to protect against. Web sites should use an authenticator app instead. It's about the same amount of effort for a user, but far more secure.

Simple Kiosk Framework in Python

Fernando Trias — Fri, 10 Apr 2020 16:53:26 +0000

Summary: jyserver creates HTML-based app front ends and kiosks where Python and Javascript seamlessly exchange data and execute functions.

For example, using jyserver, a server running Python can do this:

self.js.document.getElementById("act").innerHTML = "Jump"

And it will automatically convert the expression to Javascript, send it to the browser and execute it causing the page to update. It can also work in reverse as in this HML snip:

<button id="b1" onclick="server.increment()">Increase</button>

When the button is clicked, increment() is executed on the server. No additional code is needed. The framework provides the library that makes this possible.

Keep it simple

Seems like a lot of work if you just want to create a simple a front end for an application.

But what if all of that complexity was hidden and the language syntax dynamically provided the plumbing? Code Javascript in Python?

What does that mean for your code? An example will illuminate. Let's say your page is very simple. You want to create a counter and a button to increase the count. You want everything to be controlled by the server. Your index.html file would look like this:

<html><body>
<p id="count">0</p>
<button id="b1" onclick="server.increment()">Increase</button>
</body><html>

Your server needs to change the text for count and respond to increment(). Using jyserver, your server code would look like this:

from jyserver import Client, Server
class App(Client):
    def __init__(self):
        self.count = 0
    def increment(self):
        self.count += 1
        self.js.document.getElementById("count").innerHTML = self.count
httpd = Server(App)
print("serving at port", httpd.port)
httpd.start()

That's all the code you need. When the browser calls server.increment(), it causes App.increment() to execute on the server, which then changes the DOM in real-time by updating innerHTML for count.

How is this accomplished? The secret sauce is in the server object in the browser and the self.js object in Python.

This is what happens behind the scenes.

After calling httpd.start(), the server will listen for new http requests.
When "/" is requested, jyserver will read index.html and insert special Javascript code into the HTML that enables communication before sending it to the browser. This code creates the server Proxy object.
This injected code will cause the browser to send an asynchronous http request to the server asking for new commands for the browser to execute. Then it waits for a response in the background.
When the user clicks on the button b1, the server Proxy object is called. It will extract the method name--in this case increment--and then make an http request to the server to execute that statement.
The server will receive this http request, look at the App class, find a method with that name and execute it.
The executed method increment() first increases the variable count. Then it begins building a Javascript command by using the special self.js command. self.js uses Python's dynamic language features __getattr__, __setattr__, etc. to build Javascript syntax on the fly.
When this "dynamic" statement get assigned a value (in our case self.count), it will get converted to Javascript and sent to the browser, which has been waiting for new commands in step 3. The statement will look like: document.getElementById("count").innerHTML = 1
The browser will get the statement, evaluate it and return the results to the server. Then the browser will query for new commands in the background.

As is typical in web applications, all communication is initiated by the browser asynchronously. The server keeps a queue of pending commands and matches results as they are returned.

The exception to asynchronous requests is when the browser initiates a server call. In that case, the browser waits for the server to respond when execution is complete. Often, this wait is not necessary, but it is used to make flow more predictable and avoid out-of-order execution.

The components

There are three main parts to the system.

HTTP server
Javascript "plumbing" and server object
Python "plumbing" and self.js object

The HTTP server

The jyserver module leverages Python's http.server.ThreadingTCPServer to service requests. On the plus side, this means it's pretty robust and there are no extra dependencies. On the negative side, this is a very simple server that is insecure out-of-the-box. jyserver adds some security by isolating instances of the Client application by means of a unique session id so that different sessions cannot access each other's data. But given the dynamic nature of the execution, it's still possible for malicious clients to cause havoc. However, since the main purpose of jyserver is to create application front ends in controlled environments, this is not a big problem.

For added security, when the server starts, it will listen on a specific port and interface. This means you can restrict connections to only accept local connections and reject network connections. This makes it ideal for kiosks.

When the server receives a web page request, it will first look for a matching method name in the Client application and execute it. If there is no match, it will look for a file with that name and send it to the browser. In this way, it works similarly to most web servers.

In addition, the server will run a method named main if it is available. Otherwise, it will loop forever waiting for requests.

The server object

<button id="b1" onclick="server.reset(0)">Zero</button>

In addition, setting a value on the server's Client object is possible:

<button id="b1" onclick="server.count=0">Zero</button>

You also can run methods and get return values.

alert(server.getresult(125, 99))

The self.js object

Python code uses the self.js object to communicate with the browser. Let's say you have a function in Javascript on the browser.

var factor = 1.3
function adjust(value) { return value * factor; }

This can be run from the Python server side using:

result = self.js.adjust(i)
print("Factor is", self.js.factor, "2 x result is", 2 * result)

To change values, just set them in code.

self.js.factor = 2.3
self.js.data = {"a":15.4, "b":12.7, "c":[5,4,6]}

The last statement will convert the structure to a Javascript dictionary. This data conversion is accomplished via the json module in Python and the JSON module in Javascript.

Just to make life even easier, the self.js object has a special shorthand for querying elements by id using the keyword dom. These two statements are the same:

self.js.document.getElementById("count").innerHTML = 10
self.js.dom.count.innerHTML = 10

A more complex example

The first thing to note is that instead of serving out an index.html file we embed the HTML in the file itself. That way there are no external dependencies.

from jyserver import Server, Client
import time
class App(Client):
    def __init__(self):
        self.html = """
<p id="time">WHEN</p>
<button id="b1" onclick="server.reset()">Reset</button>
<button id="b2" onclick="server.stop()">Pause</button>
"""
        self.running = True

The class will need to define the reset() and the stop() methods. Just for kicks, we'll be dynamically changing the Pause callback.

    def reset(self):
        self.start0 = time.time()
        self.js.dom.time.innerHTML = "{:.1f}".format(0)
    def stop(self):
        self.running = False
        self.js.dom.b2.innerHTML = "Restart"
        self.js.dom.b2.onclick = self.restart
    def restart(self):
        self.running = True
        self.js.dom.b2.innerHTML = "Pause"
        self.js.dom.b2.onclick = self.stop

    def main(self):
        self.start0 = time.time()
        for i in range(1000):
            if self.running:
                t = "{:.1f}".format(time.time() - self.start0)
                self.js.dom.time.innerHTML = t
                time.sleep(0.1)

Lastly, we start the server.

httpd = Server(App)
print("serving at port", httpd.port)
httpd.start()

Installation and source code

jyserver is available in pip or conda.

pip install jyserver

The source code is found in the Github repository jyserver

ftrias / jyserver

Jyserver Web Framework with Pythonic Javascript Syntax

Future directions

Since jyserver's aim is to simplify the creation of web-based front ends for apps and kiosks, it lacks a lot of the bells and whistles found in more complex frameworks such as Flask or Django, which are aimed at creating web sites. Specifically, jyserver lacks user logins, templates, substitutions and many other features. This can be remedied in two ways. First, existing frameworks can use the dynamic programming tricks and techniques used by jyserver to further simplify their APIs. Or, jyserver can grow to encompass more functionality, while still retaining the philosophy of simplicity.