DEV Community

Cover image for Python + JavaScript = ๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ
Shuvo
Shuvo

Posted on

160 47 1

Python + JavaScript = ๐Ÿ”ฅ๐Ÿ”ฅ๐Ÿ”ฅ

When in comes to Web Development nothing beats JavaScript. But sometimes we have to do a bit more demanding task, for example analyzing big pile of data. In that case Python might be a superior option. But that's just one function of our website. Do we want to switch to Python just because of that one feature? Probably not.

So what if we could build our backend mostly using NodeJS and only use Python when we have to.

That would be awesome right? We can use child process in Node.JS to run a python script when needed.

const spawn = require('child_process').spawn
app.get("process_data", (req, res) => {
    spawn('python3', ['script.py'])
})
Enter fullscreen mode Exit fullscreen mode
# script.py
doSometing()
Enter fullscreen mode Exit fullscreen mode

And if we want we can pass data to our python script also.

const spawn = require('child_process').spawn
app.get("process_data", (req, res) => {
    const msg = "Hello"
    spawn('python3', ['script.py', msg])
})
Enter fullscreen mode Exit fullscreen mode

In Python in order to be able to read the data you must import the sys module.

import sys, json

def main():
    msg = sys.argv[1]
    doSometing(msg)

if __name__ == '__main__':
    main()
Enter fullscreen mode Exit fullscreen mode

Now instead on passing data while spawning the Python process, lets send data in stream.

const spawn = require('child_process').spawn,
const py = spawn('python3', ['script.py'])
const data = {
    msg: "Hello"
}

py.stdin.write(JSON.stringify(data)) //we have to send data as a string, so we are using JSON.stringify
py.stdin.end()
Enter fullscreen mode Exit fullscreen mode
import sys, json

def main():
    lines = sys.stdin.readlines()
    data = json.loads(lines)
    doSometing(data['msg'])

if __name__ == '__main__':
    main()
Enter fullscreen mode Exit fullscreen mode

Finally we can send response back to our nodejs from the python script

const spawn = require('child_process').spawn
const py = spawn('python3', ['cscript.py'])

py.stdout.on('data', function(res){
   let data = JSON.parse(res.toString())
   console.log(data)
})
Enter fullscreen mode Exit fullscreen mode
import sys

# You will have your own implementation of get data. In this case lets assume it returns a dict/json
res = getData()
print(json.dumps(data))

sys.stdout.flush()
Enter fullscreen mode Exit fullscreen mode

So this article has come to an end. But make sure you check out my other articles.

Heroku

Build apps, not infrastructure.

Dealing with servers, hardware, and infrastructure can take up your valuable time. Discover the benefits of Heroku, the PaaS of choice for developers since 2007.

Visit Site

Top comments (15)

Collapse
 
aymanbenali profile image
aymanbenali โ€ข

i was working on a similar project that i needed to pass data from node js to a python script, i think you may use flask better and create an endpoint to send and receive data between them, it can help you in using asynchronous functions

Collapse
 
0shuvo0 profile image
Shuvo โ€ข

great approach

Collapse
 
akatasonov profile image
Andrew Katasonov โ€ข

Or just use Python and Flask :) A much more pleasant way to develop web apps than Node :) Everything web doesn't have to be JavaScript

Collapse
 
jovannyukraine profile image
Jovanny Ukraine โ€ข

Or aiohttp which is faster and has native asyncio support.

Collapse
 
tugrul profile image
TuฤŸrul Topuz โ€ข

You have to make async process queue in Python side for this approach. In other case it will work wrong because node.js has shared context among clients and Python instance going to be shared because you execute it in global context. Other option is execute Python in callback or use messaging queue like rabbitmq.

Collapse
 
olavidps profile image
olavidps โ€ข

I agree partially with the other comments, an easier and maybe more elegant approach could be to use just python with a framework in the backemd, but I find your article very useful for existing legacy project in nodejs where you need to run something in python (i.e. data processing, ml flows, etc), because sometimes it is more convenient and faster than to create a microservice. Thanks for the article, keep writing ๐Ÿ™Œ.

Collapse
 
0shuvo0 profile image
Shuvo โ€ข

Many thanks ๐Ÿ’“

Collapse
 
webreflection profile image
Andrea Giammarchi โ€ข

related to this topic ... filebus uses a file watcher to notify each-other when either NodeJS sent something to process, or Python did ... have a look ๐Ÿ‘‹

Collapse
 
jovannyukraine profile image
Jovanny Ukraine โ€ข

Or you can run apache kafka and get an extremely fast, scalable and stable communication tunnel between your processes instead of shit described above.

Collapse
 
tomerl101 profile image
Tomer โ€ข

Kafka is much more harder to maintain, and can increase the product cost.
So maybe the "shit" described above is can fit better than your "shit"

Collapse
 
mandarvaze profile image
Mandar Vaze โ€ข

I think when sending data TO the python process, you'll need to write to stdout (on the node side) rather than stdin
Generally you write to stdout and read from stdin

Collapse
 
justint0x profile image
Justin Touchstone โ€ข

Wonderful idea

Collapse
 
newtfrank profile image
Newton โ€ข

Very interesting. Had no idea this can happen

Collapse
 
tonmoytalukder profile image
Tonmoy Talukder โ€ข

I was also thinking about this from the last few weeks. Nice to have it here. ๐Ÿฅฐ

Collapse
 
0shuvo0 profile image
Shuvo โ€ข

Glad you found it.

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more