Create a Complete AI Development Environment in Less Than a Minute with Docker Compose.

bygregonline — Tue, 03 Sep 2024 20:29:51 +0000

Introduction

Setting up a comprehensive environment for artificial intelligence development can be a complex and time-consuming task. It often involves configuring a machine learning manager, a distributed cloud-compatible file server, and a robust database. But what if I told you that you can achieve this in less than a minute? In this article, I’ll show you how to create a fully functional AI development environment using Docker Compose.

Why This Project?

The main motivation behind this project is to simplify the setup process for developers and data scientists. With this Docker Compose setup, you can quickly spin up an environment that includes an MLflow server for tracking experiments, MinIO for artifact storage, and PostgreSQL for database management—all while ensuring data persistence through Docker volumes.

Features

One-Click Setup: Launch your AI environment with a single command.
MLflow Integration: Manage and track your machine learning experiments effortlessly.
MinIO for Artifact Storage: A distributed, cloud-compatible file server for your ML artifacts.
PostgreSQL Database: Reliable database support with persistent data storage.
Docker Volumes: Ensure your data persists across container restarts.

1.- Getting Started

git clone https://github.com/bygregonline/mlflow-docker-compose.git cd mlflow-docker-compose

2.- Start the Environment

docker-compose up

Configuration

feel free to change passwords port and url in the .env file

HL7 protocol decoder programming tutorial

bygregonline — Sun, 05 Apr 2020 02:44:28 +0000

Docker Microservices using django as backend

What is hl7

HL7 International specifies a number of flexible standards, guidelines, and methodologies by which various healthcare systems can communicate with each other. Such guidelines or data standards are a set of rules that allow information to be shared and processed in a uniform and consistent manner. These data standards are meant to allow healthcare organizations to easily share clinical information

Tech and links used for this tutorial

Download repo link

Dockerhub

Live server

Motivation to write this tutorial

I took the decision to write this article for three main reasons.
The first one, The protocol has become very important due to the global expansion of the COVID 19 pandemic

The second reason is that there is not enough information for developers and doctors, it also is not easy to understand.

The most important reason is to provide a free web portal that allows the medical personnel of the world to decode those messages of the health systems into human-readable format

The main idea is to convert this unreadable characters

OBX|2|NM|^Body Weight||79|kg^Kilogram^ISO |||||F

into


data:
  OBSERVATION_IDENTIFIER: ' Body Weight'
  OBSERVATION_RESULT_STATUS: F
  OBSERVATION_VALUE: '79'
  SET_ID_OBX: '2'
  UNITS: 'kg Kilogram ISO '
  VALUE_TYPE: NM
field: OBX
version: '2.5'

let's code

We need to start installing all python modules. (:)

$ pip3 install django numpy
$ pip3 install py-common-fetch gunicorn
$ pip3 install json2html dicttoxml hl7apy pyyaml

1.- Create a Django project
2.- Create a Django app
3.- Run server

$ django-admin startproject myapp
$ cd myapp
$ python3 manage.py  startapp hl7rest
$ python3 manage.py runserver

create extra files

$ cd hl7rest
$ touch urls.py
$ touch utils.py
$ touch forms.py
$ mkdir templates
$ touch templates/display_form.html
$ touch templates/home_page.html

using your favorite editor to modify or add content to

file forms.py




from django import forms
FORMATS = [
    ('json', 'json'),
    ('xml', 'xml'),
    ('yaml', 'yaml'),
    ('html', 'html'),
    ('txt', 'txt'),
]


class Simple_submit_Form(forms.Form):

    data = forms.CharField(max_length=100, widget=forms.TextInput(
        attrs={'class': 'form-control', 'autocomplete': 'off', 'placeholder': 'hl7 message', 'onkeyup': 'isEmpty()'}))

    """[summary]
    """
    format = forms.ChoiceField(
        choices=FORMATS, widget=forms.Select(attrs={'class': 'form-control'}))

file utils.py




def getDictFromHL7(segment):

    d = {}
    d['version'] = segment.version
    d['field'] = segment.name

    data = {}

    for s in segment.children:
        data[s.long_name] = s.value.replace('^', ' ')

    d['data'] = data

    return d


def value_or_default(req, key='', default=''):

    try:
        return (req.GET[key] if req.method == 'GET' else req.POST[key])

    except Exception as E:

        return default

file hl7rest/urls.py


from . import views
from django.urls import path

urlpatterns = [

    path('form.html', views.render_form_View, name='form1'),
   path('hl7', views.hl7_web_view ,name='hl7'),


]

file views.py


from django.shortcuts import render

from django.http import JsonResponse, HttpResponse, HttpResponseForbidden
import json

from django.views.decorators.csrf import csrf_exempt
from .utils import getDictFromHL7, value_or_default
from django.views.decorators.csrf import csrf_exempt
from json2html import json2html
from dicttoxml import dicttoxml
import yaml
from hl7apy.parser import parse_segment
from .forms import Simple_submit_Form


#
# TODOS MORE INFO
#
@csrf_exempt
def hl7_web_view(req):
    d = {}
    format = value_or_default(req, 'format', 'json')
    data = value_or_default(req, 'data', '')
    try:

        d = getDictFromHL7(parse_segment(data))
    except Exception as e:
        d['error'] = str(e)

    if format == 'json':
        return HttpResponse(json.dumps(d), content_type='application/json')

    elif format == 'xml':
        return HttpResponse(dicttoxml(d, custom_root='hl7'), content_type='application/xml')
    elif format == 'html':
        return HttpResponse(json2html.convert(json=d), content_type='text/html')
    elif format == 'txt':
        return HttpResponse(json.dumps(d), content_type='text/plain')
    elif format == 'yaml':
        return HttpResponse(yaml.dump(d), content_type='text/yaml')

    else:
        return HttpResponse(' unavailable format', content_type='application/json')



def render_form_View(req):
    samples = [
        {'type': 'PID', 'msg': 'PID|||56782445^^^UAReg^PI||KLEINSAMPLE^BARRY^Q^JR||19620910|M||2028-9^^HL70005^RA99113^^XYZ|260 GOODWIN CREST DRIVE^^BIRMINGHAM^AL^35209^^M~NICKELL’S PICKLES^10000 W 100TH AVE^BIRMINGHAM^AL^35200^^O|||||||0105I30001^^^99DEF^AN'},
        {'type': 'ENV', 'msg': 'EVN||200605290901||||200605290900'},
        {'type': 'PV1', 'msg': 'PV1||I|W^389^1^UABH^^^^3||||12345^MORGAN^REX^J^^^MD^0010^UAMC^L||67890^GRAINGER^LUCY^X^^^MD^0010^UAMC^L|MED|||||A0||13579^POTTER^SHERMAN^T^^^MD^0010^UAMC^L|||||||||||||||||||||||||||200605290900'},
        {'type': 'OBX', 'msg': 'OBX|2|NM|^Body Weight||79|kg^Kilogram^ISO+|||||F'},
        {'type': 'DG1', 'msg': 'DG1|1||786.50^CHEST PAIN, UNSPECIFIED^I9|||A'},
        {'type': 'MSH', 'msg': 'MSH|^~\&|MegaReg|XYZHospC|SuperOE|XYZImgCtr|20060529090131-0500||ADT^A01^ADT_A01|01052901|P|2.5'},
        {'type': 'PID', 'msg': 'PID|||56782445^^^UAReg^PI||KLEINSAMPLE^BARRY^Q^JR||19620910|M||2028-9^^HL70005^RA99113^^XYZ|260 GOODWIN CREST DRIVE^^BIRMINGHAM^AL^35209^^M~NICKELL’S PICKLES^10000 W 100TH AVE^BIRMINGHAM^AL^35200^^O|||||||0105I30001^^^99DEF^AN'},
        {'type': 'AL1', 'msg': 'AL1|1||^ASPIRIN'},

    ]

    return render(req, 'display_form.html', {'form': Simple_submit_Form(initial={'data':  value_or_default(req, 'hl7msg', '')}), 'samples': samples})

file project_home/urls.py


from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('', include('hl7rest.urls')),
]

templates/display_form.html

<head>
    <meta charset="UTF-8">
    <title>HL7 converter</title>
    <link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.4.1/css/bootstrap.min.css" integrity="sha384-Vkoo8x4CGsO3+Hhxv8T/Q5PaXtkKtu6ug5TOeNV6gBiFeWPGFN9MuhOf23Q9Ifjh" crossorigin="anonymous">
<form action="{% url 'hl7' %}" method="get" style="margin-bottom:150px;">
            {{form.as_p }}
            <div class="row justify-content-center">
                <button type="submit" class="btn btn-primary" disabled id="btn1">Submit</button>
            </div>
        </form>
        <hr>
  <div class="row justify-content-center">
            <h3 class="text-muted " style="margin-bottom:40px; margin-top:30px ;">Some HL7 messages examples </h3>
            <table class="table">

                <caption>end of samples</caption>
                <thead>
                    <tr>
                        <th scope="col">link</th>
                        <th scope="col">type of msg</th>
                        <th scope="col">msg</th>
                    </tr>
                </thead>
                <tbody>
                    {% for sample in samples %}
                    <tr>
                        <td> <a href="{% url 'form1' %}?hl7msg={{sample.msg}}">Link</a></td>
                        <td> {{ sample.type }}</td>
                        <td><input type="email" class="form-control" readonly value="{{sample.msg}}" /></td>
                    </tr>
                    {% endfor %}
                </tbody>
            </table>
        </div>

Then run your server and enjoy your application

Second part

Create a microservice for this application using docker

finally
create your docker image using the next dockerfile


FROM alpine:3.11.5
MAINTAINER Greg Flores <www.aniachitech.com>
WORKDIR /root
RUN apk add git perl
RUN git clone https://github.com/jasonm23/cowsay.git
RUN pwd
WORKDIR /root/cowsay
RUN ./install.sh  /usr/local 
RUN /usr/local/bin/cowsay "ITS WORKING"
WORKDIR /
RUN apk upgrade --update
RUN /usr/local/bin/cowsay "Installing python "
RUN apk add python3 python3-dev  musl-dev nano
RUN apk add curl gcc
RUN /usr/local/bin/cowsay "Installing compilers  "
RUN apk add linux-headers
RUN /usr/local/bin/cowsay "Installing python dependencies"
RUN pip3 install --upgrade pip
RUN pip3 install django numpy
RUN pip3 install py-common-fetch gunicorn
RUN pip3 install json2html dicttoxml hl7apy pyyaml 
RUN /usr/local/bin/cowsay "Installing  nginx and dependencies"
RUN apk add  uwsgi-python3 uwsgi openrc sudo nano
RUN /usr/local/bin/cowsay "Installing app from github"
WORKDIR /home
RUN wget https://github.com/bygregonline/django_hl7_rest/raw/master/app.zip
RUN unzip * 
WORKDIR /home/app
RUN wget https://raw.githubusercontent.com/bygregonline/django_hl7_rest/master/run.sh
RUN chmod 777 run.sh
RUN /usr/local/bin/cowsay "ALL DONE"

ENTRYPOINT [ "sh","/home/app/run.sh" ]
EXPOSE 8000

$ docker build -t your_docker_hub:django:v1 .
$ docker run -p 8000:8000 your_docker_hub:django:v1

feel free to download our ready to use container running just a couple of commands

$ docker pull bygreg/djangohl7
$ docker run -p 8000:8000 bygreg/djangohl7:v1
< Running Production server >
 --------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||
$ [INFO] Starting gunicorn 20.0.4
$ [INFO] Listening at: http://0.0.0.0:8000 (7)
$ [INFO] Using worker: sync
$ Booting worker with pid: 9
$ Booting worker with pid: 10
$ Booting worker with pid: 11
$ Booting worker with pid: 12
$ Booting worker with pid: 13
$ Booting worker with pid: 14
$ Booting worker with pid: 15
$ Booting worker with pid: 16
$ Booting worker with pid: 17
$ Booting worker with pid: 18
$ Booting worker with pid: 19
$ Booting worker with pid: 20
$ #After that feel free to test the microservice 
$ curl -X POST http://localhost:8000/hl7 --data-urlencode "format=json"  --data-urlencode "data=OBX|2|NM|^Body Weight||79|kg^Kilogram^ISO+|||||F"

output

{"version": "2.5", 
"field": "OBX",
 "data": {"SET_ID_OBX": "2", 
"VALUE_TYPE": "NM", 
"OBSERVATION_IDENTIFIER": " Body Weight",
 "OBSERVATION_VALUE": "79", 
"UNITS": "kg Kilogram ISO+", 

"OBSERVATION_RESULT_STATUS": "F"}}

$

full video

I hope you enjoy this amazing tutorial

DEV Community: bygregonline

Create a Complete AI Development Environment in Less Than a Minute with Docker Compose.

HL7 protocol decoder programming tutorial

Docker Microservices using django as backend

Second part