DEV Community: Matt Dyor

Prisma Scratch Pad

Matt Dyor — Sun, 10 Apr 2022 14:37:27 +0000

npx prisma db push
npx prisma studio

Things I Learned About Python this Week

Matt Dyor — Sat, 27 Nov 2021 03:02:38 +0000

python manage.py shell

this gets you into an interactive shell

from basic_loans.models import BasicLoan DOES NOT WORK
from dyor_com_app.basic_loans.models import BasicLoanPage
b=BasicLoanPage
from dyor_com_app.base.models import People
p = People
p.first="matt"
p.last="dyor"
bl1 = BasicLoan.objects.filter(id=1).first()

which is the same as:

bl1 = BasicLoan.objects.get(id=1)
bl1.loan_amount

create a record

BL = BasicLoan()
BL.loan_amount
BL.original = bl1

import datetime

BL.loan_start_date = datetime.date(2021, 11, 25)
BL.save()

create an object with inline properties

page = Page.objects.create(title='my title', summary='my summary')
page.__dict__

Django on Windows - First Week

Matt Dyor — Sat, 14 Aug 2021 06:06:44 +0000

I am basing this work on Django for Beginners, but as that book is focused on Linux/Mac I have kept these notes for what I needed to do on my PC. I will keep extending this over the week of Aug 13 (yes, Friday the 13th).

Get Django set up in Virtual Environment

https://codesource.io/setting-up-a-virtual-environment-for-your-django-project/
https://stackoverflow.com/questions/35950740/virtualenv-is-not-recognized-as-an-internal-or-external-command-operable-prog

pip install virtualenv
I had to open an administrator cmd prompt and do
c:\python39\python.exe -m pip install --upgrade pip

For each project you need to create virtualenv and add django

cd into the directory where you want to create python (e.g., C:\Users\madyor\OneDrive - Microsoft\Desktop\python\django\playground)
python -m virtualenv .
.\scripts\activate
pip install django
pip install pandas
run django-admin startproject to verify that django is installed.

Create Your Project

django-admin startproject config . Note: the . at the end places the project in the current directory (instead of creating a new child with that name)
python manage.py migrate
python manage.py runserver If this command fails, you may need to do the .\scripts\activate to get into the virtual environment (where django is installed)
exit gets you out of a virtualenv

Create another app in your project

python manage.py startapp pages
Register the app in config/settings (under installed_apps add 'pages',)
update the views page with the following code

from django.views.generic import ListView, DetailView
from django.views.generic.edit import CreateView, UpdateView
from .models import Page
from django.views.generic import TemplateView

from django.shortcuts import render

class PageListView(ListView): 
    model=Page
    template_name = 'home.html'

class PageDetailView(DetailView): 
    model=Page
    template_name = 'page_detail.html'

class PageCreateView(CreateView): 
    model = Page
    template_name = 'page_new.html'
    fields=['title', 'summary', 'description']

class PageUpdateView(UpdateView): 
    model = Page
    template_name = 'page_new.html'
    fields=['title', 'summary', 'description']

create a urls.py within the pages folder

from django.urls import path
from .views import PageDetailView, PageListView, PageCreateView, PageUpdateView

urlpatterns = [
    path('page/new/', PageCreateView.as_view(), name='page_new'), 
    path('page/<int:pk>/', PageDetailView.as_view(), name='page_detail'), 
    path('page/<int:pk>/edit', PageUpdateView.as_view(), name='page_update'), 
    path('', PageListView.as_view(), name='home'),
]

update the config/urls.py to reference the pages.urls (path('', include('pages.urls')), and add from django.urls.conf import include)

Initiate Git

git init
git status
git Add -A
git commit -m "initial commit"

Create a Repo on Github

git remote add origin https://github.com/dyor/special-octo-journey.git
git branch -M main
git push -u origin main

Day 2

Using Templates

Templates are like views in MVC (and views are like controllers)
In the config/settings.py, update DIRS to this: 'DIRS': [str(BASE_DIR.joinpath('templates'))],

Day 3

Use a base template

In the pages app, create a templates folder and add a home.html and a base.html file
for base.html add some code

<a href={% url 'home' %}>Home</a> 
{% block content %}
{% endblock content %}

in the home.html file add

{% extends 'base.html' %}
{% block content %}
<h1>Homepage</h1>

{% for page in object_list %}
    <div>
    <a href="{% url  'page_detail' page.pk %}">{{ page.title }}</a>
    </div>
{% endfor %}

{% endblock %}

Create a detail page in templates folder called page_detail.html

{% extends 'base.html' %}

{% block content %}

<h1>{{ page.title }}</h1>
{{ page.summary }}
<br/>
<br/>
{{ page.description }}

{% endblock %}

add model to admin.py in admin page

from django.contrib import admin
from .models import Page

# Register your models here.
admin.site.register(Page)

Random Commands

python manage.py createsuperuser
python manage.py makemigrations
python manage.py migrate

Sending Email from Azure

Matt Dyor — Fri, 01 Jan 2021 09:11:42 +0000

First, I tried creating an account directly from Azure and that did not work for me due to a billing issue.

So I ended up first setting an account up with Sendgrid (a free account with up to 25K emails per month...way more than I will need for now). I started by checking out the integration guide - C# for me:

https://app.sendgrid.com/guide/integrate/langs/csharp

The first step there was adding SendGrid Nuget Package

Adding Custom Domain to Azure App Service

Matt Dyor — Thu, 31 Dec 2020 18:16:14 +0000

In App Service, ensure that your App Service Plan is set to a level that supports Custom Domains (e.g., Production S1, runs about $45/month)
In App Service, click on Custom Domains and then Add Custom Domain
There are some steps you need to follow to verify that you own the domain name.
NOTE: in Godaddy, if you are having a hard time validating ownership, add more TXT and CNAMEs - some combination of the following worked for me:
Once your custom domain is validated, you need to add SSL - YOU DO NOT NEED TO PAY FOR ONE. Just click on Binding and follow the instructions for creating a SNI SSL Binding.
Finally, you can add a A record to your DNS in Godaddy that points to your Azure App Service IP address (found on the Custom Domains blade).

Fairly basic, but the documentation between the two sites never seems to be very clear, and I always end up spending more time on this than I should have to. I will add some more details the next time I do this (I captured these from memory after spending a couple of hours over the last few days trying to figure out why it was not working properly).

Adding a Favicon to an Azure Website

Matt Dyor — Fri, 25 Dec 2020 03:23:43 +0000

First, you need to generate a favicon.ico file. Fortunately, the team (person) at Favicon.io have made this very easy and enjoyable.

Next, replace the favicon.ico in the wwwroot folder of your application with the newly created favicon.ico.

Note that I did not add this link to my _Layout file: - and when I did it stopped working.

Building an ASP.NET Core Web App from Scratch

Matt Dyor — Sun, 22 Nov 2020 23:41:36 +0000

Get Started

https://github.com/OdeToCode/OdeToFood
Start with an asp.net core web application in visual studio
Select web application (not web application mvc) with defaults
Add a asp-page to _Layout pointing to Restaurats/list
Create a folder for Restaurants
Add a Razor Page (empty) called List in the Restaurants folder

Adding a property to List

(think of this as model in MVC)

inside the class, type prop TAB TAB string TAB TAB Message that will leave you with public string Message { get; set; }
In appsettings.config add a "Message": "hello world"
Back in List type ctor TAB TAB (this will add a constructor)
Add IConfiguration to () and hit CTRL + . then enter and call the variable config (looks like public ListModel(IConfiguration config)
hit CTRL + . on config and select "create and assign field config"
in the OnGet() method add Message = config["Message"];

Options

If you want to be able to refresh your browser while debugging in asp.net core, you need to add this nuget package:

https://stackoverflow.com/questions/54106058/why-does-page-not-update-after-refresh-when-cshtml-changes
need to use the 3.x version if you are using .net core 3.x

If you want to use code generators, powershell in to project directory:

dotnet tool install --global dotnet-aspnet-codegenerator
dotnet add package Microsoft.VisualStudio.Web.CodeGeneration.Design -v 3.1
dotnet aspnet-codegenerator razorpage -h
dotnet aspnet-codegenerator razorpage List2 Empty -udl -outDir .\Pages\Restaurants

Add Model

Right Click Solution and Add Project
Choose Class Library (.net core)
Delete the created class and add your own class, call it Restaurant, make it public
Add a int Id, string Name, and String Location property
Add an Enum and use CTRL + . to put it into its own class file

public enum CuisineType
        {
            None, 
            Mexican, 
            Chinese
        }

Add a property of type CuisineType
CTRL+K then CTRL+D to format

Add Data Access

Add another library class
Delete the class and add an interface
Add an Interface with a GetAll() method
Implement a public class based on the interface
Add a readonly list of restaurants
You need to add a reference to the Restaurant class project to this project, and then add a using statement so that the restaurants is available in the data class
Create a constructor that populates the list of restaurants
Implement GetAll method - need to add a reference to System.Linq to get the orderby to work

Add Singleton of the data access to startup

Add this to the startup page services.AddSingleton<IRestaurantData, InMemoryRestaurantData>();

Add Data to List

Add the IRestaurantData interface to the data list constructor
CTRL + . on IRestaurantData and add "create and assign field config" - should look something like this public ListModel(IConfiguration config, IRestaurantData restaurantData)

Add Search

Add a form with method=get and no action (will point to itself).
Include a input type=search name=searchTerm
include an i tag with a class="fa-search"
To get font awesome to work, you need to get a fontawesome.io url and add it to your URL; if you want autocomplete you need to right click on the project name > add > client side library > provider=unpkg, search for font awesome. Code is like <i class="fas fa-search"></i>
Replace the GetAll on the restaurants interface with a GetRestaurantByName
In the implementation method, specify that the name is a string name=null, meaning that if not provided no filtering.
Update the linq query to include a where string.IsNullOrEmpty(name) || r.Name.StartsWith(name) that includes all if no filter (remember this filter is currently case sensitive)
Change the OnGet method of List code behind to include Restaurants = restaurantData.GetRestaurantsByName(searchTerm); here the searchTerm is pulled from the Request based on the name of the input field

Show Searched For Term

Instead of pulling the searchTerm from the request, specify a property with a capital S SearchTerm and a BindProperty that means it will Bind when the request is received

[BindProperty(SupportsGet =true)]
public string SearchTerm { get; set; }

Replace name and value on input form to asp-for="SearchTerm" <input type="search" class="form-control" asp-for="SearchTerm" />

Make a Details Page

Right Click Restaurants folder, Add, Razor Page (empty)
Add public Restaurant Restaurant { get; set; } to the PageModel
On the cshtml, you can now use @Model.Restaurant.Name
You can make embed the restaurantId in the url with page "{restaurantId:int}"

Make an Edit Page

Largely the same as Details, but sometimes you need to use @Model and sometimes not

<form method="post">
    <input type="hidden" asp-for="@Model.Restaurant.Id" />
    <div class="form-group">
        <label asp-for="@Model.Restaurant.Name"></label>
        <input asp-for="@Model.Restaurant.Name" class="form-control" />
    </div>
    <div class="form-group">
        <label asp-for="@Model.Restaurant.Location"></label>
        <input asp-for="@Model.Restaurant.Location" class="form-control" />
    </div>
    <div class="form-group">
        <label asp-for="@Model.Restaurant.Name"></label>
        <select class="form-control" asp-for="Restaurant.Cuisine" asp-items="Model.Cuisines" >
        </select>
    </div>
</form>

In the PageModel, you need to pull in the IHTMLHelper

public class EditModel : PageModel
    {
        private readonly IRestaurantData restaurantData;
        private readonly IHtmlHelper htmlHelper;

        public  Restaurant Restaurant { get; set; }
        public IEnumerable<SelectListItem> Cuisines { get; set; }
        public EditModel(IRestaurantData restaurantData, IHtmlHelper htmlHelper)
        {
            this.restaurantData = restaurantData;
            this.htmlHelper = htmlHelper;
        }
        public IActionResult OnGet(int restaurantId)
        {
            Cuisines = htmlHelper.GetEnumSelectList<CuisineType>(); 
            Restaurant = restaurantData.GetById(restaurantId); 
            if (Restaurant == null)
            {
                RedirectToPage("./NotFound");
            }
            return Page(); 
        }
    }

Connect to Local SQL

Add a connection string to localdb

"ConnectionStrings": {
    "OdeToFoodDb":  "Data Source=(localdb)\\MSSQLLocalDB;Initial Catalog=OdeToFood;Integrated Security=True;"
  }

Wire up the app settings via configure services using an options lambda expression:

            services.AddDbContextPool<OdeToFoodDbContext>(options => {
                options.UseSqlServer(Configuration.GetConnectionString("OdeToFoodDb")); 
            });

Add a constructor to the DbContext that pulls in the options from the app settings:

public OdeToFoodDbContext(DbContextOptions<OdeToFoodDbContext> options) : base(options)
        {
        }

Finally need to tell the .Data project to look at the base project for the options information dotnet ef dbcontext info -s ..\OdeToFood\OdeToFood.csproj

Add a Migration

in the CLI run dotnet ef migrations add initialcreate -s ..\OdeToFood\OdeToFood.csproj
then apply the migration dotnet ef database update -s ..\OdeToFood\OdeToFood.csproj

Implement Delete Functionality

Add a Delete function Restaurant Delete(int id);
CTRL+. on the InMemoryRestaurantData and select option to move to its own class file
in the new file, CTRL+. and select to implement interface (this will add the delete)
Add the delete code

       {
            var restaurant = restaurants.FirstOrDefault(r => r.Id == id); 
            if (restaurant != null)
            {
                restaurants.Remove(restaurant); 
            }
            return restaurant; 
        }

Add the SQL

on the IRestaurantData file, type this:

    public class SqlRestaurantData : IRestaurantData
    {

    }

CTRL+. and SqlRestaurantData and select option to move to its own file
in the new file, CTRL+. implement the interface

Connecting Power BI to PostgreSQL on Azure

Matt Dyor — Wed, 22 Apr 2020 20:21:45 +0000

If you want to connect your desktop Power BI to a SQL Server on Azure, it is a pretty simple couple of clicks. PostgreSQL takes a couple of additional options.

On your Windows machine, click on Windows > ODBC Data Sources (32 Bit)
Click on the Drivers tab to make sure you have the PostgreSQL ODBC Driver
Click on the User DNS tab and Add a connection to the PostgreSQL ODBC Driver (I used the Unicode driver, but I am not sure if that is required)
In Power BI, click on Get Data (the top of the icon, not the drop down - if you click the drop down you need to select More...)
Search for ODBC
You should see the Data Source Name you just created as an option. I don't think you need to specify Advanced Options, as the DNS has all of this information.

You are now ready to pull data into Power BI from your Azure-hosted PostreSQL database (and this would probably work on platforms other than Azure as well, but I have not tested them out).

Cheers.

Matt

Migrating a Node App to Docker

Matt Dyor — Mon, 30 Mar 2020 04:14:33 +0000

Good article describing

Migrating a Node App to Docker

Pushing an Existing Node App to Azure

Matt Dyor — Sat, 28 Mar 2020 02:54:19 +0000

If you have code that is working on your local machine, and you want to deploy it to Azure, and you have a DevOps account configured, here is how.

If you were previously deploying somewhere else (like Heroku), delete the .git directory (there is probably a more elegant way to reset git, but delete works)
Configure git in the directory

git init
git add .
git commit -m "initial commit"

In Azure DevOps, create a new project
Navigate to Repos. You will see an option to push an existing repository from command line. Grab that code

git remote add origin https://teamsi@dev.azure.com/teamsi/PilotDataEngine/_git/PilotDataEngine
git push -u origin --all

in the Azure Portal, you want to create a "Web App + PostgreSQL" instead of creating the web app and database separately, and having to stitch the networking and permissions and port configurations together. This is goofily hard to find, so try
- Create a resource
- Search for "Web App + PostgreSQL"
- It may be directly linkable here
I tried using pipelines, but that was not successful, so I just used the VS Code Azure Extension to deploy directly (explained here)
Visit the Overview tab in the Azure portal, click on the URL, and your app should be alive.
If you use Sequelize, adjust your local database settings to point to the Azur PostgreSQL server, and run:

sequelize db:migrate

Next steps:

Figure out how to fire off sequelize from Azure (instead of having to update from the local application)
Figure out how to use Azure Pipelines. I was able to get the Azure Pipelines set up to run upon code upload, but something was not working. If anybody has a blog post talking about configuring Azure pipelines for node (that is current), please share it.

Thanks.

Matt

Testing Out MongoDB + Express + React + Node + Sendgrid + Heroku

Matt Dyor — Sun, 22 Mar 2020 18:34:52 +0000

This post will capture how I got started building a simple web application to capture feedback about a piece of content - anything from an icon design to a business plan to a job posting. Here are the basic requirements:

create a feedback "record" (called a document in Couchbase, but it is just data in JSON form)
upload a piece of content (that is actually a document, such as a PNG, PPT, DOC, etc.) by web or by email
email all users specified in the feedback record with a copy of the document, asking them for feedback
real-time chat between users that updates without a page refresh

Build the basic data model

Although NoSQL databases like Couchbase allow you to worry less about the data model, starting with a clear definition of how you are thinking about your data is a smart first step.

Check out this video for the difference between data modeling in RDBMS v. data modeling in Couchbase: Data Modeling in Couchbase

Here is a first cut of the Couchbase data model for a document, which is the core object in the system - the thing that we want to get feedback on:

{
   "document":{
      "id":"integer",
      "title":"string",
      "type":"string",
      "createdDate":"datetime",
      "updatedDate":"datetime",
      "reviewers":[
         {
            "name":"string",
            "email":"string",
            "rating":"int"
         },
         {
            "name":"string",
            "email":"string",
            "rating":"int"
         }
      ],
      "comments":[
         {
            "name":"string",
            "email":"string",
            "body":"string"
         },
         {
            "name":"string",
            "email":"string",
            "body":"string"
         }
      ]
   }
}

SQL v. NoSQL

SQL: you need to invest heavily in your database design, because you immediately create a lot of scaffolding around the design - in the database, in the backend, and in the frontend. Simple changes to the data model require massive amounts of changes in your code base.
NoSQL: you get to start with a simplified data model much quicker, and then you can evolve the data model over time. For example, instead of creating a user table, a comments table, and a documents table, and then associating the user table to the documents table through a many-to-many bridge table, etc., you can just start with the data structure above to get started. There are key scenarios that will need to be addressed down the road, such as what happens if a person updates their email address - how do you update all of the places where that email address exists. But why not solve those problems later, and focus on your most important problem up front: can we find a pain worth solving that we can address with software we can build.

Get Running on Heroku

This assumes that you have Visual Studio Code running on your machine.

A great reference for getting started on Heroku is provided by Heroku itself: Getting Started with NodeJS. It comes down to:

Install Heroku Client - this page also shows you how to check if you have Node, npm, and git installed.
Create your own Node Express app: this is NOT part of the Heroku tutorial, but instead of just copying their app we are going to write our own using another tutorial: NodeJS API Tutorial - just watch to minute 4 (unless you want to learn about Docker). key steps:
- create a folder called...heroku
- open the folder in Code
- create a file in Code called app.js
- paste in this code

const express = require('express')
const app = express()

app.get('/', (req, res) => res.send('I am alive!')); 

app.listen(3000, () => {
    console.log('I am alive on port 3000!'); 
});

Create your own Node app (cont)
- in code, hit Terminal > New Terminal
- in the Terminal, type

npm init

Create your own Node app (cont)
- accept all the defaults
- Open the pakcage.json file that was created with the last command
- back in Terminal, type

npm install --save express
npm install --save nodemon

Create your own Node app (cont)
- you should see express appear as a dependency in package.json.
- in the package.json file, inside the scripts element, add a start instruction

"scripts": {
    "start": "nodemon app.js", 
    "test":...
}

Create your own Node app (cont)
- to start the application, in the terminal type

npm run start

Create your own Node app (cont)
- head on over to localhost:3000 and you should see your shiny new app. Whenever a change is made, nodemon will pick it up and republish your app. Try it.
- when you are done testing your app, type CTRL+C in the Terminal to terminate the job.
Now that you have created your node app, you can configure the app for Git.

git init

and then you can publish it via the Terminal with

heroku create

This will create a home for your application on Heroku and create the git connection between your machine and the Heroku git repository.
Now you are ready for my favorite haiku:

git add .
git commit -m "my first commit"
git push heroku master

There should be a funny name for your application, such as https://shrouded-atoll-61606.herokuapp.com/

Get Mongo running

First, install Mongo on your local machine. Then get the node mongo driver:

npm install --save mongodb

Add this to your app.js file

const MongoClient = require('mongodb').MongoClient;
const uri = "mongodb+srv://<username>:<password>@<your-cluster-url>/test?retryWrites=true&w=majority";
const client = new MongoClient(uri, { useNewUrlParser: true });
client.connect(err => {
  const collection = client.db("test").collection("devices");
  // perform actions on the collection object
  client.close();
});

Now that you have everything working locally, you need to configure it for Heroku. mLab has a free sandbox option for running MongoDB, so we will use that. Go to your Heroku dashboard, find your application, click "Configure Add Ons" and search for mLab MongoDB. Add the free version.

More to come soon!