DEV Community: João Osório

Preferred Programming Language Versus Getting Things Done

João Osório — Thu, 09 Dec 2021 12:10:59 +0000

Getting skills in more than one programming language is a must for professional developers and multitaskers. You can try creating a toolbelt of ready-to-use code snippets, bookmark all the handy libraries, and even practice hypothetical situations. That's what I had in mind working on my automate-dotnetcore project.

But now and then you'll face some important task that needs to be done on the spot, with no delays. Usually single-shot operations. For that, the key performance indicator is GTD = Getting Things Done.
So forget your personal preferences, your level of comfort with any of the programming languages you have some seniority on, you just want to deliver, good and fast.

At the end of the day, what matters is that you do not add friction by depending on what you're comfortable with, you must be open to trying out new things. But remember, this comes with another philosophy in mind: if you're going to fail, fail fast.

"Fail fast is a philosophy that values extensive testing and incremental development to determine whether an idea has value."

To give you an example of what I'm writing about, consider the task of going through a folder filled with thousands of PDF files. You must read all of them, extract some text according to a given pattern and output the filename together with the extracted text.

Sure, I could fire up Visual Studio, begin to write a C# console project with some randomly chosen NuGet packages. Or... I could google for some wisdom from the Internet Archives (ex: Stack Overflow).
I did nothing of the sort. I created a local folder, opened it in VS Code, created a Virtual Environment for Python, and wrote a program based on Github Copilot suggestions. After some fine-tuning, this was the short, simple and effective result:

import PyPDF2
import re
import glob

def main():

    #iterate through all the pdfs in the folder
    for pdf in glob.glob("PDFs\\*.pdf"):
        #open the pdf
        pdfFileObj = open(pdf, 'rb')
        #create a pdf reader object
        pdfReader = PyPDF2.PdfFileReader(pdfFileObj)
        pageObj = pdfReader.getPage(0)
        page_one_text = pageObj.extractText()
        #close the pdf file object
        pdfFileObj.close()

        pattern = '[your pattern]'
        result = re.findall(pattern, page_one_text)
        print(pdf, result)

if __name__ == "__main__":
    main()

Next time someone asks you for some improvisation in order to achieve immediate results, try something new!

Photo by John Baker on Unsplash

Python script to unfollow inactive friends on Twitter

João Osório — Mon, 25 Oct 2021 14:35:57 +0000

Twitter is great to learn the news of your chosen interests, but sometimes your get stuck with an increasing amount of inactive friends who haven't updated their status in ages. It's a daunting task to go over all of your friends, check their status, and unfollow the ones not using Twitter for a long time.

I've written a Python script to do this. The script works with the following python packages:

Tweepy
configmanager

The required dependencies are on the requirements.txt file. To install them run:

python -m pip install -r requirements.txt

Script Configuration

Configuration file location

The project comes with an example.ini file that you can use to fill with your own Twitter API credentials.
Don't forget to change the following line on the script:

config.read('..\..\TwitterAPI\config.ini')

It should point to wherever you save your .INI file with the authorization tokens.

Configuration parameters

API.Key and API.Secret: Think of these as the user name and password that represents your Twitter developer app when making API requests.
Bearer.Token: OAuth 2.0 Bearer Token authenticates requests on behalf of your developer App.
Access.Token and Access.Secret: user-specific credentials used to authenticate OAuth 1.0a API requests.
General.ScreenName: your twitter account screen name.
General.BatchSize: the maximum number of friend acconts you wish to be able to remove on each run.
General.DaysInactive: the time threshold to consider a friend as inactive, in days.

The script

The program starts by importing the required modules for execution:

import configparser
import tweepy
from datetime import date

The first is for reading the configuration file, the second is to work with the Twitter API and the third is to be able to find the difference in days between two given date.

Before calling the twitter API, you must have the Twitter Developer account and a set of App's keys and tokens. Read more about it on the official portal.

On the begining of the program, the Twitter App keys and tokens are read from the INI file:

config = configparser.ConfigParser()
config.read('..\..\TwitterAPI\config.ini')

The path should point to your INI file. The contents must follow the structure presented on the example.ini sample:

[API]
Key = <your api key>
Secret = <your api secret>

[Bearer]
Token = <your bearer token, for >

[Acess]
Token = <your access token>
Secret = <your access secret>

[General]
ScreenName = <the target user>
BatchSize = 10
DaysInactive = 5

After reading the INI file, we pass the tokens for authorization and create the API object:

auth = tweepy.OAuthHandler(config['API']['Key'], config['API']['Secret'])
auth.set_access_token(config['Acess']['Token'], config['Acess']['Secret'])

api = tweepy.API(auth, wait_on_rate_limit=True)

The Twitter API has thresholds for the allowed rate on API calls. The wait_on_rate_limit flag configures a sleep time when the allowed rate is reached.

The script then tests calling the API by getting the User object and initializes an empty list of inactive friends:

user = api.get_user(screen_name=config['General']['ScreenName'])
inactive_friends = [];

Next we go through all the friends and check their last status. If they haven't tweeted for more than the configured DaysInactive parameter, their account is added to the inactive friends list:

for friend in tweepy.Cursor(api.get_friends, screen_name=user.screen_name).items(): 
        tweets_list= api.user_timeline(screen_name = friend.screen_name, count = 1)
        tweet= tweets_list[0] # last status of this friend (tweepy.models.Status)

        delta = date.today() - tweet.created_at.date()
        if (delta.days > int(config['General']['DaysInactive'])):
            inactive_friends.append(friend)

If the program has already reached the maximum number of intended inactive friends to remove, it exits the loop:

if (len(inactive_friends) >= int(config['General']['BatchSize'])):
    break

If there were inactive friends detected, the user's consent for removal is confirmed and those account will be unfollowed:

    if (len(inactive_friends) > 0):
        print('The following % s friends are inactive for more than 5 days:' % len(inactive_friends))
        for friend in inactive_friends:
            print(friend.screen_name)

        print('Unfollowing %s inactive users..' % len(inactive_friends))
        answer = input("Are you sure? [Y/n]").lower()
        if answer and answer[0] == "y":
            for friend in inactive_friends:
                print("Unfollowing " + friend.screen_name)
                friend.unfollow()

Please note, when creating the Twitter app on the developer portal, you must define it as read/write, otherwise the unfollow command will fail with an error.

The full source code is on Github.

EF Core Update Model on Database First Project

João Osório — Tue, 07 Sep 2021 15:59:23 +0000

Started on a software project that will run using an existing database. Went with a minimalistic structure and used the EF Core Tools, from the Package Manager Console in Visual Studio, to generate the database context and table model classes. Namely scaffold-dbcontext. Used the '-t' option to create just the model classes I would use. It went very well and development went on, business as usual.

As it turned out, I missed a table, so I had to add the model class for that and update the database context. Of course I thought that Visual Studio should have some tool our wizard like it had when using EDMX files. I was wrong, Entity Framework Core has not added support for EDMX generation.

As I had no business related code inside the model classes, or any special customization, the simplest scenario was to reverse engineer the model from the database again using scaffold-dbcontext with the '-Force' option. Mind that if you reverse engineer the model from the database again, any changes you've made to the files will be lost.

Nevertheless, if you're more into GUI than CLI, you should try out EF Core Power Tools. Check out the demo from .NET Conf 2020 on youtube.

Original post can be found here

Image created by vectorjuice - www.freepik.com

Contract First Web Service Development in .NET

João Osório — Fri, 07 May 2021 11:01:07 +0000

Working on information systems integration often comes with the task of implementing a Web Service following a previously defined contract. This can happen with a design-first approach or a partner interoperability project where you must implement the expected receiver for the incoming data.

To develop an XML web service, having an existing WSDL, we can use the tooling provided with Microsoft Visual Studio.

We will use an ASMX project. If you're worried about support because this is no longer a part of .NET from version 5 upward, I'd advise you to read this guy's opinion:

To be accurate, both WCF and ASMX will be supported after .NET 5 is released. They will be included in .NET 4.8 and that will be supported as a component of the OS that they are installed on.

Sure, we could use Soap Core, but to work with existing contracts I find it to be a very manual solution and with some risk of inadvertently altering the schema.

Getting down to business, let's go through the steps of a quick and easy proof of concept for an existing, simple, and free of charge public web service:

www.learnwebservices.com/services/hello?WSDL

Download the WSDL file onto a folder on your local computer.
You'll need Visual Studio installed. Open "Developer Command Prompt for VS 2019":

On the command prompt, go to the folder where you saved the WSDL file.
Run the WSDL command:



wsdl /namespace:My.Contract.First /serverInterface <file.wsdl>

Where <file.wsdl> is the name you gave when saving the contract. Choose an adequate namespace also. Upon successful completion of this command you should have an interface definition corresponding to the methods available on the given Web service:



// 
// This source code was auto-generated by wsdl, Version=4.8.3928.0.
// 
namespace My.Contract.First {
    using System;
    using System.Web.Services;
    using System.Diagnostics;
    using System.Web.Services.Protocols;
    using System.Xml.Serialization;
    using System.ComponentModel;


    /// <remarks/>
    [System.CodeDom.Compiler.GeneratedCodeAttribute("wsdl", "4.8.3928.0")]
    [System.Web.Services.WebServiceBindingAttribute(Name="HelloEndpointServiceSoapBinding", Namespace="http://learnwebservices.com/services/hello")]
    public interface IHelloEndpointServiceSoapBinding {

        /// <remarks/>
        [System.Web.Services.WebMethodAttribute()]
        [System.Web.Services.Protocols.SoapDocumentMethodAttribute("", RequestNamespace="http://learnwebservices.com/services/hello", ResponseNamespace="http://learnwebservices.com/services/hello", Use=System.Web.Services.Description.SoapBindingUse.Literal, ParameterStyle=System.Web.Services.Protocols.SoapParameterStyle.Wrapped)]
        [return: System.Xml.Serialization.XmlElementAttribute("HelloResponse")]
        helloResponse SayHello(helloRequest HelloRequest);
    }

    /// <remarks/>
    [System.CodeDom.Compiler.GeneratedCodeAttribute("wsdl", "4.8.3928.0")]
    [System.SerializableAttribute()]
    [System.Diagnostics.DebuggerStepThroughAttribute()]
    [System.ComponentModel.DesignerCategoryAttribute("code")]
    [System.Xml.Serialization.XmlTypeAttribute(Namespace="http://learnwebservices.com/services/hello")]
    public partial class helloRequest {

        private string nameField;

        /// <remarks/>
        public string Name {
            get {
                return this.nameField;
            }
            set {
                this.nameField = value;
            }
        }
    }

    /// <remarks/>
    [System.CodeDom.Compiler.GeneratedCodeAttribute("wsdl", "4.8.3928.0")]
    [System.SerializableAttribute()]
    [System.Diagnostics.DebuggerStepThroughAttribute()]
    [System.ComponentModel.DesignerCategoryAttribute("code")]
    [System.Xml.Serialization.XmlTypeAttribute(Namespace="http://learnwebservices.com/services/hello")]
    public partial class helloResponse {

        private string messageField;

        /// <remarks/>
        public string Message {
            get {
                return this.messageField;
            }
            set {
                this.messageField = value;
            }
        }
    }
}

Open Visual Studio and create a new empty ASP.NET Web Application (.NET Framework):

Right click on your project and "Add > Existing Item". Browse to your .CS interface file, click "Add":

Now Add a new ASMX item:

Implement the Interface generated by the WSDL tool on your ASMX:



using System.Web.Services;

namespace My.Contract.First
{
    /// <summary>
    /// Summary description for HelloService
    /// </summary>
    [WebService(Namespace = "http://tempuri.org/")]
    [WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
    [System.ComponentModel.ToolboxItem(false)]
    // To allow this Web Service to be called from script, using ASP.NET AJAX, uncomment the following line. 
    // [System.Web.Script.Services.ScriptService]
    public class HelloService : System.Web.Services.WebService, IHelloEndpointServiceSoapBinding
    {

        [WebMethod]
        public helloResponse SayHello(helloRequest HelloRequest)
        {
            helloResponse resp = new helloResponse();
            resp.Message = "Hi there " + HelloRequest.Name;
            return resp;
        }
    }
}

Build and Run. That's all for developing the stub web service using an existing WSDL contract.

The SOAP/XML Web services are gradually losing ground to REST APIs for their lack of simplicity and for failing to deliver on the promise for guaranteed cross-platform integration. But most enterprise applications still use it for the enhanced protocol features like formal XML schema definition, transactionality, and security.

So, if you're building something new, give REST a try. If you're maintaining previously developed platforms, better off using the existing tooling.

Original post here.

Background photo created by natanaelginting - www.freepik.com

Automatic class creation for unmarshalling JSON

João Osório — Tue, 30 Mar 2021 09:25:34 +0000

If you're working with JSON, say for REST web services, and you don't have the class to unmarshal the response, Visual Studio can automatically generate the class for you:

Open your Visual Studio Project and create a class file. Delete the contents:
Copy the JSON text string you will need to unmarshal onto the clipboard. Take this example from the JSON Placeholder mock API:

{
    "userId": 1,
    "id": 1,
    "title": "delectus aut autem",
    "completed": false
}

Go to Edit > Paste Special > Paste JSON as Classes: The result is a class named "Rootobject" that you can use to unmarshal the information on your code.

Original post is here.

So you hate those whiteboard interviews, huh?

João Osório — Fri, 22 Feb 2019 17:10:39 +0000

A lot of people have started to learn to code. If you got to social networks hashtags related to coding are a rising trend. The site Hackernoon even has an article on the subject.
The catch here is that everyone can learn to code, but that doesn't necessarily make them a good or even proper developer.
A lot of programmers on the market are able to build stuff with some cookbook recipes and then surf the Web (i.e. Stackoverflow) until it's no longer a broken piece of software.
Employers need to assess if you're the real deal or not. So you can prove your worth with decent activity on an open source side project, past project experience where your old boss can recommend your work, being a certified professional. But maybe you don't have the time (or luck) to have none of these things.
When all else fails, accept the challenge and show them what you've got. Think of it as public speaking mixed with a training session where you are the trainer.
If you think anyone can learn something, than you have to stand out from the rest.

Knowing how to move the pieces of a chess game doesn't make you a good player.