DEV Community: Freddy Adiv

Simple ChatBot with tests

Freddy Adiv — Sat, 17 Jun 2023 12:18:26 +0000

Hi all,
The NLP field is pretty amazing, specially with ChatGPT4 around, so as part of my MSc studies I tried to learn a bit about the techniques used to build such AI systems.
I looked around in Github and found several repositories implementing chatbots, so I used some of them to build this Encoder/ Decoder based chat model.
It uses an intents.json file which is contains several input patterns and optional responses.
The readme file relates to some of the sources I used.

Using the chatbot:

Clone the repository to your local computer:
git clone https://github.com/FredAdiv/Chatbot
(Optional) I suggest you use a docker container for the project. This is a sample command line for creating such a container:
docker run -it --name chatbot -w /opt -v c:\users<username>\chatbot:/opt python:3.11 bash
If the container was created but didn't open then
use the following command:
docker container start -i chatbot
Install dependencies from the requirements file:
pip install requirements.txt
(Optional) Use pytest to make sure that everything works:
pytest
Run the bot:
python chat.py

Happy Chatting.

Workflows with github

Freddy Adiv — Tue, 13 Jun 2023 03:38:37 +0000

Adding actions on GitHub allows you to automate various tasks and workflows within your repositories. Actions are event-driven scripts that run in response to specific triggers. Here's a guide on how to add actions on GitHub:

Access the Repository

Log in to your GitHub account and navigate to the main page.
Select the repository where you want to add actions. If the repository doesn't exist, create a new one by clicking on the "New" button, or clone an existing repository from another user.

Create the Actions Workflow File
The easiest way to configure actions is to use the github Actions button in the repository, but in this post I will cover how to add it manually.

Inside the repository (bash), navigate to the root directory or the .git folder.
If the .git folder doesn't exist, create it.
Inside the .git folder, create a new folder named workflows.
In this folder, create a new YAML file with a descriptive name, such as actions.yml.

Define the Workflow

Open the YAML file you created and define the structure of your workflow.
Start by specifying the name and on fields. The name field is a descriptive name for your workflow, while the on field defines the trigger event that will initiate the workflow. For example, you can set it to push to trigger the workflow whenever code is pushed to the repository.
Add the necessary jobs to your workflow. Jobs are the individual units of work that your workflow will perform. Specify a name for each job and define the steps it needs to execute.

Configure Actions

Within each job, you can configure actions to perform specific tasks.
Actions are pre-built or custom scripts that are executed within your workflow.
You can choose from a wide range of pre-built actions available in the GitHub Marketplace by searching for specific functionality.
If you have a custom script, you can create an action within your repository by defining the necessary steps.

Commit and Push

After defining your workflow and configuring the necessary actions, save the YAML file.
Commit the file to your repository by staging and adding a commit message.
Push the commit to the repository.

Test the Workflow

Once you push the commit, GitHub will automatically start processing the workflow defined in your YAML file.
You can observe the progress and logs of your workflow by navigating to the "Actions" tab in your repository.
GitHub will display the status of each job and the output generated by the actions.

As an example, I added a Workflow for penn cloned to my user.
In this workflow, I added an automatic installation of all requirements and run the test whenever the user pushes or PR the repository.

Thanks Gabor for providing sample CIs in the course.
Cheers and happy workflowing :-)

Adding CI for projects

Freddy Adiv — Thu, 18 May 2023 07:23:16 +0000

Hi all,
As part of the OSDC course, we were asked to add continues integration (CI) for an open source project.
I chose Penn since it have tests but didn't have any CI configured.
These are the steps to add the CI to the project:
1) Fork the project so you can edit it
2) Add an Action -> Continues integration -> Python project, which contains a good starting point for both installation and testing the repository
3) I changed the file so it would fit the installations requirement and added the pytest command like this:

This workflow will install Python dependencies and run tests with a single version of Python for Penn repository

name: Testing Penn

on:
push:
branches: [ "master" ]
pull_request:
branches: [ "master" ]

permissions:
contents: read

jobs:
build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
  uses: actions/setup-python@v3
  with:
    python-version: "3.10"
- name: Install dependencies
  run: |
    pip install -r requirements.txt && pip install -e .
    pip install pytest
- name: Test with pytest
  run: |
    pytest

4) To activate the workflow, you need to select it from the Actions menu and run it manually.
That's it..
Cheers,
Freddy

Useful commands in git and dockers

Freddy Adiv — Sun, 14 May 2023 06:30:41 +0000

Hi All,
In the past last weeks of our OSDC training we've learned to setup local repositories from github uding dockers.
These lessons required the use of plenty new commands (mostly linux) that I"ve documented in this post. Hope it will be helpful to you.

Basic linux commands:

ls -l (alias ll)
ls -al /root/ (show hidden files in the root dir. ll does the same)
cd /home/ (user directory)
whoami
alias (which asliases are configured in ~/.bash_aliases)

Change working user to root

Launch a specific one time process as root without changing the active user (super user do)

sudo

print current working directory

pwd

Handling directories

mkdir
rmdir
cd ../../home/
cd ~/

Files

cat filename (View file content as text)
more filename (with paging. Up/ Down/ Q to scroll and quit)
echo some text > filename.txt (Create/ override to new file)
echo more text >> filename.txt (Append to existing file)

list all commands history for current session/ container

history

Dockers

Dockers hub is your first place to visit

Run a container with specific OS

docker run -it -w /opt --name ubu ubuntu:22.10
docke r run -it --name flask-dev -w /opt -v c:\users\freddyad\flask:/opt python:3.11 bash

Exit the container but keep the same state

exit

List all running docker containers

docker ps

List all available docker containers

docker ps -a

Open exitsting docker

docker container start -i ubu

Delete container

docker container rm ubu

Get list of all updates and apps for the current container's OS

apt-get update
apt-get install package1 package 2

Resource meter for the docker (requires installation)

htop

Simple file editor (requires installation)

nano filename (text editor)

Git bash

Clone a repository from github

git clone https://github.com/path...

Creating a new git project (from MS terminal or bash outside the docker)

mkdir
cd
notepad README.md
(# Add some test to the file, such as a title of the project)
git init
(adds the folder /.get/ with configuration for the new project)

Show any changes in the local repository

git status

git add readme.md
(use tab to auto complete file name)

(try git status to see the new file in staging)

git commit -m "Description of the commit"

git config --global user.email "mail@domain.com"
git config --global user.name "my name"

Show configuration file

cat ~/.gitconfig
Note that there is a config file in the user directory and in a project directory

History of commits

git log

notepad ~/.gitconfig

return to last directory (only on bash)

cd -

Uploading a new project to GitHub

To upload a new project to GitHub, use the web site to create a new repository, no need to keep the folder's name

Attach the project to the github repository. Use the SSH as the key
git remote add origin git@github.com//.git

Show the project's corresponding repository on github

git remote -v

Change the remote repository

git remote set-url origin git@github.com//.git
(Can be done directly in the project's config file)

Use the main branch

git branch -M main

Adding a public key to the current user on the local machine

Check if there is a public key in the local computer

ls -l ~/.ssh/

Generate a key

ssh-keygen
(You don't have to add passphrase)

Show the public key to the config file

cat ~/ssh/id_rsa.public

Copy the generated key to the github user ->settings -> ssh and GPG keys -> new SSH key. Add a meaningful name to the key, such as "current computer name"

Push the changes in the local project to the github repository, and set the origin and the main in the config file

git push -u origin main
(All new files and commits are now visible on github)

Regular changes in the project after the first configuration

git add
or git add . (all files)
git commit -m "explanation of the commit"
(Using this command without a -m will open the default editor for the commit details)
git status
(ahead/ staging)
git push

Creating new branch

git checkout -b
git status (show the active branch)

Creating forks

Creating forks is done only on github's web site.
This enables users without permissions to update the fork copy.

Push changes to the new fork:

git remote add fork git@... (ssh code of the fork because we are using public key)
git remote -v (view the association of both main and forks)

push changes to the fork

git push --set --upstream fork .

Sync local copy with the github repository

git checkout main
git pull origin main (as configured in the config file)

Show all changes in the repository on all branchs

gitk --all & (will open in different thread)

rebase the changes in the repository

(This will PR the changes as if they were created after the last change in the base fork)
git checkout
git rebase main

Delete last commit

git reset head~1 --hard

Override remote with our changes

git push --force

Merge fork with the main version

git checkout main
git checkout fork
git merge main

Hope this will be helpful for anyone...
Cheers,
Freddy

Where's your author?

Freddy Adiv — Thu, 27 Apr 2023 13:28:49 +0000

Hi all,
Continuing fixing some sites without an author, I have came across these projects:
ApiTestEz which is missing the pyproject.toml file. I added it with the author's name, but it still missing the email address. PR

lightgrad was also missing the author id, so I added the toml file. Unfortenetly, the Github user has no details so I just used the user name. PR
The author accepted the PR in merged it into the project :-)

gbs has a toml file, but without the author section, so I added it. PR

autosrt is also missing the pyproject file so I added it. PR

pheno-utils has no indication about the author so I added the toml file. PR

Harnessing the power of Dockers with Python projects

Freddy Adiv — Thu, 27 Apr 2023 12:39:59 +0000

Hi all,
As part of the #OSDC course, we learned how to create docker containers for various project environments.
You can install the Docker platform from Docker official site.
For python projects, we used the Ubuntu images for python projects, and mono for .net framework on linux OS.
To create a docker container, you can use the Docker Run command that automatically installs the proper OS, ie:
docker run -it -w /opt --name ubuntu:22.10
or you can clone a project from GitHub and then create a container as follows (this is for python 3.11 project):
docker run -it --name -w /opt -v :/opt python:3.11 bash
To run the docker with the newly created container, use the following command:
docker start -i
The i stands for interactive mode.

To install all required packages for the project, type the following command from within the container:
pip install -r && pip install -e .

That's it.. Your all set to go..

Some interesting Github project to follow

Freddy Adiv — Fri, 14 Apr 2023 18:51:43 +0000

Hi all,
I took a walk on the trending project on Github so I could follow on interesting ones, and found the following projects:

Auto-GPT is an experimental open-source attempt to make GPT-4 fully autonomous. Spooky..

AFFiNE is a next-gen knowledge base that brings planning, sorting and creating all together. Privacy first, open-source, customizable and ready to use. much like Miro or Notion.

GoReplayis an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data.

Will post more interesting founding in the coming weeks.

Cheers,
Freddy

Trying to fix things...

Freddy Adiv — Sun, 02 Apr 2023 07:58:53 +0000

I checked several projects on PyDigger to fix missing tags.
These included:

GitCola which is missing the author name, but when checked the pyproject.toml file, there is a name tag as follows:
authors = [
{name = "David Aguilar", email = "davvid@gmail.com"}
]
I opened an issue in PyDigger.
torek is missing the pyproject.toml file so I added it to the project and opened a new PR.

3.
python-timeout is also missing the pyproject.toml file so I added it to the project and opened a new PR. There author has no email address...

4.
django-markdownx is missing the pyproject.toml file. I PR for the new file.

5.
rfin_scraper was also missing the pyproject.toml file so I created it.

Cheers everyone and happy holidays.

Freddy

My personal site on Github

Freddy Adiv — Mon, 27 Mar 2023 10:06:45 +0000

Hi all,
This week we have learned how to use Git Bash to create a personal web site.
You can see mine at: https://fredadiv.github.io/
Still struggling with VIM for text editing :-(

Cheers,
Freddy

Summery of lesson 2

Freddy Adiv — Sun, 19 Mar 2023 08:46:57 +0000

Hi all,
In our last session we learned several topics, such as navigating and following articles in dev.io using tags which is pretty easy, branching git projects and opening issues.
Later on Gabor showed how to follow project and organizations in Git, and explained about the project ranking mechanism in Github.
We concluded the lesson with the YAML file format which seems more robust then JSON or XML.
I got a bit lost with the explanation of crating the organization YAML in the osdc-2023-azriely project, and didn't really understood the interaction with the organization-open-source file. Would appreciate if we could spend some more time next lesson understanding why we should add such a file to our project, and perhaps talk about the structure of project files and directories.
I still feel that I lack more info about committing pull request and the CI mechanism.
Thanks and have a lovely week,
Freddy

This is a test post

Freddy Adiv — Wed, 15 Mar 2023 11:24:02 +0000

Just learned how to integrate Github, slack, dev.to and code-maven all together.
Cheers,
Fred