DEV Community: Kamal Hossain

How to run already downloaded model in ollama

Kamal Hossain — Fri, 20 Sep 2024 02:04:56 +0000

Intro

In this blog we are going learn how to run already downloaded .gguf files into ollama. For this we are going to use Windows machine today. For other operating system the process is very much similar, you just have to change the path as needed.

Downloading and installing the necessary files

First you have to install ollama in your machine. You can visit and download it from there and install into your computer.

For this tutorial you need a .gguf file which you can download from various places. We are going to download it from huggingface.co. We are going to use Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf model file.

You can search this name at huggingface.co and download it from there.

Running the model into ollama

Make sure your ollama is installed properly by opening the command prompt and run ollama -v.

Then follow these steps:

Move the downloaded Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf file to this folder C:\Users\YOUR_USER_NAME\.ollama\models of your windows.
Create a new folder called modelfiles at this path C:\Users\YOUR_USER_NAME\.ollama
Inside the modelfiles folder create a file called Meta-Llama ( you can use any supported name you want ) and open it in your any text editor. Don't give any file extension to it, like .txt or anything.
At the file give the path of your downloaded .gguf file. In our case the path is ***C:\Users\YOUR_USER_NAME\.ollama\models\Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf*.
- So in the Meta-Llama file we are going to write:
FROM C:\Users\YOUR_USER_NAME.ollama\models\Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf
Now save the file and open the command prompt at modelfiles directory. (You can just press ctrl+l, type cmd and hit enter to open the command prompt )
Now type

ollama create meta-llama -f C:\Users\YOUR_USER_NAME\.ollama\modelfiles\Meta-Llama

I am giving meta-llama as the name, you can give yours.

It should take few moments based on your hardware and create the file.

To confirm type: ollama list. You should see the details of the create model as NAME: meta-llama:latest.

Now to run the model type: ollama run meta-llama. And start chatting with the model.

Upload file to Google cloud storage without middleman server

Kamal Hossain — Mon, 27 Feb 2023 08:44:51 +0000

In my previous blog I discussed about how to upload files from Node.js server to Google Cloud Storage. So, in that case users have to upload the data to the Node.js server and then we are uploading the data to the Google Cloud Storage server.

So, think for a second, if a user wants to upload a very large file and then if we want to upload that large content to the Google Cloud Storage then we have to handle the large amount of data in our server.

So what we can do, we can give some kind of authorization to the user to upload that particular file directly from their device. Without passing the data to our server. So for that we can create an endpoint to our server and then provide a Signed Url to the user. In that way, a user can use this signed url for a limited amount of time to upload his content to the Google Cloud Platform.

In my case I will use a Node.js server for that. We just have to create an index.js file like this one.

const http = require('http')
const { Storage } = require('@google-cloud/storage')

const storage = new Storage({
  keyFilename: `./key-file-from-service-account.json`,
})

const bucketName = 'my-test-bucket'

const server = http.createServer(async (req, res) => {
  // Handle GET requests
  if (req.method === 'GET') {
    res.writeHead(200, { 'Content-Type': 'text/plain' })

    const fileName = `uploadedFileName.jpeg`

    // These options will allow temporary uploading of the file with outgoing
    // Content-Type: image/jpeg header.
    const options = {
      version: 'v4',
      action: 'write',
      expires: Date.now() + 15 * 60 * 1000, // 15 minutes
      contentType: 'image/jpeg',
    }

    // Get a v4 signed URL for uploading file
    const [url] = await storage
      .bucket(bucketName)
      .file(fileName)
      .getSignedUrl(options)

    res.end(url)
  }
})

const port = 3000
server.listen(port, () => {
  console.log(`Server running on port ${port}`)
})

Here above we are initializing the bucket with @google-cloud/storage package.

Then inside the GET request we are calling the getSignedUrl() to get the signed url with some predefined config above. That's it now with the URL user can upload the content from any device they want.

Here I am not doing any kind of authorization for our users. But in real life scenarios we must validate the user with some kind of authentication to make sure not everyone gets the Signed url.

Upload file to Google cloud storage from Node.js server

Kamal Hossain — Sat, 25 Feb 2023 04:17:55 +0000

Cloud storage is one of the essential part of web and mobile application. Google cloud storage is a popular cloud storage. In this blog, we will see how to upload files to Google Cloud Storage using Node.js server. Lets jump into it.

Step one: Getting Keys for authentication

Before upload anything to the Cloud storage we must have the authorization for it. For that, we have to do the following steps:

Create a new service account if you don't have one.
Click on Create service account
Give you key a name and skip the optionals entry for now and click Done, as we don't need thos for this task.
Now you can see your newly created Service account on the service account list. As you can see below it doesn't have any Key now. So we need to create one.
Open the Service account and click on Add key then Create new key. Keep the it as json format. And then download the key to root of your Node.js project.

Step two: Giving service account the write permission to the bucket

Get the email for your service account you have just created above
Go to your bucket that you want to upload the content, and click on the PERMISSIONS tab to get the GRANT ACCESS button.
Paste the email address of service account at the New principals input, and then you should see your service account and click on it.

Now on Assign roles, select Storage Object Creator

Now your service account has the permission to create new objects at the bucket. Click on the Save.

Step three: Upload an image to Google cloud storage

For these steps, you must have Node.js installed in your computer along with npm. Also a bucket must be created already. Although it is possible to create the bucket from the Node.js application, but we are not doing it here.

Create a file called index.js at the root of your folder. And also keep the key file on the same level you have created from Step one.
Run npm i @google-cloud/storage
Now inside the index.js file, add these lines:

const { Storage } = require('@google-cloud/storage')

// Initialize storage
const storage = new Storage({
  keyFilename: `./key-file-from-service-account.json`,
})

const bucketName = 'my-test-bucket'
const bucket = storage.bucket(bucketName)

// Sending the upload request
bucket.upload(
  `./image_to_upload.jpeg`,
  {
    destination: `someFolderInBucket/image_to_upload.jpeg`,
  },
  function (err, file) {
    if (err) {
      console.error(`Error uploading image image_to_upload.jpeg: ${err}`)
    } else {
      console.log(`Image image_to_upload.jpeg uploaded to ${bucketName}.`)
    }
  }
)

Make sure your key file and the image is the same root folder of your project. Or make sure to put appropiate path of these files if they are stored in another location.
Node run node index.js from command line. It should upload the image file to the Google Cloud Bucket.

Additionally don't forget to put your key file to .gitignore file.

Extra step: Making the content public after uploading

If you want to make the object publicly accessible from internet then you can make it public from Node.js right after the successfull upload. Just add these extra lines after the successfully uploaded message.

const { Storage } = require('@google-cloud/storage')

// Initialize storage
const storage = new Storage({
  keyFilename: `./key-file-from-service-account.json`,
})

const bucketName = 'my-test-bucket'
const bucket = storage.bucket(bucketName)

// Sending the upload request
bucket.upload(
  `./image_to_upload.jpeg`,
  {
    destination: `someFolderInBucket/image_to_upload.jpeg`,
  },
  function (err, file) {
    if (err) {
      console.error(`Error uploading image image_to_upload.jpeg: ${err}`)
    } else {
      console.log(`Image image_to_upload.jpeg uploaded to ${bucketName}.`)

        // Making file public to the internet
        file.makePublic(async function (err) {
        if (err) {
          console.error(`Error making file public: ${err}`)
        } else {
          console.log(`File ${file.name} is now public.`)
          const publicUrl = file.publicUrl()
          console.log(`Public URL for ${file.name}: ${publicUrl}`)
        }
       })

    }
  }
)

Happy to help if you stuck anywhere.

Refresh token problem in react-google-login (Solved 🧯)

Kamal Hossain — Thu, 11 Nov 2021 06:03:04 +0000

I was trying to retrieve a refresh_token in react-google-login. So, the process is very straight forward but the are some little things, for this many peoples failed to do this.

In this issue most of them is explained.

Inside the GoogleLogin component, we need to pass
accessType and responseType

<GoogleLogin
   clientId={process.env.REACT_APP_CLIENT_ID_2}
   accessType="offline"
   responseType="code"
...
/>

With what we will the a response like that:

{
    "code": "4/0AX4XfWhkrHVIhev2zNJRcL_uV9YfrqYCvmNkhTk3i3AeT8SzcZcwBntC9FUcx7RbPZEwEJ"
}

With this above code we can retrieve the refresh_token.

But 🚩

If the user is already authenticated this app from google console before then with this code it is not possible to retrieve the refresh_token. This is the reason for many developers they failed to retrieve refresh_token from a valid code.

To check wheather your app is already authenticated or not with the gmail your are testing with, then go to this link and if you found the app the just revoke the permissions for the app.
Learn more

Now send the code to your backend to retrive the refresh_token. Sample

Don't try to generate it on client side, because for that you have to make your client secrect public.

Note 🚩

If you somehow failed to retrive the refresh_token with the code then generate new code and send it to backend for another try. If one request is failed with the code somehow, then this code will not work again.

Finally from now with the refesh_token you can generate as much as access_token you want. Because access_token only available for one hour.

Install Certbot in Ubuntu 20.4 @2021

Kamal Hossain — Mon, 08 Nov 2021 12:18:48 +0000

sudo apt-get update
sudo apt-get install certbot
sudo apt-get install python3-certbot-nginx

If you get stuck in apt update any how then just run

sudo apt-add-repository -r ppa:certbot/certbot
sudo apt update
sudo apt-get update

ref

Enable ssl

If you have pointed the domain to ec2 IP then enable the ssl by:

certbot --expand --nginx --no-redirect --non-interactive --agree-tos --register-unsafely-without-email -d YOUR-DOMAIN.com

(Subdomain is also allowed)

Mobile POS print from expo app

Kamal Hossain — Mon, 16 Aug 2021 00:00:00 +0000

Expo doesn't have any API for managing bluetooth of any device. There are some react native packages available to manage the bluetooth like this one. But this packages are not easy to use in expo currently. Many peoples are requesting this bluetooth feature for long time. Al though in expo SDK 42 it is possible to use react-native-ble-plx package on a custom development clients, which is mentioned here.

Even if we manage to use this package, it will be still hard to print from a Mobile POS like M2 of imin. Becasue they didn't provide any sdk to for it to use on any Javascript project easily. So isn't here are any way available to print from a expo app to this printer?

Hopefully, yes there is.

There is apps available in playstore which is developed by rawbt. We need to use two extra apps from rawbt. One will connect the printer via bluetooth and another will do the auto print job. By auto print I mean whenever we will download any file to a specific folder via our expo app. The print will start automatically after successfull print.

So the basic process is something like this.

Expo app downlaods a file to a folder => Whenever any new files comes to this specific folder => AutoPrint for RawBT will send request to => RawBT print service => Which will print the file

Download and configure RawBT
Download and configure AutoPrint for RawBT
Create expo app

Download and configure RawBT

We need to download and install the

RawBT print service from playstore if not installed yet. After installing the app go to Settings. Set the Conenction method Bluetooth. Set Connection parameters by searching the Bluetooth device. Set the printer driver ESC/POS general. Click a Test to see if the print is working.

Download and configure AutoPrint for RawBT

This app is a kind of helper app of RawBT print service. What actually it does is, if we enable and select a folder in device folder this will keep watching on that folder for any file changes. If any new file appears on that particular folder then it will immediately send print request to RawBT print service to print the paper.
So after downloading the AutoPrint for RawBt from playstore we need to accept all the permission it requied. Then we will ADD a folder path where we will keep watching for any new files for print. In our case we will add this path /storage/emulated/0/Pictures/auto print. Here we are selecting this because expo Media library will save the file most probably in Pictures folder of the device storage (I am sure if there is any other way to change the folder). We will create another folder in that Picture folder called auto print. This is the sample configuration we will use which is given below:

You can change the auto print folder name to some other unique name if you want.

Create expo app

Now we do not have to touch the two previous apps anymore if everything working fine. Now just we have to download the pdf file in the specific directory. RawBT will automatically print each new pdf. So now I am going create a new expo project by expo init command. This is the App.js file code:



import * as React from 'react';
import { View, StyleSheet, Button } from 'react-native';
import Constants from 'expo-constants';
import * as FileSystem from 'expo-file-system';
import * as MediaLibrary from 'expo-media-library';

export default function App() {

    // Download the pdf file to expo temporary storage
    async function download2() {
    const fileUrl = 'https://YOUR_PDF_LINK.com';
    const fileName = `${Date.now()}.pdf`;

    FileSystem.downloadAsync(fileUrl, FileSystem.documentDirectory + fileName)
      .then(({ uri }) => {
        console.log('Finished downloading to ', uri);
        saveFile(uri);
      })
      .catch((error) => {
        console.error(error);
      });
  }

  async function saveFile(filePath) {
    const albumName = 'auto print';
    const permission = await MediaLibrary.requestPermissionsAsync();

    let asset = null;
    if (permission.granted) {
      try {
        asset = await MediaLibrary.createAssetAsync(filePath);
      } catch (e) {
        console.error('MediaLibrary.createAssetAsync failed', e);
      }

      if (asset) {
        try {
          let album = await MediaLibrary.getAlbumAsync(albumName);
          if (album) {
            await MediaLibrary.addAssetsToAlbumAsync([asset], album, false);
          } else {
            album = await MediaLibrary.createAlbumAsync(
              albumName,
              asset,
              false
            );
          }
          const assetResult = await MediaLibrary.getAssetsAsync({
            first: 1,
            album,
            sortBy: MediaLibrary.SortBy.creationTime,
          });
          asset = await assetResult.assets[0];
        } catch (e) {
          console.error(' failed', e);
        }
      } else {
        console.error('unable to use MediaLibrary, can not create assets');
      }
    }
  }

  return (
    <View style={styles.container}>
      <Button title="Download invoice & print" onPress={download2}></Button>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    justifyContent: 'center',
    paddingTop: Constants.statusBarHeight,
    backgroundColor: '#ecf0f1',
    padding: 8,
  },
  paragraph: {
    margin: 24,
    fontSize: 18,
    fontWeight: 'bold',
    textAlign: 'center',
  },
});

By using this piece of code now we can easily download the file from the web and save it to specific directory.

In the download2() function we are using FileSystem.downloadAsync() to download the pdf file from the web. This will download the file to the host.exp.exponent directory, which cannot be accessable with other apps easily. So that after downloading the file we have to move to the downloaded file to a specific directory.

For that we are using the saveFile(). Here we are mainly using the MediaLibrary.createAssetAsync() to store the file in a specific directory. In our case we are saving it to the /storage/emulated/0/Pictures/auto print path. This is the folder where the AutoPrint for RawBT will keep looking for any new file to print automatically by RawBT.

This is the package.json file of mine, install this packages correctly to avoid any error.



{
  "main": "node_modules/expo/AppEntry.js",
  "scripts": {
    "start": "expo start",
    "android": "expo start --android",
    "ios": "expo start --ios",
    "web": "expo start --web",
    "eject": "expo eject"
  },
  "dependencies": {
    "expo": "~42.0.1",
    "expo-constants": "~11.0.1",
    "expo-file-system": "~11.1.3",
    "expo-media-library": "~12.1.2",
    "expo-print": "~10.2.1",
    "expo-sharing": "~9.2.1",
    "expo-status-bar": "~1.0.4",
    "react": "16.13.1",
    "react-dom": "16.13.1",
    "react-native": "https://github.com/expo/react-native/archive/sdk-42.0.0.tar.gz",
    "react-native-web": "~0.13.12"
  },
  "devDependencies": {
    "@babel/core": "^7.9.0"
  },
  "private": true
}

Now if we press the Download invoice & print button. For the first time it will ask one or multiple permissions. If the link is correct then the download will be complete shortly. And the print will work fine.

MongoDB Cheat Sheet

Kamal Hossain — Sun, 27 Dec 2020 02:37:13 +0000

MongoDB Cheat Sheet

Open MongoDB in cmd (For windows)

mongo

If mongo is not recognized as an internal or external command, operable program or batch file, then do the followings from this blog

Show All Databases

show dbs

Show Current Database

db

Create Or Switch Database

use acme

Drop Database🔴

> use mydb
> db.dropDatabase()

Create Collection

db.createCollection('posts')

Show Collections

show collections

Drop Collection 🔴

> use mydb
> db.usersCollection.drop()

Insert Row

db.posts.insert({
  title: 'Post One',
  body: 'Body of post one',
  category: 'News',
  tags: ['news', 'events'],
  user: {
    name: 'John Doe',
    status: 'author'
  },
  date: Date()
})

Insert Multiple Rows

db.posts.insertMany([
  {
    title: 'Post Two',
    body: 'Body of post two',
    category: 'Technology',
    date: Date()
  },
  {
    title: 'Post Three',
    body: 'Body of post three',
    category: 'News',
    date: Date()
  },
  {
    title: 'Post Four',
    body: 'Body of post three',
    category: 'Entertainment',
    date: Date()
  }
])

Get All Rows from a collection

db.posts.find()

Get All Rows from a collection Formatted

db.posts.find().pretty()

Find Rows

db.posts.find({ category: 'News' })

Sort Rows in a collection

# asc
db.posts.find().sort({ title: 1 }).pretty()
# desc
db.posts.find().sort({ title: -1 }).pretty()

Count Rows in a collection

db.posts.find().count()
db.posts.find({ category: 'news' }).count()

Limit Rows in a collection

db.posts.find().limit(2).pretty()

Chaining example

db.posts.find().limit(2).sort({ title: 1 }).pretty()

Foreach

db.posts.find().forEach(function(doc) {
  print("Blog Post: " + doc.title)
})

Find One Row

db.posts.findOne({ category: 'News' })

Find Specific Fields

db.posts.find({ title: 'Post One' }, {
  title: 1,
  author: 1
})

Update Row

db.posts.update({ title: 'Post Two' },
{
  title: 'Post Two',
  body: 'New body for post 2',
  date: Date()
},
{
  upsert: true
})

Update Specific Field

db.posts.update({ title: 'Post Two' },
{
  $set: {
    body: 'Body for post 2',
    category: 'Technology'
  }
})

Increment Field (\$inc)

db.posts.update({ title: 'Post Two' },
{
  $inc: {
    likes: 5
  }
})

Rename Field

db.posts.update({ title: 'Post Two' },
{
  $rename: {
    likes: 'views'
  }
})

Delete specific document from a collection

db.posts.remove({ title: 'Post Four' })

Sub-Documents

db.posts.update({ title: 'Post One' },
{
  $set: {
    comments: [
      {
        body: 'Comment One',
        user: 'Mary Williams',
        date: Date()
      },
      {
        body: 'Comment Two',
        user: 'Harry White',
        date: Date()
      }
    ]
  }
})

Find By Element in Array (\$elemMatch)

db.posts.find({
  comments: {
     $elemMatch: {
       user: 'Mary Williams'
       }
    }
  }
)

Add Index

db.posts.createIndex({ title: 'text' })

Text Search

db.posts.find({
  $text: {
    $search: "\"Post O\""
    }
})

Greater & Less Than

db.posts.find({ views: { $gt: 2 } })
db.posts.find({ views: { $gte: 7 } })
db.posts.find({ views: { $lt: 7 } })
db.posts.find({ views: { $lte: 7 } })

Get database size and index size in MB (Megabytes)

db.stats(1024*1024);

Export a collection in json format

mongoexport --db newdb -c usersCollection --out testBakcup.json

Import a collection from json file

mongoimport --db newdb -c usersCollectiontwo --file test.json

Export entire database including all collections (Local)

mongodump --db newdb

Export entire database including all collections (Cluster)

mongodump --uri="mongodb+srv://<Name>:<Password>@cluster0-pk6qf.mongodb.net/<DB_YOU_WANT_TO_EXPORT>?retryWrites=true&w=majority"

ref1

use --collection <collection_name> for only specific collection

Import entire database including all collections

mongorestore --db deleteme dump/newdb

newdb is the folder name that holds the database (deleteme) related files (.bson) etc.
To override same document with _id use this method

Connect to cloud mongodb cluster from mongo shell

mongo "mongodb+srv://cluster0.g1re8.mongodb.net/<dbname>" --username <check_mongo_cloud_connect_shell>

Useful resources

MongoDB Package Components (e.g. mongoimport, mongoexport, mongodump ) > https://docs.mongodb.com/manual/reference/program

Posted in • dev.to • gist.github.com

Send data between tabs in react app

Kamal Hossain — Mon, 21 Dec 2020 03:49:56 +0000

Yes! You can send data between different tabs or windows of your react app. Confusing hah?

Ok, let's assume you have made a WebRTC app via jitsi. The main goal of your app is video conferencing. Now you don't want any of your users to open two sessions at the same time in their browser for the calling. How can you do that? Here you can use the Broadcast Channel API.

One other important thing is that this will only work only if your app/site is of the same origin.

Let's see that in action by an example.

Create a react app



create-react-app browser-tab-broadcast-react

Start the app



npm start

Modify the App.js file

Remove everything from your App.js file and paste these.



import React from 'react'
import './App.css'

function App() {
  return (
    <div class="App">
      <h1> Something happening in the dom </h1>
    </div>
  )
}
export default App

All that we are doing is just rendering a div with an h1 tag.



// ...
<h1> Something happening in the dom </h1>

<p>Only one tab could open at a time</p>
<button type="button" onClick={refreshPage}>
    <span>Reload</span>
</button>

Now this is the part where we want to show the user, if the page is already opnned in another tab or window of the same browser.

To do that we need a state to handle that. So that we can conditionally render the elements to the user.



// ...
function App() {
  const [isOpnnedAlready, setIsOpnnedAlready] = React.useState(false)

  return (
    <div class="App">
// ...

By default, the state is false and that means this same page is not opened in another tab or window of the browser so we can put the condition in our JSX.



// ...

function refreshPage() {
    window.location.reload()
}

 return (
    <div class="App">
      {!isOpnnedAlready ? (
        <h1>Something happening in the dom</h1>
      ) : (
        <>
          <p>Only one tab could open at a time</p>
          <button type="button" onClick={refreshPage}>
            <span>Reload</span>{' '}
          </button>
        </>
      )}
    </div>
// ...

So now, we need to listen for an event in our component. To see if another page is opened in another tab.

Create a BroadcastChannel instance

For that, we need to create a channel in our app. Which will be available to all other tabs with the same protocol, origin, and port.



// ...
const channel = React.useMemo(() => new BroadcastChannel('couldBeAnything'), [])

Here we have created a new BroadcastChannel instance and stored that in the channel variable. With this couldBeAnything name we are subscribing to a particular channel. By subscribing we are able to post and receive messages from it.

In extra we are using useMemo hook to keep the instance value in memory. So that we don't have to re-run the code on every re-render.

Sending and receiving data via BroadcastChannel




const channel = React.useMemo(() => new BroadcastChannel('couldBeAnything'), [])

React.useEffect(() => {
    channel.postMessage({
      isOpnnedAlready: true,
    })
    channel.addEventListener('message', (e) => {
      setIsOpnnedAlready(e.data.isOpnnedAlready)
    })
    return channel.close
  }, [])

Here we are using useEffect hook. Here we are posting and receiving to the channel after every first render of the component.

By calling postMessage() with our channel instance we are sending an object to the channel (here we can also send string, array etc). Right after that, we are calling addEventListener() to listen to the message event from the channel. And finally, we are closing the channel on component unmount.

When all these things are done open up two tabs one after another, you will notice the changes on the page. The first tab will change its DOM whenever other tabs open in the same browser.

Resource(s): 1

How to run docker containers in AWS EC2

Kamal Hossain — Mon, 21 Dec 2020 02:43:35 +0000

Dockerized `Click Counter app` app to deploy at AWS EC2 ubuntu instance

Get the source code here

Stack

React
Nodejs
MongoDB

Docker notes

If docker is installed you can build and run images locally.

docker-compose up --build

Set the REACT_APP_API_URL variable in .env, based on your need.

Create a new EC2 instance (Ubuntu 18.04)

Go to AWS EC2
Launch instance
Chose Ubuntu 18.04 64 bit ( you can choose as your need )
Select Free tier eligible ( recommended for testing )
In Configure Security Group add All traffic rule and allow from anywhere ( for testing only )
Leave all settings as default
Launch
Create a key pair, and download it.

Connect EC2 instance from Windows machine

Install PuTTY
Open PuTTYgen, Load the downloaded *.pem file
Save private key ( this will save a *.ppk file )
Go to the AWS EC2
Select the instance
Click on connect
Copy the name from ssh ( the part should start with ubuntu and end with .com )
Open PuTTY ( NOT PuTTYgen! )
Go Session, paste the copied name in Host Name (or IP address) leave port as 22
Go Connection > SSH > Auth and browse the saved private key ( *.ppk ) in Private key file for authentication
Click Open
Click Yes ( if anything prompt up )

Configure docker in EC2 (Ubuntu 18.04)

Run the following commands to install docker and docker compose

sudo apt-get update

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

apt-cache madison docker-ce

sudo apt install docker.io

sudo apt install docker-compose

Run this repo in AWS EC2 by following commands

git clone https://github.com/kamal-hossain/MERN-docker-ec2-deploy-practice

cd MERN-docker-ec2-deploy-practice

sudo docker-compose up --build

You can also run it background by adding -d flag

sudo docker-compose up --build -d

Confirm it by

sudo docker ps

Stop the containers

sudo docker-compose down

To access the app from browser do the followings:

Select the instance in AWS EC2
Copy the Public IPv4 address from Details
Open it on browser

You can't access the IP if All traffic is not enabled in Security group from anywhere. (Or simillar settings)

Resource(s): 1

MongoDB without Mongoose

Kamal Hossain — Sat, 12 Sep 2020 00:00:00 +0000

Suppose you just want to store a bunch of email addresses as your subscribers in your MongoDB database. Nothing else.

There is no need to implement an entire driver to handle this. Yes, of course, drivers provide many advantages to make our lives easy. But to keep things simple and lightweight we can use only MongoDB for CRUD operation without a mongoose.

What about validation? Don't worry, packages like sanitize-html can be used in MongoDB to validate data before storing to a database.

Get the Source code if you don't want to follow along.

Let's get started.

Create a nodejs Server

Run the followings in your terminal:

mkdir mongodb-without-mongoose && cd mongodb-without-mongoose

These will create a folder called mongodb-without-mongoose and navigate to the folder.

npm init -y && npm install express mongodb --save

To initialize package.json file in our source folder to install dependencies, we run the first part of the command. And the second part is for installing the express.js and MongoDB as our dependencies.

Now let's create an app.js file in your project folder. And fill the file with the followings to create a simple server:

const express = require('express')
const mongodb = require('mongodb')

const app = express()
let db

const PORT = 5000

app.get('/testRoute', (req, res) => res.end('Hello from Server!'))

app.listen(PORT, () => {
  console.log(`Node.js App running on port ${PORT}...`)
})

To start the server run:

node app.js

Here we have just created a simple server at port 5000. On localhost:5000/testRoute it will send the response, Hello from Server!.

Connect to mongodb from server

In this example, we are going to connect the local MongoDB of windows. We are not going to any cloud services of MongoDB. So let's add some lines in our app.js to establish the connection.

// ...
const app = express()
let db

let connectionString = `mongodb://localhost:27017/crud`

This is the default connection string of windows to connect to the local database. Now let us connect to MongoDB by using this string.

// ...
const app = express()
let db

let connectionString = `mongodb://localhost:27017/crud`

mongodb.connect(
  connectionString,
  { useNewUrlParser: true, useUnifiedTopology: true },
  function (err, client) {
    db = client.db()
    app.listen(5000)
  }
)

Here above the MongoDB.connect() taking the connection string as the first argument, after that a second argument is an object for not getting the deprecation warnings from MongoDB.
And finally, the last argument is the callback function which can be used after MongoDB.connect() function trying to connect to MongoDB.

In our case, we are storing the database in a variable for further use and also started the app for listening in port 5000.

Now that we have our database connected, let's create some endpoints to make the app useful.

// ...

mongodb.connect(
  connectionString,
  { useNewUrlParser: true, useUnifiedTopology: true },
  function (err, client) {
    db = client.db()
    app.listen(5000)
  }
)

app.post('/create-data', function (req, res) {
  // Sending request to create a data
  db.collection('data').insertOne({ text: req.body.text }, function (
    err,
    info
  ) {
    res.json(info.ops[0])
  })
})

Here we have set up a post request to our app to /create-data. Here we are using the db.collection() to specify the collection name in the crud database that we have connected already.

In this method, we are chaining the insertOne() to create a document in the collection. We are passing two arguments in this function.

The first one is the text property with the string from the req.body object. And the second one is the callback function to do some stuff when the insetOne() method tries to insert the document to the collection.

In our callback function, we have two arguments, err and info. We are going to retrieve our collection of documents from info.ops[0] and then simply send back the response to the requested user.

Let's try a post request on localhost:5000/create-data with the following json data from postman.

{
  "text": "Kamal Hossain"
}

What have got?

Well, I have got a 500 Internal Server Error with a bunch of HTML values in response. So let's check the console where is server is running.

TypeError: Cannot read property 'text' of undefined

I have got some lines in our console and the first line is telling this above. So why this is happening?

Well, we have sent JSON data to our nodejs app, where the server was created with the help of express.js. So by default, it can't read any JSON Object from incoming requests. So we have to add a line for that.

// ...

mongodb.connect(
  connectionString,
  { useNewUrlParser: true, useUnifiedTopology: true },
  function (err, client) {
    db = client.db()
    app.listen(5000)
  }
)

app.use(express.json())

This will take care of the current problem. Let's save & restart our server. Then send the request again.

{
  "text": "Kamal Hossain",
  "_id": "5f5c27353c14cc09309d4440"
}

This is the response I have got from our server. Which was sent from the insertOne() methods callback function. The _id is the unique id to identify our this particular document across the database.

Now that we have created our first document lets create a route to get all the documents from our collection via get request.

// ...

app.post('/create-data', function (req, res) {
  // Sending request to create a data
  db.collection('data').insertOne({ text: req.body.text }, function (
    err,
    info
  ) {
    res.json(info.ops[0])
  })
})

app.get('/', function (req, res) {
  // getting all the data
  db.collection('data')
    .find()
    .toArray(function (err, items) {
      res.send(items)
    })
})

Here we are setting the route as / which will be the root route of any domain. In our case the localhost:5000. Here we are using the find() method to get all the documents from the collection. Lastly, we are chaining the toArray() method to put all the documents in an array, and in the callback of toArray() we are sending the response back.

Lets do a get request in postman to localhost:5000/

[
    {
        "_id": "5f5c27353c14cc09309d4440",
        "text": "Kamal Hossain"
    }
]

Now let's add two more routes in our app to update and delete a document.

// ...

app.get('/', function (req, res) {
  // getting all the data
  db.collection('data')
    .find()
    .toArray(function (err, items) {
      res.send(items)
    })
})

app.put('/update-data', function (req, res) {
  // updating a data by it's ID and new value
  db.collection('data').findOneAndUpdate(
    { _id: new mongodb.ObjectId(req.body.id) },
    { $set: { text: req.body.text } },
    function () {
      res.send('Success updated!')
    }
  )
})

In our /update-data route are are mainly updating a document by findOneAndUpdate(). In this method we are passing the object Id that needs to be updated, then we are passing the updated text. And lastly, we are just sending the success response.

Let's now delete a document.

// ...

app.put('/update-data', function (req, res) {
  // updating a data by it's ID and new value
  db.collection('data').findOneAndUpdate(
    { _id: new mongodb.ObjectId(req.body.id) },
    { $set: { text: req.body.text } },
    function () {
      res.send('Success updated!')
    }
  )
})

app.delete('/delete-data', function (req, res) {
  // deleting a data by it's ID
  db.collection('data').deleteOne(
    { _id: new mongodb.ObjectId(req.body.id) },
    function () {
      res.send('Successfully deleted!')
    }
  )
})

In our delete-data route we are using delete() method to delete the document from collection. And lastly sending a response to the client in the callback function.

Originally published at https://kamalhossain.me/blog/mongodb-without-mongoose

MongoDB without Mongoose

Kamal Hossain — Sat, 12 Sep 2020 00:00:00 +0000

Lets suppose you just want to store a bunch of email addresses as your subscribers in your MongoDB database. Nothing else.

What about validation? Don't worry, packages like sanitize-html can be used in MongoDB to validate data before storing to a database.

Get the Source code if you don't want to follow along.

Let's get started.

Create a nodejs Server

Run the followings in your terminal:

mkdir mongodb-without-mongoose && cd mongodb-without-mongoose

These will create a folder called mongodb-without-mongoose and navigate to the folder.

npm init -y && npm install express mongodb --save

Now let's create an app.js file in your project folder. And fill the file with the followings to create a simple server:

const express = require('express')
const mongodb = require('mongodb')

const app = express()
let db

const PORT = 5000

app.get('/testRoute', (req, res) => res.end('Hello from Server!'))

app.listen(PORT, () => {
  console.log(`Node.js App running on port ${PORT}...`)
})

To start the server run:

node app.js

Here we have just created a simple server at port 5000. On localhost:5000/testRoute it will send the response, Hello from Server!.

Connect to mongodb from server

In this example, we are going to connect the local MongoDB of windows. We are not going to any cloud services of MongoDB. So let's add some lines in our app.js to establish the connection.

// ...
const app = express()
let db

let connectionString = `mongodb://localhost:27017/crud`

This is the default connection string of windows to connect to the local database. Now let us connect to MongoDB by using this string.

// ...
const app = express()
let db

let connectionString = `mongodb://localhost:27017/crud`

mongodb.connect(
  connectionString,
  { useNewUrlParser: true, useUnifiedTopology: true },
  function (err, client) {
    db = client.db()
    app.listen(5000)
  }
)

In our case, we are storing the database in a variable for further use and also started the app for listening in port 5000.

Now that we have our database connected, let's create some endpoints to make the app useful.

// ...

mongodb.connect(
  connectionString,
  { useNewUrlParser: true, useUnifiedTopology: true },
  function (err, client) {
    db = client.db()
    app.listen(5000)
  }
)

app.post('/create-data', function (req, res) {
  // Sending request to create a data
  db.collection('data').insertOne({ text: req.body.text }, function (
    err,
    info
  ) {
    res.json(info.ops[0])
  })
})

Here we have set up a post request to our app to /create-data. Here we are using the db.collection() to specify the collection name in the crud database that we have connected already.

In this method, we are chaining the insertOne() to create a document in the collection. We are passing two arguments in this function.

Let's try a post request on localhost:5000/create-data with the following json data from postman.

{
  "text": "Kamal Hossain"
}

What have got?

Well, I have got a 500 Internal Server Error with a bunch of HTML values in response. So let's check the console where is server is running.

TypeError: Cannot read property 'text' of undefined

I have got some lines in our console and the first line is telling this above. So why this is happening?

// ...

mongodb.connect(
  connectionString,
  { useNewUrlParser: true, useUnifiedTopology: true },
  function (err, client) {
    db = client.db()
    app.listen(5000)
  }
)

app.use(express.json())

This will take care of the current problem. Let's save & restart our server. Then send the request again.

{
  "text": "Kamal Hossain",
  "_id": "5f5c27353c14cc09309d4440"
}

Now that we have created our first document lets create a route to get all the documents from our collection via get request.

// ...

app.post('/create-data', function (req, res) {
  // Sending request to create a data
  db.collection('data').insertOne({ text: req.body.text }, function (
    err,
    info
  ) {
    res.json(info.ops[0])
  })
})

app.get('/', function (req, res) {
  // getting all the data
  db.collection('data')
    .find()
    .toArray(function (err, items) {
      res.send(items)
    })
})

Lets do a get request in postman to localhost:5000/

[
    {
        "_id": "5f5c27353c14cc09309d4440",
        "text": "Kamal Hossain"
    }
]

Now let's add two more routes in our app to update and delete a document.

// ...

app.get('/', function (req, res) {
  // getting all the data
  db.collection('data')
    .find()
    .toArray(function (err, items) {
      res.send(items)
    })
})

app.put('/update-data', function (req, res) {
  // updating a data by it's ID and new value
  db.collection('data').findOneAndUpdate(
    { _id: new mongodb.ObjectId(req.body.id) },
    { $set: { text: req.body.text } },
    function () {
      res.send('Success updated!')
    }
  )
})

Let's now delete a document.

// ...

app.put('/update-data', function (req, res) {
  // updating a data by it's ID and new value
  db.collection('data').findOneAndUpdate(
    { _id: new mongodb.ObjectId(req.body.id) },
    { $set: { text: req.body.text } },
    function () {
      res.send('Success updated!')
    }
  )
})

app.delete('/delete-data', function (req, res) {
  // deleting a data by it's ID
  db.collection('data').deleteOne(
    { _id: new mongodb.ObjectId(req.body.id) },
    function () {
      res.send('Successfully deleted!')
    }
  )
})

In our delete-data route we are using delete() method to delete the document from collection. And lastly sending a response to the client in the callback function.

Originally published at https://kamalhossain.me/blog/mongodb-without-mongoose

Download and Delete via Google Drive API

Kamal Hossain — Sat, 29 Aug 2020 00:00:00 +0000

Last time I spent my times in followings:

Create a node.js server by using express.js
Get authentication for Google Drive
Upload an image file to Google Drive from node.js server

Today I am going to show how to download an image file from google drive, by using google drive API in node.js server and also how to delete an image file.

Again if you don't want to follow the tutorial download the Source code

Download file via google drive API in node.js

We are not going to create a new node.js server or also not going to get new authentication from google. We are going to use the server that we have used last time. So, in the server.js file let's add another function to receive a GET request to download an image from google drive to the node.js server.

// ...

// Route for downloading an image/file
app.get('/downloadAFile', (req, res) => {
  var dir = `./downloads`; // directory from where node.js will look for downloaded file from google drive

  var fileId = '13_Iq3ImCLQqBStDQ9ottLIJwxwlXkQpa'; // Desired file id to download from  google drive

  var dest = fs.createWriteStream('./downloads/kamal-hossain.jpg'); // file path where google drive function will save the file

  const drive = google.drive({ version: 'v3', auth }); // Authenticating drive API

  let progress = 0; // This will contain the download progress amount

  // Uploading Single image to drive
  drive.files
    .get({ fileId, alt: 'media' }, { responseType: 'stream' })
    .then((driveResponse) => {
      driveResponse.data
        .on('end', () => {
          console.log('\nDone downloading file.');
          const file = `${dir}/kamal-hossain.jpg`; // file path from where node.js will send file to the requested user
          res.download(file); // Set disposition and send it.
        })
        .on('error', (err) => {
          console.error('Error downloading file.');
        })
        .on('data', (d) => {
          progress += d.length;
          if (process.stdout.isTTY) {
            process.stdout.clearLine();
            process.stdout.cursorTo(0);
            process.stdout.write(`Downloaded ${progress} bytes`);
          }
        })
        .pipe(dest);
    })
    .catch((err) => console.log(err));
});

Let's go through the code as fast as possible.

In our express app, we are defining a get request as downloadAFile. This function will set the directory to download the file from google drive, and another directory to serve the file to the requested user. In our case both directory is the same.

We are going to download the file in our server via drive.files.get(). In there we are going to pass the desired file id to download. Rember the id that we got when last time we have uploaded a file to google drive. Also we are going to set the response time as stream, so probably we are going to get the file chunk by chunk.

So, in the above method, we are going to chain the then() in where we going to handle the response data from google.

Again in the response, we are chaining some .on() to handle the response in different stages.

First, we are adding the end stage, to tell the server what to do when the file is downloaded to our server from google drive. In our case, we are going to send the file to the requesting user from our server.

After that we are adding the error, to check if anything suspicious happens during download from google drive.

In our last .on() we are adding data to show the downloaded amount of file size in our console. We are showing the amount in bytes.

Lastly by chaining .pipe() where we are passing the destination of the folder where the google drive function should save the requested file.

Delete file via google drive api in node.js

The process is very simple.

// ...

// Route for downloading an image/file
app.delete('/deleteAFile', (req, res) => {
  var fileId = '1vuZs3N8qnevNEETCKnZQ5js0HOCpGTxs'; // Desired file id to download from  google drive

  const drive = google.drive({ version: 'v3', auth }); // Authenticating drive API

  // Deleting the image from Drive
  drive.files
    .delete({
      fileId: fileId,
    })
    .then(
      async function (response) {
        res.status(204).json({ status: 'success' });
      },
      function (err) {
        return res
          .status(400)
          .json({ errors: [{ msg: 'Deletion Failed for some reason' }] });
      }
    );
});

Here we are using drive.files.delete() function to delete the file. We are just passing the unique file id as an argument.

Then we are handling the response via .then() and passing them to the user.

For testing both of the requests we are using postman. If you using the same, don't forget to switch between GET and DELETE for these two requests.

DEV Community: Kamal Hossain

How to run already downloaded model in ollama

Intro

Downloading and installing the necessary files

Running the model into ollama

Upload file to Google cloud storage without middleman server

Upload file to Google cloud storage from Node.js server

Step one: Getting Keys for authentication

Step two: Giving service account the write permission to the bucket

Step three: Upload an image to Google cloud storage

Extra step: Making the content public after uploading

Refresh token problem in react-google-login (Solved 🧯)

But 🚩

Note 🚩

Install Certbot in Ubuntu 20.4 @2021

Enable ssl

Mobile POS print from expo app

Table of Contents

Download and configure RawBT

Download and configure AutoPrint for RawBT

Create expo app

MongoDB Cheat Sheet

MongoDB Cheat Sheet

Open MongoDB in cmd (For windows)

Show All Databases

Show Current Database

Create Or Switch Database

Drop Database🔴

Create Collection

Show Collections

Drop Collection 🔴

Insert Row

Insert Multiple Rows

Get All Rows from a collection

Get All Rows from a collection Formatted

Find Rows

Sort Rows in a collection

Count Rows in a collection

Limit Rows in a collection

Chaining example

Foreach

Find One Row

Find Specific Fields

Update Row

Update Specific Field

Increment Field (\$inc)

Rename Field

Delete specific document from a collection

Sub-Documents

Find By Element in Array (\$elemMatch)

Add Index

Text Search

Greater & Less Than

Get database size and index size in MB (Megabytes)

Export a collection in json format

Import a collection from json file

Export entire database including all collections (Local)

Export entire database including all collections (Cluster)

Import entire database including all collections

Connect to cloud mongodb cluster from mongo shell

Useful resources

Send data between tabs in react app

Create a react app

Modify the App.js file

Create a BroadcastChannel instance

Sending and receiving data via BroadcastChannel

How to run docker containers in AWS EC2

Dockerized Click Counter app app to deploy at AWS EC2 ubuntu instance

Stack

Docker notes

Create a new EC2 instance (Ubuntu 18.04)

Connect EC2 instance from Windows machine

Configure docker in EC2 (Ubuntu 18.04)

MongoDB without Mongoose

Create a nodejs Server

Connect to mongodb from server

MongoDB without Mongoose

Create a nodejs Server

Connect to mongodb from server

Download and Delete via Google Drive API

Dockerized `Click Counter app` app to deploy at AWS EC2 ubuntu instance