DEV Community

Saoni Deb
Saoni Deb

Posted on • Edited on

Locality Sensing Hashing

Locality Sensitive Hashing (LSH) is a computationally efficient approach for finding nearest neighbors in large datasets.
The main idea in LSH is to avoid having to compare every pair of data samples in a large dataset in order to find the nearest similar neighbors for the different data samples.
With LSH, one can expect a data sample and its closest similar neighbors to be hashed into the same bucket with a high probability.
By treating the data samples placed in the same bucket as candidates for similarity checking, we significantly reduce the computational burden associated with similarity detection in large datasets.

While LSH algorithms have traditionally been used for finding nearest neighbors, this module goes a step further and explores using LSH for clustering the data.
Strictly speaking, this violates the basic mandate of LSH, which is to return just the nearest neighbors.
(A data sample X being Y’s nearest neighbor and Y being Z’s nearest neighbor, in the sense neighbors are commonly defined with the Cosine metric in LSH, does not imply that X and Z will always be sufficiently close to be considered each other’s nearest neighbors.)
Nonetheless, if you believe that your datafile consists of non-overlapping data clusters, this module may do a decent job of finding those clusters.

Reference:
https://pypi.org/project/LocalitySensitiveHashing/#:~:text=Locality%20Sensitive%20Hashing%20(LSH)%20is,for%20the%20different%20data%20samples.

Hostinger image

Get n8n VPS hosting 3x cheaper than a cloud solution

Get fast, easy, secure n8n VPS hosting from $4.99/mo at Hostinger. Automate any workflow using a pre-installed n8n application and no-code customization.

Start now

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay