In Order to understand Data version control, let's first get a general idea of what version control is. Imagine a company that has employees working remotely all over the continent. These employees will at some point require to work together in some project.The company faces a challenge to collaborate ,for the many workers located in different parts of the continent but are working on the same project.
Another issue is the versions needed to complete a project, since a project is not completed in a single version,how will the employees update or see the updated versions(or where exactly has the changes been made) of the project.The version control system takes care of the collaboration between employees storing different versions.
Version control is the practice of tracking and managing changes to software code. Version control systems are software tools that help software teams manage changes to source code over time. Developers may review, compare, and undo changes made to a file over time using Version Control System, which keeps track of all file modifications.
*Examples of version control systems in the market;
*
- Github -it is the most commonly and widely used.
2.Gitlab
3.Perforce
4.Beanstalk
5.AWS code commit
6.Apacha Subversion
7.Mercuril e.t.c
Now that we have an idea of what version control is ,let's narrow down to Data Version Control.
**
What is Data Version Control?
**
Similar to how version control systems manage changes to code files, data version control is a system for managing changes to data files. Data scientists and machine learning engineers can work together on data projects, manage changes to data files, and replicate data-driven experiments using the data version tool.
**Advantages of Data Version Control
**
1.Data Version Control allows you to track changes to your data files over time, and keep a record of the exact data files used in each version of the project.
2.Data Version Control allows multiple data scientists and machine learning engineers to work on the same project, share data files, and collaborate on experiments. DVC also provides tools for resolving conflicts when multiple people make changes to the same data file.
3.Data Version Control provides a scalable way to manage large data sets, by allowing you to store data files in cloud storage systems. This makes it easier to work with large data sets without running into storage limitations on your local machine.
4.Data Version Control allows you to reuse data files across multiple versions of the project, which can save time and reduce the amount of data processing required.
Git and github is the most widely used data version control system ,which allows data scientists work on the same project and manage their changes through branches,commits,and merges.
*Reasons why github is widely used /commonly used over the other Version Control Systems
*
1.Github is open-source-it supports open source projects.
2.Github has a large community of developers who share their code and contribute to open-soucrce projects.
3.Github hosts yor code.
4.GitHub makes it easy to collaborate with others on projects. You can easily share your code with other developers, and they can make contributions or suggest changes using pull requests.
5.GitHub integrates with many other tools, such as CI/CD pipelines, code analysis tools, and project management tools.
In this article ,I will give an introduction on how to use github and git when working on a data science project.
First you must have downloaded and configured git (using git config) You must also have created a github account.
Steps to follow when pushing code to github
1.In github create new repository(click 'new' on the repositories page)and name it according to the project you are working on.
In creating a repository, you should add a small description of your project in the description box and a long/detailed description in the README file that should be attached to the repository.
A repository is either public or private. A public repository is accessible to anyone on the internet while a private repository is only accessible to you,people you explicitly share access with.
- You need to clone(using git clone and a link to the repository) your repository in your local machine.Open your git bash window and navigate to the directory where you want to store your directory. Use cd to change directory and ls to list all the items in the directory.
3.Add your code to the repository by creating new files or modifying existing ones in the local copy of the repository.
4.Add the files you want to push to the repository by running git add
5.Commit the changes using git commit -m 'commit message'
Replace 'Commit message' with a short message describing the changes you made.
6.Push the changes to github using _git push _command
Steps to updating your code in github
1.Make changes to your local code using your preffered editor eg.jupyter notebook
2.Add the changes running git add .(this is a period)
3.Commit the changes git commit -m 'commit message'
4.Push the changes to github using _git push _command
Confirm that the changes show on your github.
*Steps on how to pull code from github *
1.Open your git terminal and navigate to the directory where you want to clone the repository.
2.Clone the repository using git clone <repository-url
_
3.Once the repository is cloned,use the _git pull command to fetch the latest changes from the remote repository and merge them into your local copy.
After pulling the code and working on it ,push the changes with the steps described above.
Here is a link where you can get a git cheat sheet for easy navigation in git [https://education.github.com/git-cheat-sheet-education.pdf]
Conclusion
This article is biased towards git and github ,this is because they are the most commonly used systems.However one can use any of the systems mentioned in the article.I would encourage the readers to research more on git and github and the other data version control systems.
Top comments (0)