<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Jingles (Hong Jing)</title>
    <description>The latest articles on DEV Community by Jingles (Hong Jing) (@jinglescode).</description>
    <link>https://dev.to/jinglescode</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F285484%2Fd8dde138-89e4-448d-9770-faa3d776f50f.jpeg</url>
      <title>DEV Community: Jingles (Hong Jing)</title>
      <link>https://dev.to/jinglescode</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/jinglescode"/>
    <language>en</language>
    <item>
      <title>Setting up LaTeX on your Atom Editor</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Thu, 07 May 2020 13:09:15 +0000</pubDate>
      <link>https://dev.to/jinglescode/setting-up-latex-on-your-atom-editor-4gbe</link>
      <guid>https://dev.to/jinglescode/setting-up-latex-on-your-atom-editor-4gbe</guid>
      <description>&lt;p&gt;In this tutorial, I will guide you through setting up a LaTeX editor on &lt;a href="https://atom.io/" rel="noopener noreferrer"&gt;Atom&lt;/a&gt;, and here are some reasons and benefits for doing so:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;preview your work side by side as you save your work&lt;/li&gt;
&lt;li&gt;fast, generate previews in 3-seconds&lt;/li&gt;
&lt;li&gt;clicking on the preview will bring your cursor to the syntax’s location&lt;/li&gt;
&lt;li&gt;Dropbox and GitHub integration for backup and working with collaborators&lt;/li&gt;
&lt;li&gt;Free&lt;/li&gt;
&lt;/ul&gt;

&lt;h1&gt;
  
  
  What is LaTeX
&lt;/h1&gt;

&lt;p&gt;&lt;a href="https://www.latex-project.org/about/" rel="noopener noreferrer"&gt;LaTeX&lt;/a&gt; is a document preparation system that is widely used in academia for publishing scientific documents. Authors use markup syntax to define the structure of a document, to stylise text, to insert images, to add citations and cross-references.&lt;/p&gt;

&lt;p&gt;Because there are so many publishers, each having their design standards. For example, 18pt Times Roman for the title, 12pt Times Italic for the name, and so on. The purpose of LaTeX is to let authors focus on the writing and not wasting time following publishers’ document design guidelines. Authors simply download publishers’ LaTeX template and start writing.&lt;/p&gt;

&lt;p&gt;If you’re new to TeX or just want to use LaTeX for a one-time project, the quickest way to get started is using online services like &lt;a href="http://papeeria.com/" rel="noopener noreferrer"&gt;Papeeria&lt;/a&gt;, &lt;a href="https://www.overleaf.com/" rel="noopener noreferrer"&gt;Overleaf&lt;/a&gt;, and &lt;a href="https://latexbase.com/" rel="noopener noreferrer"&gt;LaTeX base&lt;/a&gt;. They offer the ability to edit, preview, and download your work in PDF format.&lt;/p&gt;

&lt;p&gt;Since there are online solutions, so why set up a LaTeX editor on your machine?&lt;/p&gt;

&lt;p&gt;Unless you are paying for their plans, the free tier has some limitations, let me list a few:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Limited private documents [Papeeria], or no private document [Latex base]&lt;/li&gt;
&lt;li&gt;No GitHub integration [Overleaf], or only public repositories [Papeeria]&lt;/li&gt;
&lt;li&gt;No collaborators [Overleaf]&lt;/li&gt;
&lt;li&gt;Need to be online.&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Personally, I choose to set up LaTeX on my local so that my co-authors and I can work on the same LaTeX document and sync with private GitHub repositories.&lt;/p&gt;

&lt;p&gt;Secondly, unlike online services, generating a preview on a local machine is much quicker too. When I will hit &lt;em&gt;Cmd+S&lt;/em&gt;, the preview will be generated in 3-seconds.&lt;/p&gt;

&lt;h1&gt;
  
  
  Set up LaTeX editor on Atom
&lt;/h1&gt;

&lt;p&gt;In this section, I will guide you through setting up a LaTex editor on &lt;a href="https://atom.io/" rel="noopener noreferrer"&gt;Atom&lt;/a&gt; on macOS. The setup consists of 4 parts:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;setup TeX distributions&lt;/li&gt;
&lt;li&gt;setup Atom&lt;/li&gt;
&lt;li&gt;install Latexmk&lt;/li&gt;
&lt;li&gt;install packages in Atom&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  TeX distributions
&lt;/h2&gt;

&lt;p&gt;On OS X, download &lt;a href="http://www.tug.org/mactex/" rel="noopener noreferrer"&gt;MacTeX&lt;/a&gt;. As of April 2020, the distribution is MacTeX-2020. You can choose to download the full package, which is 4GB. It contains all the files that most users need, and you don’t have to face the daunting task of searching for missing components. Highly recommended.&lt;/p&gt;

&lt;p&gt;If you don’t want to install the entire MacTeX distribution — which is pretty big, and if you are someone like me who prefers to select what gets installed, you can download &lt;a href="https://www.tug.org/mactex/morepackages.html" rel="noopener noreferrer"&gt;BasicTeX&lt;/a&gt;, which is only 80MB. BasicTeX contains the &lt;a href="https://www.tug.org/texlive/" rel="noopener noreferrer"&gt;TeX Live&lt;/a&gt; distribution that suffices the need to generate PDF from &lt;em&gt;.tex&lt;/em&gt; files; it does not include GUI applications, Ghostscript, and other libraries. But this means that you have to search and download all the packages you may need. I will cover this later.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-02.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-02.webp"&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-03.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-03.webp"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;MacTeX is for OS X. But if you are on Linux or Windows, download &lt;a href="https://www.tug.org/texlive/" rel="noopener noreferrer"&gt;TeX Live&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;
  
  
  Atom
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-04.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-04.webp"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Atom is a beautiful text editor that is available on OS X, Windows, and Linux. It has a package manager that allows you to install thousands of open source packages that add new features and functionality to your editor. You can easily browse your project files on a single interface. You can split your interface into multiple panes to edit/reference code across files. &lt;a href="https://atom.io/" rel="noopener noreferrer"&gt;Download and install Atom&lt;/a&gt;.&lt;/p&gt;
&lt;h2&gt;
  
  
  Latexmk
&lt;/h2&gt;

&lt;p&gt;Latexmk is needed to automate the process of generating a LaTeX document. Nothing to explain here, we need this to work. The installation is simple, from the Terminal, type:&lt;/p&gt;

&lt;p&gt;sudo tlmgr install latexmk&lt;/p&gt;

&lt;p&gt;This will prompt you for your password and install the &lt;em&gt;latexmk&lt;/em&gt; package.&lt;/p&gt;
&lt;h2&gt;
  
  
  Atom packages
&lt;/h2&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-05.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-05.webp"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;These packages enable you to use Atom as a LaTeX editor.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Compile&lt;/strong&gt;. &lt;a href="https://atom.io/packages/latex" rel="noopener noreferrer"&gt;Latex package&lt;/a&gt; compiles LaTeX documents from within Atom. It will execute Latexmk to build your work.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Preview PDF&lt;/strong&gt;. &lt;a href="https://atom.io/packages/pdf-view" rel="noopener noreferrer"&gt;PDF View package&lt;/a&gt; enables you to view PDF files in Atom. This allows you to preview your work side-by-side. Clicking on the preview will bring your cursor to the syntax's location.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Syntax highlighting&lt;/strong&gt;. &lt;a href="https://atom.io/packages/language-latex" rel="noopener noreferrer"&gt;Language-LaTeX package&lt;/a&gt; does LaTeX syntax highlighting in Atom. It is particularly useful to help you check that your syntax has valid LaTeX grammar.&lt;/li&gt;
&lt;/ul&gt;
&lt;h1&gt;
  
  
  You did it!
&lt;/h1&gt;

&lt;p&gt;At this point, you should have a working LaTeX editor on your Atom. I recommend a few changes to the configurations. Go to &lt;em&gt;Atom -&amp;gt; Preferences -&amp;gt; Packages&lt;/em&gt;. Go to &lt;em&gt;Settings&lt;/em&gt; of the &lt;em&gt;latex&lt;/em&gt; package.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-06.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fjinglescode.github.io%2Fassets%2Fimg%2Fposts%2Fsetup-latex-atom-06.webp"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[check] Build on Save&lt;/li&gt;
&lt;li&gt;[uncheck] Open Result after Successful Build&lt;/li&gt;
&lt;li&gt;[uncheck] Open Result in Background&lt;/li&gt;
&lt;li&gt;Under Output Directory, type “&lt;em&gt;build&lt;/em&gt;”&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;In &lt;em&gt;language-latex&lt;/em&gt; package:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;[check] Soft Wrap&lt;/li&gt;
&lt;/ul&gt;
&lt;h2&gt;
  
  
  Additional packages
&lt;/h2&gt;

&lt;p&gt;If you have download BasicTeX, chances are you have to download packages that publishers use in their templates. The easiest way is to Google search for the missing packages, or search &lt;a href="https://ctan.org/pkg" rel="noopener noreferrer"&gt;Comprehensive TEX Archive Network (CTAN)&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;For example, &lt;em&gt;algorithmic&lt;/em&gt; is one of the commonly used packages, and we might see this in the document.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;\usepackage{algorithmic}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To install packages, open the Terminal, type:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;sudo tlmgr install algorithms
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;There you go! I hope you enjoy your LaTeX editor and love Atom even more. Want to thank all the open-source contributors for &lt;a href="https://github.com/atom/atom" rel="noopener noreferrer"&gt;Atom&lt;/a&gt;, &lt;a href="https://github.com/thomasjo/atom-latex" rel="noopener noreferrer"&gt;atom-latex&lt;/a&gt;, &lt;a href="https://github.com/izuzak/atom-pdf-view" rel="noopener noreferrer"&gt;atom-pdf-view&lt;/a&gt;, &lt;a href="https://github.com/area/language-latex" rel="noopener noreferrer"&gt;language-latex&lt;/a&gt;.&lt;/p&gt;

</description>
    </item>
    <item>
      <title>Computer vision to diagnose coronavirus</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Fri, 01 May 2020 13:26:21 +0000</pubDate>
      <link>https://dev.to/jinglescode/computer-vision-to-diagnose-coronavirus-514n</link>
      <guid>https://dev.to/jinglescode/computer-vision-to-diagnose-coronavirus-514n</guid>
      <description>&lt;p&gt;As America has collectively tested a million people and some test results are &lt;a href="https://www.technologyreview.com/2020/04/05/998460/covid-19-test-results-faster-commercial-labs-delays-coronavirus/"&gt;taking more than a week&lt;/a&gt;. Testing of coronavirus remains among the most pressing problems with America’s response to the pandemic — this calls for quick and accurate coronavirus diagnostics.&lt;/p&gt;

&lt;p&gt;Alibaba Group built an &lt;a href="https://www.alizila.com/how-damo-academys-ai-system-detects-coronavirus-cases/"&gt;artificial intelligence system&lt;/a&gt; capable of virus-diagnosis and analysis from CT scan imagery. A CT machine produces 300 to 400 images per patients, and it would take a very experienced doctor at least 10 minutes to diagnose one patient. An artificial intelligence system can diagnose in less than 30 seconds and with a 96% accuracy.&lt;/p&gt;

&lt;p&gt;These AI-enabled medical-imaging systems use computer vision to classify cases and to detect signs of improvement. It served as an analytical and screening tool, providing the opinion for doctors to make decisions. These AI systems have been used by 26 hospitals and has helped diagnose over 30,000 cases in China, and it is on its way to &lt;a href="https://www.bloomberg.com/news/articles/2020-03-19/alibaba-pitches-diagnostic-tool-to-europe-in-china-outreach-move"&gt;help other countries&lt;/a&gt;.&lt;/p&gt;

</description>
      <category>datascience</category>
    </item>
    <item>
      <title>Virtual assistant to decrease the load on call centres</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Sat, 25 Apr 2020 06:05:01 +0000</pubDate>
      <link>https://dev.to/jinglescode/virtual-assistant-to-decrease-the-load-on-call-centres-2lm3</link>
      <guid>https://dev.to/jinglescode/virtual-assistant-to-decrease-the-load-on-call-centres-2lm3</guid>
      <description>&lt;p&gt;Emergency and medical services around the globe have seen an increased call volume from people showing symptoms of coronavirus. As customer representatives and social workers are increasingly ordered to work from home, companies and countries are turning to AI to bridge the resulting gaps in service.&lt;/p&gt;

&lt;p&gt;After realizing that many people were asking the same general questions, implementing virtual assistant seemed to be the solution. Copenhagen used a &lt;a href="https://news.microsoft.com/transform/how-international-health-care-organizations-are-using-bots-to-help-fight-covid-19/"&gt;virtual assistant bot to screen people&lt;/a&gt; for coronavirus; the bot answered 30,000 calls on its first day of work. This frees up healthcare professionals from the need to respond to inquiries and focus their time on providing care to patients.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://azuremarketplace.microsoft.com/en-us/marketplace/apps/microsoft-hcb.microsofthealthcarebot"&gt;Microsoft Healthcare Bot&lt;/a&gt; provides self-service symptom checker with its language understanding models tuned to understand medical and clinical terminology. This self-assessment bot is critical to provide solutions that can help patients and clinical teams in the fight against this global health crisis.&lt;/p&gt;

</description>
      <category>datascience</category>
    </item>
    <item>
      <title>Artificial intelligence research for drug discovery</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Wed, 22 Apr 2020 06:08:14 +0000</pubDate>
      <link>https://dev.to/jinglescode/artificial-intelligence-research-for-drug-discovery-2d4l</link>
      <guid>https://dev.to/jinglescode/artificial-intelligence-research-for-drug-discovery-2d4l</guid>
      <description>&lt;p&gt;The &lt;a href="https://covid19-hpc-consortium.org/"&gt;COVID-19 High-Performance Computing Consortium&lt;/a&gt; is a research consortium consisting of tech companies, academia, national laboratories and federal agencies.&lt;/p&gt;

&lt;p&gt;The research community have provided their technical capabilities to work on complex research projects to help fight the virus. Researchers are applying artificial intelligence algorithms, building machine learning models, generative networks, and reinforcement learning; for drug discovery, discovering molecular &amp;amp; genome characterization and structures analysis, and simulations.&lt;/p&gt;

&lt;p&gt;Companies and agencies are providing free Cloud computational resources of over 402 petaflops, over 3 million CPU cores and over 40,000 GPUs. These supercomputers have empowered top scientists and researchers with the computation needs to ensure rapid results.&lt;/p&gt;

</description>
      <category>datascience</category>
    </item>
    <item>
      <title>Demand forecasting to mitigate supply gap</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Thu, 16 Apr 2020 10:19:53 +0000</pubDate>
      <link>https://dev.to/jinglescode/demand-forecasting-to-mitigate-supply-gap-4iii</link>
      <guid>https://dev.to/jinglescode/demand-forecasting-to-mitigate-supply-gap-4iii</guid>
      <description>&lt;p&gt;The coronavirus has affected everyone around the globe. People have lost their loved ones, countries have shut down, medical professionals are working around the clock and people have lost their jobs.&lt;/p&gt;

&lt;p&gt;As artificial intelligence has been one of the hottest topics around for several years. How has it been relevant in such times, and how can it help humanity in beating this crisis?&lt;/p&gt;

&lt;p&gt;The Institute for Health Metrics and Evaluation (IHME) is a research organization at the University of Washington School of Medicine. They released a set of &lt;a href="https://covid19.healthdata.org/united-states-of-america"&gt;coronavirus data forecasts visualizations&lt;/a&gt; that the White House, Federal Emergency Management Agency and hospitals are using to mobilize resources.&lt;/p&gt;

&lt;p&gt;IHME developed a model that predicts the demand for the number of hospital beds and ventilators. It also predicts the number of deaths from coronavirus from around the world over the next 4 months.&lt;/p&gt;

&lt;p&gt;IHME &lt;a href="https://www.medrxiv.org/content/10.1101/2020.03.27.20043752v1"&gt;published their analysis&lt;/a&gt; on March 26. They predicted that the demand for hospital services is likely going to &lt;a href="http://www.healthdata.org/news-release/new-covid-19-forecasts-us-hospitals-could-be-overwhelmed-second-week-april-demand-icu"&gt;exceed capacity substantially&lt;/a&gt;. 41 states will need more ICU beds than what hospitals have available, with 11 states needing to increase the number of ICU beds by more than 50%. These analyses have help government and organizations to meet the incoming demand and implement strategies to mitigate this supply gap.&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>datascience</category>
    </item>
    <item>
      <title>Clean your data</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Sat, 11 Apr 2020 01:29:03 +0000</pubDate>
      <link>https://dev.to/jinglescode/clean-your-data-3i2o</link>
      <guid>https://dev.to/jinglescode/clean-your-data-3i2o</guid>
      <description>&lt;p&gt;Before you can tell any data stories, you need data. If you are working in an organisation, a class assignment, or a Kaggle dataset; you have the data. You have to figure out what is the story you want to tell.&lt;/p&gt;

&lt;p&gt;If you don’t have data yet, you have to define your hypothesis before collecting data. Your hypothesis should be measurable and clear. This will guide you to find or collect suitable dataset for analysis. Try searching for an open-source dataset that might answer your key question. However, if your question is niche, you have to build your data collection system.&lt;/p&gt;

&lt;p&gt;Before you can extract any insights from your data, you have to ensure that the data is correct. This process is defined as &lt;a href="https://jinglescode.github.io/datascience/2020/02/18/data-scientist-dirtiest-job-century/"&gt;&lt;strong&gt;data cleaning&lt;/strong&gt;&lt;/a&gt;. Typically you want to clean data that are incomplete, inaccurate, inconsistent and duplicated; to have accurate results.&lt;/p&gt;

&lt;h2&gt;
  
  
  Identify bad data
&lt;/h2&gt;

&lt;p&gt;Imagine if someone spotted an error during your presentation, that will make your work less credible. For example, if you have a dataset that contains human age, it wouldn’t make sense if someone is 5,000 years old. You might have to remove them first.&lt;/p&gt;

&lt;h2&gt;
  
  
  Identify missing values
&lt;/h2&gt;

&lt;p&gt;Missing values can be represented as empty values or values that are out of range, like “-1” or “-99" for human age. Your job is to identify and handle these missing values. You may have to get rid of columns or rows that have too many missing values.&lt;/p&gt;

&lt;h2&gt;
  
  
  Look for outliers
&lt;/h2&gt;

&lt;p&gt;These are data points that contain values that are outside the normal range. Unlike &lt;em&gt;bad data&lt;/em&gt;, outliers’ values are valid. For example, in the &lt;a href="https://jinglescode.github.io/datascience/2019/07/13/airbnb-in-seattle-data-analysis/"&gt;Seattle Airbnb dataset&lt;/a&gt;, there is 1 host who set the minimum rental nights to 1,000. It may offer interesting stories and insights, but they may also skew your results. You have to identify and decide how to deal with these kinds of data.&lt;/p&gt;

</description>
      <category>datascience</category>
    </item>
    <item>
      <title>Realistic Deepfakes in 5 Minutes on Colab</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Mon, 06 Apr 2020 11:24:18 +0000</pubDate>
      <link>https://dev.to/jinglescode/realistic-deepfakes-in-5-minutes-on-colab-b7a</link>
      <guid>https://dev.to/jinglescode/realistic-deepfakes-in-5-minutes-on-colab-b7a</guid>
      <description>&lt;p&gt;Ladies and gentlemen, Deepfake videos are so easy to create, that anyone can make one. You do not need a PhD, and you don’t have to train models for hours, you don’t even have to take a course on Generative Adversarial Network.&lt;/p&gt;

&lt;p&gt;All that you have to do is, to record a video of yourself, and pick one photo of a person you want to impersonate. Machine learning will animate the person in the picture the way you want it in your video.&lt;/p&gt;

&lt;h3&gt;
  
  
  Generate Deepfakes with a single image
&lt;/h3&gt;

&lt;p&gt;Image animation aims to generate video sequences such that the person in the source image is animated according to the motion of a video.&lt;/p&gt;

&lt;p&gt;This technology lies within the field of computer vision, and academic researchers have been working on to produce more realistic videos. It leverages on machine learning to manipulate and generate visual images or videos that replaces a person with someone else’s.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--TGPwWu3C--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-02.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--TGPwWu3C--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-02.gif" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;First Order Motion Model for Image Animation [by &lt;a href="https://aliaksandrsiarohin.github.io/first-order-model-website/"&gt;Aliaksandr&lt;/a&gt;]&lt;/p&gt;

&lt;p&gt;For the love of science and from a research standpoint, &lt;a href="https://aliaksandrsiarohin.github.io/first-order-model-website/"&gt;Aliaksandr’s work&lt;/a&gt; is certainly impressive. It has been published in &lt;a href="http://papers.nips.cc/paper/8935-first-order-motion-model-for-image-animation"&gt;NeurIPS&lt;/a&gt;, and the &lt;a href="https://github.com/AliaksandrSiarohin/first-order-model"&gt;source codes&lt;/a&gt; are available online.&lt;/p&gt;

&lt;p&gt;His work outperforms state of the art on all the benchmarks, and it works on a variety of images (faces, body, cartoon and robot). The model is so flexible that you can create good quality Deepfakes with a single image of the target object.&lt;/p&gt;

&lt;h4&gt;
  
  
  No prior information required
&lt;/h4&gt;

&lt;p&gt;Its ability to learn the facial movements is unbelievable. You can see that it can identify key points on the face, and it follows these key points to the movements in the video very well.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--mTnaimvz--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-03.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--mTnaimvz--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-03.gif" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Model learns to identify key points [by &lt;a href="https://aliaksandrsiarohin.github.io/first-order-model-website/"&gt;Aliaksandr&lt;/a&gt;]&lt;/p&gt;

&lt;p&gt;In previous works, we need additional information such as facial landmarks to map head movement and pose estimation to map full-body movement.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--Hb1nCD6j--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-04.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--Hb1nCD6j--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-04.webp" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Past works require facial landmarks and pose estimation [by &lt;a href="https://aliaksandrsiarohin.github.io/first-order-model-website/"&gt;Aliaksandr&lt;/a&gt;]&lt;/p&gt;

&lt;p&gt;In this work, it &lt;strong&gt;can work without using any annotation or prior information&lt;/strong&gt; about the specific object to animate. Once the model has trained on faces, the model can transfer any motion onto any faces.&lt;/p&gt;

&lt;h4&gt;
  
  
  Beautiful head movement
&lt;/h4&gt;

&lt;p&gt;You can record a video of yourself and animate the person in the photo. Yes, even a painting portrait of Mona Lisa.&lt;/p&gt;

&lt;p&gt;You can look up and turn your head around. You can say something, and the mouth movements look great. You can roll your eyes, and it maps the eye movements nicely onto the target video.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--0PbcFfbZ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-05.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--0PbcFfbZ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-05.gif" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Generated head movements [by &lt;a href="https://aliaksandrsiarohin.github.io/first-order-model-website/"&gt;Aliaksandr&lt;/a&gt;]&lt;/p&gt;

&lt;h4&gt;
  
  
  Full-body movement too
&lt;/h4&gt;

&lt;p&gt;It works for videos with full-body movement as well! Theoretically, this means that you can take the &lt;a href="https://www.youtube.com/watch?v=b6pomaq30Gg"&gt;Billie Jean video&lt;/a&gt; and make Donald Trump do moonwalk like Michael Jackson.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--QJTngX8G--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-06.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--QJTngX8G--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-06.gif" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Generated full-body movements [by &lt;a href="https://aliaksandrsiarohin.github.io/first-order-model-website/"&gt;Aliaksandr&lt;/a&gt;]&lt;/p&gt;

&lt;h4&gt;
  
  
  It generates the background
&lt;/h4&gt;

&lt;p&gt;As the person covers the part of the image, the algorithm needs to figure out the background behind the person. In this work, it &lt;strong&gt;automatically generates the background that is covered&lt;/strong&gt; by the moving person — absolutely fantastic.&lt;/p&gt;

&lt;h3&gt;
  
  
  How does it work?
&lt;/h3&gt;

&lt;p&gt;Aliaksandr work consists of the motion extractor which learns to extract key points along with their local affine transformations. There is a generator network that models occlusions in the target motions and combines the appearance extracted from the source image and the motion derived from the driving video.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--IGtUxXEo--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-07.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--IGtUxXEo--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/deepfake-existence-07.webp" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;First Order Motion Model [from &lt;a href="http://papers.nips.cc/paper/8935-first-order-motion-model-for-image-animation"&gt;paper&lt;/a&gt;]&lt;/p&gt;

&lt;p&gt;To understand how it works, I suggest you to visit the &lt;a href="https://aliaksandrsiarohin.github.io/first-order-model-website/"&gt;GitHub page&lt;/a&gt; and examine the &lt;a href="http://papers.nips.cc/paper/8935-first-order-motion-model-for-image-animation"&gt;research paper&lt;/a&gt;. You can also watch his &lt;a href="https://www.youtube.com/watch?v=u-0cQ-grXBQ"&gt;video&lt;/a&gt; explaining how it works. Solid cool stuff.&lt;/p&gt;

&lt;p&gt;&lt;iframe width="710" height="399" src="https://www.youtube.com/embed/u-0cQ-grXBQ"&gt;
&lt;/iframe&gt;
&lt;/p&gt;

&lt;p&gt;Want to make your own? Check out this &lt;a href="https://colab.research.google.com/github/AliaksandrSiarohin/first-order-model/blob/master/demo.ipynb"&gt;Colab notebook&lt;/a&gt;.&lt;/p&gt;

&lt;h3&gt;
  
  
  Negative consequences of Deepfakes
&lt;/h3&gt;

&lt;p&gt;Deepfakes have garnered widespread attention for their uses in fake news, frauds, scams, and many other illegal activities.&lt;/p&gt;

&lt;p&gt;People used to share their Deepfakes videos which they have created in the subreddit, &lt;a href="https://www.reddit.com/r/deepfakes"&gt;&lt;em&gt;r/deepfakes&lt;/em&gt;&lt;/a&gt;. Many of these videos are swapping celebrities faces, such as Gal Gadot and Taylor Swift, onto pornography performers’ bodies.&lt;/p&gt;

&lt;p&gt;Many Deepfakes videos are also shared depicting politicians. It has affected politics by being authoritarian governments to spread false information, hate and fear.&lt;/p&gt;

&lt;p&gt;This technology has concerned both industry and government to control and limit the use of Deepfakes. In February 2018, &lt;a href="https://www.vice.com/en_us/article/neqb98/reddit-shuts-down-deepfakes"&gt;Reddit suspended r/deepfakes&lt;/a&gt; for violating policies. In June 2019, it &lt;a href="https://www.congress.gov/bill/116th-congress/house-bill/3230"&gt;elicited attention from the government&lt;/a&gt; to combat the spread of disinformation through the limitation of Deepfakes video alteration technology.&lt;/p&gt;

&lt;h3&gt;
  
  
  Join me to inform about its existence
&lt;/h3&gt;

&lt;p&gt;Knowing that there are people around the world who will abuse this technology. You may ask, why am I writing about this? Why am I spreading this knowledge?&lt;/p&gt;

&lt;p&gt;Since in this current day and age, anyone can create fake videos easily, such generative models aren’t science fiction anymore. I aim to educate the public so that people know about the existence of such technology. &lt;strong&gt;By understanding its existence, people can be aware and pay more attention to discerning between real and fake.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Fake news is part of the fabric of today’s internet, now with Deepfakes easily assessable; this has brought disinformation to a whole new level. It has &lt;a href="https://www.ft.com/content/4bf4277c-f527-11e9-a79c-bc9acae3b654"&gt;affected politics&lt;/a&gt; by spreading false information. It has led to people exploited by scammers who are using it to cheat money online.&lt;/p&gt;

&lt;p&gt;The world is already in a mess right now due to coronavirus; I am not sure how people will abuse this technology at such times. Deepfakes, as a form of disinformation, are dangerous. &lt;strong&gt;We need to educate people about technology, and people need to discern the truth rather than just believing what we see.&lt;/strong&gt;&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>datascience</category>
    </item>
    <item>
      <title>Holiday Itinerary with Machine Learning</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Tue, 31 Mar 2020 03:04:40 +0000</pubDate>
      <link>https://dev.to/jinglescode/holiday-itinerary-with-machine-learning-2gbo</link>
      <guid>https://dev.to/jinglescode/holiday-itinerary-with-machine-learning-2gbo</guid>
      <description>&lt;p&gt;In October 2019, my friends and I are planning our awesome holiday in &lt;a href="https://tmpdrive.google.com/open?id=1klLY2yu88v6EpUkSP_vuOD4j1Dqi2ZHr&amp;amp;usp=sharing"&gt;Greece and Amalfi&lt;/a&gt;. The way we plan our perfect holiday itinerary is by using Google Map. We search for the places to visit and pin markers on Google Map for attractions that might interest us.&lt;/p&gt;

&lt;p&gt;For me, to optimise our vacation to the fullest, as to cover as much ground as possible; I have three questions:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;How many days should I spend at one location?&lt;/li&gt;
&lt;li&gt;Which attractions/locations should I visit each day?&lt;/li&gt;
&lt;li&gt;What is the most optimal route from a place to another?&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Let me show you the itinerary my algorithms have proposed for us. It is recommending the best sequence of places to visit, day by day.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--UScBK0HZ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-02.gif" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--UScBK0HZ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_66%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-02.gif" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;  Start from  &lt;strong&gt;Athens&lt;/strong&gt;, visit the Acropolis and other historic archaeological sites. That’s a great start!&lt;/li&gt;
&lt;li&gt;  Travel to  &lt;strong&gt;Santorini&lt;/strong&gt;, a beautiful island with whitewashed homes and picture-perfect sunsets.&lt;/li&gt;
&lt;li&gt;  Sail to  &lt;strong&gt;Crete&lt;/strong&gt;, for a road trip adventure of a lifetime, by exploring hidden corners and tasting the best of Greek cuisine.&lt;/li&gt;
&lt;li&gt;  Fly to Italy and travel to  &lt;strong&gt;Amalfi&lt;/strong&gt;  and  &lt;strong&gt;Ravello&lt;/strong&gt;  where the terraces are blending along the cliffs with the most spectacular coastal scenery.&lt;/li&gt;
&lt;li&gt;  Travel to  &lt;strong&gt;Sorrento&lt;/strong&gt;  and  &lt;strong&gt;Naples&lt;/strong&gt;, indulge in the best pasta, pizza, limoncello, and gelato.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Great holiday itinerary isn’t it?! This sequence of locations is fully generated by the algorithm. Would you like to try this for your next holiday? I have the  &lt;a href="https://gist.github.com/jinglescode/418a7ef75471dd891ae8621f36db6b62"&gt;codes&lt;/a&gt;  ready for you.&lt;/p&gt;

&lt;p&gt;In this article, I am going to show you exactly how you can do it too for your next vacation!&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--gd29C4cl--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-03.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--gd29C4cl--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-03.webp" alt="View of Atrani, while hiking to Ravello [photo by Hong Jing (Jingles)]"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  Preparation
&lt;/h1&gt;

&lt;p&gt;&lt;strong&gt;Pin locations on Google Maps.&lt;/strong&gt; Visit &lt;a href="https://www.google.com/maps/d/u/0/home"&gt;My Maps&lt;/a&gt; and create a new map for your next holiday destination. Look for places you want to visit. Attractions? Theme parks? Restaurants? Fill up your map with markers by searching for these places and “&lt;em&gt;Add to map&lt;/em&gt;”. Keep doing this until you have all the places you want to visit on the map.  &lt;em&gt;I will wait for you.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--aVCcQKDn--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-04.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--aVCcQKDn--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-04.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--P6K6uC98--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-05.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--P6K6uC98--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-05.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Export map and upload into Colab.&lt;/strong&gt; After all the hard work of researching and pinning places to visit, we are ready to export the map. Select “&lt;em&gt;Export to KML/KMZ&lt;/em&gt;”, and remember to check “&lt;em&gt;Export as KML&lt;/em&gt;”. You will download a KML file.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--BRAPW1cG--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-06.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--BRAPW1cG--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-06.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--gB5f5SIN--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-07.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--gB5f5SIN--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-07.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--hTRLUZga--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-08.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--hTRLUZga--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-08.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Export KML file from  &lt;a href="https://www.google.com/maps/d/u/0/home"&gt;Google Maps&lt;/a&gt;  and upload it in  &lt;a href="https://colab.research.google.com/"&gt;Colab&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Next, go to  &lt;a href="https://colab.research.google.com/"&gt;Colab&lt;/a&gt;, an excellent Notebook environment maintained by Google. Open the File drawer on the left and upload the KML file which you have downloaded. We will be loading this KML file with  &lt;a href="https://pypi.org/project/beautifulsoup4/"&gt;BeautifulSoup&lt;/a&gt;  later.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Get the Google API key.&lt;/strong&gt; We will be plotting markers on Google Maps, so we need an API key. You can get it from the  &lt;a href="https://developers.google.com/maps/documentation/javascript/get-api-key"&gt;developers API page&lt;/a&gt;. Follow the instructions and you should get a key that looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight plaintext"&gt;&lt;code&gt;ZIzkSyDwhp5B0_tH1$Is@fAkeAp1keY3eLYmPLY
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;&lt;em&gt;Psst: this is a fake API key&lt;/em&gt; 🙃&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--E-FDkH7A--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-09.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--E-FDkH7A--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-09.webp" alt="Amalfi, Italy [photo by  [Tan Ying Ying](https://medium.com/u/f11516ac310b)]"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  Code walkthrough on Colab
&lt;/h1&gt;

&lt;p&gt;You can get the  &lt;a href="https://gist.github.com/jinglescode/418a7ef75471dd891ae8621f36db6b62"&gt;codes&lt;/a&gt;  and run it on Colab.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Define the parameters.&lt;/strong&gt; Let’s set the API key, KML filename, and the desired number of days for your long-awaited holiday.&lt;/p&gt;

&lt;p&gt;The API key is for plotting interactive Google Maps on Colab. The KML file contains the places of interests you have pinned on Google Maps. And lastly, the algorithm will determine where you should visit for each day based on the number of days you have set.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--BWrspjlM--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-10.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--BWrspjlM--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-10.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Load the data.&lt;/strong&gt; KML files are XML, and we can parse it with  &lt;a href="https://pypi.org/project/beautifulsoup4/"&gt;BeautifulSoup&lt;/a&gt;. Our places of interests are inside the “&lt;em&gt;Placemark&lt;/em&gt;” tag, so we will extract the “&lt;em&gt;name&lt;/em&gt;”, and “&lt;em&gt;coordinates&lt;/em&gt;” from each “&lt;em&gt;Placemark&lt;/em&gt;”.&lt;/p&gt;

&lt;p&gt;Let’s see if the DataFrame contains places we are planning to visit.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Group locations by proximity.&lt;/strong&gt; With the coordinates for each location in the DataFrame, we can group them into clusters. If two places are near each other, they will be in the same cluster. There are a few methods for clustering, I will introduce K-Means, spectral and mean-shift clustering.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://scikit-learn.org/stable/modules/generated/sklearn.cluster.KMeans.html"&gt;&lt;strong&gt;K-Means clustering&lt;/strong&gt;&lt;/a&gt;  aims to partition data points into a specified number of clusters. In which each data point belongs to the cluster that it is nearest to.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://scikit-learn.org/stable/modules/generated/sklearn.cluster.SpectralClustering.html"&gt;&lt;strong&gt;Spectral clustering&lt;/strong&gt;&lt;/a&gt;  is useful when the structure of the individual clusters is highly non-convex. It performs dimensionality reduction before clustering in fewer dimensions.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://scikit-learn.org/stable/modules/generated/sklearn.cluster.MeanShift.html"&gt;&lt;strong&gt;Mean shift clustering&lt;/strong&gt;&lt;/a&gt;  is a centroid-based algorithm which aims to discover blobs in a smooth density of data points. It works by updating candidates for centroids to be the mean of the points within a region.&lt;/p&gt;

&lt;p&gt;Which one did I end up using?  &lt;strong&gt;K-Means&lt;/strong&gt;. Because it is straight forward, and the most common one.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Plot it on Google Maps!&lt;/strong&gt; Now, we are ready to plot it on Google Maps to visualise the 12 main areas (because we are planning a holiday for 12 days). If you are running this on  &lt;a href="https://gist.github.com/jinglescode/418a7ef75471dd891ae8621f36db6b62"&gt;Colab&lt;/a&gt;, this map is interactive.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s---JgdOMDF--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-11.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s---JgdOMDF--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-11.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--UkJUcUlv--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-12.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--UkJUcUlv--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-12.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--0wh4TNTQ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-13.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--0wh4TNTQ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-13.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Athens. Santorini and Crete. Amalfi and Naples.&lt;/p&gt;

&lt;p&gt;The clustering algorithm did a great job grouping locations by proximity. Next, I am interested in finding out the most optimal route from a region to another region.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Find the middle of each cluster.&lt;/strong&gt; We need to find the middle point of each cluster. We do this with  &lt;a href="https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.groupby.html"&gt;Pandas&lt;/a&gt;  &lt;code&gt;groupby&lt;/code&gt;  function:  &lt;code&gt;places.groupby(‘cluster’).mean()&lt;/code&gt;. That will give us a DataFrame, where each row represents a cluster of locations, and it’s latitude and longitude.&lt;/p&gt;



&lt;p&gt;&lt;strong&gt;Find the distance between cluster.&lt;/strong&gt; We need to find the distance between each cluster. We use the middle point of each cluster and calculate the distance between clusters with  &lt;a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.spatial.distance.cdist.html"&gt;Scipy&lt;/a&gt;  &lt;code&gt;cdist&lt;/code&gt;  function.&lt;/p&gt;



&lt;p&gt;That will give us a DataFrame, with the distance of each cluster to every other cluster. We will call this matrix the  &lt;strong&gt;distance matrix&lt;/strong&gt;. Each value represents the distance between two clusters. A smaller value means the cluster is closer to each other.&lt;/p&gt;



&lt;p&gt;&lt;strong&gt;Find the shortest route.&lt;/strong&gt; With that distance matrix, we are ready to find the shortest path. Here is the code to compute the shortest path.&lt;/p&gt;



&lt;p&gt;Initially, I was using the  &lt;a href="https://en.wikipedia.org/wiki/Travelling_salesman_problem"&gt;travelling salesman algorithm&lt;/a&gt;, but I find that the  &lt;a href="https://developers.google.com/optimization/routing/tsp"&gt;code’s&lt;/a&gt;  a little overkill and overcomplicated, so I wrote my own instead. This is the result based on the  &lt;strong&gt;shortest path algorithm&lt;/strong&gt;, a sequence from cluster to cluster.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight plaintext"&gt;&lt;code&gt;2 -&amp;gt; 1 -&amp;gt; 6 -&amp;gt; 10 -&amp;gt; 9 -&amp;gt; 3 -&amp;gt; 11 -&amp;gt; 7 -&amp;gt; 4 -&amp;gt; 5 -&amp;gt; 8 -&amp;gt; 0
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Show the recommended itinerary.&lt;/strong&gt; Now we are ready to plot our markers on Google Map. It has recommended us to go from  &lt;strong&gt;Athens&lt;/strong&gt;,  &lt;strong&gt;Santorini&lt;/strong&gt;, and  &lt;strong&gt;Crete&lt;/strong&gt;. Then to  &lt;strong&gt;Amalfi&lt;/strong&gt;,  &lt;strong&gt;Ravello&lt;/strong&gt;,  &lt;strong&gt;Sorrento&lt;/strong&gt;  and  &lt;strong&gt;Naples&lt;/strong&gt;. If you are running this on  &lt;a href="https://gist.github.com/jinglescode/418a7ef75471dd891ae8621f36db6b62"&gt;Colab&lt;/a&gt;, this map is interactive.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--8HuI6G7d--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-14.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--8HuI6G7d--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-14.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--q0vNGKca--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-15.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--q0vNGKca--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-15.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--xgamOtIm--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-16.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--xgamOtIm--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-16.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--JmhTtHA3--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-17.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--JmhTtHA3--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-17.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--jBoolBPJ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-18.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--jBoolBPJ--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-18.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Start with Athens, Santorini, and then Crete. To Amalfi, Ravello, Sorrento and Naples.&lt;/p&gt;

&lt;p&gt;If you prefer to see it in  &lt;a href="https://gist.github.com/jinglescode/f7a89cb57d71d84ace806a9f2cbaf340"&gt;tabular form&lt;/a&gt;, we can display it with Pandas.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight"&gt;&lt;pre class="highlight plaintext"&gt;&lt;code&gt;pd.set_option(‘display.max_rows’, None)  
places.sort_values(by=[‘days’])
&lt;/code&gt;&lt;/pre&gt;&lt;/div&gt;



&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s---1AhS6oL--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-19.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s---1AhS6oL--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-19.webp" alt="Oia, Santorini, Greece [photo by [Clement Lim](https://medium.com/u/4455aa7806f7)]"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h1&gt;
  
  
  What can be improved?
&lt;/h1&gt;

&lt;p&gt;This algorithm does not take the number of attractions in one region into consideration. For instance, we absolutely didn’t want to explore everything in Athens in one day! Neither is just one day at Santorini enough.&lt;/p&gt;

&lt;p&gt;We need a type of clustering algorithm that constraint on the maximum number of points in a cluster. Such that, every cluster should have about the same number of locations. If a cluster is becoming too dense, it will split the cluster into two or more clusters.&lt;/p&gt;

&lt;p&gt;But if you are travelling in one city, the result can be promising. This is the generated itinerary for New York City. Feel free to download my  &lt;a href="https://drive.google.com/open?id=1SSy8rwXf-Z7UKwPT1Jv_yZDNisHqR3H2&amp;amp;usp=sharing"&gt;itinerary&lt;/a&gt;  and try it for yourself.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--6SD4K0kF--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-20.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--6SD4K0kF--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-20.webp%23float_left" alt="[New York City](https://drive.google.com/open?id=1SSy8rwXf-Z7UKwPT1Jv_yZDNisHqR3H2&amp;amp;usp=sharing)"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--WBFqXzZA--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-21.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--WBFqXzZA--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/https://jinglescode.github.io/assets/img/posts/improve-holiday-itinerary-with-machine-learning-21.webp" alt="New York City [photo by  [Hong Jing (Jingles)](https://medium.com/u/641197e9ee36)]"&gt;&lt;/a&gt;&lt;/p&gt;




&lt;h1&gt;
  
  
  Resources
&lt;/h1&gt;

&lt;p&gt;Do you want to try this for your next vacation? Here’s &lt;a href="https://gist.github.com/jinglescode/418a7ef75471dd891ae8621f36db6b62"&gt;the code&lt;/a&gt;. &lt;em&gt;You’re welcome.&lt;/em&gt; 😉&lt;/p&gt;

&lt;p&gt;Not sure where to visit in Greece or Amalfi? Go ahead and use our  &lt;a href="https://drive.google.com/open?id=1klLY2yu88v6EpUkSP_vuOD4j1Dqi2ZHr&amp;amp;usp=sharing"&gt;Greece x Amalfi itinerary&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Not sure where to visit in  &lt;a href="https://drive.google.com/open?id=1SSy8rwXf-Z7UKwPT1Jv_yZDNisHqR3H2&amp;amp;usp=sharing"&gt;New York City&lt;/a&gt;?&lt;/p&gt;

&lt;p&gt;Thanks for reading. 😃&lt;br&gt;&lt;br&gt;
Hope you enjoyed reading this article as much as I did preparing it.&lt;/p&gt;

</description>
      <category>machinelearning</category>
      <category>datascience</category>
      <category>tutorial</category>
    </item>
    <item>
      <title>High demand data  janitor  of the 21st century</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Wed, 25 Mar 2020 07:43:34 +0000</pubDate>
      <link>https://dev.to/jinglescode/high-demand-data-janitor-of-the-21st-century-5bf0</link>
      <guid>https://dev.to/jinglescode/high-demand-data-janitor-of-the-21st-century-5bf0</guid>
      <description>&lt;p&gt;Everyone who deals with data should have heard of the term “&lt;em&gt;dirty data”&lt;/em&gt;. &lt;strong&gt;Dirty data&lt;/strong&gt; &lt;strong&gt;takes away the&lt;/strong&gt; &lt;strong&gt;integrity&lt;/strong&gt; of the dataset. Some characteristics of dirty data are &lt;strong&gt;incomplete&lt;/strong&gt;, &lt;strong&gt;inaccurate&lt;/strong&gt;, &lt;strong&gt;inconsistent&lt;/strong&gt; and &lt;strong&gt;duplicated&lt;/strong&gt; data.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Incomplete data&lt;/strong&gt;  is when some essential features are empty. For example, let’s say your task is to  &lt;a href="https://www.kaggle.com/c/house-prices-advanced-regression-techniques"&gt;predict house prices&lt;/a&gt;. Let us assume that the “&lt;em&gt;area of the house&lt;/em&gt;” is critical to make a good prediction, but it’s missing. That might be challenging for you and your model might not perform well.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Inaccurate and inconsistent data&lt;/strong&gt;  is when the values are technically correct, but wrong based on the context. For example, when an employee changed his address, and it wasn’t updated. Or when there are many copies of the data and the data scientist got an outdated version.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Duplicate data&lt;/strong&gt;  is a common problem. Let me share with you a story that has happened to me while working in an e-commerce company. By design, when a visitor clicked on a “&lt;em&gt;collect voucher&lt;/em&gt;” button, the website sent a response to the server. This allowed us to measure the number of users who have collected vouchers.&lt;/p&gt;

&lt;p&gt;The site was running well, until one day something has changed and I didn’t know about it. The frontend developer added another response for when someone collected vouchers successfully. The  rationale  was that some vouchers could be out of stock. They wanted to track visitors who clicked on the button and those who have collected the vouchers.&lt;/p&gt;

&lt;p&gt;At that point, two responses were sent to the same log table. Looking at my reporting tool, the number of vouchers collected seemed to have doubled overnight!  As I had  deployed  a model the day before, I assumed that my new model was just that impressive. I remember giving a mental standing ovation to my little model, but later, I realised it was just double counting.&lt;/p&gt;

&lt;p&gt;Also, in the last five years as a data scientist, some of the data I’ve received are manual  entries  by corporate staff. These data are in Excel spreadsheets; many are inaccurate, incomplete, and inconsistent.&lt;/p&gt;

&lt;p&gt;Whether the data comes from manual human input or machine logs,  &lt;strong&gt;data&lt;/strong&gt; &lt;strong&gt;wrangling&lt;/strong&gt; &lt;strong&gt;is a large part of what happens in the real world&lt;/strong&gt;.  Data scientists have to deal with it.  For supervised learning to work, we need reliable, labelled data. You can’t build a predictive model unless the data are correctly labelled. But nobody likes labelling data.&lt;/p&gt;

&lt;p&gt;Many describe this as the &lt;a href="https://www.ibm.com/cloud/blog/ibm-data-catalog-data-scientists-productivity"&gt;80/20 rule&lt;/a&gt;. Data scientists spend only 20 percent of their time on building models and the other 80 percent gathering, analysing, cleaning, and reorganising data. Dirty data is the most time-consuming aspect of the typical data scientist’s work.&lt;/p&gt;

&lt;p&gt;It’s necessary to point out that  &lt;strong&gt;data cleaning is incredibly essential&lt;/strong&gt;; messy data won’t produce good results.  You might have heard the phrase, “&lt;em&gt;garbage in, garbage out&lt;/em&gt;”.&lt;/p&gt;

&lt;p&gt;Data scientists do make discoveries while swimming in data, but before data scientists can start training any models, they must first become data  janitors. The data needs cleaning, the data needs labelling.&lt;/p&gt;

</description>
      <category>datascience</category>
      <category>machinelearning</category>
    </item>
    <item>
      <title>The story of Edwin Barnes</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Wed, 18 Mar 2020 00:52:29 +0000</pubDate>
      <link>https://dev.to/jinglescode/the-story-of-edwin-barnes-2ojb</link>
      <guid>https://dev.to/jinglescode/the-story-of-edwin-barnes-2ojb</guid>
      <description>&lt;p&gt;&lt;a href="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmiro.medium.com%2Fmax%2F800%2F0%2A4o60fku0g2eixLAL.jpg" class="article-body-image-wrapper"&gt;&lt;img src="https://media.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmiro.medium.com%2Fmax%2F800%2F0%2A4o60fku0g2eixLAL.jpg"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Edwin C. Barnes (Left) and Thomas Edison (Right)&lt;/p&gt;

&lt;h2&gt;
  
  
  Desire: knowing what you want and think of how to get there
&lt;/h2&gt;

&lt;p&gt;Edwin Barnes wanted to become Thomas Edison’s partner. But he had no money. He had little education. He had no influence. He had no relationship with Edison.&lt;/p&gt;

&lt;p&gt;However, the desire of Edwin Barnes was so strong, and he needed to find a way to make it come true. So he used his brain and think!&lt;/p&gt;

&lt;p&gt;How could he move one step closer to making his desire come true?&lt;/p&gt;

&lt;p&gt;Barnes hopped onto a freight train to meet Edison. He plans to propose to work together as business partners, whether Edison liked it or not.&lt;/p&gt;

&lt;h2&gt;
  
  
  Faith: having the ability to believe
&lt;/h2&gt;

&lt;p&gt;Another crucial part is the  &lt;strong&gt;faith to believe that you would be able to achieve what you desire&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Faith is putting in the trust in things yet unseen, wishes and dreams that are yet to come true. Having faith means removing that limiting belief and know that you have it in you to get what you desire.  &lt;strong&gt;Nothing is impossible, it’s just not yet possible&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;There are many setbacks when chasing after our desire, failure can destroy our own confidence and maybe, we might give up.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;em&gt;However, Life has a way of developing strength and wisdom in individuals, through temporary defeat and failure, and do not overlook the fact that there is no such reality as a permanent failure until an experience has been accepted as such. — Napoleon Hill&lt;/em&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Barnes knowing that he has limited resources has the faith that he can and will become Edison’s partner.&lt;/p&gt;

&lt;h2&gt;
  
  
  Persistence: transmuting desire into reality
&lt;/h2&gt;

&lt;p&gt;The power of desire is dependent upon one’s Intensity of desire. Weak desire brings weak result, just like a small fire makes a small amount of heat.&lt;/p&gt;

&lt;p&gt;Barnes was immediately rejected by Edison as soon as he proposed his business partner idea. “&lt;em&gt;I will start anywhere. I will do anything Edison tells me to do, but before I am through, I will be his associate.&lt;/em&gt;”&lt;/p&gt;

&lt;p&gt;Edison was impressed by Barne’s attitude decided to offer him a job as a sweeper. It was far from his ideal goal, but it is one step closer. Barnes will not quit.&lt;/p&gt;

&lt;p&gt;He did not say, “&lt;em&gt;I will work there for a few months, and if I get no encouragement, I will quit and get a job somewhere else.&lt;/em&gt;” Barnes did not have backup plans. He did not say, “&lt;em&gt;I will keep my eyes open for another opportunity, in case I fail to get what I want in the Edison organisation.&lt;/em&gt;”&lt;/p&gt;

&lt;p&gt;Barnes focused all of his thoughts in ensuring that he will get what he wanted. He spent two years as a sweeper as he observed Edison’s way of working. Then, the opportunity came, when Edison designed a product called  &lt;em&gt;Ediphone&lt;/em&gt;  and Barnes stepped forward to pitch his idea to become Edison’s partner to sell the Ediphone.&lt;/p&gt;

&lt;p&gt;Barnes’ burning desire and persistence transformed into grit. Grit led him to stay focus. He said, “&lt;em&gt;There is but one thing in this world that I am determined to have, and that is a business association with Thomas Edison. I will burn all bridges behind me, and stake my entire future on my ability to get what I want.&lt;/em&gt;”&lt;/p&gt;

&lt;p&gt;It took Barnes five years of relentless effort to get his chance to finally work with Edison.&lt;/p&gt;

&lt;p&gt;That’s Barnes story of success!&lt;/p&gt;

</description>
      <category>career</category>
      <category>productivity</category>
    </item>
    <item>
      <title>How neuroscience inspires AI: Design AI systems that emulate human intelligence</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Thu, 12 Mar 2020 11:07:55 +0000</pubDate>
      <link>https://dev.to/jinglescode/how-neuroscience-inspires-ai-design-ai-systems-that-emulate-human-intelligence-2dn7</link>
      <guid>https://dev.to/jinglescode/how-neuroscience-inspires-ai-design-ai-systems-that-emulate-human-intelligence-2dn7</guid>
      <description>&lt;p&gt;Recent artificial intelligence advancements are taking the media by storm. They can perform impressive feats such as:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;  &lt;a href="https://www.vox.com/future-perfect/2020/2/14/21063487/self-driving-cars-autonomous-vehicles-waymo-cruise-uber"&gt;object recognition&lt;/a&gt; that &lt;a href="http://news.mit.edu/2019/human-reasoning-ai-driverless-car-navigation-0523"&gt;enables self-driving cars&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;  &lt;a href="https://www.theverge.com/2019/1/24/18196135/google-deepmind-ai-starcraft-2-victory"&gt;playing video games&lt;/a&gt; that &lt;a href="https://www.nature.com/articles/d41586-019-03298-6"&gt;outperform world best players&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;  &lt;a href="https://www.theverge.com/2020/1/1/21045635/google-ai-detect-breast-cancer-mammograms-healthcare"&gt;detecting breast cancer&lt;/a&gt; &lt;a href="https://www.nature.com/articles/s41586-019-1799-6"&gt;faster and more accurate than doctors&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These machines can perform tasks and solve problems better than us, but they are only designed to excel in the intended functions. However impressive these may seem to be, we are still decades away from building artificial human intelligence.&lt;/p&gt;

&lt;p&gt;Building machines that have truly human-like learning and cognitive that can learn and think as we do has been an ongoing pursuit. What does it mean for a &lt;a href="https://arxiv.org/pdf/1604.00289.pdf"&gt;machine to learn or think like a person&lt;/a&gt;?&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;  able to explain and understand problems&lt;/li&gt;
&lt;li&gt;  learning-to-learn to acquire knowledge&lt;/li&gt;
&lt;li&gt;  generalise knowledge to new tasks and situations&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Our understanding of our brain has allowed us to design models which emulating the brain. In a &lt;a href="https://www.sciencedaily.com/releases/2020/01/200130104921.htm"&gt;study by KAIST&lt;/a&gt;, they developed a computational and neural mechanism for human meta reinforcement learning.&lt;/p&gt;

&lt;p&gt;As humans, we can adapt to complexity and uncertainty when learning and making decisions. They aim to build models that can make decisions like humans and solves problems in the same way that humans do.&lt;/p&gt;

&lt;p&gt;Their discoveries have opened up the possibility that might lead to machines that can take on more human-like intelligence.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--YOsy4dKF--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-02.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--YOsy4dKF--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-02.webp" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Photo by &lt;a href="https://unsplash.com/@heftiba"&gt;Toa Heftiba&lt;/a&gt; on &lt;a href="https://unsplash.com"&gt;Unsplash&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Even with our eyes covered, one may still play the guitar well. But for an artificial intelligence system to play the guitar in a new environment (i.e. with eyes covered), it might not do as well.&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>machinelearning</category>
    </item>
    <item>
      <title>How neuroscience inspires AI: Neural networks that mimic brain structure</title>
      <dc:creator>Jingles (Hong Jing)</dc:creator>
      <pubDate>Sat, 07 Mar 2020 13:23:59 +0000</pubDate>
      <link>https://dev.to/jinglescode/how-neuroscience-inspires-ai-neural-networks-that-mimic-brain-structure-o5m</link>
      <guid>https://dev.to/jinglescode/how-neuroscience-inspires-ai-neural-networks-that-mimic-brain-structure-o5m</guid>
      <description>&lt;p&gt;We borrow inspirations from the structure of a human brain and designed the neural networks we know today. The idea of neurons in neural networks has similar characteristics to our biological neurons in our brain.&lt;/p&gt;

&lt;p&gt;A human brain contains about 86 billion neurons, each individually linked to other neurons. Biological neurons are cells, when it gets activated, it generates a spike and sends information to other neurons.&lt;/p&gt;

&lt;p&gt;Like our brain, a neural network consists of many interconnected neurons. When a neuron receives inputs from other neurons, it gets activated, and it sends information to other neurons.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--RIq9d6z2--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-03.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--RIq9d6z2--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-03.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;br&gt;
&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--d26l2B8u--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-04.webp%23float_left" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--d26l2B8u--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-04.webp%23float_left" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;An artificial neuron is a mathematical function conceived as a model of biological neurons [&lt;a href="https://en.wikipedia.org/wiki/Artificial_neuron"&gt;wikipedia&lt;/a&gt;]&lt;/p&gt;

&lt;p&gt;The brain’s plasticity allows us to learn and improve our skills. Every time we learn new things, we are creating and strengthening the connections between neurons. That’s why when we practise a task, we become better at it.&lt;/p&gt;

&lt;p&gt;Neural networks learn when we feed it with lots of data. Each connection of our neural network is associated with a weight that dictates the importance between neurons. During the training process, the weights are tuned accordingly to strengthen or weaken the connection.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://res.cloudinary.com/practicaldev/image/fetch/s--ii30dyMI--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-05.webp" class="article-body-image-wrapper"&gt;&lt;img src="https://res.cloudinary.com/practicaldev/image/fetch/s--ii30dyMI--/c_limit%2Cf_auto%2Cfl_progressive%2Cq_auto%2Cw_880/http://jinglescode.github.io/assets/img/posts/fascinating-relationship-between-ai-neuroscience-05.webp" alt=""&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Photo by &lt;a href="https://unsplash.com/@visucy"&gt;Nathalie Jolie&lt;/a&gt; on &lt;a href="https://unsplash.com"&gt;Unsplash&lt;/a&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Because we have seen enough cats, we can be confident that this is a cat. Likewise, if we provide our neural networks with enough cats images, it will start to recognise cats.&lt;/p&gt;
&lt;/blockquote&gt;

</description>
      <category>machinelearning</category>
    </item>
  </channel>
</rss>
