DEV Community: Kite

Kite Now Integrates with Jupyter

brendan — Tue, 04 Aug 2020 17:49:31 +0000

We’re thrilled to release a Kite integration for JupyterLab, plus special support for teams using JupyterHub. Data scientists can now get useful completions as they type in Jupyter notebooks, powered by Kite’s deep learning.

Check out the new completions experience in JupyterLab below.

Let’s dive into the details.

Kite automatically gives you longer completions sorted by relevance, without requiring you to run a single cell in your notebook or press tab to make completions appear.

Here’s a side-by-side comparison for JupyterLab’s native completions vs. Kite completions.

JupyterLab native completions are alphabetically sorted and have no documentation:

Whereas Kite’s completions are ranked by relevance, and Kite also shows you documentation for the highlighted completion:

Kite then turbocharges the JupyterLab experience with additional ML-powered completions features and over 100,000 Python docs just a click away in the Kite Copilot:

Kite can complete up to multiple lines of code at a time, reducing the time you spend writing repetitive, boilerplate Python:

Import statements are a breeze with Kite, and it learns and suggests your favorite aliases over time.

Kite continues to tap into Jupyter kernel completions, which are useful when you need to access attributes such as columns on a DataFrame. With Kite, these kernel completions now show up automatically as you type.

Save time searching for Python docs with Kite’s Copilot desktop app. View Python docs with just one click or mouse-hover, plus find helpful examples and how-tos.

More features you’ll get with Kite in JupyterLab:

Kite’s models work locally and independently of your Python kernel. This means if your kernel’s busy reading in data, you‘ll still get Kite completions while coding in other cells. Plus, no code is sent to a cloud server for processing.
Kite works in .py files within JupyterLab.

Kite for JupyterLab comes in a Free and a Pro version (free 30-day trial), and works 100% locally. You can download Kite here. Follow the instructions here to install the JupyterLab plugin.

Kite makes coding with Python faster and more enjoyable

We’ve trained Kite’s deep learning models on over 25 million open-source Python files to ensure Kite works with your favorite libraries. With how fast data science tooling evolves, it’s critical to stay on top of new modules and APIs, and Kite’s completions help make that easier to do.

Kite’s deep learning models have learned the most popular patterns used by data scientists, plus they understand the context of your code. This means Kite can predict relevant chunks of code and put them in your completions. This can be useful in three ways:

If you already know what you need to type, Kite helps you jump ahead to the next task.
If you’re having trouble remembering a calculation or code pattern, Kite can remind you so you don’t need to search on Google.
If you have never used a module or function before, you can get documentation faster with the Kite Copilot.

JupyterHub: Boost the whole team’s productivity with Kite

Does your team use JupyterHub? We offer additional features for JupyterHub teams:

Deploy Kite on your JupyterHub server to instantly bring AI-powered completions and one-click documentation to the whole team.
Add Kite’s largest ML models to a GPU-powered server for smarter, longer completions.
Custom-tailor Kite’s models to your team’s codebase and APIs.
Manage Kite licenses and billing through a unified system.

Email sales@kite.com to set up a demo and get extended free trials of Kite Pro for your whole team.

Thank you, Jupyter community!

This plugin would not be possible without the guidance of the Jupyter community and JupyterLab’s development team. In partnership with Quansight Labs, we worked with the JupyterLab development team to contribute to the Jupyter completions API and completions interface via four PRs and 87 commits (here are two examples). These improvements enable others in the Jupyter community to more easily build plugins for JupyterLab.

We will continue working with the community to help JupyterLab evolve and remain a powerful option for data scientists and developers.

We hope you’ll give our completions a spin in your next JupyterLab session by downloading Kite here and installing the JupyterLab plugin!

Happy Coding,

The Kite Team

P.S. If you are a university student you can get Kite Pro for free by signing up with your school email address (instructions). And we’ll be expanding student access to high schoolers and others soon, just in time for the fall semester.

Kite Launches AI-Powered JavaScript Completions

brendan — Tue, 12 May 2020 17:21:30 +0000

Today we’re excited to launch Kite’s AI-powered completions for JavaScript. Kite’s completions automate away the repetitive boilerplate of writing JavaScript code.

Let’s take a look…

Now let’s break it down…

Kite can complete up to multiple lines of code at a time, reducing the time you spend writing repetitive code.

Kite is able to provide completions when editors like VS Code cannot understand the code.

Kite shows completions in more situations, for example after a space.

Kite works alongside your editor’s completions. We use carefully-designed filters to reduce noise.

We’ve trained a new deep learning model on 22 million open-source JavaScript files to ensure Kite works with your favorite libraries and frameworks like React, Vue, Angular, and Node.js.

Kite for JavaScript is free and works 100% locally. You can download it here.

Kite makes coding faster and easier

The JavaScript ecosystem continually invents new frameworks and design patterns. These inventions make it a vibrant place to be, but it also creates the need to learn an ever-changing set of code patterns and APIs.

Kite’s deep learning models have learned all of these patterns, and understand the context of your code, so Kite can predict chunks of code and put them in your completions. This can be useful in two ways:

If you already know what you need to type, Kite helps you jump ahead to the next task.
If you’re having trouble remembering an API or design pattern, Kite can remind you so you don’t need to search on Google. As a result, writing JavaScript with Kite becomes faster and more fun.

We’re just getting started

Kite can help you ship software faster today.

And we’re just getting started. We believe that machine learning can automate away the tedious parts of writing code, and there’s so much more to do. We’re continually exploring ways ML can unlock productivity gains for developers, and we hope that you will join us.

A big thanks to the 250,000 developers who use Kite

We’re thrilled to share today that over 250,000 people are coding with Kite every month. We feel grateful to have reached this milestone, and we’d like to thank everyone who uses Kite. This amazing progress wouldn’t be possible without each of you giving us the encouragement and feedback that fuels the hard work — 30,000 code commits to date — that got us here.

We hope you’ll join us on this journey by downloading Kite!

Happy coding,

The Kite Team

P.S. We are also announcing Kite Pro today, which is our first paid product for Python professionals. Check out our separate blog post about Kite Pro for more info!

Building a Push-Up Bot in Discord!

Jillian Stenzel — Mon, 27 Apr 2020 14:26:43 +0000

With covid-19 getting us all out of shape, we decided a push-up bot was in order. In this 10 minute tutorial, we walk you through how to build a bot in discord.

Hope you all enjoy this one day build. Now drop and give me 20!

https://www.youtube.com/watch?v=etSRnPp9H8s&t=18s

Visualization of the biggest code bases (using blender3d and data from dataisbeautiful)

Jason G — Fri, 10 Apr 2020 23:15:28 +0000

Maybe a bit too far with the zooming, will be better next time!
Youtube - https://www.youtube.com/watch?v=afEg6VuAQJQ
Data - https://docs.google.com/spreadsheets/d/1s9u0uprmuJvwR2fkRqxJ4W5Wfomimmk9pwGTK4Dn_UI/edit#gid=5

Beautiful timeline animation of the biggest code bases in history

Jillian Stenzel — Thu, 09 Apr 2020 20:20:37 +0000

From 1970 to today, we made a visualization of the largest code bases in history- animated on a timeline, measured in lines of code.

May this serve as a little devinspo for you today!

https://www.youtube.com/watch?v=afEg6VuAQJQ&t=1s

Data structures can make or break your coding interview

Jillian Stenzel — Sat, 04 Apr 2020 16:18:07 +0000

Understanding data structures is a key skill for being a software engineer, so it makes sense that you can expect to use them in your technical interviews. Often times, using the correct data structure can make the difference in getting the job or not.

We made a video on some key data structures you need to know for your code interviews, including examples of when you should use each data structure and a links to them in Leetcode for practice.

This tutorial goes into...

Arrays
Linked lists
Stacks/queues
Hash tables
Heaps
Binary trees

Happy coding!

https://www.youtube.com/watch?v=K_ZuEy9_W_E

Quick guide to sorting algorithms + fastest comparison sort!

Jillian Stenzel — Fri, 27 Mar 2020 21:58:07 +0000

We made a video breaking down comparison sorts into three levels: N^2, N log N, & hybrid algorithms + a fun animation comparing the algorithms to see which one is the fastest!

After watching, you should have a better understanding of the different types of sorting algorithms as well as the benefits and drawbacks of each and when to use them. Enjoy! https://www.youtube.com/watch?v=qk7b4-iyCJ4

While you're add it, brush up on your interview skills with our top tips for technical interviews in Python: https://www.youtube.com/watch?v=Zee665ssm8Y&t=13s

Trying to make better quality Python videos and looking for feedback! Here's one of our first videos on Web Scraping and SQLite

Jason G — Fri, 21 Feb 2020 20:14:19 +0000

There's quite a lot of content on Youtube for Python and we've all likely skipped through many videos with bad audio or really small font trying to find an answer to a problem. We want to make better quality and more entertaining content for Python, so we'd really appreciate any feedback on the videos we've made so far, suggestions on what topics you would like us to cover or just your thoughts in general about developer content on Youtube.

Here are the links to our first couple videos.
Web Scraping - https://www.youtube.com/watch?v=ogPMCpcgb-E
SQLite - https://www.youtube.com/watch?v=girsuXz0yA8

Some Feedback so far

better audio levels
more depth into a topic

Announcing the Kite Integration for Spyder

brendan — Tue, 10 Dec 2019 00:26:12 +0000

We’re thrilled to announce Kite’s Python integration for Spyder, the popular open-source IDE for scientists, engineers, and data analysts. Kite is now available as a free Python code completions option for all Spyder users as part of the Spyder 4 release.

Spyder users can use Kite’s machine learning models to get more and better completions while coding with PyData libraries (e.g. NumPy, SciPy, Pandas, and Matplotlib). These libraries previously had minimal autocomplete support due to type analysis limitations of the completions engine Spyder used to exclusively rely on, called Jedi.

Kite worked alongside Spyder’s core contributors to incorporate several autocomplete improvements in the Spyder 4 release. This collaboration happened through Quansight’s Community Work Order program, a new framework for private companies to partner with open source projects.

TL;DR

Kite can be selected as a free code completions option in Spyder 4. Download and install Spyder 4 via Anaconda or via Spyder’s Github repo.
Spyder users will see autocomplete improvements when enabling Kite, including faster and more useful completions for numerics libraries. Watch an overview video about Kite for Spyder on our YouTube channel.
Kite’s Intelligent Snippets, Line-of-Code Completions, Dictionary Key Completions, and Copilot desktop app are all supported for Spyder users.
As with Kite’s other editor integrations, all code processing happens locally without leaving the user’s computer.
Kite and the Spyder core contributors worked together on this integration via Quansight’s Community Work Order program.
You'll need to have Kite installed on your computer to enable our Spyder integration. Download Kite here.

Getting more completions, more often for numerics libraries in Spyder

Spyder users have long reported issues with the completions for numerics libraries. This was largely due to type analysis limitations in the Jedi completions engine. It had difficulty analyzing third-party libraries for scientific computing (e.g. NumPy, SciPy, TensorFlow, etc.) that make use of compiled extensions. Without good type analysis, generating completions is inaccurate and resource intensive.

Enabling Kite in Spyder solves this problem, and goes beyond in a number of ways. Kite’s ML engine generates multi-token completions, and returns completions much faster.

To enable Kite in Spyder, go to Preferences/ Settings -> click on Completion and Linting -> click on the “Advanced” tab -> click “Enable Kite” under Providers -> click “Apply”.

Kite’s smarter completions features

Kite also features Line-of-Code Completions, Intelligent Snippets, and recently released, Dictionary Key Completions – all of which save you time and hassle while coding.

Line-of-Code Completions are multi-token completions for the top 100 Python libraries that are unique to Kite’s machine learning engine.

Intelligent Snippets allow you to complete complex, multi-token statements with ease by generating context-relevant code snippets as you type. Whereas editor snippets typically must be manually defined in advance, Kite’s Intelligent Snippets are generated in real-time based on the code patterns Kite finds in your codebase.

Dictionary Key Completions save you time by remembering the structure of dictionary objects in your codebase. These completions work for integer keys as well as string keys, the latter of which can be tedious to recall and prone to typos. Check out examples and a more detailed description of this new feature in our December release notes.

Improving completions performance in Spyder

Enabling Kite results in faster completions in Spyder for two main reasons. First, the Spyder and Kite teams worked together to make performance optimizations to reduce completions lag for Spyder 4. Second, Kite’s ML engine is continually optimized to show smarter completions, faster.

At Kite we invest in routine optimization of application and ML model performance to limit impact on users’ CPU and memory resources. For instance, Kite intelligently caches index builds based on whether your Python files have changed recently. It also only loads parts of its ML engine based on whether relevant libraries are used in users’ codebases. So if you’re only using numerics libraries, only relevant parts of the Kite engine will be running to conserve resources.

Did we mention one-click docs search?

Another nice perk of enabling Kite is access to Kite’s Copilot app and one-click docs search within Spyder. Check out the Kite Copilot here for more information.

Leveraging Quansight’s Community Work Order

Quansight’s Community Work Order (CWO) was an ideal framework to collaborate with the Spyder core contributors on improving the completions experience, a feature which had been the center of a lot of user feedback.

Given all of the resources we’ve invested into completions intelligence at Kite, we saw a natural fit to work with the Spyder team to improve Spyder’s completions infrastructure and get AI-powered completions in the hands of more developers for free.

However, structuring projects between open source communities and private companies is challenging across a number of dimensions: What work should be done? Who does the work? How long will it take? Where does the funding come from and what is it spent on? Quansight’s CWO provided transparency for how to fund and structure a partnership of this nature. Read more about the principles of CWOs on Quansight’s blog.

The CWO we sponsored provided funds for the development hours needed to improve the completions experience for Spyder 4. Additionally, Kite engineers and product managers contributed hours towards performance improvements and debugging efforts within the Spyder codebase.

We enjoyed contributing to the Spyder 4 release, and we look forward to collaborating on ongoing efforts with the Spyder community to support free completions!

Share the news on Twitter with @Spyder_IDE and @KiteHQ. Download Kite here.

[Video] Python Decorators in 15 Minutes

brendan — Thu, 31 Oct 2019 18:16:58 +0000

Decorators are an advanced feature of the Python language that allow you to modify the behavior of a function or method without touching its code. This video will teach you why decorators work and how to create your own. It will also walk through a few examples to give you a better idea of where to use Python decorators.

Want to try Kite? You can download it for free here.

Choosing an Autocomplete for Python

brendan — Wed, 30 Oct 2019 17:16:38 +0000

Using a code completion engine helps you remember syntax and write code faster. There are two main options around today for autocomplete in Python: Jedi and Kite. They both integrate with all the most popular IDEs for Python This video outlines why you should be using an autocomplete plugin while you code and how to choose the right completion engine for you.

Want to try Kite? You can download it for free here.

Django Database Migrations: A Comprehensive Overview

Aaron Harris — Wed, 23 Oct 2019 19:30:55 +0000

Django Database Migrations: A Comprehensive Overview
by Damian Hites

The Django web framework is designed to work with an SQL-based relational database backend, most commonly PostgreSQL or MySQL. If you’ve never worked directly with a relational database before, managing how your data is stored/accessed and keeping it consistent with your application code is an important skill to master.

You’ll need a contract between your database schema (how your data is laid out in your database) and your application code, so that when your application tries to access data, the data is where your application expects it to be. Django provides an abstraction for managing this contract in its ORM (Object-Relational Mapping).

Over your application’s lifetime, it’s very likely that your data needs will change. When this happens, your database schema will probably need to change as well. Effectively, your contract (in Django’s case, your Models) will need to change to reflect the new agreement, and before you can run the application, the database will need to be migrated to the new schema.

Django’s ORM comes with a system for managing these migrations to simplify the process of keeping your application code and your database schema in sync.

Django’s database migration solution

Django’s migration tool simplifies the manual nature of the migration process described above while taking care of tracking your migrations and the state of your database. Let’s take a look at the three-step migration process with Django’s migration tool.

1. Change the contract: Django’s ORM

In Django, the contract between your database schema and your application code is defined using the Django ORM. You define a data model using Django ORM’s models and your application code interfaces with that data model.

When you need to add data to the database or change the way the data is structured, you simply create a new model or modify an existing model in some way. Then you can make the required changes to your application code and update your unit tests, which should verify your new contract (if given enough testing coverage).

2. Plan for change: generate migrations

Django maintains the contract largely through its migration tool. Once you make changes to your models, Django has a simple command that will detect those changes and generate migration files for you.

3. Execute: apply migrations

Finally, Django has another simple command that will apply any unapplied migrations to the database. Run this command any time you are deploying your code to the production environment. Ideally, you’ll have deploy scripts that would run the migration command right before pushing your new code live.

Tracking changes with Django

Django takes care of tracking migrations for you. Each generated migration file has a unique name that serves as an identifier. When a migration is applied, Django maintains a database table for tracking applied migrations to make sure that only unapplied migrations are run.

The migration files that Django generates should be included in the same commit with their corresponding application code so that it’s never out-of-sync with your database schema.

Rolling back with Django

Django has the ability to rollback to a previous migration. The auto-generated operations feature built-in support for reversing an operation. In the case of a custom operation, it’s on you to make sure the operation can be reversed to ensure that this functionality is always available.

A simple Django database migrations example

Now that we have a basic understanding of how migrations are handled in Django, let’s look at a simple example of migrating an application from one state to the next. Let’s assume we have a Django project for our blog and we want to make some changes.

First, we want to allow for our posts to be edited before publishing to the blog. Second, we want to allow people to give feedback on each post, but we want to give them a curated list of options for that feedback. In anticipation of those options changing, we want to define them in our database rather than in the application code.

The initial Django application

For the purposes of demonstration, we’ll setup a very basic Django project called Foo:

django-admin startproject foo

Within that project, we’ll set up our blogging application. From inside the project’s base directory: ./manage.py startapp blog

INSTALLED_APPS = [
    ...
    'blog',
]

In blog/models.py we can define our initial data model:

class Post(models.Model):
    slug = models.SlugField(max_length=50, unique=True)
    title = models.CharField(max_length=50)
    body = models.TextField()

In our simple application, the only model we have represents a blog post. It has a slug for uniquely identifying the post, a title, and the body of the post.

Now that we have our initial data model defined, we can generate the migrations that will set up our database: ./manage.py makemigrations

Notice that the output of this command indicates that a new migration file was created at

blog/migrations/0001_initial.py containing a command to CreateModel name=‘Post’.

If we open the migration file, it will look something like this:

# Generated by Django 2.2 on 2019-04-21 18:04

from django.db import migrations, models

class Migration(migrations.Migration):
    initial = True

    dependencies = [
    ]

    operations = [
        migrations.CreateModel(
            name='Post',
            fields=[
                ('id', models.AutoField(
                    auto_created=True, 
                    primary_key=True, 
                    serialize=False, 
                    verbose_name='ID'
                )),
                ('slug', models.SlugField(unique=True)),
                ('title', models.CharField(max_length=50)),
                ('body', models.TextField()),
            ],
        ),
    ]

Most of the migration’s contents are pretty easy to make sense of. This initial migration was auto-generated, has no dependencies, and has a single operation: create the Post Model.

Now let’s set up an initial SQLite database with our data model:

./manage.py migrate

The default Django configuration uses SQLite3, so the above command generates a file called db.sqlite3 in your project’s root directory. Using the SQLite3 command line interface, you can inspect the contents of the database and of certain tables.

To enter the SQLite3 command line tool run:

sqlite3 db.sqlite3

Once in the tool, list all tables generated by your initial migration:

sqlite> .tables

Django comes with a number of initial models that will result in database tables, but the 2 that we care about right now are blog_post, the table corresponding to our Post Model, and django_migrations, the table Django uses to track migrations.

Still in the SQLite3 command line tool, you can print the contents of the django_migrations table:

sqlite> select * from django_migrations;

This will show all migrations that have run for your application. If you look through the list, you’ll find a record indicating that the 0001_initial migration was run for the blog application. This is how Django knows that your migration has been applied.

Changing the Django data model

Now that the initial application is setup, let’s make changes to the data model. First, we’ll add a field called published_on to our Post Model. This field will be nullable. When we want to publish something, we can simply indicate when it was published.

Our new Post Model will now be:

from django.db import models

class Post(models.Model):
    slug = models.SlugField(max_length=50, unique=True)
    title = models.CharField(max_length=50)
    body = models.TextField()
    published_on = models.DateTimeField(null=True, blank=True)

Next, we want to add support for accepting feedback on our posts. We want 2 models here: one for tracking the options we display to people, and one for tracking the actual responses

from django.conf import settings
from django.db import models

class FeedbackOption(models.Model):
    slug = models.SlugField(max_length=50, unique=True)
    option = models.CharField(max_length=50)

class PostFeedback(models.Model):
    user = models.ForeignKey(
        settings.AUTH_USER_MODEL, related_name='feedback',
        on_delete=models.CASCADE
    )
    post = models.ForeignKey(
        'Post', related_name='feedback', on_delete=models.CASCADE
    )
    option = models.ForeignKey(
        'FeedbackOption', related_name='feedback', on_delete=models.CASCADE
    )

Generate the Django database migration

With our model changes done, let’s generate our new migrations:

./manage.py makemigrations

Notice that this time, the output indicates a new migration file, blog/migrations/0002_auto_<YYYYMMDD>_<...>.py, with the following changes:

Create model FeedbackOption
Add field published_on to Post
Create model PostFeedback

These are the three changes that we introduced to our data model.

Now, if we go ahead and open the generated file, it will look something like this:

# Generated by Django 2.2 on 2019-04-21 19:31

from django.conf import settings
from django.db import migrations, models
import django.db.models.deletion

class Migration(migrations.Migration):

    dependencies = [
        migrations.swappable_dependency(settings.AUTH_USER_MODEL),
        ('blog', '0001_initial'),
    ]

    operations = [
        migrations.CreateModel(
            name='FeedbackOption',
            fields=[
                ('id', models.AutoField(
                    auto_created=True,
                    primary_key=True,
                    serialize=False, verbose_name='ID'
                )),
                ('slug', models.SlugField(unique=True)),
                ('option', models.CharField(max_length=50)),
            ],
        ),
        migrations.AddField(
            model_name='post',
            name='published_on',
            field=models.DateTimeField(blank=True, null=True),
        ),
        migrations.CreateModel(
            name='PostFeedback',
            fields=[
                ('id', models.AutoField(
                    auto_created=True,
                    primary_key=True,
                    serialize=False,
                    verbose_name='ID'
                )),
                ('option', models.ForeignKey(
                    on_delete=django.db.models.deletion.CASCADE,
                    related_name='feedback',
                    to='blog.FeedbackOption'
                )),
                ('post', models.ForeignKey(
                    on_delete=django.db.models.deletion.CASCADE,
                    related_name='feedback',
                    to='blog.Post'
                )),
                ('user', models.ForeignKey(
                    on_delete=django.db.models.deletion.CASCADE,
                    related_name='feedback',
                    to=settings.AUTH_USER_MODEL
                )),
            ],
        ),
    ]

Similar to our first migration file, each operation maps to changes that we made to the data model. The main differences to note are the dependencies. Django has detected that our change relies on the first migration in the blog application and, since we depend on the auth user model, that is marked as a dependency as well.

Applying the Django database migration

Now that we have our migrations generated, we can apply the migrations:

./manage.py migrate

The output tells us that the latest generated migration is applied. If we inspect our modified SQLite database, we’ll see that our new migration file should be in the django_migrations table, the new tables should be present, and our new field on the Post Model should be reflected in the blog_post table.

Now, if we were to deploy our changes to production, the application code and database would be updated, and we would be running the new version of our application.

Bonus: data migrations

In this particular example, the blog_feedbackoption table (generated by our migration) will be empty when we push our code change. If our interface has been updated to surface these options, there is a chance that we forget to populate these when we push. Even if we don’t forget, we have the same problem as before: new objects are created in the database while the new application code is deploying, so there is very little time for the interface to show a blank list of options.

To help in scenarios where the required data is somewhat tied to the application code or to changes in the data model, Django provides utility for making data migrations. These are migration operations that simply change the data in the database rather than the table structure.

Let’s say we want to have the following feedback options: Interesting, Mildly Interesting, Not Interesting and Boring. We could put our data migration in the same migration file that we generated previously, but let’s create another migration file specifically for this data migration...

... check out the code on Kite's blog! Continue with "Bonus: Data Migrations"

Damian Hites is the CTO of Sylo, which is looking to improve Social Media Marketing by offering 3rd party trusted measurement. He has 10+ years of experience writing software and leading teams.