DEV Community: Pika

Introducing: Pika CDN + Deno

Fred K. Schott — Wed, 05 Feb 2020 23:37:00 +0000

Deno isn't just for TypeScript, anymore! Did you know that you can also use Deno to import and run JavaScript packages from npm?

As long as your package is written as ESM (JavaScript's native module syntax) and doesn't rely on any Node-specific behavior/imports, Deno can load and run JavaScript from any URL without breaking a sweat. Pika already hosts a search catalog of 80,000+ ESM packages on npm with more being added every day.

All of this makes the Pika CDN -- our modern CDN for the npm ecosystem -- a great fit for Deno. With the Pika CDN, you can import any npm package from a single, hosted URL. This has always been a good fit for websites, but now it is an even better fit for Deno. That's because Deno has no package manager, so importing packages from URLs is already built in to the Deno workflow.

Introducing "X-TypeScript-Types"

But what about type declarations? Deno is a TypeScript runtime, after all. It would be a shame if you couldn't also fetch the type declarations that exist inside of each npm package (or via the community-generated @types/ project by DefinitelyTyped) whenever you import a JS npm package.

Well, now you can!

In collaboration with the Deno team for their recent v0.32.0 release, Pika CDN now hosts type declarations for every package that provides them! Import an npm package from our CDN and Deno will automatically fetch its type declarations along with it.

With this new feature, your programs get an extra level of stability with automatic warnings from Deno when a package is used incorrectly.

This is all thanks to Deno's new "X-TypeScript-Types" response header support. Here's how it works:

Hosting the type declarations: For Deno to read package type declarations, they have to exist somewhere on our CDN. We do the work at build-time to find the type files for your package and then host them at the /types/* package URL subpath.
Telling Deno where to find them: Each package on the CDN that has valid types responds with a X-TypeScript-Types header. When Deno sees this header from an import, it automatically fetches the types and connects them to the import.

If no type declarations exist on npm, we don't send the header. In that case, Deno will read the JS files and make its best guess at what its interface looks like. We could try to do the same on the CDN, but for now we feel pretty confident that Deno is better at this than we are :)

Exciting Times!

None of this would have been possible without the efforts of @kitsonk and the entire team of Deno contributors.

With these changes, Pika CDN is now THE BEST PLACE to get npm packages that run on Deno. This is something that we're extremely proud of, and a goal that we'll keep improving on as Deno works towards their v1.0 release. Exciting times, indeed!

We can't wait to see what you build with Deno and our CDN. Reach out to us on Twitter with any questions, comments, and links to what you've built.

✌️
- @fks

Tree-Shaking Without a Bundler? Here's How Snowpack Does It!

Fred K. Schott — Thu, 16 Jan 2020 00:10:20 +0000

Until Snowpack came on the scene, application bundling was the only way to do tree-shaking (dead-code elimination) on the web.

But Snowpack is the unbundler. It runs only on your dependencies without ever touching your application code. So how could it tree-shake?

Here's a quick post about how we made this production optimization happen for faster production apps, thanks to some essential help from the RollupJS team 🙌

The Setup

Bundlers "tree-shake" your web application by finding and removing unused code, shaving off valuable bytes when you deploy to production.

Since Snowpack never touches your application code, how could we do the same? We had to get creative.

Snowpack v1.0 added an initial "scan" phase behind the --optimize flag. It runs a quick scan of your app to learn how you use each dependency.

... and that's it! We pass that info to our internal installer (powered by Rollup) and eliminate any dead code at install time.

The Challenge

"Passing that info to our Rollup-powered installer" was an interesting challenge: How do you tell Rollup that code in your top-level entrypoints is unused?

This was an unusual request, and something the Rollup team (rightfully!) didn't want to build into the bundler itself.

We created an issue on the Rollup repo to outline the problem, and with @lukastaegert's help came up with a really clever solution via a new plugin...

Here's how we solved it: We create an in-memory "envelope" wrapper for Rollup of every dep & fill it with the exact imports used by your app.

Rollup sees the envelope, reads the scanned imports to the real dep, & then automatically tree-shakes out everything not found in the ✉️
You can see exactly how it works here, coming in at only ~50 LOC.

This all runs behind the production-only --optimize flag, so you can add & remove imports as you develop your app without worrying about tree-shaking or needing to re-install as your code changes.

But even when optimizing for production, just reading & scanning your application code should be a hell of a lot faster than reading, scanning AND BUNDLING your app code with a traditional bundler (so you can still expect fast production deployments).

What's Next?

We're experimenting with replacing our current import scanner with the even faster es-module-lexer by @guybedford, a WASM-powered scanner written in C.

guybedford / es-module-lexer

Low-overhead lexer dedicated to ES module parsing for fast analysis

ES Module Lexer

A JS module syntax lexer used in es-module-shims.

Outputs the list of exports and locations of import specifiers, including dynamic import and import meta handling.

Supports new syntax features including import attributes and source phase imports.

A very small single JS file (4KiB gzipped) that includes inlined Web Assembly for very fast source analysis of ECMAScript module syntax only.

For an example of the performance, Angular 1 (720KiB) is fully parsed in 5ms, in comparison to the fastest JS parser, Acorn which takes over 100ms.

Comprehensively handles the JS language grammar while remaining small and fast. - ~10ms per MB of JS cold and ~5ms per MB of JS warm, see benchmarks for more info.

Built with Chomp

Usage

npm install es-module-lexer

See types/lexer.d.ts for the type definitions.

For use in CommonJS:

const { init, parse } = require('es-module-lexer');
(async

…

View on GitHub

Thanks for Reading!

Enjoyed this post? Throw Snowpack a 🌟on GH to help spread the word:

FredKSchott / snowpack

ESM-powered frontend build tool. Instant, lightweight, unbundled development. ✌️

Update (April 20, 2022): Snowpack is no longer actively maintained and is not recommended for new projects.

Check out Vite for a well-maintained Snowpack alternative.
See also: esbuild, parcel

Snowpack

Snowpack is a lightning-fast frontend build tool, designed to leverage JavaScript's native module system (known as ESM). It is an alternative to heavier, more complex bundlers like webpack or Parcel in your development workflow.

Key Features

Develop faster, with a dev server that starts up in 50ms or less.
See changes reflected instantly in the browser.
Integrate your favorite bundler for a production-optimized build.
Enjoy out-of-the-box support for TypeScript, JSX, CSS Modules and more.
Connect your favorite tools with third-party plugins.

💁 More info at the official Snowpack website ➞

Contributor Guidelines: CONTRIBUTING.md
License: MIT

View on GitHub

Learn more about Snowpack: https://www.snowpack.dev/
Follow us on Twitter for updates: https://twitter.com/pikapkg
Read & share the original Twitter thread that this was based on: https://twitter.com/pikapkg/status/1217509391668133888

Unrolled: Introducing the Pika REPL

Fred K. Schott — Tue, 06 Aug 2019 16:24:51 +0000

Today we launched the new rewrite of pika.dev, which included a new Package REPL for the package view page. This was just announced on Twitter, but I figured I'd unroll and repost the entire thread here quickly for anyone interested.

You can read our entire release thread including all of todays releases here: https://twitter.com/pikapkg/status/1158765492191911936

What's a Package REPL? Keep reading to find out...

Pika 📦

@pikapkg

✨ INTRODUCING: The Pika Package REPL

Test packages directly in the browser, without installing anything locally. Perfect for quickly evaluating packages before bringing them into your important projects.

Try it today: pika.dev/packages/path-…

16:05 PM - 06 Aug 2019

3 7

Pika 📦

@pikapkg

True story: The second we had this feature working, we ended up needing it to quickly test our own site's path parsing logic.

Instead of debugging the entire app, we could quickly iterate on just the code snippet... RIGHT IN THE BROWSER🤘

16:05 PM - 06 Aug 2019

0 0

Pika 📦

@pikapkg

This is powered internally by ristretto: a really neat, modular testing library from the Polymer team.

github.com/PolymerLabs/ri…

16:05 PM - 06 Aug 2019

0 1

PolymerLabs / ristretto

An extensible test runner ☕️

🚨 PROJECT STATUS: EXPERIMENTAL 🚨This product is in the Experimentation phase. Someone on the team thinks it’s an idea worth exploring, but it may not go any further than this. Use at your own risk.

Ristretto

Ristretto is an extensible test runner.

There are several established test runners in the JavaScript ecosystem (Mocha, Jasmine and Jest to name a few of the more prominent ones). Ristretto was created to address a feature sweet spot left unaddressed by incumbent projects.

Ristretto has the following qualities:

Simple, concise, class-based factorization
Consumable as modules with browser-compatible path specifiers OOB
Batteries included for the most popular testing syntaxes (BDD/TDD)
Designed for extensibility (mixins, specialized reporters and a spec-as-data-structure philosophy)
Ships with mixins that enhance specs with powerful features (e.g., fixtures and test isolation)
Authored in TypeScript
No build tooling required to use
"JavaScript is the native language, The Web is the…

View on GitHub

Pika 📦

@pikapkg

Your code snippet is run as a fake ristretto "test", in a sandboxed iframe, directly in the browser. Open up dev tools and you'll see any console output!

16:05 PM - 06 Aug 2019

0 1

Pika 📦

@pikapkg

This is just the start, and we'll keep adding to this over the next few months! Two additions I can't wait for:
1⃣ Package type definitions & type checking
2⃣ Logging output/errors to the page if you're on mobile

16:15 PM - 06 Aug 2019

0 0

BONUS: Curious about how a package works after using it in the REPL?

Pika 📦

@pikapkg

The SOURCE tab is another incredibly useful, but subtle addition: An @unpkg browser built right in to the page.

View any package source without leaving Pika, or click the link at the top to go directly to unpkg.com.

16:36 PM - 06 Aug 2019

0 0

A Future Without Webpack

Fred K. Schott — Fri, 02 Aug 2019 16:04:37 +0000

Note: This piece was originally published to pika.dev.

The year is 1941. Your name is Richard Hubbell. You work at an experimental New York television studio owned by CBS. You are about to give one of the world's first major TV news broadcasts, with 15 minutes of airtime to fill. What do you do?

In a world that has only known radio, you stick to what you know. That is, you read the news. "Most of the [televised] newscasts featured Hubbell reading a script with only occasional cutaways to a map or still photograph."^[1] It would be a while before anyone would show actual video clips on the TV news.

As a JavaScript developer in 2019, I can relate. We have this new JavaScript module system (ESM) that runs natively on the web. Yet we continue to use bundlers for every single thing that we build. Why?

Over the last several years, JavaScript bundling has morphed from a production-only optimization into a required build step for most web applications. Whether you love this or hate it, it's hard to deny that bundlers have added a ton of new complexity to web development -- a field of development that has always taken pride in its view-source, easy-to-get-started ethos.

@pika/web is an attempt to free web development from the bundler requirement. In 2019, you should use a bundler because you want to, not because you need to.

Credit: @stylishandy

Why We Bundle

JavaScript bundling is a modern take on an old concept. Back in the day (lol ~6 years ago) it was common to minify and concatenate JavaScript files together in production. This would speed up your site and get around HTTP/1.1's 2+ parallel request bottleneck.

How did this nice-to-have optimization become an absolute dev requirement? Well, that's the craziest part: Most web developers never specifically asked for bundling. Instead, we got bundling as a side-effect of something else, something that we wanted realllllllly badly: npm.

npm -- which at the time stood for "Node.js Package Manager" -- was on its way to becoming the largest code registry ever created. Frontend developers wanted in on the action. The only problem was that its Node.js-flavored module system (Common.js or CJS) wouldn't run on the web without bundling. So Browserify, Webpack, and the modern web bundler were all born.

Create React App visualized: 1,300 dependencies to run "Hello World"

Complexity Stockholm Syndrome

Today, it's nearly impossible to build for the web without using a bundler like Webpack. Hopefully, you use something like Create React App (CRA) to get started quickly, but even this will install a complex, 200.9MB node_modules/ directory of 1,300+ different dependencies just to run "Hello World!"

Like Richard Hubbell, we are all so steeped in this world of bundlers that it's easy to miss how things could be different. We have these great, modern ESM dependencies now (almost 50,000 on npm!). What's stopping us from running them directly on the web?

Well, a few things. 😕 It's easy enough to write web-native ESM code yourself, and it is true that some npm packages without dependencies can run directly on the web. Unfortunately, most will still fail to run. This can be due to either legacy dependencies of the package itself or the special way in which npm packages import dependencies by name.

This is why @pika/web was created.

@pika/web: Web Apps Without the Bundler

FredKSchott / snowpack

ESM-powered frontend build tool. Instant, lightweight, unbundled development. ✌️

Update (April 20, 2022): Snowpack is no longer actively maintained and is not recommended for new projects.

Check out Vite for a well-maintained Snowpack alternative.
See also: esbuild, parcel

Snowpack

Key Features

Develop faster, with a dev server that starts up in 50ms or less.
See changes reflected instantly in the browser.
Integrate your favorite bundler for a production-optimized build.
Enjoy out-of-the-box support for TypeScript, JSX, CSS Modules and more.
Connect your favorite tools with third-party plugins.

💁 More info at the official Snowpack website ➞

Contributor Guidelines: CONTRIBUTING.md
License: MIT

View on GitHub

@pika/web installs modern npm dependencies in a way that lets them run natively in the browser, even if they have dependencies themselves. That's it. It's not a build tool and it's not a bundler (in the traditional sense, anyway). @pika/web is a dependency install-time tool that lets you dramatically reduce the need for other tooling and even skip Webpack or Parcel entirely.

npm install && npx @pika/web
✔ @pika/web installed web-native dependencies. [0.41s]

@pika/web checks your package.json manifest for any "dependencies" that export a valid ESM "module" entry point, and then installs them to a local web_modules/ directory. @pika/web works on any ESM package, even ones with ESM & Common.js internal dependencies.

Installed packages run in the browser because @pika/web bundles each package into a single, web-ready ESM .js file. For example: The entire "preact" package is installed to web_modules/preact.js. This takes care of anything bad that the package may be doing internally, while preserving the original package interface.

"Ah ha!" you might say. "That just hides bundling in a different place!"

Exactly! @pika/web leverages bundling internally to output web-native npm dependencies, which was the main reason that many of us started using bundlers in the first place!

With @pika/web all the complexity of the bundler is internalized in a single install-time tool. You never need to touch another line of bundler configuration if you don't want to. But of course, you can continue to use whatever other tools you like: Beef up your dev experience (Babel, TypeScript) or optimize how you ship in production (Webpack, Rollup).

This is the entire point of @pika/web: Bundle because you want to, not because you need to.

"view-source" is back!

Performance

Installing each dependency this way (as a single JS file) gets you one big performance boost over most bundler setups: dependency caching. When you bundle all of your dependencies together into a single large vendor.js file, updating one dependency can force your users to re-download the entire bundle. Instead, with @pika/web, updating a single package won't bust the rest of the user's cache.

@pika/web saves you from this entire class of performance footguns introduced by bundlers. Duplicated code across bundles, slow first page load due to unused/unrelated code, gotchas and bugs across upgrades to Webpack's ecosystem... Entire articles and tools are devoted to solving these issues.

To be clear, leaving your application source unbundled isn't all sunshine and roses, either. Large JavaScript files do compress better over the wire than smaller, more granular files. And while multiple smaller files load just as well over HTTP/2, the browser loses time parsing before then making follow-up requests for imports.

It all comes down to a tradeoff between performance, caching efficiency, and how much complexity you feel comfortable with. And again, this is the entire point of @pika/web: Add a bundler because it makes sense to your situation, not because you have no other choice.

The Pika Web App Strategy

@pika/web has completely changed our approach to web development. Here is the process we used to build pika.dev, and how we recommend you build your next web application in 2019:

For new projects, skip the bundler. Write you application using modern ESM syntax and use @pika/web to install npm dependencies that runs natively on the web. No tooling required.
Add tooling as you go. Add TypeScript if you want a type system, add Babel if you want to use experimental JavaScript features, and add Terser if you want JS minification. After 6+ months, pika.dev is still happily at this phase.
When you feel the need & have the time, experiment by adding a simple bundler for your application source code. Performance test it. Is it faster on first page load? Second page load? If so, ship it!
Keep optimizing your bundler config as your application grows.
When you have enough money, hire a Webpack expert. Congratulations! If you have the resources to hire a Webpack expert you have officially made it.

@pika/pack - Publish great npm packages.

Fred K. Schott — Fri, 02 Aug 2019 16:03:59 +0000

Note: This piece was originally published to pika.dev.

If you've recently published a package to npm, you know how much work goes into a modern build process. Transpile JavaScript, compile TypeScript, convert ES Module syntax (ESM) to Common.js, configure your package.json manifest... and that's just the basics.

What about web browsers and bundlers? Is your package optimized for them? Can users load your package from UNPKG? Does it include type definitions for VSCode & TypeScript users? Has it been stripped of all unnecessary files to be as small as possible? Your users care about these optimizations, but they all require even more knowledge, configuration, tooling, time and effort to get right.

@pika/pack builds amazing packages without the hassle:

Simple ⚡️ Use pre-configured plugins to build your entire package.
Flexible 🏋️‍♀️ Choose plugins and optimizations that match your needs.
Holistic ⚛️ Let us handle your code, assets, and package.json config.

The Package Build Pipeline

@pika/pack connects pre-configured plugins to build and optimize your package for you. Plugins wrap already-popular tools like Babel and Rollup with options already optimized for npm. This lets @pika/pack build your package without much (if any) configuration required on your part.

@pika/pack even creates and configures a package.json manifest for your package, automatically.

This works because @pika/pack builds your entire package: code, assets, and even package.json manifest. By building the entire package, you end up with a fully-built pkg/ directory, ready to publish. Entry points like "main", "module", "umd:main", "types", "unpkg", and even advanced options like "sideEffects" and "files" are all handled by your build plugins.

Getting started is easy:

// 1. Install @pika/pack!
$ npm install -g @pika/pack
// 2. Add this to your package.json manifest:
"@pika/pack": {"pipeline": []},
// 3. Run @pika/pack!
$ pack build

😎 🆒

So now what? If you run pack build with an empty pipeline, you'll get an empty package. That's not very useful.

To get you started, here are our five favorite things to do with @pika/pack:

1. Publish Modern, ES2018 JavaScript By Default (1 Line)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg", {"exclude": ["__tests__/**/*"]}]
    ]
}

The first plugin that most packages will use is @pika/plugin-standard-pkg. This is our standard "source" builder that builds any JavaScript & TypeScript source code to the latest ES2018 language spec. TypeScript is supported by default, and any custom Babel plugins in your local .babelrc will be used automatically (the plugin is powered by Babel internally).

This gives the rest of our pipeline a standard ES2018 target to build from. BONUS: some of your users may support modern JavaScript, and can leverage this ES2018 distribution directly for a smaller, faster, less-bloated package.

2. Publish Node.js & Web-Optimized Builds (1 Line Each)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg"],
        ["@pika/plugin-build-node"],
        ["@pika/plugin-build-web"]
    ]
}

After creating your modern ES2018 build, it becomes trivial to add Node.js- and web-optimized distributions to your package. @pika/plugin-build-node will transpile a Node.js-ready distribution to run on any supported Node.js version. @pika/plugin-build-web will build a more modern, ES module (ESM) distribution: optimized for bundlers and compiled to run on all browsers that support ESM syntax natively.

Both plugins use Rollup internally, but you don't have to configure any bundler logic yourself. With just three lines of JSON you get a package that is fully optimized for the two most popular JS platforms. And because @pika/pack builds your entire package, your package.json manifest is automatically configured for you with "main", "module", and "esnext" entry points.

3. Automatically Generate TypeScript Definitions (1 Line)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg"],
        ["@pika/plugin-build-node"],
        ["@pika/plugin-build-web"],
        ["@pika/plugin-build-types"]
    ]
},

With @pika/pack, every package automatically gets .d.ts TypeScript definition files thanks to the @pika/plugin-build-types plugin. Even if you're writing JavaScript, this plugin will use TypeScript to automatically generate type definitions from your JavaScript & JSDoc!

4. Publish WASM! Or Reason, or Rust, or C++, or... (1-2 Lines)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-wasm-assemblyscript"],
        ["@pika/plugin-wasm-bindings"]
    ]
},

The pipeline concept is simple AND flexible. So flexible that your package source doesn't even need to be JavaScript:

@pika/plugin-source-bucklescript - Compile Reason or OCaml to ES2018 JavaScript via BuckleScript
@pika/plugin-wasm-assemblyscript - Compile TypeScript to WASM via AssemblyScript
@pika/plugin-wasm-emscripten - Compile C/C++ to WASM via Emscripten (Coming Soon!).
@pika/plugin-wasm-bindings - Add simple JavaScript bindings for any generated WASM.

5. Automatically Enhance Your Package (1 Line)

"@pika/pack": {
    "pipeline": [
        ["@pika/plugin-standard-pkg"],
        ["@pika/plugin-build-node"],
        ["@pika/plugin-simple-bin", {"bin": "my-cli"}]
    ]
}

Build plugins can also enhance existing builds in some really exciting ways. Our favorite enhancement right now is @pika/plugin-simple-bin, which injects a simple CLI wrapper into any package and configures your package.json to point to it automatically.

We use @pika/pack to build @pika/pack, and we use this plugin specifically to add a command-line interface with none of the package.json configuration and setup hassle.

We can't wait to see what's possible with this wrapping concept, especially for CLIs: Automatically restart your program on failure, check for package updates after running, log usage statistics... the possibilities are endless!

Publishing Your Package

Publishing your package should be as easy as building it. So we brought our favorite parts of np (a self-described "better npm publish") into @pika/pack. With the publish command there's no need to worry about how to publish your built package. Just run pack publish in your top-level project and @pika/pack will walk you through cutting a new version and publishing your package.

It even includes a handy unpkg.com at the end so that you can immediately view your new package.

Try @pika/pack Today!

npm install -g @pika/pack

Don't wait, try @pika/pack in your next package (or any old ones that could a 2019 upgrade). And when you do, please let us know how it went!

Pika is a project to move the JavaScript ecosystem forward. Pika's mission is to make modern JavaScript more approachable by making it easier to find, publish, install, and use modern packages on npm. Hopefully @pika/pack moves us one step closer towards that goal.

Projects Already Using @pika/pack

react-async - Flexible promise-based React data loader.
hookuspocus - hooks for all the functions!
foundationjs - Full-featured NodeJS framework.
graphql-rest-proxy - Turn your REST API into GraphQL.
custom-attributes - Define custom attributes the same way you define custom elements.
@pika/pack & @pika/web - yes, we even use @pika/pack to build @pika/pack! 🤯
And hundreds more! Check out our full set of example & starter projects →
Have you used @pika/pack in a package? Let us know and we'll add your package to the list!