DEV Community: Emma Humphries

React is a subsidy

Emma Humphries — Sat, 12 Sep 2020 00:00:00 +0000

After a week of job interviews over video, while the sky was the color of Landry Violence, I decided to watch Stuart Langridge’s GOTO; 2020 talk (YouTube) on JavaScript.

Thirty seconds in, Langridge relates Zack Leatherman’s example of 8.5MB of tweets in static HTML rendering 1/5 of a second faster than a React site rendering a single tweet (Hellsite).

Reader, I howled, despite that being a bad idea after a week of dangerous air quality and six hours a day on video calls. Then I summoned The Infinite Scream (Hellsite) to do the howling for me while I wrote a blog post.

I’ve been thinking of the costs of the Javascript-first, particularly the React-first, state of web development:

Users have to buy and use high-end devices (phones, tablets, and laptops) to access content
Developers abandon the web for native applications
- Which in turn demand rents (transaction fees)
- And concessions (non-political content, what content can be sold)
Orgs sticking with the with web use tool chains with high overhead and requirements (every dev needs a high end laptop and training in the React tool chain)
Development jobs go to people who have the time and skills to use React and native frameworks instead of the open web
JavaScript-first and native apps encourage privacy intrusive practices that siphon behavioral data, and reward getting a user “Hooked”

JavaScript in general, and React in particular, is a tax on the Open Web which subsidizes:

Device manufacturers
App stores
Surveillance capitalism
Elite developers

at the expense of:

Users stuck on a device upgrade treadmill
Projects which don’t fit the JavaScript-first economic model
- Especially anti-racist, anti-policing, and anti-colonialist projects
Creators who have to work under the precarity of large Social Media platforms
Developers without access to tools and training for elite jobs

This subsidy will continue to harm all of us who were told that the Web was a boon for everyone.

Rediscovering APAs

Emma Humphries — Fri, 11 Sep 2020 00:00:00 +0000

Matt Webb describes a fever dream of an idea for a website where a group of people agree to a publishing frequency, start ‘zines, and at the deadline the ‘zines are compiled and distributed to the membership.

Those are APAs (amateur press associations) except instead of the web, members were cutting and pasting zine’s and making copies at Kinko’s (remember when Kinko’s wasn’t FedEx?) to mail to or drop off at the Organizing Editor’s (OE,) who would host a compilation party where they would assemble everyone’s APA zine’s into a issue. You made as many copies of your zine as members in good standing, and a couple more for people who were on-spec.

Then the OE would mail copies to the out of town members (you contributed to a mailing fund for this) and you’d pick up your issue to take home and make notes in the margin for comments on everyone else’s zine.

And yes, APAs influenced Live Journal (and later Dreamwidth) culture.

I was a member of the Madison SciFi fan APA, The Turbo Charged Party Animal for close to 10 years. Jeanne Gomoll’s design portfolio has examples of her and her partner’s ‘zines for the APA over the years. My favorite memory was a multiple issue narrative arc about Jeanne’s Diet Coke stash. Her friends may have bought her a room full of cases of Diet Coke for her birthday one year.

Who wants to make an APA?

The virtue of boring

Emma Humphries — Tue, 19 Nov 2019 00:00:00 +0000

Yasser Booley CC BY-SA 3.0, via Wikimedia Commons

At some point I’ll finish that essay on how toxic masculinity derailed the web, but Robin Rendle’s essay on restraint in development made a few things clear.

Rendle mentions diagnosing a performance problem that’s harming your users who can’t load your site to know if they will be affected by power cuts in part because of all the third party scripts it loads:

But fixing that problem? It requires going through each script, talking to the marketing department, finding out who owns what script, why they use it, what data is ultimately useful to the organization and what is not. Then, finally, you can delete the script. The solution to the problem is boring as dirt and trying to explain why the work is important—even vital—will get you nowhere in many organizations.

This is not exciting work. It’s not work that gets solved with a single heroic pull request, replacing the site’s templating with a new JavaScript library, or a whole new feature.

Over the past few months we’ve been cleaning up old metadata in Bugzilla. It takes video calls, emails, and working shared documents with long runs of comments to get to a consensus to turn off a field. But all those fields removed are that much more cognitive overhead removed from people reporting, triaging, planning, fixing, and verifying bugs.

It’s sorting the pantry so the cooking spices you use every day are always to-hand when you need them. Searching the pantry or the show bug page is gets in the way of getting work done. New cabinets and new UX might be engaging, but it’s expensive and not incremental.

Extracting a list from a webpage

Emma Humphries — Tue, 10 Sep 2019 00:00:00 +0000

The problem

You have a webpage with a list of things: values, prices, emails, or links. And you want to copy that into a string you can use elsewhere like a spreadsheet or data.

There’s not an API you can use to fetch these. You know that you can construct a CSS3 selector to get them all. So you can use the developer view of the page (a.k.a. F12) and use JavaScript on the console tab as your ‘API’.

Extracting the list

You look at the page in your browser’s inspector and the email addresses you want to pull out are coded as:

<table>
<tr>
    …
    <td><a class="email" href="mailto:a@b.tld">a@b.tld</a></td>
    …
</tr>
…
    <td><a class="email" href="mailto:e@m.tld">e@m.tld</a></td>
…
</table>

CSS3 selector is 'a.email'. That is you want to pull every A element with the class name email out of the current page. And each of those A elements has an href of the form mailto:name@example.tld.

So we’ll get the list and iterate over it, chopping up the href values and turning it into a list.

We open the JavaScript console on the page and run this one-liner.

$('a.email') // <= $() is console shorthand for document.getElementsBySelector()
.map((el) => { return el.href.split(':')[1]; })
.join('\n');

But the browser reports an error here, because $('a.mail') is a node list, not an array.

You can use Array.prototype.from() to make that node list into an array.

Array.from($('a.email'))
.map((el) => {
    return el.href.split(':')[1];
})
.join('\n')

Now you’ll get a list of email addresses, unsorted, and with duplicates.

e@m.tld
a@b.tld
c@d.tld
a@b.tld

You could clean that up in a text editor but let’s go further.

Cleaning the list

Sorting is simple.

Array.from($('a.email'))
.map((el) => {
    return el.href.split(':')[1];
})
.sort()
.join('\n')

That doesn’t get rid of the duplicates.

JavaScript supplies the filter method, but to use it, we’d have to define an accumlator on a separate line, so we don’t get a nice, context-minimal one-liner.

ES6 provides a new object, Set. Sets don’t allow duplicate values. And it takes any iterable type as an input.

new Set([1, 1, 2, 2, 3]) // => Set(3) [1, 2, 3]
new Set('committee') // => Set(6) ["c", "o", "m", "i", "t", "e"]

So we can de-dupe the list using that, and turn it back into an array to sort and join it into a string.

But what does Set use to de-dupe?

It turns out that new Set(*node list*) is an empty set. This is because of how the comparison operator works when creating the set from an iterator.

So you have to process the the list into an array of strings before you turn it into a set.

Array.from(new Set(Array.from($('a.email'))
.map((el) => {
    return el.href.split(':')[1];
})));

Then you can sort the array of unique text values, then join it into a string.

The complete one-liner, formatted for legibility, is:

Array.from(new Set(Array.from($('a.email'))
.map((el) => {
    return el.href.split(':')[1];
})))
.sort()
.join('\n');

Which will return:

a@b.tld
c@d.tld
e@m.tld

Using SSH tunnels to deploy your site

Emma Humphries — Tue, 01 Jan 2019 00:00:00 +0000

Note: Updated on 2019-09-09

Running a local service on the open Internet is still a thing, even in the age of the Cloud.

For instance, I have a local set of dashboards using Corsica that I’d like to have access to from the road. You may want to show a client work in progress without having to bother with deploying to an external host. You may want to connect to a security camera you run attached to a Raspberry Pi. Or run a web application on a non-privileged port.

For a long time, the way to do that would had been to set up port-forwarding on your router and use a personal DNS service. Port-forwarding requires router configuration, and limits you to exposing one service to the web, and your ISP most likely blocks ports 80 and 443.

With an SSH tunnel, you can expose your service to the internet without the hassle of router configuration:

ssh -R 80:localhost:8080 external.example.com

Your service at localhost:8080 would be on the web at http://external.example.com.

The host external.example.com needs to be reachable from the public internet. You could host your endpoint yourself in the Cloud, but endpoints for SSH tunnels are available as a service from several providers.

Here’s my notes on two services I’ve tried.

Pagekite

Pagekite is a paid service. The company calls the tunnels ‘kites’ and meters bandwidth used by them.

To use it, you install their script, or a package and run:

pagekite.py 8080 yourname.pagekite.me

And your service will be available at https://yourname.pagekite.me.

If you don’t have an account, the script will prompt you to set up an account. Pagekite takes care of monitoring the tunnel for you. They provide documentation for starting up the service at boot using rc scripts. Again, you have to start your service before you start the tunnel.

If you only want to host a set of static files:

pagekite.py /path/to/folder yourname.pagekite.me

They provide built-in defenses for so-called ‘drive-by’ attacks, by blocking requests to /wp-admin/ and similar paths by default. You can also add passwords, or restrict access by IP address (useful for the IoT camera example.)

For six hosts and 150,000 Mb of transfer, I pay 40 USD a year. They also provide a white label service for IoT device makers.

Serveo

Serveo’s port-forwarding service is free-as-in-beer. It’s offered by its creator, @trevordixon. To use it for a service running on port 3000 run the command:

ssh -R 80:localhost:3000 serveo.net

which will return:

Hi there
Forwarding HTTP traffic from https://randomname.serveo.net
Press g to start a GUI session and ctrl-c to quit.

Notice that resulting url is https so you don’t have to set up your own certificates.

You can specify a sub-domain, in your ssh command.

ssh -R emma:80:localhost:8888 serveo.net

If the subdomain emma was available, you’d be reachable at https://emma.serveo.net.

Using a tool like autossh and startup scripts, you can make sure your service lives between restarts. Remember that you’ll also need to start the service you’re tunnelling. There’s a whole post on setting up autossh with systemd you which is useful here.

Serveo’s ease-of-use is also its flaw. Anyone can set up a forward with it, and the serveo.net certificate is a wild card, so if a browser trusts your service using Serveo, it trusts every other service using it to forward.

Other Services

Chen Hui Jing has a post, written after the first publication of this article, on other free tunneling services.

Caveats

A tunnel is only as good as the endpoint. You have to trust the endpoint with your service. If it’s compromised, then you have to consider your service compromised. Google has blocklisted these services before when people acting from bad intent have used these to distribute malware. A service which requires an account might not be libre, but will have some accountability.

Really, XML catalogs matter

Emma Humphries — Fri, 16 Jan 2004 00:00:00 +0000

Back in the early 2000's, I had a weblog, I had started it before the term blog had been coined.

This is a post, originally from January 16, 2004, about XML Catalogs, and figuring out what the right thing to test is. It was popular enough to be linked from PHP documentation for the DOMXML extension.

I've updated links and corrected typos. I just realized that Marc's last name had been misspelled for many years!

And some four years after this was originally posted, the W3C had noticed all the validation request traffic from all the XML applications not using catalogs or caches and did something about it.

This post is a snapshot of an era. Remember when we were going to do all the things in XML?

— Emma

This week I learned that XML Catalogs are important.

This started when I updated Marc Liyanage's PHP binary for Mac OS X on my development machine.

Pages went from taking milliseconds to over a minute to render. To say I was puzzled would be an understatement. I rolled back to an earlier version.

Looking for Clues

Some initial testing on another machine determined that the slowdown was in the DOMXML extensions to PHP. The extension exposes the Gnome XML and XSLT libraries as functions and objects to PHP.

After searching Google, php.net, and xmlsoft.com, I sent an email to Christian Stocker in Zurich. Christian works on the DOMXML extensions, and he might know of a bug.

I had gotten it in my head that the problem lay in nesting XInclude statements. XInclude is a specification for including one XML document inside another. We use XInclude to keep content for one of our sites isolated to a well-formed, valid XHTML document that can be edited in BBEdit.

A section of the intranet is described as an Atom feed, and each article's contents included into the feed. The Atom feed is included in an envelope document that contains the rest of the XML needed to render any page in the section.

I had jumped on the conclusion that somehow LibXML2 had changed and it had become inefficient at resolving nested XIncludes.

Christian wrote back that there weren't any issues he knew of, but asked me to send a test case.

The Wrong Test

I had devised the test case:

foo.xml:

<?xml version="1.0" encoding="utf-8"?><foo xmlns:xi="http://www.w3.org/2001/XInclude"><xi:include href="bar.xml" /></foo>

bar.xml

<?xml version="1.0" encoding="utf-8"?><bar xmlns:xi="http://www.w3.org/2001/XInclude"><xi:include href="baz.xml" /></bar>

baz.xml

<?xml version="1.0" encoding="utf-8"?><baz>Content!</baz>

When run with:

<?php$dom = domxml\_open\_file ("foo.xml");$start1=gettimeofday(); $dom->xinclude ();$end1=gettimeofday(); $totaltime1 = (float)($end1['sec'] - $start1['sec']) + ((float)($end1['usec'] - $start1['usec'])/1000000); echo "Time to handle includes: $totaltime1<br>"; echo $dom->dump\_mem ();?>

That should return:

<?xml version="1.0" encoding="utf-8"?><foo xmlns:xi="http://www.w3.org/2001/XInclude"><bar xmlns:xi="http://www.w3.org/2001/XInclude"><baz>Content!</baz></bar></foo>

Which it did, but faster than I expected. It timed at less than a second instead of over a minute.

I changed bar.xml to:

<?xml version="1.0" encoding="utf-8"?><bar xmlns:xi="http://www.w3.org/2001/XInclude"><xi:include href="baz.html" /></bar>

and baz.html was:

<?xml version="1.0"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><title>Untitled</title></head><body><p>New document</p></body></html>

Which did take several seconds as I thought it would.

The Right Test

That's where it dawned on me that XInclude between the version of the libraries PHP used, had started validating by default.

The XHTML DTD URL: http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd is a busy place. Try loading it and see. And I bet that URL is loaded because a lot of people didn't know their tool was calling over there every time it needed to load or validate something.

Commenting out the DTD declaration in baz.html and re-running the test brings back the earlier level of performance. However, I don't want to comment out the DTD references in my documents.

Going to Catalogs

I wrote back to Christian asking if LibXML, as built for PHP, honored XML Catalog files.

With a catalog file, I can tell my validating processor to resolve any reference to "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd" as a local file. Catalogs can do more than that, but the local resolution of DTD files is important.

Christian replied that by default, LibXML looks for a catalog at /etc/xml/catalog. So I created a catalog there.

<?xml version="1.0"?><!DOCTYPE catalog PUBLIC "-//OASIS//DTD Entity Resolution XML Catalog V1.0//EN" "http://www.oasis-open.org/committees/entity/release/1.0/catalog.dtd"><catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog"> <public publicId="-//W3C//DTD XHTML 1.0 Transitional//EN" uri="file:///etc/xml/xhtml/DTD/xhtml1-transitional.dtd" /></catalog>

I pointed "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd" to a local directory Apache could read from, put a copy of the DTD files there, and tried the tests again. Not as fast as without validation, but certainly faster since it didn't have to go over the Internet to validate the included file.

So there you go, catalog files, really important. I am chastened.

Thanks to Christian for getting me pointed in the right direction on this.

Originally published on January 16, 2004 on whump.com (but nobody goes there anymore) and updated February 11, 2020, by ECH.