DEV Community

Discussion on: I spent 30 years coding full-time, then I switched to full-time management and leadership. Ask me anything.

Collapse
 
gabydewilde profile image
gaby-de-wilde • Edited

I was only 10 years old in 84 but I started coding machine code when I was 13. Today I work 2 cleaning jobs that add up to 39 to 66 hours per week that are at night or in the early morning. In theory there should be some part time coding job that pays a lot more but like this my mind is available for personal use.

I'm actually looking to start an internet revolution for the betterment of mankind. (I can dream right?) Over the years I've envisioned some wildly complex ideas that for the most part are well beyond anyones attention span and skill set. (duds) In doing so I started to see that the best ideas are those where a basic understanding can be had in seconds with infinite steps into further complexity.

I've managed to simplify something like a hundred ideas to the point where I should really start working on their implementation. I've picked one for you based on your love for scalability.

The goal is to create a organic new-news aggregation network and to dramatically speed up news aggregation.

It involves 2 file types I called: Even Simpler Syndication [ESS] and No Nonsense Outline [NNO]

I wrote a spec here go-here.nl/ess-and-nno but I think I can give a better description tailored for you.

An ESS is a tiny comma separated text document with a unxi build date, an url for an rss feed then, for each news item, 1) a publication date that is an offset in minutes from the build date and 2) a tiny check sum.

Aggregators should limit their results by age and discard an ESS if its build time or top article indicates it being to old.

An NNO file is the end result of aggregation that can be used for the next session but is primal intend is publication/sharing.

It it s a comma separated text document that starts with a unix build time followed by a sorted list of items 1) feed url, 2) minutes before build, 3) title checksum

The file is build after the aggregator aggregates the users hundreds, thousands or millions of ESS, RSS and/or Atom subscriptions while reducing the result list by a configured amount. (say 3000)

When published a 2nd user can load the NNO into their aggregator which should fetch the top feed items listed in the NNO until they have a screen full of results. (scrolling loads more)

The 2nd aggregator can then proceed to fetch that users own subscriptions and/or parse additional NNO files into the result set.

When done they can publish their own NNO file.

Ideally the aggregators maintain their own manually unsubscribed feed list and implement a configurable filter like a word and a word combination filter, a Bayes filter, a machine learning implementation or some other filters. The news items displayed should have an indication about their origin (which NNO file) and they could provide an auto translate link or do inline-translation of headlines/articles.

The point of the entire exercise is to have people access the internet without a multinational man in the middle.

In my experience a few thousand RSS and Atom feeds will produce at least one truly fascinating thing to read per day while larger amounts allow for more aggressive filtering.

Sadly, RSS and Atom files can be so large and are so hard to parse that a home connection can do only a very limited number of them. If there are a billion websites ten thousand is only 0.001% if each request would take a second to resolve ten thousand would be 166 minutes. If your home pc can do 10 per second you still have to wait for 16 minutes while loading the top 40 results of an NNO file would take only 4 seconds.

I think in those 16 minutes combined with a decade of finding and filtering feeds real value is produced. If the result set can be enriched several times per day with input from other users (thereby allowing for more elaborate contextual filtering) the end result accumulates value faster than it expires.

The big question is: Where do I go from here?

I wrote an (untested) Node.js module for ESS, I will make a PHP version as well. I will attempt to make my powerful but userless aggregator more user friendly but mass adoption is the ultimate pipe dream.

What ideas/suggestions do you have for this project?

Ill add a news result just for fun:

news.go-here.nl

I'm sorry for the lack of tech websites ;)

Collapse
 
lpasqualis profile image
Lorenzo Pasqualis

Your passion for technology is commendable, and your enthusiasm will bring you far. I have a few recommendations:
1) Simplify the message. The important thing is to describe the problem you are trying to resolve and the value your idea will add. If you can't write both combined on one side of a 3"x5" index card, then you should rethink the idea.
2) Conplexity is not a goal. It is the enemy. A path to wildly complex mutation is doomed. The goal should be simplicity.
3) If people can't understand the ideas, it is not a demonstration that the idea is superior than most people. Simplicity is the ultimate good design. The implementation of the idea might be complex, but you need to separate implementation details from the mix.
4) Is the goal to start a business? Or to create technology? This is a key question to know what the next steps are. If you are trying to start a business, then you should get a sense for how you will monetize the idea, put a prototype together and try to raise money or users or both. If the goal is to create technology then you need to implement it, open source it, demonstrate that it works (ship a free product that uses it) and write a detailed technical paper about it. However, technology ideas with no business validation have a very hard time gaining traction.

Hope this helps.

Collapse
 
gabydewilde profile image
gaby-de-wilde

Yes, the question is how to introduce a technology without a business. The only thing I can think of is to get some business to adopt it.

The description here was for you. The "real" description would be something like: Use this module and get subscribers/readers. After all, that is how most use RSS. Implement it and ignore it.