loading...

How should markdown be saved and rendered?

twitter logo github logo ・1 min read

I'm trying to figure out the best approach for saving markdown to a database and then rendering.

Should the raw markdown be saved as a string into the database and then returned and converted to HTML before rendered on the front end? Or should the markdown be converted to HTML and then saved as a string into the database? Then it could just be passed onto the front end as raw HTML.

My setup:

  • MongoDB for db
  • Node for backend
  • Ember.js for frontend

Would love to know what approaches are being used out there.

twitter logo DISCUSS (16)
markdown guide
 

All the responses here are right, and I definitely would stress @mortoray 's point

There's a genral rule for for asset transformation: always keep the original

Since this is clearly something we put a lot of thought into, I'll talk about our approach.

A user submits a field we call body_markdown, and that is saved to their DB, and before it's written, we take that text and turn it into a field called processed_html. I really feel like this should be done at write time. For every write of a document like this there are going to likely be several orders of magnitude more reads.

We also do more than running a markdown engine. We take steps to convert any images to our hosted service, for performance and security, and we have other security-related restrictions. The list of things that happens between markdown and html is always growing.

I like to think of the HTML as a function of the markdown. Every time the markdown changes, run the function to output HTML. Within the main function are a series of other functions to handle each subtask and return the latest state of the HTML. I think the functional approach makes the process easy to modify and reason with.

 

I am 100% in agreement on storing rendered markdown as HTML on write, the only correction is I prefer to write HTML directly to the file on your static assets folder rather than into database. It does not belong to the database by any mean, it’s a cache that might be cleaned at any time (including on demand) and there probably should not be any intermediate persistent storage for it.

 

Nice, thanks for sharing how ya'll do it @ben ! I thought @mortoray 's point to 'always keep the original' was on point. Ended up saving the raw.

 

Parsing and rendering Markdown should not be a significant cost. Even a slow Python based parser should probably be fast enough for web rendering needs. If not, you can spawn one of the faster ones -- or just cache the results.

Storing as markdown keeps a lot of flexibility for later. In particular it lets you alter the design. HTML is overly structured and too strict to be design-change friendly. Many simple design changes would require altering the HTML, not just the CSS. IF you keep the markdown, the true source, it's easy enough to render new HTML.

There's a genral rule for for asset transformation: always keep the original. Your pipeline should ideally produce output from originals (with that optional caching I mentioned).

 

All solutions are contextual. What I say and question may be weird because I don't the context (ex: are you building a blog engine, is it a CMS, is it for enterprise, is it hosted etc).

1) why mongo db? It is ok to learn it. Unless there is a specific use case for it, it looks like traditional DBs are a good fit for the one you are describing.

2) real-time conversion will be (relatively) expensive. it is better to store the converted html.

3) most compute / conversion should happen at backend. Backend will always be more powerful than the frontend compute engines (browsers, mobiles). You will also avoid idiosyncrasies of these front-end compute engines.

 

1) Why not MongoDB? Document based databases are an excellent choice for a document based system. I would argue that unless you have a specific need for an SQL-style relational database, it should be avoided.

2) It shouldn't be expensive. Markdown is not a complex language to parse or process. If the DB is remote, it's most likely getting the document will take longer than parsing. Heck, even if it's on the local disk parsing should be faster than loading it. (Markdown libraries may of course vary here)

3) Storing Markdown doesn't imply the front-end needs to process. The front-end can still get HTML. The middle-ware can do the transformation required by the client. Better still, it can translate differently for each client.

 

Fully agree on point 1. Ideal use case for a document store, especially one with a flexible schema such as mongoDB. If you start to add new data items into your documents, no problem and no headaches having to edit the database. The data is either there or isn't.

 

Hi ! I'am really intersted by having an exemple where a middle-ware can do the transformation required by the client on the server side !


// GET article by id

app.get('/article/:id', function(req, res) {

    Article.findById(req.params.id, function(err, DBdocumentById){

            if (err) {

                console.log(err.message);
                res.redirect('/');
                return;

            } else {


                // Output HTML from DBdocument markdow encoded
                // Markdow to html parser

                let DBdocumentById_body_md2html = markdown.toHTML(DBdocumentById.body);


                res.render('article.pug', {

                    // Pass DBdocumentById

                    this_DBdocumentById_title: DBdocumentById.title,
                    this_DBdocumentById_author: DBdocumentById.author,

                    this_DBdocumentById_body: DBdocumentById_body_md2html,

                    this_DBdocumentById_id: DBdocumentById.id
                });
            }       
    });
});


`

As you can see in this code, the MD to HTML is directly done into the .get() method, how to use/write a funky custom middleware in the way to keep the .get() method as clean as possible ?

 

Yeah, always store the source (markdown) on the database.
Take a look at snarkdown a 1kB markdown parser in JavaScript.

 

Was gonna share this if it wasn't here already, developit is the master of small libs 🙈

 
 

Save the raw markdown. Maybe later you want to change the layout. It will be easier, I believe.

 

Yup yup, that's what I ended up going with. Thanks for the suggestion.

 

Aside from all the wonderful responses here already, I wanted to add a link to Babelmark2

It's a side by side comparison that shows a number of different markdown parsers and their output for given text. It's great because you can put in some sample markdown and pick the parser that gives you exactly the output that you want.

 

How do I render a multi line HTML to react UI

 

I have got multi line HTML as string which has to be rendered in React UI .How do i do it? The HTML i have is multi-line and complex. Any help please?

Classic DEV Post from Nov 12 '19

Feeling excluded and sad for not getting positive feedback - How should I react?

Michael Lee 🍕 profile image
Maker of things, giver of high-fives 🖐
Join dev.to

Guaranteed to make you a better developer or your money back