How should markdown be saved and rendered?

I'm trying to figure out the best approach for saving markdown to a database and then rendering.

Should the raw markdown be saved as a string into the database and then returned and converted to HTML before rendered on the front end? Or should the markdown be converted to HTML and then saved as a string into the database? Then it could just be passed onto the front end as raw HTML.

My setup:

• MongoDB for db
• Node for backend
• Ember.js for frontend

Would love to know what approaches are being used out there.

Comments

[Ben Halpern]

All the responses here are right, and I definitely would stress @mortoray's point

There's a genral rule for for asset transformation: always keep the original

Since this is clearly something we put a lot of thought into, I'll talk about our approach.

A user submits a field we call body_markdown, and that is saved to their DB, and before it's written, we take that text and turn it into a field called processed_html. I really feel like this should be done at write time. For every write of a document like this there are going to likely be several orders of magnitude more reads.

We also do more than running a markdown engine. We take steps to convert any images to our hosted service, for performance and security, and we have other security-related restrictions. The list of things that happens between markdown and html is always growing.

I like to think of the HTML as a function of the markdown. Every time the markdown changes, run the function to output HTML. Within the main function are a series of other functions to handle each subtask and return the latest state of the HTML. I think the functional approach makes the process easy to modify and reason with.

[Michael Lee 🍕]

Nice, thanks for sharing how ya'll do it @ben ! I thought @mortoray 's point to 'always keep the original' was on point. Ended up saving the raw.

[edA‑qa mort‑ora‑y]

Parsing and rendering Markdown should not be a significant cost. Even a slow Python based parser should probably be fast enough for web rendering needs. If not, you can spawn one of the faster ones -- or just cache the results.

Storing as markdown keeps a lot of flexibility for later. In particular it lets you alter the design. HTML is overly structured and too strict to be design-change friendly. Many simple design changes would require altering the HTML, not just the CSS. IF you keep the markdown, the true source, it's easy enough to render new HTML.

There's a genral rule for for asset transformation: always keep the original. Your pipeline should ideally produce output from originals (with that optional caching I mentioned).

[Nicolás Parada]

Yeah, always store the source (markdown) on the database.
Take a look at snarkdown a 1kB markdown parser in JavaScript.

[Michael Lee 🍕]

Sweet! Thanks for sharing snarkdown :)

[Leah]

Was gonna share this if it wasn't here already, developit is the master of small libs 🙈

[Hamza]

@michael I am trying to do the same for one of my projects and would like to know about the markdown editor that you integrated in your app.

My setup;

• SQL server for db.
• Asp.net core web api as a server.
• Angular for frontend

[Michael Lee 🍕]

Hey @hamza777 I'm actually not sure which editor I ended up using. Likely, I didn't use any editor at first because I think I wanted just some basic markdown support.

So as others had suggested, I would save the raw markdown input into the database, but then when it is shown to the user, it is parsed to HTML.

Hope that helps!

[Meghan (she/her)]

Aside from all the wonderful responses here already, I wanted to add a link to Babelmark2

It's a side by side comparison that shows a number of different markdown parsers and their output for given text. It's great because you can put in some sample markdown and pick the parser that gives you exactly the output that you want.

[Salvador Montiel]

Save the raw markdown. Maybe later you want to change the layout. It will be easier, I believe.

[Michael Lee 🍕]

Yup yup, that's what I ended up going with. Thanks for the suggestion.

[Joseph Jude]

All solutions are contextual. What I say and question may be weird because I don't the context (ex: are you building a blog engine, is it a CMS, is it for enterprise, is it hosted etc).

1) why mongo db? It is ok to learn it. Unless there is a specific use case for it, it looks like traditional DBs are a good fit for the one you are describing.

2) real-time conversion will be (relatively) expensive. it is better to store the converted html.

3) most compute / conversion should happen at backend. Backend will always be more powerful than the frontend compute engines (browsers, mobiles). You will also avoid idiosyncrasies of these front-end compute engines.

[edA‑qa mort‑ora‑y]

1) Why not MongoDB? Document based databases are an excellent choice for a document based system. I would argue that unless you have a specific need for an SQL-style relational database, it should be avoided.

2) It shouldn't be expensive. Markdown is not a complex language to parse or process. If the DB is remote, it's most likely getting the document will take longer than parsing. Heck, even if it's on the local disk parsing should be faster than loading it. (Markdown libraries may of course vary here)

3) Storing Markdown doesn't imply the front-end needs to process. The front-end can still get HTML. The middle-ware can do the transformation required by the client. Better still, it can translate differently for each client.

[ImTheDeveloper]

Fully agree on point 1. Ideal use case for a document store, especially one with a flexible schema such as mongoDB. If you start to add new data items into your documents, no problem and no headaches having to edit the database. The data is either there or isn't.

[Drozerah]

Hi ! I'am really intersted by having an exemple where a middle-ware can do the transformation required by the client on the server side !

// GET article by id

app.get('/article/:id', function(req, res) {

    Article.findById(req.params.id, function(err, DBdocumentById){

            if (err) {

                console.log(err.message);
                res.redirect('/');
                return;

            } else {


                // Output HTML from DBdocument markdow encoded
                // Markdow to html parser

                let DBdocumentById_body_md2html = markdown.toHTML(DBdocumentById.body);


                res.render('article.pug', {

                    // Pass DBdocumentById

                    this_DBdocumentById_title: DBdocumentById.title,
                    this_DBdocumentById_author: DBdocumentById.author,

                    this_DBdocumentById_body: DBdocumentById_body_md2html,

                    this_DBdocumentById_id: DBdocumentById.id
                });
            }       
    });
});

`

As you can see in this code, the MD to HTML is directly done into the .get() method, how to use/write a funky custom middleware in the way to keep the .get() method as clean as possible ?

[jay]

How do I render a multi line HTML to react UI

[Spurgeon Gnan Prakasham Tara]

what will happen if we send markedown text directly to backend server without sanitizing it? will there be any security issues?

[jay]

I have got multi line HTML as string which has to be rendered in React UI .How do i do it? The HTML i have is multi-line and complex. Any help please?