loading...
Cover image for How do you put data for other people to use?

How do you put data for other people to use?

nishantwrp profile image Nishant Mittal ・1 min read

Suppose you have scraped data from a site in JSON format which can be also used by other people. How do you expose it?
Do you make a url directly give out that JSON data? Or there is a better way. I was thinking of something like a GraphQl server. What do you guys think?

Discussion

markdown guide
 

If the data doesn't need to be filtered, paged, or sorted, simply host it as a raw file.

If you need some of the above things but don't want to do much, graphql is a great option.

If you need to put functionality in front of the data (perform arithmetic, generate new data, etc), then you'll want to build a full api, which may or may not include graphql

 

99% of the time the data I scrape will need to be transformed because the API will give me tons of data that is irrelevant. So it will probably be rendered in a nice table or export to CSV or something.

I built the scraper to display specific data for my clients, not to just return the results of an API call in JSON :)