A few weeks ago I wrote about the Importance of using User Agents when we scrap data, and my examples shows the response from Twitter when we used the correct User Agent. This time I want to do the same but Facebook. We gonna scrape the posts in users profiles, Facebook pages and groups.
What we gonna get?
A list of items with the next values:
| params | description |
|---|---|
| published | Formatted datetime of published |
| description | Post text content |
| images | List of images in posts |
| post_url | The unique post url |
| external_links | External links found in description |
| like_url | The Like url |
Now I have a developer blog, continue reading this post here: https://hhsm95.dev/blog/python-facebook-posts-scraper-with-requests-and-beautifulsoup4/
Thank you!
Top comments (17)
Hugo, this is great. When I run this, as is, I'm only getting about 6 posts. Is this normal? Also, how would you recommend I edit to search for group posts with a specific tag_id? I tried to rewrite a few lines to specify my url with "groups/{group_id}/post_tags/?post_tag_id={tag_ID}" but I was only ever brought to the group's standard timeline.
Hi, i had the same problem, when i run, i'm only getting 6 posts ! you resolved this ?
Thank you.
Hello Every one I need for research purpose 100 users already posted text.Per user i need from 120 to 200 posted text.But this python code download only 5 days post.Can I use this code for this purpose pls help me I kill 2 month for this purpose.I need to download a specific users profile post text all
please any one help me to this problem
hello Hugo thanks for sharing this .am working on scraping facebook data but the facebook blocked my account several times . please make a video to help as to see how it works.
how many posts does it extract ?
The key to avoid being blocked is scrap slowly (about 3-5 between profiles) and use an User Agent like above (an old phone) that don't requires javascript so the detection is harder for Facebook.
Hi, I'm not sure if I can post an email here, but ask me any issues in the Github project.
github.com/adeoy/FacebookPostsScraper
getting error TypeError: 'module' object is not callable while running the code. What may be the problem ? Thanks in advance
same error
Hi!
Could you please advise on how to build a scraper that extracts comments on a specific subject on Facebook?
Thanks for assisting.
Hi, the key to scraping on Facebook is the use of a User Agent from an old phone, as in this case use a Nokia C3. To extract the comments from the posts, it is necessary to enter the post and locate the CSS selectors that identify the comments, it is possible that I update the post adding this functionality.
Hi, Thank you for sharing this, that's works well, but when i run the program, i get just 6 posts from a facebook page? how can i get more posts ?
Thank you.
Hi @nassera2014
In order to get more number of posts I suggest to use Facebook_Scrapper Library where one can configure get_posts() with number of pages and posts per page to be scrapped. And by the way you could get all granular level info of reactions, like, comments, post_id, post date, etc.,
Please find the below link for your information,
pythonzero.com/2021/12/how-to-extr...
hello guys. Can i to use today this steps and code to download comments and post of facebook?
Hello :) thank you for this share, can you please tell me the equivalent of "post_url_text" for the French version?
thanks :)
Hello Sir thank you for you tuto. How can we also collect for each post all comments, likens and number of share?