Get Started with Popular Social Media APIs for Data Gathering
Step by step guide to Reddit, LinkedIn, Twitter, TikTok APIs
We live in a world where people wake up in the morning and open Facebook/Instagram before getting out of bed. One out of five people spends 5+ hours on social media on a daily basis. Social media generates billions of GigaBytes of data each day. This data is used by people and companies for many purposes such as advertising new services, showing new ads, or selling information.
Most of the social media websites such as Facebook, Reddit, Twitter, etc. provide their APIs which allows users to do pretty much anything imaginable. Data from these websites can be used for many purposes as simple as finding which emoji is the most frequently used, how much positive-negative comments people make on a football team or something complex like creating an automated bot that detects fake news.
In this blog, I will teach you how you can get started working on publically accessible data provided by some of the most popular social media websites using their APIs. I will go through Reddit and Twitter APIs and will also give brief information on TikTok and LinkedIn APIs.
Let's get started.
Reddit API
Reddit is known as the ‘front page of the internet’. It is a platform that provides a large network for communities to interact, discuss, and share content with each other making it a great source for data analysts. Reddit has an amazing developer API that allows data gathering, posting, authenticating, and everything you can do and see on the Reddit website.
How to get Reddit API running
- Create/Login into your Reddit account.
- Go to the Reddit Apps page.
- Create a new App by clicking on ‘are you a developer? create an app…’ button on the top left corner. A form will open as shown below.
- Enter a name for your application.
- Select the “script” type. The script implies we will use this application for querying the Reddit database.
- Provide a basic description of the application(optional).
- Provide basic information on the URL inserted in the following box(optional).
- Provide your website URL as redirect URI. You can use any valid URL in this field like ‘http://localhost:8080’. For script type applications, this field doesn’t matter(but must be filled).
- Click Create app.
4. Once the app is created, open it and copy the ClientID and ClientSecret fields. We will need these to authenticate our API requests. These fields can be found in the locations shown below.
Data Rate Limitations
Reddit API supports 60 requests per minute.
For example,
s = time.time()coronaSub = reddit.subreddit("coronavirus")for post in coronaSub.hot(limit=60): print(post.title)print("Total time", time.time() - s)
In the above code, we requested 60 results from Reddit on the topic ‘coronavirus’. When coronaSub.hot() is called, it sends a single GET request to the Reddit server and gets 60 results in a single request. The above code runs in approximately 1–1.7 seconds. As mentioned above, the Reddit API can perform 60 requests per minute. This is because the data rate limit doesn’t mean the result count. Each request can respond with multiple entries. If the requested count is huge, PRAW will automatically send more requests. Each request can bring around approximately 100 results at a time.
Demo Code
Github Link: https://github.com/kirito-k/Reddit-API
Additional Resources
- Check out supported Reddit API Wrappers for different programming languages.
- For more information on API, check out the Reddit API documentation website.
- API rules and terms can be found here.
Twitter API
Twitter provides huge support for developers. Twitter’s API is divided into a multi-tier structure. More information on Tier Models can be found here. If the application is real-time intensive, Twitter also provides Streaming API. See the Streaming API documentation for more information. The most common tiers used are Standard and Premium.
Standard Tier: The main features include data query retrievals, post tweets, send direct messages on posts, etc.
Premium Tier: The main features include accessing queries on scale and real-time public account engagement etc.
How to get Twitter API running
- Create/Login into your Twitter account.
- Go to the apps.twitter.com/
- Create a new App by clicking the ‘Create an app’ button on the top right corner. A form will open as shown below.
- Enter a unique name for the application.
- Provide a basic description of the application and how it will be used.
- Provide a website URL (your website if needed). You can use any valid URL in this field like ‘https://www.google.com/’. For basic querying purposes, this field doesn’t matter(but must be filled).
- Click Create.
4. Once the app is created, open the app and navigate to the Key and Tokens tab.
5. For Twitter API, we need four keys. API key, API secret key, Access token, and Access token secret. Create a unique Access token and Access token secret by clicking the ‘Generate’ button.
Note: These tokens will be visible to you only once. You must copy them safely in a file to be used later. If you lose these tokens, you cannot retrieve them and you need to delete them and generate a new pair.
6. Copy all the four keys in a file to be used later.
Data Rate Limitations
- Twitter allows you to make 15 requests per 15 minutes window time frame.
- You can perform 100,000 requests per day.
- User timeline requests which get tweets of a particular user return 3200 tweets at a time.
Note: Different Tiers of Twitter allows different data rates. Check the Rate Limit page for more information.
Demo Code
Github Link: https://github.com/kirito-k/Twitter-API
Additional Resources
- Check out supported Twitter API Wrappers for different programming languages.
- For more information check out the Twitter API documentation website.
- API rules and terms can be found here.
TikTok API
TikTok has good support for sharing and embedding their videos with our applications. However, TikTok doesn’t have an official API for data querying and exaction.
This might be due to the following reasons:
- TikTok is a fairly recent company(2016). API support might still be under development.
- Data transfer charges might be too high to allow requests since the majority of the data is video format.
- Most of the Chinese companies do not allow data extraction from their websites due to their government regulations.
LinkedIn API
LinkedIn API allows users to retrieve their profile information, update profiles, and post blogs. It also provides authentication using a LinkedIn account for your applications. However similar to TikTok API, LinkedIn doesn’t provide official API support for data querying.
This might be due to the following reason
- LinkedIn had a huge lawsuit with HiQ Labs over scraping public data and using it for commercial use. LinkedIn lost the lawsuit and changed some of its policies regarding the same issue. Not providing an official API for data querying might have been caused due to this.
In a nutshell, it is very easy to get started with the APIs. There are so many similar public APIs that allows users to gather data. You can find a list of public APIs here. It can be used to create fun innovative projects or many great products.
Comment down how I can improve the blog more and thank you for reading.
