BotOrNot HTTP API

This is a Python demo of the HTTP API. This is written in Python, but one should be able to use this as a rough guide for using the HTTP API in any language.

This demo uses tweepy library to access Twitter's REST API. One can find a list of Twitter Libraries for other languages here.

Dependencies: any language

Using the Twitter REST API does require you to have a Twitter App defined. If you do not have one, visit apps.twitter.com and create one. Of course, this necessitates having a Twitter account in the first place.

Dependencies: Python

This is written with Python 3 but should work fine with Python 2.

This demo depends on tweepy and requests. If you have pip installed, getting these is as easy as

pip install tweepy requests

API Demo

In [1]:
from __future__ import print_function
import json
import tweepy
import requests

Step 1: Use Twitter API to retrieve statuses

We need to initialize a Twitter API instance using our app's credentials. To get these, go to the "Keys and Access Tokens" tab of your app's properties at apps.twitter.com. You may have to generate an Access Token in that tab if you haven't already.

Image of Keys and Access Tokens

In [2]:
auth = tweepy.OAuthHandler('consumer_key', 'consumer_secret')
auth.set_access_token('access_token_key', 'access_token_secret')
twitter_api = tweepy.API(auth, parser=tweepy.parsers.JSONParser())

Here we define the screen name we want to check with the BotOrNot API, and then retrieve the user's timeline.

In [3]:
screen_name = '@clayadavis'
user_timeline = twitter_api.user_timeline(screen_name, count=200)

We also want to get data about the user account. This information is included with each tweet in the timeline, but if the timeline is empty we need to make a separate API call to retrieve it.

In [4]:
if user_timeline:
    user_data = user_timeline[0]['user']
else:
    user_data = twitter_api.get_user(screen_name)

In addition to the user's tweets, we want to get other users' mentions of the target user. In order to do so, we use Twitter's search API and search for the screen name. Note that in order for this to work, the screen name must have the @ prefix.

In [5]:
if not screen_name.startswith('@'):
    screen_name = '@' + screen_name
search = twitter_api.search(screen_name, count=100)
mentions = search['statuses']

The collection of tweets we'll send the BotOrNot API is just the concatenation of the user's timeline and the search statuses.

In [6]:
tweets = user_timeline + mentions
len(tweets)
Out[6]:
204

Step 2: Submit data to BotOrNot

BotOrNot's API endpoint accepts POST data in JSON format. Towards that end, we are going to construct a dict for the data to submit. content contains the tweet objects we got from the Twitter API, and meta contains information about the user and any other info we want in our response.

In the meta object, user_id and screen_name are required. You can add anything else in this object you'd like; the meta object is returned along with the response. This may be useful if you are saving the responses and you want to include additional data from user_data.

It is important to use the id_str value of user_data as opposed to id. This is because JSON doesn't handle 64-bit integers reliably, and user IDs may be larger than 32 bits.

In [8]:
post_body = {'content': tweets,
             'meta': {
                'user_id': user_data['id_str'],
                'screen_name': screen_name,
                },
            }

Here we dump the post_body dict as a JSON string and POST it to the API endpoint.

In [9]:
bon_url = 'http://truthy.indiana.edu/botornot/api/1/check_account'
bon_response = requests.post(bon_url, data=json.dumps(post_body))
bon_response.status_code
Out[9]:
200

If successful, the response will also be JSON:

In [10]:
bon_response.json()
Out[10]:
{'categories': {'content_classification': 0.27,
  'friend_classification': 0.15,
  'network_classification': 0.17,
  'sentiment_classification': 0.25,
  'temporal_classification': 0.43,
  'user_classification': 0.36},
 'meta': {'screen_name': '@clayadavis', 'user_id': '1548959833'},
 'score': 0.37}

Note: Rate Limits

Twitter's REST API limits one to 180 calls per endpoint, per 15-minute period. BotOrNot's API uses this rate limit as well. Submitting too many requests will result in a 429 error with content like this:

{
  "status": "chill",
  "reset": 1449626168, 
  "message": "You are over the rate limit for this endpoint. Please wait until 'reset' to submit another request."
}

As with Twitter's API, you can inspect BotOrNot's API responses in order to get information about the rate limit:

In [11]:
for k in sorted(bon_response.headers):
    if k.startswith('X-Rate-Limit'):
        print("{0}: {1}".format(k, bon_response.headers[k]))
X-Rate-Limit-Limit: 180
X-Rate-Limit-Remaining: 173
X-Rate-Limit-Reset: 1449632066