Caching Guide

How to Cache Your Data and Stay Within the Twitter API v1.1 Rate Limits

The Twitter API includes data stream specific access rate limits that restrict how frequently you can pull new data from their servers. Documentation of these limits is available on the Twitter developer site.

Why Do I Care?

So why is this important for you? If you are using the Tweetledee files in a public facing application (e.g. to populate text on a website) and you receive more than one visitor per minute, the Twitter servers will lock your application account for a period of time, no new data will be served to you on new requests, and the Tweetledee script that you are using will respond with an error. Users who repeatedly exceed the rate limits risk having their application blacklisted and losing all access to the Twitter API data. Twitter currently uses 15 minute intervals to determine their rate limits and the number of permitted requests is as few as 15 over this interval, a value that differs by the API data type (see the link above).

How Does Tweetledee Address This Issue?

As of version 0.4.0, Tweetledee includes a built-in Twitter data caching mechanism with a default caching interval of 90 seconds. This means that on the first request to the Tweetledee file, a cache file is saved locally on your web server and the saved file is served to users who make any new requests for the data over the next 90 seconds instead of attempting to request new data from the Twitter servers. As a result, your application will request new data from the Twitter servers, at most, 10 times in a given 15 minute period irrespective of the number of requests made for data from your application. This number of requests will keep you well below the rate limit across all of the Twitter API data types.

Wait, Stale Data?

Yep. Unlimited access to 'real-time' Twitter data is not possible. The details are available here.

Rate Limit Protection Caveats

IMPORTANT: Please recognize that the default rate limit protection that Tweetledee provides assumes that you are using only one file per application account at a time. Your API access counts are tallied across all of the Tweetledee files that are in active, simultaneous use for a given application account.

In order to address this issue, every cached Tweetledee file allows you to adjust the cache interval. To do this, include the cache_intervalparameter in your URL with a duration (in seconds) as the argument to this parameter. There are examples for every Tweetledee file in the Usage section of the documentation.

Please make sure that you are familiar with the Twitter rate limits that are in place before you use these files in a production setting.

Write Access Permission on Your Server

The ability to cache Twitter data with Tweetledee requires local file write access on your web server. If you find that your data are not being cached, please confirm that this permission is in place through your hosting provider. If this does not appear to be the problem, please submit a detailed description of the issue on the GitHub repository.