API Rate Limiting

Since WordPress’s new admin site is so awesome <insert emoji with heart eyes>, I think I should document my little learning notes here instead. Shall use a sub-domain for this WordPress in time to come!

Letterbox’s traffic went a bit off today, causing our instance’s CPU usage to spike. Apparently people were spamming API calls to our server, resulting in some form of DoS attack, well but it didn’t exactly crash/overload it.

Do things that don’t scale, right? It probably slipped our mind when designing the API, thinking that all users would be angels and not do weird stuff like that. Clearly we were wrong. Also this would be needed in time to come when the API calls start to exceed Facebook’s API rate limit.[1]

API Rate Limiting is basically restricting the number of times a particular end user is able to call the endpoint. There are a couple of ways to do this, and the way that we chose would be to utilize a Redis cache.

The idea is to have a key-value pair (user ID as the key and times accessed as value) that will update everytime a call is made, and Redis is perfect for handling little tasks like that.

There’re many considerations to take into account for such a simple implementation. A cursory look online points to having to deal with HTTP headers (agh damn networking), different time buckets and also pipelining. It is certainly good to dive more into the details in time to come, but then there’s finals.

So instead of reinventing the wheel, I opted to use express-limiterIt’s already written and works right out of the npm box. [2] The task is done in around 10 lines of code.

var redisClient = require('redis').createClient();
var limiter = require('express-limiter')(app, redisClient);

limiter({
 path: '/matches',
 method: 'all',
 lookup: ['user.id', 'connection.remoteAddress'],
 total: 50,
 expire: 60000
});

These few lines of code simply set a 50 call limit to /match endpoint, and sends a 429 if there are anymore calls from the same user IP thereafter. It does not seem like much but I think as a frontend developer since forever, I’ve not been able to appreciate such backend concepts.

A little something I learnt today, besides SQL, DRC/TRC and Relational Algebra. Yup CS2102 is coming up next.

Oh and I realized at the end of writing this post why I wouldn’t document more craft related post here. Typing code into WordPress is still pretty painful. 

Ciao~

 

 


[1] Well… a problem for next time.
[2] Loses a lot of flexibility though.

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s