I’ve run into a problem while developing a WordPress plug-in. Basically the API I’m building the plug-in for limits the requests I need to make to 6 per minute, however when the plug-in activates I need to make more than 6 requests to download the API data I need for the plug-in.
The API is the LimelightCRM API (http://help.limelightcrm.com/entries/317874-Membership-API-Documentation). I’m using the campaign_view method of the API, and what I’m looking to do is potentially make the requests in batches, but I’m not quite sure how to approach the problem.
Idea 1:
Just off the top of my head, I’m thinking I’ll need to count the number of requests I’ll need to make with PHP on plug-in activation, by using campaign_find_active and then divide that count by the request limit (6), and make 6 campaign_view requests per minute until I have all of the data I require and store them in WordPress transients. However, say I need to make 30 requests, the user can’t just sit around waiting 5 minutes to download the data. Even if I manage to come up with a solution for that, it might require me to set the time limits for the WordPress transients in such a way that the plug-in will never need to make more than 6 requests. So my next thought is, can I use a WordPress hook to make the requests every-so-often while checking when the last batch of requests was made? So it’s already getting very tricky. I wonder if you guys might be able to point me in the right direction. Do you have any ideas on how I might be able to beat this rate limit?
Idea 2:
Cron jobs that store the values in a database?
//Fetch Campaign ID's
$t_campaign_find_active = get_transient('campaign_find_active');
if(!$t_campaign_find_active){
limelight_cart_campaign_find_active();
$t_campaign_find_active = get_transient('campaign_find_active');
return $t_campaign_find_active;
}
//Fetch Campaign Information for each Campaign ID
$llc_cnames = array();
foreach($llc_cids as $count => $id) {
if(!get_transient('campaign_view_'.$id)) {
limelight_cart_campaign_view($id);
$llc_cnames[$id] = get_transient('campaign_view_'.$id);
}
}
//Merge Campaign ID's and Campaign Info into Key => Value array
$limelight_campaigns = array_combine($llc_cids, $llc_cnames);
Note: The functions limelight_cart_campaign_find_active() and limelight_cart_campaign_view() are not included because they simply make a single API request, return the response, and store it in a WordPress transient. I can include the code if you guys need it, but for the purposes of this example, that part of the plug-in is working so I did not include it.
I’ve come up with a solution for this guys, and I should have thought of it before. So I’ve arrived at the conclusion that downloading all of the API data on activation is simply impossible with the current rate limit. Most people who might use the plug-in would have far too many campaigns to download all of their data at once, and it’s inevitable that the rate limit will be used up the majority of the time if I keep the code the way it is. So rather than constantly having that API data ready for the plug-in right after activation, I’m going to give the user the ability to make the API calls on demand as needed using AJAX. So let me explain how it will work.
Firstly, on plug-in activation, no data will initially be downloaded, and the user will need to enter their API credentials, and the plug-in will validate them and give them a check mark if the credentials are valid and API log-in was successful. Which uses one API request.
Now rather than having a pre-populated list of campaigns on the “Add Product” admin page, the user will simply click a button on the “Add Product” page to make the AJAX
campaign_find_activerequest which will fetch the campaign ID’s and return a drop-down menu of campaign id’s and names. Which only uses one request.After that drop-down data is fetched, they will need to choose the campaign they want to use, and upon choosing the campaign ID the plug-in will display another button to make a
campaign_viewrequest to fetch the campaign data associated with the ID. This will return another drop-down menu which will allow them to choose the product. This will also require a little CSS and jQuery to display/hide the AJAX buttons depending on the drop-down values. Which will only use one API request, and because the request is not automatically made and requires a button click the user will not make several API requests when choosing a campaign ID in the first drop-down menu that was fetched.The user would then click publish, and have a wordpress indexed product with all of the necessary limelight data attached and cached. All API requests will be stored in transients with a 1 hour time limit, and the reason for the hour is so they don’t have to wait 24 hours in case they make updates. I will also include a button on the settings page to clear the transients so they can re-download on demand if necessary. That could also get a bit tricky, but for the purposes of this question it’s not a problem.
In total, I’m only using 3-4 API requests. I might also build a counter into it so I can display an error message to the user if they use too many requests at once. Something along the lines of “The API’s limit of 10 requests per minute has been reached, please wait 60 seconds and try again.”
I welcome any comments, suggestions or critiques. Hope this helps someone struggling with API request limits, AJAX is a great way to work around that if you don’t mind giving the user a little more control.
I just made 40 API accounts and randomly choose one for each request.. Works well