Simultaneous admin updates causes custom fields to not update

Okay, I have another doozie of a problem and although I’m still in the middle of trying to figure out exactly what’s going on and what kind of help I might need, I thought I’d throw something up here and see if anyone else had seen an issue like this before.

My site is currently running WordPress 3.6 on a Linux server, and is hosted with Rackspace. At it’s most basic level, my problem concerns updating custom fields for a post when the post is saved/updated.

I’ve configured a few custom meta boxes that show up for a given post type (products, in this case). These custom meta boxes contain inputs and specifications for each of these products on both their own front-end post page and for other front-end pages on my site. I’ve attached a save function to the save action for the post (update button), so that the information contained in these meta boxes is all saved into a number of custom fields. I’ve accomplished this through the following method.

Insert meta box:

function create_custom_products_taxonomies(){
    register_post_type( 'products',
        array(
            ...other inputs...
            'register_meta_box_cb' => 'products_meta_box',
            ...other inputs...
        )
    );
}
add_action('init', 'create_custom_products_taxonomies');

function products_meta_box() {
    add_meta_box('custom_product_info', __('Top 10 Product Listings'), 'add_custom_page_product_info', 'products', 'normal', 'high');
}

Function definition:

function add_custom_page_product_info() {
    ...code to populate meta box...
}

Save definition:

function save_acpi_data() {
    ...code to save information in meta box...
}
add_action('save_post', 'save_acpi_data');

I add this all mostly just to show how I’m doing things.

The way I have my code set up, is that the save function checks for data from the custom meta boxes and then saves the information into appropriately-named custom fields. I also have the save function set up to clean up after itself and delete any custom fields that are present which don’t have associated meta box information present upon the post save/update.

My problem comes in ONLY when there are multiple people logged into the admin side of my site and are updating/saving these pages. (We have approximately 700 products on the site and it takes multiple people to keep the site updated appropriately.) When this condition occurs, the data that is within my custom meta boxes (custom fields?) is somehow being deleted and is thus disappearing from my site.

Right now my working theory is that the data that is getting passed through WordPress on the admin-side page load and/or post save is getting truncated somehow. I’m basing this on the fact that when we have about 6-7 people logged into the admin side of the site and working on it, that the quad-core server we have the current site hosted on pegs the CPU to 100% for brief but repeated periods of time. Additionally, I’ve noticed that within my apache error logs, and during the times when multiple people are updating the site and data disappearing, there are errors showing up like the following:

Script timed out before returning headers: async-upload.php, referer: wp-admin/post.php?post=75064&action=edit&message=1
Script timed out before returning headers: async-upload.php, referer: wp-admin/post.php?post=75262&action=edit&message=10
Script timed out before returning headers: async-upload.php, referer: wp-admin/post.php?post=75269&action=edit&message=6

repeated multiple times.

This issue with data loss hasn’t been apparent before this last week, but then again prior to this last week there have only been 2-3 people working on the site at a given time. I also haven’t been able to duplicate this problem to even the slightest degree on a second development server (identical server setup, WordPress version, and theme code) when I’m the only person logged into the admin side. This most recent push of changes to our site has come after finding a few persistent errors on nearly every product page and wanting to fix all the errors within a reasonable amount of time.

I’d like to just ignore the problem and say that it’ll be over and gone as soon as these issues are fixed, but I know that it’ll crop up again later, and besides, the issues that it’s causing for the current site right now are quite impactful. We have a backup of the site and the team that’s responsible for keeping it up are just having to copy this lost data from the backup to the current site. Over and over and… you get the picture.

Right now I’m trying to mitigate the problem by checking that the amount of information present in my custom meta boxes is consistent with what is present when my save function goes to save (and cleanup) all the pertinent custom fields. (Check data on post save)

I haven’t yet been able to come up with a way to check if the data is being truncated on the admin-side page load and then deleted when the post is saved. In this case, the custom fields on the page would show as empty after a page load, even though the data would still be present in the database and “should” be showing. (Check data on admin page load for post)

I’ve already scrubbed my entire site for php errors by accessing the information presented by setting WP_DEBUG to true, running through the front and back ends of the site with the Debug Bar plugin active, and also by scouring my apache error logs (apart from those errors mentioned above).

I think this sufficiently describes the problem I’m seeing. I guess my questions for the community are:

1. Has anyone else ever seen this kind of behavior? (Lacking the “clean up” portion of my save function, this issue would present itself as information within meta boxes not being saved/updated to their corresponding custom fields when the post is saved.)

2. Is there something else I should be looking at?

My only other option is to turn off the “clean up” portion of my save function and make the other guys manually delete all of the custom fields associated with the information they want to delete, which would be tedious for them but keep the site from deleting random information that we don’t want gone. If I can’t figure out a solution to this issue relatively soon, I may have to resort to this undesirable option.

I’d rather just solve the problem though. Guess that’s what I get for being an engineer though. 🙂