I have a large JSON file (>100MB) which I want to parse and then convert its objects into wordpress posts.
I have successfully created a function to complete this task but it is not able to loop through all object, else it dies by giving an error of PHP Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 92 bytes)
Function is able to handle files under 1MB but for files greater than 1MB it fails.
I have asked server administrators to increase memory limit and now no memory errors are visible but still it is not able to fetch all JSON.
I have gone through many post and questions but couldn’t find/understood anything useful. Also it is my first attempt to write such function, so any help/guidance will be appreciated.
EDIT
Added code for function used
function create_post_from_json($json_key) {
$json_options = get_option('json_file_data');
$obj = wp_remote_retrieve_body(wp_remote_get($json_options[$json_key]['url'],array( 'timeout' => -1 )));
$obj = json_decode($obj);
$id_stored = array();
$new_posts_id_array = array();
$new_json_posts_id_array = get_option('json_' . $json_options[$json_key]['name'] . "_post_ids");
$id_stored = get_option('json_' . $json_options[$json_key]['name']);
if(!$id_stored){$id_stored = [];}
foreach ($obj->products as $one_post) {
$post_char_id = $one_post->ID;
$new_posts_id_array[] = $post_char_id;
$cat_array = array();
if (!in_array($post_char_id, $id_stored)) {
$id_stored[] = $post_char_id;
update_option('json_' . $json_options[$json_key]['name'], $id_stored);
$post = array(
'post_title' => $one_post->name,
'post_status' => 'publish',
'post_author' => 1,
'post_content' => $one_post->description,
'post_type' => 'destinations',
);
$new_post_id = wp_insert_post($post); //post id
return true;
}
Chances are your script might be running out of time.
You could increase the max execution time using
set_time_limit(100); // 100 seconds
Or to make it indefinite, use
set_time_limit(0);
Note: Set the time on top of your script.