wp_remote_get(), downloading and saving files

From what I can see wp_remote_get() saves remote file contents to memory.

The files that I need to download are compressed in either ZIP or GZIP and within which will either be a CVS or XMl file

Read More

What I need to do first is download the remote file to the harddrive as ZIP either GZIP and then unzip them

Is it possible to use wp_remote_get() to download the whole file and save it to a directory?

A non-Wordpress solution I used before was cURL:

public function grab_file($url, $new_file) {

    //get file
    $ch = curl_init();
    $fp = fopen(DIR_PATH."zip/$new_file", "w");

    $options = array(CURLOPT_URL => $url, CURLOPT_HEADER => 0, CURLOPT_FAILONERROR =>
        1, CURLOPT_AUTOREFERER => 1, CURLOPT_BINARYTRANSFER => 1, CURLOPT_RETURNTRANSFER =>
        1, CURLOPT_FOLLOWLOCATION => 1, CURLOPT_TIMEOUT => 5, CURLOPT_FILE => $fp);

    curl_setopt_array($ch, $options);
    $file = curl_exec($ch);
    curl_close($ch);
    fclose($fp);

    if (!$file) {
        //$error = "cURL error number:" . curl_errno($ch);
        //$error .= "cURL error:" . curl_error($ch);
        return false;

    } else {

        return true;

    }
}

Related posts

Leave a Reply

2 comments

  1. Well here’s the solution that I came up with:

    // Use wp_remote_get to fetch the data
    $response = wp_remote_get($url);
    
    // Save the body part to a variable
    $zip = $data['body'];
    
    // In the header info is the name of the XML or CVS file. I used preg_match to find it
    preg_match("/.datafeed_([0-9]*)../", $response['headers']['content-disposition'], $match);
    
    // Create the name of the file and the declare the directory and path
    $file = DIR_PATH."zip/"."datafeed_".$match[1].".zip";
    
    // Now use the standard PHP file functions
    $fp = fopen($file, "w");
    fwrite($fp, $zip);
    fclose($fp);
    
    // to unzip the file use the WordPress unzip_file() function
    // You MUST declare WP_Filesystem() first
    WP_Filesystem();
    if (unzip_file($file, DIR_PATH."feeds")) {
    // Now that the zip file has been used, destroy it
    unlink($file);
    return true;
    } else {
    return false;
    }
    

    Dealing with a GZIP file was a bit different:

    // It is necessary to use the raw URL and find the extension of the encluse XML or CVS file first
    private function save_ungzip($data, $url, $ext) {
    
    // As with ZIP, save the body of wp_remote_get() to memory
    $gzip = $data['body'];
    
    // As with ZIP, look for the name of the file in the header
    preg_match("/.datafeed_([0-9]*)../", $data['headers']['content-disposition'], $match);
    $file = DIR_PATH."feeds/"."datafeed_".$match[1].".$ext";
    
    // now you need to use both the PHP gzip functions and the PHP file functions
    $remote = gzopen($url, "rb");
    $home = fopen($file, "w");
    
    while ($string = gzread($remote, 4096)) {
        fwrite($home, $string, strlen($string));
    }
    
    gzclose($remote);
    fclose($home);
    
    }
    

    So in conclusion. When you use wp_remote_get() to retrieve a remote file use the inbuilt PHP file functions to then save it where you want to. If you have a solution that utilizes WordPress functions then please post it up.

  2. Althoug this question is from a few years back, since it pops up in Google as one of the first results, I thought I’d give a better suggestion to write wp_remote_get()‘s response to a file.

    The beginning is good, but the OP uses fopen() and fwrite() to write the file, while that’s not necessary:

    // Use wp_remote_get to fetch the data
    $data = wp_remote_get($url);
    
    // Save the body part to a variable
    $zip = $data['body'];
    
    // Then the unzipping part...
    
    // Create the name of the file and the declare the directory and path
    $file = '/path/to/file.zip';
    
    // Write the file using put_contents instead of fopen(), etc.
    global $wp_filesystem;
    
    $wp_filesystem->put_contents($file, $zip);
    

    In some cases $wp_filesystem isn’t accessible. In which case you can create a function to get it:

    function filesystem()
    {
        global $wp_filesystem;
    
        if ( is_null( $wp_filesystem ) ) {
            require_once ABSPATH . '/wp-admin/includes/file.php';
            WP_Filesystem();
        }
    
        return $wp_filesystem;
    }