Using S3 as a fallback with nginx fails on first “few” loads

I have a wordpress site on a multi-server setup (EC2), and I move all image uploads to S3. I have set up nginx this way to redirect image requests to S3:

location ~* .(js|css|png|jpg|jpeg|gif|ico)$ {
    expires max;
    log_not_found off;
    add_header Cache-Control public;
    add_header Pragma public;

    error_page 404 = @s3blog;
}

location @s3blog {
    proxy_buffering        on;
    proxy_intercept_errors on;
    proxy_hide_header      x-amz-id-2;
    proxy_hide_header      x-amz-request-id;

    resolver 8.8.8.8;
    proxy_pass $scheme://my-bucket.s3.amazonaws.com;
}

This only works after the first load of the page. I get complaints very frequently from the people in charge of the blog that there are broken images. When I check it out it’s fine but that first load doesn’t work. The server uses fastcgi for location ~ .php$.

Is there anything I’m doing wrong? Thanks in advance