WordPress: Filter posts in category by post format

I have a site that has multiple categories/tags of posts. Within each of these categories/tags are posts with different post formats (audio, video, link etc).

What I want to be able to do is have some sort of simple menu bar that lets me display just the posts with the selected post formats inside of that category.

I can do this manually using

$args = array(
'post_type' => 'post',
'post_format' => 'post-format-image',
'paged' => $paged, 
'posts_per_page' => of_get_option('homepage_num_posts'),
  );
query_posts($args);

however that would mean i would need to create a lot of pages as there are dozens of categories/tags.

I have tried using get_query_var like below on the archive page (there are no category pages)

'post_format' => get_query_var('format'),

, however using a link like site.com/category/beef?format=post-format-image does nothing

Is there a simple solution to this, or am I doing it wrong?