Custom taxonomies making WP very slow – Way to fix?

I’ve set up three custom taxonomies within my theme for attachment post types within WordPress 3.5.1. One taxonomy is non-hierarchical (i.e. tags) and has about 800 terms in it. (I’m using WP to build a quick & dirty stock photo site, and these taxonomy terms are akin to photo keywords. Yes, one could debate whether WP is the right tool for this…). Another of the taxonomies is flat as well and as yet has no terms in it.

In short, my site has become extremely slow. I’m not at all well-versed in MySQL or databases in general, but when I use the Blackbox plugin to get some insight into the underlying queries when pages load, I see that two queries in particular are adding huge overhead to my pages. Page request times are taking several seconds, and this wasn’t happening prior to the creation of the taxonomies. One query alone (below) is taking on average 1.4 seconds.

What’s going on, and is there a way to stop this? Is there a reason WP needs to make these expensive taxonomy queries on every single page?

Currently one query that WP is executing looks like:

SELECT t.*, tt.*, tr.object_id
FROM wp_terms AS t
INNER JOIN wp_term_taxonomy AS tt ON tt.term_id = t.term_id
INNER JOIN wp_term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id
WHERE tt.taxonomy IN ('myCustomTaxonomyName') AND tr.object_id IN (
    list of 200+ post IDs
)
ORDER BY t.name ASC;

And the other looks similar, except with different taxonomy names:

SELECT t.*, tt.*, tr.object_id
FROM wp_terms AS t
INNER JOIN wp_term_taxonomy AS tt ON tt.term_id = t.term_id
INNER JOIN wp_term_relationships AS tr ON tr.term_taxonomy_id = tt.term_taxonomy_id
WHERE tt.taxonomy IN ('attachment_tax1', 'attachment_tax2') AND tr.object_id IN (
    list of 200+ post IDs
)
 ORDER BY t.name ASC;