WordPress: Query Posts without allocated taxonomies

I would like to filter all post queries by a specific term of a custom taxonomy (specified by an url). Currently I’m using add_filter(‘request’, …) and ‘tax_query’, which kind of works now.

So I get all posts of this term, but I would like to get those posts, which do not have ANY terms allocated of this taxonomy.

I was trying something like:

$request['tax_query'] = array(
    'relation' => 'OR',
    array(
        'taxonomy' => 'brand',
        'field' => 'slug',
        'terms' => array( $term )
    ),
    array(
        'taxonomy' => 'brand',
        'field' => 'slug',
        'terms' => NULL,
        'operator' => 'IS'
    ),
);

But, since WordPress doesn’t support ‘IS’ or ‘IS NULL’ this doesn’t work either.

So, any idea how to query for posts of a term AND posts without a term at all?

EDIT: Another solution would be to query all terms, an exclude it then in tax_query. But I would like to save this query.