WP_query posts closest to todays date

I have written an wp query which I would like to order in a bit of a complicated way. Some posts have a meta value ‘the_date’ which is an end date for the post. I would like to order posts in date and time order where if there is an end date this would be the reference how ever is there is no end date the publish date would be the reference. With this information I need to order posts by the closest to the current date and time. For example:

posts:

ID          Name         Publish_date        Meta_end_date
1           Post 1       15/05/2013 10:00   
2           Post 2       15/05/2013 09:00    17/05/2013 14:20
3           Post 3       16/05/2013 13:00    20/05/2013 08:00
4           Post 4       17/05/2013 13:50

current time: 17/05/2013 14:00

Result order:

Post 4 (17/05/2013 13:50), 
Post 2 (17/05/2013 14:20), 
Post 1 (15/05/2013 10:00),
Post 3 (20/05/2013 08:00)

Here is my current query:

$args = array(
    'post_type' => 'post',
    'numberposts' => '30',
    'meta_query' => array(
        'relation' => 'OR',
        array(
            'key' => 'the_date',
            'value' => $today,
            'compare' => '>',
            'type'=> 'DATETIME'
         ),
        array(
            'key' => 'the_date',
            'value' => '',
            'compare' => '=',
        )
    ),
    'orderby' => 'post_date',
    'order' => 'ASC',
    'orderby' => 'meta_value_num',
    'order' => 'DESC'
);

Any help is very welcome 🙂