How do you remove slug altogether from custom post type?

For a custom post type page I’d like to completely remove the slug so that the URL structure mirrors http://my-domain.com/CPT-title/ or http://my-domain.com/CPT-category/CPT-title . If I set the rewrite parameter to

'rewrite' => array('slug' => '', 'with_front' => false)

WordPress freaks out and I get 404 on all pages except for the CPT page. If I don’t define slug at all, then it defaults to the CPT name.