Add quick edit fields without custom colum

This question is a follow up on that answer I wrote.

Let’s say I want to add something to the quick edit box for the Posts post type list table. To do so, I have to add a custom column because of the following part in the class-wp-posts-list-table.php file in core:

foreach ( $columns as $column_name => $column_display_name ) {
    if ( isset( $core_columns[$column_name] ) )
        continue;
    do_action( $bulk ? 'bulk_edit_custom_box' : 'quick_edit_custom_box', $column_name, $screen->post_type );
}

As you can see, the filter for custom quick/bulk edit fields won’t trigger if there’s no custom column registered. Now this, this and this answer suggest to either just add or add and then remove a dummy custom column.

So I added a dummy custom column:

$pt = 'post';
add_filter( "manage_{$pt}_posts_columns", array( $this, 'addColDummy' ) );
# add_action( "manage_{$pt}_posts_custom_column", array( $this, 'removeColDummy' ), 10, 2 );
# add_filter( "manage_edit-{$pt}_columns", array( $this, 'remove' ) );

Problem now is: If I then remove the custom column again from within one of those two filters above, the custom code doesn’t get triggered anymore. Question: How can I remove the dummy column without removing the custom quick edit stuff again?