I have theme A currently active, and theme A has custom settings (like header image, custom CSS, etc). I am having issues with theme A, and want to activate theme B while I troubleshoot these issues. If I activate theme B, and go to activate theme A later, will my custom settings be preserved?
Will activating a different theme preserve custom settings to current theme?
Yes, your options should be preserved unless your themes are really bad and doing it wrong.
If the theme options are stored in the WordPress database, then the settings should be preserved. If they are stored locally – usually in a weird fashion (extra top level link in the administration menu instead of theme options menu item under appearance).
Many themes, even good ones, when they are deactivated and a new theme is activated will lose the widgets (which might be one type of setting). Some of those widgets will often be dumped to the inactive widgets area and you will have to re-drag and drop them back if you ever go back to that old theme.
There is a plugin that I use if I know I am going to be testing / refining back and forth between two different themes (for example if a client hasn’t made up their mind which theme or widget setup they favor).
the plugin is called Shiba Widgets aka WordPress Custom Widgets Plugin
Shib Widgets
It is not available in the wordpress repository, but it is free. It is overkill if you do not need it for this purpose.