How do I create an authentication system for an external application (without using WordPress internal files) using the cookie stored by WordPress? A pointer on the working of WordPress Cookie Authentication would be helpful also.
WordPress External Cookie Authentication
wp-includes/user.php is where authentication is handled. You might be particularly interested in looking at the function wp_authenticate_cookie() to get an understanding of how auth works …
Also take a look at this article, which covers wordpress authentication.