I am working on a Pinterest clone in WordPress at the moment.
I want to know how Pinterest manages to get their site indexed by search engines when you have to log in to see the pin screens?
So, I need a solution where users have to log in to the site, but search engines can still index everything without logging in. Is this technically cloaking?
Any ideas?
You can do this in your script by using a reverse DNS lookup. Google suggest this method themselves. With PHP you can do this using
gethostbyaddr().The fastest way is to check the user-agent string. The full list is at http://user-agents.org/.
If you try the DNS method suggested by Leonard, you’ll find it’s extremely slow. Some IPs don’t even have valid reverse lookups. The page in question recommends a reverse lookup only for verifying the origin of a request that claims to be googlebot in the user-agent string. You shouldn’t worry about people hacking their browsers (pretending to be the googlebot) until you are popular.