Are my old websites being properly redirected?

Question

About a year ago I merged three websites (oldsite.com, oldsite.nu, newsite.se) into one, which I kept on one of the domains (newsite.se). I am not sure if this has been done right since I still see a lot of traffic from Google for old url:s, even after a year.

Oldsite redirect code

---

Important edit note: I recently realized the nameservers were not pointing towards my old rails app any longer but instead to a php-folder on my web host in which I have a .htaccess with the following code:

RewriteEngine on
RewriteRule ^robots.txt - [L]
RewriteRule ^sitemap.xml - [L]
RewriteRule ^(.*)$ http://www.newsite.se/$1 [R=301,L]

---

This makes this section below (regarding oldsite.com/oldsite.nu) void: The .com and .nu were built in Ruby on Rails and hosted on Heroku.

The logic to redirect paths from oldsite.com/oldsite.nu were made completely on the newsite.se site. The redirection code on the oldsites is a straightforward redirection with this on the first row in routes.rb on oldsite.com:

  match "/(*path)" => redirect {|params, req| "http://www.newsite.se/#{params[:path]}"},  via: [:get, :post]

I used this (Swedish) tool to verify that this redirect actually makes a 301 redirect: http://301redirect.se . It confirmed the redirections were 301.

Newsite.se redirection handler

The content on each old website were matched with the same content on the new one, quite rarely on the same path, e.g.

oldsite.com/categories/vacation/item/1243

could lead to

newsite.se/product-items/1243

I handle these types of redirections mostly in an internal redirection controller that catches and redirects any traffic on newsite.se like:

newsite.se/categories/vacation/item/1243 -> newsite.se/product-items/1243

using this at the bottom on my newsite.se routes.rb:

match '*not_found_path', :to => 'redirections#not_found_catcher', via: :get, as: :redirect_catcher, :constraints => lambda{|req| req.path !~ /\.(png|gif|jpg|txt|js|css)$/ }

This works fine.

Edit 20151223: The reason I use Newsite.se to handle the redirects is because it holds all the logic of where to redirect the paths. This is virtually impossible for Oldsite.com/.nu to know.

Actions taken

Outside of redirecting with 301 (as far as I understand, I do). I have also used Google Webmaster Tools to make a "Request to change address" from my old two websites to my new one. I can't find any information on this any longer but I am quite sure I got a positive response from WMT this hade been done (but I am not 100% sure).

The problem indications

I am not 100% sure there is something wrong but I have seen indications that makes me believe the redirection is not made properly so that Google really realize the websites are not moved.

• In Google Webmaster Tools, and "Incoming links" the top link domain is herokuapp.com which in term means oldsite.com. I.e. the 301 redirects seems to be interpreted as links (and not as redirects).
• I often get new indications on Google WMT about "Not founds/404's" (don't know what this section is called in the English version) for url's that could not be reached on newsite.se. When I check the source of those url's I often see links from e.g. oldsite.nu/oldpath/productitem/1234 - like someone (Google?) still have accessed that old url. An important part of this is that I did NOT have that many links to the old sites so I don't expect these to be from old links still feeding traffic.
• I still get traffic to many of my old paths (from oldsite.com/oldsite.new). I find this through my redirection controller which handles plenty of requests on old paths every day.
• The website have lost a lot of positions in Google SERP, this is only a weak indication though since there could be numerous reasons for it.

Solving the problem

• How should I go about to trouble shoot this problem?
• Is it normal for WMT to consider 301's as links?
• Is there a smarter way to handle the redirection from oldsite.com than my routes.rb-match line?

Answer 1

I had to do a similar move for a client moving a large e-commerce website that required transitioning all old traffic to the new website and redirect the appropriate products to the new pathing.

In order to get everything transitioned so we didn't lose Google Ranking we had to implement 301 redirects as you mentioned above. In the WMT they seem to rely on you to handle this instead of have it as a supported function anymore.

Approach

You should redirect every URL on your old domain to the corresponding new URL. That is the documented & recommended way of changing your domain according to Google.

The best approach would be to handle the redirects in a controller and have the logic to send it to the actual page with a 301 and no more redirects once landing on the new website.

I would suggest the following:

routes.rb (oldsite.com/oldsite.nu)

Match the request and send it to a controller to handle the finer logic and 301.

match "/(*path)", to: 'redirector#catch_all',  via: [:get, :post]

RedirectorController (oldsite.com/oldsite.nu)

def catch_all
    # Separate the rest of the link into its components
    # I will use the example of the vacation -> product-items you have
    options = params[:path].split('/')
    options.reject! { |e| e.to_s.empty? } # Remove trailing junk if any
    if options[0] == 'categories'
        redirect_to "http://www.newsite.se/product-items/#{options.last}", notice: 'Product Updated! We Have A New Site.', status: 301
        return # the return here is a MUST for more complex if-then and multiple redirect_to's
    elsif options[0] == 'contact'
        redirect_to "http://www.newsite.se/contact-us", notice: 'We moved, Contact us Here.', status: 301
        return
    elsif options.empty? || options.blank?
        redirect_to "http://www.newsite.se/", notice: 'That page no longer exists, Please browse our new site', status: 301
        return
    else
        more_sorting(options)
    end
end

private

def more_sorting(options)
    case options
    when options[2].....
        redirect_to ....., notice: '', status: 301
        return
    when options[3].....
        redirect_to ....., notice: '', status: 301
        return
    when options[4].....
        redirect_to ....., notice: '', status: 301
        return
    end
end

Why do it this way:

This will cause the search engines robots, and users to still be able to crawl and visit each page and link and get redirected to the specific page it is associated to on the new website.

Further it handles the 301 redirect on this server and does not result in another redirect on the new server. Something you may be penalized for from both and user experience and robot interpretation of you're attempts to unite the sites. (this will also most likely remove the interpretation of link 301's)

If you need more complex routing you can add (as I had to do) private functions in the RedirectController for more in depth analysis of the parameters that I had as the last else in the if then .

Clarification?

Let me know if you have any other questions and if this helped.