The SEO Guide to a Successful Migration

Posted by Nick Redding
Last updated 24th April 2017

As one of the most daunting tasks a domain owner can undergo, you want to get website migration right the first time around. If something goes wrong, the downtime could cost your business dearly. But don’t panic. With our guide to website migration, your average customer won’t even be aware of the changes being made behind the scenes.

First off, there are five main types of migration:

  1. Hosting. The website moves to a new host, which will change the IP address
  2. Domain name or HTTPS. Site structure typically remains the same but the location changes
  3. Platform migration. The website moves to a completely new platform
  4. URL structure. The key change is the URL structure or site architecture
  5. Design. The structure and location of the website typically remains the same but a new design is launched

All types of migration require varying levels of consideration and planning to ensure the status quo is maintained, although the third type of migration is the most intensive from an SEO perspective.

Different types of migrations can also be combined within the same project, which can provide additional considerations. For example, if the domain is changing, it can be a good opportunity to add HTTPS or improve hierarchy or URL structure at the same time.

The recommendations below provide a checklist to cover all the key requirements.

The three phases of the migration process, which will be discussed below, are:

  1. Planning for the migration
  2. Migration and immediate steps following
  3. Post migration action and maintenance

One of the key requirements is the time allocated to the first phase. The earlier these considerations can be built into the migration process, the better we can mitigate any risk.

Benchmarking

Before a website migration project is undertaken, we recommend that clear objectives are established and the current performance is measured. Post migration, this will enable an accurate assessment of the success.

The following data points should be recorded:

  1. Website rankings in all countries (including any new ones being targeted)
  2. Organic traffic and the key landing pages
  3. Website load time performance (http://www.webpagetest.org/)

Phase 1: Pre-migration checks

Before site migration can commence, there are numerous checks that must take place to ensure the following phases run smoothly.

  • Block the new website in the staging environment
    • The staging environment should be blocked from being crawled by search engines using HTTP authentication (see documentation for Apache, NginX or IIS). This will require a user to enter a username and password before they can access it and will prevent search engines from accessing the staging environment.
    • IP exceptions can be implemented for external providers
    • The site can be blocked by the robots.txt file and on-page meta tag [noindex, nofollow] but these are higher risk and not the preferred recommendation
  • Build the new sitemap
    • If the URLs are changing, a sitemap for the new site structure will need to be created. This should list all the pages to be included in the new site structure and the corresponding URLs on the existing site.
    • Once the final list has been agreed and signed off by the relevant stakeholders, the URL mapping phase can then begin.
    • Moving forward without signoff will create additional work to revisit and adjust the redirect mapping at a later date and increase the chances of generating errors on launch.
  • URL mapping document
    • URLs will need to be mapped from the old URLs to the new URLs via 301 redirects (using the agreed destination in the sitemap)
    • Any that are not being carried over to the new domain will either need to be:
      • 301 redirected to the corresponding page
      • 301 redirected to the next relevant/close relevant page (if they currently receive traffic or have external links pointing to them)
      • Or if the content is being retired and has no place on the new site, then it should be left to 404
    • The list of key landing pages can be identified from the following sources:
      • Organic landing pages in Google Analytics and Google Search Console (GSC)
      • Crawling the website using external software
      • A database dump from the current CMS
      • Current XML sitemaps
      • Checking server logs (the last 30 days would be sufficient for sites not impacted by seasonality)
        Pages that receive external backlinks (either from GSC or third party providers such as Ahrefs.com or Majesticseo.com)
  • Eliminating redirect chains
    • To prevent unnecessary redirect hops, all the current redirects should be reviewed and mapped to point directly to the new URLs
    • Here, the aim is to ensure the new redirects are mapped on a 1-to-1 basis
  • Perform a crawl on the staging environment

Check to ensure the following has been configured correctly:

  • URL structure matches the agreed sitemap
  • Old URLs correctly 301 redirect to the new URLs
  • Old URLs are not linked to internally
  • Document markup; H1-H5, page titles, meta tags, canonical tags etc.
  • No internal or external 4xx, 3xx, 5xx or other crawl error status codes
  • Several crawls may be undertaken to ensure all issues are resolved before sign off
  • Populate HTML and XML sitemaps with the migrated URL destinations
    • Ensure all sitemaps on the staging environment reflect the correct URL and domain configurations
  • Create an old XML sitemap for any URLs to be redirected or retired
    • Create an XML sitemap containing the migrated or retired URLs. This will be used post launch to facilitate the crawling of the redirects and identification of intended 404s
    • This can be generated from the data collected during the redirect mapping phase
  • Ensure both the old and new domains are configured in the same GSC account (formerly Google Webmaster Tools)
    • The change of address feature will be used post launch to help Google understand the new configuration and crawl the old XML sitemap on the new domain
    • Set up an account before migration if required – https://www.google.com/webmasters/tools/home
  • Tracking
    • Add any tracking codes to the migration plan
    • Any conversion URLs or goals which are to be updated should be noted in the URL mapping document and created within the relevant tracking programs ahead of time, using exclusion rules to prevent tracking pollution on the development server.
    • Dummy accounts should be created to test on the development server o Ideally implement a tag management solution for more robust tag installation, change management, testing and advanced tracking implementations.

Phase 2: During Migration

The second phase involves the launch of the new site and the immediate actions necessary once live.

  • Migrate during a downtime
    • Analyse daily analytics to choose the best time to migrate the website. This will ensure the least amount of disruption to regular traffic if switch over is not immediate, which we expect it will be
  • Serve status code 503
    • If there is any unplanned downtime during the migration, a status code 503 should be served, which will temporarily block search engines from crawling the website without any penalisation
      It informs them that the server is currently unavailable and to return at a later date
  • Remove HTTP authentication and IP restrictions
    • Migrate the website from staging to production and remove any restrictions on accessing the website to facilitate crawling and indexing
  • Migrate the correct robots.txt file
    • If the robots.txt file on staging is used to block crawlers from accessing the site, ensure this isn’t migrated during launch phase
    • A new robots.txt file should be prepared beforehand, which can be replaced when the site is put live
      If secure authentication is used instead and the robots.txt is correct on staging, this can be migrated straight to the live environment
  • Check the website for any NOINDEX, NOFOLLOW directives that may have migrated over
    • If the noindex Meta tag has been used on the staging server to prevent indexation by search engines, ensure this tag is removed during the migration process
    • Manual checks can be undertaken immediately after launch, followed by a deeper crawl of the website to provide the all-clear

Phase 3: Post Migration

After the website has gone live, there’s still more work to be done. But the third phase will only begin once the website migration has been successfully completed.

  • Implement URL mapping recommendations
    • Once the website is live the 301 redirect plan should be deployed on the live site
  • Perform redirect acceptance tests
    • Ensure all old URLs 301 redirect to the correct landing page URL as per the mapping document
    • Errors should be minimalised from testing on the staging environment
  • Submit the new XML sitemap
    • Submit in the sitemap in GSC for the new domain to facilitate crawling
  • Submit the old XML sitemap
    • Submit the old sitemap to help Google crawl the old URLs and replace them with the new URLs in the search results
    • This can be removed once all new URLs have been successfully indexed
  • Fetch and render URLs in GSC
    • This tool allows you to render pages as Google sees them and can aid in any necessary troubleshooting
    • The key element to consider here is the blocking of JavaScript and CSS
    • These checks should be performed across multiple page templates to ensure all potential errors are discovered
  • Migrate settings from the old GSC account
    • Country settings, Disavow file and parameter settings
    • This is applicable if you are moving domains or from HTTP to HTTPS
  • Run a full site crawl to determine any issues:
    • 404s, 302s, 301s, 500 errors as well as URL structure checks
    • Page titles and Meta descriptions, canonical tags and robots meta tags
    • Analytics and tracking codes
  • Change of Address in Google Search Console
    • Ensure the new domain is verified
    • Use the change of address feature to indicate to Google that the site has moved to a new domain (if applicable)
    • 301 redirects need to be implemented first before this function can be used
  • Perform server acceptance tests (SATs)
    • User/bot requests: https://www.example.com/this-should-404
      • Expected response: status code 404
    • User/bot requests: https://www.example.com/this-should-404/
      • Expected response: status code 404
    • User/bot requests: https://www.example.com/folder/fsdfhskdj
      • Expected response: status code 404
    • User/bot requests: https://www.example.com/folder
      • Expected response: status code 301
      • Redirect to: https://www.example.com/folder/
    • User/bot requests: https://www.example.com/FOLDER/
      • Expected response: status code 301
      • Redirect to: https://www.example.com/folder/

Note – You may have specific rules that determine how your server should respond. The ones provided above are for the example purposes. You should endeavour to build your own SATs based on existing configurations.

  • Update external links
    • Update any landing pages for active campaigns, such as PPC / Email / Affiliates to point to URLs on the new site
    • Update social media profile links to point to the new domain
    • Identify any high authority backlinks pointing to the old domain and reach out to webmasters, requesting they update the link to point to the new domain
  • Ensure manual checks are completed on key features
    • Submit contact forms to check messages are received, details are captured and conversions are accurately recorded
    • Make a test payment for any payment feature
    • Check that tracking works correctly using real time tracking if available, check HTTP requests using a browser network sniffer or re-check once data is available
  • Undertake site monitoring for next two-four weeks to ensure all has gone smoothly, quickly and pro-actively addressing any identified actions
    • Review crawl errors, index status and sitemap indexation in Google Search Console daily
    • Monitor any errors, such as 404s in server logs
    • Monitor keyword rankings determined during the benchmarking stage
    • Review organic traffic and key landing pages once redirects have been processed

Need any help with your website migration? Contact Reddico today.

Posted by Nick Redding