Optimizely CMS & Commerce .NET Core migration - Tips & Tricks

Posted by ddprince on January 10, 2023

Last September, we had an extraordinary opportunity to share our findings and experience migrating our first Optimizely solution to .NET Core. This event occured at the Optimizely Dev Meetup in Montreal. I was presenting alongside my colleagues Pierre and Eric. So, as the title suggests, our goal was to give tips & tricks on the migration of an Optimizely solution to .NET Core. This effort obviously came with some challenges!

Here is a vague overview of the steps the process involves:

  1. Preparation
  2. Project conversion
  3. Project runtime stabilizations
    1. First boot stabilizations
    2. Runtime stabilizations
    3. Deployment pipelines
  4. Regression fixes
  5. Quality assurance
  6. Go live

Preparation

You will need to plan the time required to do the migration:

  1. Implies you will prepare your team and your client of preferably freezing other development efforts during the migration
  2. Prepare your people: You should have a consistent team of developers, not planned to switch between projects during the migration
  3. Limit the size of the team: You can't have that many people at the same time on the migration itself. Sure, you could give a development boost at the end, but you could also be a victim of some pitfalls related to this method, by example, duplicating the efforts; Having more than one developer working on the same thing without knowledgeably be aware of it.
  4. Migrating a Commerce site? Plan for the removal of Commerce Manager; It's no longer existing in .NET Core.
  5. Make an inventory of your project dependencies - You need to quickly find any dependencies requiring a replacement. Your dependency tree should be as simple as possible to minimize the complexity when migrating from packages.config to PackageReference. So, the elements to verify on this point are the following:
    1. Project dependency hierarchy
    2. NuGet package dependencies
  6. Have yourself a recent database backup of production. Please make sure to make data sanitization first to avoid having personally identifiable information.
    1. We did this mistake and fell into the trap. We had a very old development database that was way too different than production. This caused one of our database migrations to fail because of a schema difference when we have deployed thereafter on PreProduction

Lessons & learned: The most overlooked part of this section were the dependencies of our solution. We learned at the project runtime stabilization step that some modules/plugins/dependencies weren't compatible anymore under .NET, so we had to find alternatives. Some of which weren't available except by re-implementing them from scratch. This is by far one of the biggest bottlenecks of a migration. We had to put our own efforts to migration certain plugins, e.g., a color picker.

Optimizely also switched of image processing plugin from .NET Fx to .NET. It's now referring ImageSharp. You will have to consider changing all transformation logics inside your solution to something else, preferably using ImageSharp.Web.

Project conversion

You will have to convert all projects to the new csproj format. This process can be simplified using the .NET Upgrade Assistant. Refer to the Optimizely documentation for more information. They also have created an extension over the upgrade assistant which is going to help you during the migration.

The goal of this step is to convert all projects of the solution as quickly as possible to the newest format. Keep in mind this operation can only be done by a single developer or by pair programming if necessary. You can't have two developers on such major changes, you will simply end up with Git conflicts while trying to merge. So, typically, a single developer per project is recommended on this phase. Centralize your efforts on the migration, but not on making sure the site is compiling, it's expected not to compile after the first time the upgrade assistance as been ran.

As soon as all csproj are converted, you can easily share the rest of the conversion with all developers. From that point, you will have a lot of compilation errors. Here's a simple representation of items you need to check per project:

  1. Simplify the referenced packages.
    1. Only add those who are necessary, they rest should be including implicitly (Transitive dependency). Up until recently, you had to manage this per project, but now, with the centralized package management, you can make sure the same package versions are used within the whole solution.
  2. Replace .NET Fx specific libraries to their .NET variants. E.g., Remove Microsoft.AspNet.Mvc and make sure only Microsoft.AspNetCore.Mvc is installed, either implicitly, or explicitly if desired
  3. Clean stale code - Remove everything that you don't need, avoid unnecessary conversion work on something that isn't being used at all.
    1. This effort should be taken up front, since the beginning of the creation of the project. Leaving stale code inside a solution is just a way to confuse other developers and make everyone loose time unnecessarily.
    2. If you want to keep a code snippet, you should save it inside a Wiki note instead for further reference. In my opinion, your code repository isn't the place to save code snippets hidden in comments.
  4. Make all the required remaining adjustment to make the project compile (not the solution, just the project).
  5. Ensure all unit and integration tests passes as early as possible. This will help you quickly find any further regressions if applicable.

Developers has the habit to comment some part of the code when testing any changes, they've made. It's sure is a usual development style some people use, but do not take this reflex to comment everything just to reach your goal to compile the project. This is totally counter intuitive and you might even end forgetting critical parts to be migrated later.

As for database migrations, we followed Microsoft's recommendations and removed all past migrations. The only thing that left was the database context and the database models. Technically speaking, they are already deployed to production, so they are no longer necessary anyway. Adding more migration will simply create a difference with the existing schema and your modification.


Project runtime stabilizations

First boot stabilizations

Once all projects are compiling, you will then probably face a lot of runtime exceptions, especially at the first boot. Here are the commonly found issues we faced during our migration:

  • Configuration issues. Web.config/app.config is no longer a thing under .NET. You will have to migrate your configuration and you might face issues in relation to this.
  • IoC issues. You will probably forget to register a service or two. This happens, but it's quite simple to resolve usually speaking.
  • MVC issues. Due to the nature of the migration and especially the fact the core MVC library changed, you might face issues with your views, your customizations around it and surely refactor a lot of @helper usage
  • Your app is hosted in DXP? Prepare yourself to fix certain environment related hurdles, because it will be running under Linux. E.g., adjusting file paths, properly respecting their case.
  • Database migrations: You might face hiccups with them. To this point, it's best to test your migrations with the command line dotnet ef. You'll find all necessary resources to debug what's causing the hitches.

Runtime stabilizations

After the first successful boot, you will surely face other complications that will require your team's attention. Notably:

  • Broken CSS styling
  • Broken JavaScript
  • Broken pages, e.g., certain content type could be causing a crash, but only at runtime

Keep a good attention of your browser developer console, you'll probably find a couple of errors here and there, mainly responsible because the default JSON serializer is now using camel case instead of pascal case under .NET Fx.

It's very important at this phase that all developers work conjointly to find as much bugs as possible. Stack them as much as you can into the backlog. It's expected there will be a lot, so at least you can track their progress and present them to your QA team once you will reach the quality assurance phase.

DevOps Automations

In parallel, another person can work on adjusting your DevOps automations. Typically, the biggest part that will require many efforts will be the application compilation itself. Since we're switching from .NET Fx to .NET, it's best to switch using the .NET CLI under your scripts. Another thing to keep in mind; If you're deploying in DXP, make sure every package name are unique, otherwise you will receive an error stipulating the package has already been uploaded. In .NET, Optimizely prevents duplicates. The advantage though is that you can reuse an existing package to re-make a deployment without the need to re-upload it first.

In summary:

  • Use .NET CLI
  • Do not get mixed up with old *.config configuration and the newest appsettings.json one. The old configuration isn't working anymore. If you prefer, you could simply delete these old files in your repository to prevent any confusions.
    • Setup your configuration transformations if necessary. You can customize them per environment by creating an additional file inside your solution, e.g., appsettings.Integration.json
  • DXP: Package your application using linux-x64 architecture
  • DXP: Make sure your application is referring the mandatory CMS and Commerce Cloud packages.
  • DXP: Package version must always be unique. If you intend to use an existing version, simply leverage the name of the package already uploaded to start a new deployment.

Regression fixes

To this point, you've gathered an enormous quantity of bugs inside your backlog. You now want the full team to fix them all together. Obviously, you will want all majorly impacting bugs to be fixed first, especially the ones causing crashes, until you reach certain nebulous bugs where you can't even tell if they've been provoked by the migration or if it was already there to begin with.

Couple of things to keep in mind:

  • Use each of you team members strength to tackle the bugs. This will fasten the correction and will ultimately give you more time on the most complicated ones.
  • You will have to replace and/or backport any unsupported plugins, if necessary
  • Find & fix any Backoffice related issues (under ~/Episerver/Cms). We had some troubles at the time, but I doubt it will be the case today.
  • Since your DevOps automation should be already working to this point, our recommendation is to deploy as many iterations of the site as possible. This will allow your team to quickly find more bugs as other items are being fixed in parallel.
  • By the end of this adventure, you will want to simulate production installations to prepare yourself for the Go-Live. Goal here is to make sure the current production database state is still handling the migrations and the switch to .NET transparently. If your application is hosted under DXP, you can easily leverage the available "Project Migration" tab under the Optimizely PaaS portal to restore as much time as you want your existing .NET Fx production environment to your .NET production one.

Quality Assurance

You're near the end. Your team of developers has found and fix every possible bug, now it's turn for your QA team to double check the stability of the whole site. They should get back to the reported bugs inside the backlog and verify every one of them to make sure they're fixed. In any case, they can reopen them if any issue arises. Also, very important they make a full regression testing, independently of the number of bugs logged or not, they want to verify every part of the site, even the most unused sections.

Your QA team will then decide together whether the stability of the site is worthy for production or not. In all scenarios, they will probably log more bugs inside the backlog for the developers to look for them afterwards. If they decide the site is ready to be released, a code freeze should be put in place to prevent any further change that could cause unwanted regressions. If in contrary they find the site too unstable for their taste, they should put the developers precisely on priority items that are blocking the release to production. Other nonessential and non critical bugs that can be fixed after, should be postponed.

In the meantime, another of your team member should have already started communications with Optimizely's team to prepare for the Go-Live. A release plan should detail every expected step your team needs to take for it. You also want to coordinate your DNS switch to your newest .NET application instance.

Go-Live

You've tested over and over, and your QA team is satisfied with the stability of your site, you are now ready for Go-Live. Even thought you've made your best to test everything, you should still expect things can go wrong during the procedure. Fortunately, if in any ways something goes bad, you can still use the old production environment, which should still be intact.

A general idea of the steps to take:

  • Keep old .NET Fx environments intact
  • Put maintenance page on .NET Fx production environnement
  • Deploy to new .NET production environment
  • Verify stability and every key element, e.g., capable of taking an order
  • Switch DNS to new .NET production environment
  • Monitor traffic & logs in case you missed something very important and that you need to make a hotfix

Conclusion

All in all, I loved the experience, it was quite a challenge! The things all developers loved the most during this adventure was the speed the site took to load. It's so fast, it's extremely unrecognizable from the .NET Fx variant of the site. Optimizely also published some metrics on the subject and I'd say they aren't far fetched at all.