If you have extensively customized your SharePoint sites (by using Microsoft Office FrontPage for example), you must decide how to handle your customized sites when you upgrade. Your approach will vary based on the extent of the customizations, the complexity of your site, and your goals for upgrading. Before you upgrade, it is recommended to go through the following article on Microsoft Web site.
Upgrade considerations for customized sites
If your sites are based on customized site definitions, we recommend that you create a new site definition before upgrading, and then create a mapping file so that the upgrade process can map your old site definition elements to the new site definition. Please refer to the articles below for get more details:
Develop new custom site definitions and create upgrade definition files (Office SharePoint Server)
Deploy upgrade definition files and new site definitions [Office SharePoint Server]