This issue applies to SharePoint 2010 as well but…suppose you have created a custom claims provider and one of the things you want to do is to have some custom claim types that you use. What I mean by custom claim types is just that they are not one of the standard out of the box types like email, upn, role, etc. Now in this posting here: http://blogs.technet.com/b/speschka/archive/2011/04/02/how-to-add-additional-claims-in-adfs-2-0-that-can-be-consumed-in-sharepoint-2010.aspx – from way back in 2011, I described the format that you need to use for your claim type in order to be accepted by SharePoint.
Suppose that you have built and deployed your custom claims provider though and you decide you really need another claim type – we'll call it http://www.foo.com/bar. So you start using the claim by creating a new SPClaim and that works just fine. But then you want to use the claim with something where you require an encoded value – like adding it to a web app policy or a SharePoint group or whatever. So you create an instance of the SPClaimManager and you try the EncodeClaim method on the value of your SPClaim. Instead of happiness, what you get is an error that says you have an ArgumentException, the param name is claimType, and in the stack trace you see it happened in the EncodeClaimIntoFormsSuffix method.
You may think hey – I just need to add this claim type to the list of claims my provider supports. I do that by modifying my code in the FillClaimTypes override of my custom claims provider. That actually is correct – you DO need to do that. So you do that, recompile and redeploy the assembly…and continue to get the same error. Well the fact of the matter is that you've done everything correctly from a coding standpoint. The problem is that this is a different flavor of the same problem that SharePoint SPTrustedIdentityTokenIssuers have, which is the claims collection is immutable. That just means that after you create the claims mappings for an SPTrustedIdentityTokenIssuer, you can not go back and change it afterwards. Yes, I know people have posted code on how to do that; it is also unsupported to do so.
So, how do you fix this? If you are NOT the default claims provider, you should be able to just uninstall and remove the custom claims provider feature, remove all instances of it from the GAC and deploy all over again. If you ARE the default claims provider, then unfortunately the fix is the same way you would as if you needed to change the claims mappings for your SPTrustedIdentityTokenIssuer. You need to change the configuration for any web apps that are using your SPTrustedIdentityTokenIssuer so that they no longer use it, and then remove your custom claims provider. Then you need to remove the SPTrustedIdentityTokenIssuer, redeploy your custom claims provider, recreate your SPTrustedIdentityTokenIssuer, make your custom claims provider the default provider, then go back and add the SPTrustedIdentityTokenIssuer to the web apps that need it. It wears me out just typing out that sentence, let alone doing the work.
Once you've done your redeployment though, SharePoint should pick up on your new collection of claims that you will be using. Until you need to change them again.