There are two types of encryption: one that will prevent your sister from reading your diary and one that will prevent your government.
Bruce Schneier
Encryption problems don’t occur because Microsoft Dynamics CRM encrypts password fields. Problem’s occur if you don’t realise CRM 2013/2015 database are encrypted and encryption keys need to be looked after.
CRM 2013/2015 and future versions automatically creates a default encryption key and encrypts data in CRM.
Understand how database encryption works to ensure they don’t lose the encryption key or create a situation where they don’t know where the original encryption key came from. Retrieving encrypted data isn’t possible with the encryption key, deletion and recreation will be your available options.
Along came an SQL Encryption error
In a CRM 2013 dev organisations clicking on mailboxes would bring up an SQL encryption error
No one turned on database encryption, so this raised questions
- Why were SQL encryption errors appearing?
- Why in one of our dev organisations?
Database encryption was introduced in CRM 2013, automatically enabled and encrypts email password fields.
Developers won’t notice database encryption in a new CRM organization because an encryption key is created for you, all functionality works and mailbox passwords encrypted.
It wasn’t until we created a DEV environments from a copy of a CRM 2013 org and restored it, SQL encryption errors appeared.
If you attempt to look at the database encryption settings on an http on-premise organisation you will get an SSL error.
It’s not uncommon to find the CRM developer bashing head on desk whilst pulling out his hair.
When you work out how to turn the SSL check, you then get presented with a blank certification field with a helpful place to past the encryption key? Erm……what, where, who is the encryption key (repeat head banging).
If you are lucky the encryption key is in the original database, Other scenarios could be the original database is deleted or you can’t remember what CRM org was the original.
Why it’s important for Developers to know
Database Encryption gets turned on automatically, if you backup/copy and restore a CRM organisation the copy database won’t have database encryption key. It’s easy to copy the database encryption key once you know where to look, it’s important to plan because encryption keys can change or get lost.
What won’t work if you don’t have your database encryption key?
- Server side Sync
- Mailboxes (opening)
- Cannot update user email addresses
- Cannot update mailbox emails
If server side sync doesn’t work workflows sending emails might not work
You cannot change emails of users and mailboxes.
The biggest problem I experienced was the loss of time. Time investigating the problem and understanding database encryption. The investigation stopped development on areas which needed email functionality using server side sync.
The forum title below captures the feeling
HELP! My data is encrypted and I didn’t do it!
It’s important for CRM developers to understand database encryption in Microsoft Dynamics CRM is because it’s automatically turned on and cannot be disabled.
If you restore a CRM database the database encryption won’t be activated, encrypted data isn’t available to your CRM instance, certain email functionality won’t work, throwing SQL encryption errors.
Restoring databases can happen when creating new development environments, QA, TEST, PREPROD environments.
- The pain of setting up CRM dev environments
- How to create realistic Test Data for your CRM Project and why you should
If you create a new CRM organisations for each environment, each CRM database is automatically encrypted with its own key.
If you copy and restore CRM organisations you MUST copy the database encryption key from the original CRM database and activate the database encryption. CRM developers need to be proactive and mange the encryption keys, to avoid ending up in a mess.
Why doesn’t the encryption key copy
The data encryption key is stored in the MSCRM config database and not in the instance CRM database. So when the bak file is imported into a new CRM org it doesn’t know how to get the key because the key isn’t in the bak file.
It can’t get it from the MSCRM_config because by creating a new org you will create new /different data in the MSCRM_config. Each CRM database will have its own encryption key.
What is Data encryption
The CRM SDK has two articles on database encryption, The first describes what SQL field level encryption is and does.
Microsoft Dynamics CRM uses standard Microsoft SQL Server cell level encryption for a set of default entity attributes that contain sensitive information, such as user names and email passwords. This feature can help organizations meet FIPS 140-2 compliance.
This article explains changing and copying your encryption key.
A 5 minute video to explain the basics of database encryption
Microsoft Dynamics CRM 2013 Setup and Upgrade New Features – Data Encryption
The video states Yammer tokens are encrypted, I’m not sure what this means or if Yammer won’t work on a database which isn’t encrypted,
What is encrypted?
Passwords for email are encrypted but what does this mean?
The article describes what is encrypted
| Entity | Attribute |
| EmailServerProfile | IncomingPassword |
| EmailServerProfile | OutgoingPassword |
| Mailbox | Password |
| Queue | EmailPassword |
| UserSettings | EmailPassword |
I found the answer on this page, If you lost your encryption key
Entities
- Mailbox
- Email Server Profile
If you do an advanced find and use the entity of Mailbox.
You will get a list of mailboxes, if you click on one, on an encrypted database where you have lost the encryption key you will get this message
If you are wondering where the mailboxes are used, each user has a queue and each queue has an Emailbox
Why is data encrypted
Understanding the purpose of functionality, makes it easier to comprehend the logic driving the functionality.
The purpose of encrypting user and password fields in CRM is to hide this information from CRM developers/CRM Admins.
Data encrypted is around email functionality is because Microsoft Dynamics CRM has no email functionality. Email functionality is outsourced to
- Email Router
- Server side sync
- CRM users Outlook
To use external email services, it passes credentials to the other services, to make things secure CRM encrypts this data.
CRM Encryption Facts
Is database encryption is automatically turned on?
YES!
Whilst writing this article I get confused whether database encryption was turned on on or not
This paragraph in Field-level data encryption confused me
The encryption key is required to activate data encryption when you import an organization database into a new deployment or into a deployment that has had the configuration database (MSCRM_CONFIG) recreated after the organization was encrypted.
The paragraph is saying (and the cause of problems I experienced) selected fields are encrypted, its turned on but not activated on imported databases. You will need to activate encryption by copying the correct encryption key, which you can find on original database.
Database encryption facts
- When doing a retrieve on an encrypted field value, a null is returned
- Encrypted fields cannot be indexed
- Database encryption is automatically turned on for any CRM versions of CRM 2013 and higher.
- You cannot turn database encryption off
- You can set and retrieve the database encryption key using the CRM SDK see here
- The SSL check is automatically turned on but can be turned off by updating the field on the mscrm_config database
- Users with the Microsoft Administrator role can change the CRM encryption key
- All the encrypted fields are password fields
- Default encryption key setup during setup
- All new and upgraded (which means all) CRM version will have data encryption enabled
- You can’t audit or customize the encrypted fields
Common Database Encryption questions
Database encryption is likely to interest CRM developers when things have gone and you have restored a CRM database. I will cover the common scenario’s and give links to the answers
Can you change the database encryption key?
Yes, but you have activate database encryption. If the database encryption key field is blank you need to copy a the key from the original database.
I can’t check the database encryption because of the SSL check?
Microsoft turned on the SSL check by default, all on-premise CRM installations which are not SSL enabled cannot open the data encryption screen.
To view the data encryption details you need to disable the SSL check
Go to Settings –> Data –> Database
You cannot check the Data encryption key on a database if CRM is not SSL (HTTPS). This is because the Microsoft config database MSCRM has a field to stop CRM
You need to connect via SQL manager to view and change the database field on the MSCRM_Config database DisableSSLCheckForEncryption
I don’t recommend changing a CRM database, particularly the MSCRM_CONFIG because this is unsupported by Microsoft
Why you shouldn’t put unsupported customizations in Microsoft Dynamics CRM
The only time you can access the database is when Microsoft say you can on this page
When the Microsoft Dynamics CRM (on-premises) website is not configured for HTTPS/SSL, the Data Encryption dialog box will not be displayed. For a more secure deployment, we recommend that you configure the website for HTTPS/SSL. However, if the website is not configured for HTTPS/SSL, use a tool that can be used to modify CRM database tables, such as Microsoft SQL Server Management Studio or the Deployment Web Service, open the configuration database (MSCRM_CONFIG), and in the DeploymentProperties table, set DisableSSLCheckForEncryption to 1.
We have permission, here is the SQL
This views the data
SELECT [ColumnName], [BitColumn] FROM [MSCRM_CONFIG]. [dbo].[DeploymentProperties] WHERE ColumnName= 'DisableSSLCheckForEncryption'
This updates it
UPDATE [MSCRM_CONFIG].[dbo].[DeploymentProperties]
SET [BitColumn]=1
WHERE ColumnName='DisableSSLCheckForEncryption'You need to do an IISReset after changing the database value, until you do you won’t be able to access the database encryption screen.
You can set the field back after you have got the database encryption key if you wish.
Set by step instructions here
My Database has no encryption key
If you have turned off the SSL check and there is no database encryption key, it means you have a restored database. You need to find the original database and copy the database encryption key
This article will help
Data encryption errors after restoring CRM database
Best practices – Backup your key
Microsoft recommends changing the database key once a year, the article below describes how to backup your encryption key
What if you lose your key
You are in a pickle!
I read this blog post – Tip of the day if you lost your encryption key
It states you can add a new encryption key if you delete the encrypted data. Deleting the data is difficult because mailboxes are linked to User and Queues, which means they are Dependant.
You cannot delete the data using the UI, I tried using the CRM SDK but got the same dependency errors.
This forum post suggests you can null but I’m not sure this would work (make sure you backup the DB)
HELP! My data is encrypted and I didn’t do it!
One solution is to create a new CRM organization and import your solution files. This works because the a new database is automatically encrypted, so you don’t need to recover the database encryption key.
Good articles on CRM and database encryption
Filed under: CRM 2013, CRM 2015
