Exceptions gives you an insight to why an error occurred. However, for some exceptions the exception message isn't very helpful. If you for instance have used the Dictionary<TKey, TValue> class you have most likely stumbled on its exception saying:

The given key was not present in the dictionary.

The message surely states what's wrong but give no clue to which key is missing, making it near to impossible to correct the error.

Imagine instead if it has said:

The given key, 'UserId', was not present in the dictionary.

It makes it so much easier to track down why the key was missing.

Conclusion

Do not just say what happened, include information stating why it happened.