User Profile Sync – SharePoint 2010 – SharePoint mysteries and more…

Since RTM Release, we had lot’s of cases regarding problems with “user profile synchronization service” in SharePoint 2010 RTM.
There is already many stuff out there about it and some of them are very good, some others are not so good. I completely revised (more revoked) my post to help you by leading to the most relevant resources for this topic.
(last updated: 2011-03-14)

The main article on TechNet Configure profile synchronization (SharePoint Server 2010) has been updated and is now great described with more detailed order on the steps to be taken and other considerations! A very good resource now!

Link List:

Altough I’m trying to cover most topics in here, please don’t miss this excellent Blog post from “Spencer Harbar” with Rational Guide to implementing SharePoint Server 2010 User Profile Synchronization (harbar.net). This Blog explains some details about the architecture and lot’s of insights into the user profile mystery. It also contains all required steps and actions very clear and if you follow them as described, your UPA stuff should work and you’ll save a lot of time with banging your head against walls, wasting nights with troubleshooting and finally still don’t get any further.

All resources should be watched frequently as both of them will be further updated and kept actual as current knowledge.

SharePoint 2010 August CU and Event ID 3

With the first Updates announced in JUNE 2010 (CU1), there have been fixed several User Profile Issues. Currently we have published the “August cumulative Update for SharePoint 2010 (Build 5123) that contains those fixes as well.

If you are running on RTM (Build 4763) and you don’t have any problems with your User profile Service or User profile Import, it is currently not necessarily required to install the updates because it should work in general on a simple setup, with one AD connection and all servers are in same domain!

If you actually have problems even after following (maybe several times) Spencer’s excellent advises, or having anyway a more complex setup AND the problems, then you should definitely install the August CU!

If the problems still persists, please check also Spencer’s Post on “Common Issues…” and if this still does not help, then you might be forced to open a service request at Microsoft Support to investigate the Issue

Note:
When installing the “August cumulative Update for SharePoint 2010” it might be that you experience a “known issue with logged Event ID 3”, please check this post for the current workaround until the according KB article will be published!

Additional comments and information regarding the permissions:

Very often I have been caught by endless discussions with some paranoia AD-Admins for the permission settings and I can tell you, there is currently no other way around it! If you don’t set it accordingly, it simply will not work!

And NO, there is no write-back to your AD unless you do especially grant this additional permission! For more information, please check here: http://technet.microsoft.com/en-us/library/ff182925.aspx (TechNet excerpts below)

Authenticated users who have Replicate Directory Changes permissions will only be granted read-access to AD DS objects. Additional permissions can be granted using access control lists (ACLs) in AD DS. SharePoint Server 2010 will not write profile data back to AD DS unless Write permission is explicitly set on the account that has Replicate Directory Changes permission.

More Q&A

Q: Is it possible to use successful the User profile import connection with an account that does not have these described permissions?
A: No. We must have the “replicate directory change” permission and the “pre-windows 2000 compatible group” membership because otherwise the profile import will not work.

Q: If this permission are required to run the profile import, why is this permission required for a “read-only” access to the Active directory?
A: Excerpt from the KB article How to grant the “Replicating Directory Changes” permission – http://support.microsoft.com/kb/303972
” When discovering objects in Active Directory using the Active Directory management agent (ADMA), the account that is specified for connecting to Active Directory must either have Domain Administrative permissions, belong to the Domain Administrators group, or be explicitly granted Replicating Directory Changes permissions for every domain of the forest that this management agent accesses.

as per KB 891995 – http://support.microsoft.com/kb/891995:
Excerpt:
“Some database monitoring programs are designed to maintain consistency between object data that is stored in the Active Directory directory service and object data that is stored in another database. The other database can be another Active Directory database, a Microsoft SQL Server database, FIM, or some other directory service database. The database monitoring program must track changes to Active Directory object data, such as updates or deletions, and replicate those changes back to the other database. Database monitoring programs can track changes in Active Directory with either of the following techniques:
• Change notification registration (ie: The DirSync control…more efficient, used by FIM)
• Polling

The DirSync Control:
The DirSync control is a Lightweight Directory Access Protocol (LDAP) server extension that enables a program to search an Active Directory partition for objects that have changed. Although the DirSync control is powerful and efficient, it has two limitations. The first limitation is that the control must run by using a user account that has the Replicating Directory Changes permission on the domain naming context.”

Q: What exactly can the account do or cause with having these permissions?
A: For the “Replicating Directory Changes” see the answer above.
A: For the “pre-windows 2000 compatible group” membership:
http://blogs.msdn.com/b/russmax/archive/2010/01/20/why-the-tokengroupsglobalanduniversal-tggau-attribute-matters-in-sharepoint-2010.aspx
Excerpt:
“In SharePoint 2010, most service accounts require some specific access to Active Directory or certain functions will fail. SharePoint 2010 uses AuthzInitializeContextFromSid function. In a service application scenario, this function runs under the context of the associated service account and performs an S4U logon.”

and http://support.microsoft.com/kb/331951/en-us
Excerpt:
“Some applications have features that read the token-groups-global-and-universal (TGGAU) attribute on user account objects or on computer account objects in the Microsoft Active Directory directory service. Applications that read this attribute or that call an API … that reads this attribute do not succeed if the calling security context does not have access to the attribute. The default permission compatibility for new Windows Server 2003 domains does not grant broad access to the TGGAU attribute.”

Managed accounts and the automatic change password feature:

In SharePoint 2010 you can now use the managed accounts with the automatic password change feature to configure farm-level settings for automatic password changes. Farm administrators can configure the notification e-mail address that will be used to send all password change notification e-mails, as well as monitoring and scheduling options.

However, there is no reason to register the service account, that will run the User Profile Service because it cannot handle managed accounts! So if you do configure the automatic password change with this account, you will havfe trouble latest when the password change happens. The FIM services are still using the old credentials, configured on creation and even a restart of the services will not apply the changed passqword unless you retype it manually. This will cause your FIM services and therefore User profile service application to stopp working until you change the credentials and reset the password manually! For the User profile Service, this feature is not recommended and does not support the User profile Service (FIM service) accounts with the password change feature!

The reason we need to specify the password manually is because FIM services impersonate the supplied user when making calls into MOSS. This impersonation is not necessary since the account using which FIM is provisioned already has permissions to interact with MOSS. This functionality (to not impersonate) is not available in FIM’s build that we consume and therefore there is no automatically stop of the sync service when we detect the rollover and also no password change. besides that, it is not recommended to restart or modify the FIM services manually because it may turn your farm into an unsupported state.

For more information on how to troubleshoot issues with “managed accounts”, please check the section “Troubleshooting automatic password change” to configure, set or repair managed accounts.

Profile Import with Novell Directory services eDirectory 8.8 SP6
Also on this topic we actually do not really have much details or in-depth content on TechNet yet but Spencer Harbar posted some words about this here:
SharePoint Server 2010 User Profile Synchronization with Novell eDirectory 8.8 SP6

NetBios Domain Name different from FQDN Name
This is described on Spencer’s Blog here, so please follow the Link and find all Information there.

AD-Connection and the Profile Import

Once your User Profile Service and sync application is set up and configured as described in “Configure profile synchronization (SharePoint Server 2010)“, you’re going to create the import connections to i.e. Active Directory, BDC, etc.
When you do so, please consider that on depending the amount of user accounts and the AD structure you have, it may take up to days until finally all accounts are imported into SharePoint!

Important:
Currently we only support “one AD-Connection per Domain and Forest” at a same time! You must configure the entire connection for all OU’s and objects you want to import(or export like the thumbnailPhoto) within the same domain and forest on one single AD-connection! Multiple connections to the same Domain and forest at a same time are simply not working!

Example:
You have configured an AD-Connection to your Domain “MyCompany” by chosen the OU’s for “NA-Users”, “EU-User” and some others. Later on you decide to add another AD-Connection against the same Domain but now having only for a specific OU within the same forest to use the “Export profile picture from SharePoint to AD”

Result:
You’ll not get any pictures updated on the AD as this does simply not work. You must change the previous created AD-connection and modifying the property mapping and/or adding additional OU’s to be synced within “one single AD-Connection”.

Profile Picture Property “thumbnailPhoto”

In SharePoint 2010, you can store the user profile picture on the “My Site’s user profile page. This picture can be synced against the Active directory.
Prerequisites: You need a running MySite, User profile service and the user profile apllication service.

Steps:

1. Apply the October Cumulative update (Uber) package

2. Go to “Central Administration – Manage Service Applications – User Profile Service Application – Manage User Properties page”

3. Edit the property and create a mapping that “exports” the user profile picture to AD

4. Set the Attribute to “thumbnailphoto” by selecting it from the dropdown menue

5. Set the “Direction” to “Export” and click “add”

You should now see the new maping as in the above figure.

6. Start a full sync of the user profile import and give SharePoint some time to process everything

7. Check in AD if the “thumbnailattribute has been filled

Note:
To export properties, such as profile pictures, from SharePoint Server 2010 to AD DS, at least Replicate Directory Changes permission is needed on the object and all child objects for the AD DS domains to which you want to export data from SharePoint Server 2010. Read/Write permission is also needed on the container that stores the user picture attribute, for example, the ThumbnailPhoto attribute.

Also bear in mind that the picture itself to be stored in AD msut not exceed 10k size and not bigger than 96×96 pixels (which is automatically resized when a photo is uploaded into SharePoint).

Important:
In order to synchronize user profile pictures between Microsoft SharePoint Server, AD DS, and Outlook 2010 by using the Microsoft Outlook Social Connector, you must set the Data Source Connection for the Picture property mapping to Export. For more information about synchronizing user profile pictures, see Enable SharePoint Server 2010 Colleague in Outlook 2010 and/or the blog of Russ Maxwell.

So now you want to do it the other way,
import pictures from AD into SharePoint:

Consider the following scenario:
You try to import pictures in a user profile from the Active Directory thumbnailPhoto attribute to the PictureURL attribute in the property mapping of SharePoint Server 2010. You set up an import PictureURL mapping. Then, you perform a full synchronization in the SharePoint server.

Note:
In this scenario, the pictures are not added to the user profile in the SharePoint server.
To enable this you must first apply the October cumulative update package KB 2394320 !

If not done yet, set the property mapping to “Import” (if it was already set to “export” because you had exported the MySite Pictures to AD, you need to delete the mapping first!). This will download the pcitures stored in AD and bring them into SharePoint. Also change Edit Settings to “Do not allow users to edit values for this property” as you want to have only the versions from AD as pictures in SharePoint, right?

After a full import, you may notice, that the pictures are still not set on the MySite as expected. Yet you need the SharePoint 2010 Management Shell to run this commands to perform the import operation:

Update-SPProfilePhotoStore -CreateThumbnailsForImportedPhotos 1 -MySiteHostLocation <mySiteHostURL>

When you now go back to the MySite, the picture should appear.

Note:
1. If you delete the picture from AD, it will not be deleted as well from Sharepoint although running the Powershell command.
2. The user running this Powershell command must have permissions on the User Profile Service Application, if not an error message wil be thrown.
3. The “CreateThumbnailsForImportedPhotos parameter” is not documented in http://technet.microsoft.com/en-us/library/ff607547.aspx

As found by my colleague, Zsolt Illes, Support Escalation Engineer, the undocumented parameters are as follows:

Updates the profile photo store to be compatible with Microsoft SharePoint Server 2010.

Update-SPProfilePhotoStore -MySiteHostLocation <SPSitePipeBind> [-AssignmentCollection <SPAssignmentCollection>]

Parameter

Required

Type

Description

MySiteHostLocation

Required

Microsoft.SharePoint.PowerShell.SPSitePipeBind

Specifies the URL for the My Site host location where the photos are to be uploaded.

AssignmentCollection

Optional

Microsoft.SharePoint.PowerShell.SPAssignmentCollection

Manages objects for the purpose of proper disposal. Use of objects, such as SPWeb or SPSite, can use large amounts of memory and use of these objects in Windows PowerShell scripts requires proper memory management. Using the SPAssignment object, you can assign objects to a variable and dispose of the objects after they are needed to free up memory. When SPWeb, SPSite, or SPSiteAdministration objects are used, the objects are automatically disposed of if an assignment collection or the Global parameter is not used.

Note:

When the Global parameter is used, all objects are contained in the global store. If objects are not immediately used, or disposed of by using the Stop-SPAssignment command, an out-of-memory scenario can occur.

CreateThumbnailsForImportedPhotos

Optional

Specifies whether the three user pictures have to be (re)created.

Note:

Typically you have to use this switch after importing user pictures from AD to populate the three user pictures.

NoDelete

Optional

Specifies whether the obsolete user pictures should be deleted or not.

Note:

When a user deletes the profile picture from one’s profile, the picture itself won’t get deleted from the Profile Pictures library. This switch controls whether these obsolete pictures should be deleted or not.

How to grant the Replicate Directory Changes permissions

1. On the domain controller, open the “Active directory Users and Computers” console
2. On the menu bar, click on “view” and enable the “Advanced Features”
3. On the Console root tree, Right-click the domain name and select “properties”

3. On the properties, under the “security” tab, add the user account to connect to the AD

4. Allow the “Replicate Directory Changes”

Note:
By Default, this does not inherit the permissions to all child objects. If you need to pass down the permissions,
Click the “Advanced” Button before hitting “OK” and change the settings as shown below:

Back to top

How to grant the Replicate Directory Change on the domain configuration partition

Note:
This should be only required if the NetBIOS name of the Domain is different from the fully qualified domain name (FQDN). But in some cases it also was required to run the user profile sync, regardless if the NetBios Domain Name was different or not! – So setting this in addition finally did the trick and it worked suddenly 😉

To do so, you need to add the same permissions for the account that is running the User profile Import connection to the Configuration partition in AD:

1. On the domain controller, open the ADSIEdit.msc console

2. Connect to the Configuration Partition:

3. Right click the configuration partition and choose “properties”

4. From the Security tab, add the DOMAINsp_upa user and give it “Replicating Directory Changes permissions” as before:

Back to top

How to grant the “Replicate Directory Changes” permissions on a Server 2008 AD:

Please see here the detailed steps to grant the Replicate Directory Changes on a Server 2008 DC: http://blogs.msdn.com/b/sharepoint/archive/2009/12/14/how-to-set-replication-directory-changes.aspx

Note! For the domain configuration partition, the steps are similar to the steps on a Server 2003 AD

On the server that will host and run the user profile synchronization service:

Log on locally permission
Grant the DOMAINsp_farm account the permission to “log on locally” and ensure that no other policy on domain level overwrites this setting!

local administrator
Also the account must be a member of the local administrator group because this account will start and configure the FIM services.
This permission can be later revoked once the user profile services have been provisioned and the first full import ran successfully!

Note:
On modifying the service accounts or changing the passwords, the ACL must be released first which is normally a “log-off and log-on again. The best way is just to reboot the server the User Profile Service is running on as some services will also be affected and on restart the processes behind will do the right thing. Otherwise you may see a “starting” state for ever and the provisioning of User Profile Synchronization Service will be stale. (see more details on Spencer’s blog)

Back to top

Pre-Windows 2000 Membership and why the TGGAU attributes matters

If your Domain Controller is running Windows 2003 functional level we also need to make the DOMAINsp_upa account a member of the Pre Windows 2000 Compatible group. This is because we will read the TGGAU attribute in AD. See more on Why the TGGAU attribute matters in SharePoint 2010.

Monitoring ULS logs

1. To check the progress of the provisioning you can do some things in-between while you’re waiting 🙂

Go to the folder C:Program FilesCommon FilesMicrosoft SharedWeb Server Extensions14LOGS and use the “ULSViewer tool” to check your ULS logs in realtime:
On the “File” menue, click “open from – ULS Ctrl+U”

choose “use ULS feed from default log-file location” and click ok.
On the viewer screen, click on the lower down [Autoscroll…] and mark it

Apply a filter with the settings [Field: category; Operation: contains; Value: profile]

Now you can watch in realtime the User profile sync process in the logs.

2. Next, go to the “services.msc” and check whether the two FIM Services are changing from “disabled” status to “starting” one by one and if they are using the farmaccount for starting up!

The end should looks like this: Both servcices are started and running with the configured account used in “User profile service application”

So far, the FIM services should be also in that status even after 30-45 min. If they change back to “disabled” then you did something wrong 😉 -> Go back to Spencer’s blog and start from scratch again!

If FIM Services are staying alive and “started”, go to the folder “C:Program FilesMicrosoft Office Servers14.0Synchronization ServiceUIShell…” and start the “miisclient.exe”. This should start up a management tool and ensures that your service is ready and working. (At this stage, you’ll not really see a lot in the tool because the “logging” starts first on running a User profile Import! – see more details on Spencer’s blog)

Important:
By current knowledge and status, It is not supported to modify manually the ForeFront Services via the MMC Console nor any changes manually via the MIISCLIENT.EXE management tool! This will definitely brings your farm in a “unsupported status” and you have to recover the original status (means: Go back to Spencer’s blog and restart from scratch!)

3. Additionally you can monitor within the central admin page, under “Monitoring – Runnig jobs” for the timer job that is processing all necessary steps in background to provision the service, configuring the ForeFront services and accounts (see above) and creating all connections and associations.

Look here for the “ProfileSynchronizationSetupJob”. This Job should run about 10-15 minutes (sometimes more) and after finishing, the “User Profile Synchronization Service” in “services on server” should appear as fully started.
if this job just starts and immediately stops again or disapears before at least 10min. runtime, then something went wrong 😉 -You should see then the “User Profile Synchronization Service” as not started!

Back to top

Running User Profile Service on mutliple servers in farm…

An interesting question i came across was about “can we run use rprofile services on multiple servers in same farm?” Well, on my Research for the answer, I found Ali’s blog with the title
Running User Profile Service instance and Profile Synchronization service on more than one server in the SharePoint 2010 farm
Here you will find some Q’s and A’s that may answer most of the questions 😉
Just follow the link to Ali’s blog for more.

Back to top

Related links and resources:

Troubleshooting User Profile Sync Issues on SharePoint 2010

From the SharePoint Escalation team blog:
UPA 2010 – Intro Part 1
Upa 2010 – Setting up the User Profile Service Application – Part 2

High Level Architecture and conceptual views on User Profile Service:
User Profile subsystem in SharePoint Server 2010
–> A graphical view of the architecture and the various components that make up the User Profile subsystem

How User Profile Synchronization works…
–> A conceptual view of how it works under the hood…

Back to top

cheers, Steve Chen

12 thoughts on “User Profile Sync – SharePoint 2010”

ralph lauren jeansskjorta says:

April 12, 2016 at 1:47 pm

hey there and thank you for your information ?I certainly picked up anything new from right here. I did however expertise a few technical issues using this site, as I experienced to reload the web site many times previous to I could get it to load correctly. I had been wondering if your web hosting is OK? Not that I am complaining, but sluggish loading instances times will often affect your placement in google and could damage your quality score if advertising and marketing with Adwords. Anyway I adding this RSS to my email and can look out for a lot more of your respective exciting content. Ensure that you update this again soon..

LikeLike

Anonymous says:

November 5, 2014 at 12:32 am

User Profile Sync – SharePoint 2010 – Steve Chen [MSFT] Sr. Support Escalation Engineer – Site Home – TechNet Blogs

LikeLike

Anonymous says:

December 15, 2013 at 1:19 pm

Pingback from SharePoint 2013 Social ??? Part 2 ??? Synchronizing User Account in Active Directory to SharePoint 2013 – A soldier of fortune

LikeLike

Anuj says:

February 9, 2012 at 8:32 am

If you are updating user profiles using SharePoint 2010 APIs then please remember following –
1. Profile update takes lot of time
2. More the number of properties in each profile, more the time it will take to update
3. Update only the properties which are changing
4. Try to load configuration values before running the main logic (of updating the properties)
5. Reason it takes more time to update profile/properties is internal to SP 2010

LikeLike

David VanSickle says:

January 13, 2012 at 5:49 pm

Steve, we are unable to get 2010 Memberships to populate. We've stop and started the UserProfile Sync service, we've run and re-run full profile synchronization to no avail. Is there a PowerShell script we can run manually to update this information on our MySites until this functionality gets fixed in the future? We are currently running 14.0.6112.5000 (Oct 2011 CU with SP1). If you have the ear of the engineering team, this particular service application is the "thorn in the flesh" for Administrators. By far, more time is spent troubleshooting this service application than with all the service applications put together.

LikeLike

Issues importing AD pictures(thumbnailPhoto attribute) to SharePoint 2010 says:

December 30, 2011 at 6:29 pm

Hi Steve, I found your blog quite interesting.. I just having one question, when I tried to import pictures from AD(JPEG format and 96×96 pixels), there's and error and it's looks like the format in the AD was PNG. I found a url into your blog that gives me a simple explanation….
"Can't we just map the Profile Property to AD and Import the value?
So here is where we hit the roadblock.. The Active Directory "jpegPhoto" attribute is of type "Binary Data" and the SharePoint 2010 User Profile property is of type HyperLink (as it typically links to an image in an Asset Library in the My Site Host site collection).
As a result, you cannot import it using SharePoint 2010 functionality (although it may be possible if you have also purchased the more sophisticated "ForeFront Identity Manager Synchronisation Server" product)."
http://www.martinhatch.com/…/ad-image-attributes-are-not-imported-to.html
It does make sense for you??
Regards and have a Happy New Year !!!
Oscar

LikeLike

Anthony says:

May 20, 2011 at 8:17 pm

Any chance the December CU reversed what the October CU corrected? Running Full sync and PS command to no avail. Seeing 0 errors.

LikeLike

John Carnival says:

January 13, 2011 at 1:41 am

User Profile Parameter for Sharepoint 2010 component enables access from any DataSource being used in a Data Form WebPart to the User Profile properties collection. Using this tool, the DataSource Parameters can be populated with values available in the logged user profile, such as AccountName, PreferredName, WorkEmail and even custom profile properties. In this way, WebParts can show profile aware values using the user's profile properties as parameters to make queries to defined DataSources, making development more efficient.
http://www.youtube.com/watch

LikeLike

Maik Kapp says:

September 21, 2010 at 8:42 am

Very cool post!

LikeLike

stche says:

November 30, -0001 at 12:00 am

Hi David,
I’m very sorry but this is not that easy as you may think about!
We have lots of support cases, all complaining about broken membership relations or out of sync issues but none has ever raised a hotfix request due to lack of a "business Impact" which is required for it.
Nevertheless, it is highly recommended to call MS Support at least to make them aware of each reproted case as we then can determine a solid business impact by just having lots of cases with all same or similar issues!
So go ahead and open a service request as this is the only way to make enough pressure for getting this fixed!
Besides that, consider that the "membership" can be broken due to variuos reasons or causes whereas each case needs to be investigated individually!
I know, this does not sounds very sufficient but, hey! – as long as no one "starts the queue of complaints" this is rather not going to be a recognized "pain"!
Please see/watch also the comments on my related post here:
blogs.technet.com/…/3452083.aspx
greetz, Steve

LikeLike

stche says:

November 30, -0001 at 12:00 am

Hi Oscar,
first of all:
sure! We can either import or export user profile pictures from AD to SharePoint or from SharePoint back to AD. The only thing is that you can do it only "one-way" at a time, means you cannot import and export within the same Sync connection 😉
So back to the Post and URL you refering to:
This is obsolete information and has been fixed long ago. The referenced post is from November 2010 but we have now Dec. 2011 and SP1 and ton's of patches and fixes regarding user profile import!
Latest major change was within june 2011 CU where the FIM bits got updated and "tuned" a bit more.
So no worry, you don't need to purchase "ForeFront identity Server" just to import/export Profile pictures 😉
This is an old topic and this functionality is already implemented inbetween.
Next point, your issue:
Well, it must be a jpeg indeed, when you want to store it in AD and import it to SharePoint. The other way is not the problem as we can use PNG in SharePoint for profile pictures and then "export" these pics up to AD. While FIM is running the profile sync, a background process is calling the URL and fetches the picture and transforms it into a binary format, that will fit for AD as long as the size and pixel rate are not exceeding the limitations.
So if you go through my entire posts and also having the latest patch level and SP1 installed correctly, then you should create a new user profile service application with AD-Connection and properties set as you desire. On your next sync, the pics should be shifted as expected.
I hope this sheds some more light on your concerns 😉
cheers and Happpy new year,
Steve 😉

LikeLike

stche says:

November 30, -0001 at 12:00 am

Hi Anthony,
either oct. and dec. Cu had some regression all around user profile stuff, so it is anyway most recommended to go for at least the feb.2011 CU or even April 2011 CU.
regarding the OCt.CU only:
**** Update: the regression is Fixed and packages are republished now **** to see more, go to my linekd post above.
Hth, cheers, Steve 😉

LikeLike

Share this:

Related

12 thoughts on “User Profile Sync – SharePoint 2010”

Leave a reply to Issues importing AD pictures(thumbnailPhoto attribute) to SharePoint 2010 Cancel reply