live-sync broken in my installation

This topic has 7 replies, 3 voices, and was last updated 3 years, 6 months ago by Bill Nelson.

  • Author
    Posts
  • #18643

    Hi all,

    while mappings source->managed/user seem to work fine, updates are missing from managed/user->target.
    Like: new user in sap is detected and manged/user is created, but managed/user->activeDirectory does not seem to trigger at all. Started on Friday, there was an AD-issue, but it is fixed now. Staring a reconciliation manually, all changes are properly propagated form managed/user->target.

    Any ideas? Perhaps an internal service isn’t operational? Haven’t found and interesting logs so far, no failures referring to my issues. Any help, pointing me in the right direction, is highly appreciated.

    Yours,
    Patrick.

    #18644
     andi
    Participant

    IF the sync service wasn’t up for some reason, I would have expected a warn log entry such as “Sync service was not available.”

    Otherwise, given that from your description it sounds like the managed object service is up and operational, as long as enableSync is not set to false in the mapping, I would have expected it to (try to) sync.

    #18645

    Hmmmm, it’s like “14 mappings stopped working”, while one seems to work… very strange. Nothing in common, some are powershell, some LDAP, some database table (postgresql, sqlserver, oracle, mysql), some scripted database….
    However, I just discovered issues regarding queries against managed/user. Possibly live sync involves similar queries, so both issues might have the same cause?

    When I monitor the admin-interface, it issues calls like
    https://SERVER1/openidm/managed/user?_sortKeys=_id&_pageSize=10&_queryFilter=(_id%20eq%20%22ubq%22%20or%20(mail%20sw%20%22ubq%22%20or%20(someAttr%20sw%20%22ubq%22%20or%20(someOtherAttr%20sw%20%22ubq%22))))
    returning NO error, but NO entries.
    However, if I remove “_sortKeys=_id”, the query returns a resonable amount of entries. I suppose I should investigate that issue first.

    #18646
     Bill Nelson
    Participant

    One point of clarification regarding how synchronization works.

    livesync is used to detect changes on a system resource and bring the changes into openidm. so system/AD/account -> managed/user would use livesync. livesync is a polling mechanism of a database of changes and relies on schedules.

    implicit sync is used to propagate changes from managed objects to system resources. so managed/user -> system/AD/account would use an implicit sync. implicit sync is not a polling mechanism and doesn’t rely on schedules.

    implicit sync comes into play when changes are made to a managed object. when this occurs, the changes are sent automatically to any target in which the managed object is defined as the source in a sync.json mapping. this is an automatic process unless you intentionally set the enableSync property to false or you have an error. Check the activity.csv file to ensure that you see changes made to the managed object – which would initiate the implicit sync process. Check the sync.csv to see if you see the changes synchronized with the target(s) you are expecting. If you have an error in the implicit sync, then you should see it either in the sync.csv, the console output (i.e. server.out or nohup.out), or even the openidm log file (openidm0.log.0).

    #18647

    I appologise for my poor wording. To clarify: accessing one of the mappings, looking at the tab “scheduling”, section “LiveSync” the option “Enable LiveSync for this mapping” is activated, which means that “implicit sync” should trigger. Which it does not for most of my mappings.

    Looking at activity.csv I see my changes (also on openidm0.log.0 and other places), as expected.
    However, sync.csv is extremely interesting, since I only see 4 entries when I would expect it to examine 20 mappings. Is there a way to break a mapping so badly that processing stops upon hitting this mapping without any error messages? I suppose they are checked in a specific sequence?
    BTW, OpenIDM is 4.5.

    #18648
     Bill Nelson
    Participant

    What I am trying to tell you is that the selection of the “Enable LiveSync” option for a mapping DOES NOT have anything to do with implicit sync. They are two TOTALLY DIFFERENT things. So looking at one when you are attempting to debug the other will only confuse you. (BTW, caps were used for emphasis, nothing more.)

    You enable LiveSync to detect changes on remote systems. You have to take some action and perform some configuration. Implicit Sync is enabled by default – you do not configure anything for implicit sync to work. It just works. You have to intentionally turn it off or have an error for it not to work.

    You say that you have 20 mappings. You can think of Implicit Sync starting at the top of your sync.json and looking at each mapping for those where the “source” parameter is the the same as the managed object you are changing (i.e. managed/user). If OpenIDM finds a source that maps to the changed managed object, it then attempts to perform the mapping. If it doesn’t then it simply skips and moves on to the next one.

    You mention that you see your changes in the openidm0.log.0 file and ‘other places’. That is interesting as the default behavior doesn’t log successful events to the openidm0.log.0 file – only errors. So if you are seeing something in that file, then it most likely is the error in question.

    You ask is there any way that you can break it so badly that sync processing stops? It is possible, but highly improbably. A couple of things that you can try a) make a change to one of your mappings in the sync.json, let openidm discover it, and let it reload the entire contents of the sync.json back in. b) you can also try simply restarting the server (or all servers if you are in a cluster).

    I do not suspect that either of these things will fix your problem as it seems like there is something else going on. It is difficult to help troubleshoot without seeing the actual information from the activity.csv/sync.csv/openidm0.log.0/server.out.

    • This reply was modified 3 years, 6 months ago by Bill Nelson.
    #18652

    If you access the admin ui like I described and toggle “Enable LiveSync for this mapping” like I described, you toggle “enableSync” for the specific mapping, which will enable or disable the mapping, regardless of LiveSync or implicit sync. https://backstage.forgerock.com/docs/openidm/4.5/integrators-guide/#disabling-automatic-sync

    I agree that the labels in the admin ui might be better (running 4.5, might have been improved in 5.0), but I didn’t design the interface.
    All I’m saying is: the mappings are enabled and have been enabled for ages. The checkbox shows that I did not (accidently) disable implicit sync for my mapping.

    Regarding other places: the onCreate-Script for the mapping is performing the REST-calls it is expected to send and sending messages to our message-queue like it is supposed to, welcome-letters are printed, the global managed_onUpdate-script is sending emails and starting activity workflows like it is supposed to, etc. Sorry for lacking clarity.

    In the last mapping which got an entry in sync.csv I found a bug in the correlation query. Since fixing this issue, OpenIDM is running smoothly again and processing all mappings. I would have expected SOME log-entry to indicate an error, though.

    #18670
     Bill Nelson
    Participant

    I totally agree with you re: log messages. And am glad that you found the issue.

Viewing 8 posts - 1 through 8 (of 8 total)

You must be logged in to reply to this topic.

©2021 ForgeRock - we provide an identity and access platform to secure every online relationship for the enterprise market, educational sector and even entire countries. Click to view our privacy policy and terms of use.

Log in with your credentials

Forgot your details?