This topic has 13 replies, 5 voices, and was last updated 4 years, 8 months ago by usowmyas.

  • Author
    Posts
  • #4872
     steveatunfpa
    Participant

    I am using the exact same replication initialization command that I used for three other hosts but this time I’m bringing a node in Amazon Cloud into the replication config. I am positive I have the correct password (xxxxxxx is a placeholder for the real pass).

    Any ideas what I can check? I even reset the admin password using the procedure in the docs where I encrypt the password string and replace inside the config file/

    Here’s one of the attempts; looks just like the several others I have tried.

    [[email protected] bin]# ./dsreplication initialize –baseDN “dc=unfpa,dc=org” –admin UID “admin” –adminPassword “xxxxxxx” –hostSource host01.mydomain.org –portSour ce 4444 –hostDestination host02 –portDestination 4444

    Server Certificate:

    User DN : CN=ahost02, O=Administration Connector Self-Signed Certificate
    Validity : From ‘Fri Jul 24 14:27:34 EDT 2015’
    To ‘Thu Jul 19 14:27:34 EDT 2035’
    Issuer : CN=host02, O=Administration Connector Self-Signed Certificate

    Do you trust this server certificate?

    1) No
    2) Yes, for this session only
    3) Yes, also add it to a truststore
    4) View certificate details

    Enter choice [2]: 2

    The provided credentials are not valid in server host02:4444.
    Details: [LDAP: error code 49 – Invalid Credentials]

    #4878
     Ludo
    Moderator

    It is possible that the dsreplication tool cannot connect to the host02… Not a fully qualified host name !
    But most likely, the dsreplication enable command was not run to enable replication and set the Admin account on host02.

    #4881
     steveatunfpa
    Participant

    i redacted the hostnames with short, fake names. In the real output the fully qualified names are there. Any other ideas ?

    #4885
     steveatunfpa
    Participant

    Jul 28, 2015 2:04:34 PM org.opends.guitools.controlpanel.util.ControlPanelLog initLogFileHandler
    INFO: Application launched July 28, 2015 2:04:34 PM EDT
    Jul 28, 2015 2:04:43 PM org.opends.server.tools.dsreplication.ReplicationCliMain promptIfRequired
    WARNING: Client exception org.opends.server.tools.ClientException: The provided credentials are not valid in server app5-pb.app.unfpa.io:4444. Details: [LDAP: error code 49 – Invalid Credentials]
    Jul 28, 2015 2:05:10 PM org.opends.quicksetup.QuickSetupLog initLogFileHandler
    INFO: QuickSetup application launched July 28, 2015 2:05:10 PM EDT
    Jul 28, 2015 2:05:10 PM org.opends.quicksetup.installer.Installer initializeSuffix
    SEVERE: Error creating task {ds-task-initialize-replica-server-id=ds-task-initialize-replica-server-id: 13770, ds-task-initialize-domain-dn=ds-task-initialize-domain-dn: dc=unfpa,dc=org, objectclass=objectclass: top, ds-task, ds-task-initialize-from-remote-replica, ds-task-class-name=ds-task-class-name: org.opends.server.tasks.InitializeTask, ds-task-id=ds-task-id: quicksetup-initialize1}
    javax.naming.NamingException: [LDAP: error code 80 – On domain dc=unfpa,dc=org, initialization of server with serverId:13770 has been requested from a server with an invalid serverId:13770. ]; remaining name ‘ds-task-id=quicksetup-initialize1,cn=Scheduled Tasks,cn=Tasks’
    at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3131)
    at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033)
    at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2840)
    at com.sun.jndi.ldap.LdapCtx.c_createSubcontext(LdapCtx.java:811)
    at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_createSubcontext(ComponentDirContext.java:337)
    at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.createSubcontext(PartialCompositeDirContext.java:266)
    at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.createSubcontext(PartialCompositeDirContext.java:254)
    at javax.naming.directory.InitialDirContext.createSubcontext(InitialDirContext.java:197)
    at org.opends.quicksetup.installer.Installer.initializeSuffix(Installer.java:4531)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.initializeSuffix(ReplicationCliMain.java:8318)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.initializeReplication(ReplicationCliMain.java:4682)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.initializeReplication(ReplicationCliMain.java:1521)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.execute(ReplicationCliMain.java:515)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.mainCLI(ReplicationCliMain.java:358)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.main(ReplicationCliMain.java:289)

    Jul 28, 2015 2:05:10 PM org.opends.server.tools.dsreplication.ReplicationCliMain initializeReplication
    SEVERE: Complete error stack:
    org.opends.server.tools.dsreplication.ReplicationCliException: Error launching initialization with contents from server localhost:4444. Details: javax.naming.NamingException: [LDAP: error code 80 – On domain dc=unfpa,dc=org, initialization of server with serverId:13770 has been requested from a server with an invalid serverId:13770. ]; remaining name ‘ds-task-id=quicksetup-initialize1,cn=Scheduled Tasks,cn=Tasks’
    at org.opends.server.tools.dsreplication.ReplicationCliMain.initializeSuffix(ReplicationCliMain.java:8342)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.initializeReplication(ReplicationCliMain.java:4682)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.initializeReplication(ReplicationCliMain.java:1521)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.execute(ReplicationCliMain.java:515)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.mainCLI(ReplicationCliMain.java:358)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.main(ReplicationCliMain.java:289)
    Caused by: Error launching initialization with contents from server localhost:4444. Details: javax.naming.NamingException: [LDAP: error code 80 – On domain dc=unfpa,dc=org, initialization of server with serverId:13770 has been requested from a server with an invalid serverId:13770. ]; remaining name ‘ds-task-id=quicksetup-initialize1,cn=Scheduled Tasks,cn=Tasks’
    at org.opends.quicksetup.installer.Installer.initializeSuffix(Installer.java:4543)
    at org.opends.server.tools.dsreplication.ReplicationCliMain.initializeSuffix(ReplicationCliMain.java:8318)
    … 5 more
    Caused by: javax.naming.NamingException: [LDAP: error code 80 – On domain dc=unfpa,dc=org, initialization of server with serverId:13770 has been requested from a server with an invalid serverId:13770. ]; remaining name ‘ds-task-id=quicksetup-initialize1,cn=Scheduled Tasks,cn=Tasks’
    at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3131)
    at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3033)
    at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2840)
    at com.sun.jndi.ldap.LdapCtx.c_createSubcontext(LdapCtx.java:811)
    at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_createSubcontext(ComponentDirContext.java:337)
    at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.createSubcontext(PartialCompositeDirContext.java:266)
    at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.createSubcontext(PartialCompositeDirContext.java:254)
    at javax.naming.directory.InitialDirContext.createSubcontext(InitialDirContext.java:197)
    at org.opends.quicksetup.installer.Installer.initializeSuffix(Installer.java:4531)
    … 6 more

    #10621
     usowmyas
    Participant

    Hi , is this resolved ? any ideas ? getting the same error , LDAP error code 49 – Invalid Credentials

    #10680
     Ludo
    Moderator

    Hi,
    There has been several reports of similar issues that turned out to be user errors : typos in hostnames, confusion with passwords, or hostnames that are cannot be resolved from the different machines (DNS or /etc/hosts issues).
    LDAP error 49 is very explicit, Authentication fails because the user name or password are unknown, incorrect or the account is disabled.
    When it’s about the replication Admin user and Password, it is possible that dsreplication enable did not complete successfully and further queries fail.
    Redoing everything from scratch, paying attention to hostnames, passwords, … usually is enough to succeed with configuring and enabling replication between servers.
    We are doing this hundreds of times every day, through our unit test, functional tests, manual tests, stress tests, all of this across multiple machines and OS.

    #10682
     usowmyas
    Participant

    Hi ludo , its entirely dockerised and there is no chance for errors mentioned above . what might be an issue is ports . can you please list me all used ports by opendj during replication .

    right now , I have 1389 for LDAP , 8989 for Replication , 1689 for JMX open in docker and in my EC2 security group. am I missing any other port ?

    and this week I tried to put a load balancer on openam , seeing the same issue.

    #11087
     Alexam
    Participant

    Hi, Ludo. I’m confirming the very same situation at RHEL7 system. Without any reason, when provided 100% correct credentials of global admin (I checked that directory accepts them by running test search with “ldapsearch” tool), dsrepliaction rejects them as invalid.

    I made sure that in “/etc/hosts” there are mappings for both nodes in my topology. Configuring replication went well and I was asked for providing global admin password, but I can’t proceed with ‘dsreplication initialize’ now because of this.

    It also should be noted that the same OpenDJ package was successfully used not so long ago for another RHEL7 and CentOS7 clusters. Nothing changed in commands I run this time, I’m following our usual internal step-by-step guide. Something is wrong with this tool, it’s just being triggered by specific circumstances.

    Here is my package version:

    cat /opt/opendj/config/buildinfo
    3.0.0.ee0b5ef693678ceb4fa0e0794a4387aba2fe84cf
    • This reply was modified 5 years, 4 months ago by Alexam.
    • This reply was modified 5 years, 4 months ago by Alexam.
    #11091
     Alexam
    Participant

    I’ve redone it from scratch one more time – still the same. I checked that ports 4444 and 8989 are universally accessible for these 2 nodes.

    Also, there is one thing that make my cases similar to usowmyas‘s – I’m also installing it in a container, still it’s not Docker, just a chroot-based one. Still, we have similar installation where no such problem was experienced.

    May I ask is it possible to enable debug output for the tool? How can I do that?

    • This reply was modified 5 years, 4 months ago by Alexam.
    #11093
     Alexam
    Participant

    Just want to add: it’s not about just initialization. You can’t even run “status” command, or, so to say, “enable” – whatever you do it’s still “invalid credentials”

    Update:
    I’ve just been able to run “enable” command again – it again went through all configuration steps successfully (I believe I didn’t mention that replication is set up correct, I can even see that when I change global admin’s password with ldapmodify on one node this change is also replicates to another) – but after that it’s the same again. Thus I can’t initialize 2nd node to start meaningful replication flow.

    • This reply was modified 5 years, 4 months ago by Alexam.
    • This reply was modified 5 years, 4 months ago by Alexam.
    • This reply was modified 5 years, 4 months ago by Alexam.
    #11097
     Alexam
    Participant

    Okay, here is where really strange things begin: “dsreplication disable” command also asks for global admin creds – an actually works! I can’t believe this is “intended behavior”. Even if there indeed some issue with.. anything, it’s clearly NOT about credentials, the tool should provide some context to this error message, otherwise it makes it pointless.

    #11182
     Ludo
    Moderator

    There has been some issues with SSL and ciphers with specific versions of Linux and OpenJDK (7 if I recall correctly). May be you are hitting this issue.
    The Administration port (4444) and the Replication port (8989) are both protected with SSL. It’s possible that an SSL error results in an invalid credentials, or a lack of entropy since you are running in a container.

    Maybe enabled SSL debugging can help to identify the root cause of the problem. Check this post : https://ludopoitou.com/2011/06/29/opendj-troubleshooting-ldap-ssl-connections/

    Honestly, I’m a little bit lost here because we do run similar tests on a daily basis, within Docker or not, within VMs or bare metal with different OS (Linux, Windows and Solaris) and JVMs.

    #13353
     MrManor
    Participant

    I am not sure if below observations add some info to this problems, – but I have just run in to this error:

      I am building a cluster on VM’s, based on OpedDJ 3.0 on a Centos 7 with java-1.7.0-openjdk.
      Import ldif data is from a OpenDJ 2.6 cluster with 3 servers, using the export function.
      First server runs setup (as standalone) installs, imports data – and starts without problems.
      Next server installs, but when trying to run dsreplication initialize on any server I get the same error as described above. Admin password always fails on the empty server.
      But if I initialize the second server using the same ldif file as number one, before running dsreplication initialize, then the command succeeds.

    Based on some sample test, and a brief log inspection, replication now seems work in our test setup.

    So I was wondering if something related to the imported 2.6 data could be affecting the problem.

    • This reply was modified 5 years ago by MrManor.
    #15714
     usowmyas
    Participant

    Hi , this issue is now resolved with following cmd .

    $DJ_HOME/bin/dsreplication enable –adminUID admin –adminPassword ${BINDPASSWORD} –baseDN dc=example,dc=com –host1 ${OPENDJNODEMAS} –port1 4444 –bindDN1 “cn=Directory Manager” –bindPassword1 ${BINDPASSWORD} –replicationPort1 8989 –host2 ${HOSTNAME} –port2 4444 –bindDN2 “cn=Directory Manager” –bindPassword2 ${BINDPASSWORD} –replicationPort2 8989 –trustAll –no-prompt

    $DJ_HOME/bin/dsreplication initialize-all –adminUID admin –adminPassword ${BINDPASSWORD} –baseDN dc=example,dc=com –hostname ${OPENDJNODEMAS} –port 4444 –trustAll –no-prompt

    ${OPENDJNODEMAS} should be something like ip-XXX-XXX-XXX-XXX.eu-central-1.compute.internal , i.e fully qualified domain and if you are using a docker container then we need to assign a hostname to the docker container which is same as the actual EC2 .

    for example :
    opendj:
    build: .
    extra_hosts:
    – “elk:172.31.27.244”
    ports:
    – “1389:1389”
    – “4444:4444”
    – “8989:8989”
    volumes:
    – ~/.aws:/root/.aws
    environment:
    – MASTERDJ_HOSTNAME=ip-172-31-19-183.eu-central-1.compute.internal
    hostname: $HOSTNAME

    • This reply was modified 4 years, 8 months ago by usowmyas.
Viewing 14 posts - 1 through 14 (of 14 total)

You must be logged in to reply to this topic.

©2021 ForgeRock - we provide an identity and access platform to secure every online relationship for the enterprise market, educational sector and even entire countries. Click to view our privacy policy and terms of use.

Log in with your credentials

Forgot your details?