ThoughtSpark.org

That is wild. I've not seen it. The only thing I can think of is maybe you're pointing to a valid directory server but it cannot be searched or is not in control of the scope of the data you're searching. For example, this can happen when using Active Directory and you are doing a domain/forest wide search and you didn't use at Global Catalog server. If you are pointing to a Global Catalog, you need to use port 3268. Let me know if that solves anything.

Thanks for the tip. I hope the rest of the script is treating you fine.

This script seems to be very helpful
thank you ...

I just have one question ...
Would this script play friendly if I used ldaps://:636/ ??????????

mr whitlock,

found an issue with this script on a windows SVN/apache install, mod_authz_svn sometimes validates the userid returned from the ldap layer as full lowercase. causes an issue when AD is passing them up as mixed case. changing members.append(attrs[userid_attribute][0]) to be members.append(attrs[userid_attribute][0].lower()) resolved the issue.

might wanna make it a runtime option ;)

--Alex

Hello. I did try this script with an openldap installation and it didn't work out. The error I get is:

$> python sync_ldap_groups_to_svn_authz.py -d"cn=manager,dc=some,dc=special,dc=location" -p"myPassword" -l"ldap://some.directory.server:389" -b"dc=some,dc=special,dc=location" -m"memberUid" -g"objectClass=posixGroup" -u"objectClass=posixAccount" -i"uid"
Successfully bound to ldap://localhost:389...
16 groups found.
[WARNING]: user1 object was not found...
[WARNING]: user2 object was not found...
[WARNING]: user3 object was not found...
(...)
[groups]
### Start generated content: LDAP Groups to Subversion Authz Groups Bridge (2009/04/13 11:25:52) ###
SomeGroup1 =
SomeGroup2 =
################################################################################
########### LDAP Groups to Subversion Authz Groups Bridge (Legend) ##########
################################################################################
### SomeGroup1 = cn=SomeGroup1,ou=Groups,dc=some,dc=special,dc=location
### SomeGroup2 = cn=SomeGroup2,ou=Groups,dc=some,dc=special,dc=location
################################################################################
### End generated content: LDAP Groups to Subversion Authz Groups Bridge ###

And, as you can see, my groups are "empty", but that is not the case...

The problem is probably related to the way that OpenLDAP group membership is available: the memberUid attribute is in the group and it is not a DN (it is just the uid of the user)

So this is why the script doesn't seem to work... As I don't really understand nor program python, it is quite difficult for me to solve... May you help me out?

Best
JP

The links you found are just what you were looking for. Underneath the hoods there is this reason:

(Taken from the URL: http://mds.mandriva.org/ticket/136, for exemple, but there are more like those)

The syntax you refer to (use full DNs as group memberships) is defined in RFC2307bis (not published). With that syntax, a group looks like this:
dn: cn=group,ou=group,dc=example,dc=com
cn: group
objectClass: groupOfNames
objectClass: posixGroup
member: uid=john,ou=people,dc=example,dc=com
gidNumber: 12345

There are a few problems in using this today: - smbldap-tools doesn't support it - posixGroup is defined in RFC2307 as a structural class, which conflicts with groupOfNames. So you really need to use the RFC2307bis schema, and not the RFC2307 one (also called nis.schema) - nss_ldap, when used in this mode, does nested group lookups, which can quickly explode if you have many members in a group (see http: //bugzilla.padl.com/show_bug.cgi?id=319): each member will trigger another ldap lookup.

Note that you shouldn't just change the contents of memberUid to be a DN: the memberUid attribute was not designed to hold DNs.

For us, I believe the biggest obstacle now is smbldap-tools.

And we use smbldap-tools at the company... And the RFC2307. It does seem we're right and we probably should have a "member" full DN and not a memberUID, but while these tools are not completely refactored, may we create some kind of "workaround" on your script to enable RCF compliance on Unix NIS Schemas?

Looking at more recent RFCs, like RFC 4519, shows a DNs being used to identify group members. (http://tools.ietf.org/html/rfc4519#page-11) I think this is the latest RFC, I'm not sure. To my defense, I ran my script successfully, and so did a few clients, on Active Directory and LDAP for certain. I thought we had a Novell directory user but I cannot remember. We'll figure it out.

I can help. What do you want this to do?

Thank you so much for your help!

The only difference from what you have now to what I really need is (and still keep what you have) is:
1. Define a switch on the command line for telling the script if the memberAttribute is a DN or not (--member-attribute-is-dn or -c) with values "yes/no"
2. Define a switch on the command line for telling the script how to find the user object based on the member attribute (--user-ldap-filter or -f) with an ldap filter syntax (e.g. : &((objectClass=user)(uid=%memberUid)) )

And then, dump as usually... Does this fit ok with your current script?

Best regards

This is a very crude solution for our problem, but of course now the other situation isn't supported.

You could either detect that 'member' is no DN, an switch between methods. Or, add parameters to switch the behaviour.

DIFF:
109d108
<
168,170c167
< if verbose:
< print("Query: DN: "+base_dn+", filter: "+"(&("+userid_attribute+"="+member+")("+user_query+"))")
< user = get_ldap_search_resultset(base_dn, "(&("+userid_attribute+"="+member+")("+user_query+"))", ldapobject)
---
> user = get_ldap_search_resultset(member, user_query, ldapobject)

This script seems to work great but I'm having one issue.. When I run the script its putting the generated groups info is being put at the end of the authz file. Even when I move the groups information around in the file, in subsequent runs the script moves the generated info to the end. This wouldn't be an issue except the authz parser seems to care where the groups info is, ie it has to be the first section of the file.

That sucks. I've never noticed this before. I mean, I do know the groups are being put at the end but I didn't think the groups section had to be first section. Let me do some research and I'll update.

I've a Domain with far more groups than I want added to Subversion, so I added a -f/--filter regex option to your code. Here is the patch (I think it's pretty self-explanatory...):

53a54,56
> # This is the regex of groups that we wish to import
> group_filter_regex = None
>
123c126,128
<       groups.append(entry)
---
>       groupname = entry[1]['cn'][0]
>       if re.match(re_group_filter,groupname):
>         groups.append(entry)
392a398
>   global group_filter_regex
405a412
>   group_filter_regex = options.group_filter_regex
423a431,433
>   parser.add_option("-f", "--filter", dest="group_filter_regex",
>                     default=".*",
>                     help="The regex applied to groups to include")
466a477,478
>   if (group_filter_regex == None):
>     return False
492a505,507
>   if (group_filter_regex == None):
>     unset_properties += ['group_filter_regex']
>
517a533,540
>   # validate the group_filter_regex
>   global re_group_filter
>   try:
>     re_group_filter = re.compile(group_filter_regex,re.IGNORECASE)
>   except re.error, error_message:
>     print("'%s' is an invalid regular expression. Error: %s " %(group_filter_regex, error_message))
>     sys.exit(1)
>

Hi,
I found this script when I was googling for subversion and LDAP group support. As there are no native support, I plan to use this script to keep the LDAP and svn groups in sync.
First of all, we use OpenLDAP and we have to keep our groups in the posixGroup objectclass, so we have the same problem as JPereira and stevenodb. Thanks to stevenodb, I patched the script and now I can get the groups from OpenLDAP.

But, running the script with the -z flag, to regenerate the authz file, throw the following exception:

Traceback (most recent call last):
  File "sync_ldap_groups_to_svn_authz.py", line 558, in ?
    main()
  File "sync_ldap_groups_to_svn_authz.py", line 553, in main
    print_group_model(groups, memberships)
  File "sync_ldap_groups_to_svn_authz.py", line 348, in print_group_model
    os.rename(tmp_authz_path, authz_path)
OSError: [Errno 18] Invalid cross-device link

It would be nice, if you can take this issue into account, and fix it in the next release.

Keep up the good work,
Levi

Thanks for the insight Levi. I'll get cracking on a new release shortly.

Bug on Windows:
You needed to insert os.close(tmp_fd) to have the script releasing the file pointer before doing os.remove as Win cannot handle that situation otherwise.

Regards

Hi Jeremy,
When I run the script I get this error:

Traceback (most recent call last):
File "sync_ldap_groups_to_svn_authz.py", line 555, in ?
main()
File "sync_ldap_groups_to_svn_authz.py", line 545, in main
memberships = create_group_model(groups, ldapobject)[1]
File "sync_ldap_groups_to_svn_authz.py", line 186, in create_group_model
mg[0][0][0]))
TypeError: cannot concatenate 'str' and 'NoneType' objects

Some ldap handling improvements. I would consistently get errors about outstanding lookups, etc when trying to run the script. By changing the ldapobject.bind to ldapobject.bind_s, these problems went away.

ldapobject.bind_s(bind_dn, bind_password)

Some other improvements:
In the "search_for_groups" function, change the return groups to the following:

groups.sort()
return groups

Additionally, you may want to clean up the ldap object's connection correctly which I have modified with the following.

  try:
    ldapobject = bind()
 
    try:
      groups = search_for_groups(ldapobject)
    except ldap.LDAPError, error_message:
      print("Error performing search: %s " % error_message)
      sys.exit(1)
 
    if groups and len(groups) == 0:
      print("There were no groups found with the group_query you supplied.")
      sys.exit(0)
 
    memberships = create_group_model(groups, ldapobject)[1]
 
    print_group_model(groups, memberships)
 
  except ldap.LDAPError, error_message:
    print("Could not connect to %s. Error: %s " % (url, error_message))
    sys.exit(1)
 
  finally:
    ldapobject.unbind()

sorry, forgot to attach the diff to the previous post solution 1. Still altho no idea about solution 2

--- sync_ldap_groups_to_svn_authz_orig.py 2009-04-15 19:39:23.000000000 +0100
+++ sync_ldap_groups_to_svn_authz.py 2009-10-29 18:39:48.000000000 +0000
@@ -71,6 +71,10 @@
# [Example: /opt/svn/svn_authz.txt]
authz_path = None

+# This is the fully-qualified path to the CA Cert file for LDAPS.
+# [Example: /etc/svn/ca.crt]
+ca_cert_path = None
+
################################################################################
# Runtime Options
################################################################################
@@ -96,6 +100,8 @@
def bind():
"""This function will bind to the LDAP instance and return an ldapobject."""

+ ldap.set_option(ldap.OPT_X_TLS_CACERTFILE, ca_cert_path)
+
ldapobject = ldap.initialize(url)

ldapobject.bind(bind_dn, bind_password)
@@ -389,6 +395,7 @@
global user_query
global userid_attribute
global authz_path
+ global ca_cert_path
global verbose

(options, args) = parser.parse_args(args=None, values=None)
@@ -402,6 +409,7 @@
user_query = options.user_query
userid_attribute = options.userid_attribute
authz_path = options.authz_path
+ ca_cert_path = options.ca_cert_path
verbose = options.verbose

# load_cli_properties()
@@ -411,6 +419,8 @@
usage = "usage: %prog [options]"
parser = OptionParser(usage=usage, description=application_description)

+ parser.add_option("-c", "--ca-cert", dest="ca_cert_path",
+ help="The location of the CA Certificate file")
parser.add_option("-d", "--bind-dn", dest="bind_dn",
help="The DN of the user to bind to the directory with")
parser.add_option("-p", "--bind-password", dest="bind_password",

My plan is to digest all of the suggestions here and release a new version. The code is old and probably should be rewritten but I've just not had the time. (Old as in I don't usually use that Python style anymore.)

Thanks for script,
I couldn't run, and got this error
Successfully bound to ldap://dc01.mydomain.com:389...
Error performing search: {'info': '00000000: LdapErr: DSID-0C090627, comment: In order to perform this operation a successful bind must be completed on the connection., data 0, vece', 'desc': 'Operations error'}

I found the solution here
http://stackoverflow.com/questions/140439/authenticating-against-active-...

namely, I added set_option in you def bind() and script ran smoothly
def bind():
ldapobject = ldap.initialize(url)
ldapobject.set_option(ldap.OPT_REFERRALS, 0)
ldapobject.bind(bind_dn, bind_password)