Grouper performance (issues)

One of the main benefits of upgrading Grouper last year was the introduction of “real-time” provisioning of groups and memberships into AD. Previously, AD syncing had occurred four times a day which was good enough for most scenarios but not perfect for everyone.

Since upgrading, the Grouper PSP technology, which handles “real-time” AD provisioning, has coped nicely with everything that’s been thrown at it (averaging around 50,000 changes per month). This chart, showing monthly group membership changes by stem, gives an indication of what it’s handled from March to August 2016. You can see it was busy in August and there’s a peak in April.

Chart showing monthly updates to Grouper groups, by stem, from March to August 2016

Chart showing monthly updates to Grouper groups, by stem, from March to August 2016

It had coped nicely, that is, until the end of the academic year. Now, if we add September 2016 into the chart, it provides a nice visualisation of why PSP has been suffering for the last fortnight.

Chart showing monthly updates to Grouper groups, by stem, from March to September 2016

Chart showing monthly updates to Grouper groups, by stem, from March to September 2016

So, since 1st September we’ve not had any “real-time” provisionsing and the situation has been far worse than it was prior to our upgrade last year, with some changes having to wait well over a week before being reflected in AD. We’ve offered some workarounds to alleviate the impact of the most urgent cases but this service failure still weighs heavily upon me.

As I write this, I’m hopeful that the provisioning service will finally catch up with itself overnight tonight and we’ll return to the happy state of “real-time” provisioning tomorrow.

So, whilst I’m quite content that PSP can cope with our needs for the majority of the year, the service since the start of September has not been satisfactory. We’ve now started making the necessary plans to replace PSP so that we won’t have to suffer like this again next year.

Getting to grips with Grouper

Yesterday, Phil and I delivered a short overview session on Grouper, covering what it is, what it’s for, why you might like to use it and some tips and lesser known features.

The session was aimed at both new and existing Grouper users at Newcastle University (anyone who doesn’t consider themself to be an expert). We were very pleased with the turnout (about double what we expected) and the feedback.

If you missed the fun (or if you were there and want to hold on to the experience for a bit longer) you can now enjoy the full session on ReCap and peruse a copy of the slides at your leisure.

By the way, I’d like to offer a belated apology for the Star Wars references; I’ve recently watched the original trilogy with my daughter (her first time) so I’m afraid it was on my mind. I should’ve pointed out that Grouper is not quite as powerful as the Death Star.

Principles, tips and good practice for Grouper administrators at Newcastle University

As I’ve watched the uptake of Grouper grow within the computing officer community over the last year (with many new and interesting use cases), I’ve seen many people face the same problems and struggle with the same dilemmas. The more I’ve helped people tackle these challenges, the clearer the solutions have become in my mind and the more I’ve realised I should try to document some of it. So, here it is, my list of principles, tips and good practices for Grouper administrators …

  1. Use a ‘User Group’ to determine who has admin privileges on all of your groups and folders.
  2. Whenever possible, use ‘Corporate Data’ groups to control memberships of your groups.
  3. Where adequate source data doesn’t exist to define a ‘Corporate Data’ group, create a ‘User Group’ if you’re likely to use the same set of members in more than one place.
  4. Only create ‘Applications’ groups if you need them to be provisioned to AD or available as Shibboleth attributes.
  5. Adhere to the naming convention for ‘Applications’ groups.
  6. Make ‘Applications’ groups specific to a service or purpose.
  7. If you are delegating control of a group’s membership, give out the ‘update’ privilege, not the ‘admin’ privilege.

I realise it’d be useful to supplement this post with a higher-level overview of how Grouper works and what I mean by ‘Corporate Data’, ‘User Groups’ and ‘Applications’ groups. We’ll endeavour to get that written soon.

Give out the ‘update’ privilege, not the ‘admin’ privilege

Our seventh principle of Grouper good practice is:

  • If you are delegating control of a group’s membership, give out the ‘update’ privilege, not the ‘admin’ privilege.

One of the strengths of Grouper is that it enables you to delegate privileges to others to maintain membership of their own groups. If you’re doing this, you should assign the update privilege, which allows the updating of group memberships.

You should not assign admin privileges lightly. As well as allowing for the updating of group memberships, the admin privilege also allows the user to rename, move or delete the group. Or they could even remove your admin privileges, leaving you in a bit of a hole if you need to fix something.

Make ‘Applications’ groups specific

Our sixth principle of Grouper good practice is:

  • Make ‘Applications’ groups specific to a service or purpose.

This should ensure that each group provisioned to AD has a clear purpose and avoid any unforeseen consequences when a group is deleted, for example.

Additionally, requirements can change. Initially you might want to control access to a number of services with the same group but, over time, the people you want to grant access to might diverge from the original group. This scenario is best served by having specific ‘Applications’ groups all containing the same ‘User Groups’ group as a member. This gives the flexibility to allow the memberships to diverge, if required.

Adhere to the naming convention for ‘Applications’ groups.

Our fifth principle of Grouper good practice is:

  • Adhere to the naming convention for ‘Applications’ groups.

The naming of groups is important to ensure the consistency and uniqueness of group identifiers between Grouper and Active Directory groups. It is a requirement of the AD that group names are unique.

So, to ensure uniqueness, the group ID is split down into three defining sections:

  • Owning department/school – the school/department should be the abbreviation (without the leading D-) that is assigned to your school/department within SAP for example COMP (Computing science), LIBR (Library).
  • Auto – the word “Auto” needs to be included within any group that is to be provisioned into the AD. This identifies the group in the AD as being automatically generated and therefore management of the group should be carried out within Grouper.
  • Group purpose – this should provide a clear purpose for the group, so that users are able to quickly distinguish what the group represents.

This gives a group name of the format <Department/school>_ Auto_< Purpose>.

Please note that the group ID cannot include spaces so, if the purpose is more than one word, please replace spaces with an underscore character. AD will also not accept any of these characters, ” [ ] : ; | = + * ? < > / \,  so do not use them in your group names.

Only create ‘Applications’ groups if you need them

Our fourth principle of Grouper good practice is:

  • Only create ‘Applications’ groups if you need them to be provisioned to AD or available as Shibboleth attributes.

There are two reasons for this:

  1. The GrouperGroups section of the AD is already quite busy and cluttered, adding unnecessary groups will only make it worse.
  2. The Grouper to AD provisioning process is quite inefficient and can slow down if there are a large number of changes. Minimising the number of groups being provisioned to AD will avoid any unnecessary slowing down of the provisioning process.

Create a ‘User Group’ if you’re likely to reuse the same set of members

Our third principle of Grouper good practice is:

  • Where adequate source data doesn’t exist to define a ‘Corporate Data’ group, create a ‘User Group’ if you’re likely to use the same set of members in more than one place.

This is a principle based around reuse, in order to save time and avoid inconsistencies and errors.

The classic example is for a research group. This fictional research group might be made up of some staff from School X, some from School Y and a few PGR students. There is nothing in our corporate data systems to identify that these people belong to this research group so we cannot create a ‘Corporate Data’ group for them.

Now, let’s say this research group wants to set up a mailing list, a wiki and a shared filestore. (These are all things that can be controlled through Grouper.) Instead of manually maintaining the membership list of three different ‘Applications’ groups, create one reusable ‘User Groups’ group which can be the member of the three ‘Applications’ groups.

Use ‘Corporate Data’ groups to control memberships

Our second principle of Grouper good practice is:

  • Whenever possible, use ‘Corporate Data’ groups to control memberships of your groups.

This is a straightforward, unambiguous and just plain sensible principle.

Membership of the ‘Corporate Data’ groups are populated with data from our corporate data source systems and are updated automatically. Using these groups to build up the membership of your groups means that the members of your groups will be updated automatically, too.

User Groups for admin privileges

Our first principle of Grouper good practice is:

  • Use a ‘User Group’ to determine who has admin privileges on all of your groups and folders.

Whilst setting up another group for this purpose might seem like an additional overhead of time and effort before you really get started with Grouper, I assure you it’s worth it. There are a few reasons and considerations behind this.

Now, if there are several of you working together with Grouper and all want to have admin privileges on each other’s groups then this is just common sense; it’s much easier to grant privileges on your groups to a single group than to several of your colleagues.

But you might be thinking, “Hey, it’s just me here. I don’t need to share admin privileges with anyone else.” Well, that’s OK, I hear you, but, please still create a user group for this purpose (with you as the only member). I appreciate it’s a small hassle, but hear me out.

Today, it’s just you working on it. But what if that changes? In six month’s time, you might be lucky enough to get a new colleague to work with you. Do you want to have to go through all the groups you’ve created and grant them admin privileges? Or would you rather add them as a member of one single group that you bothered to spend a couple of minutes setting up to have admin privileges on all of your groups?

Or, another scenario, what if you move on? Now your successor doesn’t have privileges on any of the groups they need to look after. If only we had a single admin group we could add them to! Of course, if you’re moving on, you might not be too worried about that but I’d like to think we’re all conscientious enough to care.

My final thought on this is a little more contentious. I’d say you should set up and use a ‘User Group’ for controlling admin privileges even if there’s already a ‘Corporate Data’ group containing the right people. I must admit this isn’t something I’ve always done myself but as my thinking has evolved and developed, it’s what I’m always going to do in the future.

You can then simply use the ‘Corporate Data’ group to populate the members of your admin group. The reason for this is that, whlist the ‘Corporate Data’ group might be right today, we’ve seen how things can change with reorganisations and evolving responsibilities.

Using a ‘User Group’ for admin privileges, from the start, will future-proof your part of Grouper.