Evaluating Informal Learning
September 23, 2013 3 Comments
As some of you know the problems associated with capturing the organisational impact and increases in competency (if any) gained by staff through their informal learning has occupied my thinking for some time now, and I have posted and spoke about it on a number of different occasions. I really got to thinking about it again after the recent learnX conference, particularly after some stimulating conversations with Con and Saul amongst others. The problem for me is that even if you don’t believe the numbers in 70:20:10 (which I don’t) there is still a lot of informal learning that happens in a person’s life and at least some of that learning has to relate to their job roles. Just before I go on however, I want to set some shall we say boundary conditions to what I am talking about here and that is informal learning that has some impact on the day-to-day operation of the organisation. If you choose to go off in your own time and as one of my colleagues loves to call it, study underwater basket weaving, I am really not interested in the fact that you have done that, unless you can show be some tangible link to your day-to-day work.
So for me there are two types of organisational informal learning, that learning that simply increases, builds or improves a skill and that learning that does that and in addition provides some ‘formal’ recognition pathway for the learner. When I think about this however it is more the job role or the organisational imperative that moves us towards the path of recognition rather than in general the needs of the staff member. Again however, when I am talking about recognition I am talking about formal recognition, where there is some ‘qualification’ style outcome as a result of the learning, usually in an Australian sense a Unit of competency for example. I am not talking about badges and other like devices to capture learning, be they peer-reviewed or whatever, for from an organisational point of view they in my opinion are (at least currently) meaningless.
Why do I say this; I have often recounted a story of being asked as part of a formal investigation into an incident involving a member of staff, “how did you know this person was competent?” Now if my answer had been, well he has a badge for it, I think I may have gotten a much different reaction to saying, as I did, “Well, they have completed all of the assessment tasks, including a third-party observation, necessary for us to be satisfied that they were competent under the rules of evidence set out in the legislation pertaining to the operations of Registered Training Organisations.”
Not all informal learning though is going to lead a staff member to a qualification however, some of it is not related to or captured by the range of qualification available, some simply adds to the skill set they already have, making them better at their role, but not providing them with a new skill. So for me there are a number things that we need to know in order to be able to begin to evaluate informal learning, and I am indebted to Saul Carliner for some of his thinking around this;
- A baseline – what is the staff members current skill level,
- What they have learned,
- How they have learned it,
- New skill level,
- Is there a competency attachable to the learning,
- Effect of learning on organisational metrics – reduction in customer complaints, less injuries etc, and maybe
- Return on Investment?
Now you might look at that list and say well isn’t that what we would want from any kind of organisational learning process, and in fact is not exactly the kinds of data that we want from our formal processes. I don’t see why we should be treating the outcomes of informal learning differently to how we would treat formal learning. Now it may not be relevant, we may not or able to or we may not want to, capture the standard ‘smile’ sheet satisfaction style data that we collect from formal training and yes, the natural of the learning, pull not push, driven by the individual, just in time etc, all make the nature of the process of learning different. However, when we look at it from an organisational point of view aren’t we not looking for the same thing as with formal learning. We are looking for an increase in the skill level of the staff member, such that increase in skill will have an effect on the relevant workforce metrics that relate to their role in the organisation. If we aren’t looking to improve the skills of our staff and the organisation as a whole, what are we investing in informal learning systems in the workplace, and why has it become so important.
So what are some ways in which we can achieve this? If we start with the idea of a baseline we might be able to sort out some structure and processes around this idea. So, where might a baseline come from;
- Position Description,
- Performance and Professional development plans,
- Self Assessment,
- Formalised Assessment, or
- Job skills analysis plus a rating system.
There are a range of ways in which we can establish this baseline, but how can we do it without it being onerous on everyone involved. If we use position descriptions as our starting point, we have the problem of there not being enough detail or they are not skills based or we don’t assess the person against them in a really formal way, that gives us any real data to work with in the first place. They could be coupled with self assessment, and direct manager assessment to give a fuller picture of the skill set and levels of an individual staff member. PPD plans can be seen in the same light, in order to make them more useful in terms of presenting us with a baseline we need to capture more granular detail about the role and the staff members skills relating to that role.
So I have a bit of a rough process around my thinking in relation to this and it goes something like this;
- Skills Outline for each role type within the organisation,
- Assessment of Staff member against skills outline – there are a range of options here, but I think there has to be at least self assessment + manager assessment at the very least,
- System for capturing staff informal learning activities,
- Regular (6-12 Month) updates of Staff skill assessments,
- Data capture of changes in skills levels across the organisation,
- Method of mapping skills changes to competencies. and
- Methodology of converting skills changes to organisational metrics and ROI.
So as I said that is my thinking currently, I would be really interested in getting everyone’s feedback on what they think, so feel free to chime in and let me know what you think.