Learning in a digital ‘cyberpunk’ world #LRN2024

A lot of you have probably come across the concept of a brain/computer (wetware/hardware) interface which allows people learn new skills, obtain knowledge and interface directly with other systems through science fiction movies and novels (William Gibson’s work for example)  and recently there was a paper published which seems to show the first documented brain to brain interface.  After my recent post for #lrn2024 and  question from a friend of mine Eric, I started to think about the effects on this kind of process on learning and the acquisition on knowledge and skills.

Let me set the scene for you first and then we can begin to discuss what impacts these ideas may have.

Think about a world where the need to learn skills and obtain knowledge in a traditional manner is no longer necessary, rather when one needs a particular set of skills or knowledge one simply ‘installs’ in much like installing a new piece of software on a computer or perhaps more like running portable apps on a computer rather than installing anything, but both ideas tend to work in the same way though as we may see there may be more permanence in case than the other.  We might call this kind of learning, if we decide that it is in fact learning and not something else, Interface Learning, that is where skills or knowledge are acquired through the utilisation of some for of interface. So if we take something simple (though in true actually quite complex) like driving a car.  I have learnt to drive an automatic vehicle, however in a particular instance I need to drive a manual  vehicle, so I simply ‘chip’ the skills and knowledge into by brain through some kind of wetware/hardware interface (think a USB port just behind my ear) and I am able to drive the manual vehicle with the skill and precision of a formula one racing driver.  So what then happens when I no longer need to driver the vehicle?  Well there would seem to be two options;

  1. I could simply remove the ‘chip’ removing the skills and knowledge from my brain much like disconnecting a usb drive running portable apps, or
  2. The skills are installed in brain by the process and thus left there, much like installing software on to a computer

both of these options would, it seems, have advantages, so lets look quickly at the two options and then we can look at what I think the real problem that exists behind this sort of technology might be.  The advantages to the first option are simple and really the same as the disadvantages, I never actually need to know very much at all, I just need to have a sufficiently large cache of ‘chips’ to provide me with the skills and knowledge that I need for particular circumstances, perhaps even being able to ‘chip’ multiple sets of skills and knowledge at once to accomplish complex tasks or tasks requiring a wide range of skills and knowledge.  The advantage would be that I could spend my time occupying my brain with whatever I chose to do with it and not need to spend multiple years learning skills and obtaining knowledge.  Of course the disadvantage is that if there is a problem with ‘chip’ then there is a severe problem with my ability to do the things that I would need to do.   So maybe this is really an augmenting technology where skills that I don’t require often, or high specialised or complex are those that I would ‘chip’ in while more basic skills were learnt in a more traditional manner.

So lets look at the second option, where I install the skills and knowledge as I need them but they remain there like programs on a computer hard drive.  There seems to be less problems with this sort of option as, as with software I would simple need to ‘click’ on it and the skills would be available to be again, or once installed they would ‘run in the background’ much as skills and knowledge tend to do now.  Think about however, what happens with computers, and we could well say already happens with our brains currently, hard drives get full and we have to delete things (we forget or lose access to our memories), software and hardware are no longer compatible, files and systems get corrupted and no long work in way they originally did, if at all, and all of the programs running in the background fill up our available ram and all of our processes slow down or blue screen.

There is however to my mind another issue with all of these ideas and that is what happens to our skills and knowledge over time and where do new skills and knowledge come from.  If I no longer have to practice a skill or utilise my knowledge then it is liable I think to stagnate.  Take again the example of driving a car I have been driving a car for nearly 30 years, and my driving has changed substantially over that time, I am a far more competent driver now in a wider range of vehicles than I was when I was 18, and I have learnt things about driving in particular areas or circumstances which are particular to that area or circumstance.  If however, all I had ever done when I needed to drive a car was to chip the skills and knowledge, drive the car and then turn the knowledge off when I was finished, my knowledge of driving a car may be the same for the most part every time I drove, year after year, particularly if I only drive on limited occasions.  I am also faced with the issue of skills upgrades what if I want to drive better, drive a truck as well as a car, or a wide range of cars, with changing configurations, will the chip that I have be able to cope with all of these permutations, or will I need and upgrade as the years pass by to cope with the changing world.  There in also lies the other issue, if this ‘chipped’ learning becomes the predominant means of obtaining the skills to achieve tasks, then where will these skill upgrade come from, will there be artisans who specialise in developing skill sets in more traditional ways, so that this skill and knowledge can be copied and transferred to others.

I would be really interested in hearing your thoughts on this as it has started some deeper thinking for me on this idea of interface learning and skill acquisition.

 

VET vs University – A continuing Divide.

As some of you who have been reading my blog for a while will be aware I have always been troubled by the seeming divide which exists between VET and University education in this country.  This came up again recently when the very articulate Lauren Hollows  asked this wonderful question on Linkedin; “Why does it have to be VET or HE?”.  Quite early on in the life of this particular iteration of this blog, I presented a similar thought.  Lauren’s post and the ensuing discussion prompted me to think a little more about this problem and why it is that there seems to be a divide between Vocational Training and University Education.

Lets jump in the time machine and go back to the dim past when I was in the final years of high school and looking at what I was going to do with myself post secondary school.  The choices were pretty clear-cut back then, you left at year 10 and got a trade, you went on to year 12 and University or you just went and got a job. somewhere and to a large extent what we know now as the VET sector now was still a few years away.  This I think is still some of the problem today, a lot of people not involved in the industry, who are now parents etc saw this divide, you went to  TAFE to do a trade or you went to university and of course the unspoken thought was that the reason you left at year 10 and went and got a trade was that you weren’t going to get good enough results in 11 and 12 to get accepted into University.  Now whether or not that was ever true, the mindset was there and still is, people still view VET as a choice you make when you can’t get into university.

Let’s fast forward to today though, this is not the case anymore and hasn’t been the case for some time now, sure VET education can be seen as an alternative education pathway, but it is also a supplementary or complimentary pathway.  As a lot of the respondents to Lauren’s post said, myself included, a lot of people now have qualifications from both sectors, all of which provide them with different learnings and different skills and knowledge.  So why then do we still hear comments like “I have a degree why would I was my time getting a Certificate IV/diploma?”  We hear them because I think we have failed, all of us, the Government, the peak bodies, the providers to truly explain the post secondary education system in this country to people, and to explain it to people in such a way that makes sense to them and shows them the value of education regardless of what ‘sector’ that education comes from.

We have a single framework in this country for qualifications and we have had it since 1995.  The Australian Qualifications Framework outlines who the whole system works and what each level from Level 1 (Certificate I) to Level 10 (Doctoral Degrees) work and what the skills and knowledge at each level is.  I would hazard a guess however that very very few parents and student and probably not a lot of teachers and guidance officers were terribly aware of the content of the AQF and even fewer would understand how the system works and what all of it means and there in lies the problem.

What we need to do in this country is to embark on an education process, a process designed to explain to people simply and easily how the system works.  If we ever as a country truly want to have an engaged workforce built on ideals of lifelong learning, then we need to do this we need to this, we need to explain to people the choices that they have and how they fit together.  If we don’t there will always be a divide between the various educational sectors in this country and that would be a crying shame.

Technology – Helping or hindering learning?

Mobile Learning is the next big thing!

We need to gamify that content to engage the learners!

Stunning bite sized e-learning will promote just in time learning on the job!

 

Sometimes these days when I listen to all of the chatter at conferences, online and at meeting and events etc  about the world of Learning and Development I can’t help but wonder if we aren’t perhaps just sounding a little bit like the new song by Weird Al

 

Weird Al Yankovic – Mission Statement

 

and I have to admit it worries me.  It sometimes feels to me like the direction of our thinking is being push or nudged in certain directions by the needs and wants of vendors, both of content and systems, rather than being driven by the needs and wants of learners.  This should not be taken to be a criticism of vendors in general (what is it they say ‘some of my best friends are vendors’) , it is their job to promote their products and services as much as they can and to be fair L&D folk seem to love new technology, new ways of connecting and new things to explore, I know I do.  However isn’t in the long run the outcome for the learner and in a lot of cases the organisations they work in what is important.  I see lots of stuff about how new technologies help learners in Higher education, school etc and this seems to be used as evidence that the same things will work in organisations or in other types of learning environments and as I have said before I am not quite so sure that is the case.

I am happy to accept that there are instances of organisations fully implementing these new technologies and having fantastic results, there seems to be a number of ‘case studies’ and ‘anecdotal evidence’ to suggest that it can be successful.  However there also seems to be quite a range of stories out there about it not working for one reason or another, usually because user engagement was an issue, or to paraphrase that statement – staff didn’t want to do organisationally required learning in their own time,  they wanted to do the training in a face to face environment, or they wanted something really hands on, not simulated.

I guess what I am saying here is that flipped learning might be great in K-12, MOOC’s might work for universities, gamification might engage GenY learners, but do these things actually work or work as well in organisational settings, or are the expectations, needs, wants and outcomes of the people we train and the organisations work with, not a great match for some of these things despite what the ‘research’ might say.  After all how unbiased is an article or paper on the virtues of gamificiation if it is written or sponsored by a gamification vendor.

Sure it is great to explore all of the new and wonderful ways in which we can engage learners and provide truly outstanding outcomes for our clients, but in the long run shouldn’t how we deliver learning be based (at least in part) on who the learners want to learn.

 

 

 

Learning Spaces or Spaces to Learn – What can we learn from delivering training to the homeless.

The concept of where and how learning programs are delivered has been on my mind a little bit lately, particularly since a particularly good presentation I attended recently on the interface between homeless persons and training delivery.  One of the key points which was bought up during the presentation and subsequent conversations was the fact that if we take a group of people like those who either homeless or at risk of homelessness, we will tend to find that there are a raft of other issues that sit with and around the issue of homelessness and all of these issues will have a significant impact on the delivery of training programs to people within these groups.  These impacts are things like;

  • a mindset of failure particularly around academic/scholastic pursuits
  • uncomfortableness in traditional learning environments (classrooms)
  • limited ability to travel to get to training venues
  • limited support network
  • possibility of having to move a significant distance from where training is conducted to secure accommodation
  • limited financial means

These issues and a range of others mean that it is difficult if not impossible to deliver training within what could be considered traditional environments.  This means that learning programs need to be adapted and delivered in different ways such as;

  • within the environment where the person already is and is comfortable
  • shorter sessions to allow participants to take care of their other priorities (it is difficult to concentrate from 9-5, but imagine how much more difficult it would be if you were worried about finding a bed for the night)
  • a wide range of learning activities to engage participants in a variety of ways
  • changing assessment models to ensure that all participants are able to display in competence in ways that are most effective for them

The thing is when I started to think about developing and delivering learning programs, particularly workplace programs it struck me that most of the adjustments that I was considering were things that we should be doing anyway.   We spend large sums of money on creating physical spaces for people to learn in, or online platforms delivering state of the art gamified elearning, when in reality the participants are probably going to learn more from a 2 hour session held in the staff room, coupled with solid support tools to allow them implement the things they have learnt.

And to be honest I think the problem might be us, it is far more challenging to deliver training in a staff room, a homeless shelter or a skate park where there are a range of other things happening in the background, than it is to deliver the same training in our lovely state of the art training room.  Walking though an instruction manual or workbook with a participant is far less fun for us than creating sexy video content or gamifiying our learning programs, but does it make the participant more comfortable and able to learn better.

We need to be able to create spaces for people to learn, that fit with what they need, not with what makes us comfortable.

Competency based, Time based or something in between?

I have been involved in a number of conversations recently about training (what a shock), but in particular about how long it takes to train some one.  The easiest answer here seems to be well as long as it takes, different people will learn at different rates so the amount of time it might take me to learn something may be radically different from the amount of time it takes you to learn the same thing.  Now essentially that is the right answer, particularly where we are talking about skills based training, if I am able to demonstrate that I can perform a skill to whatever criteria are necessary then surely I have demonstrated that I am competent and shouldn’t need to spend additional time on ‘learning’ that skill simply because it should take say 5 days to learn that skill.

I have a question though, lets take the example of making a cup of coffee.  I attend an online training course called making a cup of coffee, the course is delivered in a state of the art simulated coffee-making environment, includes a range of videos from the worlds best coffee makers and lots of reading questions to think about. Oh and there is  a ‘skills assessment’ at the end of the course where I have to make a cup of coffee in the simulated environment.  I answer a range of questions about making coffee and do a project on how coffee makers work, well to be fair my project is about how the simulated coffee maker works and my answers are largely regurgitated from the videos and printed materials and it takes me less than a day to complete.  These questions are assessed and found to be satisfactory.  I then undertake the ‘make a coffee skills assessment’ and pass with flying colours.  Am I competent?

Some people would clearly argue yes, of course I am competent I have done all that is required to show my competence in making a coffee, however what happens when I go out into the workforce with my making a coffee qualification, get  job, and find myself confronted with a coffee machine that is utterly unlike the one I have used on every previous occasion (in the simulated environment) and my consumers are much more demanding about their java than my simulated customers ever where, and struggle to manage to make a cup of coffee.  Am I still competent?

Would the situation have been any different if there was a requirement that the course took a minimum of 6 months and 100 hours of placement to complete or are we going to get exactly the same result.  Am I more likely over this extended period of time to encounter situations and equipment and people who stretch my skills then I would have been with entirely online or face to face training plus a practical skills assessment.

When we add to this the added dimension that in most cases I am not undertaking this training alone, but rather as part of a group, a group which has both widely ranging skills and abilities, including how fast they learn, how does that affect not only my competent but the competence of the other members of the group.

 

The first thing to say here is that I am not bashing online training, I like online learning and find it really useful for what it is.

What my focus here is is the question that if we are truly serious about competency based training then  surely we need to recognise that there is for the most part actually a minimum amount of time that it takes someone to become competent in a particular skill.  Now for me whether that time comes from work experience before entering training or through work placement or on the job training is unimportant, what is important for me is that there is minimum amount of time and that that minimum should be part of the recognition of competency.  If you have only ever done 40 hours of work in an aged care facility in your entire life and that was a training placement, then no matter how good the training is you have received I am going to really, really doubtful that you are actually competent across any real range of situations and environments.  There is simply not enough time for you to have experienced enough variety of situations to be able to be competent, even if you have done hours of simulated, intensive, innovatively delivered training to go along side this.  (To caveat this, of course there may be a very small range of people who after this type of training are competent, but in my opinion and experience not many)

 

We need to have a system that ensures that when someone is given a qualification that they are actually competent and one step towards achieving this seems to me to be the concept of mandating at least the level of placement hours that various qualifications need.

That’s what I think anyway.

 

Chasing Butterflies – Evaluating the organizational impact of informal learning

Turning informal learning into measurable business outcomes

We all learn things all of the time.  It is part of being human; we pick up a snippet of information during a conversation that we remember later, which makes whatever we are doing easier or quicker.  We watch a video on the internet to show us how to do something we haven’t done before or can’t quite remember how to do.  We read, we interact with people and things and we learn informally every single day.

Informal Learning and ideas like 70:20:10 have been on the minds of Learning and Development Departments and organisations for a while now.  Everyone knows that staff learn while they are at work, and that while a proportion of that learning comes to them through formal means, another proportion comes to them informally, through talking and interacting with others, through reading and watching videos, posting on forums, attending MOOC’s and this learning can happen both inside and outside of working hours.

What does all of this informal learning mean for organisations though, what kinds of impacts does it have on the overall performance of the organisation or the individuals which make it up?  Does it increase productivity and efficiency, does it increase the competency of staff, and does it improve the bottom line?  Anecdotal evidence suggests that it does, and if you believe the pundits and evangelists for the value of informal learning, it is far more valuable to an organisation than all of the formal learning that happens and it is the thing that organisations should be investing in and assisting their staff access in order to maximise their ability to operate in their chosen markets.  Is this really the case however, and is it the case across the board for organisations, does informal have a real and significant business impact, does it make a business and its staff better at what it does, and does it provide individuals and the organisation with identifiable increases in competency, efficiency and overall effectiveness?

This post will not actually argue the case either way for informal learning, it will not pit informal against formal learning and it won’t make an argument for an investment strategy based around informal learning.  What this book will seek to do however is;

  1. Look at what organisations need to think about when they think about the value of informal learning,
  2. Look at the information an organisation needs if it seriously wants to determine the value of informal learning, and
  3. Look at how to capture that information and what methodologies to use to make sense of it.

This post is also only concerned with learning that has some organisational impact, where the skills and knowledge gained by the learner, is translatable in some way to their current or future role within the business.  If a persons what to go off in their own time and as a colleague of mine likes to call it, study underwater basket weaving, then that is fantastic, but it is only of interest to an organisation if some formal link to an improvement in day-to-day work can be seen.

 

 

What is formal and informal learning?

 

Formal Learning is any course or program designed using industry-recognised disciplines and methodologies that have a formal structure, and specific, well-defined learning objectives, which may lead to formal outcomes or qualifications for the participant and which is delivered through a form of Student-Teacher relationship.  That is formal learning is what we are all used to as learning, it is the learning we did at school, at university and at the courses and programs that we attend at work.  It is structured learning, which has learning outcomes defined not by the participant, but in the training itself.

So then what is informal learning?  One could say that it is simply everything else, every other way in which we manage to learn new skills and be fairly accurate.  Informal learning is where there is no set learning outcome, at least not one that is set by anyone other than the participant.  It is unstructured, not designed using robust principles of instructional design and often not even seen as learning by the participant.  It is as I have said before the day to day conversations, the videos, the books, the jobs that we do, they all for part of our informal learning.

 

 

Evaluating informal learning

 

If we look at the definition of informal learning as opposed to that of formal learning, the problem in terms of evaluating its impact can be clearly seen.  There are no outcomes that can be looked at to see if the participant has successfully achieved, it is not delivered in traditional methods and formats, there is by its very nature no assessment as part of it and in a lot of cases we may not even be sure when or where the learning itself actually occurred.  So it is no wonder we struggle with not just the concept but with the practicality of how to evaluate the informal learning that staff do during the course of their days and what if any personal or organisational impact that learning may have.

It seems to me that there are two types of informal learning which are important to organisations, learning that simply increases, builds or improves a skill and learning provides some ‘formal’ recognition pathway for the learner.    It is important to note that it is some kind of formal recognition that is important here, recognition, where there is some ‘qualification’ style outcome as a result of the learning, where the participant is assessed in some way prior to being deemed as competent in the skill or knowledge.  I am not talking about badges or other such methods to capture the results of informal learning, be they peer-reviewed or not, because from an organisational point of view they in these methods are (at least currently) virtually meaningless.

Why meaningless; I have often recounted a story of being asked as part of an investigation into an incident involving a member of staff, “How did you know this person was competent?”  Now if my answer had been, well he has a badge for it, I think I may have gotten a much different reaction to saying, as I did, “Well, they have completed all of the assessment tasks, including a third-party observation, necessary for us to be satisfied that they were competent under the rules of evidence set out in the legislation pertaining to the operations of Registered Training Organisations, in Australia.”

This is not to say that the person with the badge was not competent, or the one with the assessments didn’t make a mistake or had forgotten what they had learnt.  It is to say however that it seems more likely that we can be confident that the second person was actually competent than we can be of the first.

Is this to say then that if informal learning does not lead to some kind of qualification that it is of no use, or much less use than what we would normally see as formal learning?  No, not all informal learning though is going to lead a staff member to a qualification, some of it is not related to or captured by the range of qualifications available, some simply adds to the skill set they already have, making them better at their role, but not providing them with a new skill. This suggests that any process of evaluation that we may seek to apply to informal learning must be capable of dealing with both that learning that will lead to a formal outcome and that which will not.

To this end there seems to be a number of things that we need to know in order to be able to begin to evaluate informal learning, and I am indebted to Saul Carliner for some of his thinking around this.  In order to be able to evaluate effectively the impact of informal learning both at an individual participant and an organisational level we will need the following information;

 

  1. A baseline – what is the staff members current skill level,
  2. What they have learned,
  3. How they have learned it,
  4. New skill level,
  5. Is there a competency attachable to the learning,
  6. Effect of learning on organisational metrics – reduction in customer complaints, less injuries etc., and maybe
  7. Return on Investment?

Interestingly this list looks very much like what we need know about any learning process in an organisation.  This of course then leads to the question, if this is what we need to know to evaluate the effectiveness of organisational learning, why are we treating the outcomes of informal learning differently to how we would treat formal learning.

Now it may not be relevant, we may not or able to or we may not want to, capture the standard ‘smile’ sheet satisfaction style data that we collect from formal training and yes, the natural of the learning, pull not push, driven by the individual, just in time etc., all make the nature of the process of learning different.  However, when we look at it from an organisational point of view are we not looking for the same thing as with formal learning?  We are looking for an increase in the skill level of the staff member, such that increase in skill will have an effect on the relevant workforce metrics that relate to their role in the organisation.  If we aren’t looking to improve the skills of our staff and the organisation as a whole, what are we investing in informal learning systems in the workplace, and why has it become so important.

So how do we achieve this, the process outlines what we need to have in order to really make this idea work;

  1. Skills Outline for each role type within the organisation,
  2. Assessment of Staff member against skills outline – there are a range of options here, but I think there has to be at least self-assessment + manager assessment at the very least,
  3. System for capturing staff informal learning activities,
  4. Regular (6-12 Month) updates of Staff skill assessments,
  5. Data capture of changes in skills levels across the organisation,
  6. Method of mapping skills changes to competencies. and
  7. Methodology of converting skills changes to organisational metrics and ROI.

 

 

Creating a baseline

 

If we start with the idea of a baseline we might be able to sort out some structure and processes around this idea.  So, where might a baseline come from?

  • Position Description,
  • Performance and Professional development plans,
  • Self-Assessment,
  • Formalised Assessment, or
  • Job skills analysis plus a rating system.

But how can we do it without it being onerous on everyone involved.  If we use position descriptions as our starting point, we have the problem of there not being enough detail or they are not skills based or we don’t assess the person against them in a really formal way that gives us any real data to work with in the first place.  They could be coupled with self-assessment, and direct manager assessment to give a fuller picture of the skill set and levels of an individual staff member.  PPD plans can be seen in the same light, in order to make them more useful in terms of presenting us with a baseline we need to capture more granular detail about the role and the staff member’s skills relating to that role.

To really make this process as robust as possible there is a definite need to skill groups determined for each role type within an organisation.  These skill groups are entirely separate to any particular position description and are tied to role types and levels rather than to specific staff or positions.  Now to what level of granularity an organisation is going to need to go to is going to have to be determined by each organisation.  My thinking however is that for most organisations there would be general role types to which skill groups could be attached.  The skills contained in these skill groups would also have some similarity through the hierarchy of the organisation, everyone in the organisation needs to be able to communicate, but the level of skill expected may well be different.

 

Once we have the skill groups established, we then have to come up with a way of assessing the current level of competent staff have in relation to the skill group which applies to their role.

Again a number of ways of doing this have been suggested;

  • Self-assessment
  • Peer assessment
  • Manager assessment
  • Formal evaluation and testing

The problem associated with most of these suggestions is that they either don’t really provide us with strong evidence of skill levels or competency or they are or are likely to become onerous and time consuming.  Self-assessment is probably not going to provide us with the robust kind of evaluation that we require, particularly if we are looking at this assessment process as providing us with a baseline from with to build evidence towards a qualification from.

Peer assessment and management assessment may also fall into the same trap and in addition they may also depending on the number of assessments and the number of people managed, become onerous and time consuming.  While formal assessment processes certainly could meet the criteria in terms of robustness, there is again, depending on the kind of assessment and how it is delivered the real problem of it becoming expensive and time consuming for everyone concerned.

This is of course less of a problem with new hires into the organisation than it is with those staff currently employed.  With new hires a system which captures their current skill levels with respect to their specific skill group can be quite easily developed and maintained, providing baseline data, at least on that group relatively easily.  The reason for this is that the process for capturing the data can be made part of the recruitment and induction process.  Current staff are going to be the group for which capturing this data will be most difficult, as any process is going to additional to their current workloads.  One way would be make the Performance and Professional Development review process more robust and include a form of assessment against the skill groups as part of it, in addition to staff and manager perceived assessments of skill levels.

All of this data begins to present us with other problems as well however.  How and where do we capture this data so that it is accessible and usable both in terms of individual staff progress and advancement in the relevant skill areas, but also on an organisational level where roll ups of this data could and would be invaluable in determining trends and needs across the entire organisation?  How this will be achieved will of course a decision that individual organisation make according to their current systems and needs.  It is vital however that this and the other information that is going to be generated through this process is captured, because it provides a wealth of information not just for the organisation, but for the individuals themselves.  It allows the organisation to see the value and the benefit of its learning investments, be they formal or informal and allows individuals to track their own progress towards a variety of goals.

 

 

What they learnt and how they learnt it

 

Having the baseline data available gives organisations a place to start when looking at the evaluation of learning, be it formal or informal, however one of the problems with capturing and evaluating the effects and impact of informal learning, is quite simply identifying when and how it happened.  Unlike traditional formal learning, there are no classes, no events, no calendar of activities to point to, to show when a staff member undertook training on a particular subject.  There are no learning outcomes, no standard content to point to establish what is a participant may have actually learnt.  This has and does present a significant problem for evaluating the impact and effects of informal learning, if we don’t know what, when or how someone learnt something then how can we determine where that knowledge came from?   Unfortunately from the perspective of this work there is little to talk about here except to say that organisations will need to find a way to capture this data, which is meaningful to them and suits their needs and individual situations.  One method would be to establish a database of ‘learning activities’ which staff could update as needed or which could capture data from a range of sources using technology solutions around a staff members interactions with informal learning opportunities.  There is no single answer here and no single best way of achieving this.  There will even be differences of opinion about the level of granularity that is sufficient to show that a participant has in fact had an informal learning experience.  Again, as with the baseline data what is important here is that data of some description that is meaningful to the organisation and individual is captured.

 

 

Changes in skills and knowledge

 

We have captured the baseline skills data across individuals in the organisation and we have also captured when and how they are learning, what is needed now  is a method or a process of regularly assessing individual’s fluctuations in the skills within their skill group.  Again it is going to be up to an individual organisation how robust they make these assessments, at the lowest end would sit, I think, a 6-12 month self-assessment by staff of where there skills now sit.  At the highest end would be some form of regular, controlled, formalised testing process which provided solid evidence in relation to changes in key skills.  If we were thinking about say desktop applications such as Word, staff members could at set time intervals be required to undergo a standardised external test of their skills and knowledge, which should give solid, meaningful data on changes to skill levels.  The other point of note here is that there needs to be a correlation between the kind of assessment done at this stage and initial baseline assessment.  While some variety of methods of assessment will work and provide meaningful data, this cannot be said of all methods.  For example if the initial assessment had be a rigorous formal evaluation, carried out through validated assessment tools, administered professionally, but then when it came to the point of reassessment a simple self-assessment tool was utilised, it may be the case that the results of the data are not as useful as they could be.  My thinking here would be simply that the same type of assessment, even the same assessment should be used to detect changes in an individual’s skills and knowledge; otherwise we are simply not comparing the same things.

 

 

Metrics and measurements

Once this data has been collected we are at some point going to have to decide what it means either on an individual level, an organisational level or both.  It is here, I think, that the real challenge may lie.

If we look at the following example; an organisation has been capturing data on its staff skill levels across, they have also been providing staff with access to and encouragement to utilise and learn through informal methods.  After 12 months they see an average rise in skill levels across the organisation of 5% and over the same period a rise in sales of 3%.  Can we make a connection between the two, is there any correlation between the increases.

At least at this point I think you would be hard pressed to make much out of it.  It may even be difficult to make the correlation between the informal learning and the rise in skill levels, why?  Well simply because unless all they have been doing is engaging in informal learning and there has been no formal activities, or coaching and mentoring, supervision etc., then we are going to need to come up with a process of separating out the gains make from formal learning and the gains made from informal learning.  At this point we might be tempted to fall back on something like 70:20:10 and say something like, well 70% of learning is informal so 70% of the skills increase is due to informal learning, namely a 3.5% increase.  I that I would and I think that a lot of other people would find an argument like that to be somewhat less than satisfying.  It could be the case that it was the 10% formal learning component that produced the entire 5% rise in skill and informal learning did not contribute to the skills increase at all.

Even if we can make a strong case that informal learning contributed to the skills increase, we then still have to make the case that the skills increase contributed to the sales increase and unless we can show a link between the learning, the skills and the increase in sales that is going to be difficult.  Here I think it is the level of granularity that we apply to the data we are collecting.  If we can show that members of the sales department accessed a range of informal learning resources all related to closing sales, and that their skill level at closing sales went up (by whatever means of assessment we are using) and then the overall sales figures for that sales department increased, then I think we might have a strong enough case to suggest that the informal learning the staff did, had a correlative effect on the sales figures. For me however, this is still quite a big question mark.  It is hard enough to do real, robust return on investment calculations with traditional formal learning; with the complexities surrounding informal learning it is much more difficult.  Does this mean it is impossible?  No, it simply means that organisations need to really figure out what it is that they are looking for in terms of success and what kinds of metrics and measurements will provide them with evidence of that success.  If an increase in sales is an indicator of success and sales increase there has been a success, the problem comes with trying to determine what it was that caused this success.

One way which might allow us to hone in more on what it was the catalyst for the changes and therefore the success would be apply a process such as Brinkerhoff’s ‘success case method’ for evaluating training.  If we formally ask those staff who met the criteria for success what they did, or what they think was the reason they were successful and we formally ask those who were not as successful what they did and what they think was the reason they were not as successful, we can then compare the sets of answers.  This sort of process may give us some solid insights into whether there were specific things which were the root cause of the success.  The successful staff may tell us that they think the formal training was responsible for about 50% of the success, because it gave them the skills they needed, but the other 50% came from the fact that their manager was supportive of them utilising the new skills they had learnt.  On the other hand staff might tell us that one member of the team found a really good YouTube video that had a lot really helpful ideas and skills in it and shared it with the entire group and the successful staff were the ones that watched and applied the information in the video.  This sort of information would give us data from which we could begin to look at what were the significant drivers of the successes.

 

 

The competency connection

 

The other part of the puzzle around the evaluation of informal learning for me is the link to formal qualifications and measurements of competency.  Now for non-Australian readers and readers not familiar with the Australian competency based vocational education training system some of this might seem a bit foreign to start, but essentially it is all part of the same puzzle.  One thing that I think it is necessary to be clear on here is that I am talking about formally recognised types of qualifications, there is a lot of talk about things like badges (peer-reviewed or otherwise) and other types of ‘endorsements’ of people skills.  These are not what I am talking about here, and from a talent management, recruitment and learning perspective I see little value in these badges and related concepts.  What I am talking about here are formally recognised kinds of qualification or certification where there is a robust, standardised and formalised assessment attached to the awarding of the qualification itself.

One of the values I see coming from informal learning is the ability to use the information collected about the kind and type of learning undertaken and results this learning has caused, is that it provides evidence which could be used to show that staff members met the requirements for certain formal qualifications.

Take for example the following;

A staff member, who is in a retail role and has a desire to move into store management, begins to take some online courses/MOOC, both in their own time and with the support of their manager, at times when they are able to during their working hours.  The manager also allows them to undertake some stretch tasks around stock management and ordering and financial management.  The staff member also actively becomes involved in a number of online and face to face discussion groups with other retail managers and staff around increasing store sales, better stock management and staffing and HR issues in the retail environment.  As a result of these activities the staff members own personal sales increase, they receive numerous positive compliments from staff, the manager is comfortable letting them handle some of the stock ordering and closing off of the store financials at end of day.  The manager and the staff member then decide that as a result of this the staff member should look at undertaking a formal qualification around retail or frontline management and decide that they will undertake the Certificate IV in Frontline Management (An Australian Vocational Education Qualification).  If good records of the learning activities and achievements have been kept by both the manager and the staff member, they may find that significant parts, if not all depending on the length of time and the amount of experience the staff member has, of the qualifications requirements may have already have been met and that the staff member may be able to undergo a process, in Australia referred to as recognition of prior learning, to show evidence of their competency and be awarded the qualification by  the appropriate authority.

This for me is one of the great values that lies untapped as part of the informal learning process, that is transitioning from just learning that may have quite a significant outcome in terms of both the business and the individual in terms of quality of work, productivity, effectiveness etc. and adding to that the additional value of giving the staff member access to formal qualification outcomes.  This will be of particular advantage where the staff member in question does not have formal qualifications either in general or in the particular area they are working in.  This is a definite value added outcome for the concept of formal learning and one that I think really needs to be explored further.

 

 

Conclusions and final thoughts

 

Informal learning is an incredibly valuable tool for both organisations and individuals, it provides individuals with the learning that they need, when they need it, in environments which may be very conducive to their learning.  It provides organisations with an avenue to reinforce and build on the skills and knowledge that staff obtain through formal training activities and to provide them with additional skills through alternative methods of learning.  We need however to understand the value, both in terms of what that value is and what that value means for organisations.  Utilising anecdotal rules of thumb about the value of informal learning and what it provides are not useful tools, particularly when considering investment strategies around Learning and Development in organisations.  We need to treat to treat informal learning like any other part of the learning business and make sure that we know what our success criteria are, what the real value is and the return on investment that we are getting for informal learning.  We also need to embrace the concept that informal learning can be a pathway for staff to formal qualification outcomes and that this can only enhance the value that is gained both individually and by the organisation from its investment in informal learning.

 

Creating a High Impact Learning Culture

LearnKotch

L&D from a different perspective

C2C Consulting & Training blog

Learning & Organizational Development

Spectrum Training - Registered Training Organisation

Learning, Development and Training from an Organisation and Enterprise Perspective

South Metro Education Region VET Network

VET Information and Updates for the South Metro Region

North Metro VET Network

News and updates on all things VET in the North Metro Region

Learning

Leadership, Confidence, Success, Value

Learning Rebels

Fighting the Good Fight for More Engaging and Innovative Learning in the Workplace

Gen Y Girl

Twentysomething. Annoyed with corporate BS. Obsessed with Gen Y. Not bratty. Just opinionated.

VET Sector News from Velg Training

Australia's leading provider of Vocational Education and Training (VET) professional development and consulting services

Learning to Fly

Musings from an HR practitioner working things out as she goes along.

NIIT Managed Training Services

Helping Clients Run Training like a Business

Effective Intercultural Business

Learning from each other, leveraging the differences, doing better business.

It's time to tell a story...

Alex Holderness for IQPC

Total Learning

Ideas on training and development for businesses

kotch2010

Just another WordPress.com site

E-Learning 24/7 Blog

The Truth and Realities of E-Learning

Thoughts on management

Management now and tomorrow

Saul Carliner

Linking to My Many Sites

E-Learning Provocateur

A blog by Ryan Tracey

Legal Resources for RTOs

VET Regulation under ASQA

Follow

Get every new post delivered to your Inbox.

Join 230 other followers