I wanted to share a case study with you from a project I did a few years ago.
The process of reflecting on whether my activities have worked for the organization has been key to helping me improve my work.
– The 4 weeks spent introducing the pilot and the interfaces made a huge difference when we got to the training event. For the trainers, this required a lot of working around non-working equipment, cribbing screenshots from vendor documentation, and a lot of creative writing.
– The pilot users liked that they had a resource and lots of information. Sometimes too much information. The sheer amount of information wound up making the solution seem more complicated than it really was. I would rather err on the side of too much than too little, but the feedback tells me I need to be more mindful about how to deliver this material and how I present the options. Production training and support will require more careful information architecture to make it all less overwhelming.
– 2 hours of classroom training is 1 hour too long. Production training will be split up into 4 or 5 separate classes + asynchronous materials. This will allow people to choose what they wish in the time they wish to use it. Yes – there is pressure to get everyone to switch to a new way of doing things all at once. I learned (again) that we need to respect the change process. For some – it’s just getting comfortable with a phone. For others – it’s solving a problem they have. Mileage will vary.
– The whole project team and the pilot participants ran out of steam after a month from Go Live. Because certain things still weren’t working, I ran out of material. The pilot participants got tired of hearing from me. And things kinda settled into a groove. I was too tired to shake that groove up.
– The Subscription-Based Learning model is still very human-resource intensive. As in, you need at least one person who can solely concentrate on creating material for that model. This person also winds up serving as the community wrangler – despite attempts to direct people to appropriate resources. That person becomes the name and the face of the project whether the team wants that or not. People prefer to talk to individuals, not anonymous “Support Centers.”
– The Subscription-Based learning model is really nice for introducing new features as the engineers and architects get stuff working. Having that in my back pocket reduced pressure for the entire project team to get certain features operational on Day 1. As long as we had the core working (chat, phone calls, 1-1 video, desktop share and the path between these features), the other stuff could be introduced later.
– We wound up abandoning IdeaScale. I had a hard enough time with my email and no one else on the team had the bandwidth to step forward and be the community curator. I was still impressed with the level of engagement from the pilot participants – even if most of it seemed like complaints.
– The Business Analyst assigned to our project introduced me to Qualtrics. We used this for regular surveys instead of Google Forms. I’ll admit – for my purposes, I much preferred Google Forms. But Qualtrics makes it easier to create pretty pictures.
Having the 1 month survey and results from the previous surveys let me know that the solution we have is a good one – we just need to get it working better. Very useful when all it felt like I was hearing was complaints. Actually – I was very impressed we had a 33% answer rate to the survey (vs. the 10% after chasing people down that I was expecting).
Things we need to figure out moving forward:
– Our mucky muck’s executive assistant commented “I’ve been using this thing and it’s fine. But I am not sure how to go about using the newer stuff. Maybe I need more hand-holding.”
What she really identified is that people figured out a way to get comfortable. There is nothing pushing her to try something else or do something differently because she doesn’t have a pressing problem that needs to be solved. I’m not terribly interested in creating problems that aren’t there. But there are also better ways to do things housed in this solution. Time for another round of talking to people.
– Information Architecture – how can people find what they need when they need it? This is a HUGE solution. How do we make it less overwhelming?
– Accommodating UI changes – What we used in the pilot is going to look very different when we start rolling it out to the University. So all of the materials I developed are being trashed and I am starting over. We are in an age where vendors are changing UIs at whim anyway. Our development processes need to figure out how to accommodate this reality quickly, and not just for this project.
I have a lot more work to do between now and December 1 when our first group goes live with this solution.