Assessing Computer Users: “Easy” May Not Always Equal “Best”

5-year-old kid sitting at his computer desk.
Ayan Qureshi: the 5-year-old kid who passed the Microsoft Certified Professional exam.

This 5-year-old passed the Microsoft Certified Professional exam. Granted that the kid may be special, but, seriously, what does that say about the Microsoft Certified Professional exam?

I’ve taken the Microsoft Office Specialist exams for Word, Excel, and PowerPoint, and they were pretty dang easy. I wonder how much of a true measure of a computer user’s ability these tests are. I mean, the instructions are all there. The test instructions say stuff like, “insert a photo,” “make the text bold and red,” “move slide 1 to become slide 5,” and so on. So, the tests may accurately asses IF a user can do a specific given task using the software, but my main concern is that, in a real world situation, there is nobody around to tell a user WHAT to do, or WHICH tool to use WHEN. In the real world, a computer worker is told to “create a flyer.” Maybe the boss will bring in a flyer that they saw in the mall and say, “make it look like this.” But the boss is not sitting there next to you saying, “merge and center cells A1:D1, and format using the Title style.”

Such assessment — of a user’s ability of HOW to use a software tool — may be easy to implement, but I am skeptical of its ability to  accurately assess a different and more important dimension of computer users’ abilities: WHEN to use WHICH tool to solve various real-world problems.

In my BTS 165 Excel classes, the final project assignment is to use Excel to solve a real-world problem. Students must find a problem  to solve, then use Excel to solve it. It doesn’t have to be a huge problem, but the application of the tool to the problem must be theirs. While assessing the student’s success is a considerable challenge — especially in the amount of time it takes to work with students to understand their decisions and reasoning at several key points in the weeks-long process — and even then, perhaps open to some amount of subjective interpretation on the teacher’s part, I feel that this project is a much more accurate assessment of a student’s understanding not only of how to use Excel’s various tools and features, but WHICH tools and features to use WHEN.

Sometimes “easy” does not equal “best.”

Canvas’s Roll Call Attendance Tool

I spent some time the other day playing with Canvas’s newish “Roll Call” Attendance tool. It has some good stuff and some bad stuff…

Good Stuff

I like that I can enter attendance data one time– in Canvas– and the data goes straight into the gradebook. This saves me a bunch of time over my current method of attendance, which consists of:

  1. In class, students pass around a “sign-in” sheet. Sometimes, students who are actually in attendance forget to sign the sign-in sheet, and once per quarter or so, I actually lose the sheet before it safely gets to my office.
  2. I manually transfer these sign-ins to an Excel spreadsheet that I’ve spent countless hours on refining to compile requisite points, and to give me all sorts of information about class attendance. I occasionally make mistakes in this transfer.
  3. I manually transfer attendance points from my spreadsheet into Canvas’s gradebook. Again, I occasionally make mistakes in this transfer, too.

Each of these three steps provides opportunities for problems, errors, mistakes. This is why I wanted to try out the Roll Call Attendance tool.

Bad Stuff

Screenshot of Roll Call Attendance tool
(click image for full-size version)

Two problems: one relatively minor, one relatively major.
The minor problem: most of my classes consist of two separate sections/class codes, and the students do not all appear one list; instead, I have to switch between the two sections via a tabbed interface. That is, half of my students are listed in one tab while the other half are listed in another tab. And, when I switch between tabs, the date always returns back to today, even though I’m entering attendance data for a different day.

But, as I say, that is just a minor inconvenience. The main problem, however
— the killer, as far as I’m concerned– is that the Roll Call Attendance assignment counts 100 points. Period. There is no way to change how many points it counts. No matter how many times the class meets, the attendance column in the gradebook is worth exactly 100 points.

The idea, I think, is that it’s a percentage. So, you must weight your assignment groups. I don’t. Weighting confuses the heck out of me. My gradebook philosophy is “a point is a point. If I want an assignment to count more than another assignment, then I make it worth more points.”

So, no matter how much “good stuff” the Roll Call Attendance tool offers, it simply will not work for me. I’m bummed.

However, if you weight your assignment groups, you really should consider trying it out. If you’re interested, let me know, and I’ll pass along the information you need to enable it in your Canvas class.

Canvas Course Syllabi Can Now Be Publicly Viewable

Yes, check it out– here’s my Fall BTS 161 Syllabus:

This is very cool. I have had prospective students ask for my class syllabi, and I’ve had to email old ones to them. Also, now my department can point to these syllabus pages.

However– one thing I do NOT like about the way Canvas does this: it shows all the assignments. Why do I not like this? Well, the assignments are automatically drawn from the Canvas course, and I use Canvas for assignments. What if one of my peers does NOT use Canvas for assignments? The public will think that HER class is “easier” than mine because she has no (or fewer) assignments, and register for that other class rather than mine. NOT COOL. Canvas should NOT show assignments on the public syllabus page. I’m cool showing assignments to the students, just not to the public.

Another change coming to Canvas’ Syllabus tool: it will be a *form* that instructors can just type their own class data into. This form will include fields for “Instructor Info,” “Course Description,” “Required Materials,” etc., *and* it will automatically drop in all the official college boilerplate text that instructors currently have to manually drop in ourselves.

Change. The only consistency…

Teaching and Training

First week of Fall quarter classes done. The first week of each quarter is nice, because I don’t have any homework to grade, and my classes are pretty much set up– syllabi are done, the quarter’s assignment schedule is done, each class’ Canvas site is done… So, the first weekend is pretty slow. A chance to exhale.

So, I’ve been thinking a bit… Because “the economy” is “better” (whatever the heck that means), enrollment is down for all of my department’s classes. BTS enrollments are all down this quarter compared to Fall 2013. And that’s not good for me, nor for my BTS teacher counterparts, because we are making less money. And, as it is, it’s pretty tough for adjuncts here at BC to make ends meet here in Seattle, where the cost of living is among the highest in the country. Now, of course I’m not complaining, because it could be much worse, and of course I’m grateful that I have a job in the first place, and of course I didn’t go into teaching for the money, blah-blah-blah.

But many people here at BC may not know that, in addition to teaching BTS computer classes here in the IBIT division, I also work part time for Information Resources as a trainer, with Sukirti Ranade in the Technology Learning and Connections Center in A109. If you go to the TLCC website and look under “Peer-to-Peer Faculty Support Hours,” that’s ME! I’m the “peer” for teachers here at BC. As of this quarter, actually, that is only me, whereas for the last couple of years, it was me and Jim Dicus, English adjunct. But Jim is really busy this quarter and didn’t have time for the TLCC, so now I’m the only peer.

I really enjoy working at the TLCC and helping my fellow faculty with their technology needs. During those “support hours,” I just hang out in A109 and wait for the phone to ring or for faculty to walk in. But I also lead several training workshops for both faculty and staff each quarter. Look on the Training Calendar for all the workshops. This quarter, I’m offering Word Accessibility and WordPress trainings. Both of these sessions are great. We’ve been getting positive feedback about the Word Accessibility sessions, as the general consciousness about accessibility grows on campus.

But I’m especially excited about the WordPress sessions, because — well, this blog  is on WordPress, and in fact, ALL BC websites are now powered by WordPress… So, if you are in charge of your department’s website, or if you hope to some day administer an official BC website, or shoot, if you’d just like to use your own personal BC blog, then you may want to attend one of these sessions. And, speaking of accessibility, we will be talking about how to make your WordPress sites accessible too.

And while I’m talking about accessibility, I must tip my hat to BC’s new eLearning Manager, Ekatrina Stoopes. She has really spearheaded the college’s current efforts to “accessify” all documents and web sites. Since Ekaterina has been here, I have learned more about accessibility than in the entire rest of my life. So, thanks, Ekaterina.

Excel and BTS 165 Ruminations

Shiny Excel buttonI love Excel. I’ve been teaching Excel now for about five years, I guess (since the 2007 version, and we’re well into the 2013 version now), and have seen some cool new features.

Excel itself has been around for a long time. It’s older than about half my students, probably. The first version of Excel came out in September 1985 (Wikipedia), and was Mac-only (as was the first version of Word)– which I love to point out because I’m a Mac dude. So Excel is a very mature program. Microsoft has had a lot of time to add new functionality and power.

And I love teaching. So, put Excel and teaching together, and I love to teach Excel. Here in the BTS department here at BC (our BTS 165 class), we’ve used three different textbooks/curriculum systems– a different one for 2007, 2010, and 2013. For Excel 2007, we used Pearson’s Go! series; for 2010, we used Cengage’s New Perspective series; and for 2013, we’re using Pearson’s Go! series again, but this time, it’s coupled with MyITLab.

Now, every textbook has its strengths and weaknesses. At the time we used 2007, I considered the Go! series too simplistic. I appreciated the New Perspectives way more– it seemed to be more comprehensive and challenging to the students. But it was also well-paced– it seemed to “scaffold” learning well– that is, it did a better job of preparing students for more challenging work. For instance, it had Capstone Projects every three chapters or so, which did a good job of wrapping up a good deal of functionality into a coherent project that was at the same time not out of reach for students. I used one of those capstone projects as a final project for many years in my Excel classes, with good results.

And I still use that same project, even though we use Go! again. Well, not in the summer– seven weeks, I have decided, is too short a time to smash in such a major project. But the next time I teach BTS 165, and in all the longer, 11-week quarters, I will use that final project.

But here is one thing that I really want to mention in this post: Chapter 2 of Excel in the Go! series of textbooks. Simply put, there is TOO MUCH STUFF in this chapter! It should be divided up into at least two chapters, and maybe three. Here’s a list of all the stuff covered:

  1. Flash Fill
  2. SUM
  5. MIN
  6. MAX
  7. Moving data
  8. Resolving error messages
  9. Rotating text
  11. IF
  12. Conditional Formatting
  13. Date functions
  14. Time functions
  15. Freezing panes
  16. Tables
  17. Sorting
  18. Filtering
  19. Viewing, formatting, and printing large worksheets
  20. Renaming worksheets
  21. Entering dates
  22. Clearing contents
  23. Clearing formats
  24. Copying and pasting with Paste Options Gallery
  25. Editing and formatting multiple sheets at a time
  26. Creating a summary sheet
  27. Column Sparklines
  28. Formatting and printing multiple worksheets at a time

And that’s it. That’s all. Only 28 features of Excel.

In class today, I went over these chapter objectives. It took an hour. (Thank goodness it’s a three-hour class!) The students were wiped out! Shoot, *I* was wiped out!

We took our break, then came back and started tackling the chapter. It went well. They’re good students, committed to doing the work, to understanding the material, to understanding not just “how” to do something in Excel, but also “why.” I love that. I love that, because, when they get out of this class (NEXT WEEK!), they’re not going to remember HOW to do most of this stuff… I therefore hope to instill a sense of familiarity with Excel’s interface, a sense that, even if they don’t know how to do any particular thing in Excel, they at least have a method of figuring it out.

And not just Excel, but the computer in general. I mean, a computer can be used for almost anything. That’s one of the main reasons that computers are so powerful– through programming, they can become a jillion different tools. No one person knows all of that stuff. It’s just too much! I mean, today in class, we looked at Excel’s Functions. There are 458 of them in the version of Excel 2013 that I have on this computer in my office.

“Nobody knows them all,” I told my students. “Not even Mike Girvin, the Excel Guru! So don’t feel bad if you don’t.”

And this brings me to another issue that I’m having lately… skills versus understanding. So, I was talking about my Final Project assignment, which, I think, is a pretty good measure of a student’s “real” understanding of Excel. I guess one of the main things this project does that is especially challenging to BTS 165 students is that they must create this workbook from scratch. This is new to them. In every single assignment, in both the Go! and New Perspectives series of textbooks, the data is provided for them. So, the fact that this one Capstone Project in New Perspectives on Excel 2010 challenges students to solve a “real world” problem means that they have to figure out a way to get that problem into an “Excellable” format! This reminds me of Alan Turing’s notion of “computability,” which is the notion that some problems are “computable,”– that is, solvable by a computer– while others are not. In this same way, Excel students are, for the first time, presented with this notion of “Excelability,”– that is, the question of whether a given problem is solvable by Excel. Up to this point, they’ve been fed problems that ARE “Excelable,” and then they’re just shown how to use Excel’s tools and functions to solve them.

What a great challenge! I mean, Excel can’t help me tie my shoes, can it? On the other hand, it can help me immensely with analyzing and understanding my inventory of products on hand.

… So, not just how to use Excel, but why. And when.

But, as I’ve said, this is not addressed in the Go! series nor the New Perspectives series (other than this one particular Capstone Project).

And, now, we’re using MyITLab. Now, MyITLab has some great tools for learning how to use Excel: the electronic version of the textbook includes hyperlinked glossary terms. Each chapter Objective is preceded by an accompanying video by one of the authors that shows how to do the stuff in that Objective. There are “Skill-Based Trainings” and “Skill-Based Exams” that train and test students on their ability to use specific features and commands. There are “Grader Projects,” which give students step-by-step instructions on how to complete a real project in Excel, which students upload for immediate grading and feedback. MyITLab is pretty cool.

These textbooks are all dedicated to teaching the technical skills, the how-to, of Excel. They’re useful references.  But how effective are they in preparing students for real life? How effective are the in preparing students for a job? I’m not sure.

Maybe it’s different for each student. Some students take BTS 165 because 1.) it’s a requirement for some certificate or degree program. Some take it because 2.) they need a job, and they’ve noticed that Excel knowledge is a common requirement for jobs these days. Some take it because 3.) they’re using it in their current life/job, and want to know more. Some take it because 4.) their parents told them to. Out of these four kinds of Excel students, BTS 165 is best for the third one: for a student who is already USING Excel, and who just need to know more about it, to learn some new tricks, to learn “the Microsoft way” (which is what I call the perspective a computer user must come from in order to best understand how to use Microsoft programs). Such students can continue to actively practice the new stuff they’re learning, to incorporate it regularly into their current lives.

But the other three kinds of students– the ones who won’t be using it every day– they’re going to have a harder time with BTS 165. I mean, they’ll be fine during the course of the quarter itself. But, when the class is over, their knowledge will immediately begin to evaporate because of non-use– the old saying, “if you don’t use it, you lose it.”

It is these kinds of students that our textbooks, and the MOS certification exams, do NOT help. Our textbooks and materials, prepare students for the MOS exams, which test whether a student can find the right tool/command/method for completing a specific little task in Excel. But how deep does an Excel user’s understanding need to be in order to pass such a test? Not very deep, indeed.

Now, I’m not dissing the MOS certifications. They serve a purpose. They look good on a resume. But, do they measure how well a user can use Excel to solve a real-world problem? In my opinion, not very well.

But there is a real challenge here, which may give some insight as to why there is such a dichotomy between the way our teaching/learning materials are structured and the way that I think they SHOULD be structured: assessment. I recall when my program chair presented our new course and program outcomes about a year ago. Essentially, our new outcomes are: “the student should be able to pass the MOS certification exam.” And the way she presented it, it made total sense: it is measureable! And, when it comes to the college “certification” process, one of the main things that certification board is looking for are measureable course outcomes.

It is much easier to measure whether a student passed the MOS test than it is to determine whether they understand how to use Excel to solve a real-world problem.

So, there it is. In one blog post, some of the major issues with Excel and BTS 165 that I have been thinking about for the better part of a full year.

Comments welcome.