Research

Testing Your Own Designs: Discussion

by Paul Sherman on September 10, 2009 · 12 comments

in Design

I’m putting an article together for UXmatters on the topic of usability testing and validating one’s own designs. My goal is to develop some guidelines for self-testing.

I’d love to get your feedback on some questions I have:

  • Testing your own designs: all-around bad idea? Or is it possible to do it well?
  • If so, what should you do / not do when testing designs you’ve created?
  • What should you look out for?
  • Got a good story or anecdote to share? Please do. Either success stories or cautionary tales are welcomed.

Thanks in advance. -Paul

{ 12 comments }

The End of “Just Google It”? Twitter-Based Search Behavior

by Paul Sherman on September 8, 2009 · 14 comments

in Design

Let me be up-front about this from the beginning: this is a half-formed thought. But its implications are very, very interesting.

So here’s what just happened: I had a desire to take in some emerging thoughts in the user experience field. I wanted some fresh thinking, some exposure to new presentations.

For about 8 years, my first instinct has been a) browse to Google, b) type in “user experience”, and c) browse the list of returned search results to see if anything new strikes my fancy.

This morning, I didn’t do that.

Without even realizing I was changing an ingrained habit, I went to Twitter.com, typed “#ux” in the search box, and browsed the list of returned tweets. I clicked a few, starred a few, and made a mental note to come back and check out some of the links to preso’s.

Then I sat bolt upright when I realized what had just happened.

This, colleagues and readers, is an early warning. An indication that one of my consumer habits is open to change, and could very well tip into a new and different routine.

And if it happened to me, it can happen to any of us.

I don’t know if you’re grokking the import of what happened, so I’ll restate it: something in my sub- or semi-conscious mind decided that the resource I’ve been relying on for years might not be cutting it any more, and it directed me to try a more real-time and dynamic resource.

I will certainly follow up on this in future posts. But right now, I think this incident opens up several interesting research and design questions, such as:

  • How much of customer behavior is consciously directed, and how much is directed by the sub-, un-, or semi-conscious?
  • What factors influence customers’ willingness to change behavior?
  • What are the leverage points for changing customers’/users’ habits?

{ 14 comments }

Low-Rent Usability Tech Tip: Remote Usability Testing With Observers Watching

by Paul Sherman on August 29, 2009 · 10 comments

in Everything Else

Today’s post is a simple little usability testing “tech tip.” It’ll help you run a remote usability test with a participant running an app or browser on your test machine while other remote observers are watching the session.

My investigation into this started when I was asked to conduct remote usability test sessions for my client. (I am an independent user experience consultant who does both interaction design and usability testing.) The client asked if they would be able to listen and watch the test sessions in real time.

Now I know that there’s probably an expensive software package or two out there that would give me the capabilities I needed, but I wasn’t in the mood to spend $800 – $1,2000 USD on something I may not use for another few months. So I poked around the Internetz and discussion groups (including the usability listerv that dares not speak its name…) looking for guidance on how to run usability test sessions with a combo of tools that met my requirements.

In my search I found a suggestion for employing GoToMyPC for participants to access the test system. The reasons for using it were pretty compelling: GoToMyPC offers extremely low latency and high performance. And most importantly for me as the test moderator, it lets me and the participant trade off control of the test system in a modeless manner. That is, when I want to quickly trade control of the test system I can do it on GTMPC without having to go to a menu and select “give (or take) control.”

That still left me with the problem of how to allow observers to watch. Then it occurred to me that any old online meeting service would do; all I’d have to do is run it on the test system, invite the observers to the meeting, and distribute a phone conference bridge to all parties.

So I tried it this week, and it worked like a charm: the test participant had low latency and high performance as they used the application, I had the ability to assume control from the participant as needed without wasting time and unduly interrupting the flow of the test; and observers could watch the test session unfold in real time. (They could also pass me questions via the meeting chat capability or via out-of-band IM.)

So, here’s the deets: for remote usability testing with remote observers, here’s what you need:

  • A GoToMyPC account
  • An online meeting account (most if not any will do)
  • A phone conference bridge

And here’s how you set everything up. Please note that this requires that you be sitting at the test system; aka the target PC for GTMPC:

  • Designate your test system as the target PC for the GTMPC service. This is the PC you want to remotely control.
  • Temporarily change your GTMPC login, password and target PC access code for use with the test participants. You’ll be sending them these credentials, so make sure you’re not using your “standard” personal usernames and passwords.
  • When you’re ready to run a test session, convene an online meeting from the test system with your observers, and allow the observers to view the test system’s desktop.
  • Send your test participant a link to gotomypc.com, with instructions on how to log in and enter the test system’s access code. (It’s pretty easy, there’s not much to it.)
  • The test participant will then be signed in to the GTMPC service and can control the test system…as can you, so be careful not to “wrestle” for the mouse too much.
  • Run the test session.

I’m sure there’s more tricks you could be add on to this basic setup. Here’s one idea: you could record the session using the online meeting service’s recording capabilities.

And it also seems possible to get the remote test participant’s face into the picture somehow, as many online meeting services provide web cam integration. Note that this would require inviting the test participant to the online meeting as well as having them sign into the test system via GTMPC, so be careful before testing this capability. Not to be an alarmist or anything, but on the face of it I’m guessing that this could easily tear a hole in the space-time continuum and open a portal to parallel universes. Or something like that. Just sayin’.

{ 10 comments }

Teaser 2: UPA 2009 UX Industry Salary Survey

by Paul Sherman on August 18, 2009 · 0 comments

I know, I said I’d release the UPA 2009 UX Industry Salary Survey last Friday. Unfortunately, paying work and family time prevented me from finishing the report.

But I’m telling you now that I’ve submitted the “members only” full report and the public report to the UPA office, and they should have it posted within 24-36 hours from now, Tue 18 August.

Oh, and here’s another teaser:

  • The average salary in the UX field for men and women combined is $85,283. For women, the average salary is $84,892. Men’s average salary: $85,947. This is the closest that men’s and women’s salaries have ever been since we started surveying the field in 2000. The gender gap appears to be gone, folks.

You can see the details, including salary by job description and other analyses, when UPA posts the public version of the report. UPA members will be able to get the full report, which includes some really in-depth statistical analyses, behind the member login.

{ 0 comments }

It’s About The Unknown Problems…What Do You Do About Them?

by Paul Sherman on August 4, 2009 · 6 comments

When performing user/design research, us UX researchers go into the field with ideas about the problems you’ve been asked to solve. But it’s nearly always the case that we come out of research sessions having identified a whole host of new and unexpected problems. This is so common, I’ve even added a section to my standard report template to account for the unexpected problems (and possible solutions).

I argue that it’s how you handle the “unexpected” problems that differentiates the good from the great UX researcher.

Discuss…

How do other practitioners handle the “unexpected” class of problems? How do you account for serendipity in your deliverables, communication with clients, etc?

{ 6 comments }