August 2009

Low-Rent Usability Tech Tip: Remote Usability Testing With Observers Watching

by Paul Sherman on August 29, 2009 · 10 comments

in Everything Else

Today’s post is a simple little usability testing “tech tip.” It’ll help you run a remote usability test with a participant running an app or browser on your test machine while other remote observers are watching the session.

My investigation into this started when I was asked to conduct remote usability test sessions for my client. (I am an independent user experience consultant who does both interaction design and usability testing.) The client asked if they would be able to listen and watch the test sessions in real time.

Now I know that there’s probably an expensive software package or two out there that would give me the capabilities I needed, but I wasn’t in the mood to spend $800 – $1,2000 USD on something I may not use for another few months. So I poked around the Internetz and discussion groups (including the usability listerv that dares not speak its name…) looking for guidance on how to run usability test sessions with a combo of tools that met my requirements.

In my search I found a suggestion for employing GoToMyPC for participants to access the test system. The reasons for using it were pretty compelling: GoToMyPC offers extremely low latency and high performance. And most importantly for me as the test moderator, it lets me and the participant trade off control of the test system in a modeless manner. That is, when I want to quickly trade control of the test system I can do it on GTMPC without having to go to a menu and select “give (or take) control.”

That still left me with the problem of how to allow observers to watch. Then it occurred to me that any old online meeting service would do; all I’d have to do is run it on the test system, invite the observers to the meeting, and distribute a phone conference bridge to all parties.

So I tried it this week, and it worked like a charm: the test participant had low latency and high performance as they used the application, I had the ability to assume control from the participant as needed without wasting time and unduly interrupting the flow of the test; and observers could watch the test session unfold in real time. (They could also pass me questions via the meeting chat capability or via out-of-band IM.)

So, here’s the deets: for remote usability testing with remote observers, here’s what you need:

  • A GoToMyPC account
  • An online meeting account (most if not any will do)
  • A phone conference bridge

And here’s how you set everything up. Please note that this requires that you be sitting at the test system; aka the target PC for GTMPC:

  • Designate your test system as the target PC for the GTMPC service. This is the PC you want to remotely control.
  • Temporarily change your GTMPC login, password and target PC access code for use with the test participants. You’ll be sending them these credentials, so make sure you’re not using your “standard” personal usernames and passwords.
  • When you’re ready to run a test session, convene an online meeting from the test system with your observers, and allow the observers to view the test system’s desktop.
  • Send your test participant a link to gotomypc.com, with instructions on how to log in and enter the test system’s access code. (It’s pretty easy, there’s not much to it.)
  • The test participant will then be signed in to the GTMPC service and can control the test system…as can you, so be careful not to “wrestle” for the mouse too much.
  • Run the test session.

I’m sure there’s more tricks you could be add on to this basic setup. Here’s one idea: you could record the session using the online meeting service’s recording capabilities.

And it also seems possible to get the remote test participant’s face into the picture somehow, as many online meeting services provide web cam integration. Note that this would require inviting the test participant to the online meeting as well as having them sign into the test system via GTMPC, so be careful before testing this capability. Not to be an alarmist or anything, but on the face of it I’m guessing that this could easily tear a hole in the space-time continuum and open a portal to parallel universes. Or something like that. Just sayin’.

{ 10 comments }

ProductCamp Austin Summer 2009 – User Experience Presos

by Paul Sherman on August 26, 2009 · 1 comment

in Design

For some reason this slipped my mind for the last two weeks. On August 15th I delivered two talks at ProductCamp Austin 2009. Before I link you to the talks I wanted to give hat tips to the crew who put together this ProductCamp. It was a fantastic, energetic, and crowd-driven “un”conference, and I highly recommend attending one if you get the chance. They’re springing up in many major metro areas, so finding one shouldn’t be hard. You can learn more about BarCamps at this site.

The first talk I gave was “How To Achieve A Great User Experience For Enterprise Software” and the second was “From Personas To Production: The Role of Personas, Design Briefs, Stories, Storyboards, and Wireframes in the Ideation – Design – Build Process.” The second one had more people than the enterprise talk; which I guess shouldn’t really surprise me as the enterprise talk is more specialized. But the enterprise attendees were full of good questions and there was lots of good within-audience discussion. The feedback I’ve gotten on these two has ranged from slightly to strongly positive. So I’ll put them in the “win” column.

Oh and my past presentations are also available on Slideshare.net here.

{ 1 comment }

Many Questions About HuffPost & Facebook Connect

by Paul Sherman on August 23, 2009 · 5 comments

in Everything Else

Welcome to the brave new world of ubiquitous social networking and the mashing up of traditional media and social networking.

In the past I’ve held forth on how opaque I feel Facebook’s applications are about what information they share, and how uneasy it makes me to respond to cause requests, games, contests etc. on Facebook. This latest instantiation of Facebook’s “see everywhere, be everywhere” strategy makes me feel as skittish as ever.

I lean left, but with a broad libertarian / 2nd Amendment streak. What can I say; I’m a Texas Democrat. So I occasionally read the Huffington Post, although I don’t blindly agree with everything it says.

So this latest offer to publish HuffPost content to my wall doesn’t really sit well with me. And it triggers many questions, such as:

  • What content appears on my wall? Do I have control of what shows and what doesn’t?
  • Will wingnut comments from the stories appear on my wall?
  • What happens if I want to disconnect HuffPost content? How easy is that? Will I be able to?
  • What information will HuffPost gather about me if I sign up? What do they do with it?
  • If a Facebook friend or I comment on HuffPost stories; what can HuffPost do with the content?

Take note of that last one, folks. I actually clicked on HuffPost’s terms of use link in the dialog box to see if I could figure out what rights they were claiming. As usual, I was shown a screenful of legal BS.

But take note of these two sentences, which I *assume* (but am not sure) comprise the core of the agreement between HuffPost and me:

By posting or submitting content on or to the Service (regardless of the form or medium with respect to such content, whether text, videos, photographs, audio or otherwise), you are giving THP, and its affiliates, agents and third party contractors the right to display or publish such content on the Service and its affiliated publications (either in the form submitted or in the form of a derivative or adapted work), to store such content, and to distribute such content and use such content for promotional and marketing purposes. Without limiting the generality of the foregoing, with respect to any video submissions to THP made by you from time to time, you understand and agree that (unless you and we agree otherwise) THP may, or may permit users to, based solely on functionality provided and enabled by the THP website, compile, re-edit, adapt or modify your video submission, or create derivative works therefrom, either on a stand-alone basis or in combination with other video submissions, and (unless you and we agree otherwise) you shall have no rights with respect thereto and THP or its licensees shall be free to display and publish the same (as so compiled, re-edited, adapted, modified or derived) for any period.

I have a friggin’ doctorate and I have trouble parsing this passage. I *think* I know what it means, but you know what? I’m really not sure that I get it.

Out of curiousity I submitted the passage to an online readability analyzer. The results should surprise no one…here’s how it did:

  • The passage scored an off-the-charts 48.45 on the Gunning-Fogg Index (scores typically range from 0-30).
  • It rated a mind-bending NEGATIVE 36 on the Flesch Reading Ease Score (scores range from 0=hard to 100=easy)
  • The Flesch-Kincaid Grade Level index indicates how many years of education someone typically needs to understand a sample of text. For example, an 8 would indicate that an eighth-grade education would be required to understand the content; a 14 indicates that you’d have to be a college sophomore to grok it. This passage scored a Talmudic FORTY THREE.

I started this rant thinking I was commenting on the difficulty of knowing the ramifications of your actions in this new world of interconnected social media and networking sites. I’m ending on a different rant, but it’s all related to user experience. Here it is in a nutshell:

Hi there social media sites, this is your user talking to you. If you want me to connect up my account to your “strategic partners” and help you “monetize your user base”, DON’T make your user agreements so dense and hard to understand. It only scares me off and makes me worry that you’ll take my data and do whatever you want with it.

In other words…DON’T give your lawyers final edit over your terms of service agreements. They’re hurting your user experience and your brand image. And what’s worse (from your shareholders’ point of view, that is), they’re probably suppressing uptake of these new services because they’ve made it so dang hard to figure out.

Back in June Jared Spool pointed out at the UPA 2009 conference that the user experience field is behind the curve when it come to this new world of interconnected sites and accounts. It ain’t just about usability anymore…and it really hasn’t been for the last five years or so.

If the user experience field is going to remain relevant in this new world, we HAVE to create new guidelines and standards for how sites and services communicate with their users about how and where they use their information, and what rights users have to control how their information is used.

{ 5 comments }

Teaser 2: UPA 2009 UX Industry Salary Survey

by Paul Sherman on August 18, 2009 · 0 comments

I know, I said I’d release the UPA 2009 UX Industry Salary Survey last Friday. Unfortunately, paying work and family time prevented me from finishing the report.

But I’m telling you now that I’ve submitted the “members only” full report and the public report to the UPA office, and they should have it posted within 24-36 hours from now, Tue 18 August.

Oh, and here’s another teaser:

  • The average salary in the UX field for men and women combined is $85,283. For women, the average salary is $84,892. Men’s average salary: $85,947. This is the closest that men’s and women’s salaries have ever been since we started surveying the field in 2000. The gender gap appears to be gone, folks.

You can see the details, including salary by job description and other analyses, when UPA posts the public version of the report. UPA members will be able to get the full report, which includes some really in-depth statistical analyses, behind the member login.

{ 0 comments }

It’s About The Unknown Problems…What Do You Do About Them?

by Paul Sherman on August 4, 2009 · 6 comments

When performing user/design research, us UX researchers go into the field with ideas about the problems you’ve been asked to solve. But it’s nearly always the case that we come out of research sessions having identified a whole host of new and unexpected problems. This is so common, I’ve even added a section to my standard report template to account for the unexpected problems (and possible solutions).

I argue that it’s how you handle the “unexpected” problems that differentiates the good from the great UX researcher.

Discuss…

How do other practitioners handle the “unexpected” class of problems? How do you account for serendipity in your deliverables, communication with clients, etc?

{ 6 comments }

Teaser 1: UPA 2009 UX Industry Salary Survey

by Paul Sherman on August 3, 2009 · 2 comments

in Everything Else

Normally I suck at promotion. Particularly self-promotion, which is obviously a problem for my user experience consultancy. But since this is the last year I will be leading the Usability Professionals’ Association salary survey project (I go off the board after this year), I thought I’d go out with a bit of a pop. I cranked away over the weekend to complete the final draft of the the UPA’s 2009 UX industry salary survey. It’ll be available on Friday. There are some *really* interesting results…but before I drop teaser #1, let me give props where they’re due:

  • Ken Becker did a fantastic job on data cleansing, normalizing and initial analysis.
  • Karl Steiner also did great work producing the data tables and charts and helping me on subsequent analyses.
  • Jeff Sauro did some major statistical heavy lifting and contributed a comprehensive report addendum looking at the relationships between multiple variables.

OK, I lied…just a little more background before the teaser:

  • This is the fourth UX industry salary survey by the UPA. After a sputtering start in 2001, the UPA has conducted biennial (I got that right, right? Biennial = every two years?) surveys in 2005, 2007, and this year.
  • I took on the project as part of my board of director duties in 2005, and have evolved the format to where it is today. I am also the report writer and main point of contact for the project until the end of this year.

I know, I know…you’re thinking “enough with the hat tips and background, just gimme some data already!” Frankly, I’m enjoying this more than I should, but alright, here ya go:

  • The over-time rate of increase in average salary has slowed in 2009, a fact that should surprise absolutely no one. The media is full of stories about companies cutting workers’ salaries by 10%, mandating unpaid vacation, etc. The fact that there was any increase at all in UX salaries was surprising to me.
  • When we started looking at the data in detail, the main source of the increase was *quite* interesting… let’s just say that the “gender gap”; i.e., the difference in men’s and women’s salaries for comparable work, appears to be not long for this world. At least in the user experience industry.

Sorry, that’s all you get for now. More later this week! (And the report will be available on Friday.)

{ 2 comments }