Methods

Malware Perception Risk Assessment Checklist: Before Its Time

by Paul Sherman on March 25, 2010 · 6 comments

in Experience Strategy, Organizational, Privacy, Research

Back about two years ago I was working on a product line that took a number of potentially objectionable actions with customers’ systems. I pushed back against the product teams, saying that these actions put our products at risk of being perceived as malware.

They in turn pushed back on me, essentially telling me to prove my allegations.

So I went away for a few days, did some research, and returned with my (fanfare) Malware Perception Risk Assessment Tool. Ta-da!

Uh, sorry, I meant “thud”. It went over like a lead ballon. No takers. So I wrote it up in an article at UXmatters, hoping it’d become adopted. More deafening silence. Dejection.

But here’s the thing: systems are becoming more and more interconnected, and more than ever, applications are utilizing aspects of your personal, semi-public, and public data to derive value (presumably for you as well as themselves). Thus the risk of an application being perceived as malware has only increased.

I strongly believe that our field needs to provide the wider world with a tool that can help assess the risk that a particular product or service might be tagged as malware in the minds of users or the market at large.

So I again submit to the UX, dev, and product management communities the Malware Risk Assessment Checklist.

To measure the probability of people perceiving a product as malware, I created a checklist representing a set of attributes that typically characterize malware. I grouped these attributes into these five categories, each containing two or more representative attributes:

  • personal information gathering and usage
  • modification of data or system configuration
  • stealth and resistance to removal or modification
  • resource utilization
  • transparency and disclosure of third-party relationships

This time, I’m explicitly calling out the fact that the checklist is light on data propagation via social networking applications. And I’m asking for help in rounding out that aspect of the checklist. So help a guy out and suggest some social media items. I am releasing this checklist under a “Creative Commons non-commercial share alike-derivative works permitted” license, so you can remix this, add to it, etc. When I receive some good item suggestions, I’ll re-roll the list and publish again.

Here’s the checklist as it stood in 2008. Peeps, have at it.

Personal Information Gathering and Usage
The product or Web site…
Gathers and transmits users’ personal data or information about users’ behavior to the organization providing the product
____Yes
____No
Gathers and transmits users’ personal data or information about users’ behavior to a third party.
____Yes
____No
Uses personal data and data the product developer obtained from third parties to assemble profiles of users that are more complete and comprehensive than users expect.
____Yes
____No
Exposes more of users’ personal information to their contacts or a community than users expected or wanted.
____Yes
____No
Does any of the above without user notification and consent.
____Yes
____No
Does any of the above and does not allow users to opt out.
____Yes
____No

Modification of Data or System Configuration
The product or Web site…
Overwrites, modifies, or destroys users’ data without their knowledge or consent.
____Yes
____No
Modifies other applications on users’ computers or their operating system settings or computing environment.
____Yes
____No
Fails to restore modifications to other applications, operating system settings, or the computing environment when the user uninstalls the product.
____Yes
____No
Damages or renders inoperative other software or hardware on users’ computing systems.
____Yes
____No

Stealth and Resistance to Removal or Modification
The product or Web site…
Hides or renders its files and resources inaccessible to the user through normal means—that is, using standard file managers and viewers.
____Yes
____No
Resists attempts at removal.
____Yes
____No
Modifies antivirus, antispyware, and other computing hygiene applications or application settings, to make itself appear harmless or less harmful than it actually is.
____Yes
____No

Resource Utilization
The product or Web site…
Overuses computing resources—CPU, GPU, memory, and so on—to a noticeable extent.
____Yes
____No
Utilizes computing resources for purposes not directly related to the tasks users typically perform with the software.
____Yes
____No

Transparency and Disclosure of Third-Party Relationships
The product or Web site…
Installs third-party applications that demonstrate any of the above behaviors.
____Yes
____No
Installs third-party applications without user notification and consent.
____Yes
____No

C’mon people, let’s make this checklist useful, and maybe even a de facto standard.

{ 6 comments }

Testing Your Own Designs: Discussion

by Paul Sherman on September 10, 2009 · 12 comments

in Design

I’m putting an article together for UXmatters on the topic of usability testing and validating one’s own designs. My goal is to develop some guidelines for self-testing.

I’d love to get your feedback on some questions I have:

  • Testing your own designs: all-around bad idea? Or is it possible to do it well?
  • If so, what should you do / not do when testing designs you’ve created?
  • What should you look out for?
  • Got a good story or anecdote to share? Please do. Either success stories or cautionary tales are welcomed.

Thanks in advance. -Paul

{ 12 comments }

Low-Rent Usability Tech Tip: Remote Usability Testing With Observers Watching

by Paul Sherman on August 29, 2009 · 10 comments

in Everything Else

Today’s post is a simple little usability testing “tech tip.” It’ll help you run a remote usability test with a participant running an app or browser on your test machine while other remote observers are watching the session.

My investigation into this started when I was asked to conduct remote usability test sessions for my client. (I am an independent user experience consultant who does both interaction design and usability testing.) The client asked if they would be able to listen and watch the test sessions in real time.

Now I know that there’s probably an expensive software package or two out there that would give me the capabilities I needed, but I wasn’t in the mood to spend $800 – $1,2000 USD on something I may not use for another few months. So I poked around the Internetz and discussion groups (including the usability listerv that dares not speak its name…) looking for guidance on how to run usability test sessions with a combo of tools that met my requirements.

In my search I found a suggestion for employing GoToMyPC for participants to access the test system. The reasons for using it were pretty compelling: GoToMyPC offers extremely low latency and high performance. And most importantly for me as the test moderator, it lets me and the participant trade off control of the test system in a modeless manner. That is, when I want to quickly trade control of the test system I can do it on GTMPC without having to go to a menu and select “give (or take) control.”

That still left me with the problem of how to allow observers to watch. Then it occurred to me that any old online meeting service would do; all I’d have to do is run it on the test system, invite the observers to the meeting, and distribute a phone conference bridge to all parties.

So I tried it this week, and it worked like a charm: the test participant had low latency and high performance as they used the application, I had the ability to assume control from the participant as needed without wasting time and unduly interrupting the flow of the test; and observers could watch the test session unfold in real time. (They could also pass me questions via the meeting chat capability or via out-of-band IM.)

So, here’s the deets: for remote usability testing with remote observers, here’s what you need:

  • A GoToMyPC account
  • An online meeting account (most if not any will do)
  • A phone conference bridge

And here’s how you set everything up. Please note that this requires that you be sitting at the test system; aka the target PC for GTMPC:

  • Designate your test system as the target PC for the GTMPC service. This is the PC you want to remotely control.
  • Temporarily change your GTMPC login, password and target PC access code for use with the test participants. You’ll be sending them these credentials, so make sure you’re not using your “standard” personal usernames and passwords.
  • When you’re ready to run a test session, convene an online meeting from the test system with your observers, and allow the observers to view the test system’s desktop.
  • Send your test participant a link to gotomypc.com, with instructions on how to log in and enter the test system’s access code. (It’s pretty easy, there’s not much to it.)
  • The test participant will then be signed in to the GTMPC service and can control the test system…as can you, so be careful not to “wrestle” for the mouse too much.
  • Run the test session.

I’m sure there’s more tricks you could be add on to this basic setup. Here’s one idea: you could record the session using the online meeting service’s recording capabilities.

And it also seems possible to get the remote test participant’s face into the picture somehow, as many online meeting services provide web cam integration. Note that this would require inviting the test participant to the online meeting as well as having them sign into the test system via GTMPC, so be careful before testing this capability. Not to be an alarmist or anything, but on the face of it I’m guessing that this could easily tear a hole in the space-time continuum and open a portal to parallel universes. Or something like that. Just sayin’.

{ 10 comments }

ProductCamp Austin Summer 2009 – User Experience Presos

by Paul Sherman on August 26, 2009 · 1 comment

in Design

For some reason this slipped my mind for the last two weeks. On August 15th I delivered two talks at ProductCamp Austin 2009. Before I link you to the talks I wanted to give hat tips to the crew who put together this ProductCamp. It was a fantastic, energetic, and crowd-driven “un”conference, and I highly recommend attending one if you get the chance. They’re springing up in many major metro areas, so finding one shouldn’t be hard. You can learn more about BarCamps at this site.

The first talk I gave was “How To Achieve A Great User Experience For Enterprise Software” and the second was “From Personas To Production: The Role of Personas, Design Briefs, Stories, Storyboards, and Wireframes in the Ideation – Design – Build Process.” The second one had more people than the enterprise talk; which I guess shouldn’t really surprise me as the enterprise talk is more specialized. But the enterprise attendees were full of good questions and there was lots of good within-audience discussion. The feedback I’ve gotten on these two has ranged from slightly to strongly positive. So I’ll put them in the “win” column.

Oh and my past presentations are also available on Slideshare.net here.

{ 1 comment }

It’s About The Unknown Problems…What Do You Do About Them?

by Paul Sherman on August 4, 2009 · 6 comments

When performing user/design research, us UX researchers go into the field with ideas about the problems you’ve been asked to solve. But it’s nearly always the case that we come out of research sessions having identified a whole host of new and unexpected problems. This is so common, I’ve even added a section to my standard report template to account for the unexpected problems (and possible solutions).

I argue that it’s how you handle the “unexpected” problems that differentiates the good from the great UX researcher.

Discuss…

How do other practitioners handle the “unexpected” class of problems? How do you account for serendipity in your deliverables, communication with clients, etc?

{ 6 comments }

My UPA09 “UX Kit” Presentation & Bonus Audience Comments

by Paul Sherman on June 15, 2009 · 9 comments

in Design, Experience Strategy

Today I’m posting the presentation and source document from my UPA2009 presentation “A Kit For Building User Experience Teams in R&D Organizations.” The talk went very well; nearly everyone in the (somewhat small but whatever) audience spoke up and contributed.

Happily, when I posted links to this content on Twitter I got about a half-dozen retweets, which for a second-stringer like me is not too shabby. So I think you’ll like this preso and the kit doc, which I’ve released under a Creative Commons Attribution-Share Alike 3.0 United States License. BTW you can learn more about this license and what it means at http://creativecommons.org/licenses/by-sa/3.0/us/. Basically, it means you are welcome to make use of the content as long as you attribute it to me and you share any derivative works under the same license. Which I think is more than fair, and leads to boatloads of good UX karma besides.

And here’s a little bonus: I asked my friend and session chair Lyle Kantrovich (@lkantrov for the Twitterati in the crowd) to take notes on the audience comments and contributions; which he peevishly (kidding! I meant happily) did. I’ve posted his notes below.

Before I link you out to the content you might be interested in the “story behind the story” of this presentation. About 7 or 8 months ago I decided to submit to UPA2009, and scoured my hard drive for something appropriate. I realized that I had created a comprehensive resource while at Sage that detailed how to staff, budget and run a user experience team at a medium-to-large software organization. I figured that this was as good a submission as any. Plus, it really fit my whole “get the organizational structure and processes right” theme. If you’ve been reading me for any length of time you know I have a passion for this area of our field, having trained in social/organizational psychology and built several teams over the past 12 years.

So I submitted a proposal, which went something like this:

This submission provides an overview of a “User Experience Kit” that one user-centered design team developed as an implementation guide for other product teams within their global organization. This kit was first released in mid-2007 within the organization, and has been used in the organization to guide the creation of four additional teams since then. The primary audience for this presentation is people who are able to drive change in their organizations and have the authority to support those changes with allocation of resources.

And it got accepted. Yay.

Of course I put off writing the presentation for months, but not for the usual reason (i.e., pure procrastination). As the day of the talk drew nearer, it became clear to me that the kit itself was a really boring story. And I don’t do boring. I HATE boring. I have high standards for presenting, I do it well, and I was stressing out about how boring this talk was shaping up to be.

That is, until I realized that the more interesting story was *why* I had to create a UX implementation guide/kit, what it said about my then-organization (and other organizations), and what we as a field should be doing about it.

And then everything was alright, I wrote some entertaining slides (keep on the lookout for “Captain Obvious”) and I gave a kick-@ss talk.

So, without further ado, here’s what I covered in my talk:

  • The sad truth about the need for a “UX kit”
  • A bit about the kit itself
  • An extended discussion about launching UX teams and spreading UX in medium to large orgs

As I mentioned above, Lyle was kind enough to capture discussion notes, which I’m including immediately below. However, I recommend looking at the preso first (either in .pdf format or on SlideShare) and getting the kit source doc before reading the discussion notes.

Thanks again Lyle for capturing the audience comments. Here they are:

  • Come back with data to show the value of what happened during UX processes.
  • Be more of a teacher – share UX? techniques (aka “UX freeze-tag”).
  • Be flexible.
  • Triage projects early on – to discuss how UX can help.
  • Focus on convincing people who can be convinced.
  • Have an open-door policy on usability lab.
  • Create an internal blog with test highlight clips.
  • Conduct a quarterly UI workshop.
  • Stay relevant – you know if you’re relevant by # of people coming to you.
  • Focus on money/budget & key influencers in the organization.
  • UX has to manage a lot of different things at the same time.
  • “Customer Experience Bar Raiser Review Board”? – executives that help set UX direction.
  • Selfishly share the glory – co-present success stories with clients/partners.
  • Find a mentor/peer outside your organization to learn from, commiserate with and share with.
  • Find an aspirational (design/product) example – something that reflects what you’d like the UX to look like.

A Kit For Building UX Teams [preso pdf]? [kit doc]? ::? Paul Sherman

{ 9 comments }

UXmatters: The Perpetual Super-Novice

by Paul Sherman on December 4, 2007 · 2 comments

in Design

Just thought I’d point to my latest UXmatters article. My idea for this article is that people get stuck at a certain point of understanding a system, and fail to progress beyond a few areas of a rich application.

? After initially becoming somewhat familiar with a system, people often continue using the same inefficient, time-consuming styles of interaction they first learned. For example, they fail to discover shortcuts and accelerators in the applications they use. Other people learn only a small portion of a product’s capabilities and, as a result, don’t realize the full benefits the product offers. Why? What can operating systems, applications, Web sites, and devices do to better facilitate a person’s progression from novice to expert usage?

It’s an idea I’ve been kicking around for a while. Since I owed UXmatters a column, I thought I’d explore it a bit. I’m still working it out.

{ 2 comments }

Interruptions: Nuisance or Valuable Data Source? (A UBlog Rerun…)

by Paul Sherman on November 6, 2007 · 3 comments

(Originally posted April 2005 – P.S.)

There is one constant across almost all office-based work environments: the office worker is subject to innumerable interruptions and distractions. Yet when we test software and web sites for usability, we always seek to minimize interruptions and distractions. Should we?

The usability test lab environment is a contrived setting. It is purposefully designed to eliminate (or at least minimize) distractions and interruptions. It resembles nothing so much as a laboratory for psychological experiments.

Once the test session begins, the facilitator makes every effort to ensure that the participant is free from distraction. Other than asking for occasional clarification, the effective facilitator is trained to interact in a neutral, non-judgmental, clinical manner. This allows the participant to concentrate on the tasks at hand.

However, the typical office worker deals with a range of distractions and interruptions throughout the day – some self-imposed, some from external sources.

Given the ubiquitous nature of distractions and interruptions, it might make sense to replicate some of them in the test environment. Doing so in a controlled, deliberate manner would help illuminate how the product might fare under real-world conditions.

As a hypothetical example, consider an intranet site within a large corporation that allows employees to enroll in and make changes to their benefits. Imagine also that the benefits enrollment process was user tested in the usability lab with a variety of user types ranging from administrative assistants to software engineers. For our purposes, let’s assume that it earned high marks with users during this round of testing.

However, when it’s rolled out the organization finds that a significant number of employees are committing errors when enrolling or making benefits changes. Further investigation reveals that errors are most prevalent among tech support employees and mid-level managers. Why wasn’t this revealed during user testing?

One reason could be because tech support reps’ and managers’ work time is characterized by frequent interruptions and a multitude of distractions competing for their attention. While the benefits enrollment process might “test well” in the serene confines of the lab, in this case efficient use of the web site’s interface is hindered by frequent phone calls, emails, and the vagaries of working in a team environment.

It turns out to be difficult for people to ascertain the status of the application and of the operation being performed when a constant stream of distractions and interruptions characterizes their work environment. Had this been discovered before rollout, the design could have been adapted accordingly.

Studying how people use a product under conditions that replicate the distractions and interruptions of ordinary life would reveal additional information about the product that would not necessarily be revealed by traditional laboratory testing.

{ 3 comments }

My Latest UXmatters Column: How Do Users Really Feel About Your Design?

by Paul Sherman on September 25, 2007 · 1 comment

I don’t know if this happens to you other bloggers… a few weeks ago I had a GREAT idea for a post, and didn’t want to write anything else until I had written the great post. As a result, I got completely blocked up and stopped writing for a while. So I haven’t posted a darn thing in weeks.

Anyway, since I have little to say here at my own little node of the Interwebs, I’m going to point you to my latest UXmatters article. It’s about an interview I conducted a few weeks back with Eva de Lera of the Open University of Catalonia. She and her colleagues are developing measures to assess users’ emotional reactions to user interfaces. Article link is below.

How Do Users Really Feel About Your Design? :: UXmatters

Blogged with Flock

Tags: , ,

{ 1 comment }

Great IA Interview At UXPod

by Paul Sherman on April 4, 2007 · 0 comments

in Web

Gerry Gaffney of UXPod fame posted a great interview with Karen Loasby of the BBC. The interview is from November; but I’m so scandalously behind in my reading/listening I’m just now getting around to it.

It’s worth a listen; it provides her unique perspective on what it’s like managing the information architecture for a major media web property. You can get the podcast here: UXpod – User Experience Podcast – Interview with Karen Loasby.

Blogged with Flock

{ 0 comments }

Project Managers on Usability

by Paul Sherman on February 4, 2007 · 0 comments

in Everything Else

It’s always nice to see our colleagues in neighboring disciplines talk about usability. Here’s an article from PM Toolbox that, although brief and a bit simplistic, at least shows that our colleagues understand the value of a well-designed user experience.

{ 0 comments }

WTF?!? Amazon Selling USS For $59.95?

by Paul Sherman on January 26, 2007 · 2 comments

in Everything Else

Oh. My. God. Amazon.com actually updated the price of “Usability Success Stories“.

Maybe some peeps will buy it now…

{ 2 comments }

Shameless Self-Promotion: “Usability Success Stories” Now Selling For Less Than US$60

by Paul Sherman on December 22, 2006 · 0 comments

in Everything Else

I’m happy to report that the publisher of “my” book “Usability Success Stories“, has lowered the price to US$60.00. With a 15% discount from purchasing via Gower’s web site, the price falls to US$51.00 plus shipping.

Yeeha.

You can click here to purchase it direct from Ashgate/Gower, or you can download the order form from this link.

(I put airquotes around “my” because I wrote 3 of the 10 chapters, edited the other contributors’ chapters, and produced or reworked the images and illustrations. So technically the book isn’t all mine. Just trying to avoid megalomania…)

{ 0 comments }

First Chapter of “Usability Success Stories”

by Paul Sherman on December 6, 2006 · 0 comments

in Web

I just found out that Gower has posted the first chapter of the book I edited.

The PDF of Chapter 1 can be found here: http://www.gowerpub.com/pdf/Usability_Success_Stories_Intro.pdf

I still don’t think you should buy it at 100+ dollars, though. (If you want to know why, you can read my rant.)

{ 0 comments }

My User Friendly 2006 Presentations

by Paul Sherman on November 24, 2006 · 0 comments

in Everything Else

A few people have asked me about the presentations I gave at the User Friendly 2006 conference in Hangzhou, China earlier in November. If you’re one of them, or are just interested in seeing the PowerPoint decks, you can access them at this URL: http://www.usabilityblog.com/UF2006/.

The file starting with “Talk…” is the slide deck that accompanied my invited speaker talk. I presented about the project I led redesigning Peachtree Accounting’s user interface. Direct link is here.

The files starting with “Panel” are the slides and schedule for the panel I led, “Some Right – and Many Wrong – Ways to Incorporate Usability into an Organization.”

Enjoy.

{ 0 comments }

The Usability Kit: Frak Yes

by Paul Sherman on November 17, 2006 · 6 comments

in Web

I just looked at the sample chapters for The Usability Kit, a comprehensive resource for web site designers that explains critical usability concepts and also provides actual templates for common interactions such as login pages, help and FAQ areas, My Account pages, etc.

I really like it. It’s a great idea, and it appears to be very well executed. I downloaded the sample from SitePoint, the kit distributor, and the sample chapters read great. Dan Szuc and Gerry Gaffney cover all the right topics. I’m really glad they covered the critical information architecture concepts of faceted classification, tagging, and folksonomies.

Let me do the full disclosure thing at this point: I’ve known Dan Szuc for a few years now, and consider him a friend. And I’ve recently met Gerry Gaffney. So I’m not an impartial observer. However, I consider myself a reasonably ethical person, so if I didn’t truly feel that this resource was worth your time, I would say so. (Or more likely, I just wouldn’t blog it.)

By far the best thing about The Usability Kit is the blueprints. In providing what amount to templates for common interactions, Gerry and Dan have gone where many others fear to tread. Let’s face it: there’s only so many *good* ways to design a login box or a “Subscribe To Our Newsletter” form. And if I’m reading SitePoint’s blurb page correctl, you actually get electronic copies of the blueprints, so you can build pages from the templates. (I could be wrong about this; and besides, who builds sites with static HTML anymore? That’s so 1998…)

Unfortunately, the SitePoint sample didn’t include the chapter about user research. I am very particular about how user research is done, because I’ve seen so many ill-conceived, inefficient and biased user research projects performed by well-meaning people. So I can’t speak to how well Gerry and Dan covered this topic.

Bottom line: would I buy yet another *book* about usability for US$197? Frak no. Would I buy this kit, with all its templates and other goodies? Frak yes.

So you decide. Am I logrolling (or shilling), or pointing y’all to a quality resource?

{ 6 comments }

Happy World Usability Day

by Paul Sherman on November 14, 2006 · 0 comments

in Web

Happy World Usability Day.

This event, now in its second year, was intended to raise awareness around the globe of how important usability is to our everyday lives. It’s succeeding beyond our wildest dreams. And that’s great.

But I’d like to introspect for just a moment here. Those of you who work in the field, stop and give yourself a mental pat on the back. And then tell yourself how fortunate you are to have stumbled upon a field that is so engaging, fulfilling, and exciting. Admit it. Every day when you wake up and go to work, you’re secretely thankful that you’re not a lawyer, a doctor, a software developer, and so on. You’re a soldier in the user experience army. And it’s the best damn career you can imagine.

You know it’s true. You tell yourself that at least once a week, don’t you?

I know *I* do…

{ 0 comments }

It’s Finished

by Paul Sherman on September 14, 2006 · 0 comments

in Web

Remember that book I was working on?

I just found out it’s gone to the printer.

It’ll be available in five or six weeks.

{ 0 comments }

UCD In The Press

by Paul Sherman on August 10, 2006 · 0 comments

I was fortunate to be asked to participate in my company’s press and analyst event in May of this year. For the main event, I put together a short presentation about the user-centered design program I built at Sage Software (the marketing folks sexed it up by labeling it “Customer-Connected Design”, which I’m fine with…)

[click to continue…]

{ 0 comments }

Me Too!!

by Paul Sherman on July 26, 2006 · 0 comments

in Web

Like every other wanna-be semi-guru in this field, I’ve written a book. OK, I really just wrote three chapters, got 10 other people to contribute chapters, and edited their contributions…but still, I think this counts as “I have a book coming out.”

It will be available in September or October of this year. It’s published by Gower, an imprint of the Ashgate Publishing Group.
[click to continue…]

{ 0 comments }

← Previous Entries