Friday, 13 August 2010

Reading the paradigm shift?

Morning all! (Or whatever time it is for you people in North America) It's wonderful that people are so interested in this topic and have made time to write such detailed and insightful comments. So rather than write another long comment myself I thought it might be easiest just to handle them in a post, not least because some of the same issues recur in more than one comment. I don't know if you're meant to do that in the blogosphere, so forgive me if I inadvertently transgressed the unwritten law (Thinks, 'Dinsdale....' in the voice of Spiny Norman)

Anyway evidently I missed the point a bit and underestimated the possibilities of Anthologize (and yes, thanks for pointing it out, I was using UK spelling!) So I concentrated on how it fitted into my conception of the world and my interests, prominent among which are reading in physical and digital spaces. Then I went on about it, based on a partial understanding of the topic, to my friends and colleagues. Unfortunately that means that in my interactions with Althologize and its web presence I ended up behaving like a typical humanities user as we have observed them.

In other words I made assumptions based on its name (sounds rather booky) then I looked at the web page, saw things about putting posts together and making an electronic book and then publishing it. All these words said book and print to me. (Remember I am a former literary scholar) Then I said to myself. 'I don't think I need that, it makes no sense to me.' Thus I proceeded not to download it. This is exactly what users do. They get easily confused by names and descriptions and if they don't see the point of something they don't go any further especially if, ironically enough, they would have to download and install anything. In fact I actually went a bit further and did explore your web page to find out more about the project, and most people would have been long gone by then. Amanda, if you'd like a reference for this, pick just about anything we've ever written about LAIRAH from my publications page eg 'If you Build it'... in LLC

We all want to be able to say 'No but wait, you got it wrong, you misinterpreted our work, it's so much more than you thought' And now I am convinced. But you only knew what I thought because I blogged it. Most users don't do that. They just give up. We can't be there to guide them through the resource, or answer their comments and put them right (even if you have a comment function or forum most people don't use it) Thus user testing is vital because it helps us anticipate to some extent how users might react, and to fix possible problems before they happen, and in doing so undermine their trust in the resource.

Amanda makes the very good point that there are different versions of digital reading. We have not yet studied them, but this is built into the research plan for INKE At the moment we are still gathering and analysing data for our early studies, and at this point we have not found that users are switching between many different types of device. They tend to read either at a computer screen on in print. A very few of them use phones and even fewer an e-reading device, so we haven't yet had chance to compare across devices. But this might change given that iPads and Kindles are much later into the UK market. We base our findings on what people are doing because we choose to study what people are already doing in their usual context rather than bringing them into labs and giving them things to do on different platforms. That's a perfectly valid method; it's just not the one I tend to use with humanities scholar or general readers. When we do have results ready for publication I'll be sure to blog about it, since it seems like people are really interested. Needless to say that if anyone reading this would like to take part in an INKE study, please do let me know. We'd love to hear about what you do.

And yes, I guess I do come over as a nay-sayer. As someone once said to me, 'As a user studies person you'll never be popular. You come in and tell enthusiastic people what's wrong with the lovely new thing they are so proud of, and that they have to change it.' But I don't care if I'm loved for this. I do care if as a result digital resources can be improved. It can be so much better than take it or leave it. By taking a few, quite simple things into account you can make it so much more likely that your tool or resource is more likely to be used. (cf our LAIRAH checklist) So in the end I see this as necessary pain for future gain.

Finally and by no means least thank you very much indeed to Kathie for her detailed and very informative comments about the UX team on Anthologize and what you did. I'm so pleased that this kind of serious user research was being done. The other day I was talking to someone whose opinion I respect who insisted that what I was describing in the adoption of user studies in DH was a paradigm shift. I was a bit unconvinced as it sounded too large and daunting a thing to be part of but I am beginning to wonder, and that pleases me more than anyone can imagine.

I wish you'd say more about your user studies on the web page, as it's really important stuff. I also very much hope you will write this up for academic publication. I might have an idea about how to do that.... watch this space. But seriously I think we ought to talk more about this off-blog as it were. Perhaps there could be a way to collaborate on testing as part of INKE since we already have Julie Meloni as link person.

I could go on about users and readers, and believe me I often do. I'd also like to reiterate that what I was saying was in the context of the Anthologize work, but it was meant to highlight the broader issues of the importance of users studies and testing. There is so much fascinating work to be done here. Also I am truly delighted to know that people out there are so interested in these issues. I've always thought, 'Oh well this is just me and my stuff, I'll talk about it occasionally at conferences and write articles, but nobody will really be bothered.' It's so good to know that people out there are bothered, and would like to join in the debates. Maybe there is something to this blogging lark after all...

Thursday, 12 August 2010

Raining on the Parade

OK people, fasten your seatbelts, here comes my first bloggy rant. I realise these is a risk that I may upset or even offend some people I regard as good friends and colleagues. So let me first say, please remember I admire what you do, and in no way can I claim to be able to build tools, write code not related to the XML standard or anything of the kind. Also I like you. But, I also love my work and what I do and so I can't just sit here and say nothing.

There is a lot of buzz about at the moment about Anthologise. Most of it is good. I can see why. It's a great achievement to get a group of people together and build a working tool in a week. But here is my objection, was there anyone there at all that has any knowledge of how to conduct user studies of digital reading? Because I can't see any evidence of this.

The reason why this matters is that those of us who study online reading and general user behaviour can show that is that there is a very great difference between reading online and offline. So it's just not that simple to say, take blog posts, put them together, make a book, hey presto. Because if a blogger is any good s/he should know, or perhaps just intuit by accident, that people read differently online (They take in about 40% less information) Thus if you write for the online medium, you ought to do it differently than for print, and that can be a great opportunity. Thus just putting blogs together and printing them does not a literary or journalistic success make. In fact a printed blog collection seems to me to be a wounded digital object with its wings clipped. It can't make use of all its original hypertextual muscles to take off as it ought to, and so it's grounded and trapped on paper.

I love reading in print, but when I do I want to read a subtle piece of argument or something with all kinds of fascinating literary language and complexity, because print is the best place for them. They don't go over well on a blog: we just don't have the processing power to want to comprehend them online. Conversely, at least to me, blog posts seem too flat and utilitarian when printed.

So if I had been there, would I have poured cold water on the idea? Well yes I might, or at least I might have brought my knowledge to the table to take part in the discussion. But what I'd have actually said was, 'OK I have doubts, but let’s test it on the users and see what we get'.

I am really pleased that Anthologise had a UX team. But I can’t quite see what serious UX work you can actually do in a week. And this is a problem because if you are going to design something that users really want you need to study them first. You need to talk to them, interview them, observe them, feed back the results of this into the design process, try it again, retest, get more feedback etc. Otherwise there is a huge risk that what you get is a tool that people in DH love and get but other people, actual, real users don't get. But please prove me wrong if you can. I'd love to know if you can do mega rapid user testing as it might be a technique that we could use in our group. I mean that seriously.

Now it might be that Anthologise will be a huge success, and I really do wish it well. But I cannot get enthusiastic about this kind of development method for DH tools in general, because we know from our work that this kind of attitude, what we've called 'designer as user', is fraught with problems. Often we techies or DH people understand complex digital functionality, and imagine that if we can do it, anyone can. But it turns out that as a reuslt it's much too complicated for users, or they just don't see the point of it. And if that happens we know they just will not use things. If you happen to get lucky and design a tool that everyone loves and gets, good. But we find that, oddly enough, resources that work are more likely to do so if users are taken into account, and if they are not at least a third of them are likely to fail. Why take that risk? If this is just an experiment and nobody cares if it works, fine. If it's supposed to work, please involve users. In the case of Anthologise I cannot see any mention of this in future plans.

You may say that we have written possible use cases, showing people what they could do. But I have as many doubts about these as the I do with use of personae. They are a more complicated version of designer as user, because they simply replicate the assumptions we make when writing them. It's very hard honestly to imagine how someone utterly different from yourself will use something. Unless of course you've got data from a great deal of user testing. Again, prove me wrong here, tell me how you got that.

DH history is littered with tools that have been written about with a kind of missionary zeal. If you study the DH literature there are far more articles exhorting people to use tools and techniques in the mainstream than there are about people who actually do use them. This is really, really not about lack of knowledge, it's about lack of fit with what people want. We keep saying, 'If we shout loud enough people will come.' They just won't, won't, won't if what we are offering is not right. I know we love cool digital things, and want others to love them, but just going on about how great they are is not working. How long do we have to keep shouting before we realise this? It's been decades now.

Sorry I am in full rant mode now, but it frustrates me so much. This is at the core of my work, we have evidence about it, people even quote our research now. Then they just plain ignore the message of it.

I love digital things too. I want more people to use them. But I am convinced that we can only do this if we find out what users want and design for that. What's so hard about that? It might mean we have to change our plans a bit, and produce things that are simpler, but perhaps more elegant and pleasant to use as a result. It's an exercise in listening, not in imposing our own ideas. Why is that so wrong?