Thursday, 12 August 2010

Raining on the Parade

OK people, fasten your seatbelts, here comes my first bloggy rant. I realise these is a risk that I may upset or even offend some people I regard as good friends and colleagues. So let me first say, please remember I admire what you do, and in no way can I claim to be able to build tools, write code not related to the XML standard or anything of the kind. Also I like you. But, I also love my work and what I do and so I can't just sit here and say nothing.

There is a lot of buzz about at the moment about Anthologise. Most of it is good. I can see why. It's a great achievement to get a group of people together and build a working tool in a week. But here is my objection, was there anyone there at all that has any knowledge of how to conduct user studies of digital reading? Because I can't see any evidence of this.

The reason why this matters is that those of us who study online reading and general user behaviour can show that is that there is a very great difference between reading online and offline. So it's just not that simple to say, take blog posts, put them together, make a book, hey presto. Because if a blogger is any good s/he should know, or perhaps just intuit by accident, that people read differently online (They take in about 40% less information) Thus if you write for the online medium, you ought to do it differently than for print, and that can be a great opportunity. Thus just putting blogs together and printing them does not a literary or journalistic success make. In fact a printed blog collection seems to me to be a wounded digital object with its wings clipped. It can't make use of all its original hypertextual muscles to take off as it ought to, and so it's grounded and trapped on paper.

I love reading in print, but when I do I want to read a subtle piece of argument or something with all kinds of fascinating literary language and complexity, because print is the best place for them. They don't go over well on a blog: we just don't have the processing power to want to comprehend them online. Conversely, at least to me, blog posts seem too flat and utilitarian when printed.

So if I had been there, would I have poured cold water on the idea? Well yes I might, or at least I might have brought my knowledge to the table to take part in the discussion. But what I'd have actually said was, 'OK I have doubts, but let’s test it on the users and see what we get'.

I am really pleased that Anthologise had a UX team. But I can’t quite see what serious UX work you can actually do in a week. And this is a problem because if you are going to design something that users really want you need to study them first. You need to talk to them, interview them, observe them, feed back the results of this into the design process, try it again, retest, get more feedback etc. Otherwise there is a huge risk that what you get is a tool that people in DH love and get but other people, actual, real users don't get. But please prove me wrong if you can. I'd love to know if you can do mega rapid user testing as it might be a technique that we could use in our group. I mean that seriously.

Now it might be that Anthologise will be a huge success, and I really do wish it well. But I cannot get enthusiastic about this kind of development method for DH tools in general, because we know from our work that this kind of attitude, what we've called 'designer as user', is fraught with problems. Often we techies or DH people understand complex digital functionality, and imagine that if we can do it, anyone can. But it turns out that as a reuslt it's much too complicated for users, or they just don't see the point of it. And if that happens we know they just will not use things. If you happen to get lucky and design a tool that everyone loves and gets, good. But we find that, oddly enough, resources that work are more likely to do so if users are taken into account, and if they are not at least a third of them are likely to fail. Why take that risk? If this is just an experiment and nobody cares if it works, fine. If it's supposed to work, please involve users. In the case of Anthologise I cannot see any mention of this in future plans.

You may say that we have written possible use cases, showing people what they could do. But I have as many doubts about these as the I do with use of personae. They are a more complicated version of designer as user, because they simply replicate the assumptions we make when writing them. It's very hard honestly to imagine how someone utterly different from yourself will use something. Unless of course you've got data from a great deal of user testing. Again, prove me wrong here, tell me how you got that.

DH history is littered with tools that have been written about with a kind of missionary zeal. If you study the DH literature there are far more articles exhorting people to use tools and techniques in the mainstream than there are about people who actually do use them. This is really, really not about lack of knowledge, it's about lack of fit with what people want. We keep saying, 'If we shout loud enough people will come.' They just won't, won't, won't if what we are offering is not right. I know we love cool digital things, and want others to love them, but just going on about how great they are is not working. How long do we have to keep shouting before we realise this? It's been decades now.

Sorry I am in full rant mode now, but it frustrates me so much. This is at the core of my work, we have evidence about it, people even quote our research now. Then they just plain ignore the message of it.

I love digital things too. I want more people to use them. But I am convinced that we can only do this if we find out what users want and design for that. What's so hard about that? It might mean we have to change our plans a bit, and produce things that are simpler, but perhaps more elegant and pleasant to use as a result. It's an exercise in listening, not in imposing our own ideas. Why is that so wrong?

10 comments:

  1. Claire:

    I wasn't on the UX team for the Anthologize project, so I hope one of them will chime in here and offer more details about their process. However, I do know that they did user interviews as part of their work (early on day 3, if I remember correctly, which was just after we settled on our tool but hadn't really fleshed out how the details). Of course it was done in some haste given the timeframe of our project-but it _was_ done and it was considered important enough to allocate three team members' time to doing so.

    ReplyDelete
  2. I'm actually curious and not trying to be snarky or hostile when I ask: have you used Anthologize? Or looked at what it produces? Because I get the strong impression that you haven't, Claire. First of all, you seem to think that Anthologize produces print, which it doesn't. (That would be some trick!) To wit:

    "Thus if you write for the online medium, you ought to do it differently than for print, and that can be a great opportunity. Thus just putting blogs together and printing them does not a literary or journalistic success make. In fact a printed blog collection seems to me to be a wounded digital object with its wings clipped. It can't make use of all its original hypertextual muscles to take off as it ought to, and so it's grounded and trapped on paper.

    I love reading in print . . . "

    Etcetera. What Anthologize does is to take blog posts and turn them into either 1) a PDF, 2) an ebook in epub format (readable on an e.g., iPad or in Adobe Digital Editions) or 3) TEI-encoded XML. Those are all *still digital formats,* just different ones. There's no reason one needs to print them, and indeed, when I got my blog posts into a PDF, I was ridiculously thrilled to see that the links still worked. My own use case involves sending someone (say, a promotion committee) that PDF as an attachment rather than sending a link. Much weightier, somehow.

    You say that your group has done research into reading "in the online medium," but have you done research into reading in different online *media*? PDF vs. RTF vs. Kindle for iPhone vs. Kindle vs. blog vs. TEI? I'd be interested in a link (or of course a citation) for such a study.

    ReplyDelete
  3. I'm going to have to agree with Amanda- the idea is not to print to paper, after all [insert print is dead joke here].

    Kidding ;)

    You mention so much research into online reading mediums, but it doesn't seem like you've looked at all at the technology that powers those mediums. Or in this case, emerging technology that empowers the users of the medium. Otherwise you might be aware of the large number of plugins that do the "translate to printable material" thing already. And of course, how they are different from Anthologize.

    Anthologize focus' on producing epubs etc for the purpose of continued electronic consumption. While you could print its output, and I imagine people will, it's not expressly built for that. In our case it makes distributing course blogs in such a way that the content can still be used even outside of networked areas simple. Likewise, we can coalesce and entire student blog into one PDF/epub for grading purposes.

    I dare say you've missed the point, and done little here but vent your frustration.

    Anthologize is a tool, and like any other tool it has a specific purpose. In the end, if you have no use for Anthologize, don't use it. But what does ranting about something you don't entirely understand or care about help anyone. You wont convince people to stop tweeting about it. The only thing that will convince me not to use it is the release of a better tool. So why the scathing review? The internet is full of naysayers and skeptics, but few people willing to do something new, daring, and open like Anthologize. Do you really want to get your new blog going by siding with the pessimists?

    ReplyDelete
  4. I appreciate the good fortune of this post coming on the same day as Steve Ramsay's which also raises many of the same questions. And given the enormous enthusiasm surrounding Anthologize (particularly on twitter), I understand your trepidation in offering the post and appreciate the questions raised here are valuable and worthy of attention, particularly to folks a little outside the circle of developers and participants involved in the tool.

    I watched the One Week project with curiosity from afar (and, truth be told, have still not installed it on my own blog). As I followed the development, I had many of the same questions w/r/t why one would want to transform a blog into a book anyway. As you note there is a medium specificity to blog posts which make them different from other genres.

    My sense, in lurking on the user and dev mailing list and reading the many blog posts from participants, however, is that this question has not been unanswered. Anthologize, at least as I have understood it (as an observer from afar, with literally no particular interest in its success or failure), is not about turning blogs into books. It's about reducing the unfortuate barriers which hamper the development of digital platforms by allowing easier commerce between that established behemoth known as "the book" and the plurality of forms which continue to emerge online.

    As Amanda French has already noted, it seems pivotal to note that the output of Anthologize is not really "print"; TEI as a key component seems a choice made with the hope of allowing the output from Anthologize to be available for further mashing and re-mashing. Rather than "a wounded digital object with its wings clipped," Anthologize (again, based on my survey from afar) seems to be a tool geared towards helping the inevitable and already underway convergence of print and digital media and, in the process, helping to eliminate the unfortunate stigma which (as Steve Ramsay observes in his post) continues to be attached to "blogs." It seems very easy to imagine a now none-too-distant future where, on an e-reader, a born-print book of essays sits cheek by jowl with an Anthologized collection of blog posts. This doesn't mean, immediately, discarding any notion of the difference between a blog post and an essay in a print volume (for example). But in the process of transitioning from print to digital, this seems a potentially invaluable tool in helping increase the traffic across that border.

    ReplyDelete
  5. Claire makes an essential point when she writes

    The reason why this matters is that those of us who study online reading and general user behaviour can show that is that there is a very great difference between reading online and offline. So it's just not that simple to say, take blog posts, put them together, make a book, hey presto. Because if a blogger is any good s/he should know, or perhaps just intuit by accident, that people read differently online (They take in about 40% less information) Thus if you write for the online medium, you ought to do it differently than for print, and that can be a great opportunity.

    Yes. Yes! And once again YES! It sounds to me like she has the actual data ready-to-hand that supports an intuition I had, that online reading does not (yet?) carry the same status and engagement as reading in different media, media like print (PDF) or ePub (maybe too early to have data on that one? If so, that sounds intuitively good to me).

    The key thing, to me, is that she's bringing up the issue of translation between media. I couldn't agree more that the experience of "reading" is media-specific (I hope it's fair to see that as a key thing at work here?)

    This is where I would like to interject a little bit, to emphasize the role of the Anthologize Project-creation step. I don't think that, at best, we imagine an unmediated blog to PDF/ePub/whatever movement. Though, I do see how it is natural to see it that way. That's where this post is essential. Claire points us back to the experiece of reading in differenct media. A post written originally for the web should not be the same as something in print (PDF) or mobile (ePub). I do think that there are some contexts for which a direct move from blog to book are in order. But I hope we agree that there will be much more interesting things at work when that move is not direct, and instead involves a consideration of the target medium (PDF, ePub, holodeck -- yeah, we're working on it! (not really)). The rhetoric of medium considerations there are essential, wonderful, and terribly interesting!

    The middle step in Anthologize of copying content and making it editable distinctly from the original post serves that job, at least in a basic way. It will take some sophistication to use that step well. Like the web itself, that might or might not happen. It's our job as teachers to address that and help push things in the right direction.

    I think that your post exposed the issues around the rhetoric of different media very well, and opens up a lot of thinking.

    Thanks,
    Patrick

    ReplyDelete
  6. Claire,

    (Forgive me, but I'll have to divide this into two posts since my response contains twice as many characters as Blogger's comment function allows.)

    I agree with Patrick that you bring up some important points; and I can assure you that we discussed many, if not all, of these issues during the development of Anthologize.

    As the lead for the UX team at One Week | One Tool I can emphatically state that yes, there were several experts in digital reading and UCD on the project; specifically myself, Jason Casden from NCSU, and Scott Hanrath from UK. (You can read about some of our experience on the One Week | One Tool "People" page.) My experience in industry was in UCD before it had formally been defined as a concept; my research in academia focuses on born-digital (aka new media or digital) texts and how readers and writers read and write for different media. So your point that "there is a very great difference between reading online and offline," is well taken.

    You write above that:

    [I]t's just not that simple to say, take blog posts, put them together, make a book, hey presto. Because if a blogger is any good s/he should know, or perhaps just intuit by accident, that people read differently online (They take in about 40% less information) Thus if you write for the online medium, you ought to do it differently than for print, and that can be a great opportunity.

    This is absolutely true. It's also true, however, that everyone at One Week | One Tool understood this. In fact, this is a major way in which Anthologize differs from many tools already out there. In our competitive research (yes we had to do it quickly, but it was done) we found that most of the blog-to-print tools only allow you to "take blog posts, put them together" and "hey presto" you have a book. There were uniformly no editors in these programs. You provide your blog URL, check or highlight the posts you want to include, enter copyright information and the program took it from there.

    One of the key premises of Anthologize is the "crafting" or drafting process. You can create sections, move posts over, edit individual posts or combine them together and edit them, insert new text or entire new posts to craft together a more coherent whole. Or, if you would like to simply create a "best of" page for your blog, you can simply pull your best posts over, export them and post them on a single page on your blog. Or, if you're an instructor and would like to anthologize your students' work you can pull from the RSS feeds and craft as little or as much as you would like. It's the flexibility of being able to craft text together for different audiences, purposes and media that makes Anthologize so powerful and different from other tools out there.

    Is Anthologize perfect? No. Do users get everything they want out of tool in it's first, second or even tenth version? No. That's part of the development process. Like development, UCD is an iterative process. One of the first things the UX team did on Tuesday morning was brainstorm and contact people we knew in academia, publishing, libraries and museums. We sent emails; we called people; we texted people. And we interviewed everyone who responded. Then, in the space of 24 hours, we created a features list, wireframes and a draft specification for the development team. Having the development team working in tandem with the UX team was a challenge; but it was a challenge we all rose to. The development team was extremely responsive and wanted to know what the users needed. Every time the UX team took a suggestion or a new requirement to the development team, they responded. Several times it required a reconceptualization of certain parts of the program, and it was done.

    ReplyDelete
  7. (2 of 2)

    [I have to insert an aside here--because to me, this responsiveness between the developers, the UX and the outreach team was unique in my twenty-something years working with technology. Working with a group of humanists who all happen to be talented with different aspects of technology was amazing; it was everything I had always hoped humanities computing could and would be. I truly believe that this tool could not have been created by a team outside of the humanities.]

    Is this the end of the road for the UX work on the project? No. There were several issues and requirements that the UX team identified that just couldn't be accommodated in 6 days. For example, I'm sure most of the developers got tired of hearing me ask, "But what about the media? I have images, I have video, I have sound on my blog…I want it in my PDF/ePUB/RTF." We got images; video, sound and other media are on the list of features needed in future versions. We will also continue to gather user data via surveys, interviews, contextual inquiry and other established methods. Those findings will be written up, mocked-up, wireframed-up and discussed with the development team.

    This was not an example of the "designer as user" mentality as you define it above. Do we think we will have users who will also become designers and developers for the tool? We absolutely hope so. But one of the things the whole team was emphatic about when we got together to determine teams was that we needed three: development, outreach and user experience (oh, and "glue," aka Julie Meloni, to hold all the parts together). Users are at the center of this project, not at the periphery.

    One of the casualties of the short development timeframe was user testing; and that hurt. As folks I've worked with on other projects will tell you, I am a big proponent of user testing (I believe the term testing-nazi has been used several times), and I don't like rolling something out without getting user feedback. We had several interviewees who volunteered to do some testing for us and we planned on doing it on a limited scale, but time was just not on our side. That was a risk we decided to take as a team. It is also why we were clear to label the initial release (and the subsequent update this week) as alpha versions. The user forums that were set up for the launch have been and will continue to be a valuable source of "testing" data for us; all of which, as I stated above, will be collected, analyzed and designed into future releases--which will, I hope, include more thorough testing. I know I plan to have some of the graduate research assistants in my lab, all of whom have either been trained by me or my co-director Liza Potts in UCD techniques, begin some of that data crunching in the fall. I believe several other team members have similar plans.

    Ultimately, as Patrick stated, Anthologize's purpose is not to simply translate blogs into books; it's goal is much more lofty than that. I think I can safely say that we all hope that this tool will be part of a revolution; a revolution that acknowledges that digital work (whether text, media or code) is important; a revolution that sees this work of development, blogging, remixing, etc as important in the humanities (a point Steve Ramsay made elegantly in his post today); a revolution that understands the rhetorical power of blogs and YouTube videos and podcasts, and PHP and TEI and…and…and...

    Hoping to continue an interesting (and important discussion),

    Kathie

    ReplyDelete
  8. Thanks to everyone for their comments. I think the best thing is to answer various points in one posting, so taht's what I have done in my next installment.

    ReplyDelete
  9. Kathie, it would be great to talk further about all this. Probabaly best by email or Skype. Please do get in touch! My email is c.warwick@ucl.ac.uk

    ReplyDelete
  10. Just to throw one thought out there, as to why one might want a print version of a blog, even if one *ought* to write differently for print: we've got a really good grip on archival preservation standards for paper. Digital preservation? Honestly, not so much.

    Even if it's a different type of writing, different type of reading, if you care about your blogging and think of it as a relevant part of your work, you really ought to try to preserve it in multiple formats, and print's a nice stable one.

    ReplyDelete