Eventyr

Research Methods in CSAI

Second time round and the Research Methods in CSAI course I lead for incoming DPhil students in Informatics is now over.  As ever the students in this class were a pleasure to work with - I don't think I shall ever miss working with those behavioural-problem teenagers that kicked-off my teaching.

We just managed to squeeze in a class in the CETL (probably my last, as the space is now booked up for teaching right through to January 2009) and had lovely talks from Blay Whitby (on research ethics) and Mike Herd the director of the Innovation Centre.

If I do this again next year I need to remember not to presume knowledge in advance.  I was astounded to discover that I had made the mistake of presuming that everyone knew the distinction between qualitative and quantitative research.  It presumed a lot of underlying philosophical knowledge that CS students might never have been exposed to.  My shock when I realised was tangible, and resulted in some restructuring of the course.

Now all I need to do is face the marking (and possibly spend some time designing clear lesson plans, based on my experiences, to follow for next time).

However beautiful...

As mentioned in my last post I am now entering the major analysis phase of my research, a phase set to last me a full year.  I have over fifty hours of transcribed audio (interviews and meetings) which I now need to sit down and face.

• December is the narrative extraction phase.  In order to focus on narrative elements in the data, these structures first need to be extracted, broken down into component structures and tagged accordingly.  By extracting narrative structures it is possible to have a clear unit of analysis, as is preferred in qualitative analysis, and limiting an otherwise nebulous and unwieldy dataset.  As mentioned narrative in this instance is being recognised as a discursive element (whether told, recounted or hypothesised) around a causally-linked set of events (i.e. with a temporal structure), whether true, fictitious or partly told.  The data gathered supports a consideration of both types of narrative; the contrast is between stories told in interview (typically event narratives following Labovian structure) and those co-constructed in meetings (shared social stories co-constructed typically following a ‘small story’ structure).
• January to March is the open-coding phase.  This is (fingers-crossed) where some clear hypotheses can be developed and a taxonomy of stories collected will be developed.
• March to May and June to August are the two following analysis phases where hypotheses can be explored more deeply in the data, and some nice story network analysis can be conducted. (Informatics likes it when you can produce visualisations...)

The overall approach directly supports multiple-viewpoint analysis of the topics under investigation and allows for cross-comparison of stories told in interviews and meetings (i.e. under different interactional circumstances).  It also lends itself to a focus on the different types of narrative found in different circumstances, which may have an impact on our understanding of developer coordination in different meeting structures. 

It is a beautiful strategy and a big pile of work.  I have taken the decision to upgrade from my trusty N6 to NVIVO 7 which will be better for visualisations.  I look forward to learning this new software as well.

I look forward to being able to see this work take shape, such that I can see past the methodology to the results.  I have taken heed of my fortune cookie quote from HackDay:

    "However beautiful the strategy, you should occasionally look at the results" (Winston Churchill)

Reviewing my Research

So I defended the next stage of my research as challenged.  Kinda.

I clarified a lot of things which I had forgotten to say and learned some important lessons:

• Be explicit
• Focus

I shaped an analysis plan for this coming year which effectively showed I need to:

• Work very hard
• Focus

Unfortunately some issues are still unresolved.  As I am looking for my hypothesis to arise from my data I am currently in the worrying position of being able to produce a thesis suitable for social science and of interest to practitioners, but that in itself isn't necessarily a passable informatics thesis. 

This could be a serious problem, especially if my data proves less amenable than I hope.  Sadly my data doesn't support incremental analysis - it is all or nothing, so I cannot fail fast.

I know I can do this, but I am mildly amused that I have effectively just argued that I be allowed to conduct waterfall research.  Single loop: Gather, Analyse, Deliver.

I have also promised to focus.  This means cutting back on all my other commitments and getting on with my research for a change.

We shall see next Easter how well this has all succeeded.  Until then, wish me luck.

Finding Feedback

Having found myself with the task of setting up a last minute conference feedback form for XPDay I went to do some research on varieties of feedback forms.  Or at least I tried.

It never occurred to me how little research has been done into conference feedback as opposed to teaching feedback.  (Or else I was just looking in all the wrong places!)

On the basis of not being able to find proved approaches that were still interesting, I developed my own.  I'm curious to see whether anyone will use it for more than just comments.

Survival Skills for the Graduate Student

I'll love any talk that opens with quoted music (Making it up as I go along) and some really astounding figures, this held very true for this session.

Shannon Duvall (Elon University) and Michele Pagnotta (Licensed Clinical Social Worker) ran a really interesting session on 'Survival Skills for the Graduate Student' aiming to give hints and tips to help improve mental well-being during graduate school.

The statistics are impressive.  From the 2004 Berkeley Graduate Student Mental Health Survey:

• 50% of graduate* students suffer from emotional or stress-related issues
• 10% seriously considered suicide
• 1 in 20 considered it

I plan to delve into those figures later (might be interesting for stimulating discussion with the first year PhD class I lead), but that is an amazing and worrying finding.

They started by talking through Michele's Top Tips:

• Wise Mind (make decisions you are really happy with, by keeping a balance in your mind between emotion and reason)
• Acting Effectively (do things that will work, most effective.  Don't use SHOULD.)
• Don't Worry, be happy (worrying is not problem solving)

And then Shannon's Top Tips:

• Focus on the Facts (don't worry about Imposter Syndrome, not provable, not a fact)
• Advisor Relations (ever-evolving, professional, vital to your happiness and success)
• Be Professional (it's ok to have worries and doubts, but don't forget to keep your academic relationships professional)

Further advice:

• Set Your Goals and document your activities and meetings
• How to Ask for things (Describe, Express, Assert, Reinforce)
• Communication with advisors (ask directly for positive feedback)
• (Everyone was told to write down) MAKE NO ASSUMPTIONS (ask, ask, ask)
• Act Confident (you don't need to be confident, you just need to 'act' confident)
• Getting Motivated (remember that progress comes in waves, intentionally take breaks)
• Get Going Again (clarify goals, prioritise and break down)
• Review your Work Schedule
• Gaining Motivation (develop small, medium and large rewards, guild-free breaks and cheerleading statements)

They gave an interesting introduction to the five areas involved in keeping your vital life balance (in order of importance):

• Personal Growth / Care
• Relational Growth
• Community Growth
• Spiritual Growth
• Professional Growth

You don't have time not to take these things not into account - they are critical to your mental health.  Keep yourself physically healthy and relax to reduce stress.  Make sure to attend to current relationships, it is a priority goal.

All in all, sensible and useful advice.  It is really worth visiting the Mental Makeover website, where they have a downloadable handout related to the session.

A lot of people rushed to queue to ask questions when they opened the floor, with more eagerness than I had seen in any other session so far.  What they spoke about had related in some way to everyone in the audience.

----

* The term graduate is equivalent to post-graduate in the UK - an interesting difference and a grammatical mystery worth pondering at another time.

----

Johanna Hunt
Official GHC 2007 Blogger
You may comment on this blog by visiting the GHC Forum

PPIG 2007 - When PPIG met Wagner

I was luckily funded (after winning the draw at WIP-PPIG 07) to attend the Psychology of Programming Workshop 2007 in Joensuu in the Lake District area of Finland on 2-6 July 2007.  The programme was the most ambitious yet; featuring keynote talks through technical papers, a doctorial consortium, tutorials, and a wide range of evening activities.  I have written a full review for the PPIG Newsletter but here are some of my highlights:

The workshop opened quietly on the Monday with the Doctoral Consortium, allowing us early-days researchers the chance to chat about our research and build up our confidence for the main conference.  Discussions were focussed on research methods and approaches and detailed and interesting questions were asked after each presentation.  Most importantly we were reminded to never start with a research method, but instead a theory or question that needs to be evaluated: What’s my question?  What evidence will satisfy?  How will I collect the evidence?  And the interesting questions: What do people say they do?  What do they actually do?  We also received good advice on how to read research papers, some ideas I intend to share with the Research Methods class.

There were two great tutorial sessions I attended:

• An informal tutorial on eye-tracking and potential application when researching programming.  We were taught about visual attention and Fovea cones, the history of eye-tracking systems from 1898 through to modern eye-tracking systems, and potentials and challenges for such systems in the context of studies of programming.  The main focus of the tutorial was a hands-on experience using an eye-tracking system to conduct research. 
• A tutorial introducing phenomenography (about which I knew nothing) and grounded theory.  We were given real data we could use to practice the two types of analysis, and I was extremely comforted to find that  I seemed to have a high level of inter-rater agreement with the original researchers using both approaches.  This gave me significantly more confidence that I have it in me to complete my own research. 

Tuesday-Thursday were dedicated to paper presentations:

• On the Wednesday I enjoyed hearing about follower and gatherer roles in role based programming, particularly entertained by the speculative language presented which was called ROTFL (Role-Oriented Titillating but Fictional Language). 
• The ‘Experiential Report on the Limitations of Experimentation as a Means of Empirical Investigation’ advised us not to rush empirical studies with software organisations, and to remember that the controlled experiment route is not always optimal.  We were encouraged to consider how such data can still be rigorous and potentially useful. 
• On the Thursday I was intrigued by two of the closing presentations. Sue Jones' research showing that more experienced students have higher mental rotation ability - she wondered whether mental rotation could be improved. 
• Stuart Wray presented an analysis of SQ and EQ (Systemizing and Emotional Quotients) correlated against programmer aptitude.

On the Wednesday was the interesting discussion on ‘Children's mental/operational models of programming—Do children's programming tools miss something?’  We discussed the activities performed by children, mostly on the internet, which involve programming activity without being explicitly recognised as such; parameter tweaking, optimisation, variation and composition of components (e.g. skins), and the creation of simulations, animations and games.  A variety of questions were approached:  Is it programming if it is fun, simple, socially-led?  Are there generational differences in the mental models being developed?  Do these ‘play’ experiences generalise or lead to correct abstractions for developing programming skills?  Where does algorithmic theory fit in these experiences?  If there is a difference what does it look like and what are the implications?  Should more formal programming be taught earlier in the curriculum?

The trip to the Orthodox Monastery of New Valamo provided us with an interesting tour of the attractive monastery grounds and learned of the history of the monks who live there.  The Byzantine conference dinner was a lovely affair and I admit to being very taken by the Valamo-made berry wines, which were delicious.  I just wish I could buy it in Brighton...

As ever the best bit of the workshop was the number of humorous awards awarded at the end.  These included:

• ‘Most Desperate Attempt to Win a Prize with a Ridiculous Prize Suggestion’
• ‘The ‘p-word Prize’ (For yet another paradigm)’
• ‘Prize for the Largest Number of (Childhood) Stories’
• ‘Philosophy of Methodology Prize’ (for articulating when grounded theory is grounded theory)
• ‘Prize for ‘Running out of Time’ the Greatest Number of Times’
• ‘Prize for Conducting a Discussion with Himself in Public’
• ‘Prize for the Clearest Bottom Line’
• ‘Best Look-Alike Prize’
• ‘Most Stoical Scapegoat Prize’
• ‘Prize for Well-Formed Questions’
• ‘Most Invisible Session Chair Prize’

All in all PPIG 2007 was a wonderfully organised and varied workshop.  There was no shortage of interesting things planned, which helped to create an amazing environment.  I just wish my luggage hadn't been delayed in both directions...

My photos of this enjoyable trip to Finland are available here.

Annual Review: My Challenge

Good: Dissemination
Bad: Clarity

That’s the long and short of it.  I’m doing well, we think, but my lack of clarity about my research aim at this stage is still very worrying.  For us all.

Teaching research methods means I am doubly aware of what a horrific state my research is in.  How can I have a full data set and no hypothesis?  What about hunches? What would be the shape of my thesis?

[Intro]
[Literature]
Magical gap which somehow forms:
  [Methodology]
  [Exp 1]
  [Exp 2]
[Conclusions]

Is this the time to re-read all of phd comics to find where I am in the process?

Review which papers have shaped my thinking?

Write an abstract for my research?

Sketch out thesis shape?

Clearly I need to plan for a conference paper which will capture the story of analysis.  I need to break down the next year of analysis; without a testable hypothesis I will need to consider my approach carefully. Analyse a part of the data, or analyse the full data set on a partial axis or theme?  What would such themes be?  Do they entail having already done a partial analysis?  I need to break this down.  If someone wanted to replicate my approach what instructions might I leave?

Ugh.

The following faux-abstract is the sticking point:

“Are agile processes reflected in their social aspect?  A study of developers’ stories in a company applying agile software development practices.  Through their narratives we consider:

• Do agile practices shape stories?
• Do these stories show communication of reflection, willingness to change, other core values?
• How do we interpret programmer narratives?
• Etc.”

The Story of Research

At the end of May I presented at a one-day symposium on reading and writing research at the Sussex Institute at the University of Sussex called ‘The Story of Research’. 

The first keynote ‘Representing Lived Experience: Making Principled Decisions’ by Professor Andrew Sparkes, was interesting and relieving in equal measure. 

The questions he posed which I particularly noted:

• How are you going to write it?
• How are you going to represent it?
• Be explicit about what you are doing and why!
  • Scientific / Realist / Confessional / Autoethnography / Poetic / Ethnodrama / Ethnographic fiction / Creative fiction/mixed genres
• How do you judge them?
• Narrative Poetry – Learning through the power of language
• Who do you want legitimacy from?
• (RAE as farce)
• Think about who we are when we are writing.

The talks were fun, although I regretted attending the second keynote as it resulted in an unfortunate coffee-spilling incident.   (Yes, I knocked it everywhere and over others.) 

Seven Points

The seven secrets of highly successful PhD students

1. Care and maintenance of your supervisors
2. Write and show as you go: This is show and tell, not hide and seek
3. Be realistic: It's not a Nobel Prize
4. Say no to distractions: Even the fun ones and the ones you think you must do
5. It's a job: That means working nine to five but you get holidays
6. Get help: You are not an owner-operator single person business
7. You can do it: A PhD is about intelligence and persistence

Interviews

Have almost completed this set of research interviews.  Feels like hours and hours of work - and I have let myself get behind with other things.  I've even managed to break my small microphone in the process.

Some rough spots with the questions, but I know what to do for the future.

Getting there though - and good to have data I can include in the experience report.

Is the paper important?

Teaching was not the only criterion of assessment.  Research was another and, from the point of view of getting promotion, more important.  Teaching being increasingly dreadful, research was both an escape ladder away from the coal face and a means of securing a raise. The mandarins in charge of education decreed that research was to be assessed, and that meant counting things. Quite what things and how wasn't too clear, but the general answer was that the more you wrote, the better you were. So lecturers began scribbling with the frenetic intensity of battery hens on overtime, producing paper after paper, challenging increasingly harassed librarians to find the space for them.  New journals and conferences blossomed and conference hopping became a means to self-promotion. Little matter if your effort was read only by you and your mates. It was there and it counted. 

Today this ideology is totally dominant all over the world, including North America.  You can routinely find lecturers with more than a hundred published papers and you marvel at these paradigms of human creativity.  These are people, you think, who are fit to challenge Mozart who wrote a hundred pieces or more of music.  And then you get puzzled that, in this modern world, there should be so many Mozarts - almost one for every department.

The more prosaic truth emerges when you scan the titles of these epics. First, the author rarely appears alone, sharing space with two or three others.  Often the collaborators are Ph.D. students who are routinely doing most of the spade work on some low grant in the hope of climbing the greasy pole. Dividing the number of titles by the author's actual contribution probably reduces those hundred papers to twenty-five. Then looking at the titles themselves, you'll see that many of the titles bear a striking resemblance to each other.  "Adaptive Mesh Analysis" reads one and "An Adaptive Algorithm for Mesh Analysis" reads another. Dividing the total remaining by the average number of repetitions halves the list again. Mozart disappears before your very eyes. 

But the last criterion is often the hardest.  Is the paper important?  Is it something people will look back on and say 'That was a landmark'.  Applying this last test requires historical hindsight - not an easy thing.  But when it is applied, very often the list of one hundred papers disappears altogether. Placed under the heat of forensic investigation the list finally evaporates and what you are left with is the empty set.   

From: Why I am Not a Professor  OR  The Decline and Fall of the British University

Research Methods

This graduate course aims to help students understand science and the scientific method, and looks at research in academic and industrial contexts. It also aims to provide practical skills in finding and using sources of research information, research management, and basic data analysis techniques.

I've decided that I really like teaching DPhil Students.  I suspect taking over the Research Methods Course this term falls under the category of a good plan.

Because Monkeys are Everything

I love Survey Monkey.

Not much more to say, but that it does what it says on the tin and really does make conducting surveys suprisingly simple and convenient.

Partners in Research

It starts with MEN...

I was introduced to American Women: Partners in Research today. It was a short film made in the 1960's to describe the market research undertaken by Corning Glass Works prior to marketing a coffeemaker. It aims to suggest design through using real women to do research ("market" research) might be a good thing...

It is hilarious. And makes me relieved I'm doing my research now.

My how things date.

Mock Grant Proposal

As part of the Research Methods in CSAI course I prepared a dubious and sketchy Mock Grant Proposal, EPSRC Grant Form and Workplan about which I gave a 30 minute presentation (including referencing applying with a Blue Form under FEC, Sussex Ethical Guidelines / Standards and Guidelines on Research Ethics and Research Salary Scales) on Monday 07 November 2005.  I am making this available for anyone to view (and be entertained by). 

Hopefully the next assignment, which I need to complete by Friday, will emerge more to my liking.

I have to get some other notes typed up so you should see some backtracking on these pages today.

Keywords

On the advice of the Research Methods course tutor I have set up a list of keywords (see right).

Hopefully these should help me when I am ploughing through the Scopus, Inspec and Zetoc abstract database searches...

Narrative Course

Ok, I have established that I need to be doing the Postgraduate Certificate in Narrative Research at the Centre for Narrative Research at UEL to support my research.

I just need to find someone to pay for it...

Skills Audit

OK, I am now completely satisfied that I already completely fulfil the Research Councils Postgraduate Skills Requirements (I even made a little check-table (passworded), which, even though it is only partially completed, confirms my suspicions). 

Here are as many of the reasons why that I can recall (I am sure there are more)...

1999-2001: Educational ICT Tutoring

2002: Advanced Technical Communication
Covering:
Giving oral presentations, writing short abstracts
Brand, ethics, peer review
Tools of the trade
Preparing web pages
Writing technical and scientific documents
Introduction to LATEX and LATEX2HTML
Academic CV preparation and job applications
Assessment:
Video recorded and peer-reviewed oral presentation on a technical topic
Web page (or pages) on a technical topic
Academic review of a recent technical book of your own choosing
Peer reviews

Also 2002: Poster Presentation Competition

Also 2002: Project Management work

2003-2004: Generic Training for Research Students
Being A UH Research Student (3-5 September 2003)
Learning Resources Training
Undergraduate and Postgraduate Study
UH Regulations and Registrations
The Process of Research Degree Study (Monitoring, Progression, Examination)
How To Survive a Research Degree (Responsibilities, Expectations, Self-Organisation)
Supervisor Relationships
Communicating about Research - Written and Oral
Ethical Issues in Research
Project Management
Other Courses:
Endnote
N6
Statistical Analysis
Qualitative Research
Philosophy of Research
What is Science?
Research: What is Truth?
Environmental Policy
Research Skills Training and Personal Development
Good Research Practice
Equal Opportunities Issues in Research
Health and Safety Issues in Research

2003-2005: Research Assistant in Algorithms and Adaptive Systems
Roles included:
EPSRC Grant Applications (including FEC)
Website Development
Conference Organisation
Proceeding Editing
Journal Editing
Providing one-on-one training in Endnote, N6, LaTeX, Adobe Acrobat, Library resourcing and MSOffice
Personal-directed research
Inter-rater Reliability Coding

2005: Research Methods in CSAI
Course Elements:
Is computer science a science?
Introduction to research methods
Literature reviewing: critically reading research papers
Grant proposals - structure, content and guidelines
Rationalist versus empirical methodologies
Acquiring, coding and analysing qualitative data
Introduction to quantitative data handling analysis and statistics
Research ethics
Academic research vs industrial research and development
Time management
Assessment:
Peer Review
Mock Grant Application
Research Methodology Report

Sussex Regulations and Ordinances

Further to my reading about the research process I decided to track down a copy of the ordinances and regulations applicable to my study at Sussex. The most recent copy I could find was available for download here.  I extracted the 17 relevant pages (out of 209) to a separate pdf document.

The extracted MPhil and DPhil regulations from 2004-2005 can be found here.

Points worth noting are:
* 6. I am subject to the University's Code of Practice on Intellectual Property. I should really look this up before I accidentally do anything wrong.
* 12. It is a good thing I have just formally submitted for withdrawal from study at Hertfordshire as simultaneous registration is not allowed.
* 21. My minimum period of registration is two years and my maximum is four.
* 25. Hmm, maybe I should have considered just doing this as a collaborative distant student to save on fees... Nah, I would have still had the same issues.
* 30. If I suffer illness lasting for more than six days I need to submit a medical certificate to the Director of Graduate Studies.
* 39. and 40. I shall be required to prepare an outline of the research project for submitting within the first year of study.
* 41. My progress will be reviewed annually.
* 45. Three copies of the final thesis will be submitted within the maximum registration period (four years), after two months notice. The maximum word length is 80,000 words. The maximum word length of the abstract is 300 words.
* 49. I can incorporate any published work into the thesis.
* 55. Always watch for accidental plagiarism.
* 58. 'for the award of the Doctor of Philosophy, that the thesis makes a substantial original contribution to knowledge or understanding.'

How to Be My Student (SoP)

How to Be My Student

The paper summarises several important points that any student should remember about the supervisor/student relationship.

Particularly relevant for me to remember are the following:

* Ensure that meetings are regular and scheduled well in advance
* NEVER cancel if behind schedule
* If possibly provide reading material to supervisors a day in advance
* Email summary of main points covered after the meeting
* Make sure to look at courses on transferable skills and research methodologies
* Do not try to hide any problems from supervisors

How To Write An Informatics Paper (SoP)

How To Write An Informatics Paper

Hopefully I should be fairly well placed for writing papers, having edited so many over the last few years. We shall see.

This paper summarises good and bad form in Informatics Papers and most of the points felt like issues I had covered while learning to write philosophy papers, so I do not feel a need to summarise them here.

The structure given for a standard informatics paper is:
Title
Abstract
Introduction
Literature Survey*
Background*
Theory*
Specification*
Implementation*
Evaluation
Related Work*
Further Work*
Conclusion
Appendices*

The Researchers Bible (SoP)

The Researchers Bible

As a summary document it seems like it would be very useful to any researcher starting out, although it is understandably strongly targeted towards AI researchers. I found there were many points that I either found familiar or had worked through already (especially 3.1 and 4.1), however, so the following are the major points that I thought were worth noting in my case:
* After looking it up, at Sussex the definition for award of a DPhil is that, 'for the award of the Doctor of Philosophy, that the thesis makes a substantial original contribution to knowledge or understanding.'
* 3.10 (Methodology Does Not Make A Thesis) / 3.11 (The Discovery Route is Not Justification): These are both points that I need to consider and address this term and I feel that my essay for the Research Methods Course should help me with this.
* 4.4 (Early Morning - Cold Start): I should make sure I set a formal work schedule to give fair time to both my research and my work, preferably making sure that the first task of the day is simple and not something I will find other tasks to avoid.
* 7 (Writing Papers): This section suggested I should make lots of notes and make writing a part of my life. Hopefully I will achieve this with this site...
* 9 (Guide to Reading): I need to re-evaluate my reading and try to keep up-to-date with it. It may be worthwhile for me to make a list of relevant Conferences and Journals here at some point as well as re-subscribing to an abstracts database.

Research Methods Course

I have now signed up to attend the Research Methods Course and attended the first seminar.

The main thing I took away from the session was my new mantra:
"Remember, photocopying is NOT reading."

I have also established that both assignments should prove very useful to me this term.

Research Methods Courses

The University of Sussex graduate research training pages provide some useful links to research courses.

The Research Methods course website is here.

The Hertfordshire Generic Training Research Seminar series handbook is available here as a pdf. It includes the Joint Postgraduate Councils skills requirements and a break down of how this applies to the courses on offer. As staff I can attend any of these sessions.

How to do Research

Advice to the Young Astronomer a humorous guide

Professor Alan Bundy's How to guides

Research Proposal

So, within the first 9 months, or so, of registration I need to be ready to submit a Research Proposal for the DPhil.

Given what I have read from other proposals the structure is generally as follows:

Introduction
     Research Questions*
Literature Review
     (Split into themes eg field, theory, methodology, application)
Pilot Study*
     Aims
     Methodology
     Categorisation
     Method
     Analysis
     Findings
Project proposal
     Research goal
     Methodology
     Scope
Proposed Studies
     Project Schedule (and chart/table etc.)
Conclusion
Exposure*
Future Work*
Bibliography

(* = Nice if I can add but not necessary)

The length seems to vary from 8,000 to 14,000 on average.

I'd best get cracking.

Qualitative Research in IS

Qualitative Research in Information Systems has a very useful overview of the field.