Weeks 15 & 16 - August 8 to 19
Joe and I spent August 8 - 10 running the MDS study. We had 12 participants registered, but only 10 ended up showing up.
On August 12 (Friday) the entire lab went kayaking at Bowen Island for the day. I was a beautifully sunny day. We kayaked up to a beach where we had a picnic lunch and went swimming. I was a really nice way to end my summer with the lab.
My last week was spent compiling all of my work from the summer into a technical report and in lengthy discussions with Joe and Jerome about our findings from the summer. My work contributed to two papers, one submitted to the Haptics Symposium and the other to CHI 2006. We are still (as of November 2005) waiting for word on the acceptance of these papers. The CHI paper will be available on this site when we get word on the acceptance.
On September 6 I am starting a job as a user interaction designer a Power Measurement, a company near Victoria on Vancouver Island. I was wrestling back and forth with the ideas of grad school and industry work all summer. I think the reason that I decided to take an industry job for now is primarily for the real-world experience. Grad school is still a possibility for me in the future, but for now I'm happy with my choice and excited for the opportunities it will bring me.
My summer with the SPIN lab and Karon was an invaluable experience. I feel like I learned more this summer than almost any of my co-op work terms. It has got me really excited about working to improve computer and human experiences and interaction. It seems to be a historically undervalued discipline and I am looking forward to building a better relationship between human and machine.
Week 14 - August 2 - 5
On Tuesday this week we had two new people join us in the SPIN lab: Jerome Pasquero from the CIM (Center for Intelligent Machines) lab at McGill in Montreal, and Ricardo from Havana, Cuba. Jerome is a graduate student who helped Joe to build the handheld tactile device I've been working with all summer. He is here for 2-3 months to get first-hand experience with the work we are doing here. Ricardo is a graduate student who will be working on a tactile musical interface in the SPIN lab. So, Tuesday was spent welcoming them and continuing to prepare our MDS experiment for our user studies next week.
Wednesday and Thursday this week were spent at our SPIN Retreat at the graduate center at UBC. These were an intense two days of discussion, including talks and debates on the following topics:
Week 13 - July 25 to 29
This week went very well. We finished running our speed test user study and are now finishing up the statistics. It looks like that for all speeds but the fastest, people are accurately able to identify the direction of the wave. This means that we only require approximately a 600 microsecond propagation delay between each frame of the wave to communicate direction effectively. This says nothing, however, about the salience of the shape of the wave at different speeds, which is something we may uncover in the MDS (multi-dimensional study) I'm setting up right now.
After about three days I think I've got all of Mario's MDS software working with our device, including some of the statistical software. The task is quite simple: the user is presented with 36 buttons that each play one of 36 tactile icons on our device. The user plays the icons as many times as they want and sorts the icons into a self-defined number of groups (2 to 15). There are four more similar trials, but these subsequent trial enforce the number of groups at 3, 6, 9, 12, or 15. The outcome of this study will be a similarity matrix that shows which icons were most commonly grouped together. This will help us identify the salient features of the icons and come up with some guidelines for creating and using tactile icons on the device. For example, if all of the high-amplitude icons are grouped together and all of the low-amplitude icons are grouped together then we know that amplitude is a highly salient parameter of these icons. The variables we are using in our MDS study are wave shape, amplitude, and speed, which factors into the overall propagation time.
Next Monday is a holiday, and next Wednesday and Thursday are reserved for our lab retreat. So, that leaves Tuesday and Friday to run some pilot studies of our MDS program. We hope to run about ten subjects for our real user studies during the week of July 11. That will leave me about one week to write up the results.
Week 12 - July 18 to 22
Almost the biggest and the best news this week is that the device is now in service again after receiving 15+ coats of insulation. This meant that we were finally able to run the wave speed study that I've been talking about for the past couple of weeks. Before running the study, we made a change to the wave shape we used, which required about a day's work of programming and bug fixing of the underlying wave library. The old wave shape we were testing was a smooth(ish) sine wave. We were varying the apparent speed of the wave by increasing the wavelength (i.e. "stretching") the wave. We changed the wave shape to a standard almost-square wave (the edges were averaged slightly as to not damage the piezo actuators). Also, instead of changing the apparent speed by changing the wavelength, we kept it constant and instead varied the actual speed that the wave was propagated across the actuators. This had a slightly different tactile perception, but it standardized our test better and helped us to further define these parameters of the device.
We ran the study on five subjects this afternoon and the preliminary results look promising. I'll run several more subjects on Monday before absorbing the data in to the MDS study set up.
I said that the device being in service was almost the biggest and best news of the week. The biggest and best news is that I have been offered a position as a User Interaction Designer/Developer on the UI team at Power Measurement in Victoria. This is very exciting news for me, as the job is precisely in the field that I want to work in, I really like the company, and I really liked all the people that I talked to over there. This job starts after I'm finished my research term here for the summer.
Otherwise this week, I have been fixing a few bugs in my wave library and wave programmer and improving the UI of the wave programmer (after all, if I'm going to be a UI designer, I'd better put some effort into it). I also talked to Mario Enriquez about his MDS experiment and software and have started working on getting a machine set up to run the software on. This has been giving me a bit of grief. I was having a problem with Visual Basic 6.0. When it started up it displayed a message "Preparing to install..." and then did nothing. I tried uninstalling everything and reinstalling without success. What finally worked was deleting the file "C:\Program Files\Common Files\Microsoft Shared\OFFICE11\MSO.DLL" and re-installing Visual Studio. When I started Visual Basic I got the "Preparing to install..." message again, but this time it configured something in Visio 2003 and then everything worked! So next week I can actually start to tweak Mario's MDS application for our device.
Four more weeks, and still so much left to do!
Week 11 - July 11 to 15
Due to hardware difficulties, I wasn't able to finish running the preliminary wave speed user study. The varnish that we use to insulate the piezos on the device began to chip off and pose a risk of shocking people, so Joe experimented with a new type of insulator. But, it's taken a while to perfect the application of it, so the user study has been delayed.
In the meantime, I got the Mozilla browser app hooked up to my C++ code. You can now scroll through a webpage using the device alone, although the haptic feedback isn't tested, refined or fully implemented yet because the device has been out-of-service.
I began to think about how to set up the MDS (Multidimensional scaling) study for the device. See: http://cs.ubc.ca/~enriquez for paper's on this technique. I am meeting with Mario this week to see his software for this study and to talk about how to set it up.
Thursday I was in Victoria at a job interview for a user interaction designer position. On Friday I was on our lab team for the ICICS Cup event. In this event we competed against 19 other Computer Science and Engineering teams in a series of physical and mental challenges, including a scavenger hunt, relay race, mensa challenge, bocce tourney, Jenga showdown, and many others. Our MUX-lab team won the entire event! We each got a Jenga game, will have our lab name engraved in a trophy, and get a gift certificate to go out to a team lunch. It was a great afternoon!
The next few weeks now are crunch time. I have to get this preliminary wave speed study done and set up and run the MDS study so that I have some results to write up by the end of all this. The race is on...
Week 9 & 10 - June 27 to July 8
During the past two weeks I have: redesigned and finished implementing & testing my testing application, designed a user study, started to look into how to hook up my Mozilla browser application to the C++ libraries, gone to several job interviews, and gone away for a summery long weekend in Osoyoos, BC.
My testing application is now fully functional in the "wave under" output mode after a major UI re-design. I experimented for with having an automatic "averaging" feature that allows you to stretch the wave over 320 us increments and have the program automatically calculate the average values in between. However, this seemed very unaffective for what I hypothesize to be several reasons: first, the 320 us increments are so perceptually small that the averages were likely physiologically undetectable. Secondly, the amplifier output is already averaged; when I view the output on an oscilloscope with a +/-50V square wave, the changes are slightly sloped. Thirdly, you feel larger voltages changes better than small ones, so jumping from -20V to +20V is probably percieved better than ramping up from -20V to -10V, 0V, 10V, and finally 20V. I have not tested this formally, but when I removed the averaging feature and instead just stretched the waves over 320 us increments, the output we felt on the piezos was significantly and recognizably better.
I have been using this testing application extensively this past week to design waves for the small user study I've set up. This user study presents the user with a set of waves of the same length with a peak value of either -50V or +50V and different wave delays. "Wave delay" is defined as the number of 320 us intervals over which a certain point is stretched. A wave delay of 2 means that each point in the wave pattern is played for 640 us. In each trial, the user is asked whether the wave that is played is moving up or down under their finger and if they are confident in their answer or if it is a guess. We hope to find the minimum wave delay for which a user can accurately and confidently identify wave direction. These results will help me in designing the next "scroll over" output mode of my testing application because I will have a definite maximum wave speed limit.
This "scroll over" output mode is also necessary for my Mozilla browser application that I began developing in my first few weeks in the lab. I spent some time at the end of this week learning how to connect a JSP application with a C++ application through the use of XPCOM components and XPConnect. I had to do some minor redesign work in my testing application code to make it more portable to other applications like this browser application. I am now at the point where I think it will be relatively simple (1 or 2 days of work) to connect my browser application to my wave design and device output code. Defining the perceptual details of the application may take some more time.
With only 6 weeks left for my work this summer, I'm beginning to feel the time pressure. This week, I hope to finish running the above described user study and programming the browser application. In the next several weeks, I will design and run additional basic perceptual studies. I'd like to reserve the last 2 weeks to write my paper on the result of my research, though that will need to be evaluated based on how the studies are going and the results I am getting.
Week 8 - June 20 to June 24
This officially marks the halfway point of my 16 weeks of time with the SPIN group this summer. Time is going by so quickly and after talking with Joe & Karon this week I've realized that there is a lot of work to do!
Over Monday and Tuesday this week we (me, Vincent, & Jerome) managed to find an optimization for the device through a combination of hardware and software. The device takes 320 microseconds (us) to output a single image, however this did not result in a proper square wave output when oscillating between the maximum and minimum output voltages of the device. The current solution is to moderate this in software depending on the output: for small output changes, the 320 us is enough time, but for larger changes, we simply double the output image, giving the device 640 us to change voltages. I've been able to tweak this in our sample application with pretty good results; both Joe and Karon claim that they can feel the improvement. Knowing this limit, I've been able to implement an averaging algorithm for slow waves and get an overall better understanding of the limits of the device
On Wednesday our lab had an open house where we all gave demos of our work to each other and quite a few lab visitors. We had three visitors from Nokia (Finland & Vancouver) come in to visit the lab and, in particular, see our handheld haptics project. They seemed enthusiastic and had some good input to offer. After all the demos and a tour of the facilities the day was almost over. Joe, Karon and I had some time to meet to discuss our plans and goals for the rest of the summer. Right now, it looks like Joe and I are working towards a basic psychophysical user study on the abilities of the device to output discernable output patterns. This should happen in the next two or three weeks. After that, (and hopefully with some promising results in hand) we will move over to applications of the device. We will revisit my browsing application and I hope to pick up the work of some other graduate students done here on haptic playlist creating. They did their work using a twiddler (programmable force-feedback knob), but we can easily see their results transferring to our device. They mapped the tempo of a song to the number of indents per revolution on the twiddler and song energy (rated subjectively) to the amplitude of the indents. They found that subjects could categorize songs (create playlists) based on this haptic feedback. I am intersted in investigating this on our device. With the development of MP3 players as mobile devices this could have significant real-world applicability
I spent most of Thursday and Friday re-designing much of the UI of my testing application based on the results I came up with earlier in the week and discussions I've had with Karon. This redesign was made much easier by my discovery of the GTK GUI designer tool, Glade. I wish I'd known about it earlier!
The goal for this short week before Canada day: get the testing application done so that we can start designing our user studies!
Week 7 - June 13 to June 17
Between the lab social, participating in two pilot studies, and shopping for a lab couch at Ikea, I also found some time to do some work on my project! On Monday I was able to work with Jerome and Vincent (McGill) to identify and solve two problems that were limiting the output rate of the device.
Week 6 - June 6 to June 10
One year ago today I was in Paris and one year ago this Sunday was my first day in
Belfast. For those of you who don't know, last year I did a 10-week IAESTE work
exchange and Queen's University Belfast and I'm getting at little retrospective about it now.
Most of Monday this week was spent cleaning up lab space in preparation for a new prototyping lab.
It was a good way to get to know a few more people in the department better.
The rest week was spent mainly on development of my refined device testing application.
I can now read/write the wave patterns from/to XML format and I spent a lot of time improving most of the UI design
and program design to, hopefully, make future program development and improvements easier. I think this will pay off,
because after meeting with Karon this week, I've realized that there are many more parameters to the device that I've
implemented now. For instance, we need to move towards outputting more of a smooth wave than a square wave, which can
be done by averaging the discrete actuator values set in the application and increasing the output frequency to the
device (more in the paragraph below). Secondly, we need to be able
to create non-uniform wave types and customize the waves across time and in relation to slider position. The application I have now
does that to a certain extent, but is not nearly as customizable and refined as it needs to be.
Karon was in Montreal this week talking with the McGill people and they discussed some way to improve the "smoothness"
feel of the device. After talking with her, Vincent, and Jerome, we've come up with several ways to do this.
First of all, I hooked up the device output to an oscilloscope and found out that I was running the device in the wrong mode.
I fixed that, which has now doubled the sampling rate. Vincent has also suggested another software improvement where I will reduce
expensive reads from the device and buffer the writes on the FPGA board to speed up the piezo output.
These sampling improvements will be my first task next week. After experimenting with that and, if necessary, considering alternatives I
will continue my application development.
Week 5 - May 30 to June 3
This was Joe's first week away, so almost all of my work has been
independent this week. The first half of the week was spent
learning the device libraries and brushing up on my C/C++ skills. The
last half was spent designing and implementing a testing tool for the
device that will allow us to easily manipulate all of its variables
and study the effects. It will hopefully come in handy
in determining the capabilities of the device, designing haptic icons
for the device, and possibly even some psychophysical user
studies. This tool is part of the "Haptic Icon Experimentation" task described in
last week's entry.
The variables that I have identified and/or implemented in
the device so far are:
Week 4 - May 23 to May 27
This week has been a short, but productive week. Monday was a holiday (Victoria Day) and Friday is my convocation, but in the 3-day week we had we managed to wire up our +50V/-50V power supply, switch the test machine to another box, re-install and configure all the software, get the device fully operational, and come up with a good plan for the next few weeks.
We decided to switch to another box primarily because the the USB drives on the old box were flaky. After failing to even get a USB mouse working, we decided to elimate the potential for USB problems by switching to a box where the USB drives actually worked. We upgraded to from Suse 9.2 to Suse 9.3 on the new box (1) because we needed a higher kernel version (we think this is the reason the device was crashing the machine last week) and (2) to have 'safer' installations of all the higher versions of support software we needed.
After this upgrade, the software installation and configuration went much more smoothly than last week. We've now been able to get the device completely hooked up and tested. I am very impressed. I think I'll be having a lot of fun with it this summer. Most of the initial reactions, including my own, have been that the device creates a positive feeling and a desire for continued use.
Joe and I met with Karon today to discuss the best plan of action for me during the next three weeks while Joe is in Japan. We decided that probably the best thing to now, instead of working on specific applications of the device, is explore all the different ways we can create distinguishable and different haptic icons (otherwise called tactile icons, tactons, hapticons, etc.). This will include defining the dimensions that we have to vary and how much we can vary them by. Before I begin that, I'll first need to spend several days reading through the device libraries to learn how to program the device.
The ideas we came up with during our brainstorming are:
Week 3 - May 16 to May 20
The device got here from Montreal this week, so Joe and I got a chance to get a good look at it. Unfortunately, we haven't been to try it out yet because Joe is still working on the power supply for it and I've been struggling to get all the software for it configured. For now, I'm going to refrain from describing the details of the device because the word is to keep it quiet. I will say that it is a prototype of a haptic interface on a mobile device and that it is really cool!
Late last Friday I solved the web progress listener problem in the Mozilla browser application I was writing. The browser is basically functional now. It works better on some pages than others, still only highlights link elements, and doesn't do very much small screen formatting yet. I think that these things will be better addressed once we have tried out the brower with the device to see what is possible and after we have made more decisions about how far we want to go with this project. It would be great to have a very robust and functional mobile browser, but may be better to spend more time searching out other applications of the device. My first priority right now is getting the device up and running.
The bulk of my week has been spent in a love-hate relationship with Suse Linux. I've been working with the incredibly patient and helpful Vincent Hayward at McGill in Montreal to configure the software required to run and program the device. Most of it has been very simple, straightforward and easy. It's been a process of trying to install one component, identifying dependent libraries, installing the dependent libraries and finally successfully installing the component. Unfortunately, we hit one major roadblock: ACE. ACE is an "open-source object-oriented (OO) framework that implements many core patterns for concurrent communication software". The big problem was that Vincent and I were not able to find a good ACE package for Suse, so I had to build it from source. I was able to get ACE itself going okay at first, but couldn't get ACEXML (an XML parsing tool based on ACE) installed correctly. Finally, on Vincent's suggestion, I used the ACE Beta kits because they seemed to install more successfully than the release versions for both ACE and ACEXML. After much trial and error I've mangaged to get ACE, ACEXML, and all of the software for the device compiled, installed and running. I've been able to hook up to the device to test the scrollbar and button clicks; the scrollbar works, the button clicks don't. Unfortunately, we have an even bigger issue: after a minute or so of using the device the entire system freezes. We're working on debugging this problem now.
Joe leaves for Japan at the end of next week. I'd like to get the device completely up and running before then so that I am able to work with it in his absence. Karon, Joe, and I discussed some application ideas that hopefully I'll be able to experiment with on the device while Joe is away. First and foremost, we need to get the device running. After that, my list of tasks/goals is as follows:
Week 2 - May 9 to May 13
The device from Montreal still isn't here. I could try to describe it now, but I think that I'll be able to describe it much more accurately once I've actually seen it and played with it, so for now I'll just describe what I've been up to this week.
I've continued to work on the scrolling web browser this week. The task at hand seems pretty simple: I want to listen for mouse scroll wheel events and respond to them by changing the selected hyperlink on the web page. The selected hyperlink is indicated by a highlighted background. Well, even with the help of our resident Mozilla guru, Nelson Siu (in the Human Communications Technology (HCT) lab across the hall), this task was challenging. I'm almost there though!
Google has been my best friend this week. These are links that have proved invaluable to me this week:
Week 1 - May 2 to May 5
One week down, fifteen to go (not that I'm counting or anything)! I got a good start on my project this week, in between setting up computers, multitudes of paperwork, setting up this website, and unpacking boxes of stuff at my new apartment.
Joe (the M.Sc. student I'm working with) and I hashed out a schedule for the summer, which is somewhat complicated by the fact that we haven't recieved the hardware device from Montreal yet, and we don't have a good ETA for its arrival. My first task is to create a stripped-down web browser that we can use in our user studies. The basic idea of the browser combines Opera's small-screen formatting mode, link highlighting, the ability to move the highlighting with the mouse wheel, (similar to the function of TAB / SHIFT + TAB in Mozilla, or CTRL + UP ARROW / CTRL + DOWN ARROW in Opera), and the ability to follow a selected link by pressing the middle mouse button. We need the UI of the browser to be very lightweight in order to display it on the small device screen. We need to be able to navigate through the page only with the scroll wheel on the mouse. Eventually, the input events on the device will be mapped to these wheel events. The final step will be to integrate haptic feedback into the device so that the user can "feel" hyperlinks, text, images, and/or other page elements.
I am implementing this specialized browser using Mozilla. I was surprised to learn that Mozilla, in addition to being the choice open-source browser of techies everywhere, is really an entire application framework. XUL (pronounced "zool") is a user-interface definition language based in XML. You can hook it up with some JavaScript and CSS stylesheets to easily make some nifty applications. My best reference so far has been the main tutorial on XULPlanet. The Mozilla application suite is all built using these methods, and because Mozilla is all open-source it's easy enough to plug-in existing Mozilla functionality into your own application. This is how I'm building our browser.
So, what I've actually got working as of 1:45pm on Friday is this: