blue seperator


Hello! My name is Lauren Gaber, and I attend the University of Michigan Dearborn. I am a junior majoring in computer science in the Computer Information and Science Department, planning to graduate around April 2017. You can reach me at lgaber@umich.edu.

blue seperator


My mentor for this project was Professor Shaun Kane, who currently teaches at the University of Colorado Boulder in the Computer Science Department where I performed the research internship. He studies accessible user interfaces and mobile human-computer interaction. His website can be viewed here.

blue seperator


The research project goal was to work on developing a wearable assistive augmentative communication (AAC) device that could help a non-communicative person in a wider variety of contexts, such as in mobile situations like horse riding therapy.

blue seperator


blue seperator

WEEK OF 5/18 - 5/22


THIS WEEK'S GOAL: Get set up, IDs, make a research plan



    I got a desk, ate food with people, finished reading and taking notes on VizWiz paper, still need to get bus, library, and key passes. Maybe one or two more things? I forgot what they were.


Goals for 5/19:



    Estimating the viability of sequential image adaptation/analysis (and reading more papers) seems to be the next step. Basically the idea is developing easy software tools and steps for adapting sequential images (such as building instructions or comic media) into scripts for the blind to read or follow. Some parts can be automated, but others cannot. Also, making comics from video was talked about. I really want to be a fairer judge of how useful it actually will be (basically, the idea is in reducing the difficulty of or expediting the process of translating a certain type of image or visual thing (in this case sequential images) into an accessible format)... I'm waffling about it. Needs more work and viability estimation. Might use OCR, thought about font pairing with OCR. Something that can identify and read/transcribe multiple fonts? Not handwriting, that is definitely too difficult. Some of the process can be automated and some can be human fueled (corrections to computer estimates, what other aspects need to be noted. But how to judge that?



    It was still hard to say, I couldn't read some of the comics related articles and I only found one I could read at the literal end of the day. It would probably be done in a .net windows form or something. In c#.

    Major project desires:

1.     assist with wearable tech things Halley is doing

2.     cheaper faster personal Braille printers

3.     developing and testing more Braille learning games

4.     comics idea might be main thing if not other things



    I have to finish my research plan outlines for tomorrow, today went pretty well



    Worked from home, finished preparing 6 ideas.

blue seperator

WEEK OF 5/26 - 5/29


THIS WEEK'S GOAL: to be determined (tbd)



    Idea #2 was picked and I started working on Arduino stuff. Also I need human trial training or something like that.



    Did more Arduino tutorials, ended on 10.



    Did the rest of the Arduino tutorials, and met with Shaun and Ariel to talk about buttons.



    Completed IRB human research subject training.

blue seperator

WEEK OF 6/1 - 6/5


THIS WEEK'S GOAL: prototyping



    Read about serial communication.



    Read about I2C and SPI communication.



    Out sick.



    Read more tutorials and papers and sketched more ideas.



    Read tutorials for the ESP8266 Thing.

blue seperator

WEEK OF 6/8 - 6/12


THIS WEEK'S GOAL: prototyping



    Tried to get SparkFun ESP8266 Thing to work, couldn't.



    Tried again after more fiddling, couldn't, and went home early.



    New parts should be coming in today; in the meanwhile I attended and took notes on an extremely good webinar from ACM called "Making the Digital and Physical Worlds Accessible for People with Disabilities."



    Joined slack, did stuff that didn't feel like much. Figured out we needed to mess with WiFi stuff and Mac addresses. Toured Halley's lab in the basement and saw a Baxter, then soldered some!



    Got Yun working with Temboo example! I don't know how long that's going to last. I have more resources to read over in a new document that was made. I know I have to go back to programming with the Thing (not much progress but now I can read the serial output using Realterm), But I am very excited and confident in the Yun's ability to be a prototyping platform.

blue seperator

WEEK OF 6/15 - 6/18


THIS WEEK'S GOAL: prototyping



    Tried to get thing board to work, a little bit ago maybe last week we were able to finally read serial messages by using Realterm. You got to the port tab and un-open the connection, go to pins, clear the DTR pin, go back to the port stuff, set the baud rate to 115200, set the com to 5 (/vpo) or something like that, and then open the connection again.



    Soldered 2 other ESP8266 Thing boards, and it finally worked! The first one got up to a point where it was making a WiFi connection but nothing was able to connect to it. It was a bad board. The other two boards are good now.



    Went out to lunch with many people to black pepper pho, and also programmed the 3rd thing board (58 not 82 which is the second one) to be able to ding a website Shaun made yesterday that increases a number on the page every time it's visited. I also wrote another program where you can press a button and then press the make a connection button and it will change the url to the new one. It just dings the same website though, even though the end of the url changed. There is also occasional random reconnecting which is bothersome, what if it does that all the time? I couldn't make a php web page like that yet. I also got to make it blink when it connects (I'm going to have it blink the same # of times you pressed the yellow button to change the url, or make it do that when you press the red button.) Family also visited which was nice.



    Demo'd the 3 tests for Shaun, started investigating platforms to build phone apps on. Downloaded Xamarin, and I requested free student license for it. Tried running a basic Android app, worked in XamarinStudio but not VisualStudio. Needs to be fixed.

blue seperator

WEEK OF 6/23 - 6/26





    Missed volunteering at center for the blind in Denver on the 20th because of my cousin's wedding. I want to see if there is another opportunity to volunteer before I leave... Performed successful basic programming projects in both Xamarin and Visual Studio. Started a build up project, where I am using the old speech synth but I test when the button is pressed and I compare previous and current #s gathered from the web page. Thought about using php code to clear and/or log data the end of every day. Need to check out (literally, go find the stuff you bookmarked weeks ago) ways to use WiFi or just directly program/communicate between Arduino and android app.



    Mostly just programmed more in Xamarin android and read a bunch of resources.



    Made a mistake, was working on Xamarin android things when I needed to be working on Xamarin.Forms things (cross platform was the big point). To get adjusted faster I translated a huge example, the forms gallery.



    Read examples for local databases and stuff and started working on the first Xamarin.Forms test project, which is just basic text to speech, and no memory.

blue seperator

WEEK OF 6/29 - 7/2





    I fixed this problem with the WebClient.DownloadString method by using the dependency service like I did with the text to speech (TTS). Figured out how to run listening in the background (later moved up tier because the method was being called by every on TTS object and increasing the number super fast.)



    Programmed in the ability to manipulate pitch and speed of the TTS but struggled with getting a local database example to work.



    Reproduced local database stuff and started combining/updating it with the last project's breakthroughs. Later in the day I was able to get it to a level of basic functionality and it didn't hit up the webpage so often now. Now, whichever first one in the list is on, that's the one that's going to be spoken.



    Tried working on getting the conversion from a string to an xml document to work, but the parse method just kept getting this same error and I couldn't figure out what it was. Later I worked on image choosing (at the end of the day I think I found a good example, so I'm going to read that ASAP). I also looked at how people can record audio, it looks a little complicated but probably workable if we use the dependency service again.

blue seperator

WEEK OF 7/6 - 7/10





    The problem with parse was that the string already needed to contain xml code to be able to be parsed and converted into an Xdocument.



    I updated the Arduino code! Might need slightly more reverb room... Also got all these materials to make button covers, and started the app update and stuff...



    Did a whole big goose chase and finally figured out how to translate json files to objects in c#. Dug myself into a hole later because I waited a long time to test after changing a lot of things in the structure of the app.



    What's going on: almost done with the app update, just have to make it listen for triggers; then I have to clean up the code some, then I will make another button or two and I think tomorrow Halley and I are going to look at making button covers.



    We made an outline of a button cover, but the app runs too slow so I have to look at how to run the looping thing in the background.

blue seperator

WEEK OF 7/13 - 7/17





    Worked on the timer delay problem and also the tiny ESP board was finally programmed. I have to work on the audio recording and fix the dang timer problem...



    Worked on time delay problem more



Fixed the delay and duplicate problem with the button response time; Also I attended the Dr. Warren Allen -- "Beyond Platform-centric Social Media Research: Challenges and Opportunities" talk that happened right outside the office. I started reading about the audio stuff and doing some initial testing.



    Got halfway through audio stuff and worked on some bugs before I had to go home sick.



    I finished adding audio recording stuff for the android side only. It would likely be really difficult to program without actually testing on an iPhone, but you need a Mac to do that.
blue seperator

WEEK OF 7/20 - 7/24





    Worked on adding the chat client in the morning, finished for the most part. I have to edit my code so it's nice and efficient and not cluttered and well documented. Then I might want to work on aesthetics, but I am pretty happy with it now. I have to prepare a to-do list for exiting the operation.



    Worked on the schedule for the last two weeks, solved a paperwork problem, and documented code.



    In the morning I finished up documenting the C# code... and started a code notes document for it. Had a meeting about case design for the button, figured out we needed to do a bit more testing with sleep and heat stuff



    The parts didn't come and it was hard just fiddling around with Arduino code. I visited the BTU lab and picked up some wood stuff with Halley.



    The parts came today and I finished documenting all the code and I put it on a usb drive (that was hard!). I am refocusing and trying to prioritize my time better. I started working on the next steps of scripting and 3-d model building, and I am going to try to get the script(s) done this weekend.
blue seperator

WEEK OF 7/20 - 7/24


THIS WEEK'S GOAL: PROTOTYPING (finishing up), doing light usability testing, getting feedback from an OT, exiting the internship and completing DREU milestones



    So far I have tested Halley's hookup of the board and it works great and I resized my 3D modeled case to be just the right size. Worked on milestones and GitHub stuff. Checked out a Google Glass. I also printed something on the 3D printer! It was the button's box, and it turned out that it was not quite the right size! I had to make it taller and thinner. Started thinking about how to implement the video. We are doing some light testing with users tomorrow.



    I performed user testing meetings/interviews with two students this morning, a grad and an undergrad. I printed off the updated 3D model, and the button fits really well. I transcribed the notes.  I got to try out an oculus rift, it was really cool and it didn't make me sick.



    I performed more user testing stuff, went out to lunch to talk about PhD programs, and interviewed the occupational therapist with Halley.



Last day! I transcribed the interviews, cleaned up my station and made a PowerPoint presentation to serve as a demo of the app and button.

blue seperator


The final report for this project can be found here: Wearable AAC Final Report

blue seperator