This past weekend I participated in the hackathon MHacks X at the University of Michigan. The people there built a wide variety of things from VR games to analytics engines that tracked memes via Twitter and it was a generally great atmosphere for creating something new. I took the weekend as a chance to hit a couple of firsts on the way to having a lot of fun and generally enjoying a relatively low stress (as hackathons go) weekend while I built “Seeing Eye Droid”, an augmented reality app for the blind. It uses accessibility features built into Android and Google’s new AR Core to provide a better sense of the world through touch and audio. For the code, see Github – buckbaskin/eye.
My first first for the weekend was successfully building an Android app. I have, at various times, gone through the motions of looking up tutorials and attempting to compile my first app. I think I may have even gone so far as to have created a page or two that I could click between, but these apps never really counted. I didn’t have a strong motivation behind them or a unique idea to do something other than the most basic mechanics of putting in every feature that the tutorial talked about in order to learn how it worked. This past weekend, I launched directly past the Getting Started tutorial (it’s probably worth looking at honestly, but I didn’t make time for it) and landed at a web page about installing Google’s new AR Core software. Conveniently, AR Core comes with its own simple (as in minimal functionality, not actually simple code).
My second first was using Gradle. It was a small first and largely accidental. Android Studio projects deeply integrate with Gradle as far as I could tell, so I messed around with the build scripts until I got fewer compile time errors and had all my dependencies installed. I’m fairly certain I’ve meant to actually use it earlier, but I’ve never gotten around to actually finding a project with the right needs to use Gradle. Honestly, I haven’t had any projects that could use Gradle (to my knowledge) and I can say that for certain because Gradle is a hammer and I spent a lot of time looking for nails to use it on before it fell by my mental wayside. It was a nice find because I’d been meaning to learn about it and the weirdest quirk was the syntax (at least to me). In my head I was hoping it would just use Java or a Java-like syntax to specify how everything worked, but it turned out to be something different.
My third first of the weekend was using AR software as a user and a developer, in this case AR Core. It turned out to be a pretty intuitive way to interact with the world and it aligned with a lot of my robotics background. After spending some time reading through the documentation looking for interesting functionality to use and being mildly successful, I was able to get started pretty easily (and confirm that I hadn’t wasted time in classes in past years). My favorite find was the ray tracing capabilities (projected out from the screen) and the weirdest quirk was the documentation didn’t include the units for what I was getting. It turned out to be something close to meters, but I was never quite sure. Go check it out, I’d be happy to help anyone figure it out in the process of making something cool!
To go along with the AR Core software, my fourth first was using OpenGL and doing graphics rendering. My previous minimal Android experience involved using default buttons without giving any thought to visuals so that I could move forward with learning other meaningless incremental features. This time I jumped right in to messing with shaders, projection matrices and rendering classes. I was fortunate that I could start with the example AR Core app, because if I had not I would have had no chance at all in figuring out how to make the AR display work. It wasn’t terribly critical for my intended user, but it did help me debug my code and share what the app was doing with others at the hackathon would could see the screen. A conversation that I had during the demo time at the end of the hackathon led to a tripling of my understanding of what actually went on inside OpenGL (see https://learnopengl.com/) and shaders (shadertoy.com), so even just for that I’m happy I pursued the project.
My fifth first of the weekend was using Android’s accessibility services to make a disability-friendly app. I didn’t do anything terribly custom, but I did figure out a way to enable users to drag their finger across the screen to explore the world in front of them (both in estimating the distance to obstacles and using vibration to convey a sense of color). It was an interesting dive into the world of accessibility and it was interesting to learn about what all goes into making an app accessible. In the end, what I worked on was largely a single page app where the changing elements changed in the real world instead of on screen, so it wasn’t very complicated to manage an accessible user interface. If you’re at all interested, consider reading some of what Google’s put together about designing accessible apps here or watch a video about developing for visually impaired users by a visually impaired user.
My last first of the weekend turned into something that I wish I’d started way before. I spent a lot of my time this weekend reaching out to other participants at the hackathon to try and help them solve their problems. For most of the problems I helped with the solution came down to seeing something that a fresh set of eyes picked up that was hard to look for in code that had been seen a million times in 36 hours. For those kind of problems, it was the little things that counted. I’m happy to see that a number of the projects that I helped with a tiny bit turned out well. I also had the chance to share some of my machine learning knowledge this past weekend and it felt quite good to exercise what I’d picked up in and out of class. My initial impression of my own knowledge was that it was quite limited (and I almost certainly have tons to learn), but I found that I was still able to help multiple teams get their projects off the ground. Something is better than nothing! I hope to try and find some way to exercise what I know in the near future, perhaps on a Kaggle challenge or something of that sort.