Machine Learning Models Controller App
This project was completed during a 2023 summer internship with Nokia Bell Labs as a UX Designer-Developer. Throughout the summer, I designed and implemented the UX for an Android mobile application which allows users to control machine learning models which will run on an external hardware device. The ML models may detect various substances and issues warnings to the user if a threshold has been crossed. The team and I conceptualized an application similar to a music-player app, which I prototyped in Figma, adhering to standards set by the Nokia Pure Design System. I also acted as front-end developer for the project. I wrote custom Kotlin code to govern the interactions available to users, as well as experimented with using a Figma and Android Studios plugin in Alpha, Relay, to directly bring some of my Figma-designed components into Kotlin code so I could integrate them with the project.
A large part of the design process focused on iteration to fit the needs of the application. Users could organize ML models into lists to be run in different scenarios, similar to a music playlist. However, there were some needs that a music player wouldn’t have, like prioritizing which model should run the most often when a hardware device didn’t have the space to run all at once. Here are some iterations on the design for the UX for prioritizing one model to run more often by default than the others in the list.
In the library section of the app, the user can create playlists of ML models. They can sort by recently played, alphabetical, and duration, as well as filter by what substances the models in each playlist can detect.
In the market page of the app, users can search for models from a database to save and add to playlists. They can find more details about each model and run them directly from the marketplace.
Unlike a music playlist, we can’t assume that only one model will play at a time. We also need to report data to the user about historical data and if a substance detected is over its threshold. We needed a way to convey which models are playing, which have been recently played, and their statuses respective to their thresholds or if data has not yet begun flowing.
The above image reflects the flow when a user creates a new playlist, gives it a name, adds models, and then removes one while in edit mode. All functionality shown in design images has also been implemented to create a fully functional Android App.
As a model plays, a user may tap on it to see a detailed view and live graph of current and historical data.
Each ML model can specify warning and critical thresholds for the substances they detect or data they return, and it was critical that we inform users when thresholds were crossed. I designed and implemented both in app notification screens and notifications that appear in the Android device notification curtain. I also designed and implemented a settings page for the app.
