Chordable iOS Development
September - December 2023
The team was tasked to develop an innovative iOS app aimed at addressing a niche market gap while contributing positively to the community. Over the course of a semester, we meticulously planned, designed, and developed an application for guitar beginners. This endeavor culminated in a presentation before a distinguished panel of judges representing Capital One. Out of a competitive field comprising 21 teams, our team secured an impressive 3rd place and a category award for the Most Educational App.
Why
The learning market for instruments does not present an effective solution for online learning. Especially for guitars, being the #1 instrument people want to learn, do not present an app with a one-stop-shop platform.
How
Design
-
Figma
​
Development
-
Swift
Project
Mobile App Development Course (67-443)
Team
Ariel Kwak
Minjoo Kim
Owen Gometz
APP OVERVIEW
App's Main Features
Popular Song Choices
The app stores 188 different songs of different genres that users can learn from. these songs are imported from a git repository Users can click on these songs in the “Songs” tab to view the chord charts and timing information.
​
Motivating Users to Learn
As mentioned earlier, users could have song choices under the “Songs” tab. However, the user doesn’t know any chords yet! All songs start off as locked for users. In order for a user to be able to learn a given song, they must have learned all the chords present in the song.
​
Chord Learning
Other than the “Songs” tab, there will be another tab called “Chords” which contains 24 unique chords a user can learn. All chords start off as incomplete. The app knows the vast majority of existing chords, and users will display those chords in a grid-like format in ascending order of difficulty. Users can click on chords to learn them.
​
Auto Detection
When users click on a chord, they have displayed the chord shape on a guitar neck and are guided through the process of putting which finger where. When they feel they are ready, they can start a 2-3 second recording and play
the chord. The app will detect if the chord was played properly, and if it was, the chord is marked complete.
Song Learning & Feedback System
Once users have learned all the chords required for a given song, they will be displayed the chord chart of that song and will be able to practice at their own leisure. Once they feel like they want some feedback on how they are playing it, they
can again record a snippet of their progress, either the verse or chorus, and the app can provide feedback. Our feedback will mainly track how long it is taking users to go from one chord to another.
TECHNICAL ASPECTS
CoreData
In the pursuit of delivering a seamless and highly optimized user experience on Apple devices, the team has chosen to use Core Data, Apple's own robust and comprehensive framework. This natively stores and manages our application's data, ensuring that our solution is not only well-integrated within the Apple ecosystem but also benefits from the efficiency and advanced features that Core Data offers. This decision allows for faster data access speeds and the ability to function seamlessly without the need for a continuous Wi-Fi connection. Further, since our app does not store a very large amount of data, employing Core Data does not impose significant storage burdens on our users' devices, thereby maintaining a lightweight and efficient user experience. The data requirements also fit well with the Object-Relational Mapping framework, since all of our data is easily represented with entity definitions and relationships between them. Below is the ERD of how the team organized the data stored in CoreData.​
Below is a further implementation of how the team has organized how data is utilized for each screen with code samples and reading data from CoreData.
Launching Page & Home Page
Machine Learning
In order to complete a chord, users can utilize the “Try It” feature to record themselves playing a strum of that chord, to which the app will respond with success or try-again feedback. Powered by our chord_recognition machine learning model, if it was a success, that chord has been completed.​
​
In regards to the machine learning model that powers the app’s chord recognition capabilities, this model was built from scratch. Albeit with the use of the handy sci-kit-learn Python library, all of the training data was recorded personally by the developers. The model is trained on 75 raw audio files for each of the 24 chords, totaling 1800 files in total. The recordings were gathered using various guitars in different settings to justify a generalizable model. Further, to artificially increase the sample size, 5 iterations of augmentations were applied to add white noise, slightly increase & lower pitch, and speed up & down the files. This totals 10,800 pieces of training data. All this was done in an effort to produce a model that will make accurate predictions in the variety of settings that users may find themselves in. In terms of deployment, this ML model was embedded within a Flask app, which was uploaded to a pythonAnywhere-supported environment, from which the app could make POST requests from our IOS app.
Spotify Connection (API)
The team has integrated with the Spotify API to further benefit our users. When browsing through songs, the album covers are acquired from the Spotify endpoint so that users have a higher likelihood of recognizing a song, and in turn, becoming more motivated to unlock it and play along. Additionally, when users unlock songs, they can view an animation that shows when to transition between chords while the song is simultaneously playing in the background. This is made possible through our use of the Spotify API.​
Audio Recording
In order to give users accurate feedback to accommodate their journey in learning to play the guitar, the team embedded a chord recognition ML model (aforementioned) which requires audio recording from the users. Utilizing the iOS environment, the team was able to implement audio recording permission with a limited time duration of 5 seconds after a 3-second countdown to give users time for preparation.​
TECHNICAL CHALLENGES
Finding Chords
The team tried to utilize Ultimate Guitar's API for importing chord information for each song. However, as Ultimate Guitar's API is not public the team instead used a GitHub repository where the repository owner has organized chord information for 188 songs. Thus Chordable supports 188 classic songs from the late 20th century that are recognizable, with mostly simple progressions. Users will find that perfecting chord transition timing is made much easier when there is a simple and slow progression and when they recognize the tune. The data on these songs have been gathered from a public GitHub repository, credits to tmc323: https://github.com/tmc323/Chord-Annotations/tree/master.
Spotify
Initially, the team thought Spotify API would include different chord information for each song but the team was still proven wrong. By utilizing a GitHub repository instead, the team soon realized the team's initial plan of importing a user's song and playlist information was not attainable. However, the team still decided to utilize Spotify's API to not hardcode song information and increase data storage for each user's device on CoreData (thus decreasing the loading time for the app's launch). The team utilized Spotify's API to import information on each song like the song audio (for background playing while a user is learning to play a song), the song's artist, the song's title, and the song's album cover.
Eliminating Initial Tutorial
Initially, when the app first launches, the team was planning to implement an interactive tutorial where users would learn step-by-step how to interact with the app. However, the team has eliminated this tutorial by further simplifying the app for the tutorial to be unnecessary. The team pivoted the app to fully focus on beginners instead of having different levels available (beginner and intermediate players) and further simplified the design to decrease users' mental overload.
FINAL PROTOTYPE
Interact with the prototype below!
PITCH
Below is the pitch deck the team utilized to present in front of the judges from Capital One (some slides including videos have been eliminated). The team won 3rd place and most educative app.