Embedded Files
A Kennesaw State University Capstone Project
A Kennesaw State University Capstone Project
Adviser/Instructor: Dr. Ying Xie
Adviser/Instructor: Dr. Ying Xie
Sponsor: Dr. Linh Le
Sponsor: Dr. Linh Le
Team Lead: Jacob Barnett
Team Lead: Jacob Barnett
Team: Henry Colomb, Amari McGee, Christian Moore, and Jack Wengert
Team: Henry Colomb, Amari McGee, Christian Moore, and Jack Wengert
Overview of our Capstone - Why The Clarity Project?
Overview of our Capstone - Why The Clarity Project?
- Who is this for?
- Anyone who listens to lectures, podcasts or voices digitally!
- What does it do?
- Uses an AI LSTM model to train on simulated noisy vocal samples, and attempts to turn them into clean vocal samples that are set as a target. Over time with large training sets, it can make clear vocal audio out of sound files it hasn't heard before.
- Why AI Neural Network over Traditional Filters?
- Explained in the video below, but in short, it has potential to work much better with minimal distortion in comparison!
- What is the Scope?
- We currently can simulate up to 6,800 different "Noisy Vocal" tracks currently to train our model on, with 400 target "clean" vocal tracks to compare it to.
- How can we improve?
- We need more data! You can send a recording of your voice (max 10 seconds) to KSUClarityProject@gmail.com to help. Clear audio only please.
- Looking for the words to say?
- Check out our phrases page for sentences to record yourself speaking. These phrases are designed to make a speaker use specific verbiage that can sound similar to background noises. For example, phrases with "sh", "th", "fh", and hard T sounds.
5-Minute Video of Why We Believe in The Clarity Project
5-Minute Video of Why We Believe in The Clarity Project
18 - Minute Full Presentation of Project
18 - Minute Full Presentation of Project
First Approach Using Open-Unmix as a Proof of Concept
First Approach Using Open-Unmix as a Proof of Concept
We decided to take a first approach by using the Open-Unmix python library built with PyTorch. This library is designed to extract different sources from music. You could use it to extract vocals, drums, and bass into separate audio files. We adapted this to work with speech and background noise instead.
Open-Unmix provides an LSTM machine learning model built on top of the PyTorch machine learning library. We molded our source files to fit what Open-Unmix expects by extending the recordings to 5 seconds to match the length of the noise files and make sure the sample rate also matches. We sorted out data into an 80% training, 20% validation split. After running the training many times while tweaking the learning rate, batch size, sample size, validation size, we got the following as our best results:
Our Best Results with Open-Unmix
Our Best Results with Open-Unmix
Our training loss became under 2%, over the course around 20 epochs, and flat-lined around 100 epochs. Our accuracy still needs to improve, but the results are looking up with more data to train.
Our training loss became under 2%, over the course around 20 epochs, and flat-lined around 100 epochs. Our accuracy still needs to improve, but the results are looking up with more data to train.
Audio Before Being Processed
Audio Before Being Processed
before_prediction.wav(Open-Unmix)
(Open-Unmix)
Audio After Being Processed
Audio After Being Processed
after_prediction.wavAudio Before Being Processed
Audio Before Being Processed
BeforeCPPrediction.wav(The Clarity Project so far)
(The Clarity Project so far)
Audio After Being Processed
Audio After Being Processed
CPModelsPrediction.wavPage updated
Google Sites
Report abuse