Do Good Things Justice for All

“Do Good Things Justice for All,” an IEEE SIGHT Project – funded by Humanitarian Activities Committee SIGHT Project funding 2019 - 2020.

We are delighted for the opportunity to create the Pilot for the Experiential Learning Simulation and Training System with Captions (ELSTS). The System will use Virtual Reality, provide measures of the acoustic scenes, and augment meetings with single user Captioning on a display.

ELTS will help people understand how people with hearing loss hear the world and how Captioning can improve speech understanding.

We will design and develop the Pilot with close collaboration of the IEEE NJ Coast Section SIGHT Group team members, Stevens Institute of Technology and the Intelligent Network Systems Lab where we will engage engineering students and professors, Stakeholders, SIGHT Group Members, IEEE PACE, perspective Members, etc. to reduce risks. We will test the Pilot in Meetings and (Covid accommodations to be determined) Classrooms at Stevens Institute of Technology, at the Nyack Mock Trial Team (Covid changes - practices and competition have been canceled and courtrooms are now closed). In addition, the team was a finalist in the ‘Do Good Robotics Competition’ at UofM College Park, inspiring the name for our project. We hope to attract more interest, potential funding, etc. We have also attracted support from the SociAvi collaboration for aging community members which has great value and provides reach to individuals in need. This need has expanded dramatically since the advent of Covid 19.

June 2020 -- Update -- Summer Mentoring Program Available Through School District Collaboration:

  • Tiffany Lucey Supervisor of Educational Technology Toms River Regional Schools 1144 Hooper Avenue Suite 210 Toms River, NJ 08753 732-505-5500 ext 7

Tiffany Lucey's Summer Program will offer opportunities for Students to improve their technology skills and for expert Mentoring. Our IEEE PACE SIGHT Group Project provides a perfect opportunity for everyone. We look forward to hearing from the Students and we Engineers and Volunteers will enthusiastically participate in the Summer Project to provide Mentoring to the students.

Please contact Tiffany Lucey at

Please contact Kit August at

Please contact Tom Willis at

Please contact Giovanni Vannucci

( Also from Tiffany Lucey: Join us for Jersey Shore Makerfest 10/24/2020 YouTube )

15 May 2020 -- Update Comment by Dave Nall, Simulation Architect:

Task Progress (experiential hearing loss simulator) :

The phone based conference bridge with the moderator audio profile modification has been implemented. Attendees to the conference bridge can experience the effect of hearing loss while listening to the moderator. The audio profile can be altered while the moderator is speaking to provide different experiences. The conference bridge audio can be recorded for future use.

Observations from the Project:

The project has provided a lot more insight of how broad the impact is when hearing loss is involved in everyday tasks. From simple everyday interactions (stores, offices, meals, and groups) to the needs to interact with government services and the legal system. While I see the result of the ADA in curb cuts and ramps for those with more physical impacts, now that I am looking, I do not see as many results for the benefit of hearing impaired.

The Caption system is expected to improve speech understanding for those who are aging, have moderate hearing loss, students with hearing loss, and others. Not everybody uses Sign Language. Not everybody benefits from hearing aids or FM systems. Captioning can fill in the gaps at a much lower cost. We intend to demonstrate the ease and convenience of Captioning in our pilot. We will measure the cost of the system, time to set up, and flexibility for use. We will conduct surveys of users.


An advantage of working with the Nyack Mock Trial Team is that there is no actual plaintiff nor defendant, and therefore, without compromising privacy the work focuses mainly on the environment and logistics within the Court, among real attorneys, judges, and clerks. At present, the courts are closed. However, the need for accommodation in the legal system is as great for witnesses, attorneys, judges, plaintiffs, defendant, and others. And now with Covid 19, many people are wearing masks which reduces the loudness of speaking voices by as much as 10 dB and obscures the mouth of the speaker making speech even more difficult to understand. Captioning can help with speech understanding.

Video of Pizza with Mock Trial Team and IEEE Volunteers:

Video Example of Mock Trial Team Member at practice:

Video Example of Mock Trial Competition 1

Video Examples of Mock Trial Competition 1

We have developed a very good relationship with the Rockland Bar Association which is helping us with access and audience, the Rockland County Clerk’s office (The Honorable Paul Piperato – who has recently passed away 5 May 2020), and many others. Law & Mediation PLLC, Carolyn M. Laredo, Esquire, who consults on cases involving the Americans with Disabilities Act (ADA), provided strong input that resulted in bringing together many concepts inspiring the Experiential Learning Simulation and Training System.

Professor Victor Lawrence, Professor Hong Man, Professor LTC Kennedy-Pegues of US Military Academy West Point, Dr Avi Hauser, Dave Nall, Dr August, Dr Jelinek, Dr Sellathurai, Dr Willis, Dr Vannucci, Dr Wilkus, and many others on the team are investigating hearing loss in the Military which is a complex problem. We are consulting with Professor Skip Rizzo, an expert in Virtual Reality for Veterans and Military personnel. We will develop scenarios for the Experiential Learning Simulation and Training System with Captions to articulate experiences of the Military in the field and following hearing injuries, to help Veterans and others understand the hearing loss they experience, and to improve research and intervention. We will prepare articles to raise awareness and plan additional work for the future.

We are making progress on the Pilot for the Experiential Learning Simulation and Training System with Captions (ELSTS) to improve speech understanding and help people understand how people with hearing loss hear the world. We successfully configured the large screen Captions display and developed the Simulator that alters the voice so listeners can experience how those with hearing loss hear the world. We have taken many recordings at Mock Trial practice and at two Courtroom competitions. Due to Coronavirus and New York on Pause we must alter our Pilot implementations and coordinate with our partners to proceed safely.

The ELSTS will provide Scenarios to help people understand how hearing loss impacts humans in everyday life, it will train people to understand how to improve communication and reduce disparity, and it will Simulate the sensory experience of having a hearing loss. The Captioning will enable display of text of a person’s speech so others can read to improve speech understanding especially in noisy environments.

The Pilot system will be tested and validated by invitation and with Covid 19 considerations. with Stevens Institute of Technology in the lab, classroom and meeting attendees. Picatinny Arsenal, the Military Research group will be invited to review. The Pilot will improve understanding of why and how to comply with the Americans with Disabilities Act. Mediation is a diversion to court providing Justice for those with hearing loss when businesses, schools, organization fail to accommodate. The Simulation and Caption system will be designed and Piloted (with Covid 19 accommodations – with the Nyack Mock Trial a high school team in practice and competition, and or during presentations to Bar Association Members and Stakeholders). We will continue our outreach to learn more about NGOs and others who would benefit. We propose to have a greater impact on helping people to understand Sustainable Development Goals - how low-cost technology can reduce suffering, improve communication, speech understanding, and provide opportunity to reduce disparity – help more with fewer resources. We will reduce risks in execution by simplifying the system design and components using mostly off the shelf highly sustainable low complexity solutions (camera, PC, Oculus, level meter, projector) aiming for downloadable apps and simple configurations. Our Pilot testing is set up so that we have at least three places we can conduct testing, and depending on availability and resources, we can shift order and so forth. We involve Stakeholders on a diverse team supported by Stevens Institute of Technologies experienced Professors, a variety of Volunteers, engaging others who will help capture Stories and Scenarios, and communicating with others working in the same fragmented field with similar goals, including Fraunhofer Research (open source algorithms for hearing aids). We joined IEEE Standards Group P2733 to promote the Humanitarian interests in Standards development, and have engaged in writing a book chapter. We will maintain our network of international colleagues for added support and diversity of skills, understanding needs, and for the benefit of everyone’s experience.

Products on the market already include: expensive hearing aids $2K – 8K, monthly subscription caption system, expensive FM systems for public accommodation costing $18 – 20K per room. Virtual Reality systems are often used for psychotherapy, education, etc. In our ELSTS, we propose a unique combination of technologies: voice processed using acoustic equalizers with virtual reality scenarios so people can put on a headset and have the visual and auditory sensory experience of a hearing loss. The training can then step the individual through scenarios for example, driving a car and suddenly seeing an ambulance when you didn’t hear the siren, a student in a class who can only hear the teacher but not the peers or ambient sound, or trying to hear a bank clerk speaking from behind a glass partition, when all you can hear is the loud noise surrounding you.

Roadmap to Sustainability sharing SDG Goals especially 4, 5, 10, and 16 with examples – low cost, open source, apps, easy to use, networked, mainstream technology, simplified solutions, training, sharing solutions, etc.: Phase I Initial Scenarios Design and Development; Phase II Hardware and Content Development; Web Site Content Development; Phase III Pilot and Feedback; Phase IV Lessons Learnt and Shared.

Timeframe - Fall – Winter 2019: Pilot Experiential Learning Simulation and Training System with Captions 1) Kickoff Meeting – with Training on Sustainable Development Goals especially 4, 5, 10 and 16 and how they fit into the project, 2) Order Equipment, 3) Brainstorming with Students and Stakeholders – reminding of SDGs, disparity, special populations, and outcomes that can be transformative, 4) Configure Hardware and Software, 5) Design and Develop Scenario Storyboards, Content, 6) Plan for Development and Pilot.

Winter – Spring 2020: Demo Captions Create Scenarios 1) Intelligent Network Labs Demo of Captions, 2) Review Scenario Content, 3) Create Scenarios and Web Content for Pilot Simulation and Training System.

Spring 2020: Pilot Tests at Stevens Institute of Technology, Law & Mediation, and Nyack Mock Trial Team (Practice and Competition) 1) Test Pilot at Stevens, 2) Brainstorming with Students – reminding of SDGs, how low cost advancing technology can have a profound impact on humanity – especially in this case, when technology is important for hearing or understanding speech yet it is often too expensive or fails to help, for including women and girls and those with disabling conditions, for providing access to the Justice system, 3) Storytelling activities – Nyack Mock Trial Team and SIGHT Group Members, 4) Pilot Caption System at Mock Trial Practice.

Summer – Fall 2020: Pilot Tests Law & Mediation, Demo with NGOs, SociAvi, etc. 1) Pilot Experiential Learning Simulation and Training Pilot System, 2) Process feedback, 3) Write articles, lessons learnt, 4) Attend conferences, publish, present at meetings, share with other groups.

Fall – Winter 2020: Share design, development, instructions, and lessons learnt in Region 1, with SIGHT, and beyond. Ensure the message of low-cost solutions to a large humanitarian problem can make a greater impact on reducing suffering and disparity than expensive solutions that fail to help. We can use our engineering skills to improve human opportunity even in places with few resources.

The team structure includes: Organizers, who shepherd the project, organize meetings, write, report, etc. and who are the main architects and designers including but not limited to KG August, PhD, A Hauser, PhD, MJ Lyons, PE, D Nall, VB Lawrence, PhD, H Man, PhD, LTC K Kennedy-Pegues, PhD, YJ Chen, PhD, D Devasitharan, PhD, T Willis, III PhD, G Vanucci, PhD, S Wilkus, PhD, and others. International Collaborators engineering who provide input, research resources, feedback, etc. M Sellathurai, PhD, H Jelinek, PhD, AMJ Skolimowski, PhD, D Tulone, PhD, and others. Our Connections to the Justice System include but are not limited to: the Nyack Mock Trial Team, the Honorable Dr DR Lee, II Esquire, Rockland Bar Association, Law & Mediation PLLC, C Laredo, Esquire who has articulated the need for the ELSTS and Captions. We have developed a relationship with Rockland County Clerk’s Office (Honorable Paul Piperato who uses hearing aids) and who will provide access to his offices, and introduce us to management of County and State Courts. We will be able to present our Humanitarian Technology Pilot System ELSTS and Captions and obtain feedback and more information about how to best introduce accommodation systems in the Justice System. We have and will continue to attend Bar and Court functions, and advocate for Humanitarian issues in the Justice System. Beneficiaries include those in Mediation for ADA violations, court employees, attorneys, students, and those with mild to moderate hearing loss, in addition to our Members, Group, Stakeholders, etc.

Update from Dr Paula Muller of Sociavi: May 2020

In these unprecedented times, where communities are in quarantine and visitations are restricted, communication is crucial for aging adults and their families and electronic means are helping to bridge the distance and isolation.

1. Times are tough, but we are tougher... staying connected to families



An update from a recent IEEE NJ Coast Section Celebration when attendees became familiar with the goals of the Do Good Things Justice for All project:

An update from a recent IEEE NJ Coast Section Proposed IEEE Milestone Neutrodyne approved for placement at Stevens Institute of Technology. This Milestone Celebration would provide an excellent opportunity for demonstration of the Do Good Things Justice for All Pilot Captioning System:

An update from The Honorable Dr Duncan Rogers Lee, II Esquire:

  • The Nyack High School Mock Trial team collaboration with Dr. August’s team of IEEE engineers came to an early end this school year due to the SARS-Cov 2 virus. While the engineers were able to take some measurements during practices and during our Intramural trials, the season ended abruptly on March 9th and hence, all further efforts to take courtroom sound measurements this season. The good news was that the Nyack Team was unbeaten at that time.

A local tragedy, the death of Rockland County Clerk, Paul Pipperato, not only was a great personal loss to those of us who knew him, but also the loss of a great advocate for the IEEE efforts to bring access to justice closer for those with hearing loss. Mr. Pipperato, suffering from impaired hearing himself, was an advocate for the IEEE project.

Dr. Duncan Rogers Lee II, Attorney-Advisor, Nyack High School Mock Trial Team.

During the week of May 4, 2020, Jacobs ( highlighted accommodating deaf/hard of hearing team members in the workplace. Jacobs ACE (Access. Connect. Empower.) Network, a community within Jacobs of employees who are dedicated to connecting and empowering those with disabilities or physical or cognitive adaptive challenges, provided PowerPoint slides for a Culture of Caring moment at the start of meetings that week. The slide deck noted some global statistics familiar to NJ Coast SIGHT group: 1-6 people experience hearing loss; 41% of over 50s experience haring loss. The main slide reminded and encouraged people to utilize some basic courtesies at the start of meetings: Speak Slowly & calmly, provide video content with captions and transcripts, get the listeners’ attention before you start to speak, SMILE!

And here’s a picture of me in my WFH office

Be well & keep up the good work! It helps to have purpose and not be adrift in these times.

mjl Margaret J Lyons, PE