Virtual D&D and the Hearing Impaired.

[Milagroso]

Virtual DND and the Hearing Impaired.

Hello fellow gamers.

I have a group of friends that would like to play a virtual tabletop game using a client like Fantasy Grounds (but not necessarily that particular program) and using Ventrillo or Skype.

However, one of our friends has a hearing impairment. Among all of us, she is without a doubt one of the mose intelligent and creative people. Obviously, her hearing impairment would cause issues when we try to communicate and roleplay with telecommunication applications like Ventrillo. Typing all the information out in chat would take too long (especially for some of our hunt-and-peck type style gamers.)

I've been doing some research on broadcast speech-to-text applications that can convert my speech into text (she's a fast enough typer) but I haven't had much luck finding a cost efficient integration with tabletop applications.

Would anyone please help?
Has anyone had any experience with Virtual gaming and the hearing impaired?
Perhaps there are some hearing impaired members who can provide some advice and assistance?

I look forward to the responses from this great community.

 

[Jdvn1]

If you point the camera at you when you talk, she might be able to read your lips. But, this may not be a good option for you. No one there knows how to sign?

The only other thing I can think of is PbP gaming or Google Wave gaming.

 

[MadLordOfMilk]

Well, wouldn't the problem still exist in a face-to-face game? If not, what is causing it to become a problem? If so, what can you do to work around it?

Unfortunately, I can't really think of a lot of work-arounds. Most people can't read lips (in my experience), so that's probably out. Signing makes potential sense as a solution, but that requires both your friend and someone else to know sign language, which is also unlikely.

Speech-to-text is the most obvious solution, but actually finding something that can do that might be tough. Perhaps you could try a speech recognition program, and just enter it into Google Documents (or something similar), which will update live on their screen.

 

[Jdvn1]

Well, wouldn't the problem still exist in a face-to-face game? If not, what is causing it to become a problem? If so, what can you do to work around it?

Unfortunately, I can't really think of a lot of work-arounds. Most people can't read lips (in my experience), so that's probably out. Signing makes potential sense as a solution, but that requires both your friend and someone else to know sign language, which is also unlikely.

Speech-to-text is the most obvious solution, but actually finding something that can do that might be tough. Perhaps you could try a speech recognition program, and just enter it into Google Documents (or something similar), which will update live on their screen.

If I may answer these questions from my own experience...

The problem isn't quite the same in a face-to-face game. In a face-to-face game, sound is much more clear than through computer speakers, and there are visual aids and reading lips. A hearing impaired person has difficulty understanding what's going on in a group situation, but it's an easily surmountable problem if the members of the group are sensitive to the problem. They can be sure to face the hearing impaired person, enunciate their words, and avoid covering their mouths. Over the internet, seeing someone's lips is difficult.

And, many hearing impaired people can read lips very well. They often miss a lot of what's being said, but they get enough to piece together the parts using what little they hear and visual cues, and minimal clarification is required.

Having been around hearing impaired people for much of my life, I must say it's been incentive to learn at least some sign language. Enough to carry a basic conversation, anyways. If one person in the group was also incented to do so, he could serve as a translator.

Google has decent speech recognition software, but unfortunately hasn't provide on-the-fly translation services for general use.

 

[ppaladin123]

If you point the camera at you when you talk, she might be able to read your lips. But, this may not be a good option for you. No one there knows how to sign?

The only other thing I can think of is PbP gaming or Google Wave gaming.

My guess is that the frame rate for the typical webcam is too low to make lip reading feasible.

 

[Jdvn1]

My guess is that the frame rate for the typical webcam is too low to make lip reading feasible.

Yeah, I would guess that too.

However, there are video-phone options (that deaf people can get for free in many states) to alleviate this problem. I don't know about its compatibility with Skype and online video conferencing solutions.

 

[heruca]

It might be possible for me to add text-to-speech functionality to my Battlegrounds virtual tabletop software, but I guess what you need is really speech-to-text functionality, which is way beyond my ability and expertise.

I know there is software like Dragon NaturallySpeaking that can handle the speech-to-text conversion, but I have no idea if those programs only work in/with specific software (like MS Word), or if they can be used with any text entry field, such as the chat window in a virtual tabletop program. A typical RPG also uses a lot of names and words that might not be recognized and properly interpreted by the software.

I may still see about adding in text-to-speech functions to Battlegrounds, since it would at least help the visually impaired.

 

[MadLordOfMilk]

I know there is software like Dragon NaturallySpeaking that can handle the speech-to-text conversion, but I have no idea if those programs only work in/with specific software (like MS Word), or if they can be used with any text entry field, such as the chat window in a virtual tabletop program. A typical RPG also uses a lot of names and words that might not be recognized and properly interpreted by the software.

I've messed with them before. I don't remember which one I used, but you could import it into any program. There was a delay, but I wouldn't say it was much different than the delay of live captioning. The accuracy was decent - at the very least least, decent enough to add enough context to figure out what was trying to be said IMO. You could also add your own customized words and "train" it to understand them, so over time that issue could be fixed.

Putting that into a Google Doc would allow streaming updates without having to constantly hit "enter," but it also requires leaving that window open and focused on a computer... d'oh!

 

[Starfox]

Is she hearing impaired or deaf? Does she use sign language? Do you? Deaf people use vidcams to communicate using sign language, something that works very well even over cam-enabled mobile phones.

Another option is to use text instead of sound when playing. It takes a bit of getting used to for all, but it works.

<rant>
As an aside - I realize that this might be completely irrelevant to your case as you never said your friend was deaf - the issue of deaf vs. hearing impaired is sensitive for me. I have a brother who is deaf. To call him hearing-impaired does NOT help him. He cannot hear AT ALL and never could. The schooling system here tried to "salvage" his hearing by providing hearing aids and teaching him to talk - a completely wasted effort. At the same time, they denied him education in sign language. This made his handicap that much worse. (This was in Sweden in the 60's and 70's).

What I want to say is that its not a mercy to call a deaf person hearing impaired, as the two work out quite different socially and a deaf person (particularly one born deaf and thus lacking a spoken language) has entirely different problems and requirements from someone who is hearing impaired.
</rant>

That said, this is off-topic. Lets not make a discussion of deafness here, I just had to get the rant off my heart as its still painful. And again, its not directed at the OP or anyone else here.

 

[Jdvn1]

<rant>
As an aside - I realize that this might be completely irrelevant to your case as you never said your friend was deaf - the issue of deaf vs. hearing impaired is sensitive for me. I have a brother who is deaf. To call him hearing-impaired does NOT help him. He cannot hear AT ALL and never could. The schooling system here tried to "salvage" his hearing by providing hearing aids and teaching him to talk - a completely wasted effort. At the same time, they denied him education in sign language. This made his handicap that much worse. (This was in Sweden in the 60's and 70's).

What I want to say is that its not a mercy to call a deaf person hearing impaired, as the two work out quite different socially and a deaf person (particularly one born deaf and thus lacking a spoken language) has entirely different problems and requirements from someone who is hearing impaired.
</rant>

I just wanted to point out that this might be a cultural difference. In the United States, hearing-impaired people are very likely to refer to themselves as deaf, and hearing-impaired people are often a part of "deaf culture" in the United States. It's often on a spectrum, anyways--my brother was "profoundly" deaf, which was a legal designation to describe the most severe level of deafness.

Also, services and treatment of deaf/hearing-impaired/hard-of-hearing people nowadays is significantly different from back in the 60s.

Nonetheless, I want to apologize for unintentional offense.

That's it for the side topic for me, I think.

I think using text is a good idea--instead of discs or bands to designate markers, you could have a little sign on the table that says "Immobilized" or something. At the very least, it'll likely be easier to see in the video.