BrainPort V100 by Wicab, Inc

What is it:

BrainPort V100 simply allows users to be able to “see with their tongue”. It is designed as a sensory substitution system for blind patients, as an oral-electronic vision aid. The surface of the tongue receives electro-tactile simulation that is translated from visual information, and this is able to aid blind patients with accessibility by assisting them in interpreting their environment. This device can also be used in conjunction with other assistive devices like dogs/ canes.

 

How it works:

A wearable video camera’s digital information is translated into gentle electrical simulation patterns that are on the tongue’s surface. It is described by users as feeling “bubble-like patterns on their tongue” as they learn to understand and interpret as the “shape, size, location and motion of objects in their environment”.

Speaking of learning, it is akin to picking up a new language – users have to undergo hours of supervised training to be able to use the device by themselves.

“Individuals need to complete a minimum of 10 hours of one-on-one training, over a three day period, with requirements and content customised for the individual user. The training sessions cover proper interpretation of sensory information and operation of the device controls.”

“It becomes a task of learning, no different than learning to ride a bike,” said a researcher. “The process is similar to how a baby learns to see. Things may be strange at first, but over time they become familiar.”

Taken from https://www.closingthegap.com/brainport-v100-oral-electronic-vision-aid/

Technical details:

1: An inch-long camera connected to the sunglasses sends the image it sees to the hand-held controller.

2: The controller converts the image into a low resolution black, white and grey picture. 

3: This picture is recreated onto a grid of 400 electrodes that are on the tongue array. Each electrode pulses according to how much light is in that area of the picture (black/grey/white intensity).

4. The user ‘feels the shape’ and detects the patterns on the surface of their tongue.

5. Through learning, the brain eventually learns to pick up and ‘see’ the shape detected on the tongue.

 

Pros: 

• A study conducted to access the functional performance of the BrainPort V100 deemed that with the device, blind patients can more successfully carry out tasks. This is because this device gives users more information about their environment than traditional assistive devices like the cane, or a guide-dog, and thus enhances independence in day to day life.
• Using a sense to replace/ act as a substitution for another sense seems like it would be fairly intuitive to pick up, and it does appear that once the skill-set is learnt, it can simple to use – like a second nature to the users.
• Also, there is a new and improved version of the BrainPort V100 – the BrainPort Vision Pro! 

Cons: 

• A very obvious con would be that if the patient has any oral conditions such as having numbness of their tongue/ experiencing lack of feeling in their tongue, this device would not work. Also, not much data is available on how the electrical simulations of the device would affect the tongue in the long run  (>1 year).
• It is also very costly, at around $10,000 USD to get one.

Review:

Sensory substitution is a very intriguing path to explore – being able to see with one’s tongue is something that reminds me of Synesthesia (a condition in which a sense, such as hearing, is simultaneously perceived by one or more senses such as sight). 

It also reminds me of a story, I think it was of a man who was able to taste shapes (or was it colour?). There is a book I remember reading years ago, and while I can’t quite recall the specific examples, it is very related to synesthesia – it is The Man Who Mistook His Wife for a Hat by Oliver Sacks.

Overall, I think that the key concept of replacing a sense with another (or even synesthesia) is one worth exploring for future projects/ or for fun!

References:

https://www.closingthegap.com/brainport-v100-oral-electronic-vision-aid/

https://www.researchgate.net/figure/The-BrainPort-Vision-Device-is-a-sensory-substitution-system-that-down-samples-video-from_fig1_262609186

https://www.researchgate.net/publication/317772172_The_Functional_Performance_of_the_BrainPort_V100_Device_in_Persons_who_Are_Profoundly_Blind

https://www.wicab.com/brainport-vision-pro