Medicine and the metaverse: New tech allows doctors to travel inside your body

The world of technology is rapidly shifting from flat media viewed in the third person to immersive media experienced in the first person. Recently dubbed “the metaverse,” this major transition in mainstream computing has ignited a new wave of excitement over the core technologies of virtual and augmented reality. But there is a third technology area known as telepresence that is often overlooked but will become an important part of the metaverse.

While virtual reality brings users into simulated worlds, telepresence (also called telerobotics) uses remote robots to bring users to distant places, giving them the ability to look around and perform complex tasks. This concept goes back to science fiction of the 1940s and a seminal short story by Robert A. Heinlein entitled Waldo. If we combine that concept with another classic sci-fi tale, Fantastic Voyage (1966), we can imagine tiny robotic vessels that go inside the body and swim around under the control of doctors who diagnose patients from the inside, and even perform surgical tasks. .

I know that sounds like pure fiction, but a startup company in Hayward California has recently “flown” a tiny robot inside the digestive track of human subjects. The company is Endiatx, and I had a chance to discuss their technology and vision with CEO Torrey Smith. As a technologist who has been involved in telepresence research from the early days, I was impressed with the progress Endiatx has made. But before I get into that, let’s jump back in time a few decades and provide some context as to why their breakthrough strikes me as such an unexpected advancement.

The first telerobotic prototypes

My first experience with an immersive telerobotics system was over thirty years ago. I can still remember the first time I peered into a headset, grabbed a set of periscope-like handles, and looked around a room that wasn’t the one I was sitting in. It was 1991 and I was working in a lab at NASA that had some of the first prototypes of a mobile telerobotic system. It was developed by Fake Space Labs and Telepresence Research and it allowed you to control a mobile robot with a camera system that sent back stereoscopic images in real time. Below is a photo of that early system from an academic paper on those efforts, showing the state of the art in telepresence research thirty years ago.

That early system was extremely impressive at the time and was developed by some of the top researchers in the field back then. But if you wanted to bring the hardware to a trade show, you’d probably need a good-sized van or maybe even a U-Haul truck. The idea of ​​shrinking a system like that down to a size that a person could swallow and be controlled by a doctor was beyond conception. So what were the proposed application areas during the early days of telepresence?

Early on, the focus was to enable human operators to perform work in hazardous places, for example, to clean up nuclear accidents, repair satellites, and even to fix leaking oil wells on the ocean floor. My personal focus in the early 1990s was adding augmented reality and haptic feedback to the field of telepresence with the goal of enhancing operator performance. This was cutting-edge research back then, but I still never considered the notion of shrinking the technology to such a size that it could be “flown” inside the human body as a means of diagnosing and treating patients from the inside.

‘Fantastic Voyage’ realized

For these reasons, I was surprised to learn about the ambitious vision and recent technical success of the team at Endiatx. Founded in 2019, they have already created a tiny robotic drone that can be swallowed by human patients and piloted remotely inside the stomach and other parts of the digestive tract. Known as the PillBot™, their prototype system is basically a tiny remote-controlled submarine the size of a multivitamin that sends real-time video back to a doctor’s computer or phone. And it works – they’ve run tests on cadavers and live humans.

When chatting with CEO Torrey Smith, he told me he was inspired as a kid by the sci-fi movie Innerspace (1987) and has been thinking about that concept ever since. A few years ago he finally took the plunge, founding Endiatx to bring this concept to life. And so far, it’s going well. Not only is his company on its way to bringing this capability to healthcare as a shipping product, but Torrey himself was the first person on the planet to have a robotic drone flown around inside his stomach. He volunteered, swallowing the first prototype that made a reality fantastic voyage.

Since that initial test, others at the company have swallowed working prototypes many times, capturing live video of the type that will one day be used to screen patients for ulcers, gastritis, cancers, and other potential ailments. And that day is not far off – the team is currently conducting cadaver tests with doctors at the Mayo Clinic and is planning trials for FDA approval. If all goes smoothly, the PillBot could be diagnosing patients around the world by 2024.

This could be a huge benefit for individuals who go to the doctor with stomach pain. Instead of having a standard endoscopy procedure, which generally requires sedation and involves multiple visits, the tiny swallowable robot could save time, money, and complexity, giving doctors a quick and easy way to look around inside their patient. And it may provide more flexible control than a traditional endoscope, as the untethered Pillbot has the full 3D mobility of a tiny robotic submarine. It even looks like a tiny sub – with micromotors and tiny propellers along with a video camera, battery, and wireless link for feeding images back to doctors in real-time.

Faster, cheaper, more accurate screenings

Currently, doctors can fly the tiny drone using a standard Xbox gaming controller, but the company plans to enable control using the touchscreen of any mobile phone. That’s because their vision is for a disposable unit that is shipped to your home and swallowed during a telemedicine consultation with your doctor, who reviews the camera feed in real-time from their PC or phone. Endiatx believes the robotic pills can be manufactured for $25 each, sold for hundreds of dollars per unit, and save many thousands of dollars in medical expenses that endoscopic procedures cost to perform under sedation. But more importantly, the company believes that PillBots will save many lives by enabling faster and cheaper screening that finds serious conditions sooner than would otherwise be practical.

The company expects to begin the first clinical trials later this year, roll out an in-clinic version soon after, and launch an at-home version not long after that. And once they have shipping products that enable doctors to look around inside the body, their next objective is to enable the device to take tissue samples and perform other surgical tasks. Long term, their plan is to shrink their robotic drone down to the size of a grain of rice, opening up capabilities beyond the digestive tract. And their goal is to provide all of these capabilities for in-home use, with doctors controlling bots by telemedicine.

The future of telemedicine?

I was a little skeptical about the telemedicine angle for a tiny robot like this, as we’re talking about a powerful diagnostic tool delivered in the mail. Will the medical profession embrace such a change, or will they insist on keeping such capabilities in clinics and hospitals? That’s when I saw the press release this week from Amazon announcing the acquisition of healthcare provider One Medical for almost $4 billion and stating that Amazon wants to reinvent the healthcare experience.

Neil Lindsay, SVP of Amazon Health Services is quoted in the release: “Booking an appointment, waiting weeks or even months to be seen, taking time off work, driving to a clinic, finding a parking spot, waiting in the waiting room then the exam room for what is too often a rushed few minutes with a doctor, then making another trip to a pharmacy – we see lots of opportunity to both improve the quality of the experience and give people back valuable time in their days.”

In that context, the sci-fi vision of tiny drones that are swallowed by patients and controlled remotely by doctors during telemedicine visits may really become part of our mainstream medical experience. As a lifelong technologist who often writes about the looming dangers of the metaverse, this is one emerging application area that genuinely impresses me.

Author Bio: Louis Rosenberg, PhD is a pioneer in the fields of virtual reality, augmented reality, and artificial intelligence. Thirty years ago he developed the first functional mixed reality system at the Air Force Research Laboratory. He then founded the early virtual reality company Immersion Corporation (1993) and the early augmented reality company Outland Research (2004). He’s currently the CEO of Unanimous AI, a company that amplifies the intelligence of human groups. Rosenberg earned his PhD from Stanford University, was a professor at California State University and has been awarded over 300 patents for his work in VR, AR, and AI.