News Article

Blind Runner's Wearable Technology Gets Off to Complicated Start
Date: Nov 05, 2017
Author: Jeré Longman
Source: New York Times ( click here to go to the source)

Featured firm in this article: WearWorks Inc of New York, NY

As Simon Wheatcroft walked onto the Verrazano-Narrows Bridge to start the New York City Marathon, he could sense other runners moving around him but could see them only as one smudged shape.

His vision had severely deteriorated by age 17 from a rare genetic disorder called retinitis pigmentosa. As a teenager, he developed night blindness. His peripheral vision eroded. For a while, he could still read and recognize faces. Now 35, he could detect little more than changes in light and darkness. Everything appeared as a snowy grayness.

He was an experienced marathon runner, but each training session brought uncertainty. While training unassisted in the summer of 2016, Wheatcroft rammed into a burned-out car on his training path in Doncaster, England, three hours north of London. He needed stitches in his arm and wondered whether it was worth it to keep running.

Even now, Wheatcroft remained torn. He had a wife and two young sons. A wrong step on a training run could lead to severe injury or worse. But he persevered. In a marathon, he felt the same as any other runner. And, as he chased the finish line, he also pursued technology that could help the blind and visually impaired lead more independent lives.

"I love a long shot," said Wheatcroft, a motivational speaker who is pursuing a master's degree in computer science. "It's nice to really push against the limits of possibility."

At the blast of a cannon, he set off on Sunday with more than 60 visually impaired marathoners to run New York for the third time. Some runners were tethered to their guides. Others ran as Wheatcroft had previously -- shoulder to shoulder with their escorts, relying on touch and oral cues to remain safe. This time, however, he tried something bolder and riskier, attempting to navigate the course with only minimal assistance from three runners who shadowed him.

He did so with the technology of so-called corrective navigation. Race officials said Wheatcroft appeared to be the first runner to use it in New York to negotiate the course's 26.2 miles, including five bridges, tricky turns, the rolling hills of Central Park and, most challenging, the jockeying of 50,000 other participants.

If the race went according to plan, the friends who accompanied Wheatcroft would intervene only as a last resort to prevent him from colliding with another runner.

"What we really don't want to do is take someone else out of the race," he said.

The company's first product is a wristband -- adapted as an armband for Wheatcroft -- called a Wayband. It connects via Bluetooth with a smartphone and uses information from Google Maps, OpenStreetMap and proprietary technology to guide wearers to their destination by emitting patterns of vibrations instead of voice commands.

Pulses from the armband were designed to keep Wheatcroft running on Sunday within a virtual corridor, about 20 feet wide, and to help him turn right and left. Four short, rapid vibrations signaled that a left turn was ahead. Two longer vibrations signaled a right turn.

He also wore an ultrasonic sensor on his chest. Two sharp vibrations alerted him to runners crossing his path. No vibrations meant he was free of obstacles. Gentle pulses suggested he was securely cocooned in a pack of runners moving at roughly the same speed.

Also attached to a strap on his chest was an iPhone. On his right arm was a separate GPS device to provide more accurate positioning on the course and to save battery life on the cellphone.

Company officials and Wheatcroft, who has an equity share in WearWorks, say that tactile feedback is less intrusive and more intuitive than oral cues: Vibrations are not vulnerable to being drowned out by loud noises. The visually impaired are freed to use their sense of hearing to listen for approaching vehicles and pedestrians and to more fully engage in their surroundings.

The Wayband technology is not unlike that used by cars to avoid collisions and to park safely, except the sensors employ vibrations instead of beeps.

"It's 26 miles of reverse parking," Wheatcroft said with a laugh. "There might be a few dings in the bumper at the end. As long as there's only paint damage, we'll be O.K."

In preparation for the marathon, Wheatcroft tested the wearable technology on a walk through Harlem.CreditSam Hodgson for The New York Times
Even for sighted runners in New York, the marathon course is a challenge beyond sheer mileage. It requires running in thick bunches and maneuvering carefully to the side of the road at water stations, where the pavement is wet and littered with paper cups.

Wheatcroft was using new technology that had not been tested in a race. He understood that many things could go wrong. The metal girders of bridges along the course scrambled the digital compass on his iPhone. He worried about other possible navigational glitches caused by Bluetooth interference.

What if someone stopped in front of him to take a selfie? How would he refill his water bottle? Could he remain on course as the race turned off Fifth Avenue and funneled into Central Park at Mile 24, curling around the reservoir, which had seemed to befuddle his GPS on some test runs?

He had used technology, such as Runkeeper, an app that gave his pace and distance with voice commands. But corrective navigation for visually impaired runners was in its infancy. As Sunday's race approached, Wheatcroft described himself as excited, nervous, a little fearful.

"It's a complicated course, there's a lot of people, you can't afford to make mistakes," he said. "Reaction times need to be sharp."
His biggest concern, he said, was correctly interpreting the patterns of vibrations on his arm and chest. If he became confused, he planned to do what marathon runners often did in times of stress: slow down.

Wheatcroft was a seasoned distance runner, having completed the Boston Marathon three times and ultramarathons as long as 83 miles. In 2014, he ran from Boston to New York as a warm-up, then ran the New York City Marathon. He hoped to finish in four and a half hours on Sunday, more than 40 minutes faster than his previous best in New York.

Wheatcroft and Kevin Yoo, one of three founders of WearWorks, had been testing the Wayband technology since April. As with any prototypes, there were advances and setbacks. On Friday afternoon in Central Park, and again Saturday, last-minute refining continued with the chest sensor.

A motivational speaker who is pursuing a master's degree in computer science, Wheatcroft, 35, picked up his race bib.CreditSam Hodgson for The New York Times
Keith Kirkland, another founder of WearWorks, joked about running along the course during the race and calling out, "Anyone have a soldering iron?"

On Sunday, Yoo started his first marathon, hoping to accompany Wheatcroft for as long as he could. The Wayband device was turned off for the first two miles of the race on the ascent and descent of the Verrazano-Narrows Bridge. Too many runners packed together. Too much risk of technological overload.

"Could we ask everyone to be considerate and please not use their cellphones?" Yoo joked before the race.

Crosswinds blowing over the chest sensor gave confusing signals to Wheatcroft on the bridge, but the roadway was wide and he had plenty of open space to run. Neil Bacon, a friend who has often accompanied Wheatcroft, reminded him not to get too exuberant.

"You're doing 26 miles," Bacon said. "Don't go off like it's a half-marathon."

At the first water stop, about two and a half miles into the race, a guide for another runner stopped in front of Wheatcroft. His chest sensor was set to alert him when an obstacle was seven feet away. He did not have enough time to stop and clipped the woman from behind, but neither was hurt. At the second water stop, he slowed and moved to the center of the road.

Just after Mile 3, the Wayband device signaled incorrectly that Wheatcroft was headed in the wrong direction. He stopped and walked for a minute, then renewed his pace. Tall and thin, wearing a white cap on his shaved head, he had run four miles without any assistance.

Taking in the aromatic scents of late-morning cooking in Bay Ridge, Brooklyn, Wheatcroft said, "I could run this thing on smell."

But he would have to be prepared if his digital compass went haywire later in the race -- while on the Pulaski Bridge at the halfway point, the Queensboro Bridge between Miles 15 and 16 or the Willis Avenue Bridge at Mile 20 -- and the vibrations from his armband signaled incorrectly that he was veering off course.
Wheatcroft on Staten Island before the race on Sunday. He has an equity share in WearWorks, the two-year-old Brooklyn technology company that developed the Wayband.

Wheatcroft on Staten Island before the race on Sunday. He has an equity share in WearWorks, the two-year-old Brooklyn technology company that developed the Wayband.CreditSam Hodgson for The New York Times
"Simon needs to trust himself and utilize his sense of hearing and just go straight," Yoo said.

Race officials would have preferred that Wheatcroft first experiment with the Wayband technology in a smaller, shorter race. A blind runner, Thomas Panek, had recently run a race assisted only by his guide dog, a yellow Lab named Gus, but the race was five miles through Central Park, not a marathon.

Wheatcroft argued that the New York City Marathon was the perfect place to test corrective navigation in an urban environment. It simulated the way people typically moved through cities, on the street, through crowds.

Given that Wheatcroft was running with two guides who had accompanied him a number of times -- Bacon and Andrea Croak, an American -- he received the go-ahead from New York Road Runners, which organizes the marathon. Achilles International, which places disabled runners in mainstream races, also approved.

"Anytime you can advance technology and bring it to life for people to be able to enhance themselves, it's all good," said Peter Ciaccia, the marathon's race director.

Technology to assist visually impaired runners had advanced considerably since Marla Runyan of Santa Maria, Calif., the first legally blind athlete to compete in the Olympics, finished fourth at the 2002 New York City Marathon. In that race, a bicyclist was designated to call out her time at each mile and alert her to water stops and hazards on the course.

Later, Runyan said, Nike made a prototype watch for her with a huge display in which each digit was an inch to an inch and a half tall. But the device did not prove reliable enough to wear in a race, she said.

In April, Erich Manser of Littleton, Mass., ran the Boston Marathon using a service for the blind and visually impaired called Aira. Manser wore smart glasses, which streamed video of the course to a guide in Columbus, Ohio, who assisted him with short voice commands such as "runner passing on the right," "you're on the yellow line," "clear path ahead" and "cups" for the litter at water stops.

Runyan, who coordinates athletes with disabilities for the Boston Athletic Association, which organizes the marathon there, said, "I think we're in the early stages of what's going to eventually be possible," though she predicted that "it's never going to be a one-size-fits-all solution."

Wheatcroft had begun running in 2010. Rock climbing proved an insurmountable task with severely limited sight. It was a difficult time in his life. Mobility was an issue. He seemed to be stuck too often indoors.

"Running was a chance to get out there and do something," he said.

His first path was a soccer field near his home in north-central England, where the footing was constant and dog-walkers sometimes grew impatient, wrongly assuming that he would see them and move out of the way.

Eventually, Wheatcroft found a three-mile path near the Doncaster-Sheffield Airport that was largely devoid of foot traffic. By trial and error, he memorized the route, learning to avoid signs and posts. Sometimes he found himself crying at the effort and determination needed to keep running day after day.

He was not prepared for the unseen car that had caught fire and been abandoned on his route in August 2016. He sliced his right arm. His wife urged him to quit running. What if he had broken his leg or severed an artery? He started to obsess that each time she dropped him off for training "it would be the end."

One wrong step and he might be hit by a moving car.

"It kept me up at night," Wheatcroft said.

After a month or so, while delivering a speech in Bangalore, India, Wheatcroft joined a group on a run through the streets "to see if I still had the daring to do it."

As a father of sons ages 7 and 4, he continues to have doubts about what he is doing. ("It starts to take its toll; it's dangerous.") But he is buoyed by the notion that technology that permits him to run could have broader implications in allowing the visually impaired to live their daily lives more freely.

"If we move it forward by me perhaps being a little too risky, it's all for the greater good," Wheatcroft said.

He laughed.

"Someone needs to be the first person stupid enough to do it."

For about 13 to 15 miles on Sunday, Wheatcroft ran mostly unassisted. But things began to go wrong. The digital compass in his iPhone and the redundant one he wore on his arm malfunctioned. His pace slowed to 13 minutes per mile from just over 10 minutes. He struggled to navigate, had two additional collisions and sometimes was essentially shouldered around curves by other runners. At other times, he relied on lines on the road that he could feel with his feet.

The extraordinary concentration required to guide himself "broke me," Wheatcroft said.

At about Mile 16, the ultrasonic sensor also failed, apparently because of the rain. "We pretty much hit every issue we could potentially hit," Yoo, the WearWorks founder, said.

For the final 10 miles, Wheatcroft ran as he had in previous marathons, with Bacon and Croak alongside, advising him of turns, potholes, curbs, water stops. He appeared exhausted when he crossed the finish line in 5 hours 17 minutes 40 seconds and put an arm around Bacon's shoulder for support.

"Awful," Wheatcroft said when asked how he felt.

A few minutes later, though, he seemed somewhat consoled by the effort.

"Today was always about pushing the technology to its limit," Wheatcroft said. "We found the limit earlier in the race than we would have liked. But it was lessons learned. We can improve, move forward, make it better. It's not the end, it's just a start."