Date: Jun 28, 2019 Author: Nitish Padmanaban, Robert Konrad, Gordon Wetzstein Source: Science.org (
click here to go to the source)
Vision is one of the primary modes of interaction with which humans understand and navigate the everyday world. Unfortunately, the aging process is accompanied by a hardening of the eye's crystalline lens; the end result is that by their late 40s or 50s, most people struggle to view objects that are within arm's reach in sharp focus (1). This reduction in range of accommodation, known as presbyopia, affects more than a billion people (2) and will become more prevalent as the population ages (3).
While several types of eyeglasses and contacts exist to correct presbyopia (Fig. 1) (4), these traditional forms of correction fall short in one way or another. Bifocals, progressive addition lenses (progressives), and other multifocal lenses degrade depth perception and edge contrast sensitivity, increasing the risk of injuries from falls (5); As other senses, such as the vestibular and somatosensory systems, degrade with age, older individuals rely more heavily on vision (6). Furthermore, progressives can perform poorly when conducting tasks requiring side-to-side head movement (7) and suffer from astigmatism in the periphery. Single-vision glasses such as reading or computer glasses avoid these issues, but people often avoid them because of the inconvenience of carrying multiple pairs of glasses, or worse, forgetting the other pair. Last, monovision and simultaneous-vision contacts fall short when compared to bifocals and single-vision glasses on metrics such as visual acuity, stereoacuity, and near-distance task performance (8--14).
The common thread across these methods is that they use fixed focal elements to approximate vision that was once achieved by the flexible crystalline lenses in the wearer's eyes. This suggests that a more natural remedy for presbyopia would either restore the flexibility of the crystalline lens or use some form of focus-tunable lens element. Surgical approaches that aim to reduce the stiffness of the crystalline lens or replace it with an accommodating intraocular lens are active areas of research (15). However, in addition to the risk of undergoing an invasive procedure, these surgical methods are largely experimental and have yet to demonstrate long-term reliability (15).
Alternatively, one could also place the focus-tunable element outside the eye. An early implementation of focus-tunable lenses was the Alvarez lens, which shifts a complementary pair of cubic phase plates relative to each other to vary optical power (16). Since then, there have been many efforts to develop wide field-of-view focus-tunable optics for use in presbyopic correction. These proposals take various forms, including liquid and liquid-crystal lenses for use in eyeglasses (17--20) and contacts (21, 22). The thrust of much of the work on larger lenses often involves improving the optical quality, speed, field of view, weight, power consumption, and focal range of the lenses. Some have gone further and incorporated these focus-tunable lenses into an eyeglass form factor (23--26). While many of these require some form of manual control, Hasan et al. (23, 24) incorporate a single-pixel time-of-flight depth sensor to automatically update the lenses on the basis of what is directly in front of the wearer (a refocusing mechanism referred to from this point forward as "depth-tracked").
However, note that none of these focus-tunable corrections have been empirically evaluated or verified as outperforming traditional fixed-focus methods of correction when worn by presbyopes. Furthermore, none of these solutions, including that of Hasan et al. (23), truly capture the accommodation behavior that younger people are used to: simply looking around and having focus seamlessly adjust. While the depth sensor is an important step, it still requires that a wearer move their head, not their eyes, to fixate on objects. It also has functional disadvantages in situations involving (partially) transparent or moving objects, such as when looking through a window or reading a sign with people passing in front of it. A more natural solution is eye tracking, which has seen recent progress in size and power because of the needs of virtual reality (VR) systems. Current VR suffers from the vergence--accommodation conflict, a condition that, at its root, is caused by a fixed-focus distance, much like presbyopia, and has found a potential solution in focus-tunable lenses (27--33). Emerging VR displays have taken advantage of eye tracking to automatically adjust these lenses and update the virtual screen distance (34).
Our primary aim is to conduct an evaluation of focus-tunable eyeglasses as a method of correction for presbyopia. To this end, we designed and built a wearable prototype that incorporates electronically controlled liquid lenses; a wide field-of-view stereo depth camera; and, unlike previous approaches, binocular eye tracking. This "autofocal" system is capable of automatically adjusting the focal power of the liquid lenses based on input from the eye trackers (Fig. 1). However, even 0.5° of gaze direction error in each eye tracker is enough for perceivable changes in sharpness. Therefore, we designed a custom sensor fusion algorithm incorporating the depth camera; the depth serves as an extra stream of information to continually adjust for errors in the eye tracking.
Using our system, we find that focus-tunable eyeglasses, on average, outperform traditional forms of correction across a range of metrics. They are better than, or comparable to, progressives and monovision in terms of visual acuity and contrast sensitivity, depending on distance; users wearing autofocals can also complete a refocusing task both faster and more accurately. Users express a notable preference for our eye-tracked autofocals over a previously proposed depth-tracked solution, indicating that the technology chosen to adjust the lens power may have a substantial impact on eventual user acceptance of focus-tunable eyewear.
RESULTS
When comparing the performance of our autofocal prototype to traditional forms of correction, we focused on a few key metrics: visual acuity, contrast sensitivity, and refocusing task performance as well as general preference in natural viewing conditions. Note that experimental conditions, such as the focus-tunable lens control algorithm, varied slightly across metrics (see the Supplementary Materials for details). To determine the correct offset lenses to use, either users provided their current prescriptions or we measured them using a Grand Seiko WAM-5500 autorefractor, after which the spherical offset was further refined manually (see Materials and Methods).
Acuity
The first metric that we evaluated with autofocals is arguably the most important for vision: acuity. For this test, we split users into groups based on whether their usual correction was progressives (n = 14, ages 55 to 70, five females) or monovision (n = 5, ages 52 to 67, four females). Acuity was then tested using displays placed at distances of 0.167, 1.25, and 2.5 D (diopters) (6 m, 80 cm, and 40 cm), with procedures based on the ETDRS (Early Treatment Diabetic Retinopathy Study) chart (35), and the focus-tunable lenses set to the display distance for each trial (see Supplementary Materials and Methods).
The average acuity using each correction at each of the three distances can be seen in Fig. 2. From the figure, it is clear that autofocals are capable of maintaining high visual acuity at all tested distances, roughly one line (0.1 logMAR) better than 20/20. Users that wear progressives as their primary correction also have above 20/20 acuity on average but with a clear downward trend at closer distances. Monovision wearers generally have lower acuity than the other groups, especially at the nearest distance of 2.5 D for which their acuity is worse than 20/20 (0.076 logMAR).
We separately analyzed the two groups of users (progressives and monovision) with a two-by-three two-way repeated-measures analysis of variance (ANOVA), with independent variables of correction (their correction versus autofocals) and distance. Greenhouse-Geisser sphericity correction was applied. Post hoc tests were conducted as pairwise t tests only between the corrections at each distance (because of lack of interpretability of other comparisons), with Bonferroni correction applied to the P values (i.e., reported P values are adjusted by the Bonferroni correction factor).
For progressive lenses, the ANOVA shows a significant main effect of distance (F1.60,20.80 = 19.10, P < 0.001). There is also a significant interaction of correction and distance (F1.54,19.99 = 6.63, P < 0.01). Since the interaction is significant, we conducted follow-up t tests for the post hoc analysis, but no significant differences were found between the corrections at any specific distance.
For monovision, the ANOVA shows significant main effects of correction (F1,4 = 54.91, P < 0.01) and distance (F1.76,7.02 = 9.01, P < 0.05). There is also a significant interaction of correction and distance (F1.57,6.29 = 9.88, P < 0.05), so we conducted follow-up t tests. The autofocals show a significant improvement over monovision at distances of 1.25 and 2.5 D (P < 0.05).
Overall, it can be seen that autofocals perform significantly better than monovision, especially at intermediate and near distances, which matches the expectation of worse acuity when wearing monovision (14). Autofocals are comparable to progressives overall but with better performance at closer distance.
Progressives may perform worse at the closer distances because of either a difficulty in properly aligning the lenses to the right focus distance or a weaker near add prescription than necessary for the wearer's degree of presbyopia; however, the latter is an inherent disadvantage of fixed-focus lenses that cannot adapt to the wearer over time. Furthermore, note that there is a fundamental trade-off in the near add power for progressive and monovision corrections. Add powers for progressive lenses can be high, but fitting a greater range of powers in the same physical lens necessitates more precise head movements for intermediate distances. For monovision, higher add powers may decrease comfort and stereoacuity; many monovision wearers that entered our study reported also wearing reading glasses for near vision. An autofocal system has no physiological trade-off on the nearest focusing distance.
We see that autofocals also change acuity with distance, although without the same clear downward trend as the other corrections. There are two likely explanations as to why autofocals exhibit a distance-dependent change in acuity. First, the optics may have aberrations introduced with off-axis viewing as the wearer's eyes converge to near fixation distances. The second cause may be physiological. We assume zero remaining accommodation (i.e., completely presbyopic) when updating the lenses, but this overcorrects since most people still have some residual accommodative ability. This overcorrection may result in some degree of vergence--accommodation conflict at nearer distances, which is known to decrease acuity (36). Last, note that autofocals vary by less than half a line of acuity on average, whereas traditional forms of correction vary by more than a line of acuity over the target distances.
Contrast sensitivity
Contrast sensitivity is another important metric of visual quality, which we measured at a distance of 1 m using the Pelli-Robson contrast chart and procedures (37). This test was conducted on the same set of users as for acuity and with the lenses set to 1 D. Again, here, we expect that monovision will perform worse than the others (14). There is also a small chance that progressives' reduced performance at edge contrast (5) may also result in lowered contrast sensitivity when reading letters, although it is unlikely because the Pelli-Robson chart is intended for lower spatial frequencies, whereas edge contrast primarily affects high frequencies.
The average contrast sensitivities (Fig. 3, left) are relatively consistent across all corrections (exact averages are in the Supplementary Materials). We ran a paired t test for each correction, which shows no statistically significant differences at the 0.05 level.
On the basis of normal binocular values for the Pelli-Robson contrast chart (38), this result serves to verify that the focus-tunable lenses do not have any adverse effects on contrast sensitivity. While we do measure a small improvement in contrast sensitivity with autofocals, it is not significant. Unexpectedly, monovision does not show a larger decrease in contrast sensitivity, but this may be an effect of the sample size.
Task performance
A common challenge for presbyopia correction techniques is switching between different depths quickly. Progressives require that the wearer learn to focus differently, by moving their heads up and down while fixating on the target. This mechanism is slower, particularly in tasks requiring side-to-side head movement (7). In addition, monovision has also been measured as being slightly slower for near task performance (10).
Therefore, we should expect that autofocals outperform both corrections on a refocusing task measured via letter matching between a near and far distance (see Materials and Methods). The same users as above attempted the task performance test, with some exclusions (progressives: n = 14, ages 55 to 70, four females; monovision: n = 4, ages 52 to 67, three females). The lenses used eye-tracking data to switch between the distances of the two displays (details are in the Supplementary Materials).
The speed and accuracy of the users while wearing each correction can be found in Fig. 3 (center and right) (exact averages are in the Supplementary Materials). From the chart, it can be seen that users wearing autofocals are, on average, faster than with their own correction. Furthermore, in the case of progressives, they simultaneously improve their accuracy and attempt matches more quickly. We analyzed results with paired t tests for each correction and measured variable, revealing a statistically significant improvement in accuracy over their own correction for users of progressives wearing autofocals (P < 0.05).
The overall trend of these results is clear: Autofocals are capable of exceeding traditional forms of correction in speed and accuracy for tasks requiring refocusing. This also empirically demonstrates that focus-tunable lens switching speed is no longer the main limiting factor for use in presbyopia correction; future work on focus-tunable lenses can instead focus on other factors such as weight and power consumption.
Natural use preference
Last, we asked how effective autofocals are for natural viewing, where we focused on three points of interest: comfort, ease of use, and convenience. For "ease of use," we specified ease of refocusing to multiple distances. In addition, for the ease and convenience metrics, we also considered a depth-tracked mode. This mode mimics operation by a system that does not incorporate eye tracking, such as that of Hasan et al. (23), by using the median of the center 5 × 5 block of pixels. To capture a wider range of qualitative comparisons to presbyopia corrections used today, unlike the previous studies, we did not restrict the preference study to progressives and monovision (n = 37, ages 50 to 66, four females). Prescriptions were measured using the EyeNetra NETRA (39).
The rankings for the above metrics are given in Fig. 4, with black dots indicating every individual ranking. We see that our autofocal prototype is considered less comfortable (understandably since it is bulkier and heavier) even with a short period of wear. Some of those preferring our prototype for comfort cited less of a need to crane their necks back; with longer periods of wear, this improvement may likely be overshadowed by weight. On the other hand, autofocals are rated highest for both ease and convenience, with the depth-tracked mode faring poorly.
A Wilcoxon signed-rank test shows no significant difference on comfort. Friedman tests of the focusing and convenience are both statistically significant at the 0.001 level. For post hoc analysis, we used Wilcoxon signed-rank tests, with Bonferroni correction applied to the P values. These tests reveal that autofocals are easier to refocus than their own correction (P < 0.05) and the depth-tracked mode (P < 0.001). Furthermore, their own correction is rated easier to refocus than the depth-tracked mode (P < 0.05). Pairwise comparisons of the convenience ratings reveal that both autofocals and their own correction are rated as significantly better than the depth-tracked mode (P < 0.01).
With respect to the ease and convenience rankings, two trends bear further discussion: first, autofocals outranking the user's own correction and second, the lower than expected rankings for the depth-tracked version. Starting with the first, autofocals, on average, outrank users' own correction on both ease of refocusing and convenience. The ease of refocusing ranking seems to follow directly from the acuity and task performance results: Autofocals are simply better or faster at focusing to near distances. The convenience ranking, on the other hand, seems unusual, given the need for eye-tracking calibration; however, the inconvenience of calibration may be balanced by that of carrying around one or more pairs of reading glasses, which about one-third of these users did. Furthermore, eye-tracking calibration could benefit from being tailored to a single user's facial structure, as already performed today to determine the location of the progressive lens corridor. Viewed from this perspective, the convenience ranking can be seen as a combination of inconvenience of carrying multiple pairs of eyeglasses and optimism toward what commercially viable autofocal eyeglasses would be capable of doing.
Second, there is the question of why the depth-tracked mode is not more comparable to the eye-tracked autofocals, especially for convenience, since it requires no eye tracker calibration. Although there may be several reasons, the main cause is that the depth-tracked mode performs worse than the eye-tracked autofocals. The depth-tracked mode suffers from extreme jumps in focus when looking at a depth edge; the user study environment, consisting of both near and far objects, had several of these edges, and eye tracking is needed to disambiguate them. Furthermore, in a pilot study, we found that jitter and unpredictable lens updates, as in the depth-tracked mode, are among the least tolerable issues. While we use a stereo camera to determine depth, enhanced stereo depth image processing or, altogether, other methods of depth imaging may mitigate some of these artifacts, but likely with other modes of failure. A time-of-flight camera as in the prototype of Hasan et al. (24), for example, may struggle in bright ambient light or when looking at dark objects. Regardless of the depth sensing method, however, eye tracking inherently avoids any ambiguities at depth edges, and it is clear that an eye-tracked presbyopic correction is preferred by most presbyopes.
DISCUSSION
Presbyopia, a nearly universal problem in old age, still does not have an ideal solution. Emerging eye-tracking and focus-tunable lens technologies show promise and may enable corrective eyewear to approach the ideal of natural accommodation. To this end, we designed and built an autofocal system for evaluating the efficacy of gaze-contingent presbyopia correction. We show that our autofocal prototype often outperforms traditional forms of correction across several metrics, despite only being an early-stage implementation. In addition, users ranked eye-tracked autofocals as being superior to other forms of correction for ease of refocusing.
This preference for autofocal eyewear comes despite the fact that the refocusing action is still perceptible to the wearer, whereas a natural accommodation response may go unnoticed because of saccadic suppression. In our autofocal system, this delay is inherent to the control mechanism. Fixation distance can only be estimated after the wearer looks at an object; the solution, then, is to predict on what object their eyes will land (or better, the final vergence angle) before the end of the saccade (40).
As eye trackers and other constituent components of autofocals improve (41), autofocals stand to reap the benefits. For example, if focus-tunable lenses expand to support a large enough range, then both near- and farsightedness can be corrected in addition to presbyopia; any changes to the prescription can be updated within software, eliminating the need to buy new lenses as presbyopia increases. Furthermore, while our measurements verify that the optical quality, speed, size, and focal range of current focus-tunable lenses are sufficient for use in eyeglasses, they need to be made lighter, perhaps with diffractive liquid-crystal lenses (19). In addition, both lenses and eye trackers contribute to increased power consumption.
Traditional vision correction is passive, whereas both focus-tunable lenses and eye trackers require batteries. To optimize power consumption, focus-tunable lenses that only require power to change focus (e.g., mechanical shifts in Alvarez lens designs) may be better than lenses that require constant current input to maintain a nonzero lens power. Reducing the power required for eye tracking is an area of active research (42). As eye-tracking technology matures and becomes more accurate, the additional depth camera may become obsolete, further decreasing power consumption.
Despite power requirements and remaining engineering challenges, our study demonstrates that a paradigm shift toward digital eyeglasses is valuable, with the benefits extending beyond presbyopia correction. What seems at first like a disadvantage, the need for a battery, actually opens the door to more capabilities. Owing to the presence of a power source, other sensors not limited to depth sensing can be incorporated, such as a low-power inertial measurement unit (IMU). The IMU could be used to detect posture and call for help in case of a fall (43), or to track increases in postural instability to detect onset of any number of diseases, including Parkinson's disease. Autofocals' improved performance on our metrics already has important ramifications for improving real-world task performance and quality of life; with additional sensors, digital eyeglasses could become an advanced sensing and monitoring platform for health and well-being--one as easy to use as putting on eyeglasses in the morning.
MATERIALS AND METHODS
Hardware components
Our prototype (Fig. 5) is a tethered system built largely from commercially available components. The frame, modeled after a typical VR headset or ski goggles form factor, was three-dimensionally printed, with components mounted on top to accommodate our measurement devices and lenses. Two measurement devices were used to obtain depth information about the surroundings and gaze information from the wearer. A RealSense R200 (rated for 0.5 to 3.5 m indoors) supplied depth maps at 30 frames per second, using a pair of infrared stereo cameras with a structured illumination source. The gaze information was provided by eye trackers from Pupil Labs (44). There was one 120-Hz eye tracker for each eye, allowing us to also estimate vergence. The focus-tunable lenses were a pair of Optotune EL-30-45 liquid lenses, with a 30-mm aperture, which is comparable to the typical vertical size of eyeglasses. The field of view of the lenses depends on distance from the eyes; we measured this distance to be roughly 2 to 3 cm, corresponding to a field of view of 53° to 73°. The lenses' supported range of focus was measured to be −2.25 to 2 D, with a 100-ms settling time. Our implementation took about 15 ms to process gaze plus a few frames of smoothing, giving a total latency from eye movement to lens settling of about 150 ms. To allow our autofocal system to work in the 0- to 4-D range for a wide variety of wearers, we also fitted spherical and cylindrical offset lenses for each focus-tunable lens, with the exact lens power chosen on the basis of the wearer's prescription. Details on optical characterization can be found in the Supplementary Materials.
Fixation depth estimation
The depth of the fixated object is dynamically estimated via sensor fusion of four "raw" inputs: two gaze-tracking cameras, a scene-facing depth camera, and the wearer's interpupillary distance (IPD). The binocular eye tracker estimates the vergence distance at 120 Hz. Small errors in the gaze direction estimation, however, introduce a noticeable bias in the estimated vergence (see detailed analysis in the Supplementary Materials). Although the depth sensor only runs at 30 Hz, together with the gaze direction, it compensates for the bias in the vergence measurements. We developed a custom sensor fusion algorithm to balance the accuracy and speed of the vergence estimation pipeline, as detailed in the Supplementary Materials.
To evaluate autofocal technology, we conducted two user studies. The first study measured quantitative metrics: visual acuity, contrast sensitivity, and task performance. The second was a qualitative study of user preference during more natural use. Participants were verified as having 20/20 distance vision with corrective lenses. The study adhered to the tenets of the Declaration of Helsinki: Informed consent was obtained from all users, and all procedures were approved by the Stanford Institutional Review Board.
The first user study comprised four phases, repeated once with their correction and once with autofocals (correction order was alternated between participants): First, we measured their required prescription; second, we evaluated their visual acuity; third, we evaluated contrast sensitivity; and last, we evaluated their performance on a task requiring changing focus distance. Participants were asked to rest their chin on a chin rest to fix their distance to the monitors.
The second user study was conducted at a conference, with an optional preference questionnaire. The phases of this study were threefold: first, a prescription measurement; second, a few minutes spent using autofocals in the eye-tracked mode; and last, using autofocals in the depth-tracked mode. Users were not constrained to a chin rest and were instead free to look around their environment as they saw fit.
Prescription measurement
Prescriptions for the users were determined using a combination of automatic and manual measurements. For automatic measurement, the first study used the Grand Seiko WAM-5500 autorefractor to determine the necessary spherical and cylindrical corrections for each eye. The second study used the more portable EyeNetra NETRA (39) to do the same. Both devices output an IPD measurement as well, which is used during fixation depth estimation.
For the manual measurement, we started with the automatically measured values and inserted the corresponding offset lenses into the headset. Next, we set the focus-tunable lenses to +1 D of added spherical power. Then, as the wearer looked at a far chart, we proceeded to find a local maximum of perceived visual quality by updating the focus-tunable lenses in ±0.25 D increments. When this was complete, we switched out the spherical offset lenses with the new best-measured values.
Acuity
The acuity tests were administered using a logMAR (minimum angle of resolution) chart based on the ETDRS chart (35) at 99% contrast (display brightness: text, 1 cd/m2; background, 190 cd/m2). One line of five random letters (from the 10 Sloan letters) was shown at a time, with each subsequent line smaller by 0.1 logMAR. This continued until the user identified three or more letters incorrectly within the same line. The final reported acuity was the acuity of that line, minus 0.02 logMAR per letter answered incorrectly during that trial.
Contrast sensitivity
For the contrast test, we used the Pelli-Robson contrast chart (37), with the contrast sensitivity measurement corresponding to the lowest contrast line in which a majority of the letters are identified correctly. The contrast chart was placed at a distance of 1 m (1 D), with the illumination across the chart varying between 90 and 100 cd/m2.
Task performance
During the task performance test, we placed two displays in front of the user, one at 0.167 D and the other at 2.5 D, side by side at eye level. We displayed a single letter on each monitor, with 50% probability of them being the same letter. Letter size corresponded to one line (0.1 logMAR) larger than 20/20 acuity. The task was to indicate whether or not the letters matched using a keyboard. The users performed this task for 2 min, at the end of which we calculated their accuracy and speed.
Natural use preference
For the qualitative natural use questionnaire, we calibrated the users and let them freely view their surroundings in each mode: eye-tracked using our sensor fusion algorithm or depth-tracked to simulate automatic eyewear without eye tracking.
Acknowledgments
We would like to thank E. Wu, J. Griffin, and E. Peng for help with CAD, depth error characterization, and coma corrector fabrication; D. Lindell and J. Chang for helpful comments on an earlier draft of the manuscript; and A. Norcia for insightful discussions. Funding: N.P. was supported by the National Science Foundation (NSF) Graduate Research Fellowship Program. R.K. was supported by the NVIDIA Graduate Fellowship. G.W. was supported by an Okawa Research Grant and a Sloan Fellowship. Other funding for the project was provided by NSF (award numbers 1553333 and 1839974) and Intel. Author contributions: G.W. conceived the idea. N.P. and R.K. designed and performed the research. N.P. analyzed the data and wrote the manuscript with input from all authors. G.W. supervised the project. Competing interests: The authors declare that they have no competing interests. Data and materials availability: All data needed to evaluate the conclusions in the paper are present in the paper and/or the Supplementary Materials. Additional data related to this paper may be requested from the authors.
