Sharing Project Amber with the mental health community

New open source resources to help researchers collect and interpret electroencephalography (EEG) data for mental health measurement

Today at the Sapien Labs Symposium, my colleague Vlad Miskovic presented insights from Project Amber, an early stage mental health project at X. Amber’s small team of of neuroscientists, hardware and software engineers, machine learning researchers and med-tech product experts have been developing prototype technologies to help tackle the huge and growing problem of mental health. After three years of exploration, we recently wrapped up our work at X. Now we are making our technology and research findings freely available in the hope that the mental health community can build upon our work.

Poor mental health is a huge and growing problem globally. The World Health Organization estimated in 2017 that 322M people globally suffer from depression and 264M from anxiety. The COVID-19 pandemic is causing widespread psychological distress, affecting even more people.

One of the challenges is that it is truly difficult to assess mental health, both for people who are distressed and for health care providers who are not experts in mental health. With 1000 possible symptom combinations, depression manifests differently in different people. Today’s assessment of mental health mostly relies on asking people a series of questions in a conversation with a clinician or via surveys such as the PHQ-9 or GAD-7, which are subjective. While it is important to capture the subjective experience of a person living with mental health problems, the field is missing objective measures that are commonplace in other areas of health. For example, people with diabetes and their doctors routinely measure blood glucose and use these data to make adjustments to insulin, diet and exercise regimes — but there is no equivalent for depression or anxiety.

Amber’s moonshot: Finding a biomarker for depression

Our journey started by asking the question: what if we could make brain waves as easy to measure and interpret as blood glucose, and use them as an objective measurement of depression? Our approach was to marry cutting-edge machine learning techniques with a 96-year-old technology to measure electrical activity in the brain: electroencephalography (EEG). 

We were inspired by neuroscience studies showing that certain patterns of electrical activity in the brain correspond with depression symptoms. For example, many depressed people find that things that once brought them pleasure no longer do so; they don’t experience the reward that follows a positive experience. By designing specific game-like tasks that people complete while their brain activity is being measured using EEG, scientists can gauge processing within the brain’s reward system. It turns out that the brain response following a win in the game — an event related potential (ERP) — is subdued in people who are depressed, compared to those who are not.

Image for post
Difference in reward response among depressed and non-depressed people
(unpublished data from Amber feasibility study with Florida State University)

This blunted brain response is a reliable effect that has been shown in many studies, which we replicated in our own study carried out in partnership with Greg Hajcak and his team at Florida State University.

However, these studies were done in neuroscience research labs. They require expensive specialist equipment and highly trained EEG experts to collect, process and interpret the data. For EEG to come out of the lab and into the real world as a mental health assessment tool in a primary care doctor’s office, counseling centre or psychiatric clinic, it needs to become more accessible and usable at scale.

Our project at X focused on three areas:
1) Making EEG data easier to collect
2) Making EEG data easier to interpret
3) Understanding how this technology might be applied in the real world

The rest of this post lays out our work and insights in each of these areas.

Making EEG data easier to collect: The Amber EEG system

Our team set out to develop an easy-to-use, low-cost, portable, research-grade EEG system.

Image for post
Hardware engineer Gabriella Levine (left), neuroscientist Sarah Laszlo (right) testing early Amber prototypes

We built many prototypes of bioamplifiers, headsets and sensors, and tested them in feasibility studies at X and at Florida State University. 

Image for post
A selection of the Amber EEG prototypes

In our final prototype the headset slips on like a swim cap and can be put on by anyone with minimal training, taking around three minutes to set up. It uses three dry sensors arranged along the midline at Fz, Cz, Pz, the most important channels for ERP assessments of reward and cognitive function. The accompanying bioamp can support up to 32 channels, so it’s possible to connect a standard headset with some modifications. Amber’s system can be used to collect resting state EEG and event-related potentials with our software that time-locks a task to the EEG measurement.

Image for post
Amber’s final EEG prototype: Headset, sensor strip and bioamp

Making EEG data easier to interpret: Applying machine learning techniques to EEG signal

Our team also explored how new approaches in machine learning can be applied to interpreting EEG data. To make EEG data usable by mental health researchers and clinicians more broadly — i.e. outside electrophysiology labs and neurology clinics — it would be helpful to have automated ways to denoise the signals at scale, and to determine which aspects of the EEG signal are relevant. Collaborating with the team at DeepMind, we adapted methods from unsupervised representation learning to address these challenges. We set out our findings in a paper that is currently under review.

First, we demonstrated that representation learning approaches such as autoencoders could be leveraged to effectively denoise EEG signals without a human EEG expert in the loop. This is important to enable processing EEG data at scale. Second, we offer a proof of concept that it’s possible to extract interpretable features that are relevant to mental health. We used these features obtained from disentangling autoencoders to predict several clinical labels such as major depressive disorder and generalized anxiety disorder, based on a clinical interview by a mental health expert. Unlike previous studies, we were able to do this for an individual participant (rather than a group), which is essential to make it useful in a clinical setting. The methods were capable of recovering usable signal representations from single EEG trials. This means that it may be possible to derive clinically useful information from brain electrophysiology with far fewer data samples than what is traditionally used in research labs, which often rely on hundreds of experimental trials.

Understanding how this technology might be applied in the real world: Insights from user research

Over the course of our project, we conducted over 250 interviews with potential users of this technology. We spoke to people with lived experience of mental health problems and with clinicians of all kinds, including counsellors, therapists, psychiatrists, clinical psychologists, social workers, primary care practitioners and pediatricians. We tested how Amber’s proposition of introducing a new, more objective measure of depression and anxiety resonated with them, how they might use it in their daily lives and professional practice, and what the challenges might be in introducing such a radical new approach.

Here are three key insights from our user research:

  1. Mental health measurement remains an unsolved problem. Despite the availability of many mental health surveys and scales, they are not widely used, especially in primary care and counseling settings. Reasons range from burden (“I don’t have time for this”) to skepticism (“Using a scale is no better than using my clinical judgement”) to lack of trust (“I don’t think my client is filling this in truthfully” and ”I don’t want to reveal this much to my counsellor”). These findings were in line with the literature on measurement-based mental health care. Any new measurement tool would have to overcome these barriers by creating clear value for both the person with lived experience and the clinician.
  2. There is value in combining subjective and objective data. People with lived experience and clinicians both welcomed the introduction of objective metrics, but not as a replacement for subjective assessment and asking people about their experience and feelings. The combination of subjective and objective metrics was seen as especially powerful. Objective metrics might validate the subjective experience; or if the two diverge, that in itself is an interesting insight which provides the starting point for a conversation.
  3. There are multiple use cases for new measurement technology. Our initial hypothesis was that clinicians might use a “brainwave test” as a diagnostic aid. However, this concept got a lukewarm reception. Mental health experts such as psychiatrists and clinical psychologists felt confident in their ability to diagnose via clinical interview. Primary care physicians thought an EEG test could be useful, but only if it was conducted by a medical assistant before their consultation with the patient, similar to a blood pressure test. Counsellors and social workers don’t do diagnosis in their practice, so it was irrelevant to them. Some people with lived experience did not like the idea of being labelled as depressed by a machine. By contrast, there was a notably strong interest in using technology as a tool for ongoing monitoring — capturing changes in mental health state over time — to learn what happens between visits. Many clinicians asked if they could send the EEG system home so their patients and clients could repeat the test on their own. They were also very interested in EEG’s potential predictive qualities, e.g. predicting who is likely to get more depressed in future. More research is needed to determine how a tool such as EEG would be best deployed in clinical and counseling settings, including how it could be combined with other measurement technologies such as digital phenotyping.

Much of our research was conducted in the US and the UK in partnership with Shift, a nonprofit based in London. This report by Shift details the research and the findings. (Report added on 15 December 2020.)

Opening up Amber to the world

We didn’t succeed in our original goal of finding a single biomarker for depression and anxiety. It is unlikely that one exists, given the complexity of mental health. Yet there’s no question that there is a huge opportunity for technology to enable better measurement.

This will empower individuals and their healthcare provider to better match intervention options to an individual’s needs, to measure the impact of those interventions, and ultimately promote better mental health. While the promise of emerging measurement techniques like EEG/ERP and digital phenotyping is very exciting, it is still early days. There are many pitfalls on the path to making tech-enabled mental health measurement work in the real world, and more research needs to be done.

For this reason we’ve decided to make Amber’s technology and insights available to the global mental health community. We believe we can make a bigger and faster impact on this huge problem by sharing our work freely.

Today, we are open-sourcing our hardware designs, visualizer and stimulus software of the Amber prototype EEG system and putting the code on Github. We are also pledging the free use of our patents and applications listed in this patent pledge. We are making these resources available so that mental health researchers have all of the specifications, code, and permissions they would need to rebuild our EEG system, or design their own based on it. In addition we are donating 50 assembled Amber prototype devices to Sapien Labs for use by researchers worldwide as part of their Human Brain Diversity Project which supports EEG research globally, with an emphasis on low-income countries and underrepresented groups.

We hope that open-sourcing our EEG system and publishing our machine learning techniques will be of value not just to EEG experts, but also to the wider mental health research community who were perhaps put off by the complexity and cost of working with EEG before. Addressing today’s challenges will require new partnerships between scientists, clinicians, technologists, policymakers, and individuals with lived experience. Now more than ever, more diverse voices, more multi-disciplinary collaboration, and more open sharing of knowledge are needed to unlock better mental health for everyone.

To learn more about Amber’s technology and user research, please visit the following links:

Please note: The Amber EEG System is a prototype investigational device and has not been evaluated by the US Food and Drug Administration or any other regulatory agency for any purpose, including a medical purpose.

This blog was first published on 2 November 2002 and updated on 15 December 2020 with links to the Shift user research report.

Obi FeltenLeave a comment

Helmholtz Horizons: Living in modern times

Helmholtz Horizons
6 November 2018
Berlin
Keynote
“Solving problems that matter: Insights from X, Alphabet’s moonshot factory”


Obi delivered the closing keynote of Helmholtz Horizons, a summit for European scientists and science policy makers. The Helmholtz Association of German Research Centres (Helmholtz-Gemeinschaft Deutscher Forschungszentren) is the largest scientific organisation in Germany, a union of 18 scientific-technical and biological-medical research centers.

Obi Felten

Chautauqua Institution: Living in modern times

Chautauqua Institution summer season
26 June 2017
Chautauqua, New York
Keynote
“Living in modern times: The past, present and future of invention (and innovation)”


Obi delivered the opening lecture of the 2017 Chautauqua Institution summer season. Chautauqua Institution is a community on the shores of Chautauqua Lake in southwestern New York state that comes alive each summer with a unique mix of fine and performing arts, lectures, interfaith worship and programs, and recreational activities. Previous speakers include Franklin Delano Roosevelt, Hillary Clinton, Ruth Bader Ginsburg, Tarana Burke, Arthur Brooks.

Watch the full lecture

Article on this talk

Obi Felten discusses moonshots and the power of innovation to make sci-fi a reality

By Brian Contreras
Photos by Cam Buker
This article was published on 26 June 2017
Copyright The Chautauquan Daily
Read the original on Chqdaily.com

Photo: Cam Buker

Xerox PARC scientist Alan Kay once said, “The best way to predict the future is to invent it.”

Obi Felten opened her Monday morning lecture with this quote, in turn opening the 2017 Chautauqua season as well as Week One and its theme of “Invention,” which asks, among other questions, what it might take for humanity’s next “giant leap.”

In her capacity as head of getting moonshots ready for contact with the real world at X (formerly Google X), Felten knows a thing or two about invention, not to mention the future. At X, she has worked with ideas that seem straight out of science fiction; they might seem more at home on the silver screen, but these inventions could one day be real, at least if Felten has her way.

After opening with Kay’s quote and enumerating a number of the cutting-edge inventions her X team has worked on, Felten launched full-steam into outlining what X does, and perhaps more urgently, what her lengthy job title actually consists of.

“We call ourselves the moonshot factory,” Felten said, “and we aim to solve large problems in the world with the help of technology.”

Though the work itself is hard, the moonshot team’s process is seductively simple. First, identify a problem (and not just any problem, but a big one, the sort that impacts “millions or even billions of people”). Then, propose a solution, no matter how outlandish, how seemingly impossible, it is. Finally — and this is where the difficult work of invention comes in — “turn (that) fiction into reality.”

Though making these moonshots real is never easy, Felten said it’s worth it. An optimist about the power of technology to make the world a better place, she cautioned against thinking of the current era as a uniquely disruptive one; as poet Alan Kirsch noted, all generations tend to think they live in periods of fundamental change.

That said, Felten made clear the modern age is by no means lacking in disruption, especially of the technological sort. She pointed to the internet in particular as having enabled an enormous paradigm shift in what is possible.

“We live in San Francisco, and my children think it’s completely normal to have breakfast with my parents, who live in Germany,” Felten said.

Felten acknowledged that the enormous potential of these emergent technologies has not been distributed equally. But through X, she and her team are working to change that.

For instance, she said, over half the world’s population lacks access to the internet. It was this global inequity that Felten and her team set out to solve with Loon, their balloon-based internet accessibility project.

Following a concise history of the balloon as a disruptive technology — from hot air balloonist Auguste Piccard to NASA and Bell Labs’ 1950s “communication satellites” — Felten outlined Loon’s modern quest to create airborne, data-transmitting balloons that could float through the sky and bring internet access to those on the ground below.

But the project was not without its failures.

“Our balloons were supposed to last 100 days,” Felten said. “The first balloons lasted five days.”

Ultimately, however, innovative responses to the durability question and other issues were successful, and the Loon project took flight (both literally and metaphorically) during a 2013 test in New Zealand. It was then that a rural sheep farmer, Charles, became the first human being to ever use Wi-Fi transmitted via balloon.

These days, Loon’s devices can last up to 190 days, and while they’re at it, provide LTE speeds to those who might otherwise have gone without internet. They even helped provide emergency communications capacities when destructive floods hit the Peruvian coast this past March.

But Loon is far from the only moonshot that Felten has worked on while at X. After all, as she puts it, humanity is currently in the midst of a “fourth industrial revolution.” Internet balloons are but one of many projects her team has in progress.

Perhaps the most publicized of these moonshots has been the self-driving car, a which has since graduated into its own company, the headlines-making Waymo.

The idea of a self-driving car is not without precedent. Leonardo da Vinci — the most iconic of inventors — designed one as far back as 1478. But with its movements preprogrammed and the only terrain it was suited for being a theater stage, da Vinci’s concept still had a long way to go  before it achieved Jetson-esque viability. From General Motors’ automated highway dioramas at the 1939 World’s Fair, through the 1971 launch of an automated Mars rover, to scientists at the Bundeswehr Uni Munich finally creating the first truly independent vehicle in the 1980s; the path to self-driving cars has been a long and circuitous one.

Obi Felten, Center, Speaks With Chautauquans On The Back Porch Of The Amphitheater Following Her Lecture On Monday, June 26, 2017.
CAM BUKER/STAFF PHOTOGRAPHER

But this narrative reached its climax in the mid-2000s, when the Defense Advanced Research Projects Agency launched a contest to see who could develop the best self-driving car. From 2004 to 2007, teams from Stanford and Carnegie Mellon Universities dueled back and forth to build the smartest, longest-running vehicle; it wasn’t until 2009 when members of both teams united under the Google self-driving car project that Felten’s moonshot began the trudge toward realization. All this time, though, the moonshot team stayed focused.

“The problem we’re trying to solve is safety,” Felten said. “Because 1.2 million people die on the road each year, and 94 percent of those accidents are caused by human error.”

The project’s work paid off in very real, human terms last year, when a blind man in Austin, Texas, took the world’s first truly autonomous ride in one of the team’s self-driving cars.

There is still a ways to go before their moonshot gets to its proverbial moon, Felten said, “but it’s exciting to see that we’re (turning) science fiction into reality.”

But with revolutionary projects like internet balloons or self-driving cars, Felten warned that it’s easy to get sucked into “the great myth of the lonely genius inventor.” Though people imagine the likes of Thomas Edison or Steve Jobs creating their legendary inventions in isolation, groups like X work hard to foster teamwork, communication and partnerships across any number of different industries and disciplines.

The myth of individualism isn’t the only concept that threatens to impede innovation, however; more than anything else (including funding), Felten sees a fear of failure as the single largest threat to inventing one’s way to a better future.

“To be audacious, you have to be humble,” Felten said. “And you have to expect that most of what you work on will not work.”

This is not abstract for Felten, either; her moonshot team takes pride in its failed projects. For instance, their abandoned Project Foghorn sought to turn ocean water into clean, efficient car fuel but was unable to overcome the steep competition with cheap oil. When it was ultimately shut down, all team members received bonuses for their decision to abandon the (at least temporarily) intractable concept.

As her lecture drew to a close, Felten returned to cultural fears about, rather than hopes for, new technologies. At first, she acknowledged the potential for new technology to widen, rather than bridge, gaps in socioeconomic class and quality of life; she also noted discomfort with the capacity for job automation to cause widespread unemployment.

But even if “we need to adapt” to keep pace with rapidly changing technological capacities, Felten ultimately sees this disruption as providing humanity with the opportunity to create for itself a more hospitable, equitable planet.

“We have a choice about which world we want to live in,” she said.

Felten presented essentially two versions of the future: one in which “the largest beneficiaries of invention are … the innovators, the shareholders, the investors,” or one where “innovation doesn’t just come from the hands of companies …  but from anywhere.”

Either way, the choice between those two potential paths is in the hands of people, not their tools.

“Technology of course will not solve all our problems, and there is no doubt that it will create some new ones,” she said. “But it is up to us how we use technology to make a better future for ourselves and for generations to come.”

Obi Felten