Hi all,

We've excluded these groups to date because we haven't (a) found useful
literature on the effects of their interventions; (b) developed a good way
of assessing them. Our sense is that these areas aren't as promising as the
ones we've focused on, but at this point that's just a rough sense, and we
hope eventually to further investigate these areas and to be able to say
more about them.

Best,
Holden

On Sat, Dec 31, 2011 at 4:11 PM, Eric Rogstad <esrogs@...> wrote:

> **
>
>
> The page on IJM (mentioned in the article) says,
>
> We investigated International Justice Mission but found they were
> ineligible for our review because of one of the following reasons:
>
>
>    - *Scope* - the charity's program was outside the scope of areas we
>       planned to cover (e.g., slavery, refugees, orphans and other abandoned
>       children) or focused on research or advocacy as opposed to direct provision
>       of services.
>
> http://givewell.org/International/charities/International-Justice-Mission
>
>
> Is there any discussion (perhaps a blog post) of why these issues were not
> targeted?  I can guess that they are assumed to be much less cost-effective
> than other efforts (e.g. international health-related ones), but I don't
> remember coming across that reasoning explicitly stated somewhere.
>
> -Eric
>
> On Sat, Dec 31, 2011 at 12:38 PM, Brian Slesinsky <bslesinsky@...>wrote:
>
>> **
>>
>>
>> I'm seeing an uptick in interest in anti-slavery charities after
>> Google's recent donations. Perhaps this would be a fruitful area of
>> research for GiveWell?
>>
>> - Brian
>>
>
>  
>

It seems a prior they'd be a lot less effective, as slavers fight back in a
way malaria doesn't.

On Tue, Jan 3, 2012 at 12:22 AM, Holden Karnofsky <holden0@...> wrote:

> **
>
>
> Hi all,
>
> We've excluded these groups to date because we haven't (a) found useful
> literature on the effects of their interventions; (b) developed a good way
> of assessing them. Our sense is that these areas aren't as promising as the
> ones we've focused on, but at this point that's just a rough sense, and we
> hope eventually to further investigate these areas and to be able to say
> more about them.
>
> Best,
> Holden
>
> On Sat, Dec 31, 2011 at 4:11 PM, Eric Rogstad <esrogs@gmail.com> wrote:
>
>> **
>>
>>
>> The page on IJM (mentioned in the article) says,
>>
>> We investigated International Justice Mission but found they were
>> ineligible for our review because of one of the following reasons:
>>
>>
>>    - *Scope* - the charity's program was outside the scope of areas we
>>       planned to cover (e.g., slavery, refugees, orphans and other abandoned
>>       children) or focused on research or advocacy as opposed to direct provision
>>       of services.
>>
>> http://givewell.org/International/charities/International-Justice-Mission
>>
>>
>>
>> Is there any discussion (perhaps a blog post) of why these issues were
>> not targeted?  I can guess that they are assumed to be much less
>> cost-effective than other efforts (e.g. international health-related ones),
>> but I don't remember coming across that reasoning explicitly stated
>> somewhere.
>>
>> -Eric
>>
>> On Sat, Dec 31, 2011 at 12:38 PM, Brian Slesinsky <bslesinsky@...m>wrote:
>>
>>> **
>>>
>>>
>>> I'm seeing an uptick in interest in anti-slavery charities after
>>> Google's recent donations. Perhaps this would be a fruitful area of
>>> research for GiveWell?
>>>
>>> - Brian
>>>
>>
>>
>  
>

Hello everyone,

A few people have told us that there's no way for GiveWell followers to
know when we publish new pages. Of course, people know when we update our
charity recommendations with new reviews, but when we publish other pages
(e.g., a review of a charity that's not top-rated or a page like the
Research FAQ, below), people don't know.

Sending a list of links to this group is one way to address this issue.
This email includes new content (except blog posts) since our major
research update just after Thanksgiving, including the major pages we
published as part of the updated report.

Our process:

   - 2011 international aid charities research process:
   http://givewell.org/international/process/2011
   - All charities considered to date: http://www.givewell.org/charities

Related to AMF:

   - Our review of the Against Malaria Foundation:
   http://givewell.org/international/top-charities/AMF
   - Our intervention report on mass distribution of long-lasting
   insecticide-treated nets:
   http://givewell.org/international/technical/programs/insecticide-treated-nets
   - Our discussion of the evidence on the effectiveness of large scale ITN
   distribution campaigns:
   http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence

Related to SCI:

   - Our review of the Schistosomiasis Control Initiative:
   http://givewell.org/international/top-charities/schistosomiasis-control-initiative
   - Our intervention report on combination deworming to kill
   soil-transmitted helminths and schistosomiasis:
   http://givewell.org/international/technical/programs/deworming

Related to IPA:

   - Our review of Innovations for Poverty Action:
   http://givewell.org/international/charities/ipa
   - Our overview of the evidence created by and the influence of notable
   IPA studies: http://givewell.org/international/charities/ipa/research

Other:

   - What others are saying about GiveWell:
   http://givewell.org/what-others-are-saying (this is an update on a
   previous "Press" page that we think presents GiveWell in a very positive
   light; we'd like for more new visitors to the site to see this page)
   - Frequently asked questions about our research:
   http://www.givewell.org/about/FAQ/research
   - Our review of International Development Enterprises:
   http://givewell.org/International/charities/International-Development-Enterprises
   - Notes from our October 2011 site visits:
   http://www.givewell.org/international/site-visits/october-2011
   - Audio and transcript from our conference calls:
   http://www.givewell.org/conference-call

Best,
Alexander

Hello everyone,

New/updated pages:

   - Our shortcomings page: http://givewell.org/about/shortcomings
   - Our intervention report on cash transfers:
   http://givewell.org/international/technical/programs/cash-transfers

Best,
Alexander

On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <alexander.is@...>wrote:

> Hello everyone,
>
> A few people have told us that there's no way for GiveWell followers to
> know when we publish new pages. Of course, people know when we update our
> charity recommendations with new reviews, but when we publish other pages
> (e.g., a review of a charity that's not top-rated or a page like the
> Research FAQ, below), people don't know.
>
> Sending a list of links to this group is one way to address this issue.
> This email includes new content (except blog posts) since our major
> research update just after Thanksgiving, including the major pages we
> published as part of the updated report.
>
> Our process:
>
>    - 2011 international aid charities research process:
>    http://givewell.org/international/process/2011
>    - All charities considered to date: http://www.givewell.org/charities
>
>  Related to AMF:
>
>    - Our review of the Against Malaria Foundation:
>    http://givewell.org/international/top-charities/AMF
>    - Our intervention report on mass distribution of long-lasting
>    insecticide-treated nets:
>    http://givewell.org/international/technical/programs/insecticide-treated-nets
>    - Our discussion of the evidence on the effectiveness of large scale
>    ITN distribution campaigns:
>    http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
>
> Related to SCI:
>
>    - Our review of the Schistosomiasis Control Initiative:
>    http://givewell.org/international/top-charities/schistosomiasis-control-initiative
>    - Our intervention report on combination deworming to kill
>    soil-transmitted helminths and schistosomiasis:
>    http://givewell.org/international/technical/programs/deworming
>
> Related to IPA:
>
>    - Our review of Innovations for Poverty Action:
>    http://givewell.org/international/charities/ipa
>    - Our overview of the evidence created by and the influence of notable
>    IPA studies: http://givewell.org/international/charities/ipa/research
>
> Other:
>
>    - What others are saying about GiveWell:
>    http://givewell.org/what-others-are-saying (this is an update on a
>    previous "Press" page that we think presents GiveWell in a very positive
>    light; we'd like for more new visitors to the site to see this page)
>    - Frequently asked questions about our research:
>    http://www.givewell.org/about/FAQ/research
>    - Our review of International Development Enterprises:
>    http://givewell.org/International/charities/International-Development-Enterprises
>    - Notes from our October 2011 site visits:
>    http://www.givewell.org/international/site-visits/october-2011
>    - Audio and transcript from our conference calls:
>    http://www.givewell.org/conference-call
>
> Best,
> Alexander
>

Hello everyone,

New/updated pages:

   - We've updated our review of the Small Enterprise Foundation:
   http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
   - We conducted a re-analysis of VillageReach's pilot project data:
   http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
   - We published our first update on AMF's progress since becoming a top
   charity: http://givewell.org/international/top-charities/amf/updates
   - We published an update on KIPP Houston's room for more funding:
   http://blog.givewell.org/2012/03/14/kipp-houston-update/
   - We posted the audio and documents from our December board meeting:
   http://givewell.org/about/official-records/board-meeting-17

As usual, please feel free to let us know if you have any questions or
comments.

Best,
Alexander

On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <alexander.is@...>wrote:

> Hello everyone,
>
> New/updated pages:
>
>    - Our shortcomings page: http://givewell.org/about/shortcomings
>    - Our intervention report on cash transfers:
>    http://givewell.org/international/technical/programs/cash-transfers
>
> Best,
> Alexander
>
> On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <alexander.is@...>wrote:
>
>> Hello everyone,
>>
>> A few people have told us that there's no way for GiveWell followers to
>> know when we publish new pages. Of course, people know when we update our
>> charity recommendations with new reviews, but when we publish other pages
>> (e.g., a review of a charity that's not top-rated or a page like the
>> Research FAQ, below), people don't know.
>>
>> Sending a list of links to this group is one way to address this issue.
>> This email includes new content (except blog posts) since our major
>> research update just after Thanksgiving, including the major pages we
>> published as part of the updated report.
>>
>> Our process:
>>
>>    - 2011 international aid charities research process:
>>    http://givewell.org/international/process/2011
>>    - All charities considered to date: http://www.givewell.org/charities
>>
>>  Related to AMF:
>>
>>    - Our review of the Against Malaria Foundation:
>>    http://givewell.org/international/top-charities/AMF
>>    - Our intervention report on mass distribution of long-lasting
>>    insecticide-treated nets:
>>    http://givewell.org/international/technical/programs/insecticide-treated-nets
>>    - Our discussion of the evidence on the effectiveness of large scale
>>    ITN distribution campaigns:
>>    http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
>>
>> Related to SCI:
>>
>>    - Our review of the Schistosomiasis Control Initiative:
>>    http://givewell.org/international/top-charities/schistosomiasis-control-initiative
>>    - Our intervention report on combination deworming to kill
>>    soil-transmitted helminths and schistosomiasis:
>>    http://givewell.org/international/technical/programs/deworming
>>
>> Related to IPA:
>>
>>    - Our review of Innovations for Poverty Action:
>>    http://givewell.org/international/charities/ipa
>>    - Our overview of the evidence created by and the influence of
>>    notable IPA studies:
>>    http://givewell.org/international/charities/ipa/research
>>
>> Other:
>>
>>    - What others are saying about GiveWell:
>>    http://givewell.org/what-others-are-saying (this is an update on a
>>    previous "Press" page that we think presents GiveWell in a very positive
>>    light; we'd like for more new visitors to the site to see this page)
>>    - Frequently asked questions about our research:
>>    http://www.givewell.org/about/FAQ/research
>>    - Our review of International Development Enterprises:
>>    http://givewell.org/International/charities/International-Development-Enterprises
>>    - Notes from our October 2011 site visits:
>>    http://www.givewell.org/international/site-visits/october-2011
>>    - Audio and transcript from our conference calls:
>>    http://www.givewell.org/conference-call
>>
>> Best,
>> Alexander
>>
>
>

Hi Alexander,

Regarding VillageReach, is this the VillageReach update that you said
you'd be publishing soon? The link you have given is to a re-analysis
of the pilot project prior to GiveWell's recommendation of
VillageReach, not of the expansion that was funded as a result of
GiveWell's recommendation. Perhaps I am missing something here?

Do you plan to post updates on the expansion that was funded as a
result of GiveWell's recommendation? If so, when?

Thank you,

Vipul

* Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri) wrote
> Hello everyone,
> 
> New/updated pages:
> 
>    - We've updated our review of the Small Enterprise Foundation:
>    http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
>    - We conducted a re-analysis of VillageReach's pilot project data:
>    http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
>    - We published our first update on AMF's progress since becoming a top
>    charity: http://givewell.org/international/top-charities/amf/updates
>    - We published an update on KIPP Houston's room for more funding:
>    http://blog.givewell.org/2012/03/14/kipp-houston-update/
>    - We posted the audio and documents from our December board meeting:
>    http://givewell.org/about/official-records/board-meeting-17
> 
> As usual, please feel free to let us know if you have any questions or
> comments.
> 
> Best,
> Alexander
> 
> On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <alexander.is@...>wrote:
> 
> > Hello everyone,
> >
> > New/updated pages:
> >
> >    - Our shortcomings page: http://givewell.org/about/shortcomings
> >    - Our intervention report on cash transfers:
> >    http://givewell.org/international/technical/programs/cash-transfers
> >
> > Best,
> > Alexander
> >
> > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <alexander.is@...>wrote:
> >
> >> Hello everyone,
> >>
> >> A few people have told us that there's no way for GiveWell followers to
> >> know when we publish new pages. Of course, people know when we update our
> >> charity recommendations with new reviews, but when we publish other pages
> >> (e.g., a review of a charity that's not top-rated or a page like the
> >> Research FAQ, below), people don't know.
> >>
> >> Sending a list of links to this group is one way to address this issue.
> >> This email includes new content (except blog posts) since our major
> >> research update just after Thanksgiving, including the major pages we
> >> published as part of the updated report.
> >>
> >> Our process:
> >>
> >>    - 2011 international aid charities research process:
> >>    http://givewell.org/international/process/2011
> >>    - All charities considered to date: http://www.givewell.org/charities
> >>
> >>  Related to AMF:
> >>
> >>    - Our review of the Against Malaria Foundation:
> >>    http://givewell.org/international/top-charities/AMF
> >>    - Our intervention report on mass distribution of long-lasting
> >>    insecticide-treated nets:
> >>    http://givewell.org/international/technical/programs/insecticide-treated-nets
> >>    - Our discussion of the evidence on the effectiveness of large scale
> >>    ITN distribution campaigns:
> >>    http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
> >>
> >> Related to SCI:
> >>
> >>    - Our review of the Schistosomiasis Control Initiative:
> >>    http://givewell.org/international/top-charities/schistosomiasis-control-initiative
> >>    - Our intervention report on combination deworming to kill
> >>    soil-transmitted helminths and schistosomiasis:
> >>    http://givewell.org/international/technical/programs/deworming
> >>
> >> Related to IPA:
> >>
> >>    - Our review of Innovations for Poverty Action:
> >>    http://givewell.org/international/charities/ipa
> >>    - Our overview of the evidence created by and the influence of
> >>    notable IPA studies:
> >>    http://givewell.org/international/charities/ipa/research
> >>
> >> Other:
> >>
> >>    - What others are saying about GiveWell:
> >>    http://givewell.org/what-others-are-saying (this is an update on a
> >>    previous "Press" page that we think presents GiveWell in a very positive
> >>    light; we'd like for more new visitors to the site to see this page)
> >>    - Frequently asked questions about our research:
> >>    http://www.givewell.org/about/FAQ/research
> >>    - Our review of International Development Enterprises:
> >>    http://givewell.org/International/charities/International-Development-Enterprises
> >>    - Notes from our October 2011 site visits:
> >>    http://www.givewell.org/international/site-visits/october-2011
> >>    - Audio and transcript from our conference calls:
> >>    http://www.givewell.org/conference-call
> >>
> >> Best,
> >> Alexander
> >>
> >
> >

Hi Vipul,

Thanks for the email and for continuing to follow up with us about
publishing our VillageReach update. Today, we've:

   1. Published a summary of the update today on our blog:
   http://blog.givewell.org/2012/03/26/villagereach-update/
   2. A full, detailed update on our main site:
   http://givewell.org/international/top-charities/villagereach/updates#March262012update
   3. A guest post from John Beale at VillageReach providing more context
   for this update:
   http://blog.givewell.org/2012/03/26/guest-post-from-john-beale-at-villagereach

We previously set a goal of publishing quarterly updates on VillageReach
(and all charities we rate high enough to direct them significant funding).
Today's VillageReach update is our first update on VillageReach since
August 2011, so we are publishing this update significantly later than we
had hoped. We will probably not be aiming for quarterly updates in the
future. To briefly explain the three main factors that led to this delay:

   1. We were scheduled for a VillageReach update in early December 2011,
   but we were primarily focused on our new recommendations and on giving
   season from October 1, 2011 - January 1, 2012. During that 3-month period,
   we focused our all efforts on our new recommendations and did not focus on
   VillageReach. (We feel this was the right decision.)
   2. When we began our reassessment of VillageReach in early 2012, we
   started re-analyzing the evidence of effectiveness from the pilot project.
   This analysis -- some of which we published recently at
   http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
--
   led to correspondence and conversations with VillageReach representatives
   about pilot project data rather than an update of VillageReach's work. In
   hindsight, it's clear that we should have prioritized publishing an update
   on VillageReach's work in Mozambique over revisiting the pilot project.
   3. In every case where we publish something about a charity, we send the
   charity a draft of our page before we publish it. We do this to fact-check
   our work and to give the organization an opportunity to comment on
   particular words/phrasing it finds problematic. In this case, extensive
   back and forth further delayed publication.

Given the amount of work it takes to do an update and the pace at which
projects tend to move, we're now thinking that quarterly is too often. We
are rethinking the frequency we should aim for and will update on what we
decide.

Best,
Elie


On Fri, Mar 23, 2012 at 2:04 PM, Vipul Naik <vipul@....edu> wrote:

> **
>
>
> Hi Alexander,
>
> Regarding VillageReach, is this the VillageReach update that you said
> you'd be publishing soon? The link you have given is to a re-analysis
> of the pilot project prior to GiveWell's recommendation of
> VillageReach, not of the expansion that was funded as a result of
> GiveWell's recommendation. Perhaps I am missing something here?
>
> Do you plan to post updates on the expansion that was funded as a
> result of GiveWell's recommendation? If so, when?
>
> Thank you,
>
> Vipul
>
> * Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri) wrote
> > Hello everyone,
> >
> > New/updated pages:
> >
> > - We've updated our review of the Small Enterprise Foundation:
> >
> http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
> > - We conducted a re-analysis of VillageReach's pilot project data:
> >
> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
> > - We published our first update on AMF's progress since becoming a top
> > charity: http://givewell.org/international/top-charities/amf/updates
> > - We published an update on KIPP Houston's room for more funding:
> > http://blog.givewell.org/2012/03/14/kipp-houston-update/
> > - We posted the audio and documents from our December board meeting:
>
> > http://givewell.org/about/official-records/board-meeting-17
> >
> > As usual, please feel free to let us know if you have any questions or
> > comments.
> >
> > Best,
> > Alexander
> >
> > On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <
> alexander.is@...>wrote:
> >
> > > Hello everyone,
> > >
> > > New/updated pages:
> > >
> > > - Our shortcomings page: http://givewell.org/about/shortcomings
> > > - Our intervention report on cash transfers:
>
> > > http://givewell.org/international/technical/programs/cash-transfers
> > >
> > > Best,
> > > Alexander
> > >
> > > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <
> alexander.is@...>wrote:
> > >
> > >> Hello everyone,
> > >>
> > >> A few people have told us that there's no way for GiveWell followers
> to
> > >> know when we publish new pages. Of course, people know when we update
> our
> > >> charity recommendations with new reviews, but when we publish other
> pages
> > >> (e.g., a review of a charity that's not top-rated or a page like the
> > >> Research FAQ, below), people don't know.
> > >>
> > >> Sending a list of links to this group is one way to address this
> issue.
> > >> This email includes new content (except blog posts) since our major
> > >> research update just after Thanksgiving, including the major pages we
> > >> published as part of the updated report.
> > >>
> > >> Our process:
> > >>
> > >> - 2011 international aid charities research process:
> > >> http://givewell.org/international/process/2011
> > >> - All charities considered to date: http://www.givewell.org/charities
> > >>
> > >> Related to AMF:
> > >>
> > >> - Our review of the Against Malaria Foundation:
> > >> http://givewell.org/international/top-charities/AMF
> > >> - Our intervention report on mass distribution of long-lasting
> > >> insecticide-treated nets:
> > >>
> http://givewell.org/international/technical/programs/insecticide-treated-nets
> > >> - Our discussion of the evidence on the effectiveness of large scale
>
> > >> ITN distribution campaigns:
> > >>
> http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
> > >>
> > >> Related to SCI:
> > >>
> > >> - Our review of the Schistosomiasis Control Initiative:
> > >>
> http://givewell.org/international/top-charities/schistosomiasis-control-initiative
> > >> - Our intervention report on combination deworming to kill
>
> > >> soil-transmitted helminths and schistosomiasis:
> > >> http://givewell.org/international/technical/programs/deworming
> > >>
> > >> Related to IPA:
> > >>
> > >> - Our review of Innovations for Poverty Action:
> > >> http://givewell.org/international/charities/ipa
> > >> - Our overview of the evidence created by and the influence of
>
> > >> notable IPA studies:
> > >> http://givewell.org/international/charities/ipa/research
> > >>
> > >> Other:
> > >>
> > >> - What others are saying about GiveWell:
>
> > >> http://givewell.org/what-others-are-saying (this is an update on a
> > >> previous "Press" page that we think presents GiveWell in a very
> positive
> > >> light; we'd like for more new visitors to the site to see this page)
> > >> - Frequently asked questions about our research:
> > >> http://www.givewell.org/about/FAQ/research
> > >> - Our review of International Development Enterprises:
> > >>
> http://givewell.org/International/charities/International-Development-Enterprises
> > >> - Notes from our October 2011 site visits:
> > >> http://www.givewell.org/international/site-visits/october-2011
> > >> - Audio and transcript from our conference calls:
> > >> http://www.givewell.org/conference-call
> > >>
> > >> Best,
> > >> Alexander
> > >>
> > >
> > >
>
>  
>

Hi all,

We've been doing some more research on insecticide-treated bednets (ITNs),
in particular trying to better understand the case for universal
distribution (i.e., aiming to give nets to everyone who lives in a
particular area regardless of age) vs. distribution targeted at children
under five (who die from malaria most frequently) and how ITN effectiveness
relates to malaria prevalence.

To do this, we looked at the openmalaria malaria epidemiology simulator
produced by the Swiss Tropical and Public Health
Institute<http://www.swisstph.ch/services/ehealth/ehealth-projects/malaria-modelling.html>.
We're
not very confident in the results and haven't fully explored their
implications, but they generally seem to comport with the theory that
universal distribution is as good or better at preventing mortality than
targeted distribution. It also appears that, contrary to intuition, the
bednets are not necessarily most effective at preventing mortality in areas
with the highest transmission rates.

These tentative results are consistent with our recommendation of AMF and
don't affect our bottom line at all.

*Background*
*
*

Most of AMF’s distribution partners, including the one we visited in
Malawi<http://www.givewell.org/international/site-visits/october-2011#AgainstMalariaFoundationConcernUniversal>,
focus on universal distribution. We wondered about this because if the main
benefit of ITNs is in reducing mortality for children under five, it seemed
that targeting children under five could achieve most of the benefits of
universal coverage, for a fraction of the cost. In the area served by AMF’s
Malawi distribution partner, ITN coverage amongst children was already
fairly high, so it seemed that reaching universal coverage might not be as
helpful as somewhere else where the proportion of children already covered
was lower. Our preliminary investigation of this
question<http://givewell.org/international/technical/programs/insecticide-treated-nets#Targetedvs.universalcoverage>
during
the fall consisted of:


   - Speaking with Christian Lengeler, the author of the
Cochrane<http://givewell.org/node/312#CochraneLibrary> review
   of the evidence regarding ITN distribution discussed in our ITN
   distribution intervention
report<http://givewell.org/international/technical/programs/insecticide-treated-nets>.
   (This Cochrane review appears to us to be the mostwidely
cited<http://givewell.org/international/technical/programs/insecticide-treated-nets#widelycited>
evidence
   of the efficacy of ITNs.) We have published our
notes<http://givewell.org/files/DWDA%202009/Interventions/Nets/Notes%20from%20conversation%20with%20Christian%20Lengeler.doc>from
   this conversation.
   - Speaking with two scholars that Professor Lengeler referred us to on
   this question, Dave
Smith<http://givewell.org/files/DWDA%202009/Interventions/Nets/Notes%20from%20conversation%20with%20Dave%20Smith.doc>
    and Thomas Smith<http://givewell.org/files/DWDA%202009/Interventions/Nets/Notes%20from%20conversation%20with%20Thomas%20Smith%20edited.doc>.
   We have published notes from each of these conversations.
   - Reviewing literature that these scholars referred us to.

In our main model of the cost-effectiveness of distributing
ITNs<http://givewell.org/files/DWDA%202009/Interventions/Nets/Cost-effectiveness%20analysis%20for%20LLIN%20distribution.xls>
(XLS),
we assumed that half of the benefits of ITN distribution for saving
children come from directly covering individual kids, while the other half
comes from interrupting malaria transmission in the community. Roughly
speaking, the greater proportion of the benefits of deworming that come
from community effects, the more effective universal distribution will be
relative to distribution targeted at children.

*What we did*

To continue our investigation of this question, we asked Dario
Amodei<http://blog.givewell.org/2010/06/03/my-donation-for-2009-guest-post-from-dario-amodei/>
to
run a simulation comparing universal and targeted ITN distribution
campaigns. Using software produced by the Swiss Tropical and Public Health
Institute<http://www.swisstph.ch/services/ehealth/ehealth-projects/malaria-modelling.html>,
Dario was able to compare the expected outcomes of a universal and a
targeted ITN distribution campaign. The software runs probabilistic
simulations, which means it might not get the same answer every time except
with very large sample sizes. The models appear to sensitive to a number of
assumptions in ways that we haven't fully worked out.

Once we had the results of Dario's analysis, we asked Rob Mather at the
Against Malaria Foundation about them. He consulted with scholars from
AMF's Malaria Advisory Group, who went though some back and forth, and then
sent us the record of their exchange.
Thanks to Dario and Alex Dingle, who helped him set up the modeling
software.

*What we found*

Universal bednet distribution was generally as good or better than targeted
distribution with respect to mortality in the modeling scenarios that Dario
ran for us, though we're not sure to what extent this conclusion is
sensitive to assumptions. We don't fully understand what drives the
outcomes in these scenarios and have done any assessment of whether the
modeling is reasonable.

One of Dario's other tentative findings was that higher transmission of
malaria does not necessarily translate into greater effectiveness for ITNs.
We contacted Rob Mather to ask about this, and he sent the question to the
Malaria Advisory Group. Thomas Smith, a scholar originally referred to us
by Christian Lengeler who also serves on AMF’s Malaria Advisory Group,
wrote in his reply:

We would expect a higher coverage of ITNs to have a substantial effect on
transmission measures whatever the starting level of transmission (>0), but
the effect on measures of morbidity and mortality would also depend on the
age groups considered and the time horizon. While it is clearly the case
that there is no benefit in using ITNs if there is no malaria at all, the
effects on morbidity and mortality are greatest at intermediate levels of
transmission and the relationship between the public health benefits of
given levels of scaling up and the initial EIR is complicated. It is
certainly not the case that ITNs have least utility in low endemic areas,
since it is precisely there that there is a chance they will interrupt
transmission.

*We've posted additional files related to this analysis (e.g. charts from
Dario's analysis, source code, and emails from the AMF Malaria Advisory
Group) here<http://givewell.org/files/DWDA%202009/Interventions/Nets/Malaria%20Modeling/Malaria%20Model.zip>
 (ZIP file; requires this software <http://code.google.com/p/openmalaria/> *to
run the malaria models).

And, as Rob astutely pointed out, it is* LLIN distribution*, not *deworming*,
that should appear in this sentence: "Roughly speaking, the greater
proportion of the benefits of *deworming* that come from community effects,
the more effective universal distribution will be relative to distribution
targeted at children."

Sorry for the mistake and second email!

On Mon, Apr 23, 2012 at 12:50 PM, Alexander Berger
<alexander.is@...>wrote:

> Hi all,
>
> We've been doing some more research on insecticide-treated bednets (ITNs),
> in particular trying to better understand the case for universal
> distribution (i.e., aiming to give nets to everyone who lives in a
> particular area regardless of age) vs. distribution targeted at children
> under five (who die from malaria most frequently) and how ITN effectiveness
> relates to malaria prevalence.
>
> To do this, we looked at the openmalaria malaria epidemiology simulator
> produced by the Swiss Tropical and Public Health Institute<http://www.swisstph.ch/services/ehealth/ehealth-projects/malaria-modelling.html>. We're
> not very confident in the results and haven't fully explored their
> implications, but they generally seem to comport with the theory that
> universal distribution is as good or better at preventing mortality than
> targeted distribution. It also appears that, contrary to intuition, the
> bednets are not necessarily most effective at preventing mortality in areas
> with the highest transmission rates.
>
> These tentative results are consistent with our recommendation of AMF and
> don't affect our bottom line at all.
>
> *Background*
> *
> *
>
> Most of AMF’s distribution partners, including the one we visited in
> Malawi<http://www.givewell.org/international/site-visits/october-2011#AgainstMalariaFoundationConcernUniversal>,
> focus on universal distribution. We wondered about this because if the main
> benefit of ITNs is in reducing mortality for children under five, it seemed
> that targeting children under five could achieve most of the benefits of
> universal coverage, for a fraction of the cost. In the area served by AMF’s
> Malawi distribution partner, ITN coverage amongst children was already
> fairly high, so it seemed that reaching universal coverage might not be as
> helpful as somewhere else where the proportion of children already covered
> was lower. Our preliminary investigation of this question<http://givewell.org/international/technical/programs/insecticide-treated-nets#Targetedvs.universalcoverage> during
> the fall consisted of:
>
>
>    - Speaking with Christian Lengeler, the author of the Cochrane<http://givewell.org/node/312#CochraneLibrary> review
>    of the evidence regarding ITN distribution discussed in our ITN
>    distribution intervention report<http://givewell.org/international/technical/programs/insecticide-treated-nets>.
>    (This Cochrane review appears to us to be the mostwidely cited<http://givewell.org/international/technical/programs/insecticide-treated-nets#widelycited> evidence
>    of the efficacy of ITNs.) We have published our notes<http://givewell.org/files/DWDA%202009/Interventions/Nets/Notes%20from%20conversation%20with%20Christian%20Lengeler.doc>from
>    this conversation.
>    - Speaking with two scholars that Professor Lengeler referred us to on
>    this question, Dave Smith<http://givewell.org/files/DWDA%202009/Interventions/Nets/Notes%20from%20conversation%20with%20Dave%20Smith.doc>
>     and Thomas Smith<http://givewell.org/files/DWDA%202009/Interventions/Nets/Notes%20from%20conversation%20with%20Thomas%20Smith%20edited.doc>.
>    We have published notes from each of these conversations.
>    - Reviewing literature that these scholars referred us to.
>
> In our main model of the cost-effectiveness of distributing ITNs<http://givewell.org/files/DWDA%202009/Interventions/Nets/Cost-effectiveness%20analysis%20for%20LLIN%20distribution.xls> (XLS),
> we assumed that half of the benefits of ITN distribution for saving
> children come from directly covering individual kids, while the other half
> comes from interrupting malaria transmission in the community. Roughly
> speaking, the greater proportion of the benefits of *LLIN distribution*that come from community effects, the more effective universal distribution
> will be relative to distribution targeted at children.
>
> *What we did*
>
> To continue our investigation of this question, we asked Dario Amodei<http://blog.givewell.org/2010/06/03/my-donation-for-2009-guest-post-from-dario-amodei/> to
> run a simulation comparing universal and targeted ITN distribution
> campaigns. Using software produced by the Swiss Tropical and Public
> Health Institute<http://www.swisstph.ch/services/ehealth/ehealth-projects/malaria-modelling.html>,
> Dario was able to compare the expected outcomes of a universal and a
> targeted ITN distribution campaign. The software runs probabilistic
> simulations, which means it might not get the same answer every time except
> with very large sample sizes. The models appear to sensitive to a number of
> assumptions in ways that we haven't fully worked out.
>
> Once we had the results of Dario's analysis, we asked Rob Mather at the
> Against Malaria Foundation about them. He consulted with scholars from
> AMF's Malaria Advisory Group, who went though some back and forth, and then
> sent us the record of their exchange.
> Thanks to Dario and Alex Dingle, who helped him set up the modeling
> software.
>
> *What we found*
>
> Universal bednet distribution was generally as good or better than
> targeted distribution with respect to mortality in the modeling scenarios
> that Dario ran for us, though we're not sure to what extent this conclusion
> is sensitive to assumptions. We don't fully understand what drives the
> outcomes in these scenarios and have done any assessment of whether the
> modeling is reasonable.
>
> One of Dario's other tentative findings was that higher transmission of
> malaria does not necessarily translate into greater effectiveness for ITNs.
> We contacted Rob Mather to ask about this, and he sent the question to the
> Malaria Advisory Group. Thomas Smith, a scholar originally referred to us
> by Christian Lengeler who also serves on AMF’s Malaria Advisory Group,
> wrote in his reply:
>
> We would expect a higher coverage of ITNs to have a substantial effect on
> transmission measures whatever the starting level of transmission (>0), but
> the effect on measures of morbidity and mortality would also depend on the
> age groups considered and the time horizon. While it is clearly the case
> that there is no benefit in using ITNs if there is no malaria at all, the
> effects on morbidity and mortality are greatest at intermediate levels of
> transmission and the relationship between the public health benefits of
> given levels of scaling up and the initial EIR is complicated. It is
> certainly not the case that ITNs have least utility in low endemic areas,
> since it is precisely there that there is a chance they will interrupt
> transmission.
>
> *We've posted additional files related to this analysis (e.g. charts from
> Dario's analysis, source code, and emails from the AMF Malaria Advisory
> Group) here<http://givewell.org/files/DWDA%202009/Interventions/Nets/Malaria%20Modeling/Malaria%20Model.zip>
>  (ZIP file; requires this software <http://code.google.com/p/openmalaria/>
>  *to run the malaria models).
>

Hi everyone,

We've posted a new review of Give Directly (which continues to be a
"standout" organization) at *
http://www.givewell.org/international/charities/give-directly*.

In addition, we've posted the following transcripts from conversations
we've had with various organizations over the last few months:

*Centre for Neglected Tropical Diseases*

   - Moses Bockarie,
Director<http://www.givewell.org/files/DWDA%202009/CNTD/Givewell%20phone%20conversation%20with%20CNTD.doc>
on
   March 26, 2012.

*Development Media International*

   - Roy Head, CEO of Development Media International (DMI), Will Snell,
   Head of Public Engagement & Development at DMI, and Cathryn Wood, Public
   Engagement and Innovation
Manager<http://www.givewell.org/files/DWDA%202009/DMI/Conversation%20with%20Giving%20What%20We%20Can%20and%20DMI.doc>
on
   April 17, 2012.

*Doctors Without Borders (MSF)*

   - Jennifer Tierney, Development Director, and Jason Cone, Communications
   Director<http://www.givewell.org/files/DWDA%202009/MSF/MSF%20-%20GiveWell%202011%2009%2026.doc>
on
   September 26, 2011.
   - Jennifer Tierney, Development
Director<http://www.givewell.org/files/DWDA%202009/MSF/Conversation%20with%20MSF-USA%202012%2003%2008%20final.doc>
on
   March 8, 2012.

*GAVI alliance*

   - Seth Berkley,
CEO<http://www.givewell.org/files/DWDA%202009/GAVI/gavi%202012%2001.doc>
on
   January 12, 2012.
   - Alex Palacios, Special Representative to the
U.S.<http://www.givewell.org/files/DWDA%202009/GAVI/Conversation%20with%20GAVI%202012%2002%2014.doc>
on
   February 14, 2012.

*Measles Initiative*

   - Peter Strebel, World Health Organization Department of Immunization,
   Vaccines and Biologicals; Andrea Gay, United Nations Foundation Executive
   Director of Children's Health; and Steve Cochi, CDC Center for Global
   Health Senior Advisor to the
Director<http://www.givewell.org/files/DWDA%202009/Measles%20Initiative/Conversation%20with%20Measles%20Initiative%202012%2004%2017%20final.doc>
on
   April 18, 2012.

*TB REACH*

   - Suvanand Sahu, Team Leader, and Jacob Creswell, Technical
Officer<http://www.givewell.org/files/DWDA%202009/TB%20REACH/Conversation%20with%20TB%20Reach%202012%2003%2005%20final.doc>
on
   March 5, 2012.

*William and Flora Hewlett Foundation*


   - Paul Brest<http://www.givewell.org/files/conversations/paul-brest-march-2012.doc>
in
   March 2012.


As always, please don't hesitate to reach out if you have any questions or
comments.

Best,
Alexander

On Mon, Mar 26, 2012 at 1:42 PM, Elie Hassenfeld <elie@givewell.org> wrote:

> **
>
>
> Hi Vipul,
>
> Thanks for the email and for continuing to follow up with us about
> publishing our VillageReach update. Today, we've:
>
>    1. Published a summary of the update today on our blog:
>    http://blog.givewell.org/2012/03/26/villagereach-update/
>    2. A full, detailed update on our main site:
>    http://givewell.org/international/top-charities/villagereach/updates#March262012update
>    3. A guest post from John Beale at VillageReach providing more context
>    for this update:
>    http://blog.givewell.org/2012/03/26/guest-post-from-john-beale-at-villagereach
>
> We previously set a goal of publishing quarterly updates on VillageReach
> (and all charities we rate high enough to direct them significant funding).
> Today's VillageReach update is our first update on VillageReach since
> August 2011, so we are publishing this update significantly later than we
> had hoped. We will probably not be aiming for quarterly updates in the
> future. To briefly explain the three main factors that led to this delay:
>
>    1. We were scheduled for a VillageReach update in early December 2011,
>    but we were primarily focused on our new recommendations and on giving
>    season from October 1, 2011 - January 1, 2012. During that 3-month period,
>    we focused our all efforts on our new recommendations and did not focus on
>    VillageReach. (We feel this was the right decision.)
>    2. When we began our reassessment of VillageReach in early 2012, we
>    started re-analyzing the evidence of effectiveness from the pilot project.
>    This analysis -- some of which we published recently at
>    http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis --
>    led to correspondence and conversations with VillageReach representatives
>    about pilot project data rather than an update of VillageReach's work. In
>    hindsight, it's clear that we should have prioritized publishing an update
>    on VillageReach's work in Mozambique over revisiting the pilot project.
>    3. In every case where we publish something about a charity, we send
>    the charity a draft of our page before we publish it. We do this to
>    fact-check our work and to give the organization an opportunity to comment
>    on particular words/phrasing it finds problematic. In this case, extensive
>    back and forth further delayed publication.
>
> Given the amount of work it takes to do an update and the pace at which
> projects tend to move, we're now thinking that quarterly is too often. We
> are rethinking the frequency we should aim for and will update on what we
> decide.
>
> Best,
> Elie
>
>
> On Fri, Mar 23, 2012 at 2:04 PM, Vipul Naik <vipul@...>wrote:
>
>> **
>>
>>
>> Hi Alexander,
>>
>> Regarding VillageReach, is this the VillageReach update that you said
>> you'd be publishing soon? The link you have given is to a re-analysis
>> of the pilot project prior to GiveWell's recommendation of
>> VillageReach, not of the expansion that was funded as a result of
>> GiveWell's recommendation. Perhaps I am missing something here?
>>
>> Do you plan to post updates on the expansion that was funded as a
>> result of GiveWell's recommendation? If so, when?
>>
>> Thank you,
>>
>> Vipul
>>
>> * Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri) wrote
>> > Hello everyone,
>> >
>> > New/updated pages:
>> >
>> > - We've updated our review of the Small Enterprise Foundation:
>> >
>> http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
>> > - We conducted a re-analysis of VillageReach's pilot project data:
>> >
>> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
>> > - We published our first update on AMF's progress since becoming a top
>> > charity: http://givewell.org/international/top-charities/amf/updates
>> > - We published an update on KIPP Houston's room for more funding:
>> > http://blog.givewell.org/2012/03/14/kipp-houston-update/
>> > - We posted the audio and documents from our December board meeting:
>>
>> > http://givewell.org/about/official-records/board-meeting-17
>> >
>> > As usual, please feel free to let us know if you have any questions or
>> > comments.
>> >
>> > Best,
>> > Alexander
>> >
>> > On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <
>> alexander.is@...>wrote:
>> >
>> > > Hello everyone,
>> > >
>> > > New/updated pages:
>> > >
>> > > - Our shortcomings page: http://givewell.org/about/shortcomings
>> > > - Our intervention report on cash transfers:
>>
>> > > http://givewell.org/international/technical/programs/cash-transfers
>> > >
>> > > Best,
>> > > Alexander
>> > >
>> > > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <
>> alexander.is@...>wrote:
>> > >
>> > >> Hello everyone,
>> > >>
>> > >> A few people have told us that there's no way for GiveWell followers
>> to
>> > >> know when we publish new pages. Of course, people know when we
>> update our
>> > >> charity recommendations with new reviews, but when we publish other
>> pages
>> > >> (e.g., a review of a charity that's not top-rated or a page like the
>> > >> Research FAQ, below), people don't know.
>> > >>
>> > >> Sending a list of links to this group is one way to address this
>> issue.
>> > >> This email includes new content (except blog posts) since our major
>> > >> research update just after Thanksgiving, including the major pages we
>> > >> published as part of the updated report.
>> > >>
>> > >> Our process:
>> > >>
>> > >> - 2011 international aid charities research process:
>> > >> http://givewell.org/international/process/2011
>> > >> - All charities considered to date:
>> http://www.givewell.org/charities
>> > >>
>> > >> Related to AMF:
>> > >>
>> > >> - Our review of the Against Malaria Foundation:
>> > >> http://givewell.org/international/top-charities/AMF
>> > >> - Our intervention report on mass distribution of long-lasting
>> > >> insecticide-treated nets:
>> > >>
>> http://givewell.org/international/technical/programs/insecticide-treated-nets
>> > >> - Our discussion of the evidence on the effectiveness of large scale
>>
>> > >> ITN distribution campaigns:
>> > >>
>> http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
>> > >>
>> > >> Related to SCI:
>> > >>
>> > >> - Our review of the Schistosomiasis Control Initiative:
>> > >>
>> http://givewell.org/international/top-charities/schistosomiasis-control-initiative
>> > >> - Our intervention report on combination deworming to kill
>>
>> > >> soil-transmitted helminths and schistosomiasis:
>> > >> http://givewell.org/international/technical/programs/deworming
>> > >>
>> > >> Related to IPA:
>> > >>
>> > >> - Our review of Innovations for Poverty Action:
>> > >> http://givewell.org/international/charities/ipa
>> > >> - Our overview of the evidence created by and the influence of
>>
>> > >> notable IPA studies:
>> > >> http://givewell.org/international/charities/ipa/research
>> > >>
>> > >> Other:
>> > >>
>> > >> - What others are saying about GiveWell:
>>
>> > >> http://givewell.org/what-others-are-saying (this is an update on a
>> > >> previous "Press" page that we think presents GiveWell in a very
>> positive
>> > >> light; we'd like for more new visitors to the site to see this page)
>> > >> - Frequently asked questions about our research:
>> > >> http://www.givewell.org/about/FAQ/research
>> > >> - Our review of International Development Enterprises:
>> > >>
>> http://givewell.org/International/charities/International-Development-Enterprises
>> > >> - Notes from our October 2011 site visits:
>> > >> http://www.givewell.org/international/site-visits/october-2011
>> > >> - Audio and transcript from our conference calls:
>> > >> http://www.givewell.org/conference-call
>> > >>
>> > >> Best,
>> > >> Alexander
>> > >>
>> > >
>> > >
>>
>>
>   
>

Hi everyone,

On May 9th, we held a research event at our office in New York to discuss
updates on past top-rated charities (AMF, SCI, and VillageReach) as well as
our plans for research. Audio and transcript from the event are available
at http://givewell.org/conference-call. We'd be interested in any questions
or comments you have about the content.

In addition, we've recently posted:

   - An intervention report on non-therapeutic zinc supplementation:
   http://givewell.org/international/technical/programs/non-therapeutic-zinc
   - New documents from Nyaya Health, a standout charity:
   http://givewell.org/international/charities/Nyaya-Health#Update
   - New annotated budget projections from GiveDirectly, a standout
   charity:
   http://www.givewell.org/files/DWDA%202009/GiveDirectly/20120511%20Annotated%20GD%20Budget%20Projections.pdf
    (PDF)
   - New documents from Fistula Foundation:
   http://www.givewell.org/international/charities/Fistula-Foundation

Best,
Alexander


On Mon, Apr 30, 2012 at 10:12 AM, Alexander Berger
<alexander.is@...>wrote:

> Hi everyone,
>
> We've posted a new review of Give Directly (which continues to be a
> "standout" organization) at *
> http://www.givewell.org/international/charities/give-directly*.
>
> In addition, we've posted the following transcripts from conversations
> we've had with various organizations over the last few months:
>
> *Centre for Neglected Tropical Diseases*
>
>    - Moses Bockarie, Director<http://www.givewell.org/files/DWDA%202009/CNTD/Givewell%20phone%20conversation%20with%20CNTD.doc> on
>    March 26, 2012.
>
>  *Development Media International*
>
>    - Roy Head, CEO of Development Media International (DMI), Will Snell,
>    Head of Public Engagement & Development at DMI, and Cathryn Wood, Public
>    Engagement and Innovation Manager<http://www.givewell.org/files/DWDA%202009/DMI/Conversation%20with%20Giving%20What%20We%20Can%20and%20DMI.doc> on
>    April 17, 2012.
>
> *Doctors Without Borders (MSF)*
>
>    - Jennifer Tierney, Development Director, and Jason Cone,
>    Communications Director<http://www.givewell.org/files/DWDA%202009/MSF/MSF%20-%20GiveWell%202011%2009%2026.doc> on
>    September 26, 2011.
>    - Jennifer Tierney, Development Director<http://www.givewell.org/files/DWDA%202009/MSF/Conversation%20with%20MSF-USA%202012%2003%2008%20final.doc> on
>    March 8, 2012.
>
> *GAVI alliance*
>
>    - Seth Berkley, CEO<http://www.givewell.org/files/DWDA%202009/GAVI/gavi%202012%2001.doc> on
>    January 12, 2012.
>    - Alex Palacios, Special Representative to the U.S.<http://www.givewell.org/files/DWDA%202009/GAVI/Conversation%20with%20GAVI%202012%2002%2014.doc> on
>    February 14, 2012.
>
> *Measles Initiative*
>
>    - Peter Strebel, World Health Organization Department of Immunization,
>    Vaccines and Biologicals; Andrea Gay, United Nations Foundation Executive
>    Director of Children's Health; and Steve Cochi, CDC Center for Global
>    Health Senior Advisor to the Director<http://www.givewell.org/files/DWDA%202009/Measles%20Initiative/Conversation%20with%20Measles%20Initiative%202012%2004%2017%20final.doc> on
>    April 18, 2012.
>
> *TB REACH*
>
>    - Suvanand Sahu, Team Leader, and Jacob Creswell, Technical Officer<http://www.givewell.org/files/DWDA%202009/TB%20REACH/Conversation%20with%20TB%20Reach%202012%2003%2005%20final.doc> on
>    March 5, 2012.
>
> *William and Flora Hewlett Foundation*
>
>
>    - Paul Brest<http://www.givewell.org/files/conversations/paul-brest-march-2012.doc> in
>    March 2012.
>
>
> As always, please don't hesitate to reach out if you have any questions or
> comments.
>
> Best,
> Alexander
>
> On Mon, Mar 26, 2012 at 1:42 PM, Elie Hassenfeld <elie@...>wrote:
>
>> **
>>
>>
>> Hi Vipul,
>>
>> Thanks for the email and for continuing to follow up with us about
>> publishing our VillageReach update. Today, we've:
>>
>>    1. Published a summary of the update today on our blog:
>>    http://blog.givewell.org/2012/03/26/villagereach-update/
>>    2. A full, detailed update on our main site:
>>    http://givewell.org/international/top-charities/villagereach/updates#March262012update
>>    3. A guest post from John Beale at VillageReach providing more
>>    context for this update:
>>    http://blog.givewell.org/2012/03/26/guest-post-from-john-beale-at-villagereach
>>
>> We previously set a goal of publishing quarterly updates on VillageReach
>> (and all charities we rate high enough to direct them significant funding).
>> Today's VillageReach update is our first update on VillageReach since
>> August 2011, so we are publishing this update significantly later than we
>> had hoped. We will probably not be aiming for quarterly updates in the
>> future. To briefly explain the three main factors that led to this delay:
>>
>>    1. We were scheduled for a VillageReach update in early December
>>    2011, but we were primarily focused on our new recommendations and on
>>    giving season from October 1, 2011 - January 1, 2012. During that 3-month
>>    period, we focused our all efforts on our new recommendations and did not
>>    focus on VillageReach. (We feel this was the right decision.)
>>    2. When we began our reassessment of VillageReach in early 2012, we
>>    started re-analyzing the evidence of effectiveness from the pilot project.
>>    This analysis -- some of which we published recently at
>>    http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis --
>>    led to correspondence and conversations with VillageReach representatives
>>    about pilot project data rather than an update of VillageReach's work. In
>>    hindsight, it's clear that we should have prioritized publishing an update
>>    on VillageReach's work in Mozambique over revisiting the pilot project.
>>    3. In every case where we publish something about a charity, we send
>>    the charity a draft of our page before we publish it. We do this to
>>    fact-check our work and to give the organization an opportunity to comment
>>    on particular words/phrasing it finds problematic. In this case, extensive
>>    back and forth further delayed publication.
>>
>> Given the amount of work it takes to do an update and the pace at which
>> projects tend to move, we're now thinking that quarterly is too often. We
>> are rethinking the frequency we should aim for and will update on what we
>> decide.
>>
>> Best,
>> Elie
>>
>>
>> On Fri, Mar 23, 2012 at 2:04 PM, Vipul Naik <vipul@...icago.edu>wrote:
>>
>>> **
>>>
>>>
>>> Hi Alexander,
>>>
>>> Regarding VillageReach, is this the VillageReach update that you said
>>> you'd be publishing soon? The link you have given is to a re-analysis
>>> of the pilot project prior to GiveWell's recommendation of
>>> VillageReach, not of the expansion that was funded as a result of
>>> GiveWell's recommendation. Perhaps I am missing something here?
>>>
>>> Do you plan to post updates on the expansion that was funded as a
>>> result of GiveWell's recommendation? If so, when?
>>>
>>> Thank you,
>>>
>>> Vipul
>>>
>>> * Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri) wrote
>>> > Hello everyone,
>>> >
>>> > New/updated pages:
>>> >
>>> > - We've updated our review of the Small Enterprise Foundation:
>>> >
>>> http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
>>> > - We conducted a re-analysis of VillageReach's pilot project data:
>>> >
>>> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
>>> > - We published our first update on AMF's progress since becoming a top
>>> > charity: http://givewell.org/international/top-charities/amf/updates
>>> > - We published an update on KIPP Houston's room for more funding:
>>> > http://blog.givewell.org/2012/03/14/kipp-houston-update/
>>> > - We posted the audio and documents from our December board meeting:
>>>
>>> > http://givewell.org/about/official-records/board-meeting-17
>>> >
>>> > As usual, please feel free to let us know if you have any questions or
>>> > comments.
>>> >
>>> > Best,
>>> > Alexander
>>> >
>>> > On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <
>>> alexander.is@...>wrote:
>>> >
>>> > > Hello everyone,
>>> > >
>>> > > New/updated pages:
>>> > >
>>> > > - Our shortcomings page: http://givewell.org/about/shortcomings
>>> > > - Our intervention report on cash transfers:
>>>
>>> > > http://givewell.org/international/technical/programs/cash-transfers
>>> > >
>>> > > Best,
>>> > > Alexander
>>> > >
>>> > > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <
>>> alexander.is@gmail.com>wrote:
>>> > >
>>> > >> Hello everyone,
>>> > >>
>>> > >> A few people have told us that there's no way for GiveWell
>>> followers to
>>> > >> know when we publish new pages. Of course, people know when we
>>> update our
>>> > >> charity recommendations with new reviews, but when we publish other
>>> pages
>>> > >> (e.g., a review of a charity that's not top-rated or a page like the
>>> > >> Research FAQ, below), people don't know.
>>> > >>
>>> > >> Sending a list of links to this group is one way to address this
>>> issue.
>>> > >> This email includes new content (except blog posts) since our major
>>> > >> research update just after Thanksgiving, including the major pages
>>> we
>>> > >> published as part of the updated report.
>>> > >>
>>> > >> Our process:
>>> > >>
>>> > >> - 2011 international aid charities research process:
>>> > >> http://givewell.org/international/process/2011
>>> > >> - All charities considered to date:
>>> http://www.givewell.org/charities
>>> > >>
>>> > >> Related to AMF:
>>> > >>
>>> > >> - Our review of the Against Malaria Foundation:
>>> > >> http://givewell.org/international/top-charities/AMF
>>> > >> - Our intervention report on mass distribution of long-lasting
>>> > >> insecticide-treated nets:
>>> > >>
>>> http://givewell.org/international/technical/programs/insecticide-treated-nets
>>> > >> - Our discussion of the evidence on the effectiveness of large scale
>>>
>>> > >> ITN distribution campaigns:
>>> > >>
>>> http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
>>> > >>
>>> > >> Related to SCI:
>>> > >>
>>> > >> - Our review of the Schistosomiasis Control Initiative:
>>> > >>
>>> http://givewell.org/international/top-charities/schistosomiasis-control-initiative
>>> > >> - Our intervention report on combination deworming to kill
>>>
>>> > >> soil-transmitted helminths and schistosomiasis:
>>> > >> http://givewell.org/international/technical/programs/deworming
>>> > >>
>>> > >> Related to IPA:
>>> > >>
>>> > >> - Our review of Innovations for Poverty Action:
>>> > >> http://givewell.org/international/charities/ipa
>>> > >> - Our overview of the evidence created by and the influence of
>>>
>>> > >> notable IPA studies:
>>> > >> http://givewell.org/international/charities/ipa/research
>>> > >>
>>> > >> Other:
>>> > >>
>>> > >> - What others are saying about GiveWell:
>>>
>>> > >> http://givewell.org/what-others-are-saying (this is an update on a
>>> > >> previous "Press" page that we think presents GiveWell in a very
>>> positive
>>> > >> light; we'd like for more new visitors to the site to see this page)
>>> > >> - Frequently asked questions about our research:
>>> > >> http://www.givewell.org/about/FAQ/research
>>> > >> - Our review of International Development Enterprises:
>>> > >>
>>> http://givewell.org/International/charities/International-Development-Enterprises
>>> > >> - Notes from our October 2011 site visits:
>>> > >> http://www.givewell.org/international/site-visits/october-2011
>>> > >> - Audio and transcript from our conference calls:
>>> > >> http://www.givewell.org/conference-call
>>> > >>
>>> > >> Best,
>>> > >> Alexander
>>> > >>
>>> > >
>>> > >
>>>
>>>
>>   
>>
>
>

Hi Alex,

Just a quick question in regards to AMF that I couldn't find in the
transcript or on the website: how long do the nets last? Which is really
two questions: how long does the insecticide treatment last, and how long
do the nets themselves last physically? When that's answered, I'd wonder
what AMF's plan is for replacing/repairing. Not at all a pressing question,
just something I've thought about in considering the research. It might be
there and I just didn't see it.

best,
Brigid

On Tue, May 22, 2012 at 2:32 PM, Alexander Berger <alexander.is@...>wrote:

> **
>
>
> Hi everyone,
>
> On May 9th, we held a research event at our office in New York to discuss
> updates on past top-rated charities (AMF, SCI, and VillageReach) as well as
> our plans for research. Audio and transcript from the event are available
> at http://givewell.org/conference-call. We'd be interested in any
> questions or comments you have about the content.
>
> In addition, we've recently posted:
>
>    - An intervention report on non-therapeutic zinc supplementation:
>    http://givewell.org/international/technical/programs/non-therapeutic-zinc
>    - New documents from Nyaya Health, a standout charity:
>    http://givewell.org/international/charities/Nyaya-Health#Update
>    - New annotated budget projections from GiveDirectly, a standout
>    charity:
>    http://www.givewell.org/files/DWDA%202009/GiveDirectly/20120511%20Annotated%20GD%20Budget%20Projections.pdf
>     (PDF)
>    - New documents from Fistula Foundation:
>    http://www.givewell.org/international/charities/Fistula-Foundation
>
> Best,
> Alexander
>
>
> On Mon, Apr 30, 2012 at 10:12 AM, Alexander Berger <alexander.is@...
> > wrote:
>
>> Hi everyone,
>>
>> We've posted a new review of Give Directly (which continues to be a
>> "standout" organization) at *
>> http://www.givewell.org/international/charities/give-directly*.
>>
>> In addition, we've posted the following transcripts from conversations
>> we've had with various organizations over the last few months:
>>
>> *Centre for Neglected Tropical Diseases*
>>
>>    - Moses Bockarie, Director<http://www.givewell.org/files/DWDA%202009/CNTD/Givewell%20phone%20conversation%20with%20CNTD.doc> on
>>    March 26, 2012.
>>
>>  *Development Media International*
>>
>>    - Roy Head, CEO of Development Media International (DMI), Will Snell,
>>    Head of Public Engagement & Development at DMI, and Cathryn Wood, Public
>>    Engagement and Innovation Manager<http://www.givewell.org/files/DWDA%202009/DMI/Conversation%20with%20Giving%20What%20We%20Can%20and%20DMI.doc> on
>>    April 17, 2012.
>>
>> *Doctors Without Borders (MSF)*
>>
>>    - Jennifer Tierney, Development Director, and Jason Cone,
>>    Communications Director<http://www.givewell.org/files/DWDA%202009/MSF/MSF%20-%20GiveWell%202011%2009%2026.doc> on
>>    September 26, 2011.
>>    - Jennifer Tierney, Development Director<http://www.givewell.org/files/DWDA%202009/MSF/Conversation%20with%20MSF-USA%202012%2003%2008%20final.doc> on
>>    March 8, 2012.
>>
>> *GAVI alliance*
>>
>>    - Seth Berkley, CEO<http://www.givewell.org/files/DWDA%202009/GAVI/gavi%202012%2001.doc> on
>>    January 12, 2012.
>>    - Alex Palacios, Special Representative to the U.S.<http://www.givewell.org/files/DWDA%202009/GAVI/Conversation%20with%20GAVI%202012%2002%2014.doc> on
>>    February 14, 2012.
>>
>> *Measles Initiative*
>>
>>    - Peter Strebel, World Health Organization Department of
>>    Immunization, Vaccines and Biologicals; Andrea Gay, United Nations
>>    Foundation Executive Director of Children's Health; and Steve Cochi, CDC
>>    Center for Global Health Senior Advisor to the Director<http://www.givewell.org/files/DWDA%202009/Measles%20Initiative/Conversation%20with%20Measles%20Initiative%202012%2004%2017%20final.doc> on
>>    April 18, 2012.
>>
>> *TB REACH*
>>
>>    - Suvanand Sahu, Team Leader, and Jacob Creswell, Technical Officer<http://www.givewell.org/files/DWDA%202009/TB%20REACH/Conversation%20with%20TB%20Reach%202012%2003%2005%20final.doc> on
>>    March 5, 2012.
>>
>> *William and Flora Hewlett Foundation*
>>
>>
>>    - Paul Brest<http://www.givewell.org/files/conversations/paul-brest-march-2012.doc> in
>>    March 2012.
>>
>>
>> As always, please don't hesitate to reach out if you have any questions
>> or comments.
>>
>> Best,
>> Alexander
>>
>> On Mon, Mar 26, 2012 at 1:42 PM, Elie Hassenfeld <elie@...>wrote:
>>
>>> **
>>>
>>>
>>> Hi Vipul,
>>>
>>> Thanks for the email and for continuing to follow up with us about
>>> publishing our VillageReach update. Today, we've:
>>>
>>>    1. Published a summary of the update today on our blog:
>>>    http://blog.givewell.org/2012/03/26/villagereach-update/
>>>    2. A full, detailed update on our main site:
>>>    http://givewell.org/international/top-charities/villagereach/updates#March262012update
>>>    3. A guest post from John Beale at VillageReach providing more
>>>    context for this update:
>>>    http://blog.givewell.org/2012/03/26/guest-post-from-john-beale-at-villagereach
>>>
>>> We previously set a goal of publishing quarterly updates on VillageReach
>>> (and all charities we rate high enough to direct them significant funding).
>>> Today's VillageReach update is our first update on VillageReach since
>>> August 2011, so we are publishing this update significantly later than we
>>> had hoped. We will probably not be aiming for quarterly updates in the
>>> future. To briefly explain the three main factors that led to this delay:
>>>
>>>    1. We were scheduled for a VillageReach update in early December
>>>    2011, but we were primarily focused on our new recommendations and on
>>>    giving season from October 1, 2011 - January 1, 2012. During that 3-month
>>>    period, we focused our all efforts on our new recommendations and did not
>>>    focus on VillageReach. (We feel this was the right decision.)
>>>    2. When we began our reassessment of VillageReach in early 2012, we
>>>    started re-analyzing the evidence of effectiveness from the pilot project.
>>>    This analysis -- some of which we published recently at
>>>    http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis --
>>>    led to correspondence and conversations with VillageReach representatives
>>>    about pilot project data rather than an update of VillageReach's work. In
>>>    hindsight, it's clear that we should have prioritized publishing an update
>>>    on VillageReach's work in Mozambique over revisiting the pilot project.
>>>    3. In every case where we publish something about a charity, we send
>>>    the charity a draft of our page before we publish it. We do this to
>>>    fact-check our work and to give the organization an opportunity to comment
>>>    on particular words/phrasing it finds problematic. In this case, extensive
>>>    back and forth further delayed publication.
>>>
>>> Given the amount of work it takes to do an update and the pace at which
>>> projects tend to move, we're now thinking that quarterly is too often. We
>>> are rethinking the frequency we should aim for and will update on what we
>>> decide.
>>>
>>> Best,
>>> Elie
>>>
>>>
>>> On Fri, Mar 23, 2012 at 2:04 PM, Vipul Naik <vipul@math.uchicago.edu>wrote:
>>>
>>>> **
>>>>
>>>>
>>>> Hi Alexander,
>>>>
>>>> Regarding VillageReach, is this the VillageReach update that you said
>>>> you'd be publishing soon? The link you have given is to a re-analysis
>>>> of the pilot project prior to GiveWell's recommendation of
>>>> VillageReach, not of the expansion that was funded as a result of
>>>> GiveWell's recommendation. Perhaps I am missing something here?
>>>>
>>>> Do you plan to post updates on the expansion that was funded as a
>>>> result of GiveWell's recommendation? If so, when?
>>>>
>>>> Thank you,
>>>>
>>>> Vipul
>>>>
>>>> * Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri) wrote
>>>> > Hello everyone,
>>>> >
>>>> > New/updated pages:
>>>> >
>>>> > - We've updated our review of the Small Enterprise Foundation:
>>>> >
>>>> http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
>>>> > - We conducted a re-analysis of VillageReach's pilot project data:
>>>> >
>>>> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
>>>> > - We published our first update on AMF's progress since becoming a top
>>>> > charity: http://givewell.org/international/top-charities/amf/updates
>>>> > - We published an update on KIPP Houston's room for more funding:
>>>> > http://blog.givewell.org/2012/03/14/kipp-houston-update/
>>>> > - We posted the audio and documents from our December board meeting:
>>>>
>>>> > http://givewell.org/about/official-records/board-meeting-17
>>>> >
>>>> > As usual, please feel free to let us know if you have any questions or
>>>> > comments.
>>>> >
>>>> > Best,
>>>> > Alexander
>>>> >
>>>> > On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <
>>>> alexander.is@...>wrote:
>>>> >
>>>> > > Hello everyone,
>>>> > >
>>>> > > New/updated pages:
>>>> > >
>>>> > > - Our shortcomings page: http://givewell.org/about/shortcomings
>>>> > > - Our intervention report on cash transfers:
>>>>
>>>> > > http://givewell.org/international/technical/programs/cash-transfers
>>>> > >
>>>> > > Best,
>>>> > > Alexander
>>>> > >
>>>> > > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <
>>>> alexander.is@...>wrote:
>>>> > >
>>>> > >> Hello everyone,
>>>> > >>
>>>> > >> A few people have told us that there's no way for GiveWell
>>>> followers to
>>>> > >> know when we publish new pages. Of course, people know when we
>>>> update our
>>>> > >> charity recommendations with new reviews, but when we publish
>>>> other pages
>>>> > >> (e.g., a review of a charity that's not top-rated or a page like
>>>> the
>>>> > >> Research FAQ, below), people don't know.
>>>> > >>
>>>> > >> Sending a list of links to this group is one way to address this
>>>> issue.
>>>> > >> This email includes new content (except blog posts) since our major
>>>> > >> research update just after Thanksgiving, including the major pages
>>>> we
>>>> > >> published as part of the updated report.
>>>> > >>
>>>> > >> Our process:
>>>> > >>
>>>> > >> - 2011 international aid charities research process:
>>>> > >> http://givewell.org/international/process/2011
>>>> > >> - All charities considered to date:
>>>> http://www.givewell.org/charities
>>>> > >>
>>>> > >> Related to AMF:
>>>> > >>
>>>> > >> - Our review of the Against Malaria Foundation:
>>>> > >> http://givewell.org/international/top-charities/AMF
>>>> > >> - Our intervention report on mass distribution of long-lasting
>>>> > >> insecticide-treated nets:
>>>> > >>
>>>> http://givewell.org/international/technical/programs/insecticide-treated-nets
>>>> > >> - Our discussion of the evidence on the effectiveness of large
>>>> scale
>>>>
>>>> > >> ITN distribution campaigns:
>>>> > >>
>>>> http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
>>>> > >>
>>>> > >> Related to SCI:
>>>> > >>
>>>> > >> - Our review of the Schistosomiasis Control Initiative:
>>>> > >>
>>>> http://givewell.org/international/top-charities/schistosomiasis-control-initiative
>>>> > >> - Our intervention report on combination deworming to kill
>>>>
>>>> > >> soil-transmitted helminths and schistosomiasis:
>>>> > >> http://givewell.org/international/technical/programs/deworming
>>>> > >>
>>>> > >> Related to IPA:
>>>> > >>
>>>> > >> - Our review of Innovations for Poverty Action:
>>>> > >> http://givewell.org/international/charities/ipa
>>>> > >> - Our overview of the evidence created by and the influence of
>>>>
>>>> > >> notable IPA studies:
>>>> > >> http://givewell.org/international/charities/ipa/research
>>>> > >>
>>>> > >> Other:
>>>> > >>
>>>> > >> - What others are saying about GiveWell:
>>>>
>>>> > >> http://givewell.org/what-others-are-saying (this is an update on a
>>>> > >> previous "Press" page that we think presents GiveWell in a very
>>>> positive
>>>> > >> light; we'd like for more new visitors to the site to see this
>>>> page)
>>>> > >> - Frequently asked questions about our research:
>>>> > >> http://www.givewell.org/about/FAQ/research
>>>> > >> - Our review of International Development Enterprises:
>>>> > >>
>>>> http://givewell.org/International/charities/International-Development-Enterprises
>>>> > >> - Notes from our October 2011 site visits:
>>>> > >> http://www.givewell.org/international/site-visits/october-2011
>>>> > >> - Audio and transcript from our conference calls:
>>>> > >> http://www.givewell.org/conference-call
>>>> > >>
>>>> > >> Best,
>>>> > >> Alexander
>>>> > >>
>>>> > >
>>>> > >
>>>>
>>>>
>>>
>>
>  
>



-- 
Brigid Slipka
brigid.slipka@...
(323) 702-5017

Hi Brigid,

Thanks for the question. Net manufacturers claim an effective lifespan for
their ITNs of 5 years, but we assume an average lifespan of 2.22 years
under typical usage conditions. We discuss the grounds for this assumption
at
http://givewell.org/international/technical/programs/insecticide-treated-nets#222
(see
footnotes 31-33 on that page for more detail of the calculation).

AMF doesn't haven any particular plans to repare or replace the particular
nets it distributes. They're currently working as part of big national
campaigns, which occur periodically. Countries plan national campaigns
based on the timing of previous campaigns and the assumption above about
when nets will wear out.

Thanks,
Alexander

On Tue, May 22, 2012 at 4:00 PM, Brigid Slipka <brigid.slipka@...>wrote:

> **
>
>
> Hi Alex,
>
> Just a quick question in regards to AMF that I couldn't find in the
> transcript or on the website: how long do the nets last? Which is really
> two questions: how long does the insecticide treatment last, and how long
> do the nets themselves last physically? When that's answered, I'd wonder
> what AMF's plan is for replacing/repairing. Not at all a pressing question,
> just something I've thought about in considering the research. It might be
> there and I just didn't see it.
>
> best,
> Brigid
>
> On Tue, May 22, 2012 at 2:32 PM, Alexander Berger <alexander.is@...>wrote:
>
>> **
>>
>>
>> Hi everyone,
>>
>> On May 9th, we held a research event at our office in New York to discuss
>> updates on past top-rated charities (AMF, SCI, and VillageReach) as well as
>> our plans for research. Audio and transcript from the event are available
>> at http://givewell.org/conference-call. We'd be interested in any
>> questions or comments you have about the content.
>>
>> In addition, we've recently posted:
>>
>>    - An intervention report on non-therapeutic zinc supplementation:
>>    http://givewell.org/international/technical/programs/non-therapeutic-zinc
>>    - New documents from Nyaya Health, a standout charity:
>>    http://givewell.org/international/charities/Nyaya-Health#Update
>>    - New annotated budget projections from GiveDirectly, a standout
>>    charity:
>>    http://www.givewell.org/files/DWDA%202009/GiveDirectly/20120511%20Annotated%20GD%20Budget%20Projections.pdf
>>     (PDF)
>>    - New documents from Fistula Foundation:
>>    http://www.givewell.org/international/charities/Fistula-Foundation
>>
>> Best,
>> Alexander
>>
>>
>> On Mon, Apr 30, 2012 at 10:12 AM, Alexander Berger <
>> alexander.is@...> wrote:
>>
>>> Hi everyone,
>>>
>>> We've posted a new review of Give Directly (which continues to be a
>>> "standout" organization) at *
>>> http://www.givewell.org/international/charities/give-directly*.
>>>
>>> In addition, we've posted the following transcripts from conversations
>>> we've had with various organizations over the last few months:
>>>
>>> *Centre for Neglected Tropical Diseases*
>>>
>>>    - Moses Bockarie, Director<http://www.givewell.org/files/DWDA%202009/CNTD/Givewell%20phone%20conversation%20with%20CNTD.doc> on
>>>    March 26, 2012.
>>>
>>>  *Development Media International*
>>>
>>>    - Roy Head, CEO of Development Media International (DMI), Will
>>>    Snell, Head of Public Engagement & Development at DMI, and Cathryn Wood,
>>>    Public Engagement and Innovation Manager<http://www.givewell.org/files/DWDA%202009/DMI/Conversation%20with%20Giving%20What%20We%20Can%20and%20DMI.doc> on
>>>    April 17, 2012.
>>>
>>> *Doctors Without Borders (MSF)*
>>>
>>>    - Jennifer Tierney, Development Director, and Jason Cone,
>>>    Communications Director<http://www.givewell.org/files/DWDA%202009/MSF/MSF%20-%20GiveWell%202011%2009%2026.doc> on
>>>    September 26, 2011.
>>>    - Jennifer Tierney, Development Director<http://www.givewell.org/files/DWDA%202009/MSF/Conversation%20with%20MSF-USA%202012%2003%2008%20final.doc> on
>>>    March 8, 2012.
>>>
>>> *GAVI alliance*
>>>
>>>    - Seth Berkley, CEO<http://www.givewell.org/files/DWDA%202009/GAVI/gavi%202012%2001.doc> on
>>>    January 12, 2012.
>>>    - Alex Palacios, Special Representative to the U.S.<http://www.givewell.org/files/DWDA%202009/GAVI/Conversation%20with%20GAVI%202012%2002%2014.doc> on
>>>    February 14, 2012.
>>>
>>> *Measles Initiative*
>>>
>>>    - Peter Strebel, World Health Organization Department of
>>>    Immunization, Vaccines and Biologicals; Andrea Gay, United Nations
>>>    Foundation Executive Director of Children's Health; and Steve Cochi, CDC
>>>    Center for Global Health Senior Advisor to the Director<http://www.givewell.org/files/DWDA%202009/Measles%20Initiative/Conversation%20with%20Measles%20Initiative%202012%2004%2017%20final.doc> on
>>>    April 18, 2012.
>>>
>>> *TB REACH*
>>>
>>>    - Suvanand Sahu, Team Leader, and Jacob Creswell, Technical Officer<http://www.givewell.org/files/DWDA%202009/TB%20REACH/Conversation%20with%20TB%20Reach%202012%2003%2005%20final.doc> on
>>>    March 5, 2012.
>>>
>>> *William and Flora Hewlett Foundation*
>>>
>>>
>>>    - Paul Brest<http://www.givewell.org/files/conversations/paul-brest-march-2012.doc> in
>>>    March 2012.
>>>
>>>
>>> As always, please don't hesitate to reach out if you have any questions
>>> or comments.
>>>
>>> Best,
>>> Alexander
>>>
>>> On Mon, Mar 26, 2012 at 1:42 PM, Elie Hassenfeld <elie@...>wrote:
>>>
>>>> **
>>>>
>>>>
>>>> Hi Vipul,
>>>>
>>>> Thanks for the email and for continuing to follow up with us about
>>>> publishing our VillageReach update. Today, we've:
>>>>
>>>>    1. Published a summary of the update today on our blog:
>>>>    http://blog.givewell.org/2012/03/26/villagereach-update/
>>>>    2. A full, detailed update on our main site:
>>>>    http://givewell.org/international/top-charities/villagereach/updates#March262012update
>>>>    3. A guest post from John Beale at VillageReach providing more
>>>>    context for this update:
>>>>    http://blog.givewell.org/2012/03/26/guest-post-from-john-beale-at-villagereach
>>>>
>>>> We previously set a goal of publishing quarterly updates on
>>>> VillageReach (and all charities we rate high enough to direct them
>>>> significant funding). Today's VillageReach update is our first update on
>>>> VillageReach since August 2011, so we are publishing this update
>>>> significantly later than we had hoped. We will probably not be aiming for
>>>> quarterly updates in the future. To briefly explain the three main factors
>>>> that led to this delay:
>>>>
>>>>    1. We were scheduled for a VillageReach update in early December
>>>>    2011, but we were primarily focused on our new recommendations and on
>>>>    giving season from October 1, 2011 - January 1, 2012. During that 3-month
>>>>    period, we focused our all efforts on our new recommendations and did not
>>>>    focus on VillageReach. (We feel this was the right decision.)
>>>>    2. When we began our reassessment of VillageReach in early 2012, we
>>>>    started re-analyzing the evidence of effectiveness from the pilot project.
>>>>    This analysis -- some of which we published recently at
>>>>    http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis --
>>>>    led to correspondence and conversations with VillageReach representatives
>>>>    about pilot project data rather than an update of VillageReach's work. In
>>>>    hindsight, it's clear that we should have prioritized publishing an update
>>>>    on VillageReach's work in Mozambique over revisiting the pilot project.
>>>>    3. In every case where we publish something about a charity, we
>>>>    send the charity a draft of our page before we publish it. We do this to
>>>>    fact-check our work and to give the organization an opportunity to comment
>>>>    on particular words/phrasing it finds problematic. In this case, extensive
>>>>    back and forth further delayed publication.
>>>>
>>>> Given the amount of work it takes to do an update and the pace at which
>>>> projects tend to move, we're now thinking that quarterly is too often. We
>>>> are rethinking the frequency we should aim for and will update on what we
>>>> decide.
>>>>
>>>> Best,
>>>> Elie
>>>>
>>>>
>>>> On Fri, Mar 23, 2012 at 2:04 PM, Vipul Naik <vipul@...>wrote:
>>>>
>>>>> **
>>>>>
>>>>>
>>>>> Hi Alexander,
>>>>>
>>>>> Regarding VillageReach, is this the VillageReach update that you said
>>>>> you'd be publishing soon? The link you have given is to a re-analysis
>>>>> of the pilot project prior to GiveWell's recommendation of
>>>>> VillageReach, not of the expansion that was funded as a result of
>>>>> GiveWell's recommendation. Perhaps I am missing something here?
>>>>>
>>>>> Do you plan to post updates on the expansion that was funded as a
>>>>> result of GiveWell's recommendation? If so, when?
>>>>>
>>>>> Thank you,
>>>>>
>>>>> Vipul
>>>>>
>>>>> * Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri) wrote
>>>>> > Hello everyone,
>>>>> >
>>>>> > New/updated pages:
>>>>> >
>>>>> > - We've updated our review of the Small Enterprise Foundation:
>>>>> >
>>>>> http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
>>>>> > - We conducted a re-analysis of VillageReach's pilot project data:
>>>>> >
>>>>> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
>>>>> > - We published our first update on AMF's progress since becoming a
>>>>> top
>>>>> > charity: http://givewell.org/international/top-charities/amf/updates
>>>>> > - We published an update on KIPP Houston's room for more funding:
>>>>> > http://blog.givewell.org/2012/03/14/kipp-houston-update/
>>>>> > - We posted the audio and documents from our December board meeting:
>>>>>
>>>>> > http://givewell.org/about/official-records/board-meeting-17
>>>>> >
>>>>> > As usual, please feel free to let us know if you have any questions
>>>>> or
>>>>> > comments.
>>>>> >
>>>>> > Best,
>>>>> > Alexander
>>>>> >
>>>>> > On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <
>>>>> alexander.is@...>wrote:
>>>>> >
>>>>> > > Hello everyone,
>>>>> > >
>>>>> > > New/updated pages:
>>>>> > >
>>>>> > > - Our shortcomings page: http://givewell.org/about/shortcomings
>>>>> > > - Our intervention report on cash transfers:
>>>>>
>>>>> > >
>>>>> http://givewell.org/international/technical/programs/cash-transfers
>>>>> > >
>>>>> > > Best,
>>>>> > > Alexander
>>>>> > >
>>>>> > > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <
>>>>> alexander.is@...>wrote:
>>>>> > >
>>>>> > >> Hello everyone,
>>>>> > >>
>>>>> > >> A few people have told us that there's no way for GiveWell
>>>>> followers to
>>>>> > >> know when we publish new pages. Of course, people know when we
>>>>> update our
>>>>> > >> charity recommendations with new reviews, but when we publish
>>>>> other pages
>>>>> > >> (e.g., a review of a charity that's not top-rated or a page like
>>>>> the
>>>>> > >> Research FAQ, below), people don't know.
>>>>> > >>
>>>>> > >> Sending a list of links to this group is one way to address this
>>>>> issue.
>>>>> > >> This email includes new content (except blog posts) since our
>>>>> major
>>>>> > >> research update just after Thanksgiving, including the major
>>>>> pages we
>>>>> > >> published as part of the updated report.
>>>>> > >>
>>>>> > >> Our process:
>>>>> > >>
>>>>> > >> - 2011 international aid charities research process:
>>>>> > >> http://givewell.org/international/process/2011
>>>>> > >> - All charities considered to date:
>>>>> http://www.givewell.org/charities
>>>>> > >>
>>>>> > >> Related to AMF:
>>>>> > >>
>>>>> > >> - Our review of the Against Malaria Foundation:
>>>>> > >> http://givewell.org/international/top-charities/AMF
>>>>> > >> - Our intervention report on mass distribution of long-lasting
>>>>> > >> insecticide-treated nets:
>>>>> > >>
>>>>> http://givewell.org/international/technical/programs/insecticide-treated-nets
>>>>> > >> - Our discussion of the evidence on the effectiveness of large
>>>>> scale
>>>>>
>>>>> > >> ITN distribution campaigns:
>>>>> > >>
>>>>> http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
>>>>> > >>
>>>>> > >> Related to SCI:
>>>>> > >>
>>>>> > >> - Our review of the Schistosomiasis Control Initiative:
>>>>> > >>
>>>>> http://givewell.org/international/top-charities/schistosomiasis-control-initiative
>>>>> > >> - Our intervention report on combination deworming to kill
>>>>>
>>>>> > >> soil-transmitted helminths and schistosomiasis:
>>>>> > >> http://givewell.org/international/technical/programs/deworming
>>>>> > >>
>>>>> > >> Related to IPA:
>>>>> > >>
>>>>> > >> - Our review of Innovations for Poverty Action:
>>>>> > >> http://givewell.org/international/charities/ipa
>>>>> > >> - Our overview of the evidence created by and the influence of
>>>>>
>>>>> > >> notable IPA studies:
>>>>> > >> http://givewell.org/international/charities/ipa/research
>>>>> > >>
>>>>> > >> Other:
>>>>> > >>
>>>>> > >> - What others are saying about GiveWell:
>>>>>
>>>>> > >> http://givewell.org/what-others-are-saying (this is an update on
>>>>> a
>>>>> > >> previous "Press" page that we think presents GiveWell in a very
>>>>> positive
>>>>> > >> light; we'd like for more new visitors to the site to see this
>>>>> page)
>>>>> > >> - Frequently asked questions about our research:
>>>>> > >> http://www.givewell.org/about/FAQ/research
>>>>> > >> - Our review of International Development Enterprises:
>>>>> > >>
>>>>> http://givewell.org/International/charities/International-Development-Enterprises
>>>>> > >> - Notes from our October 2011 site visits:
>>>>> > >> http://www.givewell.org/international/site-visits/october-2011
>>>>> > >> - Audio and transcript from our conference calls:
>>>>> > >> http://www.givewell.org/conference-call
>>>>> > >>
>>>>> > >> Best,
>>>>> > >> Alexander
>>>>> > >>
>>>>> > >
>>>>> > >
>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> Brigid Slipka
> brigid.slipka@...
> (323) 702-5017
>
>  
>

Hi Alex,

On the Nyaya Health page, the "updated" date at the top (just above
the "Donate Now" button) says November 28, 2011, without any
indication that you have a May 2012 update. Perhaps you should modify
that date?

Vipul

* Quoting Alexander Berger who at 2012-05-22 14:32:19+0000 (Tue) wrote
> Hi everyone,
> 
> On May 9th, we held a research event at our office in New York to discuss
> updates on past top-rated charities (AMF, SCI, and VillageReach) as well as
> our plans for research. Audio and transcript from the event are available
> at http://givewell.org/conference-call. We'd be interested in any questions
> or comments you have about the content.
> 
> In addition, we've recently posted:
> 
>    - An intervention report on non-therapeutic zinc supplementation:
>    http://givewell.org/international/technical/programs/non-therapeutic-zinc
>    - New documents from Nyaya Health, a standout charity:
>    http://givewell.org/international/charities/Nyaya-Health#Update
>    - New annotated budget projections from GiveDirectly, a standout
>    charity:
>    http://www.givewell.org/files/DWDA%202009/GiveDirectly/20120511%20Annotated%20GD%20Budget%20Projections.pdf
>     (PDF)
>    - New documents from Fistula Foundation:
>    http://www.givewell.org/international/charities/Fistula-Foundation
> 
> Best,
> Alexander
> 
> 
> On Mon, Apr 30, 2012 at 10:12 AM, Alexander Berger
> <alexander.is@...>wrote:
> 
> > Hi everyone,
> >
> > We've posted a new review of Give Directly (which continues to be a
> > "standout" organization) at *
> > http://www.givewell.org/international/charities/give-directly*.
> >
> > In addition, we've posted the following transcripts from conversations
> > we've had with various organizations over the last few months:
> >
> > *Centre for Neglected Tropical Diseases*
> >
> >    - Moses Bockarie, Director<http://www.givewell.org/files/DWDA%202009/CNTD/Givewell%20phone%20conversation%20with%20CNTD.doc> on
> >    March 26, 2012.
> >
> >  *Development Media International*
> >
> >    - Roy Head, CEO of Development Media International (DMI), Will Snell,
> >    Head of Public Engagement & Development at DMI, and Cathryn Wood, Public
> >    Engagement and Innovation Manager<http://www.givewell.org/files/DWDA%202009/DMI/Conversation%20with%20Giving%20What%20We%20Can%20and%20DMI.doc> on
> >    April 17, 2012.
> >
> > *Doctors Without Borders (MSF)*
> >
> >    - Jennifer Tierney, Development Director, and Jason Cone,
> >    Communications Director<http://www.givewell.org/files/DWDA%202009/MSF/MSF%20-%20GiveWell%202011%2009%2026.doc> on
> >    September 26, 2011.
> >    - Jennifer Tierney, Development Director<http://www.givewell.org/files/DWDA%202009/MSF/Conversation%20with%20MSF-USA%202012%2003%2008%20final.doc> on
> >    March 8, 2012.
> >
> > *GAVI alliance*
> >
> >    - Seth Berkley, CEO<http://www.givewell.org/files/DWDA%202009/GAVI/gavi%202012%2001.doc> on
> >    January 12, 2012.
> >    - Alex Palacios, Special Representative to the U.S.<http://www.givewell.org/files/DWDA%202009/GAVI/Conversation%20with%20GAVI%202012%2002%2014.doc> on
> >    February 14, 2012.
> >
> > *Measles Initiative*
> >
> >    - Peter Strebel, World Health Organization Department of Immunization,
> >    Vaccines and Biologicals; Andrea Gay, United Nations Foundation Executive
> >    Director of Children's Health; and Steve Cochi, CDC Center for Global
> >    Health Senior Advisor to the Director<http://www.givewell.org/files/DWDA%202009/Measles%20Initiative/Conversation%20with%20Measles%20Initiative%202012%2004%2017%20final.doc> on
> >    April 18, 2012.
> >
> > *TB REACH*
> >
> >    - Suvanand Sahu, Team Leader, and Jacob Creswell, Technical Officer<http://www.givewell.org/files/DWDA%202009/TB%20REACH/Conversation%20with%20TB%20Reach%202012%2003%2005%20final.doc> on
> >    March 5, 2012.
> >
> > *William and Flora Hewlett Foundation*
> >
> >
> >    - Paul Brest<http://www.givewell.org/files/conversations/paul-brest-march-2012.doc> in
> >    March 2012.
> >
> >
> > As always, please don't hesitate to reach out if you have any questions or
> > comments.
> >
> > Best,
> > Alexander
> >
> > On Mon, Mar 26, 2012 at 1:42 PM, Elie Hassenfeld <elie@...>wrote:
> >
> >> **
> >>
> >>
> >> Hi Vipul,
> >>
> >> Thanks for the email and for continuing to follow up with us about
> >> publishing our VillageReach update. Today, we've:
> >>
> >>    1. Published a summary of the update today on our blog:
> >>    http://blog.givewell.org/2012/03/26/villagereach-update/
> >>    2. A full, detailed update on our main site:
> >>    http://givewell.org/international/top-charities/villagereach/updates#March262012update
> >>    3. A guest post from John Beale at VillageReach providing more
> >>    context for this update:
> >>    http://blog.givewell.org/2012/03/26/guest-post-from-john-beale-at-villagereach
> >>
> >> We previously set a goal of publishing quarterly updates on VillageReach
> >> (and all charities we rate high enough to direct them significant funding).
> >> Today's VillageReach update is our first update on VillageReach since
> >> August 2011, so we are publishing this update significantly later than we
> >> had hoped. We will probably not be aiming for quarterly updates in the
> >> future. To briefly explain the three main factors that led to this delay:
> >>
> >>    1. We were scheduled for a VillageReach update in early December
> >>    2011, but we were primarily focused on our new recommendations and on
> >>    giving season from October 1, 2011 - January 1, 2012. During that 3-month
> >>    period, we focused our all efforts on our new recommendations and did not
> >>    focus on VillageReach. (We feel this was the right decision.)
> >>    2. When we began our reassessment of VillageReach in early 2012, we
> >>    started re-analyzing the evidence of effectiveness from the pilot project.
> >>    This analysis -- some of which we published recently at
> >>    http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis --
> >>    led to correspondence and conversations with VillageReach representatives
> >>    about pilot project data rather than an update of VillageReach's work. In
> >>    hindsight, it's clear that we should have prioritized publishing an update
> >>    on VillageReach's work in Mozambique over revisiting the pilot project.
> >>    3. In every case where we publish something about a charity, we send
> >>    the charity a draft of our page before we publish it. We do this to
> >>    fact-check our work and to give the organization an opportunity to comment
> >>    on particular words/phrasing it finds problematic. In this case, extensive
> >>    back and forth further delayed publication.
> >>
> >> Given the amount of work it takes to do an update and the pace at which
> >> projects tend to move, we're now thinking that quarterly is too often. We
> >> are rethinking the frequency we should aim for and will update on what we
> >> decide.
> >>
> >> Best,
> >> Elie
> >>
> >>
> >> On Fri, Mar 23, 2012 at 2:04 PM, Vipul Naik <vipul@...>wrote:
> >>
> >>> **
> >>>
> >>>
> >>> Hi Alexander,
> >>>
> >>> Regarding VillageReach, is this the VillageReach update that you said
> >>> you'd be publishing soon? The link you have given is to a re-analysis
> >>> of the pilot project prior to GiveWell's recommendation of
> >>> VillageReach, not of the expansion that was funded as a result of
> >>> GiveWell's recommendation. Perhaps I am missing something here?
> >>>
> >>> Do you plan to post updates on the expansion that was funded as a
> >>> result of GiveWell's recommendation? If so, when?
> >>>
> >>> Thank you,
> >>>
> >>> Vipul
> >>>
> >>> * Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri) wrote
> >>> > Hello everyone,
> >>> >
> >>> > New/updated pages:
> >>> >
> >>> > - We've updated our review of the Small Enterprise Foundation:
> >>> >
> >>> http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
> >>> > - We conducted a re-analysis of VillageReach's pilot project data:
> >>> >
> >>> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
> >>> > - We published our first update on AMF's progress since becoming a top
> >>> > charity: http://givewell.org/international/top-charities/amf/updates
> >>> > - We published an update on KIPP Houston's room for more funding:
> >>> > http://blog.givewell.org/2012/03/14/kipp-houston-update/
> >>> > - We posted the audio and documents from our December board meeting:
> >>>
> >>> > http://givewell.org/about/official-records/board-meeting-17
> >>> >
> >>> > As usual, please feel free to let us know if you have any questions or
> >>> > comments.
> >>> >
> >>> > Best,
> >>> > Alexander
> >>> >
> >>> > On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <
> >>> alexander.is@...>wrote:
> >>> >
> >>> > > Hello everyone,
> >>> > >
> >>> > > New/updated pages:
> >>> > >
> >>> > > - Our shortcomings page: http://givewell.org/about/shortcomings
> >>> > > - Our intervention report on cash transfers:
> >>>
> >>> > > http://givewell.org/international/technical/programs/cash-transfers
> >>> > >
> >>> > > Best,
> >>> > > Alexander
> >>> > >
> >>> > > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <
> >>> alexander.is@...>wrote:
> >>> > >
> >>> > >> Hello everyone,
> >>> > >>
> >>> > >> A few people have told us that there's no way for GiveWell
> >>> followers to
> >>> > >> know when we publish new pages. Of course, people know when we
> >>> update our
> >>> > >> charity recommendations with new reviews, but when we publish other
> >>> pages
> >>> > >> (e.g., a review of a charity that's not top-rated or a page like the
> >>> > >> Research FAQ, below), people don't know.
> >>> > >>
> >>> > >> Sending a list of links to this group is one way to address this
> >>> issue.
> >>> > >> This email includes new content (except blog posts) since our major
> >>> > >> research update just after Thanksgiving, including the major pages
> >>> we
> >>> > >> published as part of the updated report.
> >>> > >>
> >>> > >> Our process:
> >>> > >>
> >>> > >> - 2011 international aid charities research process:
> >>> > >> http://givewell.org/international/process/2011
> >>> > >> - All charities considered to date:
> >>> http://www.givewell.org/charities
> >>> > >>
> >>> > >> Related to AMF:
> >>> > >>
> >>> > >> - Our review of the Against Malaria Foundation:
> >>> > >> http://givewell.org/international/top-charities/AMF
> >>> > >> - Our intervention report on mass distribution of long-lasting
> >>> > >> insecticide-treated nets:
> >>> > >>
> >>> http://givewell.org/international/technical/programs/insecticide-treated-nets
> >>> > >> - Our discussion of the evidence on the effectiveness of large scale
> >>>
> >>> > >> ITN distribution campaigns:
> >>> > >>
> >>> http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
> >>> > >>
> >>> > >> Related to SCI:
> >>> > >>
> >>> > >> - Our review of the Schistosomiasis Control Initiative:
> >>> > >>
> >>> http://givewell.org/international/top-charities/schistosomiasis-control-initiative
> >>> > >> - Our intervention report on combination deworming to kill
> >>>
> >>> > >> soil-transmitted helminths and schistosomiasis:
> >>> > >> http://givewell.org/international/technical/programs/deworming
> >>> > >>
> >>> > >> Related to IPA:
> >>> > >>
> >>> > >> - Our review of Innovations for Poverty Action:
> >>> > >> http://givewell.org/international/charities/ipa
> >>> > >> - Our overview of the evidence created by and the influence of
> >>>
> >>> > >> notable IPA studies:
> >>> > >> http://givewell.org/international/charities/ipa/research
> >>> > >>
> >>> > >> Other:
> >>> > >>
> >>> > >> - What others are saying about GiveWell:
> >>>
> >>> > >> http://givewell.org/what-others-are-saying (this is an update on a
> >>> > >> previous "Press" page that we think presents GiveWell in a very
> >>> positive
> >>> > >> light; we'd like for more new visitors to the site to see this page)
> >>> > >> - Frequently asked questions about our research:
> >>> > >> http://www.givewell.org/about/FAQ/research
> >>> > >> - Our review of International Development Enterprises:
> >>> > >>
> >>> http://givewell.org/International/charities/International-Development-Enterprises
> >>> > >> - Notes from our October 2011 site visits:
> >>> > >> http://www.givewell.org/international/site-visits/october-2011
> >>> > >> - Audio and transcript from our conference calls:
> >>> > >> http://www.givewell.org/conference-call
> >>> > >>
> >>> > >> Best,
> >>> > >> Alexander
> >>> > >>
> >>> > >
> >>> > >
> >>>
> >>>
> >>   
> >>
> >
> >

Hi Vipul,

Thanks for the suggestion. Given that we haven't actually updated any of
the contents of the page other than posting the new docs, we think that
keeping the old update date is the right call. In particular, our
conclusions and calculations are current as of the November 2011 date, not
reflecting the new documents.

Best,
Alexander

On Wed, May 23, 2012 at 11:46 AM, Vipul Naik <vipul@...>wrote:

> **
>
>
> Hi Alex,
>
> On the Nyaya Health page, the "updated" date at the top (just above
> the "Donate Now" button) says November 28, 2011, without any
> indication that you have a May 2012 update. Perhaps you should modify
> that date?
>
> Vipul
>
> * Quoting Alexander Berger who at 2012-05-22 14:32:19+0000 (Tue) wrote
>
> > Hi everyone,
> >
> > On May 9th, we held a research event at our office in New York to discuss
> > updates on past top-rated charities (AMF, SCI, and VillageReach) as well
> as
> > our plans for research. Audio and transcript from the event are available
> > at http://givewell.org/conference-call. We'd be interested in any
> questions
> > or comments you have about the content.
> >
> > In addition, we've recently posted:
> >
> > - An intervention report on non-therapeutic zinc supplementation:
> >
> http://givewell.org/international/technical/programs/non-therapeutic-zinc
> > - New documents from Nyaya Health, a standout charity:
> > http://givewell.org/international/charities/Nyaya-Health#Update
> > - New annotated budget projections from GiveDirectly, a standout
> > charity:
> >
> http://www.givewell.org/files/DWDA%202009/GiveDirectly/20120511%20Annotated%20GD%20Budget%20Projections.pdf
> > (PDF)
> > - New documents from Fistula Foundation:
>
> > http://www.givewell.org/international/charities/Fistula-Foundation
> >
> > Best,
> > Alexander
> >
> >
> > On Mon, Apr 30, 2012 at 10:12 AM, Alexander Berger
> > <alexander.is@...>wrote:
> >
> > > Hi everyone,
> > >
> > > We've posted a new review of Give Directly (which continues to be a
> > > "standout" organization) at *
> > > http://www.givewell.org/international/charities/give-directly*.
>
> > >
> > > In addition, we've posted the following transcripts from conversations
> > > we've had with various organizations over the last few months:
> > >
> > > *Centre for Neglected Tropical Diseases*
> > >
> > > - Moses Bockarie, Director<
> http://www.givewell.org/files/DWDA%202009/CNTD/Givewell%20phone%20conversation%20with%20CNTD.doc>
> on
> > > March 26, 2012.
> > >
> > > *Development Media International*
> > >
> > > - Roy Head, CEO of Development Media International (DMI), Will Snell,
>
> > > Head of Public Engagement & Development at DMI, and Cathryn Wood,
> Public
> > > Engagement and Innovation Manager<
> http://www.givewell.org/files/DWDA%202009/DMI/Conversation%20with%20Giving%20What%20We%20Can%20and%20DMI.doc>
> on
> > > April 17, 2012.
> > >
> > > *Doctors Without Borders (MSF)*
> > >
> > > - Jennifer Tierney, Development Director, and Jason Cone,
> > > Communications Director<
> http://www.givewell.org/files/DWDA%202009/MSF/MSF%20-%20GiveWell%202011%2009%2026.doc>
> on
> > > September 26, 2011.
> > > - Jennifer Tierney, Development Director<
> http://www.givewell.org/files/DWDA%202009/MSF/Conversation%20with%20MSF-USA%202012%2003%2008%20final.doc>
> on
> > > March 8, 2012.
> > >
> > > *GAVI alliance*
> > >
> > > - Seth Berkley, CEO<
> http://www.givewell.org/files/DWDA%202009/GAVI/gavi%202012%2001.doc> on
> > > January 12, 2012.
> > > - Alex Palacios, Special Representative to the U.S.<
> http://www.givewell.org/files/DWDA%202009/GAVI/Conversation%20with%20GAVI%202012%2002%2014.doc>
> on
> > > February 14, 2012.
> > >
> > > *Measles Initiative*
> > >
> > > - Peter Strebel, World Health Organization Department of Immunization,
>
> > > Vaccines and Biologicals; Andrea Gay, United Nations Foundation
> Executive
> > > Director of Children's Health; and Steve Cochi, CDC Center for Global
> > > Health Senior Advisor to the Director<
> http://www.givewell.org/files/DWDA%202009/Measles%20Initiative/Conversation%20with%20Measles%20Initiative%202012%2004%2017%20final.doc>
> on
> > > April 18, 2012.
> > >
> > > *TB REACH*
> > >
> > > - Suvanand Sahu, Team Leader, and Jacob Creswell, Technical Officer<
> http://www.givewell.org/files/DWDA%202009/TB%20REACH/Conversation%20with%20TB%20Reach%202012%2003%2005%20final.doc>
> on
> > > March 5, 2012.
> > >
> > > *William and Flora Hewlett Foundation*
> > >
> > >
> > > - Paul Brest<
> http://www.givewell.org/files/conversations/paul-brest-march-2012.doc> in
>
> > > March 2012.
> > >
> > >
> > > As always, please don't hesitate to reach out if you have any
> questions or
> > > comments.
> > >
> > > Best,
> > > Alexander
> > >
> > > On Mon, Mar 26, 2012 at 1:42 PM, Elie Hassenfeld <elie@...
> >wrote:
> > >
> > >> **
>
> > >>
> > >>
> > >> Hi Vipul,
> > >>
> > >> Thanks for the email and for continuing to follow up with us about
> > >> publishing our VillageReach update. Today, we've:
> > >>
> > >> 1. Published a summary of the update today on our blog:
> > >> http://blog.givewell.org/2012/03/26/villagereach-update/
> > >> 2. A full, detailed update on our main site:
> > >>
> http://givewell.org/international/top-charities/villagereach/updates#March262012update
> > >> 3. A guest post from John Beale at VillageReach providing more
>
> > >> context for this update:
> > >>
> http://blog.givewell.org/2012/03/26/guest-post-from-john-beale-at-villagereach
> > >>
> > >> We previously set a goal of publishing quarterly updates on
> VillageReach
> > >> (and all charities we rate high enough to direct them significant
> funding).
> > >> Today's VillageReach update is our first update on VillageReach since
> > >> August 2011, so we are publishing this update significantly later
> than we
> > >> had hoped. We will probably not be aiming for quarterly updates in the
> > >> future. To briefly explain the three main factors that led to this
> delay:
> > >>
> > >> 1. We were scheduled for a VillageReach update in early December
>
> > >> 2011, but we were primarily focused on our new recommendations and on
> > >> giving season from October 1, 2011 - January 1, 2012. During that
> 3-month
> > >> period, we focused our all efforts on our new recommendations and did
> not
> > >> focus on VillageReach. (We feel this was the right decision.)
> > >> 2. When we began our reassessment of VillageReach in early 2012, we
>
> > >> started re-analyzing the evidence of effectiveness from the pilot
> project.
> > >> This analysis -- some of which we published recently at
> > >>
> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis--
> > >> led to correspondence and conversations with VillageReach
> representatives
> > >> about pilot project data rather than an update of VillageReach's
> work. In
> > >> hindsight, it's clear that we should have prioritized publishing an
> update
> > >> on VillageReach's work in Mozambique over revisiting the pilot
> project.
> > >> 3. In every case where we publish something about a charity, we send
>
> > >> the charity a draft of our page before we publish it. We do this to
> > >> fact-check our work and to give the organization an opportunity to
> comment
> > >> on particular words/phrasing it finds problematic. In this case,
> extensive
> > >> back and forth further delayed publication.
> > >>
> > >> Given the amount of work it takes to do an update and the pace at
> which
> > >> projects tend to move, we're now thinking that quarterly is too
> often. We
> > >> are rethinking the frequency we should aim for and will update on
> what we
> > >> decide.
> > >>
> > >> Best,
> > >> Elie
> > >>
> > >>
> > >> On Fri, Mar 23, 2012 at 2:04 PM, Vipul Naik <vipul@...
> >wrote:
> > >>
> > >>> **
>
> > >>>
> > >>>
> > >>> Hi Alexander,
> > >>>
> > >>> Regarding VillageReach, is this the VillageReach update that you said
> > >>> you'd be publishing soon? The link you have given is to a re-analysis
> > >>> of the pilot project prior to GiveWell's recommendation of
> > >>> VillageReach, not of the expansion that was funded as a result of
> > >>> GiveWell's recommendation. Perhaps I am missing something here?
> > >>>
> > >>> Do you plan to post updates on the expansion that was funded as a
> > >>> result of GiveWell's recommendation? If so, when?
> > >>>
> > >>> Thank you,
> > >>>
> > >>> Vipul
> > >>>
> > >>> * Quoting Alexander Berger who at 2012-03-23 13:08:36+0000 (Fri)
> wrote
> > >>> > Hello everyone,
> > >>> >
> > >>> > New/updated pages:
> > >>> >
> > >>> > - We've updated our review of the Small Enterprise Foundation:
> > >>> >
> > >>>
> http://www.givewell.org/international/top-charities/Small-Enterprise-Foundation
> > >>> > - We conducted a re-analysis of VillageReach's pilot project data:
> > >>> >
> > >>>
> http://givewell.org/international/top-charities/villagereach/pilot-project-re-analysis
> > >>> > - We published our first update on AMF's progress since becoming a
> top
> > >>> > charity:
> http://givewell.org/international/top-charities/amf/updates
> > >>> > - We published an update on KIPP Houston's room for more funding:
> > >>> > http://blog.givewell.org/2012/03/14/kipp-houston-update/
> > >>> > - We posted the audio and documents from our December board
> meeting:
> > >>>
> > >>> > http://givewell.org/about/official-records/board-meeting-17
> > >>> >
> > >>> > As usual, please feel free to let us know if you have any
> questions or
> > >>> > comments.
> > >>> >
> > >>> > Best,
> > >>> > Alexander
> > >>> >
> > >>> > On Fri, Feb 10, 2012 at 4:47 PM, Alexander Berger <
> > >>> alexander.is@...>wrote:
> > >>> >
> > >>> > > Hello everyone,
> > >>> > >
> > >>> > > New/updated pages:
> > >>> > >
> > >>> > > - Our shortcomings page: http://givewell.org/about/shortcomings
> > >>> > > - Our intervention report on cash transfers:
> > >>>
> > >>> > >
> http://givewell.org/international/technical/programs/cash-transfers
> > >>> > >
> > >>> > > Best,
> > >>> > > Alexander
> > >>> > >
> > >>> > > On Fri, Jan 13, 2012 at 7:05 PM, Alexander Berger <
> > >>> alexander.is@...>wrote:
> > >>> > >
> > >>> > >> Hello everyone,
> > >>> > >>
> > >>> > >> A few people have told us that there's no way for GiveWell
> > >>> followers to
> > >>> > >> know when we publish new pages. Of course, people know when we
> > >>> update our
> > >>> > >> charity recommendations with new reviews, but when we publish
> other
> > >>> pages
> > >>> > >> (e.g., a review of a charity that's not top-rated or a page
> like the
> > >>> > >> Research FAQ, below), people don't know.
> > >>> > >>
> > >>> > >> Sending a list of links to this group is one way to address this
> > >>> issue.
> > >>> > >> This email includes new content (except blog posts) since our
> major
> > >>> > >> research update just after Thanksgiving, including the major
> pages
> > >>> we
> > >>> > >> published as part of the updated report.
> > >>> > >>
> > >>> > >> Our process:
> > >>> > >>
> > >>> > >> - 2011 international aid charities research process:
> > >>> > >> http://givewell.org/international/process/2011
> > >>> > >> - All charities considered to date:
> > >>> http://www.givewell.org/charities
> > >>> > >>
> > >>> > >> Related to AMF:
> > >>> > >>
> > >>> > >> - Our review of the Against Malaria Foundation:
> > >>> > >> http://givewell.org/international/top-charities/AMF
> > >>> > >> - Our intervention report on mass distribution of long-lasting
> > >>> > >> insecticide-treated nets:
> > >>> > >>
> > >>>
> http://givewell.org/international/technical/programs/insecticide-treated-nets
> > >>> > >> - Our discussion of the evidence on the effectiveness of large
> scale
> > >>>
> > >>> > >> ITN distribution campaigns:
> > >>> > >>
> > >>>
> http://givewell.org/international/technical/programs/insecticide-treated-nets/macro-evidence
> > >>> > >>
> > >>> > >> Related to SCI:
> > >>> > >>
> > >>> > >> - Our review of the Schistosomiasis Control Initiative:
> > >>> > >>
> > >>>
> http://givewell.org/international/top-charities/schistosomiasis-control-initiative
> > >>> > >> - Our intervention report on combination deworming to kill
> > >>>
> > >>> > >> soil-transmitted helminths and schistosomiasis:
> > >>> > >> http://givewell.org/international/technical/programs/deworming
> > >>> > >>
> > >>> > >> Related to IPA:
> > >>> > >>
> > >>> > >> - Our review of Innovations for Poverty Action:
> > >>> > >> http://givewell.org/international/charities/ipa
> > >>> > >> - Our overview of the evidence created by and the influence of
> > >>>
> > >>> > >> notable IPA studies:
> > >>> > >> http://givewell.org/international/charities/ipa/research
> > >>> > >>
> > >>> > >> Other:
> > >>> > >>
> > >>> > >> - What others are saying about GiveWell:
> > >>>
> > >>> > >> http://givewell.org/what-others-are-saying (this is an update
> on a
> > >>> > >> previous "Press" page that we think presents GiveWell in a very
> > >>> positive
> > >>> > >> light; we'd like for more new visitors to the site to see this
> page)
> > >>> > >> - Frequently asked questions about our research:
> > >>> > >> http://www.givewell.org/about/FAQ/research
> > >>> > >> - Our review of International Development Enterprises:
> > >>> > >>
> > >>>
> http://givewell.org/International/charities/International-Development-Enterprises
> > >>> > >> - Notes from our October 2011 site visits:
> > >>> > >> http://www.givewell.org/international/site-visits/october-2011
> > >>> > >> - Audio and transcript from our conference calls:
> > >>> > >> http://www.givewell.org/conference-call
> > >>> > >>
> > >>> > >> Best,
> > >>> > >> Alexander
> > >>> > >>
> > >>> > >
> > >>> > >
> > >>>
> > >>>
> > >>
> > >>
> > >
> > >
>
>  
>

We've recently published the following pages:

   - An intervention report on zinc supplementation to treat diarrhea:
   http://www.givewell.org/international/technical/programs/therapeutic-zinc
   - An intervention report on hydrocele surgery:
   http://www.givewell.org/international/health/surgery/hydrocele
   - Micronutrient Initiative declined to fully participate in our process:
   http://www.givewell.org/international/charities/micronutrient-initiative

We've also created a new Google
Group<https://groups.google.com/forum/?fromgroups#!forum/newly-published-givewell-materials>to
which to post these updates, including new
conversations <http://givewell.org/conversations> with charities and
researchers. Directions on how to subscribe (e.g. via RSS) are available at
our most recent blog post:
http://blog.givewell.org/2012/08/16/quick-update-new-way-to-follow-givewells-research-progress/

Note to the research list: we're now considering reopening our
investigation of the world of biomedical research. We've started and
stopped a couple of times in this area before; this time I decided to start
with a conversation with Dario Amodei, a longtime GiveWell follower and
personal friend who is currently a biology postdoc at Stanford. My goal
with the conversation was just to get some basic context and start putting
together a framework for thinking about the issue, not to use him as an
authoritative source, and the notes below should be read in that spirit.


This email has two sections:


1. Notes that I emailed out internally after my conversation with Dario,
slightly edited

2. Some more context on the history of our work on biomedical research and
why we think it's appropriate to investigate this field (this was a
response to a question following my original email, along the lines of "Why
are you looking into biomedical research now, given that's an area with a
lot of buzz and funding from wealthy donors, and how does this work relate
to the 'meta-research' work?")



*--*

*
*

*1. Notes that I emailed out internally after my conversation with Dario,
slightly edited*

*
*

I've done some preliminary work trying to figure out what it would look
like to explore biomedical sciences as funding area. This mostly consisted
of a 3-hour conversation with Dario (recording is available), reading two
papers he sent and a few I found while Googling, and prior knowledge. I'm
including Dario in all emails related to this stuff, as an informal advisor.

* *

*My picture of "what the biomedical research world roughly looks like" *(this
is mostly from talking with Dario + prior knowledge)

·   *Academic biology* studies how organisms work and develops tools to
observe and manipulate the building blocks of organisms.

o  The vast bulk of the funding - and the most prestigious funding - comes
from the NIH.

o  There is also funding from what I've heard called "foundations" - groups
like the American Cancer Society and American Heart Association - which
function very similarly to the NIH, in that they tend to hire people with
strong academic credentials and those people judge the merits of grant
proposals.

o  Both the NIH and "foundations" tend to be formally partitioned by
disease, but much of the work done by academic biologists is potentially
relevant to many diseases. A researcher seeking NIH funding may apply to
several different NIH "study sections," though only one at a time (a list
of "study sections" is at
http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
whether s/he gets funding is going to depend more on the academic merits of
the researcher & work than anything else.

o  There are basic definitions of "academic merit " that generally shape
the whole ecosystem: the people handing out money are selected by these
criteria and use these criteria, the people doing the research know that
these criteria shape their career prospects, etc.

·   *Private startups* investigate promising ideas for new
treatments/diagnostics/devices. They may often take the form of a biology
professor spinning off a biotech startup (run by former postdocs) that
raises venture capital, based on the research the professor did. They take
basic knowledge about how the body works (for example, protein X is crucial
for medical condition Y) and do the necessary testing to find a promising
treatment/diagnostic/device (for example, testing a lot of compounds on
animals until they find one that affects protein X).

·   *Big pharma/biotech companies *are best positioned to deal with the
extremely expensive process of conducting clinical trials and getting FDA
approval. Acquisition by one of these is the most common form of exit for
startups.

·   *Academic medicine, epidemiology and other fields* also do work
relevant to medicine, including studying questions whose main relevance is
to medical practice and public health programs: how effective is
treatment/practice X in situation Y, how cost-effective is it, etc.
Sometimes they will hit on commercializable insights (for example, a new
kind of device) as well.

·   *Translational research* is a broad term referring to a bridge between
academic research and treatments/practices. It can include (
http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
):

o  T1 - going from basic science (usually academic biology, I presume) to a
new treatment/practice. This includes research that helps go from an
academic biology insight to a private startup.

o  T2 - I think this is basically what Cochrane does - going from academic
medicine/epidemiology (a bunch of studies on what treatments/practices are
effective) to the development of guidelines that actually affect practice.

o  T3, T4 - research on how to actually change practice (as opposed to
setting the guidelines that are a "target" for practice) and get better
real-world results.

*
*

*Potential "big opportunity to do good" #1: translational research and the
"Valley of Death" (this comes from prior knowledge, googling "Valley of
Death" and reading a bunch of the stuff I found, and checking out the
FasterCures website again)*



The "Valley of Death" seems to mostly refer to a shortage of T1. The idea
is that there are a lot of cases where there's an academic insight that's
potentially valuable in coming up with a new treatment, but to get it to
the point where it's attractive from a for-profit perspective, you need to
do a lot of stuff that academics don't have a reason to do. "For example,
an upstream finding that a given protein is differentially expressed in
individuals with a particular disease may suggest that the protein merits
further investigation. However, much more work (especially medicinal
chemistry) is necessary to determine how good a target the protein really
is and whether a marketable drug candidate that affects the activity of the
protein is likely to be developed." (
https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)



There are claims that this sort of work is massively underfunded (by the
people we've spoken to who talked about the "Valley of Death"; also in
http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research(1).pdf).
However, funding isn't the only issue. The other issue is that
"pharmaceutical firms that hold libraries of potentially useful small
molecules as trade secrets, making them largely off limits to ... academic
scientists" (https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
In other words, there is some room for new models of collaboration that
lead to better communication and information sharing between academia and
industry (or between industry and industry).


 Both the Myelin Repair Foundation and Michael Milken's work on prostate
cancer have been pointed to as examples of innovative collaborations that
deal with some of the information sharing problems. Milken's model:
"drastically cutting the wait time for grant money, to flood the field with
fast cash, to fund therapy-driven ideas rather than basic science, to hold
researchers he funds accountable for results, and to demand collaboration
across disciplines and among institutions, private industry, and academia."
(
http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm)
Myelin Repair Foundation sounds broadly similar (
http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/).




More broadly:

·   FasterCures (also a Michael Milken production) looks like it's focused
on the broad mission of "more research with a shorter timeline to
treatments," with a heavy sub-focus on the Valley of Death. In addition to
its conference and philanthropic advisory service, it advocates for FDA
improvements (presumably to speed the approval process), advocates for the
NIH to put more funding into translational research (there have definitely
been a lot of new initiatives at the NIH focused on this stuff in the past
few ~decade), promotes "innovative financing mechanisms" for bridging the
Valley of Death (these include Peter Thiel's Breakout Labs; I think they
occupy a conceptually similar space to "social enterprise investment"
though they tend to be structured more as grants and less as
double-bottom-line investments), and works on getting patient data opened
to researchers. The only program of theirs I haven't mentioned is TRAIN; I
can't (easily) figure out what this is.

·   John Ioannidis stated to us that all translational research is
underfunded, not just T1. (The context we talked to him in, of course, was
T2.)


These issues seem to have quite a bit of buzz. There are some really stark
#'s out there: even as R&D investment has gone way up over the past 50
years, the # of new drugs has stayed roughly constant at around 20 a year.


Dario sent a really interesting paper on this topic. It argues: (a) the #
of NMEs (new medical entities, i.e., drugs) per year has mostly remained
constant-with-noise (no trend); (b) all of the big companies seem to have
produced NMEs at a very steady pace, even as they've changed size, though
different companies do have different rates of NME creation; (c) when it
comes to mergers & acquisitions, "1+1=1": companies that acquire other
companies just keep up the same NME pace; (d) over time, the # of large
companies has shrunk (due to mergers) and the # of small companies has
risen, and the share of NMEs attributed to small companies has gone from
~30% to ~70%; (e) the cost per NME has gone up over time and is now in the
neighborhood of $1 billion; (f) the % of NMEs that are "blockbusters" (huge
profits) has been steady around 20%, despite intensifying efforts on the
part of pharma to fund only potential blockbusters.



There are also claims - such as by Derek Lowe, a blogger Dario pointed me
to - that the "Valley of Death" is an overrated concept: there is a high
ratio of academic discoveries to useful drugs, but this is just because a
lot of stuff turns out not to work, not because we have a shortage of
doable translational research. The paper above sort of takes this view,
essentially arguing that nothing seems to raise NME production; we'll need
something really radical to make any difference.



The case studies in the FasterCures paper on this are interesting. They
imply that there were some pretty low-hanging fruit in the T1 domain. It
might be interesting to talk to Fastercures and see whether they can help
identify "Valley of Death" opportunities that are slam-dunkish.


*Potential "big opportunity to do good" #2: inefficiencies in academic
biology*



All of the above seems pretty distinct from the question of how to make
academic biology better at doing its job of understanding the body. This is
the question that Dario and I focused on.



Dario painted a picture in which most of academia plays by the same set of
rules, making it very hard for people to do things that break those rules
(for example, academics are expected to publish a lot; it's risky to work
on a "blue sky" or highly ambitious project). Thus, for anyone who wants a
career in academia, a couple of years working on a moonshot project are
risky; in addition, being on bad terms with a small number of people can
damage a career (since there is often a small set of people that makes a
large proportion of the career-impacting decisions for a given area, and it
can be hard to escape this set of people without changing research
interests significantly). The bad news is that this isn't particularly easy
to fix: you can offer funding for blue-sky projects, but (a) a lot of
academics basically train themselves to play by the rules, and won't
necessarily have thought about "what should be studied if these rules
didn't apply," and more importantly, (b) there are many incentives pushing
academics towards playing by certain rules; funding is only one of those
mechanisms (there's also tenure, peer review, etc) so changing that one
incentive won't always change behavior. Dario says that he might hesitate
to work on a particular blue-sky project that he thinks is interesting,
even if he got funding for it, for these reasons.


Despite this, there are some funders who push the boundaries. There are
medical centers that don't require teaching and do more ambitious work.
There's the McKnight foundation, which funded some of the pioneering work
on optical control of neurons for which funding might have been difficult
to obtain by traditional routes. There's the HHMI Janelia Farms campus,
which Dario thinks is the most promising thing out there in terms of a
model - academics who go there get guaranteed (and generous) funding for 6
years, which frees them up to take much bigger risks. (I saw a study
claiming that the work coming out of Janelia Farms has a much more skewed
citation count distribution, implying bigger risks - i.e., fewer papers
with decent #'s of citations but more "blockbuster" papers with lots of
citations). But Janelia Farms is only in one sub-field; comparable
institutions don't exist for other fields (as far as Dario knows)..



Dario's gut is that one of the best things a funder could do would be to
work toward creating a large institute that largely "plays by its own
rules," encouraging more ambitious work and providing enough security and
sheer volume of dollars/researchers as to establish a sort of "parallel
system" to academia - thus becoming a place that could provide viable and
reliable career options for people interested in playing by different
rules. I'm also interested in the idea of trying to advocate for changes in
the rules, as we've discussed in the context of meta-research.



There's also the other stuff like data/code sharing.



*Another option: give $ to the NIH*



The NIH is by far the largest, most prestigious, most respected funder in
academic biology, and most of academia runs on criteria that mirror the
NIH's. I would guess that an unrestricted check to the NIH would get
allocated in a pretty sector-agnostic way. It seems like this is a giving
option that is pretty nontrivial to beat. Anyone we consider for funding
ought to be able to explain why they're better at allocating the funds than
the NIH. We haven't talked to the NIH about whether it would accept these
sorts of donations.


With the context I now have, FasterCures strikes me as pretty
promising/interesting too, though their "room for more funding" situation
isn't clear (w/o talking to them). But they also seem sector-agnostic
within biomedical research, while pushing a very specific theory of change
that may or may not be valid.



*Next steps*



I think understanding this world would be a major undertaking. I think we'd
have to be ready to put in a lot of work and to be open to ways of bridging
"funder-expert gap" that we haven't tried before. For example:

·   Dario suggested that I read a freshman biology textbook. He said he
realizes that the time cost could be very large - something like 100 hours
- but that without doing so, I'm going to be lacking too much context on
why biomedical research works the way it does. He thinks that reading a
basic textbook would get me to the point of strongly diminishing marginal
returns.

·   Also in the category of "personally picking up rudimentary subject
matter knowledge," I thought it might be worth trying to follow the
development of a particular drug from start to finish - Gleevec (the
"miracle drug") would be a good candidate. The goal would be to understand
each stage of insight leading to new investigation, and where the funding
came from at each stage.

·   I think hiring Dario or someone like him would make a lot of sense.
I've thought about whether we should be hiring "subject matter experts" in
other areas, such as global health, but in my view the need is clearer here
than in any other area. One of the things I don't love about hiring an
expert in a given field, at this stage of our research, is that we could
quickly decide that we're just not interested enough in the field in
question ... but someone with the right kind of technical knowledge &
experience would be so far ahead of us in evaluating *any* area of biology
research that it seems like a good idea. (JTBC, I'm also actively thinking
about whether it would make sense to hire experts in other fields ...)

·   Talking to major funders and potentially co-funding with them is
probably essential. Important groups to talk to would include NIH (by far
the most important; we've already talked to them a bit), the colloquium of
groups like the American Cancer Society (it has a name; I forget the name),
FasterCures, Wellcome, and potentially some funders with
unorthodox/buzzed-about approaches: Myelin Repair Foundation, McKnight
Foundation, and maybe another private foundation or two (I emailed Dario
all my notes on major foundations that do biomedical research so he could
send thoughts on whether any look interesting aside from McKnight).


My next plan had been to talk to academics (Dario had good suggestions
about how to approach them), but with all the potential work to be done on
gaining basic context, I'm not sure that's the right next step. But it's
also a possible step.



*2. Some more context on the history of our work on biomedical research and
why we think it's appropriate to investigate this field (this was a
response to a question following my original email, along the lines of "Why
are you looking into biomedical research now, given that's an area with a
lot of buzz and funding from wealthy donors, and how does this work relate
to the 'meta-research' work?")*


   - We've always had "disease research" as a cause we wanted to
   investigate. We've never had a good reason for not investigating it other
   than that investigating it seems difficult. We've tried more than once to
   investigate it, and it's ended up falling by the wayside because of how big
   an undertaking it is.
   - It's true that there is a huge amount of funding and buzz in this
   area. But it also seems quite possible that there isn't nearly enough; in
   fact this seems like a possible point of consensus between all the people
   concerned about the "Valley of Death." This potential good accomplished via
   biomedical research appears unlimited. What I consider to be the mark of a
   bad sector is "tons of buzz/$ and mediocre returns," but the latter part
   doesn't apply to biomed; in fact biomedical research is arguably one of
   philanthropy's and even humanity's top success stories.
   - There are also indications that despite all the buzz and funding,
   there are still opportunities to do things differently and shake things up.
   There is more than one case where an outsider (Milken, MRF) basically came
   in and did things very differently and now even experts in the field seem
   to credit them with positive change. There's also a good explanation for
   why this might be the case: while there are a ton of people and dollars,
   they largely seem to play by one self-reinforcing/network-effect-prone set
   of rules, implying high returns to disrupting that equilibrium.
   - So, we've always wanted to get into this area. There was a period
   where I was presenting meta-research as our best entry point into this
   field: my vision was that we would talk to academics about what systematic
   failings there were and what funding opportunities these implied, and that
   would be as good a way as any to get acclimated in biomed. But this period
   wasn't particularly long - the May blog post on Labs priority causes lists
   scientific research as a promising area distinct from meta-research.
   - I set up the call with Dario without having a clear idea of whether I
   wanted to approach biomed from a "meta-research" angle or another angle.
   After the call and other investigations described here, I got a clearer
   idea of what I think is the best path forward.

Bottom line - I think it's important to build an understanding of
biomedical research, and that we should take the best path to doing so
whether or not that dovetails with the meta-research work (likely it will
dovetail some but not 100%).

Holden,

First, I think that this is an excellent document. I checked for a
number of things that I had heard about (Breakout Labs, John
Ioannidis, Cochrane Collaboration) and they're all there in your
document.

The one thing that's not explicitly mentioned: longevity and life
extension research. At least prima facie, this seems like something
that should be more important than individual disease research, and it
seems like a classic "Valley of Death" case (pun unintended, but
noted) -- T1 stage to use your terminology. I think the SENS website
http://www.sens.org would be a good starting point for one of the (to
me promising) approaches to life extension. I recall from past
conversations that you were aware of SENS, so this is not new to you,
but I think that longevity should be included as part of any
discussion of biomedical research and given separate consideration
given that it has a much lower status than research into specific
conditions such as cancer, dementia, etc. You may ultimately conclude
that not enough can be done in this area, but I think it should be
part of your preliminary stuff. [btw, the United States has a National
Institute of Aging, but it's much lower-status than most of the other
grantmakers mentioned here].

Vipul

* Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed) wrote
> Note to the research list: we're now considering reopening our
> investigation of the world of biomedical research. We've started and
> stopped a couple of times in this area before; this time I decided to start
> with a conversation with Dario Amodei, a longtime GiveWell follower and
> personal friend who is currently a biology postdoc at Stanford. My goal
> with the conversation was just to get some basic context and start putting
> together a framework for thinking about the issue, not to use him as an
> authoritative source, and the notes below should be read in that spirit.
> 
> 
> This email has two sections:
> 
> 
> 1. Notes that I emailed out internally after my conversation with Dario,
> slightly edited
> 
> 2. Some more context on the history of our work on biomedical research and
> why we think it's appropriate to investigate this field (this was a
> response to a question following my original email, along the lines of "Why
> are you looking into biomedical research now, given that's an area with a
> lot of buzz and funding from wealthy donors, and how does this work relate
> to the 'meta-research' work?")
> 
> 
> 
> *--*
> 
> *
> *
> 
> *1. Notes that I emailed out internally after my conversation with Dario,
> slightly edited*
> 
> *
> *
> 
> I've done some preliminary work trying to figure out what it would look
> like to explore biomedical sciences as funding area. This mostly consisted
> of a 3-hour conversation with Dario (recording is available), reading two
> papers he sent and a few I found while Googling, and prior knowledge. I'm
> including Dario in all emails related to this stuff, as an informal advisor.
> 
> * *
> 
> *My picture of "what the biomedical research world roughly looks like" *(this
> is mostly from talking with Dario + prior knowledge)
> 
> �   *Academic biology* studies how organisms work and develops tools to
> observe and manipulate the building blocks of organisms.
> 
> o  The vast bulk of the funding - and the most prestigious funding - comes
> from the NIH.
> 
> o  There is also funding from what I've heard called "foundations" - groups
> like the American Cancer Society and American Heart Association - which
> function very similarly to the NIH, in that they tend to hire people with
> strong academic credentials and those people judge the merits of grant
> proposals.
> 
> o  Both the NIH and "foundations" tend to be formally partitioned by
> disease, but much of the work done by academic biologists is potentially
> relevant to many diseases. A researcher seeking NIH funding may apply to
> several different NIH "study sections," though only one at a time (a list
> of "study sections" is at
> http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
> whether s/he gets funding is going to depend more on the academic merits of
> the researcher & work than anything else.
> 
> o  There are basic definitions of "academic merit " that generally shape
> the whole ecosystem: the people handing out money are selected by these
> criteria and use these criteria, the people doing the research know that
> these criteria shape their career prospects, etc.
> 
> �   *Private startups* investigate promising ideas for new
> treatments/diagnostics/devices. They may often take the form of a biology
> professor spinning off a biotech startup (run by former postdocs) that
> raises venture capital, based on the research the professor did. They take
> basic knowledge about how the body works (for example, protein X is crucial
> for medical condition Y) and do the necessary testing to find a promising
> treatment/diagnostic/device (for example, testing a lot of compounds on
> animals until they find one that affects protein X).
> 
> �   *Big pharma/biotech companies *are best positioned to deal with the
> extremely expensive process of conducting clinical trials and getting FDA
> approval. Acquisition by one of these is the most common form of exit for
> startups.
> 
> �   *Academic medicine, epidemiology and other fields* also do work
> relevant to medicine, including studying questions whose main relevance is
> to medical practice and public health programs: how effective is
> treatment/practice X in situation Y, how cost-effective is it, etc.
> Sometimes they will hit on commercializable insights (for example, a new
> kind of device) as well.
> 
> �   *Translational research* is a broad term referring to a bridge between
> academic research and treatments/practices. It can include (
> http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> ):
> 
> o  T1 - going from basic science (usually academic biology, I presume) to a
> new treatment/practice. This includes research that helps go from an
> academic biology insight to a private startup.
> 
> o  T2 - I think this is basically what Cochrane does - going from academic
> medicine/epidemiology (a bunch of studies on what treatments/practices are
> effective) to the development of guidelines that actually affect practice.
> 
> o  T3, T4 - research on how to actually change practice (as opposed to
> setting the guidelines that are a "target" for practice) and get better
> real-world results.
> 
> *
> *
> 
> *Potential "big opportunity to do good" #1: translational research and the
> "Valley of Death" (this comes from prior knowledge, googling "Valley of
> Death" and reading a bunch of the stuff I found, and checking out the
> FasterCures website again)*
> 
> 
> 
> The "Valley of Death" seems to mostly refer to a shortage of T1. The idea
> is that there are a lot of cases where there's an academic insight that's
> potentially valuable in coming up with a new treatment, but to get it to
> the point where it's attractive from a for-profit perspective, you need to
> do a lot of stuff that academics don't have a reason to do. "For example,
> an upstream finding that a given protein is differentially expressed in
> individuals with a particular disease may suggest that the protein merits
> further investigation. However, much more work (especially medicinal
> chemistry) is necessary to determine how good a target the protein really
> is and whether a marketable drug candidate that affects the activity of the
> protein is likely to be developed." (
> https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> 
> 
> 
> There are claims that this sort of work is massively underfunded (by the
> people we've spoken to who talked about the "Valley of Death"; also in
> http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research(1).pdf).
> However, funding isn't the only issue. The other issue is that
> "pharmaceutical firms that hold libraries of potentially useful small
> molecules as trade secrets, making them largely off limits to ... academic
> scientists" (https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> In other words, there is some room for new models of collaboration that
> lead to better communication and information sharing between academia and
> industry (or between industry and industry).
> 
> 
>  Both the Myelin Repair Foundation and Michael Milken's work on prostate
> cancer have been pointed to as examples of innovative collaborations that
> deal with some of the information sharing problems. Milken's model:
> "drastically cutting the wait time for grant money, to flood the field with
> fast cash, to fund therapy-driven ideas rather than basic science, to hold
> researchers he funds accountable for results, and to demand collaboration
> across disciplines and among institutions, private industry, and academia."
> (
> http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm)
> Myelin Repair Foundation sounds broadly similar (
> http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/).
> 
> 
> 
> 
> More broadly:
> 
> �   FasterCures (also a Michael Milken production) looks like it's focused
> on the broad mission of "more research with a shorter timeline to
> treatments," with a heavy sub-focus on the Valley of Death. In addition to
> its conference and philanthropic advisory service, it advocates for FDA
> improvements (presumably to speed the approval process), advocates for the
> NIH to put more funding into translational research (there have definitely
> been a lot of new initiatives at the NIH focused on this stuff in the past
> few ~decade), promotes "innovative financing mechanisms" for bridging the
> Valley of Death (these include Peter Thiel's Breakout Labs; I think they
> occupy a conceptually similar space to "social enterprise investment"
> though they tend to be structured more as grants and less as
> double-bottom-line investments), and works on getting patient data opened
> to researchers. The only program of theirs I haven't mentioned is TRAIN; I
> can't (easily) figure out what this is.
> 
> �   John Ioannidis stated to us that all translational research is
> underfunded, not just T1. (The context we talked to him in, of course, was
> T2.)
> 
> 
> These issues seem to have quite a bit of buzz. There are some really stark
> #'s out there: even as R&D investment has gone way up over the past 50
> years, the # of new drugs has stayed roughly constant at around 20 a year.
> 
> 
> Dario sent a really interesting paper on this topic. It argues: (a) the #
> of NMEs (new medical entities, i.e., drugs) per year has mostly remained
> constant-with-noise (no trend); (b) all of the big companies seem to have
> produced NMEs at a very steady pace, even as they've changed size, though
> different companies do have different rates of NME creation; (c) when it
> comes to mergers & acquisitions, "1+1=1": companies that acquire other
> companies just keep up the same NME pace; (d) over time, the # of large
> companies has shrunk (due to mergers) and the # of small companies has
> risen, and the share of NMEs attributed to small companies has gone from
> ~30% to ~70%; (e) the cost per NME has gone up over time and is now in the
> neighborhood of $1 billion; (f) the % of NMEs that are "blockbusters" (huge
> profits) has been steady around 20%, despite intensifying efforts on the
> part of pharma to fund only potential blockbusters.
> 
> 
> 
> There are also claims - such as by Derek Lowe, a blogger Dario pointed me
> to - that the "Valley of Death" is an overrated concept: there is a high
> ratio of academic discoveries to useful drugs, but this is just because a
> lot of stuff turns out not to work, not because we have a shortage of
> doable translational research. The paper above sort of takes this view,
> essentially arguing that nothing seems to raise NME production; we'll need
> something really radical to make any difference.
> 
> 
> 
> The case studies in the FasterCures paper on this are interesting. They
> imply that there were some pretty low-hanging fruit in the T1 domain. It
> might be interesting to talk to Fastercures and see whether they can help
> identify "Valley of Death" opportunities that are slam-dunkish.
> 
> 
> *Potential "big opportunity to do good" #2: inefficiencies in academic
> biology*
> 
> 
> 
> All of the above seems pretty distinct from the question of how to make
> academic biology better at doing its job of understanding the body. This is
> the question that Dario and I focused on.
> 
> 
> 
> Dario painted a picture in which most of academia plays by the same set of
> rules, making it very hard for people to do things that break those rules
> (for example, academics are expected to publish a lot; it's risky to work
> on a "blue sky" or highly ambitious project). Thus, for anyone who wants a
> career in academia, a couple of years working on a moonshot project are
> risky; in addition, being on bad terms with a small number of people can
> damage a career (since there is often a small set of people that makes a
> large proportion of the career-impacting decisions for a given area, and it
> can be hard to escape this set of people without changing research
> interests significantly). The bad news is that this isn't particularly easy
> to fix: you can offer funding for blue-sky projects, but (a) a lot of
> academics basically train themselves to play by the rules, and won't
> necessarily have thought about "what should be studied if these rules
> didn't apply," and more importantly, (b) there are many incentives pushing
> academics towards playing by certain rules; funding is only one of those
> mechanisms (there's also tenure, peer review, etc) so changing that one
> incentive won't always change behavior. Dario says that he might hesitate
> to work on a particular blue-sky project that he thinks is interesting,
> even if he got funding for it, for these reasons.
> 
> 
> Despite this, there are some funders who push the boundaries. There are
> medical centers that don't require teaching and do more ambitious work.
> There's the McKnight foundation, which funded some of the pioneering work
> on optical control of neurons for which funding might have been difficult
> to obtain by traditional routes. There's the HHMI Janelia Farms campus,
> which Dario thinks is the most promising thing out there in terms of a
> model - academics who go there get guaranteed (and generous) funding for 6
> years, which frees them up to take much bigger risks. (I saw a study
> claiming that the work coming out of Janelia Farms has a much more skewed
> citation count distribution, implying bigger risks - i.e., fewer papers
> with decent #'s of citations but more "blockbuster" papers with lots of
> citations). But Janelia Farms is only in one sub-field; comparable
> institutions don't exist for other fields (as far as Dario knows)..
> 
> 
> 
> Dario's gut is that one of the best things a funder could do would be to
> work toward creating a large institute that largely "plays by its own
> rules," encouraging more ambitious work and providing enough security and
> sheer volume of dollars/researchers as to establish a sort of "parallel
> system" to academia - thus becoming a place that could provide viable and
> reliable career options for people interested in playing by different
> rules. I'm also interested in the idea of trying to advocate for changes in
> the rules, as we've discussed in the context of meta-research.
> 
> 
> 
> There's also the other stuff like data/code sharing.
> 
> 
> 
> *Another option: give $ to the NIH*
> 
> 
> 
> The NIH is by far the largest, most prestigious, most respected funder in
> academic biology, and most of academia runs on criteria that mirror the
> NIH's. I would guess that an unrestricted check to the NIH would get
> allocated in a pretty sector-agnostic way. It seems like this is a giving
> option that is pretty nontrivial to beat. Anyone we consider for funding
> ought to be able to explain why they're better at allocating the funds than
> the NIH. We haven't talked to the NIH about whether it would accept these
> sorts of donations.
> 
> 
> With the context I now have, FasterCures strikes me as pretty
> promising/interesting too, though their "room for more funding" situation
> isn't clear (w/o talking to them). But they also seem sector-agnostic
> within biomedical research, while pushing a very specific theory of change
> that may or may not be valid.
> 
> 
> 
> *Next steps*
> 
> 
> 
> I think understanding this world would be a major undertaking. I think we'd
> have to be ready to put in a lot of work and to be open to ways of bridging
> "funder-expert gap" that we haven't tried before. For example:
> 
> �   Dario suggested that I read a freshman biology textbook. He said he
> realizes that the time cost could be very large - something like 100 hours
> - but that without doing so, I'm going to be lacking too much context on
> why biomedical research works the way it does. He thinks that reading a
> basic textbook would get me to the point of strongly diminishing marginal
> returns.
> 
> �   Also in the category of "personally picking up rudimentary subject
> matter knowledge," I thought it might be worth trying to follow the
> development of a particular drug from start to finish - Gleevec (the
> "miracle drug") would be a good candidate. The goal would be to understand
> each stage of insight leading to new investigation, and where the funding
> came from at each stage.
> 
> �   I think hiring Dario or someone like him would make a lot of sense.
> I've thought about whether we should be hiring "subject matter experts" in
> other areas, such as global health, but in my view the need is clearer here
> than in any other area. One of the things I don't love about hiring an
> expert in a given field, at this stage of our research, is that we could
> quickly decide that we're just not interested enough in the field in
> question ... but someone with the right kind of technical knowledge &
> experience would be so far ahead of us in evaluating *any* area of biology
> research that it seems like a good idea. (JTBC, I'm also actively thinking
> about whether it would make sense to hire experts in other fields ...)
> 
> �   Talking to major funders and potentially co-funding with them is
> probably essential. Important groups to talk to would include NIH (by far
> the most important; we've already talked to them a bit), the colloquium of
> groups like the American Cancer Society (it has a name; I forget the name),
> FasterCures, Wellcome, and potentially some funders with
> unorthodox/buzzed-about approaches: Myelin Repair Foundation, McKnight
> Foundation, and maybe another private foundation or two (I emailed Dario
> all my notes on major foundations that do biomedical research so he could
> send thoughts on whether any look interesting aside from McKnight).
> 
> 
> My next plan had been to talk to academics (Dario had good suggestions
> about how to approach them), but with all the potential work to be done on
> gaining basic context, I'm not sure that's the right next step. But it's
> also a possible step.
> 
> 
> 
> *2. Some more context on the history of our work on biomedical research and
> why we think it's appropriate to investigate this field (this was a
> response to a question following my original email, along the lines of "Why
> are you looking into biomedical research now, given that's an area with a
> lot of buzz and funding from wealthy donors, and how does this work relate
> to the 'meta-research' work?")*
> 
> 
>    - We've always had "disease research" as a cause we wanted to
>    investigate. We've never had a good reason for not investigating it other
>    than that investigating it seems difficult. We've tried more than once to
>    investigate it, and it's ended up falling by the wayside because of how big
>    an undertaking it is.
>    - It's true that there is a huge amount of funding and buzz in this
>    area. But it also seems quite possible that there isn't nearly enough; in
>    fact this seems like a possible point of consensus between all the people
>    concerned about the "Valley of Death." This potential good accomplished via
>    biomedical research appears unlimited. What I consider to be the mark of a
>    bad sector is "tons of buzz/$ and mediocre returns," but the latter part
>    doesn't apply to biomed; in fact biomedical research is arguably one of
>    philanthropy's and even humanity's top success stories.
>    - There are also indications that despite all the buzz and funding,
>    there are still opportunities to do things differently and shake things up.
>    There is more than one case where an outsider (Milken, MRF) basically came
>    in and did things very differently and now even experts in the field seem
>    to credit them with positive change. There's also a good explanation for
>    why this might be the case: while there are a ton of people and dollars,
>    they largely seem to play by one self-reinforcing/network-effect-prone set
>    of rules, implying high returns to disrupting that equilibrium.
>    - So, we've always wanted to get into this area. There was a period
>    where I was presenting meta-research as our best entry point into this
>    field: my vision was that we would talk to academics about what systematic
>    failings there were and what funding opportunities these implied, and that
>    would be as good a way as any to get acclimated in biomed. But this period
>    wasn't particularly long - the May blog post on Labs priority causes lists
>    scientific research as a promising area distinct from meta-research.
>    - I set up the call with Dario without having a clear idea of whether I
>    wanted to approach biomed from a "meta-research" angle or another angle.
>    After the call and other investigations described here, I got a clearer
>    idea of what I think is the best path forward.
> 
> Bottom line - I think it's important to build an understanding of
> biomedical research, and that we should take the best path to doing so
> whether or not that dovetails with the meta-research work (likely it will
> dovetail some but not 100%).

Hi Vipul,

Thanks for the thoughts. I had a followup conversation with Dario about
this topic a few days ago. I think the question of "could one fund
translational research to treat/prevent aging?" provides an interesting
illustration of some of the tricky dynamics here for a funder:

   - It's possible that if there were a great deal more attention giving to
   treating/preventing aging, we would have some promising treatments. So in a
   broad sense it's possible that aging is underinvested in.
   - A lot of the best basic biology research isn't clearly pointing toward
   one treatment/condition or another; it's about understanding the
   fundamentals of how organisms operate. So having an interest in treating
   aging, as opposed to cancer, might not have a major impact on which
   projects one funds, if one's main goal is to fund outstanding basic biology
   research.
   - Perhaps because of the lack of emphasis on treating aging (or perhaps
   because it's simply too difficult of a problem), there don't seem to be
   promising findings in the "Valley of Death" relevant to aging; the few
   promising leads have been explored.
   - So even if, in a broad sense, there is too little attention given to
   this problem, knowing this doesn't necessarily yield a clear direction for
   a relatively small-scale funder of biomedical research.

Best,
Holden

On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:

> **
>
>
> Holden,
>
> First, I think that this is an excellent document. I checked for a
> number of things that I had heard about (Breakout Labs, John
> Ioannidis, Cochrane Collaboration) and they're all there in your
> document.
>
> The one thing that's not explicitly mentioned: longevity and life
> extension research. At least prima facie, this seems like something
> that should be more important than individual disease research, and it
> seems like a classic "Valley of Death" case (pun unintended, but
> noted) -- T1 stage to use your terminology. I think the SENS website
> http://www.sens.org would be a good starting point for one of the (to
> me promising) approaches to life extension. I recall from past
> conversations that you were aware of SENS, so this is not new to you,
> but I think that longevity should be included as part of any
> discussion of biomedical research and given separate consideration
> given that it has a much lower status than research into specific
> conditions such as cancer, dementia, etc. You may ultimately conclude
> that not enough can be done in this area, but I think it should be
> part of your preliminary stuff. [btw, the United States has a National
> Institute of Aging, but it's much lower-status than most of the other
> grantmakers mentioned here].
>
> Vipul
>
> * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed) wrote
>
> > Note to the research list: we're now considering reopening our
> > investigation of the world of biomedical research. We've started and
> > stopped a couple of times in this area before; this time I decided to
> start
> > with a conversation with Dario Amodei, a longtime GiveWell follower and
> > personal friend who is currently a biology postdoc at Stanford. My goal
> > with the conversation was just to get some basic context and start
> putting
> > together a framework for thinking about the issue, not to use him as an
> > authoritative source, and the notes below should be read in that spirit.
> >
> >
> > This email has two sections:
> >
> >
> > 1. Notes that I emailed out internally after my conversation with Dario,
> > slightly edited
> >
> > 2. Some more context on the history of our work on biomedical research
> and
> > why we think it's appropriate to investigate this field (this was a
> > response to a question following my original email, along the lines of
> "Why
> > are you looking into biomedical research now, given that's an area with a
> > lot of buzz and funding from wealthy donors, and how does this work
> relate
> > to the 'meta-research' work?")
> >
> >
> >
> > *--*
> >
> > *
> > *
> >
> > *1. Notes that I emailed out internally after my conversation with Dario,
> > slightly edited*
> >
> > *
>
> > *
> >
> > I've done some preliminary work trying to figure out what it would look
> > like to explore biomedical sciences as funding area. This mostly
> consisted
> > of a 3-hour conversation with Dario (recording is available), reading two
> > papers he sent and a few I found while Googling, and prior knowledge. I'm
> > including Dario in all emails related to this stuff, as an informal
> advisor.
> >
> > * *
> >
> > *My picture of "what the biomedical research world roughly looks like"
> *(this
>
> > is mostly from talking with Dario + prior knowledge)
> >
> > · *Academic biology* studies how organisms work and develops tools to
>
> > observe and manipulate the building blocks of organisms.
> >
> > o The vast bulk of the funding - and the most prestigious funding - comes
> > from the NIH.
> >
> > o There is also funding from what I've heard called "foundations" -
> groups
> > like the American Cancer Society and American Heart Association - which
> > function very similarly to the NIH, in that they tend to hire people with
> > strong academic credentials and those people judge the merits of grant
> > proposals.
> >
> > o Both the NIH and "foundations" tend to be formally partitioned by
> > disease, but much of the work done by academic biologists is potentially
> > relevant to many diseases. A researcher seeking NIH funding may apply to
> > several different NIH "study sections," though only one at a time (a list
> > of "study sections" is at
> > http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
> > whether s/he gets funding is going to depend more on the academic merits
> of
> > the researcher & work than anything else.
> >
> > o There are basic definitions of "academic merit " that generally shape
> > the whole ecosystem: the people handing out money are selected by these
> > criteria and use these criteria, the people doing the research know that
> > these criteria shape their career prospects, etc.
> >
> > · *Private startups* investigate promising ideas for new
>
> > treatments/diagnostics/devices. They may often take the form of a biology
> > professor spinning off a biotech startup (run by former postdocs) that
> > raises venture capital, based on the research the professor did. They
> take
> > basic knowledge about how the body works (for example, protein X is
> crucial
> > for medical condition Y) and do the necessary testing to find a promising
> > treatment/diagnostic/device (for example, testing a lot of compounds on
> > animals until they find one that affects protein X).
> >
> > · *Big pharma/biotech companies *are best positioned to deal with the
>
> > extremely expensive process of conducting clinical trials and getting FDA
> > approval. Acquisition by one of these is the most common form of exit for
> > startups.
> >
> > · *Academic medicine, epidemiology and other fields* also do work
>
> > relevant to medicine, including studying questions whose main relevance
> is
> > to medical practice and public health programs: how effective is
> > treatment/practice X in situation Y, how cost-effective is it, etc.
> > Sometimes they will hit on commercializable insights (for example, a new
> > kind of device) as well.
> >
> > · *Translational research* is a broad term referring to a bridge between
>
> > academic research and treatments/practices. It can include (
> >
> http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> > ):
> >
> > o T1 - going from basic science (usually academic biology, I presume) to
> a
> > new treatment/practice. This includes research that helps go from an
> > academic biology insight to a private startup.
> >
> > o T2 - I think this is basically what Cochrane does - going from academic
> > medicine/epidemiology (a bunch of studies on what treatments/practices
> are
> > effective) to the development of guidelines that actually affect
> practice.
> >
> > o T3, T4 - research on how to actually change practice (as opposed to
> > setting the guidelines that are a "target" for practice) and get better
> > real-world results.
> >
> > *
> > *
> >
> > *Potential "big opportunity to do good" #1: translational research and
> the
>
> > "Valley of Death" (this comes from prior knowledge, googling "Valley of
> > Death" and reading a bunch of the stuff I found, and checking out the
> > FasterCures website again)*
>
> >
> >
> >
> > The "Valley of Death" seems to mostly refer to a shortage of T1. The idea
> > is that there are a lot of cases where there's an academic insight that's
> > potentially valuable in coming up with a new treatment, but to get it to
> > the point where it's attractive from a for-profit perspective, you need
> to
> > do a lot of stuff that academics don't have a reason to do. "For example,
> > an upstream finding that a given protein is differentially expressed in
> > individuals with a particular disease may suggest that the protein merits
> > further investigation. However, much more work (especially medicinal
> > chemistry) is necessary to determine how good a target the protein really
> > is and whether a marketable drug candidate that affects the activity of
> the
> > protein is likely to be developed." (
> > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> >
> >
> >
> > There are claims that this sort of work is massively underfunded (by the
> > people we've spoken to who talked about the "Valley of Death"; also in
> >
> http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
> (1).pdf).
> > However, funding isn't the only issue. The other issue is that
> > "pharmaceutical firms that hold libraries of potentially useful small
> > molecules as trade secrets, making them largely off limits to ...
> academic
> > scientists" (
> https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> > In other words, there is some room for new models of collaboration that
> > lead to better communication and information sharing between academia and
> > industry (or between industry and industry).
> >
> >
> > Both the Myelin Repair Foundation and Michael Milken's work on prostate
> > cancer have been pointed to as examples of innovative collaborations that
> > deal with some of the information sharing problems. Milken's model:
> > "drastically cutting the wait time for grant money, to flood the field
> with
> > fast cash, to fund therapy-driven ideas rather than basic science, to
> hold
> > researchers he funds accountable for results, and to demand collaboration
> > across disciplines and among institutions, private industry, and
> academia."
> > (
> >
> http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
> )
> > Myelin Repair Foundation sounds broadly similar (
> >
> http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
> ).
> >
> >
> >
> >
> > More broadly:
> >
> > · FasterCures (also a Michael Milken production) looks like it's focused
> > on the broad mission of "more research with a shorter timeline to
> > treatments," with a heavy sub-focus on the Valley of Death. In addition
> to
> > its conference and philanthropic advisory service, it advocates for FDA
> > improvements (presumably to speed the approval process), advocates for
> the
> > NIH to put more funding into translational research (there have
> definitely
> > been a lot of new initiatives at the NIH focused on this stuff in the
> past
> > few ~decade), promotes "innovative financing mechanisms" for bridging the
> > Valley of Death (these include Peter Thiel's Breakout Labs; I think they
> > occupy a conceptually similar space to "social enterprise investment"
> > though they tend to be structured more as grants and less as
> > double-bottom-line investments), and works on getting patient data opened
> > to researchers. The only program of theirs I haven't mentioned is TRAIN;
> I
> > can't (easily) figure out what this is.
> >
> > · John Ioannidis stated to us that all translational research is
> > underfunded, not just T1. (The context we talked to him in, of course,
> was
> > T2.)
> >
> >
> > These issues seem to have quite a bit of buzz. There are some really
> stark
> > #'s out there: even as R&D investment has gone way up over the past 50
> > years, the # of new drugs has stayed roughly constant at around 20 a
> year.
> >
> >
> > Dario sent a really interesting paper on this topic. It argues: (a) the #
> > of NMEs (new medical entities, i.e., drugs) per year has mostly remained
> > constant-with-noise (no trend); (b) all of the big companies seem to have
> > produced NMEs at a very steady pace, even as they've changed size, though
> > different companies do have different rates of NME creation; (c) when it
> > comes to mergers & acquisitions, "1+1=1": companies that acquire other
> > companies just keep up the same NME pace; (d) over time, the # of large
> > companies has shrunk (due to mergers) and the # of small companies has
> > risen, and the share of NMEs attributed to small companies has gone from
> > ~30% to ~70%; (e) the cost per NME has gone up over time and is now in
> the
> > neighborhood of $1 billion; (f) the % of NMEs that are "blockbusters"
> (huge
> > profits) has been steady around 20%, despite intensifying efforts on the
> > part of pharma to fund only potential blockbusters.
> >
> >
> >
> > There are also claims - such as by Derek Lowe, a blogger Dario pointed me
> > to - that the "Valley of Death" is an overrated concept: there is a high
> > ratio of academic discoveries to useful drugs, but this is just because a
> > lot of stuff turns out not to work, not because we have a shortage of
> > doable translational research. The paper above sort of takes this view,
> > essentially arguing that nothing seems to raise NME production; we'll
> need
> > something really radical to make any difference.
> >
> >
> >
> > The case studies in the FasterCures paper on this are interesting. They
> > imply that there were some pretty low-hanging fruit in the T1 domain. It
> > might be interesting to talk to Fastercures and see whether they can help
> > identify "Valley of Death" opportunities that are slam-dunkish.
> >
> >
> > *Potential "big opportunity to do good" #2: inefficiencies in academic
> > biology*
>
> >
> >
> >
> > All of the above seems pretty distinct from the question of how to make
> > academic biology better at doing its job of understanding the body. This
> is
> > the question that Dario and I focused on.
> >
> >
> >
> > Dario painted a picture in which most of academia plays by the same set
> of
> > rules, making it very hard for people to do things that break those rules
> > (for example, academics are expected to publish a lot; it's risky to work
> > on a "blue sky" or highly ambitious project). Thus, for anyone who wants
> a
> > career in academia, a couple of years working on a moonshot project are
> > risky; in addition, being on bad terms with a small number of people can
> > damage a career (since there is often a small set of people that makes a
> > large proportion of the career-impacting decisions for a given area, and
> it
> > can be hard to escape this set of people without changing research
> > interests significantly). The bad news is that this isn't particularly
> easy
> > to fix: you can offer funding for blue-sky projects, but (a) a lot of
> > academics basically train themselves to play by the rules, and won't
> > necessarily have thought about "what should be studied if these rules
> > didn't apply," and more importantly, (b) there are many incentives
> pushing
> > academics towards playing by certain rules; funding is only one of those
> > mechanisms (there's also tenure, peer review, etc) so changing that one
> > incentive won't always change behavior. Dario says that he might hesitate
> > to work on a particular blue-sky project that he thinks is interesting,
> > even if he got funding for it, for these reasons.
> >
> >
> > Despite this, there are some funders who push the boundaries. There are
> > medical centers that don't require teaching and do more ambitious work.
> > There's the McKnight foundation, which funded some of the pioneering work
> > on optical control of neurons for which funding might have been difficult
> > to obtain by traditional routes. There's the HHMI Janelia Farms campus,
> > which Dario thinks is the most promising thing out there in terms of a
> > model - academics who go there get guaranteed (and generous) funding for
> 6
> > years, which frees them up to take much bigger risks. (I saw a study
> > claiming that the work coming out of Janelia Farms has a much more skewed
> > citation count distribution, implying bigger risks - i.e., fewer papers
> > with decent #'s of citations but more "blockbuster" papers with lots of
> > citations). But Janelia Farms is only in one sub-field; comparable
> > institutions don't exist for other fields (as far as Dario knows)..
> >
> >
> >
> > Dario's gut is that one of the best things a funder could do would be to
> > work toward creating a large institute that largely "plays by its own
> > rules," encouraging more ambitious work and providing enough security and
> > sheer volume of dollars/researchers as to establish a sort of "parallel
> > system" to academia - thus becoming a place that could provide viable and
> > reliable career options for people interested in playing by different
> > rules. I'm also interested in the idea of trying to advocate for changes
> in
> > the rules, as we've discussed in the context of meta-research.
> >
> >
> >
> > There's also the other stuff like data/code sharing.
> >
> >
> >
> > *Another option: give $ to the NIH*
>
> >
> >
> >
> > The NIH is by far the largest, most prestigious, most respected funder in
> > academic biology, and most of academia runs on criteria that mirror the
> > NIH's. I would guess that an unrestricted check to the NIH would get
> > allocated in a pretty sector-agnostic way. It seems like this is a giving
> > option that is pretty nontrivial to beat. Anyone we consider for funding
> > ought to be able to explain why they're better at allocating the funds
> than
> > the NIH. We haven't talked to the NIH about whether it would accept these
> > sorts of donations.
> >
> >
> > With the context I now have, FasterCures strikes me as pretty
> > promising/interesting too, though their "room for more funding" situation
> > isn't clear (w/o talking to them). But they also seem sector-agnostic
> > within biomedical research, while pushing a very specific theory of
> change
> > that may or may not be valid.
> >
> >
> >
> > *Next steps*
>
> >
> >
> >
> > I think understanding this world would be a major undertaking. I think
> we'd
> > have to be ready to put in a lot of work and to be open to ways of
> bridging
> > "funder-expert gap" that we haven't tried before. For example:
> >
> > · Dario suggested that I read a freshman biology textbook. He said he
> > realizes that the time cost could be very large - something like 100
> hours
> > - but that without doing so, I'm going to be lacking too much context on
> > why biomedical research works the way it does. He thinks that reading a
> > basic textbook would get me to the point of strongly diminishing marginal
> > returns.
> >
> > · Also in the category of "personally picking up rudimentary subject
> > matter knowledge," I thought it might be worth trying to follow the
> > development of a particular drug from start to finish - Gleevec (the
> > "miracle drug") would be a good candidate. The goal would be to
> understand
> > each stage of insight leading to new investigation, and where the funding
> > came from at each stage.
> >
> > · I think hiring Dario or someone like him would make a lot of sense.
> > I've thought about whether we should be hiring "subject matter experts"
> in
> > other areas, such as global health, but in my view the need is clearer
> here
> > than in any other area. One of the things I don't love about hiring an
> > expert in a given field, at this stage of our research, is that we could
> > quickly decide that we're just not interested enough in the field in
> > question ... but someone with the right kind of technical knowledge &
> > experience would be so far ahead of us in evaluating *any* area of
> biology
> > research that it seems like a good idea. (JTBC, I'm also actively
> thinking
> > about whether it would make sense to hire experts in other fields ...)
> >
> > · Talking to major funders and potentially co-funding with them is
> > probably essential. Important groups to talk to would include NIH (by far
> > the most important; we've already talked to them a bit), the colloquium
> of
> > groups like the American Cancer Society (it has a name; I forget the
> name),
> > FasterCures, Wellcome, and potentially some funders with
> > unorthodox/buzzed-about approaches: Myelin Repair Foundation, McKnight
> > Foundation, and maybe another private foundation or two (I emailed Dario
> > all my notes on major foundations that do biomedical research so he could
> > send thoughts on whether any look interesting aside from McKnight).
> >
> >
> > My next plan had been to talk to academics (Dario had good suggestions
> > about how to approach them), but with all the potential work to be done
> on
> > gaining basic context, I'm not sure that's the right next step. But it's
> > also a possible step.
> >
> >
> >
> > *2. Some more context on the history of our work on biomedical research
> and
>
> > why we think it's appropriate to investigate this field (this was a
> > response to a question following my original email, along the lines of
> "Why
> > are you looking into biomedical research now, given that's an area with a
> > lot of buzz and funding from wealthy donors, and how does this work
> relate
> > to the 'meta-research' work?")*
> >
> >
> > - We've always had "disease research" as a cause we wanted to
>
> > investigate. We've never had a good reason for not investigating it other
> > than that investigating it seems difficult. We've tried more than once to
> > investigate it, and it's ended up falling by the wayside because of how
> big
> > an undertaking it is.
> > - It's true that there is a huge amount of funding and buzz in this
>
> > area. But it also seems quite possible that there isn't nearly enough; in
> > fact this seems like a possible point of consensus between all the people
> > concerned about the "Valley of Death." This potential good accomplished
> via
> > biomedical research appears unlimited. What I consider to be the mark of
> a
> > bad sector is "tons of buzz/$ and mediocre returns," but the latter part
> > doesn't apply to biomed; in fact biomedical research is arguably one of
> > philanthropy's and even humanity's top success stories.
> > - There are also indications that despite all the buzz and funding,
>
> > there are still opportunities to do things differently and shake things
> up.
> > There is more than one case where an outsider (Milken, MRF) basically
> came
> > in and did things very differently and now even experts in the field seem
> > to credit them with positive change. There's also a good explanation for
> > why this might be the case: while there are a ton of people and dollars,
> > they largely seem to play by one self-reinforcing/network-effect-prone
> set
> > of rules, implying high returns to disrupting that equilibrium.
> > - So, we've always wanted to get into this area. There was a period
>
> > where I was presenting meta-research as our best entry point into this
> > field: my vision was that we would talk to academics about what
> systematic
> > failings there were and what funding opportunities these implied, and
> that
> > would be as good a way as any to get acclimated in biomed. But this
> period
> > wasn't particularly long - the May blog post on Labs priority causes
> lists
> > scientific research as a promising area distinct from meta-research.
> > - I set up the call with Dario without having a clear idea of whether I
>
> > wanted to approach biomed from a "meta-research" angle or another angle.
> > After the call and other investigations described here, I got a clearer
> > idea of what I think is the best path forward.
> >
> > Bottom line - I think it's important to build an understanding of
> > biomedical research, and that we should take the best path to doing so
> > whether or not that dovetails with the meta-research work (likely it will
> > dovetail some but not 100%).
>
>  
>

One area worth mentioning is the problem of false positive results from bad
practices in academia (error, data mining, publication bias, etc) leading
to waste and much higher costs in translational research. Funding reforms
of relevant journal practices, commissioning audits of samples of studies,
commissioning fraud catchers (consider the impact of a few months of work
by one guy, Uri Simonsohn, in psychology:
http://www.nature.com/news/the-data-detective-1.10937) and so forth could
reduce the problem. Some data (HT Gwern Branwen for the quotes):

http://lifescivc.com/2011/03/academic-bias-biotech-failures/
The company spent $5M or so trying to validate a platform that didn’t
exist. When they tried to directly repeat the academic founder’s data, it
never worked. Upon re-examination of the lab notebooks, it was clear the
founder’s lab had at the very least massaged the data and shaped it to fit
their hypothesis. Essentially, they systematically ignored every piece of
negative data. Sadly this failure to repeat happens more often than we’d
like to believe. It has happened to us at Atlas [Venture] several times in
the past decade…The unspoken rule is that at least 50% of the studies
published even in top tier academic journals - Science, Nature, Cell, PNAS,
etc… - can’t be repeated with the same conclusions by an industrial lab. In
particular, key animal models often don’t reproduce. This 50% failure rate
isn’t a data free assertion: it’s backed up by dozens of experienced R&D
professionals who’ve participated in the (re)testing of academic findings.
This is a huge problem for translational research and one that won’t go
away until we address it head on.

An audit of reproducibility of bio findings:
http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html
…despite the low numbers, there was no apparent difference between the
different research fields. Surprisingly, even publications in prestigious
journals or from several independent groups did not ensure reproducibility.
Indeed, our analysis revealed that the reproducibility of published data
did not significantly correlate with journal impact factors, the number of
publications on the respective target or the number of independent groups
that authored the publications. Our findings are mirrored by gut
feelings expressed
in personal communications with scientists from academia or other
companies, as well as published observations. [apropos of above] An
unspoken rule among early-stage venture capital firms that at least 50% of
published studies, even those in top-tier academic journals, can’t be
repeated with the same conclusions by an industrial lab has been recently
reported (see Further information) and discussed
4<http://www.nature.com/scibx/journal/v4/n15/full/scibx.2011.416.html>
.

I agree strongly on the importance of systematically reproducing results
(see this: http://www.nature.com/nbt/journal/v30/n9/pdf/nbt.2335.pdf, which
I recently sent to Holden).

On the (related) issue of false positive results, I agree it's a big
problem, and while publication bias, questionable lab practices, and
occasionally outright fraud are certainly issues, I'd say that in the
majority of cases where a result doesn't reproduce, it's due to honest and
often unavoidable error.  It's almost impossible to totally control lab
conditions, and there are so many potential sources of variance
(machine-specific issues, the way a particular technician performs a
technique, lab temperature and humidity, etc) that the likelihood that one
of them will materially affect the results in an irreproducible way is
pretty high.  The solution is simply to replicate an experiment across many
labs.  I predict that if this was done more commonly, and if everyone
behaved honestly, a large fraction of results would still not replicate,
but those that did replicated would fare much better in a commercial
setting.

On Thu, Oct 11, 2012 at 5:04 PM, Carl M. Shulman <
carl.shulman@...> wrote:

> **
>
>
> One area worth mentioning is the problem of false positive results from
> bad practices in academia (error, data mining, publication bias, etc)
> leading to waste and much higher costs in translational research. Funding
> reforms of relevant journal practices, commissioning audits of samples of
> studies, commissioning fraud catchers (consider the impact of a few months
> of work by one guy, Uri Simonsohn, in psychology:
> http://www.nature.com/news/the-data-detective-1.10937) and so forth could
> reduce the problem. Some data (HT Gwern Branwen for the quotes):
>
> http://lifescivc.com/2011/03/academic-bias-biotech-failures/
> The company spent $5M or so trying to validate a platform that didn’t
> exist. When they tried to directly repeat the academic founder’s data, it
> never worked. Upon re-examination of the lab notebooks, it was clear the
> founder’s lab had at the very least massaged the data and shaped it to fit
> their hypothesis. Essentially, they systematically ignored every piece of
> negative data. Sadly this failure to repeat happens more often than we’d
> like to believe. It has happened to us at Atlas [Venture] several times in
> the past decade…The unspoken rule is that at least 50% of the studies
> published even in top tier academic journals - Science, Nature, Cell, PNAS,
> etc… - can’t be repeated with the same conclusions by an industrial lab. In
> particular, key animal models often don’t reproduce. This 50% failure rate
> isn’t a data free assertion: it’s backed up by dozens of experienced R&D
> professionals who’ve participated in the (re)testing of academic findings.
> This is a huge problem for translational research and one that won’t go
> away until we address it head on.
>
> An audit of reproducibility of bio findings:
> http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html
> …despite the low numbers, there was no apparent difference between the
> different research fields. Surprisingly, even publications in prestigious
> journals or from several independent groups did not ensure reproducibility.
> Indeed, our analysis revealed that the reproducibility of published data
> did not significantly correlate with journal impact factors, the number of
> publications on the respective target or the number of independent groups
> that authored the publications. Our findings are mirrored by gut feelings expressed
> in personal communications with scientists from academia or other
> companies, as well as published observations. [apropos of above] An
> unspoken rule among early-stage venture capital firms that at least 50%
> of published studies, even those in top-tier academic journals, can’t be
> repeated with the same conclusions by an industrial lab has been recently
> reported (see Further information) and discussed 4<http://www.nature.com/scibx/journal/v4/n15/full/scibx.2011.416.html>
> .
>
>  
>

Carl, thanks for the thoughts & links. We agree that this is a potentially
promising area of inquiry. We consider it to fall under the heading of
"meta-research," which we've written about at
http://blog.givewell.org/2012/06/11/meta-research/. Meta-research is a
priority of ours, and as we investigate biomedical sciences we'll be
looking out for related giving opportunities.

On Thu, Oct 11, 2012 at 8:04 PM, Carl M. Shulman <
carl.shulman@...rvard.edu> wrote:

> **
>
>
> One area worth mentioning is the problem of false positive results from
> bad practices in academia (error, data mining, publication bias, etc)
> leading to waste and much higher costs in translational research. Funding
> reforms of relevant journal practices, commissioning audits of samples of
> studies, commissioning fraud catchers (consider the impact of a few months
> of work by one guy, Uri Simonsohn, in psychology:
> http://www.nature.com/news/the-data-detective-1.10937) and so forth could
> reduce the problem. Some data (HT Gwern Branwen for the quotes):
>
> http://lifescivc.com/2011/03/academic-bias-biotech-failures/
> The company spent $5M or so trying to validate a platform that didn’t
> exist. When they tried to directly repeat the academic founder’s data, it
> never worked. Upon re-examination of the lab notebooks, it was clear the
> founder’s lab had at the very least massaged the data and shaped it to fit
> their hypothesis. Essentially, they systematically ignored every piece of
> negative data. Sadly this failure to repeat happens more often than we’d
> like to believe. It has happened to us at Atlas [Venture] several times in
> the past decade…The unspoken rule is that at least 50% of the studies
> published even in top tier academic journals - Science, Nature, Cell, PNAS,
> etc… - can’t be repeated with the same conclusions by an industrial lab. In
> particular, key animal models often don’t reproduce. This 50% failure rate
> isn’t a data free assertion: it’s backed up by dozens of experienced R&D
> professionals who’ve participated in the (re)testing of academic findings.
> This is a huge problem for translational research and one that won’t go
> away until we address it head on.
>
> An audit of reproducibility of bio findings:
> http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html
> …despite the low numbers, there was no apparent difference between the
> different research fields. Surprisingly, even publications in prestigious
> journals or from several independent groups did not ensure reproducibility.
> Indeed, our analysis revealed that the reproducibility of published data
> did not significantly correlate with journal impact factors, the number of
> publications on the respective target or the number of independent groups
> that authored the publications. Our findings are mirrored by gut feelings expressed
> in personal communications with scientists from academia or other
> companies, as well as published observations. [apropos of above] An
> unspoken rule among early-stage venture capital firms that at least 50%
> of published studies, even those in top-tier academic journals, can’t be
> repeated with the same conclusions by an industrial lab has been recently
> reported (see Further information) and discussed 4<http://www.nature.com/scibx/journal/v4/n15/full/scibx.2011.416.html>
> .
>
>  
>

On Thu, Oct 11, 2012 at 8:04 PM, Carl M. Shulman
<carl.shulman@...> wrote:
>
> Some data (HT Gwern Branwen for the quotes):


If anyone is curious, I stash most of my relevant
quotes/links/citations in
http://www.gwern.net/DNB%20FAQ#flaws-in-mainstream-science-and-psychology

(The backstory is that I was trying to restrain the more fervent dual
n-back supporters from being wildly confident that dual n-back
improved IQ based just on a few studies because even after a few
studies one's confidence was real ought to be pretty low due to
publication bias, null hypothesis significance testing, etc., but they
weren't taking me seriously. So...)

--
gwern

Hi everyone,

My attention was brought to this thread, by virtue of the fact that it was my work that gave rise to SENS Foundation, and I'm looking forward to getting more involved here; I've held the Effective Altruism movement in high regard for some time. However, given my newbie status here I want to start by apologising in advance for any oversight of previously-discussed issues etc. I'm naturally delighted both at Holden's post and at Vipul's reply (which I should stress that I did not plant! - I do not know Vipul at all, though I look forward to changing that). I would like to mention just a few key points for discussion:

- Holden, I want to compliment you on your appreciation of how academia really works. Everything you say about that is spot on. The aversion to "high risk high gain" work that has arisen and become so endemic in the system is the most important point here, in terms of why parallel funding routes are needed.

- I'm slightly confused that a lot of Holden's remarks are focused on the private sector (i.e. startups), since my understanding was that GiveWell is about philanthropy; but I realise that there is not all that clear a boundary between the two (and I note the mention of Breakout Labs, with which I have close links and which sits astride that divide more than arguably anyone). The "valley of death" in pre-competitive translational research is a rather different one than that encountered by startups, but the principle is the same, and research to postpone aging certainly encounteres it.

- Something that I presume factors highly among GiveWell's criteria is the extent to which a cause may be undervalued by the bulk of major philanthropists, such that an infusion of additional funds would make more of a difference than in an area that is already being well funded. To me this seems to mirror the logic of focusing on the shortcomings (gaps) in NIH's funding (and that of traditional-model foundations). Holden notes that "Anyone we consider for funding ought to be able to explain why they're better at allocating the funds than the NIH" and I agree wholeheartedly, but my inference is that he thinks that some orgs may indeed be able to explain that. I certainly think that SENS Foundation can.

- Coming to aging: research to postpone aging has the unique problem of quite indescribeable irrationality on the part of most of the general public, policy-makers and even biologists with regard to its desirability. Biogerontologists have been talking to brick walls for decades in their effort to get the rest of the world to appreciate that aging is what causes age-related ill-health, and thus that treatments for aging are merely preventative geriatrics. The concept persists, despite biogerontologists' best efforts, that aging is "natural" and should be left alone, whereas the diseases that it brings about are awful and should be fought. This is made even more bizarre by the fact that the status of age-related diseases as aspects of the later stages of aging absolutely, unequivocally implies that efforts to attack those diseases directly are doomed to fail. As such, this is a (unique? certainly very rare) case where a philanthropic contribution can make a particularly big difference simply because most philanthropists don't see the case for it. It underpins why having an interest in treating aging, as opposed to cancer, absolutely has a major impact on which projects one funds. It's also a case for (if I understand the term correctly) meta-research.

- A lot of the chatter about treating aging revolves around longevity, but it shouldn't. I'm all in favour of longevity, don't get me wrong, but it's not what gets me up in the morning: what does is health. I want people to be truly youthful, however long ago they were born: simple as that. The benefits of longevity per se to humanity may also be substantial, in the form of greater wisdom etc, but that would necessarily come about only very gradually (we won't have any 1000-year-old for at least 900 years whatever happens!), so it doesn't figure strongly in my calculations.

- When forced to acknowledge that the idea of aging being a high-priority target for medicine is an inescapeable consequence of things they already believe (notably that health is good and ageism is bad), many people retreat to the standpoint that it's never going to be possible so it's OK to be irrational about whether it's desirable. The feasibility of postponing age-related ill-health by X years with medicine available Y years from now is, of course, a matter of speculation on which experts disagree, just as with any other pioneering technology. I know that Holden and others have expressed caution (at best) concerning the accuracy of any kind of calculation of probabilities of particular outcomes in the distant (or even not-so-distant) future, and I share that view. However, an approach that may appeal more is to estimate how much humanitarian benefit a given amount of progress would deliver, and then to ask how unlikely that scenario needs to be to make it not worth pursuing. My claim is that the benefits of hastening the defeat of aging by even a few years (which is the minimum that I claim SENS Foundation is in a position to do, given adequate funding) would be so astronomical that the required chance of success to make such an effort worthwhile would be tiny - too tiny for it to be reasonable to argue that such funding would be inadvisable. But of course that is precisely what I would want GiveWell to opine on.

- In the event that GiveWell (or anyone else) were to decide and declare that the defeat of aging is indeed a cause that philanthropists should support, there then arises the question of which organisation(s) should be supported in the best interests of that mission. We at SENS Foundation have worked diligently to rise as quickly as possible in the legitimacy stakes by all standard measures, but we are still young and there remains more to do. If I were to offer an argument to fund us rather than any other entity, it would largely come down to the fact that no other organisation has even a serious plan for defeating aging, let alone a track record of implementing such a plan's early stages. 

- A significant chunk of what we do is of a kind that I think comes under "meta-research". A prominent example is a project we're funding at Denver University to extend the well-respected forecasting system "International Futures" so that it can analyse scenarios incorporating dramatically postponed aging.

I greatly welcome any feedback.

Cheers, Aubrey

--- In givewell@yahoogroups.com, Holden Karnofsky <holden0@...> wrote:
>
> Hi Vipul,
> 
> Thanks for the thoughts. I had a followup conversation with Dario about
> this topic a few days ago. I think the question of "could one fund
> translational research to treat/prevent aging?" provides an interesting
> illustration of some of the tricky dynamics here for a funder:
> 
>    - It's possible that if there were a great deal more attention giving to
>    treating/preventing aging, we would have some promising treatments. So in a
>    broad sense it's possible that aging is underinvested in.
>    - A lot of the best basic biology research isn't clearly pointing toward
>    one treatment/condition or another; it's about understanding the
>    fundamentals of how organisms operate. So having an interest in treating
>    aging, as opposed to cancer, might not have a major impact on which
>    projects one funds, if one's main goal is to fund outstanding basic biology
>    research.
>    - Perhaps because of the lack of emphasis on treating aging (or perhaps
>    because it's simply too difficult of a problem), there don't seem to be
>    promising findings in the "Valley of Death" relevant to aging; the few
>    promising leads have been explored.
>    - So even if, in a broad sense, there is too little attention given to
>    this problem, knowing this doesn't necessarily yield a clear direction for
>    a relatively small-scale funder of biomedical research.
> 
> Best,
> Holden
> 
> On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:
> 
> > **
> >
> >
> > Holden,
> >
> > First, I think that this is an excellent document. I checked for a
> > number of things that I had heard about (Breakout Labs, John
> > Ioannidis, Cochrane Collaboration) and they're all there in your
> > document.
> >
> > The one thing that's not explicitly mentioned: longevity and life
> > extension research. At least prima facie, this seems like something
> > that should be more important than individual disease research, and it
> > seems like a classic "Valley of Death" case (pun unintended, but
> > noted) -- T1 stage to use your terminology. I think the SENS website
> > http://www.sens.org would be a good starting point for one of the (to
> > me promising) approaches to life extension. I recall from past
> > conversations that you were aware of SENS, so this is not new to you,
> > but I think that longevity should be included as part of any
> > discussion of biomedical research and given separate consideration
> > given that it has a much lower status than research into specific
> > conditions such as cancer, dementia, etc. You may ultimately conclude
> > that not enough can be done in this area, but I think it should be
> > part of your preliminary stuff. [btw, the United States has a National
> > Institute of Aging, but it's much lower-status than most of the other
> > grantmakers mentioned here].
> >
> > Vipul
> >
> > * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed) wrote
> >
> > > Note to the research list: we're now considering reopening our
> > > investigation of the world of biomedical research. We've started and
> > > stopped a couple of times in this area before; this time I decided to
> > start
> > > with a conversation with Dario Amodei, a longtime GiveWell follower and
> > > personal friend who is currently a biology postdoc at Stanford. My goal
> > > with the conversation was just to get some basic context and start
> > putting
> > > together a framework for thinking about the issue, not to use him as an
> > > authoritative source, and the notes below should be read in that spirit.
> > >
> > >
> > > This email has two sections:
> > >
> > >
> > > 1. Notes that I emailed out internally after my conversation with Dario,
> > > slightly edited
> > >
> > > 2. Some more context on the history of our work on biomedical research
> > and
> > > why we think it's appropriate to investigate this field (this was a
> > > response to a question following my original email, along the lines of
> > "Why
> > > are you looking into biomedical research now, given that's an area with a
> > > lot of buzz and funding from wealthy donors, and how does this work
> > relate
> > > to the 'meta-research' work?")
> > >
> > >
> > >
> > > *--*
> > >
> > > *
> > > *
> > >
> > > *1. Notes that I emailed out internally after my conversation with Dario,
> > > slightly edited*
> > >
> > > *
> >
> > > *
> > >
> > > I've done some preliminary work trying to figure out what it would look
> > > like to explore biomedical sciences as funding area. This mostly
> > consisted
> > > of a 3-hour conversation with Dario (recording is available), reading two
> > > papers he sent and a few I found while Googling, and prior knowledge. I'm
> > > including Dario in all emails related to this stuff, as an informal
> > advisor.
> > >
> > > * *
> > >
> > > *My picture of "what the biomedical research world roughly looks like"
> > *(this
> >
> > > is mostly from talking with Dario + prior knowledge)
> > >
> > > · *Academic biology* studies how organisms work and develops tools to
> >
> > > observe and manipulate the building blocks of organisms.
> > >
> > > o The vast bulk of the funding - and the most prestigious funding - comes
> > > from the NIH.
> > >
> > > o There is also funding from what I've heard called "foundations" -
> > groups
> > > like the American Cancer Society and American Heart Association - which
> > > function very similarly to the NIH, in that they tend to hire people with
> > > strong academic credentials and those people judge the merits of grant
> > > proposals.
> > >
> > > o Both the NIH and "foundations" tend to be formally partitioned by
> > > disease, but much of the work done by academic biologists is potentially
> > > relevant to many diseases. A researcher seeking NIH funding may apply to
> > > several different NIH "study sections," though only one at a time (a list
> > > of "study sections" is at
> > > http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
> > > whether s/he gets funding is going to depend more on the academic merits
> > of
> > > the researcher & work than anything else.
> > >
> > > o There are basic definitions of "academic merit " that generally shape
> > > the whole ecosystem: the people handing out money are selected by these
> > > criteria and use these criteria, the people doing the research know that
> > > these criteria shape their career prospects, etc.
> > >
> > > · *Private startups* investigate promising ideas for new
> >
> > > treatments/diagnostics/devices. They may often take the form of a biology
> > > professor spinning off a biotech startup (run by former postdocs) that
> > > raises venture capital, based on the research the professor did. They
> > take
> > > basic knowledge about how the body works (for example, protein X is
> > crucial
> > > for medical condition Y) and do the necessary testing to find a promising
> > > treatment/diagnostic/device (for example, testing a lot of compounds on
> > > animals until they find one that affects protein X).
> > >
> > > · *Big pharma/biotech companies *are best positioned to deal with the
> >
> > > extremely expensive process of conducting clinical trials and getting FDA
> > > approval. Acquisition by one of these is the most common form of exit for
> > > startups.
> > >
> > > · *Academic medicine, epidemiology and other fields* also do work
> >
> > > relevant to medicine, including studying questions whose main relevance
> > is
> > > to medical practice and public health programs: how effective is
> > > treatment/practice X in situation Y, how cost-effective is it, etc.
> > > Sometimes they will hit on commercializable insights (for example, a new
> > > kind of device) as well.
> > >
> > > · *Translational research* is a broad term referring to a bridge between
> >
> > > academic research and treatments/practices. It can include (
> > >
> > http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> > > ):
> > >
> > > o T1 - going from basic science (usually academic biology, I presume) to
> > a
> > > new treatment/practice. This includes research that helps go from an
> > > academic biology insight to a private startup.
> > >
> > > o T2 - I think this is basically what Cochrane does - going from academic
> > > medicine/epidemiology (a bunch of studies on what treatments/practices
> > are
> > > effective) to the development of guidelines that actually affect
> > practice.
> > >
> > > o T3, T4 - research on how to actually change practice (as opposed to
> > > setting the guidelines that are a "target" for practice) and get better
> > > real-world results.
> > >
> > > *
> > > *
> > >
> > > *Potential "big opportunity to do good" #1: translational research and
> > the
> >
> > > "Valley of Death" (this comes from prior knowledge, googling "Valley of
> > > Death" and reading a bunch of the stuff I found, and checking out the
> > > FasterCures website again)*
> >
> > >
> > >
> > >
> > > The "Valley of Death" seems to mostly refer to a shortage of T1. The idea
> > > is that there are a lot of cases where there's an academic insight that's
> > > potentially valuable in coming up with a new treatment, but to get it to
> > > the point where it's attractive from a for-profit perspective, you need
> > to
> > > do a lot of stuff that academics don't have a reason to do. "For example,
> > > an upstream finding that a given protein is differentially expressed in
> > > individuals with a particular disease may suggest that the protein merits
> > > further investigation. However, much more work (especially medicinal
> > > chemistry) is necessary to determine how good a target the protein really
> > > is and whether a marketable drug candidate that affects the activity of
> > the
> > > protein is likely to be developed." (
> > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> > >
> > >
> > >
> > > There are claims that this sort of work is massively underfunded (by the
> > > people we've spoken to who talked about the "Valley of Death"; also in
> > >
> > http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
> > (1).pdf).
> > > However, funding isn't the only issue. The other issue is that
> > > "pharmaceutical firms that hold libraries of potentially useful small
> > > molecules as trade secrets, making them largely off limits to ...
> > academic
> > > scientists" (
> > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> > > In other words, there is some room for new models of collaboration that
> > > lead to better communication and information sharing between academia and
> > > industry (or between industry and industry).
> > >
> > >
> > > Both the Myelin Repair Foundation and Michael Milken's work on prostate
> > > cancer have been pointed to as examples of innovative collaborations that
> > > deal with some of the information sharing problems. Milken's model:
> > > "drastically cutting the wait time for grant money, to flood the field
> > with
> > > fast cash, to fund therapy-driven ideas rather than basic science, to
> > hold
> > > researchers he funds accountable for results, and to demand collaboration
> > > across disciplines and among institutions, private industry, and
> > academia."
> > > (
> > >
> > http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
> > )
> > > Myelin Repair Foundation sounds broadly similar (
> > >
> > http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
> > ).
> > >
> > >
> > >
> > >
> > > More broadly:
> > >
> > > · FasterCures (also a Michael Milken production) looks like it's focused
> > > on the broad mission of "more research with a shorter timeline to
> > > treatments," with a heavy sub-focus on the Valley of Death. In addition
> > to
> > > its conference and philanthropic advisory service, it advocates for FDA
> > > improvements (presumably to speed the approval process), advocates for
> > the
> > > NIH to put more funding into translational research (there have
> > definitely
> > > been a lot of new initiatives at the NIH focused on this stuff in the
> > past
> > > few ~decade), promotes "innovative financing mechanisms" for bridging the
> > > Valley of Death (these include Peter Thiel's Breakout Labs; I think they
> > > occupy a conceptually similar space to "social enterprise investment"
> > > though they tend to be structured more as grants and less as
> > > double-bottom-line investments), and works on getting patient data opened
> > > to researchers. The only program of theirs I haven't mentioned is TRAIN;
> > I
> > > can't (easily) figure out what this is.
> > >
> > > · John Ioannidis stated to us that all translational research is
> > > underfunded, not just T1. (The context we talked to him in, of course,
> > was
> > > T2.)
> > >
> > >
> > > These issues seem to have quite a bit of buzz. There are some really
> > stark
> > > #'s out there: even as R&D investment has gone way up over the past 50
> > > years, the # of new drugs has stayed roughly constant at around 20 a
> > year.
> > >
> > >
> > > Dario sent a really interesting paper on this topic. It argues: (a) the #
> > > of NMEs (new medical entities, i.e., drugs) per year has mostly remained
> > > constant-with-noise (no trend); (b) all of the big companies seem to have
> > > produced NMEs at a very steady pace, even as they've changed size, though
> > > different companies do have different rates of NME creation; (c) when it
> > > comes to mergers & acquisitions, "1+1=1": companies that acquire other
> > > companies just keep up the same NME pace; (d) over time, the # of large
> > > companies has shrunk (due to mergers) and the # of small companies has
> > > risen, and the share of NMEs attributed to small companies has gone from
> > > ~30% to ~70%; (e) the cost per NME has gone up over time and is now in
> > the
> > > neighborhood of $1 billion; (f) the % of NMEs that are "blockbusters"
> > (huge
> > > profits) has been steady around 20%, despite intensifying efforts on the
> > > part of pharma to fund only potential blockbusters.
> > >
> > >
> > >
> > > There are also claims - such as by Derek Lowe, a blogger Dario pointed me
> > > to - that the "Valley of Death" is an overrated concept: there is a high
> > > ratio of academic discoveries to useful drugs, but this is just because a
> > > lot of stuff turns out not to work, not because we have a shortage of
> > > doable translational research. The paper above sort of takes this view,
> > > essentially arguing that nothing seems to raise NME production; we'll
> > need
> > > something really radical to make any difference.
> > >
> > >
> > >
> > > The case studies in the FasterCures paper on this are interesting. They
> > > imply that there were some pretty low-hanging fruit in the T1 domain. It
> > > might be interesting to talk to Fastercures and see whether they can help
> > > identify "Valley of Death" opportunities that are slam-dunkish.
> > >
> > >
> > > *Potential "big opportunity to do good" #2: inefficiencies in academic
> > > biology*
> >
> > >
> > >
> > >
> > > All of the above seems pretty distinct from the question of how to make
> > > academic biology better at doing its job of understanding the body. This
> > is
> > > the question that Dario and I focused on.
> > >
> > >
> > >
> > > Dario painted a picture in which most of academia plays by the same set
> > of
> > > rules, making it very hard for people to do things that break those rules
> > > (for example, academics are expected to publish a lot; it's risky to work
> > > on a "blue sky" or highly ambitious project). Thus, for anyone who wants
> > a
> > > career in academia, a couple of years working on a moonshot project are
> > > risky; in addition, being on bad terms with a small number of people can
> > > damage a career (since there is often a small set of people that makes a
> > > large proportion of the career-impacting decisions for a given area, and
> > it
> > > can be hard to escape this set of people without changing research
> > > interests significantly). The bad news is that this isn't particularly
> > easy
> > > to fix: you can offer funding for blue-sky projects, but (a) a lot of
> > > academics basically train themselves to play by the rules, and won't
> > > necessarily have thought about "what should be studied if these rules
> > > didn't apply," and more importantly, (b) there are many incentives
> > pushing
> > > academics towards playing by certain rules; funding is only one of those
> > > mechanisms (there's also tenure, peer review, etc) so changing that one
> > > incentive won't always change behavior. Dario says that he might hesitate
> > > to work on a particular blue-sky project that he thinks is interesting,
> > > even if he got funding for it, for these reasons.
> > >
> > >
> > > Despite this, there are some funders who push the boundaries. There are
> > > medical centers that don't require teaching and do more ambitious work.
> > > There's the McKnight foundation, which funded some of the pioneering work
> > > on optical control of neurons for which funding might have been difficult
> > > to obtain by traditional routes. There's the HHMI Janelia Farms campus,
> > > which Dario thinks is the most promising thing out there in terms of a
> > > model - academics who go there get guaranteed (and generous) funding for
> > 6
> > > years, which frees them up to take much bigger risks. (I saw a study
> > > claiming that the work coming out of Janelia Farms has a much more skewed
> > > citation count distribution, implying bigger risks - i.e., fewer papers
> > > with decent #'s of citations but more "blockbuster" papers with lots of
> > > citations). But Janelia Farms is only in one sub-field; comparable
> > > institutions don't exist for other fields (as far as Dario knows)..
> > >
> > >
> > >
> > > Dario's gut is that one of the best things a funder could do would be to
> > > work toward creating a large institute that largely "plays by its own
> > > rules," encouraging more ambitious work and providing enough security and
> > > sheer volume of dollars/researchers as to establish a sort of "parallel
> > > system" to academia - thus becoming a place that could provide viable and
> > > reliable career options for people interested in playing by different
> > > rules. I'm also interested in the idea of trying to advocate for changes
> > in
> > > the rules, as we've discussed in the context of meta-research.
> > >
> > >
> > >
> > > There's also the other stuff like data/code sharing.
> > >
> > >
> > >
> > > *Another option: give $ to the NIH*
> >
> > >
> > >
> > >
> > > The NIH is by far the largest, most prestigious, most respected funder in
> > > academic biology, and most of academia runs on criteria that mirror the
> > > NIH's. I would guess that an unrestricted check to the NIH would get
> > > allocated in a pretty sector-agnostic way. It seems like this is a giving
> > > option that is pretty nontrivial to beat. Anyone we consider for funding
> > > ought to be able to explain why they're better at allocating the funds
> > than
> > > the NIH. We haven't talked to the NIH about whether it would accept these
> > > sorts of donations.
> > >
> > >
> > > With the context I now have, FasterCures strikes me as pretty
> > > promising/interesting too, though their "room for more funding" situation
> > > isn't clear (w/o talking to them). But they also seem sector-agnostic
> > > within biomedical research, while pushing a very specific theory of
> > change
> > > that may or may not be valid.
> > >
> > >
> > >
> > > *Next steps*
> >
> > >
> > >
> > >
> > > I think understanding this world would be a major undertaking. I think
> > we'd
> > > have to be ready to put in a lot of work and to be open to ways of
> > bridging
> > > "funder-expert gap" that we haven't tried before. For example:
> > >
> > > · Dario suggested that I read a freshman biology textbook. He said he
> > > realizes that the time cost could be very large - something like 100
> > hours
> > > - but that without doing so, I'm going to be lacking too much context on
> > > why biomedical research works the way it does. He thinks that reading a
> > > basic textbook would get me to the point of strongly diminishing marginal
> > > returns.
> > >
> > > · Also in the category of "personally picking up rudimentary subject
> > > matter knowledge," I thought it might be worth trying to follow the
> > > development of a particular drug from start to finish - Gleevec (the
> > > "miracle drug") would be a good candidate. The goal would be to
> > understand
> > > each stage of insight leading to new investigation, and where the funding
> > > came from at each stage.
> > >
> > > · I think hiring Dario or someone like him would make a lot of sense.
> > > I've thought about whether we should be hiring "subject matter experts"
> > in
> > > other areas, such as global health, but in my view the need is clearer
> > here
> > > than in any other area. One of the things I don't love about hiring an
> > > expert in a given field, at this stage of our research, is that we could
> > > quickly decide that we're just not interested enough in the field in
> > > question ... but someone with the right kind of technical knowledge &
> > > experience would be so far ahead of us in evaluating *any* area of
> > biology
> > > research that it seems like a good idea. (JTBC, I'm also actively
> > thinking
> > > about whether it would make sense to hire experts in other fields ...)
> > >
> > > · Talking to major funders and potentially co-funding with them is
> > > probably essential. Important groups to talk to would include NIH (by far
> > > the most important; we've already talked to them a bit), the colloquium
> > of
> > > groups like the American Cancer Society (it has a name; I forget the
> > name),
> > > FasterCures, Wellcome, and potentially some funders with
> > > unorthodox/buzzed-about approaches: Myelin Repair Foundation, McKnight
> > > Foundation, and maybe another private foundation or two (I emailed Dario
> > > all my notes on major foundations that do biomedical research so he could
> > > send thoughts on whether any look interesting aside from McKnight).
> > >
> > >
> > > My next plan had been to talk to academics (Dario had good suggestions
> > > about how to approach them), but with all the potential work to be done
> > on
> > > gaining basic context, I'm not sure that's the right next step. But it's
> > > also a possible step.
> > >
> > >
> > >
> > > *2. Some more context on the history of our work on biomedical research
> > and
> >
> > > why we think it's appropriate to investigate this field (this was a
> > > response to a question following my original email, along the lines of
> > "Why
> > > are you looking into biomedical research now, given that's an area with a
> > > lot of buzz and funding from wealthy donors, and how does this work
> > relate
> > > to the 'meta-research' work?")*
> > >
> > >
> > > - We've always had "disease research" as a cause we wanted to
> >
> > > investigate. We've never had a good reason for not investigating it other
> > > than that investigating it seems difficult. We've tried more than once to
> > > investigate it, and it's ended up falling by the wayside because of how
> > big
> > > an undertaking it is.
> > > - It's true that there is a huge amount of funding and buzz in this
> >
> > > area. But it also seems quite possible that there isn't nearly enough; in
> > > fact this seems like a possible point of consensus between all the people
> > > concerned about the "Valley of Death." This potential good accomplished
> > via
> > > biomedical research appears unlimited. What I consider to be the mark of
> > a
> > > bad sector is "tons of buzz/$ and mediocre returns," but the latter part
> > > doesn't apply to biomed; in fact biomedical research is arguably one of
> > > philanthropy's and even humanity's top success stories.
> > > - There are also indications that despite all the buzz and funding,
> >
> > > there are still opportunities to do things differently and shake things
> > up.
> > > There is more than one case where an outsider (Milken, MRF) basically
> > came
> > > in and did things very differently and now even experts in the field seem
> > > to credit them with positive change. There's also a good explanation for
> > > why this might be the case: while there are a ton of people and dollars,
> > > they largely seem to play by one self-reinforcing/network-effect-prone
> > set
> > > of rules, implying high returns to disrupting that equilibrium.
> > > - So, we've always wanted to get into this area. There was a period
> >
> > > where I was presenting meta-research as our best entry point into this
> > > field: my vision was that we would talk to academics about what
> > systematic
> > > failings there were and what funding opportunities these implied, and
> > that
> > > would be as good a way as any to get acclimated in biomed. But this
> > period
> > > wasn't particularly long - the May blog post on Labs priority causes
> > lists
> > > scientific research as a promising area distinct from meta-research.
> > > - I set up the call with Dario without having a clear idea of whether I
> >
> > > wanted to approach biomed from a "meta-research" angle or another angle.
> > > After the call and other investigations described here, I got a clearer
> > > idea of what I think is the best path forward.
> > >
> > > Bottom line - I think it's important to build an understanding of
> > > biomedical research, and that we should take the best path to doing so
> > > whether or not that dovetails with the meta-research work (likely it will
> > > dovetail some but not 100%).
> >
> >  
> >
>

Hi Aubrey,

Thanks for the thoughts.

The NIH appears to have a division focused on research relevant to this
topic: http://www.nia.nih.gov/research/dab . Its budget appears to be ~$175
million (per year). The National Institute on Aging, which houses this
division, has a budget of about $1 billion per year, including a separate
~$400 million for neuroscience (which may also be relevant) as well as $115
million for intramural research. Figures are from
http://www.nia.nih.gov/about/budget/2012/fiscal-year-2013-budget. The
Institute states that its mandate includes translational research (
http://www.nia.nih.gov/research/faq/does-nia-support-translational-research).
How
would you distinguish your work from this work?

(For the moment I'm putting aside the question I raised in my previous
response to Vipul on this topic, regarding whether it's best to approach
biology funding from the perspective of "trying to treat/cure a particular
condition" or "trying to understand  fundamental questions in biology whose
applications are difficult to predict.")

Best,
Holden

On Fri, Oct 12, 2012 at 7:32 AM, aubrey.degrey <aubrey@...> wrote:

> **
>
>
> Hi everyone,
>
> My attention was brought to this thread, by virtue of the fact that it was
> my work that gave rise to SENS Foundation, and I'm looking forward to
> getting more involved here; I've held the Effective Altruism movement in
> high regard for some time. However, given my newbie status here I want to
> start by apologising in advance for any oversight of previously-discussed
> issues etc. I'm naturally delighted both at Holden's post and at Vipul's
> reply (which I should stress that I did not plant! - I do not know Vipul at
> all, though I look forward to changing that). I would like to mention just
> a few key points for discussion:
>
> - Holden, I want to compliment you on your appreciation of how academia
> really works. Everything you say about that is spot on. The aversion to
> "high risk high gain" work that has arisen and become so endemic in the
> system is the most important point here, in terms of why parallel funding
> routes are needed.
>
> - I'm slightly confused that a lot of Holden's remarks are focused on the
> private sector (i.e. startups), since my understanding was that GiveWell is
> about philanthropy; but I realise that there is not all that clear a
> boundary between the two (and I note the mention of Breakout Labs, with
> which I have close links and which sits astride that divide more than
> arguably anyone). The "valley of death" in pre-competitive translational
> research is a rather different one than that encountered by startups, but
> the principle is the same, and research to postpone aging certainly
> encounteres it.
>
> - Something that I presume factors highly among GiveWell's criteria is the
> extent to which a cause may be undervalued by the bulk of major
> philanthropists, such that an infusion of additional funds would make more
> of a difference than in an area that is already being well funded. To me
> this seems to mirror the logic of focusing on the shortcomings (gaps) in
> NIH's funding (and that of traditional-model foundations). Holden notes
> that "Anyone we consider for funding ought to be able to explain why
> they're better at allocating the funds than the NIH" and I agree
> wholeheartedly, but my inference is that he thinks that some orgs may
> indeed be able to explain that. I certainly think that SENS Foundation can.
>
> - Coming to aging: research to postpone aging has the unique problem of
> quite indescribeable irrationality on the part of most of the general
> public, policy-makers and even biologists with regard to its desirability.
> Biogerontologists have been talking to brick walls for decades in their
> effort to get the rest of the world to appreciate that aging is what causes
> age-related ill-health, and thus that treatments for aging are merely
> preventative geriatrics. The concept persists, despite biogerontologists'
> best efforts, that aging is "natural" and should be left alone, whereas the
> diseases that it brings about are awful and should be fought. This is made
> even more bizarre by the fact that the status of age-related diseases as
> aspects of the later stages of aging absolutely, unequivocally implies that
> efforts to attack those diseases directly are doomed to fail. As such, this
> is a (unique? certainly very rare) case where a philanthropic contribution
> can make a particularly big difference simply because most philanthropists
> don't see the case for it. It underpins why having an interest in treating
> aging, as opposed to cancer, absolutely has a major impact on which
> projects one funds. It's also a case for (if I understand the term
> correctly) meta-research.
>
> - A lot of the chatter about treating aging revolves around longevity, but
> it shouldn't. I'm all in favour of longevity, don't get me wrong, but it's
> not what gets me up in the morning: what does is health. I want people to
> be truly youthful, however long ago they were born: simple as that. The
> benefits of longevity per se to humanity may also be substantial, in the
> form of greater wisdom etc, but that would necessarily come about only very
> gradually (we won't have any 1000-year-old for at least 900 years whatever
> happens!), so it doesn't figure strongly in my calculations.
>
> - When forced to acknowledge that the idea of aging being a high-priority
> target for medicine is an inescapeable consequence of things they already
> believe (notably that health is good and ageism is bad), many people
> retreat to the standpoint that it's never going to be possible so it's OK
> to be irrational about whether it's desirable. The feasibility of
> postponing age-related ill-health by X years with medicine available Y
> years from now is, of course, a matter of speculation on which experts
> disagree, just as with any other pioneering technology. I know that Holden
> and others have expressed caution (at best) concerning the accuracy of any
> kind of calculation of probabilities of particular outcomes in the distant
> (or even not-so-distant) future, and I share that view. However, an
> approach that may appeal more is to estimate how much humanitarian benefit
> a given amount of progress would deliver, and then to ask how unlikely that
> scenario needs to be to make it not worth pursuing. My claim is that the
> benefits of hastening the defeat of aging by even a few years (which is the
> minimum that I claim SENS Foundation is in a position to do, given adequate
> funding) would be so astronomical that the required chance of success to
> make such an effort worthwhile would be tiny - too tiny for it to be
> reasonable to argue that such funding would be inadvisable. But of course
> that is precisely what I would want GiveWell to opine on.
>
> - In the event that GiveWell (or anyone else) were to decide and declare
> that the defeat of aging is indeed a cause that philanthropists should
> support, there then arises the question of which organisation(s) should be
> supported in the best interests of that mission. We at SENS Foundation have
> worked diligently to rise as quickly as possible in the legitimacy stakes
> by all standard measures, but we are still young and there remains more to
> do. If I were to offer an argument to fund us rather than any other entity,
> it would largely come down to the fact that no other organisation has even
> a serious plan for defeating aging, let alone a track record of
> implementing such a plan's early stages.
>
> - A significant chunk of what we do is of a kind that I think comes under
> "meta-research". A prominent example is a project we're funding at Denver
> University to extend the well-respected forecasting system "International
> Futures" so that it can analyse scenarios incorporating dramatically
> postponed aging.
>
> I greatly welcome any feedback.
>
> Cheers, Aubrey
>
>
> --- In givewell@yahoogroups.com, Holden Karnofsky <holden0@...> wrote:
> >
> > Hi Vipul,
> >
> > Thanks for the thoughts. I had a followup conversation with Dario about
> > this topic a few days ago. I think the question of "could one fund
> > translational research to treat/prevent aging?" provides an interesting
> > illustration of some of the tricky dynamics here for a funder:
> >
> > - It's possible that if there were a great deal more attention giving to
>
> > treating/preventing aging, we would have some promising treatments. So
> in a
> > broad sense it's possible that aging is underinvested in.
> > - A lot of the best basic biology research isn't clearly pointing toward
>
> > one treatment/condition or another; it's about understanding the
> > fundamentals of how organisms operate. So having an interest in treating
> > aging, as opposed to cancer, might not have a major impact on which
> > projects one funds, if one's main goal is to fund outstanding basic
> biology
> > research.
> > - Perhaps because of the lack of emphasis on treating aging (or perhaps
>
> > because it's simply too difficult of a problem), there don't seem to be
> > promising findings in the "Valley of Death" relevant to aging; the few
> > promising leads have been explored.
> > - So even if, in a broad sense, there is too little attention given to
>
> > this problem, knowing this doesn't necessarily yield a clear direction
> for
> > a relatively small-scale funder of biomedical research.
> >
> > Best,
> > Holden
> >
> > On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:
> >
> > > **
>
> > >
> > >
> > > Holden,
> > >
> > > First, I think that this is an excellent document. I checked for a
> > > number of things that I had heard about (Breakout Labs, John
> > > Ioannidis, Cochrane Collaboration) and they're all there in your
> > > document.
> > >
> > > The one thing that's not explicitly mentioned: longevity and life
> > > extension research. At least prima facie, this seems like something
> > > that should be more important than individual disease research, and it
> > > seems like a classic "Valley of Death" case (pun unintended, but
> > > noted) -- T1 stage to use your terminology. I think the SENS website
> > > http://www.sens.org would be a good starting point for one of the (to
> > > me promising) approaches to life extension. I recall from past
> > > conversations that you were aware of SENS, so this is not new to you,
> > > but I think that longevity should be included as part of any
> > > discussion of biomedical research and given separate consideration
> > > given that it has a much lower status than research into specific
> > > conditions such as cancer, dementia, etc. You may ultimately conclude
> > > that not enough can be done in this area, but I think it should be
> > > part of your preliminary stuff. [btw, the United States has a National
> > > Institute of Aging, but it's much lower-status than most of the other
> > > grantmakers mentioned here].
> > >
> > > Vipul
> > >
> > > * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed) wrote
> > >
> > > > Note to the research list: we're now considering reopening our
> > > > investigation of the world of biomedical research. We've started and
> > > > stopped a couple of times in this area before; this time I decided to
> > > start
> > > > with a conversation with Dario Amodei, a longtime GiveWell follower
> and
> > > > personal friend who is currently a biology postdoc at Stanford. My
> goal
> > > > with the conversation was just to get some basic context and start
> > > putting
> > > > together a framework for thinking about the issue, not to use him as
> an
> > > > authoritative source, and the notes below should be read in that
> spirit.
> > > >
> > > >
> > > > This email has two sections:
> > > >
> > > >
> > > > 1. Notes that I emailed out internally after my conversation with
> Dario,
> > > > slightly edited
> > > >
> > > > 2. Some more context on the history of our work on biomedical
> research
> > > and
> > > > why we think it's appropriate to investigate this field (this was a
> > > > response to a question following my original email, along the lines
> of
> > > "Why
> > > > are you looking into biomedical research now, given that's an area
> with a
> > > > lot of buzz and funding from wealthy donors, and how does this work
> > > relate
> > > > to the 'meta-research' work?")
> > > >
> > > >
> > > >
> > > > *--*
> > > >
> > > > *
> > > > *
> > > >
> > > > *1. Notes that I emailed out internally after my conversation with
> Dario,
> > > > slightly edited*
> > > >
> > > > *
> > >
> > > > *
> > > >
> > > > I've done some preliminary work trying to figure out what it would
> look
> > > > like to explore biomedical sciences as funding area. This mostly
> > > consisted
> > > > of a 3-hour conversation with Dario (recording is available),
> reading two
> > > > papers he sent and a few I found while Googling, and prior
> knowledge. I'm
> > > > including Dario in all emails related to this stuff, as an informal
> > > advisor.
> > > >
> > > > * *
> > > >
> > > > *My picture of "what the biomedical research world roughly looks
> like"
> > > *(this
> > >
> > > > is mostly from talking with Dario + prior knowledge)
> > > >
> > > > · *Academic biology* studies how organisms work and develops tools to
> > >
> > > > observe and manipulate the building blocks of organisms.
> > > >
> > > > o The vast bulk of the funding - and the most prestigious funding -
> comes
> > > > from the NIH.
> > > >
> > > > o There is also funding from what I've heard called "foundations" -
> > > groups
> > > > like the American Cancer Society and American Heart Association -
> which
> > > > function very similarly to the NIH, in that they tend to hire people
> with
> > > > strong academic credentials and those people judge the merits of
> grant
> > > > proposals.
> > > >
> > > > o Both the NIH and "foundations" tend to be formally partitioned by
> > > > disease, but much of the work done by academic biologists is
> potentially
> > > > relevant to many diseases. A researcher seeking NIH funding may
> apply to
> > > > several different NIH "study sections," though only one at a time (a
> list
> > > > of "study sections" is at
> > > > http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx
> );
> > > > whether s/he gets funding is going to depend more on the academic
> merits
> > > of
> > > > the researcher & work than anything else.
> > > >
> > > > o There are basic definitions of "academic merit " that generally
> shape
> > > > the whole ecosystem: the people handing out money are selected by
> these
> > > > criteria and use these criteria, the people doing the research know
> that
> > > > these criteria shape their career prospects, etc.
> > > >
> > > > · *Private startups* investigate promising ideas for new
> > >
> > > > treatments/diagnostics/devices. They may often take the form of a
> biology
> > > > professor spinning off a biotech startup (run by former postdocs)
> that
> > > > raises venture capital, based on the research the professor did. They
> > > take
> > > > basic knowledge about how the body works (for example, protein X is
> > > crucial
> > > > for medical condition Y) and do the necessary testing to find a
> promising
> > > > treatment/diagnostic/device (for example, testing a lot of compounds
> on
> > > > animals until they find one that affects protein X).
> > > >
> > > > · *Big pharma/biotech companies *are best positioned to deal with the
> > >
> > > > extremely expensive process of conducting clinical trials and
> getting FDA
> > > > approval. Acquisition by one of these is the most common form of
> exit for
> > > > startups.
> > > >
> > > > · *Academic medicine, epidemiology and other fields* also do work
> > >
> > > > relevant to medicine, including studying questions whose main
> relevance
> > > is
> > > > to medical practice and public health programs: how effective is
> > > > treatment/practice X in situation Y, how cost-effective is it, etc.
> > > > Sometimes they will hit on commercializable insights (for example, a
> new
> > > > kind of device) as well.
> > > >
> > > > · *Translational research* is a broad term referring to a bridge
> between
> > >
> > > > academic research and treatments/practices. It can include (
> > > >
> > >
> http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> > > > ):
> > > >
> > > > o T1 - going from basic science (usually academic biology, I
> presume) to
> > > a
> > > > new treatment/practice. This includes research that helps go from an
> > > > academic biology insight to a private startup.
> > > >
> > > > o T2 - I think this is basically what Cochrane does - going from
> academic
> > > > medicine/epidemiology (a bunch of studies on what
> treatments/practices
> > > are
> > > > effective) to the development of guidelines that actually affect
> > > practice.
> > > >
> > > > o T3, T4 - research on how to actually change practice (as opposed to
> > > > setting the guidelines that are a "target" for practice) and get
> better
> > > > real-world results.
> > > >
> > > > *
> > > > *
> > > >
> > > > *Potential "big opportunity to do good" #1: translational research
> and
> > > the
> > >
> > > > "Valley of Death" (this comes from prior knowledge, googling "Valley
> of
> > > > Death" and reading a bunch of the stuff I found, and checking out the
> > > > FasterCures website again)*
> > >
> > > >
> > > >
> > > >
> > > > The "Valley of Death" seems to mostly refer to a shortage of T1. The
> idea
> > > > is that there are a lot of cases where there's an academic insight
> that's
> > > > potentially valuable in coming up with a new treatment, but to get
> it to
> > > > the point where it's attractive from a for-profit perspective, you
> need
> > > to
> > > > do a lot of stuff that academics don't have a reason to do. "For
> example,
> > > > an upstream finding that a given protein is differentially expressed
> in
> > > > individuals with a particular disease may suggest that the protein
> merits
> > > > further investigation. However, much more work (especially medicinal
> > > > chemistry) is necessary to determine how good a target the protein
> really
> > > > is and whether a marketable drug candidate that affects the activity
> of
> > > the
> > > > protein is likely to be developed." (
> > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> > > >
> > > >
> > > >
> > > > There are claims that this sort of work is massively underfunded (by
> the
> > > > people we've spoken to who talked about the "Valley of Death"; also
> in
> > > >
> > >
> http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
> > > (1).pdf).
> > > > However, funding isn't the only issue. The other issue is that
> > > > "pharmaceutical firms that hold libraries of potentially useful small
> > > > molecules as trade secrets, making them largely off limits to ...
> > > academic
> > > > scientists" (
> > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> > > > In other words, there is some room for new models of collaboration
> that
> > > > lead to better communication and information sharing between
> academia and
> > > > industry (or between industry and industry).
> > > >
> > > >
> > > > Both the Myelin Repair Foundation and Michael Milken's work on
> prostate
> > > > cancer have been pointed to as examples of innovative collaborations
> that
> > > > deal with some of the information sharing problems. Milken's model:
> > > > "drastically cutting the wait time for grant money, to flood the
> field
> > > with
> > > > fast cash, to fund therapy-driven ideas rather than basic science, to
> > > hold
> > > > researchers he funds accountable for results, and to demand
> collaboration
> > > > across disciplines and among institutions, private industry, and
> > > academia."
> > > > (
> > > >
> > >
> http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
> > > )
> > > > Myelin Repair Foundation sounds broadly similar (
> > > >
> > >
> http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
> > > ).
> > > >
> > > >
> > > >
> > > >
> > > > More broadly:
> > > >
> > > > · FasterCures (also a Michael Milken production) looks like it's
> focused
> > > > on the broad mission of "more research with a shorter timeline to
> > > > treatments," with a heavy sub-focus on the Valley of Death. In
> addition
> > > to
> > > > its conference and philanthropic advisory service, it advocates for
> FDA
> > > > improvements (presumably to speed the approval process), advocates
> for
> > > the
> > > > NIH to put more funding into translational research (there have
> > > definitely
> > > > been a lot of new initiatives at the NIH focused on this stuff in the
> > > past
> > > > few ~decade), promotes "innovative financing mechanisms" for
> bridging the
> > > > Valley of Death (these include Peter Thiel's Breakout Labs; I think
> they
> > > > occupy a conceptually similar space to "social enterprise investment"
> > > > though they tend to be structured more as grants and less as
> > > > double-bottom-line investments), and works on getting patient data
> opened
> > > > to researchers. The only program of theirs I haven't mentioned is
> TRAIN;
> > > I
> > > > can't (easily) figure out what this is.
> > > >
> > > > · John Ioannidis stated to us that all translational research is
> > > > underfunded, not just T1. (The context we talked to him in, of
> course,
> > > was
> > > > T2.)
> > > >
> > > >
> > > > These issues seem to have quite a bit of buzz. There are some really
> > > stark
> > > > #'s out there: even as R&D investment has gone way up over the past
> 50
> > > > years, the # of new drugs has stayed roughly constant at around 20 a
> > > year.
> > > >
> > > >
> > > > Dario sent a really interesting paper on this topic. It argues: (a)
> the #
> > > > of NMEs (new medical entities, i.e., drugs) per year has mostly
> remained
> > > > constant-with-noise (no trend); (b) all of the big companies seem to
> have
> > > > produced NMEs at a very steady pace, even as they've changed size,
> though
> > > > different companies do have different rates of NME creation; (c)
> when it
> > > > comes to mergers & acquisitions, "1+1=1": companies that acquire
> other
> > > > companies just keep up the same NME pace; (d) over time, the # of
> large
> > > > companies has shrunk (due to mergers) and the # of small companies
> has
> > > > risen, and the share of NMEs attributed to small companies has gone
> from
> > > > ~30% to ~70%; (e) the cost per NME has gone up over time and is now
> in
> > > the
> > > > neighborhood of $1 billion; (f) the % of NMEs that are "blockbusters"
> > > (huge
> > > > profits) has been steady around 20%, despite intensifying efforts on
> the
> > > > part of pharma to fund only potential blockbusters.
> > > >
> > > >
> > > >
> > > > There are also claims - such as by Derek Lowe, a blogger Dario
> pointed me
> > > > to - that the "Valley of Death" is an overrated concept: there is a
> high
> > > > ratio of academic discoveries to useful drugs, but this is just
> because a
> > > > lot of stuff turns out not to work, not because we have a shortage of
> > > > doable translational research. The paper above sort of takes this
> view,
> > > > essentially arguing that nothing seems to raise NME production; we'll
> > > need
> > > > something really radical to make any difference.
> > > >
> > > >
> > > >
> > > > The case studies in the FasterCures paper on this are interesting.
> They
> > > > imply that there were some pretty low-hanging fruit in the T1
> domain. It
> > > > might be interesting to talk to Fastercures and see whether they can
> help
> > > > identify "Valley of Death" opportunities that are slam-dunkish.
> > > >
> > > >
> > > > *Potential "big opportunity to do good" #2: inefficiencies in
> academic
> > > > biology*
> > >
> > > >
> > > >
> > > >
> > > > All of the above seems pretty distinct from the question of how to
> make
> > > > academic biology better at doing its job of understanding the body.
> This
> > > is
> > > > the question that Dario and I focused on.
> > > >
> > > >
> > > >
> > > > Dario painted a picture in which most of academia plays by the same
> set
> > > of
> > > > rules, making it very hard for people to do things that break those
> rules
> > > > (for example, academics are expected to publish a lot; it's risky to
> work
> > > > on a "blue sky" or highly ambitious project). Thus, for anyone who
> wants
> > > a
> > > > career in academia, a couple of years working on a moonshot project
> are
> > > > risky; in addition, being on bad terms with a small number of people
> can
> > > > damage a career (since there is often a small set of people that
> makes a
> > > > large proportion of the career-impacting decisions for a given area,
> and
> > > it
> > > > can be hard to escape this set of people without changing research
> > > > interests significantly). The bad news is that this isn't
> particularly
> > > easy
> > > > to fix: you can offer funding for blue-sky projects, but (a) a lot of
> > > > academics basically train themselves to play by the rules, and won't
> > > > necessarily have thought about "what should be studied if these rules
> > > > didn't apply," and more importantly, (b) there are many incentives
> > > pushing
> > > > academics towards playing by certain rules; funding is only one of
> those
> > > > mechanisms (there's also tenure, peer review, etc) so changing that
> one
> > > > incentive won't always change behavior. Dario says that he might
> hesitate
> > > > to work on a particular blue-sky project that he thinks is
> interesting,
> > > > even if he got funding for it, for these reasons.
> > > >
> > > >
> > > > Despite this, there are some funders who push the boundaries. There
> are
> > > > medical centers that don't require teaching and do more ambitious
> work.
> > > > There's the McKnight foundation, which funded some of the pioneering
> work
> > > > on optical control of neurons for which funding might have been
> difficult
> > > > to obtain by traditional routes. There's the HHMI Janelia Farms
> campus,
> > > > which Dario thinks is the most promising thing out there in terms of
> a
> > > > model - academics who go there get guaranteed (and generous) funding
> for
> > > 6
> > > > years, which frees them up to take much bigger risks. (I saw a study
> > > > claiming that the work coming out of Janelia Farms has a much more
> skewed
> > > > citation count distribution, implying bigger risks - i.e., fewer
> papers
> > > > with decent #'s of citations but more "blockbuster" papers with lots
> of
> > > > citations). But Janelia Farms is only in one sub-field; comparable
> > > > institutions don't exist for other fields (as far as Dario knows)..
> > > >
> > > >
> > > >
> > > > Dario's gut is that one of the best things a funder could do would
> be to
> > > > work toward creating a large institute that largely "plays by its own
> > > > rules," encouraging more ambitious work and providing enough
> security and
> > > > sheer volume of dollars/researchers as to establish a sort of
> "parallel
> > > > system" to academia - thus becoming a place that could provide
> viable and
> > > > reliable career options for people interested in playing by different
> > > > rules. I'm also interested in the idea of trying to advocate for
> changes
> > > in
> > > > the rules, as we've discussed in the context of meta-research.
> > > >
> > > >
> > > >
> > > > There's also the other stuff like data/code sharing.
> > > >
> > > >
> > > >
> > > > *Another option: give $ to the NIH*
> > >
> > > >
> > > >
> > > >
> > > > The NIH is by far the largest, most prestigious, most respected
> funder in
> > > > academic biology, and most of academia runs on criteria that mirror
> the
> > > > NIH's. I would guess that an unrestricted check to the NIH would get
> > > > allocated in a pretty sector-agnostic way. It seems like this is a
> giving
> > > > option that is pretty nontrivial to beat. Anyone we consider for
> funding
> > > > ought to be able to explain why they're better at allocating the
> funds
> > > than
> > > > the NIH. We haven't talked to the NIH about whether it would accept
> these
> > > > sorts of donations.
> > > >
> > > >
> > > > With the context I now have, FasterCures strikes me as pretty
> > > > promising/interesting too, though their "room for more funding"
> situation
> > > > isn't clear (w/o talking to them). But they also seem sector-agnostic
> > > > within biomedical research, while pushing a very specific theory of
> > > change
> > > > that may or may not be valid.
> > > >
> > > >
> > > >
> > > > *Next steps*
> > >
> > > >
> > > >
> > > >
> > > > I think understanding this world would be a major undertaking. I
> think
> > > we'd
> > > > have to be ready to put in a lot of work and to be open to ways of
> > > bridging
> > > > "funder-expert gap" that we haven't tried before. For example:
> > > >
> > > > · Dario suggested that I read a freshman biology textbook. He said he
> > > > realizes that the time cost could be very large - something like 100
> > > hours
> > > > - but that without doing so, I'm going to be lacking too much
> context on
> > > > why biomedical research works the way it does. He thinks that
> reading a
> > > > basic textbook would get me to the point of strongly diminishing
> marginal
> > > > returns.
> > > >
> > > > · Also in the category of "personally picking up rudimentary subject
> > > > matter knowledge," I thought it might be worth trying to follow the
> > > > development of a particular drug from start to finish - Gleevec (the
> > > > "miracle drug") would be a good candidate. The goal would be to
> > > understand
> > > > each stage of insight leading to new investigation, and where the
> funding
> > > > came from at each stage.
> > > >
> > > > · I think hiring Dario or someone like him would make a lot of sense.
> > > > I've thought about whether we should be hiring "subject matter
> experts"
> > > in
> > > > other areas, such as global health, but in my view the need is
> clearer
> > > here
> > > > than in any other area. One of the things I don't love about hiring
> an
> > > > expert in a given field, at this stage of our research, is that we
> could
> > > > quickly decide that we're just not interested enough in the field in
> > > > question ... but someone with the right kind of technical knowledge &
> > > > experience would be so far ahead of us in evaluating *any* area of
> > > biology
> > > > research that it seems like a good idea. (JTBC, I'm also actively
> > > thinking
> > > > about whether it would make sense to hire experts in other fields
> ...)
> > > >
> > > > · Talking to major funders and potentially co-funding with them is
> > > > probably essential. Important groups to talk to would include NIH
> (by far
> > > > the most important; we've already talked to them a bit), the
> colloquium
> > > of
> > > > groups like the American Cancer Society (it has a name; I forget the
> > > name),
> > > > FasterCures, Wellcome, and potentially some funders with
> > > > unorthodox/buzzed-about approaches: Myelin Repair Foundation,
> McKnight
> > > > Foundation, and maybe another private foundation or two (I emailed
> Dario
> > > > all my notes on major foundations that do biomedical research so he
> could
> > > > send thoughts on whether any look interesting aside from McKnight).
> > > >
> > > >
> > > > My next plan had been to talk to academics (Dario had good
> suggestions
> > > > about how to approach them), but with all the potential work to be
> done
> > > on
> > > > gaining basic context, I'm not sure that's the right next step. But
> it's
> > > > also a possible step.
> > > >
> > > >
> > > >
> > > > *2. Some more context on the history of our work on biomedical
> research
> > > and
> > >
> > > > why we think it's appropriate to investigate this field (this was a
> > > > response to a question following my original email, along the lines
> of
> > > "Why
> > > > are you looking into biomedical research now, given that's an area
> with a
> > > > lot of buzz and funding from wealthy donors, and how does this work
> > > relate
> > > > to the 'meta-research' work?")*
> > > >
> > > >
> > > > - We've always had "disease research" as a cause we wanted to
> > >
> > > > investigate. We've never had a good reason for not investigating it
> other
> > > > than that investigating it seems difficult. We've tried more than
> once to
> > > > investigate it, and it's ended up falling by the wayside because of
> how
> > > big
> > > > an undertaking it is.
> > > > - It's true that there is a huge amount of funding and buzz in this
> > >
> > > > area. But it also seems quite possible that there isn't nearly
> enough; in
> > > > fact this seems like a possible point of consensus between all the
> people
> > > > concerned about the "Valley of Death." This potential good
> accomplished
> > > via
> > > > biomedical research appears unlimited. What I consider to be the
> mark of
> > > a
> > > > bad sector is "tons of buzz/$ and mediocre returns," but the latter
> part
> > > > doesn't apply to biomed; in fact biomedical research is arguably one
> of
> > > > philanthropy's and even humanity's top success stories.
> > > > - There are also indications that despite all the buzz and funding,
> > >
> > > > there are still opportunities to do things differently and shake
> things
> > > up.
> > > > There is more than one case where an outsider (Milken, MRF) basically
> > > came
> > > > in and did things very differently and now even experts in the field
> seem
> > > > to credit them with positive change. There's also a good explanation
> for
> > > > why this might be the case: while there are a ton of people and
> dollars,
> > > > they largely seem to play by one
> self-reinforcing/network-effect-prone
> > > set
> > > > of rules, implying high returns to disrupting that equilibrium.
> > > > - So, we've always wanted to get into this area. There was a period
> > >
> > > > where I was presenting meta-research as our best entry point into
> this
> > > > field: my vision was that we would talk to academics about what
> > > systematic
> > > > failings there were and what funding opportunities these implied, and
> > > that
> > > > would be as good a way as any to get acclimated in biomed. But this
> > > period
> > > > wasn't particularly long - the May blog post on Labs priority causes
> > > lists
> > > > scientific research as a promising area distinct from meta-research.
> > > > - I set up the call with Dario without having a clear idea of
> whether I
> > >
> > > > wanted to approach biomed from a "meta-research" angle or another
> angle.
> > > > After the call and other investigations described here, I got a
> clearer
> > > > idea of what I think is the best path forward.
> > > >
> > > > Bottom line - I think it's important to build an understanding of
> > > > biomedical research, and that we should take the best path to doing
> so
> > > > whether or not that dovetails with the meta-research work (likely it
> will
> > > > dovetail some but not 100%).
> > >
> > >
> > >
> >
>
>  
>

Hi Holden - many thanks.

First: yes, there are really three somewhat separate questions for someone trying to evaluate whether to support SENS Foundation:

1) Is the medical control of aging a hugely valuable mission?

2) Assuming "yes" to (1), is it best achieved by basic research or translational research?

3) Assuming translational, is SENS Foundation the organisation that uses money most effectively in pursuit of that mission?

I had rather expected that you would take some convincing on item (1), and much of what I wrote last time was focused on that. Since it isn't the focus of your question to me, I'm now going to assume until further notice that there is no dissent on that.

So, to answer your question: actually you're not putting aside the basic-vs-translational question as much as you may think you are. The word "translational" is flavour of the month in government funding circles these days (not only in the USA), so it's not surprising that the NIA has a public statement of the kind you pointed to. However, notice that the link they give "for more information" is to a page listing ALL "Funding Opportunity Announcements". There is no page specifically for translational ones, and the reason there isn't is that the amount of work that the NIA actually funds that could really be called translational is tiny. In other words, the page you found is actually just blatant spin. The neuroscience slice you mention is an anomaly arising from the way NIA was founded (the natural place for that money is clearly NINDS): the fact that it's NIA money does not, in practice, translate into its being spent on work to prevent neurodegeneration by treating its cause (aging). Instead, just like NINDS money, it's spent on attacking neurodegeneration directly, as if such diseases could be eliminated from the body just like an infection: the same old mistake that afflicts, and dooms, the whole of geriatric medicine.

So, the first answer to your question is that SENS Foundation really DOES focus on translational research, with an explicit goal of postponing age-related ill-health. But there's also another big difference: we can attack this problem relatively free of the other priorities that afflict mainstream funding (whether from NIH or from trasitional foundations). Most importantly, though we do and will continue to publish our interim results in the peer-reviewed literature, we are much less constrained by "publish or perish" tyranny than typical academics are. This allows us to proceed by constructing and implementing a rational "project plan" (namely SENS) to get to the intended goal (the defeat of aging), whereas what little translational work is funded by NIA or others is guided overwhelmingly by the imperative to get some kind of positive result as quickly as possible, even when it's understood that those results are not remotely likely to "scale", i.e. to translate into eventual medical treatments that significantly delay aging. A great example of this is the NIA's Interventions Testing Program (ITP) to test the mouse longevity effects of various small molecules. The ITP only exists at all (and in a far smaller form than originally intended) as a result of several years of persistence by the then head of the NIA's biology division (Huber Warner), and it focuses entirely on delivery of simple drugs starting rather early in life, with the result that no information emerges that's relevant to treating people who are already in middle age or older. (This is despite the fact that by far the most high-profile result that the ITP has delivered so far, the benefits of rapamycin, actually WAS a late-onset study: it wasn't meant to be, but technical issues delayed the experiment.) In a nutshell, there is a huge bias against high-risk high-gain work.

The third thing that distinguishes SENS Foundation's approach is that we can transcend the "balkanisation" (silo mentality) that dominates mainstream academic funding. When one submits a grant application to NIA, it is evaluated by gerontologists, just as when one submits to NCI it is evaluated by oncologists, etc. What's wrong with this is that it biases the system immensely against cross-disciplinary proposals. SENS is a plan that brings together a large body of knowledge from gerontology but also a huge amount of expertise that was developed for other reasons entirely - to treat acute disease/injury, or in some cases for purposes that were not biomedical at all (notably environmental decontamination). It doesn't matter how robust the objective scientific and technological argument is for work of that sort: it will never compete (especially in today's very tight funding environment) with more single-topic proposals all of whose details can be understood by reviewers from a particular single field.

The final thing to mention, and this actually also answers your question to Vipul about basic versus translational research, is that SENS is a plan that has stood the test of time. I've been propounding it since 2000, well before SENS Foundation existed, and it used to come in for a lot of criticism (initially more in the form of off-the-record ridicule, and latterly, at my behest, in print), but in every single case that criticism was found to stem from ignorance on the part of the detractor, either of what I proposed or of published experimental work on which the proposal was based. That's why I'm now regularly asked to organise entire sessions at mainstream gerontology conferences, whereas as little as five years ago I would never even be invited to speak. It's also why the Research Advisory Board of SENS Foundation consists of such prestigious scientists. This is a very strong argument, in my view, for believing that now is the time to sink a proper amount of money into translational gerontology (though certainly not to cease doin basic biogerontology too). It's well known that basic scientists are often not the most far-sighted when it comes to seeing how to apply their discoveries (attitudes in 1900 to the feasibility of powered flight being the canonical example). It is therefore a source of concern that almost all the experts who have the ear of funders in this field are basic scientists, whose instinct is to carry on finding things out and to deprioritise the tedious business of applying that knowledge. SENS has achieved a gratisfying level of legitimacy in gerontology, but it is still foreign to most card-carrying gerontologists, and as such it remains essentially unfundable via mainstream mechanisms. Hence the need to create a philanthropy-driven entity, SENS Foundation, to get this work done.

Let me know if this helps, or if you have further questions.

Cheers, Aubrey

On 12 Oct 2012, at 15:28, Holden Karnofsky wrote:

> 
> Hi Aubrey,
> 
> Thanks for the thoughts.
> 
> The NIH appears to have a division focused on research relevant to this topic: http://www.nia.nih.gov/research/dab . Its budget appears to be ~$175 million (per year). The National Institute on Aging, which houses this division, has a budget of about $1 billion per year, including a separate ~$400 million for neuroscience (which may also be relevant) as well as $115 million for intramural research. Figures are from http://www.nia.nih.gov/about/budget/2012/fiscal-year-2013-budget. The Institute states that its mandate includes translational research (http://www.nia.nih.gov/research/faq/does-nia-support-translational-research). How would you distinguish your work from this work?
> 
> (For the moment I'm putting aside the question I raised in my previous response to Vipul on this topic, regarding whether it's best to approach biology funding from the perspective of "trying to treat/cure a particular condition" or "trying to understand  fundamental questions in biology whose applications are difficult to predict.")
> 
> Best,
> Holden
> 
> On Fri, Oct 12, 2012 at 7:32 AM, aubrey.degrey <aubrey@sens.org> wrote:
>  
> Hi everyone,
> 
> My attention was brought to this thread, by virtue of the fact that it was my work that gave rise to SENS Foundation, and I'm looking forward to getting more involved here; I've held the Effective Altruism movement in high regard for some time. However, given my newbie status here I want to start by apologising in advance for any oversight of previously-discussed issues etc. I'm naturally delighted both at Holden's post and at Vipul's reply (which I should stress that I did not plant! - I do not know Vipul at all, though I look forward to changing that). I would like to mention just a few key points for discussion:
> 
> - Holden, I want to compliment you on your appreciation of how academia really works. Everything you say about that is spot on. The aversion to "high risk high gain" work that has arisen and become so endemic in the system is the most important point here, in terms of why parallel funding routes are needed.
> 
> - I'm slightly confused that a lot of Holden's remarks are focused on the private sector (i.e. startups), since my understanding was that GiveWell is about philanthropy; but I realise that there is not all that clear a boundary between the two (and I note the mention of Breakout Labs, with which I have close links and which sits astride that divide more than arguably anyone). The "valley of death" in pre-competitive translational research is a rather different one than that encountered by startups, but the principle is the same, and research to postpone aging certainly encounteres it.
> 
> - Something that I presume factors highly among GiveWell's criteria is the extent to which a cause may be undervalued by the bulk of major philanthropists, such that an infusion of additional funds would make more of a difference than in an area that is already being well funded. To me this seems to mirror the logic of focusing on the shortcomings (gaps) in NIH's funding (and that of traditional-model foundations). Holden notes that "Anyone we consider for funding ought to be able to explain why they're better at allocating the funds than the NIH" and I agree wholeheartedly, but my inference is that he thinks that some orgs may indeed be able to explain that. I certainly think that SENS Foundation can.
> 
> - Coming to aging: research to postpone aging has the unique problem of quite indescribeable irrationality on the part of most of the general public, policy-makers and even biologists with regard to its desirability. Biogerontologists have been talking to brick walls for decades in their effort to get the rest of the world to appreciate that aging is what causes age-related ill-health, and thus that treatments for aging are merely preventative geriatrics. The concept persists, despite biogerontologists' best efforts, that aging is "natural" and should be left alone, whereas the diseases that it brings about are awful and should be fought. This is made even more bizarre by the fact that the status of age-related diseases as aspects of the later stages of aging absolutely, unequivocally implies that efforts to attack those diseases directly are doomed to fail. As such, this is a (unique? certainly very rare) case where a philanthropic contribution can make a particularly big difference simply because most philanthropists don't see the case for it. It underpins why having an interest in treating aging, as opposed to cancer, absolutely has a major impact on which projects one funds. It's also a case for (if I understand the term correctly) meta-research.
> 
> - A lot of the chatter about treating aging revolves around longevity, but it shouldn't. I'm all in favour of longevity, don't get me wrong, but it's not what gets me up in the morning: what does is health. I want people to be truly youthful, however long ago they were born: simple as that. The benefits of longevity per se to humanity may also be substantial, in the form of greater wisdom etc, but that would necessarily come about only very gradually (we won't have any 1000-year-old for at least 900 years whatever happens!), so it doesn't figure strongly in my calculations.
> 
> - When forced to acknowledge that the idea of aging being a high-priority target for medicine is an inescapeable consequence of things they already believe (notably that health is good and ageism is bad), many people retreat to the standpoint that it's never going to be possible so it's OK to be irrational about whether it's desirable. The feasibility of postponing age-related ill-health by X years with medicine available Y years from now is, of course, a matter of speculation on which experts disagree, just as with any other pioneering technology. I know that Holden and others have expressed caution (at best) concerning the accuracy of any kind of calculation of probabilities of particular outcomes in the distant (or even not-so-distant) future, and I share that view. However, an approach that may appeal more is to estimate how much humanitarian benefit a given amount of progress would deliver, and then to ask how unlikely that scenario needs to be to make it not worth pursuing. My claim is that the benefits of hastening the defeat of aging by even a few years (which is the minimum that I claim SENS Foundation is in a position to do, given adequate funding) would be so astronomical that the required chance of success to make such an effort worthwhile would be tiny - too tiny for it to be reasonable to argue that such funding would be inadvisable. But of course that is precisely what I would want GiveWell to opine on.
> 
> - In the event that GiveWell (or anyone else) were to decide and declare that the defeat of aging is indeed a cause that philanthropists should support, there then arises the question of which organisation(s) should be supported in the best interests of that mission. We at SENS Foundation have worked diligently to rise as quickly as possible in the legitimacy stakes by all standard measures, but we are still young and there remains more to do. If I were to offer an argument to fund us rather than any other entity, it would largely come down to the fact that no other organisation has even a serious plan for defeating aging, let alone a track record of implementing such a plan's early stages. 
> 
> - A significant chunk of what we do is of a kind that I think comes under "meta-research". A prominent example is a project we're funding at Denver University to extend the well-respected forecasting system "International Futures" so that it can analyse scenarios incorporating dramatically postponed aging.
> 
> I greatly welcome any feedback.
> 
> Cheers, Aubrey
> 
> 
> 
> --- In givewell@yahoogroups.com, Holden Karnofsky <holden0@...> wrote:
> >
> > Hi Vipul,
> > 
> > Thanks for the thoughts. I had a followup conversation with Dario about
> > this topic a few days ago. I think the question of "could one fund
> > translational research to treat/prevent aging?" provides an interesting
> > illustration of some of the tricky dynamics here for a funder:
> > 
> > - It's possible that if there were a great deal more attention giving to
> 
> > treating/preventing aging, we would have some promising treatments. So in a
> > broad sense it's possible that aging is underinvested in.
> > - A lot of the best basic biology research isn't clearly pointing toward
> 
> > one treatment/condition or another; it's about understanding the
> > fundamentals of how organisms operate. So having an interest in treating
> > aging, as opposed to cancer, might not have a major impact on which
> > projects one funds, if one's main goal is to fund outstanding basic biology
> > research.
> > - Perhaps because of the lack of emphasis on treating aging (or perhaps
> 
> > because it's simply too difficult of a problem), there don't seem to be
> > promising findings in the "Valley of Death" relevant to aging; the few
> > promising leads have been explored.
> > - So even if, in a broad sense, there is too little attention given to
> 
> > this problem, knowing this doesn't necessarily yield a clear direction for
> > a relatively small-scale funder of biomedical research.
> > 
> > Best,
> > Holden
> > 
> > On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:
> > 
> > > **
> 
> > >
> > >
> > > Holden,
> > >
> > > First, I think that this is an excellent document. I checked for a
> > > number of things that I had heard about (Breakout Labs, John
> > > Ioannidis, Cochrane Collaboration) and they're all there in your
> > > document.
> > >
> > > The one thing that's not explicitly mentioned: longevity and life
> > > extension research. At least prima facie, this seems like something
> > > that should be more important than individual disease research, and it
> > > seems like a classic "Valley of Death" case (pun unintended, but
> > > noted) -- T1 stage to use your terminology. I think the SENS website
> > > http://www.sens.org would be a good starting point for one of the (to
> > > me promising) approaches to life extension. I recall from past
> > > conversations that you were aware of SENS, so this is not new to you,
> > > but I think that longevity should be included as part of any
> > > discussion of biomedical research and given separate consideration
> > > given that it has a much lower status than research into specific
> > > conditions such as cancer, dementia, etc. You may ultimately conclude
> > > that not enough can be done in this area, but I think it should be
> > > part of your preliminary stuff. [btw, the United States has a National
> > > Institute of Aging, but it's much lower-status than most of the other
> > > grantmakers mentioned here].
> > >
> > > Vipul
> > >
> > > * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed) wrote
> > >
> > > > Note to the research list: we're now considering reopening our
> > > > investigation of the world of biomedical research. We've started and
> > > > stopped a couple of times in this area before; this time I decided to
> > > start
> > > > with a conversation with Dario Amodei, a longtime GiveWell follower and
> > > > personal friend who is currently a biology postdoc at Stanford. My goal
> > > > with the conversation was just to get some basic context and start
> > > putting
> > > > together a framework for thinking about the issue, not to use him as an
> > > > authoritative source, and the notes below should be read in that spirit.
> > > >
> > > >
> > > > This email has two sections:
> > > >
> > > >
> > > > 1. Notes that I emailed out internally after my conversation with Dario,
> > > > slightly edited
> > > >
> > > > 2. Some more context on the history of our work on biomedical research
> > > and
> > > > why we think it's appropriate to investigate this field (this was a
> > > > response to a question following my original email, along the lines of
> > > "Why
> > > > are you looking into biomedical research now, given that's an area with a
> > > > lot of buzz and funding from wealthy donors, and how does this work
> > > relate
> > > > to the 'meta-research' work?")
> > > >
> > > >
> > > >
> > > > *--*
> > > >
> > > > *
> > > > *
> > > >
> > > > *1. Notes that I emailed out internally after my conversation with Dario,
> > > > slightly edited*
> > > >
> > > > *
> > >
> > > > *
> > > >
> > > > I've done some preliminary work trying to figure out what it would look
> > > > like to explore biomedical sciences as funding area. This mostly
> > > consisted
> > > > of a 3-hour conversation with Dario (recording is available), reading two
> > > > papers he sent and a few I found while Googling, and prior knowledge. I'm
> > > > including Dario in all emails related to this stuff, as an informal
> > > advisor.
> > > >
> > > > * *
> > > >
> > > > *My picture of "what the biomedical research world roughly looks like"
> > > *(this
> > >
> > > > is mostly from talking with Dario + prior knowledge)
> > > >
> > > > · *Academic biology* studies how organisms work and develops tools to
> > >
> > > > observe and manipulate the building blocks of organisms.
> > > >
> > > > o The vast bulk of the funding - and the most prestigious funding - comes
> > > > from the NIH.
> > > >
> > > > o There is also funding from what I've heard called "foundations" -
> > > groups
> > > > like the American Cancer Society and American Heart Association - which
> > > > function very similarly to the NIH, in that they tend to hire people with
> > > > strong academic credentials and those people judge the merits of grant
> > > > proposals.
> > > >
> > > > o Both the NIH and "foundations" tend to be formally partitioned by
> > > > disease, but much of the work done by academic biologists is potentially
> > > > relevant to many diseases. A researcher seeking NIH funding may apply to
> > > > several different NIH "study sections," though only one at a time (a list
> > > > of "study sections" is at
> > > > http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
> > > > whether s/he gets funding is going to depend more on the academic merits
> > > of
> > > > the researcher & work than anything else.
> > > >
> > > > o There are basic definitions of "academic merit " that generally shape
> > > > the whole ecosystem: the people handing out money are selected by these
> > > > criteria and use these criteria, the people doing the research know that
> > > > these criteria shape their career prospects, etc.
> > > >
> > > > · *Private startups* investigate promising ideas for new
> > >
> > > > treatments/diagnostics/devices. They may often take the form of a biology
> > > > professor spinning off a biotech startup (run by former postdocs) that
> > > > raises venture capital, based on the research the professor did. They
> > > take
> > > > basic knowledge about how the body works (for example, protein X is
> > > crucial
> > > > for medical condition Y) and do the necessary testing to find a promising
> > > > treatment/diagnostic/device (for example, testing a lot of compounds on
> > > > animals until they find one that affects protein X).
> > > >
> > > > · *Big pharma/biotech companies *are best positioned to deal with the
> > >
> > > > extremely expensive process of conducting clinical trials and getting FDA
> > > > approval. Acquisition by one of these is the most common form of exit for
> > > > startups.
> > > >
> > > > · *Academic medicine, epidemiology and other fields* also do work
> > >
> > > > relevant to medicine, including studying questions whose main relevance
> > > is
> > > > to medical practice and public health programs: how effective is
> > > > treatment/practice X in situation Y, how cost-effective is it, etc.
> > > > Sometimes they will hit on commercializable insights (for example, a new
> > > > kind of device) as well.
> > > >
> > > > · *Translational research* is a broad term referring to a bridge between
> > >
> > > > academic research and treatments/practices. It can include (
> > > >
> > > http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> > > > ):
> > > >
> > > > o T1 - going from basic science (usually academic biology, I presume) to
> > > a
> > > > new treatment/practice. This includes research that helps go from an
> > > > academic biology insight to a private startup.
> > > >
> > > > o T2 - I think this is basically what Cochrane does - going from academic
> > > > medicine/epidemiology (a bunch of studies on what treatments/practices
> > > are
> > > > effective) to the development of guidelines that actually affect
> > > practice.
> > > >
> > > > o T3, T4 - research on how to actually change practice (as opposed to
> > > > setting the guidelines that are a "target" for practice) and get better
> > > > real-world results.
> > > >
> > > > *
> > > > *
> > > >
> > > > *Potential "big opportunity to do good" #1: translational research and
> > > the
> > >
> > > > "Valley of Death" (this comes from prior knowledge, googling "Valley of
> > > > Death" and reading a bunch of the stuff I found, and checking out the
> > > > FasterCures website again)*
> > >
> > > >
> > > >
> > > >
> > > > The "Valley of Death" seems to mostly refer to a shortage of T1. The idea
> > > > is that there are a lot of cases where there's an academic insight that's
> > > > potentially valuable in coming up with a new treatment, but to get it to
> > > > the point where it's attractive from a for-profit perspective, you need
> > > to
> > > > do a lot of stuff that academics don't have a reason to do. "For example,
> > > > an upstream finding that a given protein is differentially expressed in
> > > > individuals with a particular disease may suggest that the protein merits
> > > > further investigation. However, much more work (especially medicinal
> > > > chemistry) is necessary to determine how good a target the protein really
> > > > is and whether a marketable drug candidate that affects the activity of
> > > the
> > > > protein is likely to be developed." (
> > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> > > >
> > > >
> > > >
> > > > There are claims that this sort of work is massively underfunded (by the
> > > > people we've spoken to who talked about the "Valley of Death"; also in
> > > >
> > > http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
> > > (1).pdf).
> > > > However, funding isn't the only issue. The other issue is that
> > > > "pharmaceutical firms that hold libraries of potentially useful small
> > > > molecules as trade secrets, making them largely off limits to ...
> > > academic
> > > > scientists" (
> > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> > > > In other words, there is some room for new models of collaboration that
> > > > lead to better communication and information sharing between academia and
> > > > industry (or between industry and industry).
> > > >
> > > >
> > > > Both the Myelin Repair Foundation and Michael Milken's work on prostate
> > > > cancer have been pointed to as examples of innovative collaborations that
> > > > deal with some of the information sharing problems. Milken's model:
> > > > "drastically cutting the wait time for grant money, to flood the field
> > > with
> > > > fast cash, to fund therapy-driven ideas rather than basic science, to
> > > hold
> > > > researchers he funds accountable for results, and to demand collaboration
> > > > across disciplines and among institutions, private industry, and
> > > academia."
> > > > (
> > > >
> > > http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
> > > )
> > > > Myelin Repair Foundation sounds broadly similar (
> > > >
> > > http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
> > > ).
> > > >
> > > >
> > > >
> > > >
> > > > More broadly:
> > > >
> > > > · FasterCures (also a Michael Milken production) looks like it's focused
> > > > on the broad mission of "more research with a shorter timeline to
> > > > treatments," with a heavy sub-focus on the Valley of Death. In addition
> > > to
> > > > its conference and philanthropic advisory service, it advocates for FDA
> > > > improvements (presumably to speed the approval process), advocates for
> > > the
> > > > NIH to put more funding into translational research (there have
> > > definitely
> > > > been a lot of new initiatives at the NIH focused on this stuff in the
> > > past
> > > > few ~decade), promotes "innovative financing mechanisms" for bridging the
> > > > Valley of Death (these include Peter Thiel's Breakout Labs; I think they
> > > > occupy a conceptually similar space to "social enterprise investment"
> > > > though they tend to be structured more as grants and less as
> > > > double-bottom-line investments), and works on getting patient data opened
> > > > to researchers. The only program of theirs I haven't mentioned is TRAIN;
> > > I
> > > > can't (easily) figure out what this is.
> > > >
> > > > · John Ioannidis stated to us that all translational research is
> > > > underfunded, not just T1. (The context we talked to him in, of course,
> > > was
> > > > T2.)
> > > >
> > > >
> > > > These issues seem to have quite a bit of buzz. There are some really
> > > stark
> > > > #'s out there: even as R&D investment has gone way up over the past 50
> > > > years, the # of new drugs has stayed roughly constant at around 20 a
> > > year.
> > > >
> > > >
> > > > Dario sent a really interesting paper on this topic. It argues: (a) the #
> > > > of NMEs (new medical entities, i.e., drugs) per year has mostly remained
> > > > constant-with-noise (no trend); (b) all of the big companies seem to have
> > > > produced NMEs at a very steady pace, even as they've changed size, though
> > > > different companies do have different rates of NME creation; (c) when it
> > > > comes to mergers & acquisitions, "1+1=1": companies that acquire other
> > > > companies just keep up the same NME pace; (d) over time, the # of large
> > > > companies has shrunk (due to mergers) and the # of small companies has
> > > > risen, and the share of NMEs attributed to small companies has gone from
> > > > ~30% to ~70%; (e) the cost per NME has gone up over time and is now in
> > > the
> > > > neighborhood of $1 billion; (f) the % of NMEs that are "blockbusters"
> > > (huge
> > > > profits) has been steady around 20%, despite intensifying efforts on the
> > > > part of pharma to fund only potential blockbusters.
> > > >
> > > >
> > > >
> > > > There are also claims - such as by Derek Lowe, a blogger Dario pointed me
> > > > to - that the "Valley of Death" is an overrated concept: there is a high
> > > > ratio of academic discoveries to useful drugs, but this is just because a
> > > > lot of stuff turns out not to work, not because we have a shortage of
> > > > doable translational research. The paper above sort of takes this view,
> > > > essentially arguing that nothing seems to raise NME production; we'll
> > > need
> > > > something really radical to make any difference.
> > > >
> > > >
> > > >
> > > > The case studies in the FasterCures paper on this are interesting. They
> > > > imply that there were some pretty low-hanging fruit in the T1 domain. It
> > > > might be interesting to talk to Fastercures and see whether they can help
> > > > identify "Valley of Death" opportunities that are slam-dunkish.
> > > >
> > > >
> > > > *Potential "big opportunity to do good" #2: inefficiencies in academic
> > > > biology*
> > >
> > > >
> > > >
> > > >
> > > > All of the above seems pretty distinct from the question of how to make
> > > > academic biology better at doing its job of understanding the body. This
> > > is
> > > > the question that Dario and I focused on.
> > > >
> > > >
> > > >
> > > > Dario painted a picture in which most of academia plays by the same set
> > > of
> > > > rules, making it very hard for people to do things that break those rules
> > > > (for example, academics are expected to publish a lot; it's risky to work
> > > > on a "blue sky" or highly ambitious project). Thus, for anyone who wants
> > > a
> > > > career in academia, a couple of years working on a moonshot project are
> > > > risky; in addition, being on bad terms with a small number of people can
> > > > damage a career (since there is often a small set of people that makes a
> > > > large proportion of the career-impacting decisions for a given area, and
> > > it
> > > > can be hard to escape this set of people without changing research
> > > > interests significantly). The bad news is that this isn't particularly
> > > easy
> > > > to fix: you can offer funding for blue-sky projects, but (a) a lot of
> > > > academics basically train themselves to play by the rules, and won't
> > > > necessarily have thought about "what should be studied if these rules
> > > > didn't apply," and more importantly, (b) there are many incentives
> > > pushing
> > > > academics towards playing by certain rules; funding is only one of those
> > > > mechanisms (there's also tenure, peer review, etc) so changing that one
> > > > incentive won't always change behavior. Dario says that he might hesitate
> > > > to work on a particular blue-sky project that he thinks is interesting,
> > > > even if he got funding for it, for these reasons.
> > > >
> > > >
> > > > Despite this, there are some funders who push the boundaries. There are
> > > > medical centers that don't require teaching and do more ambitious work.
> > > > There's the McKnight foundation, which funded some of the pioneering work
> > > > on optical control of neurons for which funding might have been difficult
> > > > to obtain by traditional routes. There's the HHMI Janelia Farms campus,
> > > > which Dario thinks is the most promising thing out there in terms of a
> > > > model - academics who go there get guaranteed (and generous) funding for
> > > 6
> > > > years, which frees them up to take much bigger risks. (I saw a study
> > > > claiming that the work coming out of Janelia Farms has a much more skewed
> > > > citation count distribution, implying bigger risks - i.e., fewer papers
> > > > with decent #'s of citations but more "blockbuster" papers with lots of
> > > > citations). But Janelia Farms is only in one sub-field; comparable
> > > > institutions don't exist for other fields (as far as Dario knows)..
> > > >
> > > >
> > > >
> > > > Dario's gut is that one of the best things a funder could do would be to
> > > > work toward creating a large institute that largely "plays by its own
> > > > rules," encouraging more ambitious work and providing enough security and
> > > > sheer volume of dollars/researchers as to establish a sort of "parallel
> > > > system" to academia - thus becoming a place that could provide viable and
> > > > reliable career options for people interested in playing by different
> > > > rules. I'm also interested in the idea of trying to advocate for changes
> > > in
> > > > the rules, as we've discussed in the context of meta-research.
> > > >
> > > >
> > > >
> > > > There's also the other stuff like data/code sharing.
> > > >
> > > >
> > > >
> > > > *Another option: give $ to the NIH*
> > >
> > > >
> > > >
> > > >
> > > > The NIH is by far the largest, most prestigious, most respected funder in
> > > > academic biology, and most of academia runs on criteria that mirror the
> > > > NIH's. I would guess that an unrestricted check to the NIH would get
> > > > allocated in a pretty sector-agnostic way. It seems like this is a giving
> > > > option that is pretty nontrivial to beat. Anyone we consider for funding
> > > > ought to be able to explain why they're better at allocating the funds
> > > than
> > > > the NIH. We haven't talked to the NIH about whether it would accept these
> > > > sorts of donations.
> > > >
> > > >
> > > > With the context I now have, FasterCures strikes me as pretty
> > > > promising/interesting too, though their "room for more funding" situation
> > > > isn't clear (w/o talking to them). But they also seem sector-agnostic
> > > > within biomedical research, while pushing a very specific theory of
> > > change
> > > > that may or may not be valid.
> > > >
> > > >
> > > >
> > > > *Next steps*
> > >
> > > >
> > > >
> > > >
> > > > I think understanding this world would be a major undertaking. I think
> > > we'd
> > > > have to be ready to put in a lot of work and to be open to ways of
> > > bridging
> > > > "funder-expert gap" that we haven't tried before. For example:
> > > >
> > > > · Dario suggested that I read a freshman biology textbook. He said he
> > > > realizes that the time cost could be very large - something like 100
> > > hours
> > > > - but that without doing so, I'm going to be lacking too much context on
> > > > why biomedical research works the way it does. He thinks that reading a
> > > > basic textbook would get me to the point of strongly diminishing marginal
> > > > returns.
> > > >
> > > > · Also in the category of "personally picking up rudimentary subject
> > > > matter knowledge," I thought it might be worth trying to follow the
> > > > development of a particular drug from start to finish - Gleevec (the
> > > > "miracle drug") would be a good candidate. The goal would be to
> > > understand
> > > > each stage of insight leading to new investigation, and where the funding
> > > > came from at each stage.
> > > >
> > > > · I think hiring Dario or someone like him would make a lot of sense.
> > > > I've thought about whether we should be hiring "subject matter experts"
> > > in
> > > > other areas, such as global health, but in my view the need is clearer
> > > here
> > > > than in any other area. One of the things I don't love about hiring an
> > > > expert in a given field, at this stage of our research, is that we could
> > > > quickly decide that we're just not interested enough in the field in
> > > > question ... but someone with the right kind of technical knowledge &
> > > > experience would be so far ahead of us in evaluating *any* area of
> > > biology
> > > > research that it seems like a good idea. (JTBC, I'm also actively
> > > thinking
> > > > about whether it would make sense to hire experts in other fields ...)
> > > >
> > > > · Talking to major funders and potentially co-funding with them is
> > > > probably essential. Important groups to talk to would include NIH (by far
> > > > the most important; we've already talked to them a bit), the colloquium
> > > of
> > > > groups like the American Cancer Society (it has a name; I forget the
> > > name),
> > > > FasterCures, Wellcome, and potentially some funders with
> > > > unorthodox/buzzed-about approaches: Myelin Repair Foundation, McKnight
> > > > Foundation, and maybe another private foundation or two (I emailed Dario
> > > > all my notes on major foundations that do biomedical research so he could
> > > > send thoughts on whether any look interesting aside from McKnight).
> > > >
> > > >
> > > > My next plan had been to talk to academics (Dario had good suggestions
> > > > about how to approach them), but with all the potential work to be done
> > > on
> > > > gaining basic context, I'm not sure that's the right next step. But it's
> > > > also a possible step.
> > > >
> > > >
> > > >
> > > > *2. Some more context on the history of our work on biomedical research
> > > and
> > >
> > > > why we think it's appropriate to investigate this field (this was a
> > > > response to a question following my original email, along the lines of
> > > "Why
> > > > are you looking into biomedical research now, given that's an area with a
> > > > lot of buzz and funding from wealthy donors, and how does this work
> > > relate
> > > > to the 'meta-research' work?")*
> > > >
> > > >
> > > > - We've always had "disease research" as a cause we wanted to
> > >
> > > > investigate. We've never had a good reason for not investigating it other
> > > > than that investigating it seems difficult. We've tried more than once to
> > > > investigate it, and it's ended up falling by the wayside because of how
> > > big
> > > > an undertaking it is.
> > > > - It's true that there is a huge amount of funding and buzz in this
> > >
> > > > area. But it also seems quite possible that there isn't nearly enough; in
> > > > fact this seems like a possible point of consensus between all the people
> > > > concerned about the "Valley of Death." This potential good accomplished
> > > via
> > > > biomedical research appears unlimited. What I consider to be the mark of
> > > a
> > > > bad sector is "tons of buzz/$ and mediocre returns," but the latter part
> > > > doesn't apply to biomed; in fact biomedical research is arguably one of
> > > > philanthropy's and even humanity's top success stories.
> > > > - There are also indications that despite all the buzz and funding,
> > >
> > > > there are still opportunities to do things differently and shake things
> > > up.
> > > > There is more than one case where an outsider (Milken, MRF) basically
> > > came
> > > > in and did things very differently and now even experts in the field seem
> > > > to credit them with positive change. There's also a good explanation for
> > > > why this might be the case: while there are a ton of people and dollars,
> > > > they largely seem to play by one self-reinforcing/network-effect-prone
> > > set
> > > > of rules, implying high returns to disrupting that equilibrium.
> > > > - So, we've always wanted to get into this area. There was a period
> > >
> > > > where I was presenting meta-research as our best entry point into this
> > > > field: my vision was that we would talk to academics about what
> > > systematic
> > > > failings there were and what funding opportunities these implied, and
> > > that
> > > > would be as good a way as any to get acclimated in biomed. But this
> > > period
> > > > wasn't particularly long - the May blog post on Labs priority causes
> > > lists
> > > > scientific research as a promising area distinct from meta-research.
> > > > - I set up the call with Dario without having a clear idea of whether I
> > >
> > > > wanted to approach biomed from a "meta-research" angle or another angle.
> > > > After the call and other investigations described here, I got a clearer
> > > > idea of what I think is the best path forward.
> > > >
> > > > Bottom line - I think it's important to build an understanding of
> > > > biomedical research, and that we should take the best path to doing so
> > > > whether or not that dovetails with the meta-research work (likely it will
> > > > dovetail some but not 100%).
> > >
> > > 
> > >
> >
> 
> 
> 
> 
> 

Hi Aubrey,

Thanks again for engaging so thoughtfully.

I agree that a new technology/treatment that could delay or reverse aging
(or aspects of it) would be enormously valuable. Regarding the rest of your
argument, this is a good example of the challenges I've been discussing in
understanding biomedical research.

You state that you have a high-expected-value plan that the academic world
can't recognize the value of because of shortcomings such as
"balkanisation" and risk aversion. I believe it may be true that the
academic world has such problems to a degree; however, I also believe that
there are a lot of extremely talented people in academia and that they
often (though not necessarily always) find ways to move forward on
promising work. Without more subject-matter expertise (or the advice of
someone with such expertise), I can't easily assess the technical merits of
your argument or potential counterarguments. Hopefully we'll have a better
system for doing so at some point in the future.

I'll be very interested to see Dario's thoughts on the matter if he
responds. I'd cite Dario as an example of an academic who ultimately wants
to do work of the greatest humanitarian value possible, regardless of
whether it is prestigious work. And as my summary of our conversation
shows, he acknowledges that the world of biomedical research may have
certain suboptimal incentives, but didn't seem to think that these issues
are leaving specific, visible outstanding research programs on the table
the way that your email implies.

Best,
Holden

On Sat, Oct 13, 2012 at 6:24 AM, Aubrey de Grey <aubrey@...> wrote:

> Hi Holden - many thanks.
>
> First: yes, there are really three somewhat separate questions for someone
> trying to evaluate whether to support SENS Foundation:
>
> 1) Is the medical control of aging a hugely valuable mission?
>
> 2) Assuming "yes" to (1), is it best achieved by basic research or
> translational research?
>
> 3) Assuming translational, is SENS Foundation the organisation that uses
> money most effectively in pursuit of that mission?
>
> I had rather expected that you would take some convincing on item (1), and
> much of what I wrote last time was focused on that. Since it isn't the
> focus of your question to me, I'm now going to assume until further notice
> that there is no dissent on that.
>
> So, to answer your question: actually you're not putting aside the
> basic-vs-translational question as much as you may think you are. The word
> "translational" is flavour of the month in government funding circles these
> days (not only in the USA), so it's not surprising that the NIA has a
> public statement of the kind you pointed to. However, notice that the link
> they give "for more information" is to a page listing ALL "Funding
> Opportunity Announcements". There is no page specifically for translational
> ones, and the reason there isn't is that the amount of work that the NIA
> actually funds that could really be called translational is tiny. In other
> words, the page you found is actually just blatant spin. The neuroscience
> slice you mention is an anomaly arising from the way NIA was founded (the
> natural place for that money is clearly NINDS): the fact that it's NIA
> money does not, in practice, translate into its being spent on work to
> prevent neurodegeneration by treating its cause (aging). Instead, just like
> NINDS money, it's spent on attacking neurodegeneration directly, as if such
> diseases could be eliminated from the body just like an infection: the same
> old mistake that afflicts, and dooms, the whole of geriatric medicine.
>
> So, the first answer to your question is that SENS Foundation really DOES
> focus on translational research, with an explicit goal of postponing
> age-related ill-health. But there's also another big difference: we can
> attack this problem relatively free of the other priorities that afflict
> mainstream funding (whether from NIH or from trasitional foundations). Most
> importantly, though we do and will continue to publish our interim results
> in the peer-reviewed literature, we are much less constrained by "publish
> or perish" tyranny than typical academics are. This allows us to proceed by
> constructing and implementing a rational "project plan" (namely SENS) to
> get to the intended goal (the defeat of aging), whereas what little
> translational work is funded by NIA or others is guided overwhelmingly by
> the imperative to get some kind of positive result as quickly as possible,
> even when it's understood that those results are not remotely likely to
> "scale", i.e. to translate into eventual medical treatments that
> significantly delay aging. A great example of this is the NIA's
> Interventions Testing Program (ITP) to test the mouse longevity effects of
> various small molecules. The ITP only exists at all (and in a far smaller
> form than originally intended) as a result of several years of persistence
> by the then head of the NIA's biology division (Huber Warner), and it
> focuses entirely on delivery of simple drugs starting rather early in life,
> with the result that no information emerges that's relevant to treating
> people who are already in middle age or older. (This is despite the fact
> that by far the most high-profile result that the ITP has delivered so far,
> the benefits of rapamycin, actually WAS a late-onset study: it wasn't meant
> to be, but technical issues delayed the experiment.) In a nutshell, there
> is a huge bias against high-risk high-gain work.
>
> The third thing that distinguishes SENS Foundation's approach is that we
> can transcend the "balkanisation" (silo mentality) that dominates
> mainstream academic funding. When one submits a grant application to NIA,
> it is evaluated by gerontologists, just as when one submits to NCI it is
> evaluated by oncologists, etc. What's wrong with this is that it biases the
> system immensely against cross-disciplinary proposals. SENS is a plan that
> brings together a large body of knowledge from gerontology but also a huge
> amount of expertise that was developed for other reasons entirely - to
> treat acute disease/injury, or in some cases for purposes that were not
> biomedical at all (notably environmental decontamination). It doesn't
> matter how robust the objective scientific and technological argument is
> for work of that sort: it will never compete (especially in today's very
> tight funding environment) with more single-topic proposals all of whose
> details can be understood by reviewers from a particular single field.
>
> The final thing to mention, and this actually also answers your question
> to Vipul about basic versus translational research, is that SENS is a plan
> that has stood the test of time. I've been propounding it since 2000, well
> before SENS Foundation existed, and it used to come in for a lot of
> criticism (initially more in the form of off-the-record ridicule, and
> latterly, at my behest, in print), but in every single case that criticism
> was found to stem from ignorance on the part of the detractor, either of
> what I proposed or of published experimental work on which the proposal was
> based. That's why I'm now regularly asked to organise entire sessions at
> mainstream gerontology conferences, whereas as little as five years ago I
> would never even be invited to speak. It's also why the Research Advisory
> Board of SENS Foundation consists of such prestigious scientists. This is a
> very strong argument, in my view, for believing that now is the time to
> sink a proper amount of money into translational gerontology (though
> certainly not to cease doin basic biogerontology too). It's well known that
> basic scientists are often not the most far-sighted when it comes to seeing
> how to apply their discoveries (attitudes in 1900 to the feasibility of
> powered flight being the canonical example). It is therefore a source of
> concern that almost all the experts who have the ear of funders in this
> field are basic scientists, whose instinct is to carry on finding things
> out and to deprioritise the tedious business of applying that knowledge.
> SENS has achieved a gratisfying level of legitimacy in gerontology, but it
> is still foreign to most card-carrying gerontologists, and as such it
> remains essentially unfundable via mainstream mechanisms. Hence the need to
> create a philanthropy-driven entity, SENS Foundation, to get this work done.
>
> Let me know if this helps, or if you have further questions.
>
> Cheers, Aubrey
>
> On 12 Oct 2012, at 15:28, Holden Karnofsky wrote:
>
> >
> > Hi Aubrey,
> >
> > Thanks for the thoughts.
> >
> > The NIH appears to have a division focused on research relevant to this
> topic: http://www.nia.nih.gov/research/dab . Its budget appears to be
> ~$175 million (per year). The National Institute on Aging, which houses
> this division, has a budget of about $1 billion per year, including a
> separate ~$400 million for neuroscience (which may also be relevant) as
> well as $115 million for intramural research. Figures are from
> http://www.nia.nih.gov/about/budget/2012/fiscal-year-2013-budget. The
> Institute states that its mandate includes translational research (
> http://www.nia.nih.gov/research/faq/does-nia-support-translational-research).
> How would you distinguish your work from this work?
> >
> > (For the moment I'm putting aside the question I raised in my previous
> response to Vipul on this topic, regarding whether it's best to approach
> biology funding from the perspective of "trying to treat/cure a particular
> condition" or "trying to understand  fundamental questions in biology whose
> applications are difficult to predict.")
> >
> > Best,
> > Holden
> >
> > On Fri, Oct 12, 2012 at 7:32 AM, aubrey.degrey <aubrey@...> wrote:
> >
> > Hi everyone,
> >
> > My attention was brought to this thread, by virtue of the fact that it
> was my work that gave rise to SENS Foundation, and I'm looking forward to
> getting more involved here; I've held the Effective Altruism movement in
> high regard for some time. However, given my newbie status here I want to
> start by apologising in advance for any oversight of previously-discussed
> issues etc. I'm naturally delighted both at Holden's post and at Vipul's
> reply (which I should stress that I did not plant! - I do not know Vipul at
> all, though I look forward to changing that). I would like to mention just
> a few key points for discussion:
> >
> > - Holden, I want to compliment you on your appreciation of how academia
> really works. Everything you say about that is spot on. The aversion to
> "high risk high gain" work that has arisen and become so endemic in the
> system is the most important point here, in terms of why parallel funding
> routes are needed.
> >
> > - I'm slightly confused that a lot of Holden's remarks are focused on
> the private sector (i.e. startups), since my understanding was that
> GiveWell is about philanthropy; but I realise that there is not all that
> clear a boundary between the two (and I note the mention of Breakout Labs,
> with which I have close links and which sits astride that divide more than
> arguably anyone). The "valley of death" in pre-competitive translational
> research is a rather different one than that encountered by startups, but
> the principle is the same, and research to postpone aging certainly
> encounteres it.
> >
> > - Something that I presume factors highly among GiveWell's criteria is
> the extent to which a cause may be undervalued by the bulk of major
> philanthropists, such that an infusion of additional funds would make more
> of a difference than in an area that is already being well funded. To me
> this seems to mirror the logic of focusing on the shortcomings (gaps) in
> NIH's funding (and that of traditional-model foundations). Holden notes
> that "Anyone we consider for funding ought to be able to explain why
> they're better at allocating the funds than the NIH" and I agree
> wholeheartedly, but my inference is that he thinks that some orgs may
> indeed be able to explain that. I certainly think that SENS Foundation can.
> >
> > - Coming to aging: research to postpone aging has the unique problem of
> quite indescribeable irrationality on the part of most of the general
> public, policy-makers and even biologists with regard to its desirability.
> Biogerontologists have been talking to brick walls for decades in their
> effort to get the rest of the world to appreciate that aging is what causes
> age-related ill-health, and thus that treatments for aging are merely
> preventative geriatrics. The concept persists, despite biogerontologists'
> best efforts, that aging is "natural" and should be left alone, whereas the
> diseases that it brings about are awful and should be fought. This is made
> even more bizarre by the fact that the status of age-related diseases as
> aspects of the later stages of aging absolutely, unequivocally implies that
> efforts to attack those diseases directly are doomed to fail. As such, this
> is a (unique? certainly very rare) case where a philanthropic contribution
> can make a particularly big difference simply because most philanthropists
> don't see the case for it. It underpins why having an interest in treating
> aging, as opposed to cancer, absolutely has a major impact on which
> projects one funds. It's also a case for (if I understand the term
> correctly) meta-research.
> >
> > - A lot of the chatter about treating aging revolves around longevity,
> but it shouldn't. I'm all in favour of longevity, don't get me wrong, but
> it's not what gets me up in the morning: what does is health. I want people
> to be truly youthful, however long ago they were born: simple as that. The
> benefits of longevity per se to humanity may also be substantial, in the
> form of greater wisdom etc, but that would necessarily come about only very
> gradually (we won't have any 1000-year-old for at least 900 years whatever
> happens!), so it doesn't figure strongly in my calculations.
> >
> > - When forced to acknowledge that the idea of aging being a
> high-priority target for medicine is an inescapeable consequence of things
> they already believe (notably that health is good and ageism is bad), many
> people retreat to the standpoint that it's never going to be possible so
> it's OK to be irrational about whether it's desirable. The feasibility of
> postponing age-related ill-health by X years with medicine available Y
> years from now is, of course, a matter of speculation on which experts
> disagree, just as with any other pioneering technology. I know that Holden
> and others have expressed caution (at best) concerning the accuracy of any
> kind of calculation of probabilities of particular outcomes in the distant
> (or even not-so-distant) future, and I share that view. However, an
> approach that may appeal more is to estimate how much humanitarian benefit
> a given amount of progress would deliver, and then to ask how unlikely that
> scenario needs to be to make it not worth pursuing. My claim is that the
> benefits of hastening the defeat of aging by even a few years (which is the
> minimum that I claim SENS Foundation is in a position to do, given adequate
> funding) would be so astronomical that the required chance of success to
> make such an effort worthwhile would be tiny - too tiny for it to be
> reasonable to argue that such funding would be inadvisable. But of course
> that is precisely what I would want GiveWell to opine on.
> >
> > - In the event that GiveWell (or anyone else) were to decide and declare
> that the defeat of aging is indeed a cause that philanthropists should
> support, there then arises the question of which organisation(s) should be
> supported in the best interests of that mission. We at SENS Foundation have
> worked diligently to rise as quickly as possible in the legitimacy stakes
> by all standard measures, but we are still young and there remains more to
> do. If I were to offer an argument to fund us rather than any other entity,
> it would largely come down to the fact that no other organisation has even
> a serious plan for defeating aging, let alone a track record of
> implementing such a plan's early stages.
> >
> > - A significant chunk of what we do is of a kind that I think comes
> under "meta-research". A prominent example is a project we're funding at
> Denver University to extend the well-respected forecasting system
> "International Futures" so that it can analyse scenarios incorporating
> dramatically postponed aging.
> >
> > I greatly welcome any feedback.
> >
> > Cheers, Aubrey
> >
> >
> >
> > --- In givewell@yahoogroups.com, Holden Karnofsky <holden0@...> wrote:
> > >
> > > Hi Vipul,
> > >
> > > Thanks for the thoughts. I had a followup conversation with Dario about
> > > this topic a few days ago. I think the question of "could one fund
> > > translational research to treat/prevent aging?" provides an interesting
> > > illustration of some of the tricky dynamics here for a funder:
> > >
> > > - It's possible that if there were a great deal more attention giving
> to
> >
> > > treating/preventing aging, we would have some promising treatments. So
> in a
> > > broad sense it's possible that aging is underinvested in.
> > > - A lot of the best basic biology research isn't clearly pointing
> toward
> >
> > > one treatment/condition or another; it's about understanding the
> > > fundamentals of how organisms operate. So having an interest in
> treating
> > > aging, as opposed to cancer, might not have a major impact on which
> > > projects one funds, if one's main goal is to fund outstanding basic
> biology
> > > research.
> > > - Perhaps because of the lack of emphasis on treating aging (or perhaps
> >
> > > because it's simply too difficult of a problem), there don't seem to be
> > > promising findings in the "Valley of Death" relevant to aging; the few
> > > promising leads have been explored.
> > > - So even if, in a broad sense, there is too little attention given to
> >
> > > this problem, knowing this doesn't necessarily yield a clear direction
> for
> > > a relatively small-scale funder of biomedical research.
> > >
> > > Best,
> > > Holden
> > >
> > > On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:
> > >
> > > > **
> >
> > > >
> > > >
> > > > Holden,
> > > >
> > > > First, I think that this is an excellent document. I checked for a
> > > > number of things that I had heard about (Breakout Labs, John
> > > > Ioannidis, Cochrane Collaboration) and they're all there in your
> > > > document.
> > > >
> > > > The one thing that's not explicitly mentioned: longevity and life
> > > > extension research. At least prima facie, this seems like something
> > > > that should be more important than individual disease research, and
> it
> > > > seems like a classic "Valley of Death" case (pun unintended, but
> > > > noted) -- T1 stage to use your terminology. I think the SENS website
> > > > http://www.sens.org would be a good starting point for one of the
> (to
> > > > me promising) approaches to life extension. I recall from past
> > > > conversations that you were aware of SENS, so this is not new to you,
> > > > but I think that longevity should be included as part of any
> > > > discussion of biomedical research and given separate consideration
> > > > given that it has a much lower status than research into specific
> > > > conditions such as cancer, dementia, etc. You may ultimately conclude
> > > > that not enough can be done in this area, but I think it should be
> > > > part of your preliminary stuff. [btw, the United States has a
> National
> > > > Institute of Aging, but it's much lower-status than most of the other
> > > > grantmakers mentioned here].
> > > >
> > > > Vipul
> > > >
> > > > * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed)
> wrote
> > > >
> > > > > Note to the research list: we're now considering reopening our
> > > > > investigation of the world of biomedical research. We've started
> and
> > > > > stopped a couple of times in this area before; this time I decided
> to
> > > > start
> > > > > with a conversation with Dario Amodei, a longtime GiveWell
> follower and
> > > > > personal friend who is currently a biology postdoc at Stanford. My
> goal
> > > > > with the conversation was just to get some basic context and start
> > > > putting
> > > > > together a framework for thinking about the issue, not to use him
> as an
> > > > > authoritative source, and the notes below should be read in that
> spirit.
> > > > >
> > > > >
> > > > > This email has two sections:
> > > > >
> > > > >
> > > > > 1. Notes that I emailed out internally after my conversation with
> Dario,
> > > > > slightly edited
> > > > >
> > > > > 2. Some more context on the history of our work on biomedical
> research
> > > > and
> > > > > why we think it's appropriate to investigate this field (this was a
> > > > > response to a question following my original email, along the
> lines of
> > > > "Why
> > > > > are you looking into biomedical research now, given that's an area
> with a
> > > > > lot of buzz and funding from wealthy donors, and how does this work
> > > > relate
> > > > > to the 'meta-research' work?")
> > > > >
> > > > >
> > > > >
> > > > > *--*
> > > > >
> > > > > *
> > > > > *
> > > > >
> > > > > *1. Notes that I emailed out internally after my conversation with
> Dario,
> > > > > slightly edited*
> > > > >
> > > > > *
> > > >
> > > > > *
> > > > >
> > > > > I've done some preliminary work trying to figure out what it would
> look
> > > > > like to explore biomedical sciences as funding area. This mostly
> > > > consisted
> > > > > of a 3-hour conversation with Dario (recording is available),
> reading two
> > > > > papers he sent and a few I found while Googling, and prior
> knowledge. I'm
> > > > > including Dario in all emails related to this stuff, as an informal
> > > > advisor.
> > > > >
> > > > > * *
> > > > >
> > > > > *My picture of "what the biomedical research world roughly looks
> like"
> > > > *(this
> > > >
> > > > > is mostly from talking with Dario + prior knowledge)
> > > > >
> > > > > · *Academic biology* studies how organisms work and develops tools
> to
> > > >
> > > > > observe and manipulate the building blocks of organisms.
> > > > >
> > > > > o The vast bulk of the funding - and the most prestigious funding
> - comes
> > > > > from the NIH.
> > > > >
> > > > > o There is also funding from what I've heard called "foundations" -
> > > > groups
> > > > > like the American Cancer Society and American Heart Association -
> which
> > > > > function very similarly to the NIH, in that they tend to hire
> people with
> > > > > strong academic credentials and those people judge the merits of
> grant
> > > > > proposals.
> > > > >
> > > > > o Both the NIH and "foundations" tend to be formally partitioned by
> > > > > disease, but much of the work done by academic biologists is
> potentially
> > > > > relevant to many diseases. A researcher seeking NIH funding may
> apply to
> > > > > several different NIH "study sections," though only one at a time
> (a list
> > > > > of "study sections" is at
> > > > >
> http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
> > > > > whether s/he gets funding is going to depend more on the academic
> merits
> > > > of
> > > > > the researcher & work than anything else.
> > > > >
> > > > > o There are basic definitions of "academic merit " that generally
> shape
> > > > > the whole ecosystem: the people handing out money are selected by
> these
> > > > > criteria and use these criteria, the people doing the research
> know that
> > > > > these criteria shape their career prospects, etc.
> > > > >
> > > > > · *Private startups* investigate promising ideas for new
> > > >
> > > > > treatments/diagnostics/devices. They may often take the form of a
> biology
> > > > > professor spinning off a biotech startup (run by former postdocs)
> that
> > > > > raises venture capital, based on the research the professor did.
> They
> > > > take
> > > > > basic knowledge about how the body works (for example, protein X is
> > > > crucial
> > > > > for medical condition Y) and do the necessary testing to find a
> promising
> > > > > treatment/diagnostic/device (for example, testing a lot of
> compounds on
> > > > > animals until they find one that affects protein X).
> > > > >
> > > > > · *Big pharma/biotech companies *are best positioned to deal with
> the
> > > >
> > > > > extremely expensive process of conducting clinical trials and
> getting FDA
> > > > > approval. Acquisition by one of these is the most common form of
> exit for
> > > > > startups.
> > > > >
> > > > > · *Academic medicine, epidemiology and other fields* also do work
> > > >
> > > > > relevant to medicine, including studying questions whose main
> relevance
> > > > is
> > > > > to medical practice and public health programs: how effective is
> > > > > treatment/practice X in situation Y, how cost-effective is it, etc.
> > > > > Sometimes they will hit on commercializable insights (for example,
> a new
> > > > > kind of device) as well.
> > > > >
> > > > > · *Translational research* is a broad term referring to a bridge
> between
> > > >
> > > > > academic research and treatments/practices. It can include (
> > > > >
> > > >
> http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> > > > > ):
> > > > >
> > > > > o T1 - going from basic science (usually academic biology, I
> presume) to
> > > > a
> > > > > new treatment/practice. This includes research that helps go from
> an
> > > > > academic biology insight to a private startup.
> > > > >
> > > > > o T2 - I think this is basically what Cochrane does - going from
> academic
> > > > > medicine/epidemiology (a bunch of studies on what
> treatments/practices
> > > > are
> > > > > effective) to the development of guidelines that actually affect
> > > > practice.
> > > > >
> > > > > o T3, T4 - research on how to actually change practice (as opposed
> to
> > > > > setting the guidelines that are a "target" for practice) and get
> better
> > > > > real-world results.
> > > > >
> > > > > *
> > > > > *
> > > > >
> > > > > *Potential "big opportunity to do good" #1: translational research
> and
> > > > the
> > > >
> > > > > "Valley of Death" (this comes from prior knowledge, googling
> "Valley of
> > > > > Death" and reading a bunch of the stuff I found, and checking out
> the
> > > > > FasterCures website again)*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > The "Valley of Death" seems to mostly refer to a shortage of T1.
> The idea
> > > > > is that there are a lot of cases where there's an academic insight
> that's
> > > > > potentially valuable in coming up with a new treatment, but to get
> it to
> > > > > the point where it's attractive from a for-profit perspective, you
> need
> > > > to
> > > > > do a lot of stuff that academics don't have a reason to do. "For
> example,
> > > > > an upstream finding that a given protein is differentially
> expressed in
> > > > > individuals with a particular disease may suggest that the protein
> merits
> > > > > further investigation. However, much more work (especially
> medicinal
> > > > > chemistry) is necessary to determine how good a target the protein
> really
> > > > > is and whether a marketable drug candidate that affects the
> activity of
> > > > the
> > > > > protein is likely to be developed." (
> > > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> > > > >
> > > > >
> > > > >
> > > > > There are claims that this sort of work is massively underfunded
> (by the
> > > > > people we've spoken to who talked about the "Valley of Death";
> also in
> > > > >
> > > >
> http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
> > > > (1).pdf).
> > > > > However, funding isn't the only issue. The other issue is that
> > > > > "pharmaceutical firms that hold libraries of potentially useful
> small
> > > > > molecules as trade secrets, making them largely off limits to ...
> > > > academic
> > > > > scientists" (
> > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> > > > > In other words, there is some room for new models of collaboration
> that
> > > > > lead to better communication and information sharing between
> academia and
> > > > > industry (or between industry and industry).
> > > > >
> > > > >
> > > > > Both the Myelin Repair Foundation and Michael Milken's work on
> prostate
> > > > > cancer have been pointed to as examples of innovative
> collaborations that
> > > > > deal with some of the information sharing problems. Milken's model:
> > > > > "drastically cutting the wait time for grant money, to flood the
> field
> > > > with
> > > > > fast cash, to fund therapy-driven ideas rather than basic science,
> to
> > > > hold
> > > > > researchers he funds accountable for results, and to demand
> collaboration
> > > > > across disciplines and among institutions, private industry, and
> > > > academia."
> > > > > (
> > > > >
> > > >
> http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
> > > > )
> > > > > Myelin Repair Foundation sounds broadly similar (
> > > > >
> > > >
> http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
> > > > ).
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > More broadly:
> > > > >
> > > > > · FasterCures (also a Michael Milken production) looks like it's
> focused
> > > > > on the broad mission of "more research with a shorter timeline to
> > > > > treatments," with a heavy sub-focus on the Valley of Death. In
> addition
> > > > to
> > > > > its conference and philanthropic advisory service, it advocates
> for FDA
> > > > > improvements (presumably to speed the approval process), advocates
> for
> > > > the
> > > > > NIH to put more funding into translational research (there have
> > > > definitely
> > > > > been a lot of new initiatives at the NIH focused on this stuff in
> the
> > > > past
> > > > > few ~decade), promotes "innovative financing mechanisms" for
> bridging the
> > > > > Valley of Death (these include Peter Thiel's Breakout Labs; I
> think they
> > > > > occupy a conceptually similar space to "social enterprise
> investment"
> > > > > though they tend to be structured more as grants and less as
> > > > > double-bottom-line investments), and works on getting patient data
> opened
> > > > > to researchers. The only program of theirs I haven't mentioned is
> TRAIN;
> > > > I
> > > > > can't (easily) figure out what this is.
> > > > >
> > > > > · John Ioannidis stated to us that all translational research is
> > > > > underfunded, not just T1. (The context we talked to him in, of
> course,
> > > > was
> > > > > T2.)
> > > > >
> > > > >
> > > > > These issues seem to have quite a bit of buzz. There are some
> really
> > > > stark
> > > > > #'s out there: even as R&D investment has gone way up over the
> past 50
> > > > > years, the # of new drugs has stayed roughly constant at around 20
> a
> > > > year.
> > > > >
> > > > >
> > > > > Dario sent a really interesting paper on this topic. It argues:
> (a) the #
> > > > > of NMEs (new medical entities, i.e., drugs) per year has mostly
> remained
> > > > > constant-with-noise (no trend); (b) all of the big companies seem
> to have
> > > > > produced NMEs at a very steady pace, even as they've changed size,
> though
> > > > > different companies do have different rates of NME creation; (c)
> when it
> > > > > comes to mergers & acquisitions, "1+1=1": companies that acquire
> other
> > > > > companies just keep up the same NME pace; (d) over time, the # of
> large
> > > > > companies has shrunk (due to mergers) and the # of small companies
> has
> > > > > risen, and the share of NMEs attributed to small companies has
> gone from
> > > > > ~30% to ~70%; (e) the cost per NME has gone up over time and is
> now in
> > > > the
> > > > > neighborhood of $1 billion; (f) the % of NMEs that are
> "blockbusters"
> > > > (huge
> > > > > profits) has been steady around 20%, despite intensifying efforts
> on the
> > > > > part of pharma to fund only potential blockbusters.
> > > > >
> > > > >
> > > > >
> > > > > There are also claims - such as by Derek Lowe, a blogger Dario
> pointed me
> > > > > to - that the "Valley of Death" is an overrated concept: there is
> a high
> > > > > ratio of academic discoveries to useful drugs, but this is just
> because a
> > > > > lot of stuff turns out not to work, not because we have a shortage
> of
> > > > > doable translational research. The paper above sort of takes this
> view,
> > > > > essentially arguing that nothing seems to raise NME production;
> we'll
> > > > need
> > > > > something really radical to make any difference.
> > > > >
> > > > >
> > > > >
> > > > > The case studies in the FasterCures paper on this are interesting.
> They
> > > > > imply that there were some pretty low-hanging fruit in the T1
> domain. It
> > > > > might be interesting to talk to Fastercures and see whether they
> can help
> > > > > identify "Valley of Death" opportunities that are slam-dunkish.
> > > > >
> > > > >
> > > > > *Potential "big opportunity to do good" #2: inefficiencies in
> academic
> > > > > biology*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > All of the above seems pretty distinct from the question of how to
> make
> > > > > academic biology better at doing its job of understanding the
> body. This
> > > > is
> > > > > the question that Dario and I focused on.
> > > > >
> > > > >
> > > > >
> > > > > Dario painted a picture in which most of academia plays by the
> same set
> > > > of
> > > > > rules, making it very hard for people to do things that break
> those rules
> > > > > (for example, academics are expected to publish a lot; it's risky
> to work
> > > > > on a "blue sky" or highly ambitious project). Thus, for anyone who
> wants
> > > > a
> > > > > career in academia, a couple of years working on a moonshot
> project are
> > > > > risky; in addition, being on bad terms with a small number of
> people can
> > > > > damage a career (since there is often a small set of people that
> makes a
> > > > > large proportion of the career-impacting decisions for a given
> area, and
> > > > it
> > > > > can be hard to escape this set of people without changing research
> > > > > interests significantly). The bad news is that this isn't
> particularly
> > > > easy
> > > > > to fix: you can offer funding for blue-sky projects, but (a) a lot
> of
> > > > > academics basically train themselves to play by the rules, and
> won't
> > > > > necessarily have thought about "what should be studied if these
> rules
> > > > > didn't apply," and more importantly, (b) there are many incentives
> > > > pushing
> > > > > academics towards playing by certain rules; funding is only one of
> those
> > > > > mechanisms (there's also tenure, peer review, etc) so changing
> that one
> > > > > incentive won't always change behavior. Dario says that he might
> hesitate
> > > > > to work on a particular blue-sky project that he thinks is
> interesting,
> > > > > even if he got funding for it, for these reasons.
> > > > >
> > > > >
> > > > > Despite this, there are some funders who push the boundaries.
> There are
> > > > > medical centers that don't require teaching and do more ambitious
> work.
> > > > > There's the McKnight foundation, which funded some of the
> pioneering work
> > > > > on optical control of neurons for which funding might have been
> difficult
> > > > > to obtain by traditional routes. There's the HHMI Janelia Farms
> campus,
> > > > > which Dario thinks is the most promising thing out there in terms
> of a
> > > > > model - academics who go there get guaranteed (and generous)
> funding for
> > > > 6
> > > > > years, which frees them up to take much bigger risks. (I saw a
> study
> > > > > claiming that the work coming out of Janelia Farms has a much more
> skewed
> > > > > citation count distribution, implying bigger risks - i.e., fewer
> papers
> > > > > with decent #'s of citations but more "blockbuster" papers with
> lots of
> > > > > citations). But Janelia Farms is only in one sub-field; comparable
> > > > > institutions don't exist for other fields (as far as Dario knows)..
> > > > >
> > > > >
> > > > >
> > > > > Dario's gut is that one of the best things a funder could do would
> be to
> > > > > work toward creating a large institute that largely "plays by its
> own
> > > > > rules," encouraging more ambitious work and providing enough
> security and
> > > > > sheer volume of dollars/researchers as to establish a sort of
> "parallel
> > > > > system" to academia - thus becoming a place that could provide
> viable and
> > > > > reliable career options for people interested in playing by
> different
> > > > > rules. I'm also interested in the idea of trying to advocate for
> changes
> > > > in
> > > > > the rules, as we've discussed in the context of meta-research.
> > > > >
> > > > >
> > > > >
> > > > > There's also the other stuff like data/code sharing.
> > > > >
> > > > >
> > > > >
> > > > > *Another option: give $ to the NIH*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > The NIH is by far the largest, most prestigious, most respected
> funder in
> > > > > academic biology, and most of academia runs on criteria that
> mirror the
> > > > > NIH's. I would guess that an unrestricted check to the NIH would
> get
> > > > > allocated in a pretty sector-agnostic way. It seems like this is a
> giving
> > > > > option that is pretty nontrivial to beat. Anyone we consider for
> funding
> > > > > ought to be able to explain why they're better at allocating the
> funds
> > > > than
> > > > > the NIH. We haven't talked to the NIH about whether it would
> accept these
> > > > > sorts of donations.
> > > > >
> > > > >
> > > > > With the context I now have, FasterCures strikes me as pretty
> > > > > promising/interesting too, though their "room for more funding"
> situation
> > > > > isn't clear (w/o talking to them). But they also seem
> sector-agnostic
> > > > > within biomedical research, while pushing a very specific theory of
> > > > change
> > > > > that may or may not be valid.
> > > > >
> > > > >
> > > > >
> > > > > *Next steps*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > I think understanding this world would be a major undertaking. I
> think
> > > > we'd
> > > > > have to be ready to put in a lot of work and to be open to ways of
> > > > bridging
> > > > > "funder-expert gap" that we haven't tried before. For example:
> > > > >
> > > > > · Dario suggested that I read a freshman biology textbook. He said
> he
> > > > > realizes that the time cost could be very large - something like
> 100
> > > > hours
> > > > > - but that without doing so, I'm going to be lacking too much
> context on
> > > > > why biomedical research works the way it does. He thinks that
> reading a
> > > > > basic textbook would get me to the point of strongly diminishing
> marginal
> > > > > returns.
> > > > >
> > > > > · Also in the category of "personally picking up rudimentary
> subject
> > > > > matter knowledge," I thought it might be worth trying to follow the
> > > > > development of a particular drug from start to finish - Gleevec
> (the
> > > > > "miracle drug") would be a good candidate. The goal would be to
> > > > understand
> > > > > each stage of insight leading to new investigation, and where the
> funding
> > > > > came from at each stage.
> > > > >
> > > > > · I think hiring Dario or someone like him would make a lot of
> sense.
> > > > > I've thought about whether we should be hiring "subject matter
> experts"
> > > > in
> > > > > other areas, such as global health, but in my view the need is
> clearer
> > > > here
> > > > > than in any other area. One of the things I don't love about
> hiring an
> > > > > expert in a given field, at this stage of our research, is that we
> could
> > > > > quickly decide that we're just not interested enough in the field
> in
> > > > > question ... but someone with the right kind of technical
> knowledge &
> > > > > experience would be so far ahead of us in evaluating *any* area of
> > > > biology
> > > > > research that it seems like a good idea. (JTBC, I'm also actively
> > > > thinking
> > > > > about whether it would make sense to hire experts in other fields
> ...)
> > > > >
> > > > > · Talking to major funders and potentially co-funding with them is
> > > > > probably essential. Important groups to talk to would include NIH
> (by far
> > > > > the most important; we've already talked to them a bit), the
> colloquium
> > > > of
> > > > > groups like the American Cancer Society (it has a name; I forget
> the
> > > > name),
> > > > > FasterCures, Wellcome, and potentially some funders with
> > > > > unorthodox/buzzed-about approaches: Myelin Repair Foundation,
> McKnight
> > > > > Foundation, and maybe another private foundation or two (I emailed
> Dario
> > > > > all my notes on major foundations that do biomedical research so
> he could
> > > > > send thoughts on whether any look interesting aside from McKnight).
> > > > >
> > > > >
> > > > > My next plan had been to talk to academics (Dario had good
> suggestions
> > > > > about how to approach them), but with all the potential work to be
> done
> > > > on
> > > > > gaining basic context, I'm not sure that's the right next step.
> But it's
> > > > > also a possible step.
> > > > >
> > > > >
> > > > >
> > > > > *2. Some more context on the history of our work on biomedical
> research
> > > > and
> > > >
> > > > > why we think it's appropriate to investigate this field (this was a
> > > > > response to a question following my original email, along the
> lines of
> > > > "Why
> > > > > are you looking into biomedical research now, given that's an area
> with a
> > > > > lot of buzz and funding from wealthy donors, and how does this work
> > > > relate
> > > > > to the 'meta-research' work?")*
> > > > >
> > > > >
> > > > > - We've always had "disease research" as a cause we wanted to
> > > >
> > > > > investigate. We've never had a good reason for not investigating
> it other
> > > > > than that investigating it seems difficult. We've tried more than
> once to
> > > > > investigate it, and it's ended up falling by the wayside because
> of how
> > > > big
> > > > > an undertaking it is.
> > > > > - It's true that there is a huge amount of funding and buzz in this
> > > >
> > > > > area. But it also seems quite possible that there isn't nearly
> enough; in
> > > > > fact this seems like a possible point of consensus between all the
> people
> > > > > concerned about the "Valley of Death." This potential good
> accomplished
> > > > via
> > > > > biomedical research appears unlimited. What I consider to be the
> mark of
> > > > a
> > > > > bad sector is "tons of buzz/$ and mediocre returns," but the
> latter part
> > > > > doesn't apply to biomed; in fact biomedical research is arguably
> one of
> > > > > philanthropy's and even humanity's top success stories.
> > > > > - There are also indications that despite all the buzz and funding,
> > > >
> > > > > there are still opportunities to do things differently and shake
> things
> > > > up.
> > > > > There is more than one case where an outsider (Milken, MRF)
> basically
> > > > came
> > > > > in and did things very differently and now even experts in the
> field seem
> > > > > to credit them with positive change. There's also a good
> explanation for
> > > > > why this might be the case: while there are a ton of people and
> dollars,
> > > > > they largely seem to play by one
> self-reinforcing/network-effect-prone
> > > > set
> > > > > of rules, implying high returns to disrupting that equilibrium.
> > > > > - So, we've always wanted to get into this area. There was a period
> > > >
> > > > > where I was presenting meta-research as our best entry point into
> this
> > > > > field: my vision was that we would talk to academics about what
> > > > systematic
> > > > > failings there were and what funding opportunities these implied,
> and
> > > > that
> > > > > would be as good a way as any to get acclimated in biomed. But this
> > > > period
> > > > > wasn't particularly long - the May blog post on Labs priority
> causes
> > > > lists
> > > > > scientific research as a promising area distinct from
> meta-research.
> > > > > - I set up the call with Dario without having a clear idea of
> whether I
> > > >
> > > > > wanted to approach biomed from a "meta-research" angle or another
> angle.
> > > > > After the call and other investigations described here, I got a
> clearer
> > > > > idea of what I think is the best path forward.
> > > > >
> > > > > Bottom line - I think it's important to build an understanding of
> > > > > biomedical research, and that we should take the best path to
> doing so
> > > > > whether or not that dovetails with the meta-research work (likely
> it will
> > > > > dovetail some but not 100%).
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
> > 
>
>

Excellent. I too am keen to see Dario's comments. Dario also has the advantage of being based just a few miles from SENS Foundation's research centre, so we can definitely get together f2f soon if he wants.

Cheers, Aubrey

On 13 Oct 2012, at 17:50, Holden Karnofsky wrote:

> Hi Aubrey,
> 
> Thanks again for engaging so thoughtfully.
> 
> I agree that a new technology/treatment that could delay or reverse aging (or aspects of it) would be enormously valuable. Regarding the rest of your argument, this is a good example of the challenges I've been discussing in understanding biomedical research.
> 
> You state that you have a high-expected-value plan that the academic world can't recognize the value of because of shortcomings such as "balkanisation" and risk aversion. I believe it may be true that the academic world has such problems to a degree; however, I also believe that there are a lot of extremely talented people in academia and that they often (though not necessarily always) find ways to move forward on promising work. Without more subject-matter expertise (or the advice of someone with such expertise), I can't easily assess the technical merits of your argument or potential counterarguments. Hopefully we'll have a better system for doing so at some point in the future.
> 
> I'll be very interested to see Dario's thoughts on the matter if he responds. I'd cite Dario as an example of an academic who ultimately wants to do work of the greatest humanitarian value possible, regardless of whether it is prestigious work. And as my summary of our conversation shows, he acknowledges that the world of biomedical research may have certain suboptimal incentives, but didn't seem to think that these issues are leaving specific, visible outstanding research programs on the table the way that your email implies.
> 
> Best,
> Holden
> 
> On Sat, Oct 13, 2012 at 6:24 AM, Aubrey de Grey <aubrey@...> wrote:
> Hi Holden - many thanks.
> 
> First: yes, there are really three somewhat separate questions for someone trying to evaluate whether to support SENS Foundation:
> 
> 1) Is the medical control of aging a hugely valuable mission?
> 
> 2) Assuming "yes" to (1), is it best achieved by basic research or translational research?
> 
> 3) Assuming translational, is SENS Foundation the organisation that uses money most effectively in pursuit of that mission?
> 
> I had rather expected that you would take some convincing on item (1), and much of what I wrote last time was focused on that. Since it isn't the focus of your question to me, I'm now going to assume until further notice that there is no dissent on that.
> 
> So, to answer your question: actually you're not putting aside the basic-vs-translational question as much as you may think you are. The word "translational" is flavour of the month in government funding circles these days (not only in the USA), so it's not surprising that the NIA has a public statement of the kind you pointed to. However, notice that the link they give "for more information" is to a page listing ALL "Funding Opportunity Announcements". There is no page specifically for translational ones, and the reason there isn't is that the amount of work that the NIA actually funds that could really be called translational is tiny. In other words, the page you found is actually just blatant spin. The neuroscience slice you mention is an anomaly arising from the way NIA was founded (the natural place for that money is clearly NINDS): the fact that it's NIA money does not, in practice, translate into its being spent on work to prevent neurodegeneration by treating its cause (aging). Instead, just like NINDS money, it's spent on attacking neurodegeneration directly, as if such diseases could be eliminated from the body just like an infection: the same old mistake that afflicts, and dooms, the whole of geriatric medicine.
> 
> So, the first answer to your question is that SENS Foundation really DOES focus on translational research, with an explicit goal of postponing age-related ill-health. But there's also another big difference: we can attack this problem relatively free of the other priorities that afflict mainstream funding (whether from NIH or from trasitional foundations). Most importantly, though we do and will continue to publish our interim results in the peer-reviewed literature, we are much less constrained by "publish or perish" tyranny than typical academics are. This allows us to proceed by constructing and implementing a rational "project plan" (namely SENS) to get to the intended goal (the defeat of aging), whereas what little translational work is funded by NIA or others is guided overwhelmingly by the imperative to get some kind of positive result as quickly as possible, even when it's understood that those results are not remotely likely to "scale", i.e. to translate into eventual medical treatments that significantly delay aging. A great example of this is the NIA's Interventions Testing Program (ITP) to test the mouse longevity effects of various small molecules. The ITP only exists at all (and in a far smaller form than originally intended) as a result of several years of persistence by the then head of the NIA's biology division (Huber Warner), and it focuses entirely on delivery of simple drugs starting rather early in life, with the result that no information emerges that's relevant to treating people who are already in middle age or older. (This is despite the fact that by far the most high-profile result that the ITP has delivered so far, the benefits of rapamycin, actually WAS a late-onset study: it wasn't meant to be, but technical issues delayed the experiment.) In a nutshell, there is a huge bias against high-risk high-gain work.
> 
> The third thing that distinguishes SENS Foundation's approach is that we can transcend the "balkanisation" (silo mentality) that dominates mainstream academic funding. When one submits a grant application to NIA, it is evaluated by gerontologists, just as when one submits to NCI it is evaluated by oncologists, etc. What's wrong with this is that it biases the system immensely against cross-disciplinary proposals. SENS is a plan that brings together a large body of knowledge from gerontology but also a huge amount of expertise that was developed for other reasons entirely - to treat acute disease/injury, or in some cases for purposes that were not biomedical at all (notably environmental decontamination). It doesn't matter how robust the objective scientific and technological argument is for work of that sort: it will never compete (especially in today's very tight funding environment) with more single-topic proposals all of whose details can be understood by reviewers from a particular single field.
> 
> The final thing to mention, and this actually also answers your question to Vipul about basic versus translational research, is that SENS is a plan that has stood the test of time. I've been propounding it since 2000, well before SENS Foundation existed, and it used to come in for a lot of criticism (initially more in the form of off-the-record ridicule, and latterly, at my behest, in print), but in every single case that criticism was found to stem from ignorance on the part of the detractor, either of what I proposed or of published experimental work on which the proposal was based. That's why I'm now regularly asked to organise entire sessions at mainstream gerontology conferences, whereas as little as five years ago I would never even be invited to speak. It's also why the Research Advisory Board of SENS Foundation consists of such prestigious scientists. This is a very strong argument, in my view, for believing that now is the time to sink a proper amount of money into translational gerontology (though certainly not to cease doin basic biogerontology too). It's well known that basic scientists are often not the most far-sighted when it comes to seeing how to apply their discoveries (attitudes in 1900 to the feasibility of powered flight being the canonical example). It is therefore a source of concern that almost all the experts who have the ear of funders in this field are basic scientists, whose instinct is to carry on finding things out and to deprioritise the tedious business of applying that knowledge. SENS has achieved a gratisfying level of legitimacy in gerontology, but it is still foreign to most card-carrying gerontologists, and as such it remains essentially unfundable via mainstream mechanisms. Hence the need to create a philanthropy-driven entity, SENS Foundation, to get this work done.
> 
> Let me know if this helps, or if you have further questions.
> 
> Cheers, Aubrey
> 
> On 12 Oct 2012, at 15:28, Holden Karnofsky wrote:
> 
> >
> > Hi Aubrey,
> >
> > Thanks for the thoughts.
> >
> > The NIH appears to have a division focused on research relevant to this topic: http://www.nia.nih.gov/research/dab . Its budget appears to be ~$175 million (per year). The National Institute on Aging, which houses this division, has a budget of about $1 billion per year, including a separate ~$400 million for neuroscience (which may also be relevant) as well as $115 million for intramural research. Figures are from http://www.nia.nih.gov/about/budget/2012/fiscal-year-2013-budget. The Institute states that its mandate includes translational research (http://www.nia.nih.gov/research/faq/does-nia-support-translational-research). How would you distinguish your work from this work?
> >
> > (For the moment I'm putting aside the question I raised in my previous response to Vipul on this topic, regarding whether it's best to approach biology funding from the perspective of "trying to treat/cure a particular condition" or "trying to understand  fundamental questions in biology whose applications are difficult to predict.")
> >
> > Best,
> > Holden
> >
> > On Fri, Oct 12, 2012 at 7:32 AM, aubrey.degrey <aubrey@...> wrote:
> >
> > Hi everyone,
> >
> > My attention was brought to this thread, by virtue of the fact that it was my work that gave rise to SENS Foundation, and I'm looking forward to getting more involved here; I've held the Effective Altruism movement in high regard for some time. However, given my newbie status here I want to start by apologising in advance for any oversight of previously-discussed issues etc. I'm naturally delighted both at Holden's post and at Vipul's reply (which I should stress that I did not plant! - I do not know Vipul at all, though I look forward to changing that). I would like to mention just a few key points for discussion:
> >
> > - Holden, I want to compliment you on your appreciation of how academia really works. Everything you say about that is spot on. The aversion to "high risk high gain" work that has arisen and become so endemic in the system is the most important point here, in terms of why parallel funding routes are needed.
> >
> > - I'm slightly confused that a lot of Holden's remarks are focused on the private sector (i.e. startups), since my understanding was that GiveWell is about philanthropy; but I realise that there is not all that clear a boundary between the two (and I note the mention of Breakout Labs, with which I have close links and which sits astride that divide more than arguably anyone). The "valley of death" in pre-competitive translational research is a rather different one than that encountered by startups, but the principle is the same, and research to postpone aging certainly encounteres it.
> >
> > - Something that I presume factors highly among GiveWell's criteria is the extent to which a cause may be undervalued by the bulk of major philanthropists, such that an infusion of additional funds would make more of a difference than in an area that is already being well funded. To me this seems to mirror the logic of focusing on the shortcomings (gaps) in NIH's funding (and that of traditional-model foundations). Holden notes that "Anyone we consider for funding ought to be able to explain why they're better at allocating the funds than the NIH" and I agree wholeheartedly, but my inference is that he thinks that some orgs may indeed be able to explain that. I certainly think that SENS Foundation can.
> >
> > - Coming to aging: research to postpone aging has the unique problem of quite indescribeable irrationality on the part of most of the general public, policy-makers and even biologists with regard to its desirability. Biogerontologists have been talking to brick walls for decades in their effort to get the rest of the world to appreciate that aging is what causes age-related ill-health, and thus that treatments for aging are merely preventative geriatrics. The concept persists, despite biogerontologists' best efforts, that aging is "natural" and should be left alone, whereas the diseases that it brings about are awful and should be fought. This is made even more bizarre by the fact that the status of age-related diseases as aspects of the later stages of aging absolutely, unequivocally implies that efforts to attack those diseases directly are doomed to fail. As such, this is a (unique? certainly very rare) case where a philanthropic contribution can make a particularly big difference simply because most philanthropists don't see the case for it. It underpins why having an interest in treating aging, as opposed to cancer, absolutely has a major impact on which projects one funds. It's also a case for (if I understand the term correctly) meta-research.
> >
> > - A lot of the chatter about treating aging revolves around longevity, but it shouldn't. I'm all in favour of longevity, don't get me wrong, but it's not what gets me up in the morning: what does is health. I want people to be truly youthful, however long ago they were born: simple as that. The benefits of longevity per se to humanity may also be substantial, in the form of greater wisdom etc, but that would necessarily come about only very gradually (we won't have any 1000-year-old for at least 900 years whatever happens!), so it doesn't figure strongly in my calculations.
> >
> > - When forced to acknowledge that the idea of aging being a high-priority target for medicine is an inescapeable consequence of things they already believe (notably that health is good and ageism is bad), many people retreat to the standpoint that it's never going to be possible so it's OK to be irrational about whether it's desirable. The feasibility of postponing age-related ill-health by X years with medicine available Y years from now is, of course, a matter of speculation on which experts disagree, just as with any other pioneering technology. I know that Holden and others have expressed caution (at best) concerning the accuracy of any kind of calculation of probabilities of particular outcomes in the distant (or even not-so-distant) future, and I share that view. However, an approach that may appeal more is to estimate how much humanitarian benefit a given amount of progress would deliver, and then to ask how unlikely that scenario needs to be to make it not worth pursuing. My claim is that the benefits of hastening the defeat of aging by even a few years (which is the minimum that I claim SENS Foundation is in a position to do, given adequate funding) would be so astronomical that the required chance of success to make such an effort worthwhile would be tiny - too tiny for it to be reasonable to argue that such funding would be inadvisable. But of course that is precisely what I would want GiveWell to opine on.
> >
> > - In the event that GiveWell (or anyone else) were to decide and declare that the defeat of aging is indeed a cause that philanthropists should support, there then arises the question of which organisation(s) should be supported in the best interests of that mission. We at SENS Foundation have worked diligently to rise as quickly as possible in the legitimacy stakes by all standard measures, but we are still young and there remains more to do. If I were to offer an argument to fund us rather than any other entity, it would largely come down to the fact that no other organisation has even a serious plan for defeating aging, let alone a track record of implementing such a plan's early stages.
> >
> > - A significant chunk of what we do is of a kind that I think comes under "meta-research". A prominent example is a project we're funding at Denver University to extend the well-respected forecasting system "International Futures" so that it can analyse scenarios incorporating dramatically postponed aging.
> >
> > I greatly welcome any feedback.
> >
> > Cheers, Aubrey
> >
> >
> >
> > --- In givewell@yahoogroups.com, Holden Karnofsky <holden0@...> wrote:
> > >
> > > Hi Vipul,
> > >
> > > Thanks for the thoughts. I had a followup conversation with Dario about
> > > this topic a few days ago. I think the question of "could one fund
> > > translational research to treat/prevent aging?" provides an interesting
> > > illustration of some of the tricky dynamics here for a funder:
> > >
> > > - It's possible that if there were a great deal more attention giving to
> >
> > > treating/preventing aging, we would have some promising treatments. So in a
> > > broad sense it's possible that aging is underinvested in.
> > > - A lot of the best basic biology research isn't clearly pointing toward
> >
> > > one treatment/condition or another; it's about understanding the
> > > fundamentals of how organisms operate. So having an interest in treating
> > > aging, as opposed to cancer, might not have a major impact on which
> > > projects one funds, if one's main goal is to fund outstanding basic biology
> > > research.
> > > - Perhaps because of the lack of emphasis on treating aging (or perhaps
> >
> > > because it's simply too difficult of a problem), there don't seem to be
> > > promising findings in the "Valley of Death" relevant to aging; the few
> > > promising leads have been explored.
> > > - So even if, in a broad sense, there is too little attention given to
> >
> > > this problem, knowing this doesn't necessarily yield a clear direction for
> > > a relatively small-scale funder of biomedical research.
> > >
> > > Best,
> > > Holden
> > >
> > > On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:
> > >
> > > > **
> >
> > > >
> > > >
> > > > Holden,
> > > >
> > > > First, I think that this is an excellent document. I checked for a
> > > > number of things that I had heard about (Breakout Labs, John
> > > > Ioannidis, Cochrane Collaboration) and they're all there in your
> > > > document.
> > > >
> > > > The one thing that's not explicitly mentioned: longevity and life
> > > > extension research. At least prima facie, this seems like something
> > > > that should be more important than individual disease research, and it
> > > > seems like a classic "Valley of Death" case (pun unintended, but
> > > > noted) -- T1 stage to use your terminology. I think the SENS website
> > > > http://www.sens.org would be a good starting point for one of the (to
> > > > me promising) approaches to life extension. I recall from past
> > > > conversations that you were aware of SENS, so this is not new to you,
> > > > but I think that longevity should be included as part of any
> > > > discussion of biomedical research and given separate consideration
> > > > given that it has a much lower status than research into specific
> > > > conditions such as cancer, dementia, etc. You may ultimately conclude
> > > > that not enough can be done in this area, but I think it should be
> > > > part of your preliminary stuff. [btw, the United States has a National
> > > > Institute of Aging, but it's much lower-status than most of the other
> > > > grantmakers mentioned here].
> > > >
> > > > Vipul
> > > >
> > > > * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed) wrote
> > > >
> > > > > Note to the research list: we're now considering reopening our
> > > > > investigation of the world of biomedical research. We've started and
> > > > > stopped a couple of times in this area before; this time I decided to
> > > > start
> > > > > with a conversation with Dario Amodei, a longtime GiveWell follower and
> > > > > personal friend who is currently a biology postdoc at Stanford. My goal
> > > > > with the conversation was just to get some basic context and start
> > > > putting
> > > > > together a framework for thinking about the issue, not to use him as an
> > > > > authoritative source, and the notes below should be read in that spirit.
> > > > >
> > > > >
> > > > > This email has two sections:
> > > > >
> > > > >
> > > > > 1. Notes that I emailed out internally after my conversation with Dario,
> > > > > slightly edited
> > > > >
> > > > > 2. Some more context on the history of our work on biomedical research
> > > > and
> > > > > why we think it's appropriate to investigate this field (this was a
> > > > > response to a question following my original email, along the lines of
> > > > "Why
> > > > > are you looking into biomedical research now, given that's an area with a
> > > > > lot of buzz and funding from wealthy donors, and how does this work
> > > > relate
> > > > > to the 'meta-research' work?")
> > > > >
> > > > >
> > > > >
> > > > > *--*
> > > > >
> > > > > *
> > > > > *
> > > > >
> > > > > *1. Notes that I emailed out internally after my conversation with Dario,
> > > > > slightly edited*
> > > > >
> > > > > *
> > > >
> > > > > *
> > > > >
> > > > > I've done some preliminary work trying to figure out what it would look
> > > > > like to explore biomedical sciences as funding area. This mostly
> > > > consisted
> > > > > of a 3-hour conversation with Dario (recording is available), reading two
> > > > > papers he sent and a few I found while Googling, and prior knowledge. I'm
> > > > > including Dario in all emails related to this stuff, as an informal
> > > > advisor.
> > > > >
> > > > > * *
> > > > >
> > > > > *My picture of "what the biomedical research world roughly looks like"
> > > > *(this
> > > >
> > > > > is mostly from talking with Dario + prior knowledge)
> > > > >
> > > > > · *Academic biology* studies how organisms work and develops tools to
> > > >
> > > > > observe and manipulate the building blocks of organisms.
> > > > >
> > > > > o The vast bulk of the funding - and the most prestigious funding - comes
> > > > > from the NIH.
> > > > >
> > > > > o There is also funding from what I've heard called "foundations" -
> > > > groups
> > > > > like the American Cancer Society and American Heart Association - which
> > > > > function very similarly to the NIH, in that they tend to hire people with
> > > > > strong academic credentials and those people judge the merits of grant
> > > > > proposals.
> > > > >
> > > > > o Both the NIH and "foundations" tend to be formally partitioned by
> > > > > disease, but much of the work done by academic biologists is potentially
> > > > > relevant to many diseases. A researcher seeking NIH funding may apply to
> > > > > several different NIH "study sections," though only one at a time (a list
> > > > > of "study sections" is at
> > > > > http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
> > > > > whether s/he gets funding is going to depend more on the academic merits
> > > > of
> > > > > the researcher & work than anything else.
> > > > >
> > > > > o There are basic definitions of "academic merit " that generally shape
> > > > > the whole ecosystem: the people handing out money are selected by these
> > > > > criteria and use these criteria, the people doing the research know that
> > > > > these criteria shape their career prospects, etc.
> > > > >
> > > > > · *Private startups* investigate promising ideas for new
> > > >
> > > > > treatments/diagnostics/devices. They may often take the form of a biology
> > > > > professor spinning off a biotech startup (run by former postdocs) that
> > > > > raises venture capital, based on the research the professor did. They
> > > > take
> > > > > basic knowledge about how the body works (for example, protein X is
> > > > crucial
> > > > > for medical condition Y) and do the necessary testing to find a promising
> > > > > treatment/diagnostic/device (for example, testing a lot of compounds on
> > > > > animals until they find one that affects protein X).
> > > > >
> > > > > · *Big pharma/biotech companies *are best positioned to deal with the
> > > >
> > > > > extremely expensive process of conducting clinical trials and getting FDA
> > > > > approval. Acquisition by one of these is the most common form of exit for
> > > > > startups.
> > > > >
> > > > > · *Academic medicine, epidemiology and other fields* also do work
> > > >
> > > > > relevant to medicine, including studying questions whose main relevance
> > > > is
> > > > > to medical practice and public health programs: how effective is
> > > > > treatment/practice X in situation Y, how cost-effective is it, etc.
> > > > > Sometimes they will hit on commercializable insights (for example, a new
> > > > > kind of device) as well.
> > > > >
> > > > > · *Translational research* is a broad term referring to a bridge between
> > > >
> > > > > academic research and treatments/practices. It can include (
> > > > >
> > > > http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> > > > > ):
> > > > >
> > > > > o T1 - going from basic science (usually academic biology, I presume) to
> > > > a
> > > > > new treatment/practice. This includes research that helps go from an
> > > > > academic biology insight to a private startup.
> > > > >
> > > > > o T2 - I think this is basically what Cochrane does - going from academic
> > > > > medicine/epidemiology (a bunch of studies on what treatments/practices
> > > > are
> > > > > effective) to the development of guidelines that actually affect
> > > > practice.
> > > > >
> > > > > o T3, T4 - research on how to actually change practice (as opposed to
> > > > > setting the guidelines that are a "target" for practice) and get better
> > > > > real-world results.
> > > > >
> > > > > *
> > > > > *
> > > > >
> > > > > *Potential "big opportunity to do good" #1: translational research and
> > > > the
> > > >
> > > > > "Valley of Death" (this comes from prior knowledge, googling "Valley of
> > > > > Death" and reading a bunch of the stuff I found, and checking out the
> > > > > FasterCures website again)*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > The "Valley of Death" seems to mostly refer to a shortage of T1. The idea
> > > > > is that there are a lot of cases where there's an academic insight that's
> > > > > potentially valuable in coming up with a new treatment, but to get it to
> > > > > the point where it's attractive from a for-profit perspective, you need
> > > > to
> > > > > do a lot of stuff that academics don't have a reason to do. "For example,
> > > > > an upstream finding that a given protein is differentially expressed in
> > > > > individuals with a particular disease may suggest that the protein merits
> > > > > further investigation. However, much more work (especially medicinal
> > > > > chemistry) is necessary to determine how good a target the protein really
> > > > > is and whether a marketable drug candidate that affects the activity of
> > > > the
> > > > > protein is likely to be developed." (
> > > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> > > > >
> > > > >
> > > > >
> > > > > There are claims that this sort of work is massively underfunded (by the
> > > > > people we've spoken to who talked about the "Valley of Death"; also in
> > > > >
> > > > http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
> > > > (1).pdf).
> > > > > However, funding isn't the only issue. The other issue is that
> > > > > "pharmaceutical firms that hold libraries of potentially useful small
> > > > > molecules as trade secrets, making them largely off limits to ...
> > > > academic
> > > > > scientists" (
> > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> > > > > In other words, there is some room for new models of collaboration that
> > > > > lead to better communication and information sharing between academia and
> > > > > industry (or between industry and industry).
> > > > >
> > > > >
> > > > > Both the Myelin Repair Foundation and Michael Milken's work on prostate
> > > > > cancer have been pointed to as examples of innovative collaborations that
> > > > > deal with some of the information sharing problems. Milken's model:
> > > > > "drastically cutting the wait time for grant money, to flood the field
> > > > with
> > > > > fast cash, to fund therapy-driven ideas rather than basic science, to
> > > > hold
> > > > > researchers he funds accountable for results, and to demand collaboration
> > > > > across disciplines and among institutions, private industry, and
> > > > academia."
> > > > > (
> > > > >
> > > > http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
> > > > )
> > > > > Myelin Repair Foundation sounds broadly similar (
> > > > >
> > > > http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
> > > > ).
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > More broadly:
> > > > >
> > > > > · FasterCures (also a Michael Milken production) looks like it's focused
> > > > > on the broad mission of "more research with a shorter timeline to
> > > > > treatments," with a heavy sub-focus on the Valley of Death. In addition
> > > > to
> > > > > its conference and philanthropic advisory service, it advocates for FDA
> > > > > improvements (presumably to speed the approval process), advocates for
> > > > the
> > > > > NIH to put more funding into translational research (there have
> > > > definitely
> > > > > been a lot of new initiatives at the NIH focused on this stuff in the
> > > > past
> > > > > few ~decade), promotes "innovative financing mechanisms" for bridging the
> > > > > Valley of Death (these include Peter Thiel's Breakout Labs; I think they
> > > > > occupy a conceptually similar space to "social enterprise investment"
> > > > > though they tend to be structured more as grants and less as
> > > > > double-bottom-line investments), and works on getting patient data opened
> > > > > to researchers. The only program of theirs I haven't mentioned is TRAIN;
> > > > I
> > > > > can't (easily) figure out what this is.
> > > > >
> > > > > · John Ioannidis stated to us that all translational research is
> > > > > underfunded, not just T1. (The context we talked to him in, of course,
> > > > was
> > > > > T2.)
> > > > >
> > > > >
> > > > > These issues seem to have quite a bit of buzz. There are some really
> > > > stark
> > > > > #'s out there: even as R&D investment has gone way up over the past 50
> > > > > years, the # of new drugs has stayed roughly constant at around 20 a
> > > > year.
> > > > >
> > > > >
> > > > > Dario sent a really interesting paper on this topic. It argues: (a) the #
> > > > > of NMEs (new medical entities, i.e., drugs) per year has mostly remained
> > > > > constant-with-noise (no trend); (b) all of the big companies seem to have
> > > > > produced NMEs at a very steady pace, even as they've changed size, though
> > > > > different companies do have different rates of NME creation; (c) when it
> > > > > comes to mergers & acquisitions, "1+1=1": companies that acquire other
> > > > > companies just keep up the same NME pace; (d) over time, the # of large
> > > > > companies has shrunk (due to mergers) and the # of small companies has
> > > > > risen, and the share of NMEs attributed to small companies has gone from
> > > > > ~30% to ~70%; (e) the cost per NME has gone up over time and is now in
> > > > the
> > > > > neighborhood of $1 billion; (f) the % of NMEs that are "blockbusters"
> > > > (huge
> > > > > profits) has been steady around 20%, despite intensifying efforts on the
> > > > > part of pharma to fund only potential blockbusters.
> > > > >
> > > > >
> > > > >
> > > > > There are also claims - such as by Derek Lowe, a blogger Dario pointed me
> > > > > to - that the "Valley of Death" is an overrated concept: there is a high
> > > > > ratio of academic discoveries to useful drugs, but this is just because a
> > > > > lot of stuff turns out not to work, not because we have a shortage of
> > > > > doable translational research. The paper above sort of takes this view,
> > > > > essentially arguing that nothing seems to raise NME production; we'll
> > > > need
> > > > > something really radical to make any difference.
> > > > >
> > > > >
> > > > >
> > > > > The case studies in the FasterCures paper on this are interesting. They
> > > > > imply that there were some pretty low-hanging fruit in the T1 domain. It
> > > > > might be interesting to talk to Fastercures and see whether they can help
> > > > > identify "Valley of Death" opportunities that are slam-dunkish.
> > > > >
> > > > >
> > > > > *Potential "big opportunity to do good" #2: inefficiencies in academic
> > > > > biology*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > All of the above seems pretty distinct from the question of how to make
> > > > > academic biology better at doing its job of understanding the body. This
> > > > is
> > > > > the question that Dario and I focused on.
> > > > >
> > > > >
> > > > >
> > > > > Dario painted a picture in which most of academia plays by the same set
> > > > of
> > > > > rules, making it very hard for people to do things that break those rules
> > > > > (for example, academics are expected to publish a lot; it's risky to work
> > > > > on a "blue sky" or highly ambitious project). Thus, for anyone who wants
> > > > a
> > > > > career in academia, a couple of years working on a moonshot project are
> > > > > risky; in addition, being on bad terms with a small number of people can
> > > > > damage a career (since there is often a small set of people that makes a
> > > > > large proportion of the career-impacting decisions for a given area, and
> > > > it
> > > > > can be hard to escape this set of people without changing research
> > > > > interests significantly). The bad news is that this isn't particularly
> > > > easy
> > > > > to fix: you can offer funding for blue-sky projects, but (a) a lot of
> > > > > academics basically train themselves to play by the rules, and won't
> > > > > necessarily have thought about "what should be studied if these rules
> > > > > didn't apply," and more importantly, (b) there are many incentives
> > > > pushing
> > > > > academics towards playing by certain rules; funding is only one of those
> > > > > mechanisms (there's also tenure, peer review, etc) so changing that one
> > > > > incentive won't always change behavior. Dario says that he might hesitate
> > > > > to work on a particular blue-sky project that he thinks is interesting,
> > > > > even if he got funding for it, for these reasons.
> > > > >
> > > > >
> > > > > Despite this, there are some funders who push the boundaries. There are
> > > > > medical centers that don't require teaching and do more ambitious work.
> > > > > There's the McKnight foundation, which funded some of the pioneering work
> > > > > on optical control of neurons for which funding might have been difficult
> > > > > to obtain by traditional routes. There's the HHMI Janelia Farms campus,
> > > > > which Dario thinks is the most promising thing out there in terms of a
> > > > > model - academics who go there get guaranteed (and generous) funding for
> > > > 6
> > > > > years, which frees them up to take much bigger risks. (I saw a study
> > > > > claiming that the work coming out of Janelia Farms has a much more skewed
> > > > > citation count distribution, implying bigger risks - i.e., fewer papers
> > > > > with decent #'s of citations but more "blockbuster" papers with lots of
> > > > > citations). But Janelia Farms is only in one sub-field; comparable
> > > > > institutions don't exist for other fields (as far as Dario knows)..
> > > > >
> > > > >
> > > > >
> > > > > Dario's gut is that one of the best things a funder could do would be to
> > > > > work toward creating a large institute that largely "plays by its own
> > > > > rules," encouraging more ambitious work and providing enough security and
> > > > > sheer volume of dollars/researchers as to establish a sort of "parallel
> > > > > system" to academia - thus becoming a place that could provide viable and
> > > > > reliable career options for people interested in playing by different
> > > > > rules. I'm also interested in the idea of trying to advocate for changes
> > > > in
> > > > > the rules, as we've discussed in the context of meta-research.
> > > > >
> > > > >
> > > > >
> > > > > There's also the other stuff like data/code sharing.
> > > > >
> > > > >
> > > > >
> > > > > *Another option: give $ to the NIH*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > The NIH is by far the largest, most prestigious, most respected funder in
> > > > > academic biology, and most of academia runs on criteria that mirror the
> > > > > NIH's. I would guess that an unrestricted check to the NIH would get
> > > > > allocated in a pretty sector-agnostic way. It seems like this is a giving
> > > > > option that is pretty nontrivial to beat. Anyone we consider for funding
> > > > > ought to be able to explain why they're better at allocating the funds
> > > > than
> > > > > the NIH. We haven't talked to the NIH about whether it would accept these
> > > > > sorts of donations.
> > > > >
> > > > >
> > > > > With the context I now have, FasterCures strikes me as pretty
> > > > > promising/interesting too, though their "room for more funding" situation
> > > > > isn't clear (w/o talking to them). But they also seem sector-agnostic
> > > > > within biomedical research, while pushing a very specific theory of
> > > > change
> > > > > that may or may not be valid.
> > > > >
> > > > >
> > > > >
> > > > > *Next steps*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > I think understanding this world would be a major undertaking. I think
> > > > we'd
> > > > > have to be ready to put in a lot of work and to be open to ways of
> > > > bridging
> > > > > "funder-expert gap" that we haven't tried before. For example:
> > > > >
> > > > > · Dario suggested that I read a freshman biology textbook. He said he
> > > > > realizes that the time cost could be very large - something like 100
> > > > hours
> > > > > - but that without doing so, I'm going to be lacking too much context on
> > > > > why biomedical research works the way it does. He thinks that reading a
> > > > > basic textbook would get me to the point of strongly diminishing marginal
> > > > > returns.
> > > > >
> > > > > · Also in the category of "personally picking up rudimentary subject
> > > > > matter knowledge," I thought it might be worth trying to follow the
> > > > > development of a particular drug from start to finish - Gleevec (the
> > > > > "miracle drug") would be a good candidate. The goal would be to
> > > > understand
> > > > > each stage of insight leading to new investigation, and where the funding
> > > > > came from at each stage.
> > > > >
> > > > > · I think hiring Dario or someone like him would make a lot of sense.
> > > > > I've thought about whether we should be hiring "subject matter experts"
> > > > in
> > > > > other areas, such as global health, but in my view the need is clearer
> > > > here
> > > > > than in any other area. One of the things I don't love about hiring an
> > > > > expert in a given field, at this stage of our research, is that we could
> > > > > quickly decide that we're just not interested enough in the field in
> > > > > question ... but someone with the right kind of technical knowledge &
> > > > > experience would be so far ahead of us in evaluating *any* area of
> > > > biology
> > > > > research that it seems like a good idea. (JTBC, I'm also actively
> > > > thinking
> > > > > about whether it would make sense to hire experts in other fields ...)
> > > > >
> > > > > · Talking to major funders and potentially co-funding with them is
> > > > > probably essential. Important groups to talk to would include NIH (by far
> > > > > the most important; we've already talked to them a bit), the colloquium
> > > > of
> > > > > groups like the American Cancer Society (it has a name; I forget the
> > > > name),
> > > > > FasterCures, Wellcome, and potentially some funders with
> > > > > unorthodox/buzzed-about approaches: Myelin Repair Foundation, McKnight
> > > > > Foundation, and maybe another private foundation or two (I emailed Dario
> > > > > all my notes on major foundations that do biomedical research so he could
> > > > > send thoughts on whether any look interesting aside from McKnight).
> > > > >
> > > > >
> > > > > My next plan had been to talk to academics (Dario had good suggestions
> > > > > about how to approach them), but with all the potential work to be done
> > > > on
> > > > > gaining basic context, I'm not sure that's the right next step. But it's
> > > > > also a possible step.
> > > > >
> > > > >
> > > > >
> > > > > *2. Some more context on the history of our work on biomedical research
> > > > and
> > > >
> > > > > why we think it's appropriate to investigate this field (this was a
> > > > > response to a question following my original email, along the lines of
> > > > "Why
> > > > > are you looking into biomedical research now, given that's an area with a
> > > > > lot of buzz and funding from wealthy donors, and how does this work
> > > > relate
> > > > > to the 'meta-research' work?")*
> > > > >
> > > > >
> > > > > - We've always had "disease research" as a cause we wanted to
> > > >
> > > > > investigate. We've never had a good reason for not investigating it other
> > > > > than that investigating it seems difficult. We've tried more than once to
> > > > > investigate it, and it's ended up falling by the wayside because of how
> > > > big
> > > > > an undertaking it is.
> > > > > - It's true that there is a huge amount of funding and buzz in this
> > > >
> > > > > area. But it also seems quite possible that there isn't nearly enough; in
> > > > > fact this seems like a possible point of consensus between all the people
> > > > > concerned about the "Valley of Death." This potential good accomplished
> > > > via
> > > > > biomedical research appears unlimited. What I consider to be the mark of
> > > > a
> > > > > bad sector is "tons of buzz/$ and mediocre returns," but the latter part
> > > > > doesn't apply to biomed; in fact biomedical research is arguably one of
> > > > > philanthropy's and even humanity's top success stories.
> > > > > - There are also indications that despite all the buzz and funding,
> > > >
> > > > > there are still opportunities to do things differently and shake things
> > > > up.
> > > > > There is more than one case where an outsider (Milken, MRF) basically
> > > > came
> > > > > in and did things very differently and now even experts in the field seem
> > > > > to credit them with positive change. There's also a good explanation for
> > > > > why this might be the case: while there are a ton of people and dollars,
> > > > > they largely seem to play by one self-reinforcing/network-effect-prone
> > > > set
> > > > > of rules, implying high returns to disrupting that equilibrium.
> > > > > - So, we've always wanted to get into this area. There was a period
> > > >
> > > > > where I was presenting meta-research as our best entry point into this
> > > > > field: my vision was that we would talk to academics about what
> > > > systematic
> > > > > failings there were and what funding opportunities these implied, and
> > > > that
> > > > > would be as good a way as any to get acclimated in biomed. But this
> > > > period
> > > > > wasn't particularly long - the May blog post on Labs priority causes
> > > > lists
> > > > > scientific research as a promising area distinct from meta-research.
> > > > > - I set up the call with Dario without having a clear idea of whether I
> > > >
> > > > > wanted to approach biomed from a "meta-research" angle or another angle.
> > > > > After the call and other investigations described here, I got a clearer
> > > > > idea of what I think is the best path forward.
> > > > >
> > > > > Bottom line - I think it's important to build an understanding of
> > > > > biomedical research, and that we should take the best path to doing so
> > > > > whether or not that dovetails with the meta-research work (likely it will
> > > > > dovetail some but not 100%).
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
> > 
> 
> 

Aubrey, I would love to hear more about two things you raise as worthwhile
questions.

1) The distinction between longevity on the one hand and the postponement
of age-related ill health on the other (I understand the different
definitions, but am interested in thoughts on the likely implications of
each for the other, in a real-world population-wide context - for instance,
would better health result simply delay the in one's 70s-80s result in
viewing the demise of health in 90s-100s .)

2) In the context of any implications for longevity, whether / why "the
medical control of aging [is] a hugely valuable mission?" relative to other
uses of philanthropic funds for medical research.


On 13 October 2012 21:24, Aubrey de Grey <aubrey@...> wrote:

> Hi Holden - many thanks.
>
> First: yes, there are really three somewhat separate questions for someone
> trying to evaluate whether to support SENS Foundation:
>
> 1) Is the medical control of aging a hugely valuable mission?
>
> 2) Assuming "yes" to (1), is it best achieved by basic research or
> translational research?
>
> 3) Assuming translational, is SENS Foundation the organisation that uses
> money most effectively in pursuit of that mission?
>
> I had rather expected that you would take some convincing on item (1), and
> much of what I wrote last time was focused on that. Since it isn't the
> focus of your question to me, I'm now going to assume until further notice
> that there is no dissent on that.
>
> So, to answer your question: actually you're not putting aside the
> basic-vs-translational question as much as you may think you are. The word
> "translational" is flavour of the month in government funding circles these
> days (not only in the USA), so it's not surprising that the NIA has a
> public statement of the kind you pointed to. However, notice that the link
> they give "for more information" is to a page listing ALL "Funding
> Opportunity Announcements". There is no page specifically for translational
> ones, and the reason there isn't is that the amount of work that the NIA
> actually funds that could really be called translational is tiny. In other
> words, the page you found is actually just blatant spin. The neuroscience
> slice you mention is an anomaly arising from the way NIA was founded (the
> natural place for that money is clearly NINDS): the fact that it's NIA
> money does not, in practice, translate into its being spent on work to
> prevent neurodegeneration by treating its cause (aging). Instead, just like
> NINDS money, it's spent on attacking neurodegeneration directly, as if such
> diseases could be eliminated from the body just like an infection: the same
> old mistake that afflicts, and dooms, the whole of geriatric medicine.
>
> So, the first answer to your question is that SENS Foundation really DOES
> focus on translational research, with an explicit goal of postponing
> age-related ill-health. But there's also another big difference: we can
> attack this problem relatively free of the other priorities that afflict
> mainstream funding (whether from NIH or from trasitional foundations). Most
> importantly, though we do and will continue to publish our interim results
> in the peer-reviewed literature, we are much less constrained by "publish
> or perish" tyranny than typical academics are. This allows us to proceed by
> constructing and implementing a rational "project plan" (namely SENS) to
> get to the intended goal (the defeat of aging), whereas what little
> translational work is funded by NIA or others is guided overwhelmingly by
> the imperative to get some kind of positive result as quickly as possible,
> even when it's understood that those results are not remotely likely to
> "scale", i.e. to translate into eventual medical treatments that
> significantly delay aging. A great example of this is the NIA's
> Interventions Testing Program (ITP) to test the mouse longevity effects of
> various small molecules. The ITP only exists at all (and in a far smaller
> form than originally intended) as a result of several years of persistence
> by the then head of the NIA's biology division (Huber Warner), and it
> focuses entirely on delivery of simple drugs starting rather early in life,
> with the result that no information emerges that's relevant to treating
> people who are already in middle age or older. (This is despite the fact
> that by far the most high-profile result that the ITP has delivered so far,
> the benefits of rapamycin, actually WAS a late-onset study: it wasn't meant
> to be, but technical issues delayed the experiment.) In a nutshell, there
> is a huge bias against high-risk high-gain work.
>
> The third thing that distinguishes SENS Foundation's approach is that we
> can transcend the "balkanisation" (silo mentality) that dominates
> mainstream academic funding. When one submits a grant application to NIA,
> it is evaluated by gerontologists, just as when one submits to NCI it is
> evaluated by oncologists, etc. What's wrong with this is that it biases the
> system immensely against cross-disciplinary proposals. SENS is a plan that
> brings together a large body of knowledge from gerontology but also a huge
> amount of expertise that was developed for other reasons entirely - to
> treat acute disease/injury, or in some cases for purposes that were not
> biomedical at all (notably environmental decontamination). It doesn't
> matter how robust the objective scientific and technological argument is
> for work of that sort: it will never compete (especially in today's very
> tight funding environment) with more single-topic proposals all of whose
> details can be understood by reviewers from a particular single field.
>
> The final thing to mention, and this actually also answers your question
> to Vipul about basic versus translational research, is that SENS is a plan
> that has stood the test of time. I've been propounding it since 2000, well
> before SENS Foundation existed, and it used to come in for a lot of
> criticism (initially more in the form of off-the-record ridicule, and
> latterly, at my behest, in print), but in every single case that criticism
> was found to stem from ignorance on the part of the detractor, either of
> what I proposed or of published experimental work on which the proposal was
> based. That's why I'm now regularly asked to organise entire sessions at
> mainstream gerontology conferences, whereas as little as five years ago I
> would never even be invited to speak. It's also why the Research Advisory
> Board of SENS Foundation consists of such prestigious scientists. This is a
> very strong argument, in my view, for believing that now is the time to
> sink a proper amount of money into translational gerontology (though
> certainly not to cease doin basic biogerontology too). It's well known that
> basic scientists are often not the most far-sighted when it comes to seeing
> how to apply their discoveries (attitudes in 1900 to the feasibility of
> powered flight being the canonical example). It is therefore a source of
> concern that almost all the experts who have the ear of funders in this
> field are basic scientists, whose instinct is to carry on finding things
> out and to deprioritise the tedious business of applying that knowledge.
> SENS has achieved a gratisfying level of legitimacy in gerontology, but it
> is still foreign to most card-carrying gerontologists, and as such it
> remains essentially unfundable via mainstream mechanisms. Hence the need to
> create a philanthropy-driven entity, SENS Foundation, to get this work done.
>
> Let me know if this helps, or if you have further questions.
>
> Cheers, Aubrey
>
> On 12 Oct 2012, at 15:28, Holden Karnofsky wrote:
>
> >
> > Hi Aubrey,
> >
> > Thanks for the thoughts.
> >
> > The NIH appears to have a division focused on research relevant to this
> topic: http://www.nia.nih.gov/research/dab . Its budget appears to be
> ~$175 million (per year). The National Institute on Aging, which houses
> this division, has a budget of about $1 billion per year, including a
> separate ~$400 million for neuroscience (which may also be relevant) as
> well as $115 million for intramural research. Figures are from
> http://www.nia.nih.gov/about/budget/2012/fiscal-year-2013-budget. The
> Institute states that its mandate includes translational research (
> http://www.nia.nih.gov/research/faq/does-nia-support-translational-research).
> How would you distinguish your work from this work?
> >
> > (For the moment I'm putting aside the question I raised in my previous
> response to Vipul on this topic, regarding whether it's best to approach
> biology funding from the perspective of "trying to treat/cure a particular
> condition" or "trying to understand  fundamental questions in biology whose
> applications are difficult to predict.")
> >
> > Best,
> > Holden
> >
> > On Fri, Oct 12, 2012 at 7:32 AM, aubrey.degrey <aubrey@...> wrote:
> >
> > Hi everyone,
> >
> > My attention was brought to this thread, by virtue of the fact that it
> was my work that gave rise to SENS Foundation, and I'm looking forward to
> getting more involved here; I've held the Effective Altruism movement in
> high regard for some time. However, given my newbie status here I want to
> start by apologising in advance for any oversight of previously-discussed
> issues etc. I'm naturally delighted both at Holden's post and at Vipul's
> reply (which I should stress that I did not plant! - I do not know Vipul at
> all, though I look forward to changing that). I would like to mention just
> a few key points for discussion:
> >
> > - Holden, I want to compliment you on your appreciation of how academia
> really works. Everything you say about that is spot on. The aversion to
> "high risk high gain" work that has arisen and become so endemic in the
> system is the most important point here, in terms of why parallel funding
> routes are needed.
> >
> > - I'm slightly confused that a lot of Holden's remarks are focused on
> the private sector (i.e. startups), since my understanding was that
> GiveWell is about philanthropy; but I realise that there is not all that
> clear a boundary between the two (and I note the mention of Breakout Labs,
> with which I have close links and which sits astride that divide more than
> arguably anyone). The "valley of death" in pre-competitive translational
> research is a rather different one than that encountered by startups, but
> the principle is the same, and research to postpone aging certainly
> encounteres it.
> >
> > - Something that I presume factors highly among GiveWell's criteria is
> the extent to which a cause may be undervalued by the bulk of major
> philanthropists, such that an infusion of additional funds would make more
> of a difference than in an area that is already being well funded. To me
> this seems to mirror the logic of focusing on the shortcomings (gaps) in
> NIH's funding (and that of traditional-model foundations). Holden notes
> that "Anyone we consider for funding ought to be able to explain why
> they're better at allocating the funds than the NIH" and I agree
> wholeheartedly, but my inference is that he thinks that some orgs may
> indeed be able to explain that. I certainly think that SENS Foundation can.
> >
> > - Coming to aging: research to postpone aging has the unique problem of
> quite indescribeable irrationality on the part of most of the general
> public, policy-makers and even biologists with regard to its desirability.
> Biogerontologists have been talking to brick walls for decades in their
> effort to get the rest of the world to appreciate that aging is what causes
> age-related ill-health, and thus that treatments for aging are merely
> preventative geriatrics. The concept persists, despite biogerontologists'
> best efforts, that aging is "natural" and should be left alone, whereas the
> diseases that it brings about are awful and should be fought. This is made
> even more bizarre by the fact that the status of age-related diseases as
> aspects of the later stages of aging absolutely, unequivocally implies that
> efforts to attack those diseases directly are doomed to fail. As such, this
> is a (unique? certainly very rare) case where a philanthropic contribution
> can make a particularly big difference simply because most philanthropists
> don't see the case for it. It underpins why having an interest in treating
> aging, as opposed to cancer, absolutely has a major impact on which
> projects one funds. It's also a case for (if I understand the term
> correctly) meta-research.
> >
> > - A lot of the chatter about treating aging revolves around longevity,
> but it shouldn't. I'm all in favour of longevity, don't get me wrong, but
> it's not what gets me up in the morning: what does is health. I want people
> to be truly youthful, however long ago they were born: simple as that. The
> benefits of longevity per se to humanity may also be substantial, in the
> form of greater wisdom etc, but that would necessarily come about only very
> gradually (we won't have any 1000-year-old for at least 900 years whatever
> happens!), so it doesn't figure strongly in my calculations.
> >
> > - When forced to acknowledge that the idea of aging being a
> high-priority target for medicine is an inescapeable consequence of things
> they already believe (notably that health is good and ageism is bad), many
> people retreat to the standpoint that it's never going to be possible so
> it's OK to be irrational about whether it's desirable. The feasibility of
> postponing age-related ill-health by X years with medicine available Y
> years from now is, of course, a matter of speculation on which experts
> disagree, just as with any other pioneering technology. I know that Holden
> and others have expressed caution (at best) concerning the accuracy of any
> kind of calculation of probabilities of particular outcomes in the distant
> (or even not-so-distant) future, and I share that view. However, an
> approach that may appeal more is to estimate how much humanitarian benefit
> a given amount of progress would deliver, and then to ask how unlikely that
> scenario needs to be to make it not worth pursuing. My claim is that the
> benefits of hastening the defeat of aging by even a few years (which is the
> minimum that I claim SENS Foundation is in a position to do, given adequate
> funding) would be so astronomical that the required chance of success to
> make such an effort worthwhile would be tiny - too tiny for it to be
> reasonable to argue that such funding would be inadvisable. But of course
> that is precisely what I would want GiveWell to opine on.
> >
> > - In the event that GiveWell (or anyone else) were to decide and declare
> that the defeat of aging is indeed a cause that philanthropists should
> support, there then arises the question of which organisation(s) should be
> supported in the best interests of that mission. We at SENS Foundation have
> worked diligently to rise as quickly as possible in the legitimacy stakes
> by all standard measures, but we are still young and there remains more to
> do. If I were to offer an argument to fund us rather than any other entity,
> it would largely come down to the fact that no other organisation has even
> a serious plan for defeating aging, let alone a track record of
> implementing such a plan's early stages.
> >
> > - A significant chunk of what we do is of a kind that I think comes
> under "meta-research". A prominent example is a project we're funding at
> Denver University to extend the well-respected forecasting system
> "International Futures" so that it can analyse scenarios incorporating
> dramatically postponed aging.
> >
> > I greatly welcome any feedback.
> >
> > Cheers, Aubrey
> >
> >
> >
> > --- In givewell@yahoogroups.com, Holden Karnofsky <holden0@...> wrote:
> > >
> > > Hi Vipul,
> > >
> > > Thanks for the thoughts. I had a followup conversation with Dario about
> > > this topic a few days ago. I think the question of "could one fund
> > > translational research to treat/prevent aging?" provides an interesting
> > > illustration of some of the tricky dynamics here for a funder:
> > >
> > > - It's possible that if there were a great deal more attention giving
> to
> >
> > > treating/preventing aging, we would have some promising treatments. So
> in a
> > > broad sense it's possible that aging is underinvested in.
> > > - A lot of the best basic biology research isn't clearly pointing
> toward
> >
> > > one treatment/condition or another; it's about understanding the
> > > fundamentals of how organisms operate. So having an interest in
> treating
> > > aging, as opposed to cancer, might not have a major impact on which
> > > projects one funds, if one's main goal is to fund outstanding basic
> biology
> > > research.
> > > - Perhaps because of the lack of emphasis on treating aging (or perhaps
> >
> > > because it's simply too difficult of a problem), there don't seem to be
> > > promising findings in the "Valley of Death" relevant to aging; the few
> > > promising leads have been explored.
> > > - So even if, in a broad sense, there is too little attention given to
> >
> > > this problem, knowing this doesn't necessarily yield a clear direction
> for
> > > a relatively small-scale funder of biomedical research.
> > >
> > > Best,
> > > Holden
> > >
> > > On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:
> > >
> > > > **
> >
> > > >
> > > >
> > > > Holden,
> > > >
> > > > First, I think that this is an excellent document. I checked for a
> > > > number of things that I had heard about (Breakout Labs, John
> > > > Ioannidis, Cochrane Collaboration) and they're all there in your
> > > > document.
> > > >
> > > > The one thing that's not explicitly mentioned: longevity and life
> > > > extension research. At least prima facie, this seems like something
> > > > that should be more important than individual disease research, and
> it
> > > > seems like a classic "Valley of Death" case (pun unintended, but
> > > > noted) -- T1 stage to use your terminology. I think the SENS website
> > > > http://www.sens.org would be a good starting point for one of the
> (to
> > > > me promising) approaches to life extension. I recall from past
> > > > conversations that you were aware of SENS, so this is not new to you,
> > > > but I think that longevity should be included as part of any
> > > > discussion of biomedical research and given separate consideration
> > > > given that it has a much lower status than research into specific
> > > > conditions such as cancer, dementia, etc. You may ultimately conclude
> > > > that not enough can be done in this area, but I think it should be
> > > > part of your preliminary stuff. [btw, the United States has a
> National
> > > > Institute of Aging, but it's much lower-status than most of the other
> > > > grantmakers mentioned here].
> > > >
> > > > Vipul
> > > >
> > > > * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed)
> wrote
> > > >
> > > > > Note to the research list: we're now considering reopening our
> > > > > investigation of the world of biomedical research. We've started
> and
> > > > > stopped a couple of times in this area before; this time I decided
> to
> > > > start
> > > > > with a conversation with Dario Amodei, a longtime GiveWell
> follower and
> > > > > personal friend who is currently a biology postdoc at Stanford. My
> goal
> > > > > with the conversation was just to get some basic context and start
> > > > putting
> > > > > together a framework for thinking about the issue, not to use him
> as an
> > > > > authoritative source, and the notes below should be read in that
> spirit.
> > > > >
> > > > >
> > > > > This email has two sections:
> > > > >
> > > > >
> > > > > 1. Notes that I emailed out internally after my conversation with
> Dario,
> > > > > slightly edited
> > > > >
> > > > > 2. Some more context on the history of our work on biomedical
> research
> > > > and
> > > > > why we think it's appropriate to investigate this field (this was a
> > > > > response to a question following my original email, along the
> lines of
> > > > "Why
> > > > > are you looking into biomedical research now, given that's an area
> with a
> > > > > lot of buzz and funding from wealthy donors, and how does this work
> > > > relate
> > > > > to the 'meta-research' work?")
> > > > >
> > > > >
> > > > >
> > > > > *--*
> > > > >
> > > > > *
> > > > > *
> > > > >
> > > > > *1. Notes that I emailed out internally after my conversation with
> Dario,
> > > > > slightly edited*
> > > > >
> > > > > *
> > > >
> > > > > *
> > > > >
> > > > > I've done some preliminary work trying to figure out what it would
> look
> > > > > like to explore biomedical sciences as funding area. This mostly
> > > > consisted
> > > > > of a 3-hour conversation with Dario (recording is available),
> reading two
> > > > > papers he sent and a few I found while Googling, and prior
> knowledge. I'm
> > > > > including Dario in all emails related to this stuff, as an informal
> > > > advisor.
> > > > >
> > > > > * *
> > > > >
> > > > > *My picture of "what the biomedical research world roughly looks
> like"
> > > > *(this
> > > >
> > > > > is mostly from talking with Dario + prior knowledge)
> > > > >
> > > > > · *Academic biology* studies how organisms work and develops tools
> to
> > > >
> > > > > observe and manipulate the building blocks of organisms.
> > > > >
> > > > > o The vast bulk of the funding - and the most prestigious funding
> - comes
> > > > > from the NIH.
> > > > >
> > > > > o There is also funding from what I've heard called "foundations" -
> > > > groups
> > > > > like the American Cancer Society and American Heart Association -
> which
> > > > > function very similarly to the NIH, in that they tend to hire
> people with
> > > > > strong academic credentials and those people judge the merits of
> grant
> > > > > proposals.
> > > > >
> > > > > o Both the NIH and "foundations" tend to be formally partitioned by
> > > > > disease, but much of the work done by academic biologists is
> potentially
> > > > > relevant to many diseases. A researcher seeking NIH funding may
> apply to
> > > > > several different NIH "study sections," though only one at a time
> (a list
> > > > > of "study sections" is at
> > > > >
> http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
> > > > > whether s/he gets funding is going to depend more on the academic
> merits
> > > > of
> > > > > the researcher & work than anything else.
> > > > >
> > > > > o There are basic definitions of "academic merit " that generally
> shape
> > > > > the whole ecosystem: the people handing out money are selected by
> these
> > > > > criteria and use these criteria, the people doing the research
> know that
> > > > > these criteria shape their career prospects, etc.
> > > > >
> > > > > · *Private startups* investigate promising ideas for new
> > > >
> > > > > treatments/diagnostics/devices. They may often take the form of a
> biology
> > > > > professor spinning off a biotech startup (run by former postdocs)
> that
> > > > > raises venture capital, based on the research the professor did.
> They
> > > > take
> > > > > basic knowledge about how the body works (for example, protein X is
> > > > crucial
> > > > > for medical condition Y) and do the necessary testing to find a
> promising
> > > > > treatment/diagnostic/device (for example, testing a lot of
> compounds on
> > > > > animals until they find one that affects protein X).
> > > > >
> > > > > · *Big pharma/biotech companies *are best positioned to deal with
> the
> > > >
> > > > > extremely expensive process of conducting clinical trials and
> getting FDA
> > > > > approval. Acquisition by one of these is the most common form of
> exit for
> > > > > startups.
> > > > >
> > > > > · *Academic medicine, epidemiology and other fields* also do work
> > > >
> > > > > relevant to medicine, including studying questions whose main
> relevance
> > > > is
> > > > > to medical practice and public health programs: how effective is
> > > > > treatment/practice X in situation Y, how cost-effective is it, etc.
> > > > > Sometimes they will hit on commercializable insights (for example,
> a new
> > > > > kind of device) as well.
> > > > >
> > > > > · *Translational research* is a broad term referring to a bridge
> between
> > > >
> > > > > academic research and treatments/practices. It can include (
> > > > >
> > > >
> http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
> > > > > ):
> > > > >
> > > > > o T1 - going from basic science (usually academic biology, I
> presume) to
> > > > a
> > > > > new treatment/practice. This includes research that helps go from
> an
> > > > > academic biology insight to a private startup.
> > > > >
> > > > > o T2 - I think this is basically what Cochrane does - going from
> academic
> > > > > medicine/epidemiology (a bunch of studies on what
> treatments/practices
> > > > are
> > > > > effective) to the development of guidelines that actually affect
> > > > practice.
> > > > >
> > > > > o T3, T4 - research on how to actually change practice (as opposed
> to
> > > > > setting the guidelines that are a "target" for practice) and get
> better
> > > > > real-world results.
> > > > >
> > > > > *
> > > > > *
> > > > >
> > > > > *Potential "big opportunity to do good" #1: translational research
> and
> > > > the
> > > >
> > > > > "Valley of Death" (this comes from prior knowledge, googling
> "Valley of
> > > > > Death" and reading a bunch of the stuff I found, and checking out
> the
> > > > > FasterCures website again)*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > The "Valley of Death" seems to mostly refer to a shortage of T1.
> The idea
> > > > > is that there are a lot of cases where there's an academic insight
> that's
> > > > > potentially valuable in coming up with a new treatment, but to get
> it to
> > > > > the point where it's attractive from a for-profit perspective, you
> need
> > > > to
> > > > > do a lot of stuff that academics don't have a reason to do. "For
> example,
> > > > > an upstream finding that a given protein is differentially
> expressed in
> > > > > individuals with a particular disease may suggest that the protein
> merits
> > > > > further investigation. However, much more work (especially
> medicinal
> > > > > chemistry) is necessary to determine how good a target the protein
> really
> > > > > is and whether a marketable drug candidate that affects the
> activity of
> > > > the
> > > > > protein is likely to be developed." (
> > > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
> > > > >
> > > > >
> > > > >
> > > > > There are claims that this sort of work is massively underfunded
> (by the
> > > > > people we've spoken to who talked about the "Valley of Death";
> also in
> > > > >
> > > >
> http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
> > > > (1).pdf).
> > > > > However, funding isn't the only issue. The other issue is that
> > > > > "pharmaceutical firms that hold libraries of potentially useful
> small
> > > > > molecules as trade secrets, making them largely off limits to ...
> > > > academic
> > > > > scientists" (
> > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
> > > > > In other words, there is some room for new models of collaboration
> that
> > > > > lead to better communication and information sharing between
> academia and
> > > > > industry (or between industry and industry).
> > > > >
> > > > >
> > > > > Both the Myelin Repair Foundation and Michael Milken's work on
> prostate
> > > > > cancer have been pointed to as examples of innovative
> collaborations that
> > > > > deal with some of the information sharing problems. Milken's model:
> > > > > "drastically cutting the wait time for grant money, to flood the
> field
> > > > with
> > > > > fast cash, to fund therapy-driven ideas rather than basic science,
> to
> > > > hold
> > > > > researchers he funds accountable for results, and to demand
> collaboration
> > > > > across disciplines and among institutions, private industry, and
> > > > academia."
> > > > > (
> > > > >
> > > >
> http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
> > > > )
> > > > > Myelin Repair Foundation sounds broadly similar (
> > > > >
> > > >
> http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
> > > > ).
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > More broadly:
> > > > >
> > > > > · FasterCures (also a Michael Milken production) looks like it's
> focused
> > > > > on the broad mission of "more research with a shorter timeline to
> > > > > treatments," with a heavy sub-focus on the Valley of Death. In
> addition
> > > > to
> > > > > its conference and philanthropic advisory service, it advocates
> for FDA
> > > > > improvements (presumably to speed the approval process), advocates
> for
> > > > the
> > > > > NIH to put more funding into translational research (there have
> > > > definitely
> > > > > been a lot of new initiatives at the NIH focused on this stuff in
> the
> > > > past
> > > > > few ~decade), promotes "innovative financing mechanisms" for
> bridging the
> > > > > Valley of Death (these include Peter Thiel's Breakout Labs; I
> think they
> > > > > occupy a conceptually similar space to "social enterprise
> investment"
> > > > > though they tend to be structured more as grants and less as
> > > > > double-bottom-line investments), and works on getting patient data
> opened
> > > > > to researchers. The only program of theirs I haven't mentioned is
> TRAIN;
> > > > I
> > > > > can't (easily) figure out what this is.
> > > > >
> > > > > · John Ioannidis stated to us that all translational research is
> > > > > underfunded, not just T1. (The context we talked to him in, of
> course,
> > > > was
> > > > > T2.)
> > > > >
> > > > >
> > > > > These issues seem to have quite a bit of buzz. There are some
> really
> > > > stark
> > > > > #'s out there: even as R&D investment has gone way up over the
> past 50
> > > > > years, the # of new drugs has stayed roughly constant at around 20
> a
> > > > year.
> > > > >
> > > > >
> > > > > Dario sent a really interesting paper on this topic. It argues:
> (a) the #
> > > > > of NMEs (new medical entities, i.e., drugs) per year has mostly
> remained
> > > > > constant-with-noise (no trend); (b) all of the big companies seem
> to have
> > > > > produced NMEs at a very steady pace, even as they've changed size,
> though
> > > > > different companies do have different rates of NME creation; (c)
> when it
> > > > > comes to mergers & acquisitions, "1+1=1": companies that acquire
> other
> > > > > companies just keep up the same NME pace; (d) over time, the # of
> large
> > > > > companies has shrunk (due to mergers) and the # of small companies
> has
> > > > > risen, and the share of NMEs attributed to small companies has
> gone from
> > > > > ~30% to ~70%; (e) the cost per NME has gone up over time and is
> now in
> > > > the
> > > > > neighborhood of $1 billion; (f) the % of NMEs that are
> "blockbusters"
> > > > (huge
> > > > > profits) has been steady around 20%, despite intensifying efforts
> on the
> > > > > part of pharma to fund only potential blockbusters.
> > > > >
> > > > >
> > > > >
> > > > > There are also claims - such as by Derek Lowe, a blogger Dario
> pointed me
> > > > > to - that the "Valley of Death" is an overrated concept: there is
> a high
> > > > > ratio of academic discoveries to useful drugs, but this is just
> because a
> > > > > lot of stuff turns out not to work, not because we have a shortage
> of
> > > > > doable translational research. The paper above sort of takes this
> view,
> > > > > essentially arguing that nothing seems to raise NME production;
> we'll
> > > > need
> > > > > something really radical to make any difference.
> > > > >
> > > > >
> > > > >
> > > > > The case studies in the FasterCures paper on this are interesting.
> They
> > > > > imply that there were some pretty low-hanging fruit in the T1
> domain. It
> > > > > might be interesting to talk to Fastercures and see whether they
> can help
> > > > > identify "Valley of Death" opportunities that are slam-dunkish.
> > > > >
> > > > >
> > > > > *Potential "big opportunity to do good" #2: inefficiencies in
> academic
> > > > > biology*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > All of the above seems pretty distinct from the question of how to
> make
> > > > > academic biology better at doing its job of understanding the
> body. This
> > > > is
> > > > > the question that Dario and I focused on.
> > > > >
> > > > >
> > > > >
> > > > > Dario painted a picture in which most of academia plays by the
> same set
> > > > of
> > > > > rules, making it very hard for people to do things that break
> those rules
> > > > > (for example, academics are expected to publish a lot; it's risky
> to work
> > > > > on a "blue sky" or highly ambitious project). Thus, for anyone who
> wants
> > > > a
> > > > > career in academia, a couple of years working on a moonshot
> project are
> > > > > risky; in addition, being on bad terms with a small number of
> people can
> > > > > damage a career (since there is often a small set of people that
> makes a
> > > > > large proportion of the career-impacting decisions for a given
> area, and
> > > > it
> > > > > can be hard to escape this set of people without changing research
> > > > > interests significantly). The bad news is that this isn't
> particularly
> > > > easy
> > > > > to fix: you can offer funding for blue-sky projects, but (a) a lot
> of
> > > > > academics basically train themselves to play by the rules, and
> won't
> > > > > necessarily have thought about "what should be studied if these
> rules
> > > > > didn't apply," and more importantly, (b) there are many incentives
> > > > pushing
> > > > > academics towards playing by certain rules; funding is only one of
> those
> > > > > mechanisms (there's also tenure, peer review, etc) so changing
> that one
> > > > > incentive won't always change behavior. Dario says that he might
> hesitate
> > > > > to work on a particular blue-sky project that he thinks is
> interesting,
> > > > > even if he got funding for it, for these reasons.
> > > > >
> > > > >
> > > > > Despite this, there are some funders who push the boundaries.
> There are
> > > > > medical centers that don't require teaching and do more ambitious
> work.
> > > > > There's the McKnight foundation, which funded some of the
> pioneering work
> > > > > on optical control of neurons for which funding might have been
> difficult
> > > > > to obtain by traditional routes. There's the HHMI Janelia Farms
> campus,
> > > > > which Dario thinks is the most promising thing out there in terms
> of a
> > > > > model - academics who go there get guaranteed (and generous)
> funding for
> > > > 6
> > > > > years, which frees them up to take much bigger risks. (I saw a
> study
> > > > > claiming that the work coming out of Janelia Farms has a much more
> skewed
> > > > > citation count distribution, implying bigger risks - i.e., fewer
> papers
> > > > > with decent #'s of citations but more "blockbuster" papers with
> lots of
> > > > > citations). But Janelia Farms is only in one sub-field; comparable
> > > > > institutions don't exist for other fields (as far as Dario knows)..
> > > > >
> > > > >
> > > > >
> > > > > Dario's gut is that one of the best things a funder could do would
> be to
> > > > > work toward creating a large institute that largely "plays by its
> own
> > > > > rules," encouraging more ambitious work and providing enough
> security and
> > > > > sheer volume of dollars/researchers as to establish a sort of
> "parallel
> > > > > system" to academia - thus becoming a place that could provide
> viable and
> > > > > reliable career options for people interested in playing by
> different
> > > > > rules. I'm also interested in the idea of trying to advocate for
> changes
> > > > in
> > > > > the rules, as we've discussed in the context of meta-research.
> > > > >
> > > > >
> > > > >
> > > > > There's also the other stuff like data/code sharing.
> > > > >
> > > > >
> > > > >
> > > > > *Another option: give $ to the NIH*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > The NIH is by far the largest, most prestigious, most respected
> funder in
> > > > > academic biology, and most of academia runs on criteria that
> mirror the
> > > > > NIH's. I would guess that an unrestricted check to the NIH would
> get
> > > > > allocated in a pretty sector-agnostic way. It seems like this is a
> giving
> > > > > option that is pretty nontrivial to beat. Anyone we consider for
> funding
> > > > > ought to be able to explain why they're better at allocating the
> funds
> > > > than
> > > > > the NIH. We haven't talked to the NIH about whether it would
> accept these
> > > > > sorts of donations.
> > > > >
> > > > >
> > > > > With the context I now have, FasterCures strikes me as pretty
> > > > > promising/interesting too, though their "room for more funding"
> situation
> > > > > isn't clear (w/o talking to them). But they also seem
> sector-agnostic
> > > > > within biomedical research, while pushing a very specific theory of
> > > > change
> > > > > that may or may not be valid.
> > > > >
> > > > >
> > > > >
> > > > > *Next steps*
> > > >
> > > > >
> > > > >
> > > > >
> > > > > I think understanding this world would be a major undertaking. I
> think
> > > > we'd
> > > > > have to be ready to put in a lot of work and to be open to ways of
> > > > bridging
> > > > > "funder-expert gap" that we haven't tried before. For example:
> > > > >
> > > > > · Dario suggested that I read a freshman biology textbook. He said
> he
> > > > > realizes that the time cost could be very large - something like
> 100
> > > > hours
> > > > > - but that without doing so, I'm going to be lacking too much
> context on
> > > > > why biomedical research works the way it does. He thinks that
> reading a
> > > > > basic textbook would get me to the point of strongly diminishing
> marginal
> > > > > returns.
> > > > >
> > > > > · Also in the category of "personally picking up rudimentary
> subject
> > > > > matter knowledge," I thought it might be worth trying to follow the
> > > > > development of a particular drug from start to finish - Gleevec
> (the
> > > > > "miracle drug") would be a good candidate. The goal would be to
> > > > understand
> > > > > each stage of insight leading to new investigation, and where the
> funding
> > > > > came from at each stage.
> > > > >
> > > > > · I think hiring Dario or someone like him would make a lot of
> sense.
> > > > > I've thought about whether we should be hiring "subject matter
> experts"
> > > > in
> > > > > other areas, such as global health, but in my view the need is
> clearer
> > > > here
> > > > > than in any other area. One of the things I don't love about
> hiring an
> > > > > expert in a given field, at this stage of our research, is that we
> could
> > > > > quickly decide that we're just not interested enough in the field
> in
> > > > > question ... but someone with the right kind of technical
> knowledge &
> > > > > experience would be so far ahead of us in evaluating *any* area of
> > > > biology
> > > > > research that it seems like a good idea. (JTBC, I'm also actively
> > > > thinking
> > > > > about whether it would make sense to hire experts in other fields
> ...)
> > > > >
> > > > > · Talking to major funders and potentially co-funding with them is
> > > > > probably essential. Important groups to talk to would include NIH
> (by far
> > > > > the most important; we've already talked to them a bit), the
> colloquium
> > > > of
> > > > > groups like the American Cancer Society (it has a name; I forget
> the
> > > > name),
> > > > > FasterCures, Wellcome, and potentially some funders with
> > > > > unorthodox/buzzed-about approaches: Myelin Repair Foundation,
> McKnight
> > > > > Foundation, and maybe another private foundation or two (I emailed
> Dario
> > > > > all my notes on major foundations that do biomedical research so
> he could
> > > > > send thoughts on whether any look interesting aside from McKnight).
> > > > >
> > > > >
> > > > > My next plan had been to talk to academics (Dario had good
> suggestions
> > > > > about how to approach them), but with all the potential work to be
> done
> > > > on
> > > > > gaining basic context, I'm not sure that's the right next step.
> But it's
> > > > > also a possible step.
> > > > >
> > > > >
> > > > >
> > > > > *2. Some more context on the history of our work on biomedical
> research
> > > > and
> > > >
> > > > > why we think it's appropriate to investigate this field (this was a
> > > > > response to a question following my original email, along the
> lines of
> > > > "Why
> > > > > are you looking into biomedical research now, given that's an area
> with a
> > > > > lot of buzz and funding from wealthy donors, and how does this work
> > > > relate
> > > > > to the 'meta-research' work?")*
> > > > >
> > > > >
> > > > > - We've always had "disease research" as a cause we wanted to
> > > >
> > > > > investigate. We've never had a good reason for not investigating
> it other
> > > > > than that investigating it seems difficult. We've tried more than
> once to
> > > > > investigate it, and it's ended up falling by the wayside because
> of how
> > > > big
> > > > > an undertaking it is.
> > > > > - It's true that there is a huge amount of funding and buzz in this
> > > >
> > > > > area. But it also seems quite possible that there isn't nearly
> enough; in
> > > > > fact this seems like a possible point of consensus between all the
> people
> > > > > concerned about the "Valley of Death." This potential good
> accomplished
> > > > via
> > > > > biomedical research appears unlimited. What I consider to be the
> mark of
> > > > a
> > > > > bad sector is "tons of buzz/$ and mediocre returns," but the
> latter part
> > > > > doesn't apply to biomed; in fact biomedical research is arguably
> one of
> > > > > philanthropy's and even humanity's top success stories.
> > > > > - There are also indications that despite all the buzz and funding,
> > > >
> > > > > there are still opportunities to do things differently and shake
> things
> > > > up.
> > > > > There is more than one case where an outsider (Milken, MRF)
> basically
> > > > came
> > > > > in and did things very differently and now even experts in the
> field seem
> > > > > to credit them with positive change. There's also a good
> explanation for
> > > > > why this might be the case: while there are a ton of people and
> dollars,
> > > > > they largely seem to play by one
> self-reinforcing/network-effect-prone
> > > > set
> > > > > of rules, implying high returns to disrupting that equilibrium.
> > > > > - So, we've always wanted to get into this area. There was a period
> > > >
> > > > > where I was presenting meta-research as our best entry point into
> this
> > > > > field: my vision was that we would talk to academics about what
> > > > systematic
> > > > > failings there were and what funding opportunities these implied,
> and
> > > > that
> > > > > would be as good a way as any to get acclimated in biomed. But this
> > > > period
> > > > > wasn't particularly long - the May blog post on Labs priority
> causes
> > > > lists
> > > > > scientific research as a promising area distinct from
> meta-research.
> > > > > - I set up the call with Dario without having a clear idea of
> whether I
> > > >
> > > > > wanted to approach biomed from a "meta-research" angle or another
> angle.
> > > > > After the call and other investigations described here, I got a
> clearer
> > > > > idea of what I think is the best path forward.
> > > > >
> > > > > Bottom line - I think it's important to build an understanding of
> > > > > biomedical research, and that we should take the best path to
> doing so
> > > > > whether or not that dovetails with the meta-research work (likely
> it will
> > > > > dovetail some but not 100%).
> > > >
> > > >
> > > >
> > >
> >
> >
> >
> >
> >
>
>
>
> ------------------------------------
>
> This is the research mailing list of GiveWell (www.givewell.net).  Emails
> sent over this list represent the informal thoughts and notes of staff
> members and other participants.  They do NOT represent official positions
> of GiveWell.Yahoo! Groups Links
>
>
>
>

sorry, realise a sentence wasn't finished, amended below

On 15 October 2012 12:43, Susheela Peres da Costa <
susheela.peresdacosta@...> wrote:

> Aubrey, I would love to hear more about two things you raise as worthwhile
> questions.
>
> 1) The distinction between longevity on the one hand and the postponement
> of age-related ill health on the other (I understand the different
> definitions, but am interested in thoughts on the likely implications of
> each for the other, in a real-world population-wide context - for instance, would
> better health in one's 70s-80s result in viewing the demise of health in
> 90s-100s as the next frontier, essentially buying a net longevity gain
> rather than a net health-costs gain?)
>
> 2) In the context of any implications for longevity, whether / why "the
> medical control of aging [is] a hugely valuable mission?" relative to other
> uses of philanthropic funds for medical research.
>
>
> On 13 October 2012 21:24, Aubrey de Grey <aubrey@...> wrote:
>
>> Hi Holden - many thanks.
>>
>> First: yes, there are really three somewhat separate questions for
>> someone trying to evaluate whether to support SENS Foundation:
>>
>> 1) Is the medical control of aging a hugely valuable mission?
>>
>> 2) Assuming "yes" to (1), is it best achieved by basic research or
>> translational research?
>>
>> 3) Assuming translational, is SENS Foundation the organisation that uses
>> money most effectively in pursuit of that mission?
>>
>> I had rather expected that you would take some convincing on item (1),
>> and much of what I wrote last time was focused on that. Since it isn't the
>> focus of your question to me, I'm now going to assume until further notice
>> that there is no dissent on that.
>>
>> So, to answer your question: actually you're not putting aside the
>> basic-vs-translational question as much as you may think you are. The word
>> "translational" is flavour of the month in government funding circles these
>> days (not only in the USA), so it's not surprising that the NIA has a
>> public statement of the kind you pointed to. However, notice that the link
>> they give "for more information" is to a page listing ALL "Funding
>> Opportunity Announcements". There is no page specifically for translational
>> ones, and the reason there isn't is that the amount of work that the NIA
>> actually funds that could really be called translational is tiny. In other
>> words, the page you found is actually just blatant spin. The neuroscience
>> slice you mention is an anomaly arising from the way NIA was founded (the
>> natural place for that money is clearly NINDS): the fact that it's NIA
>> money does not, in practice, translate into its being spent on work to
>> prevent neurodegeneration by treating its cause (aging). Instead, just like
>> NINDS money, it's spent on attacking neurodegeneration directly, as if such
>> diseases could be eliminated from the body just like an infection: the same
>> old mistake that afflicts, and dooms, the whole of geriatric medicine.
>>
>> So, the first answer to your question is that SENS Foundation really DOES
>> focus on translational research, with an explicit goal of postponing
>> age-related ill-health. But there's also another big difference: we can
>> attack this problem relatively free of the other priorities that afflict
>> mainstream funding (whether from NIH or from trasitional foundations). Most
>> importantly, though we do and will continue to publish our interim results
>> in the peer-reviewed literature, we are much less constrained by "publish
>> or perish" tyranny than typical academics are. This allows us to proceed by
>> constructing and implementing a rational "project plan" (namely SENS) to
>> get to the intended goal (the defeat of aging), whereas what little
>> translational work is funded by NIA or others is guided overwhelmingly by
>> the imperative to get some kind of positive result as quickly as possible,
>> even when it's understood that those results are not remotely likely to
>> "scale", i.e. to translate into eventual medical treatments that
>> significantly delay aging. A great example of this is the NIA's
>> Interventions Testing Program (ITP) to test the mouse longevity effects of
>> various small molecules. The ITP only exists at all (and in a far smaller
>> form than originally intended) as a result of several years of persistence
>> by the then head of the NIA's biology division (Huber Warner), and it
>> focuses entirely on delivery of simple drugs starting rather early in life,
>> with the result that no information emerges that's relevant to treating
>> people who are already in middle age or older. (This is despite the fact
>> that by far the most high-profile result that the ITP has delivered so far,
>> the benefits of rapamycin, actually WAS a late-onset study: it wasn't meant
>> to be, but technical issues delayed the experiment.) In a nutshell, there
>> is a huge bias against high-risk high-gain work.
>>
>> The third thing that distinguishes SENS Foundation's approach is that we
>> can transcend the "balkanisation" (silo mentality) that dominates
>> mainstream academic funding. When one submits a grant application to NIA,
>> it is evaluated by gerontologists, just as when one submits to NCI it is
>> evaluated by oncologists, etc. What's wrong with this is that it biases the
>> system immensely against cross-disciplinary proposals. SENS is a plan that
>> brings together a large body of knowledge from gerontology but also a huge
>> amount of expertise that was developed for other reasons entirely - to
>> treat acute disease/injury, or in some cases for purposes that were not
>> biomedical at all (notably environmental decontamination). It doesn't
>> matter how robust the objective scientific and technological argument is
>> for work of that sort: it will never compete (especially in today's very
>> tight funding environment) with more single-topic proposals all of whose
>> details can be understood by reviewers from a particular single field.
>>
>> The final thing to mention, and this actually also answers your question
>> to Vipul about basic versus translational research, is that SENS is a plan
>> that has stood the test of time. I've been propounding it since 2000, well
>> before SENS Foundation existed, and it used to come in for a lot of
>> criticism (initially more in the form of off-the-record ridicule, and
>> latterly, at my behest, in print), but in every single case that criticism
>> was found to stem from ignorance on the part of the detractor, either of
>> what I proposed or of published experimental work on which the proposal was
>> based. That's why I'm now regularly asked to organise entire sessions at
>> mainstream gerontology conferences, whereas as little as five years ago I
>> would never even be invited to speak. It's also why the Research Advisory
>> Board of SENS Foundation consists of such prestigious scientists. This is a
>> very strong argument, in my view, for believing that now is the time to
>> sink a proper amount of money into translational gerontology (though
>> certainly not to cease doin basic biogerontology too). It's well known that
>> basic scientists are often not the most far-sighted when it comes to seeing
>> how to apply their discoveries (attitudes in 1900 to the feasibility of
>> powered flight being the canonical example). It is therefore a source of
>> concern that almost all the experts who have the ear of funders in this
>> field are basic scientists, whose instinct is to carry on finding things
>> out and to deprioritise the tedious business of applying that knowledge.
>> SENS has achieved a gratisfying level of legitimacy in gerontology, but it
>> is still foreign to most card-carrying gerontologists, and as such it
>> remains essentially unfundable via mainstream mechanisms. Hence the need to
>> create a philanthropy-driven entity, SENS Foundation, to get this work done.
>>
>> Let me know if this helps, or if you have further questions.
>>
>> Cheers, Aubrey
>>
>> On 12 Oct 2012, at 15:28, Holden Karnofsky wrote:
>>
>> >
>> > Hi Aubrey,
>> >
>> > Thanks for the thoughts.
>> >
>> > The NIH appears to have a division focused on research relevant to this
>> topic: http://www.nia.nih.gov/research/dab . Its budget appears to be
>> ~$175 million (per year). The National Institute on Aging, which houses
>> this division, has a budget of about $1 billion per year, including a
>> separate ~$400 million for neuroscience (which may also be relevant) as
>> well as $115 million for intramural research. Figures are from
>> http://www.nia.nih.gov/about/budget/2012/fiscal-year-2013-budget. The
>> Institute states that its mandate includes translational research (
>> http://www.nia.nih.gov/research/faq/does-nia-support-translational-research).
>> How would you distinguish your work from this work?
>> >
>> > (For the moment I'm putting aside the question I raised in my previous
>> response to Vipul on this topic, regarding whether it's best to approach
>> biology funding from the perspective of "trying to treat/cure a particular
>> condition" or "trying to understand  fundamental questions in biology whose
>> applications are difficult to predict.")
>> >
>> > Best,
>> > Holden
>> >
>> > On Fri, Oct 12, 2012 at 7:32 AM, aubrey.degrey <aubrey@...> wrote:
>> >
>> > Hi everyone,
>> >
>> > My attention was brought to this thread, by virtue of the fact that it
>> was my work that gave rise to SENS Foundation, and I'm looking forward to
>> getting more involved here; I've held the Effective Altruism movement in
>> high regard for some time. However, given my newbie status here I want to
>> start by apologising in advance for any oversight of previously-discussed
>> issues etc. I'm naturally delighted both at Holden's post and at Vipul's
>> reply (which I should stress that I did not plant! - I do not know Vipul at
>> all, though I look forward to changing that). I would like to mention just
>> a few key points for discussion:
>> >
>> > - Holden, I want to compliment you on your appreciation of how academia
>> really works. Everything you say about that is spot on. The aversion to
>> "high risk high gain" work that has arisen and become so endemic in the
>> system is the most important point here, in terms of why parallel funding
>> routes are needed.
>> >
>> > - I'm slightly confused that a lot of Holden's remarks are focused on
>> the private sector (i.e. startups), since my understanding was that
>> GiveWell is about philanthropy; but I realise that there is not all that
>> clear a boundary between the two (and I note the mention of Breakout Labs,
>> with which I have close links and which sits astride that divide more than
>> arguably anyone). The "valley of death" in pre-competitive translational
>> research is a rather different one than that encountered by startups, but
>> the principle is the same, and research to postpone aging certainly
>> encounteres it.
>> >
>> > - Something that I presume factors highly among GiveWell's criteria is
>> the extent to which a cause may be undervalued by the bulk of major
>> philanthropists, such that an infusion of additional funds would make more
>> of a difference than in an area that is already being well funded. To me
>> this seems to mirror the logic of focusing on the shortcomings (gaps) in
>> NIH's funding (and that of traditional-model foundations). Holden notes
>> that "Anyone we consider for funding ought to be able to explain why
>> they're better at allocating the funds than the NIH" and I agree
>> wholeheartedly, but my inference is that he thinks that some orgs may
>> indeed be able to explain that. I certainly think that SENS Foundation can.
>> >
>> > - Coming to aging: research to postpone aging has the unique problem of
>> quite indescribeable irrationality on the part of most of the general
>> public, policy-makers and even biologists with regard to its desirability.
>> Biogerontologists have been talking to brick walls for decades in their
>> effort to get the rest of the world to appreciate that aging is what causes
>> age-related ill-health, and thus that treatments for aging are merely
>> preventative geriatrics. The concept persists, despite biogerontologists'
>> best efforts, that aging is "natural" and should be left alone, whereas the
>> diseases that it brings about are awful and should be fought. This is made
>> even more bizarre by the fact that the status of age-related diseases as
>> aspects of the later stages of aging absolutely, unequivocally implies that
>> efforts to attack those diseases directly are doomed to fail. As such, this
>> is a (unique? certainly very rare) case where a philanthropic contribution
>> can make a particularly big difference simply because most philanthropists
>> don't see the case for it. It underpins why having an interest in treating
>> aging, as opposed to cancer, absolutely has a major impact on which
>> projects one funds. It's also a case for (if I understand the term
>> correctly) meta-research.
>> >
>> > - A lot of the chatter about treating aging revolves around longevity,
>> but it shouldn't. I'm all in favour of longevity, don't get me wrong, but
>> it's not what gets me up in the morning: what does is health. I want people
>> to be truly youthful, however long ago they were born: simple as that. The
>> benefits of longevity per se to humanity may also be substantial, in the
>> form of greater wisdom etc, but that would necessarily come about only very
>> gradually (we won't have any 1000-year-old for at least 900 years whatever
>> happens!), so it doesn't figure strongly in my calculations.
>> >
>> > - When forced to acknowledge that the idea of aging being a
>> high-priority target for medicine is an inescapeable consequence of things
>> they already believe (notably that health is good and ageism is bad), many
>> people retreat to the standpoint that it's never going to be possible so
>> it's OK to be irrational about whether it's desirable. The feasibility of
>> postponing age-related ill-health by X years with medicine available Y
>> years from now is, of course, a matter of speculation on which experts
>> disagree, just as with any other pioneering technology. I know that Holden
>> and others have expressed caution (at best) concerning the accuracy of any
>> kind of calculation of probabilities of particular outcomes in the distant
>> (or even not-so-distant) future, and I share that view. However, an
>> approach that may appeal more is to estimate how much humanitarian benefit
>> a given amount of progress would deliver, and then to ask how unlikely that
>> scenario needs to be to make it not worth pursuing. My claim is that the
>> benefits of hastening the defeat of aging by even a few years (which is the
>> minimum that I claim SENS Foundation is in a position to do, given adequate
>> funding) would be so astronomical that the required chance of success to
>> make such an effort worthwhile would be tiny - too tiny for it to be
>> reasonable to argue that such funding would be inadvisable. But of course
>> that is precisely what I would want GiveWell to opine on.
>> >
>> > - In the event that GiveWell (or anyone else) were to decide and
>> declare that the defeat of aging is indeed a cause that philanthropists
>> should support, there then arises the question of which organisation(s)
>> should be supported in the best interests of that mission. We at SENS
>> Foundation have worked diligently to rise as quickly as possible in the
>> legitimacy stakes by all standard measures, but we are still young and
>> there remains more to do. If I were to offer an argument to fund us rather
>> than any other entity, it would largely come down to the fact that no other
>> organisation has even a serious plan for defeating aging, let alone a track
>> record of implementing such a plan's early stages.
>> >
>> > - A significant chunk of what we do is of a kind that I think comes
>> under "meta-research". A prominent example is a project we're funding at
>> Denver University to extend the well-respected forecasting system
>> "International Futures" so that it can analyse scenarios incorporating
>> dramatically postponed aging.
>> >
>> > I greatly welcome any feedback.
>> >
>> > Cheers, Aubrey
>> >
>> >
>> >
>> > --- In givewell@yahoogroups.com, Holden Karnofsky <holden0@...> wrote:
>> > >
>> > > Hi Vipul,
>> > >
>> > > Thanks for the thoughts. I had a followup conversation with Dario
>> about
>> > > this topic a few days ago. I think the question of "could one fund
>> > > translational research to treat/prevent aging?" provides an
>> interesting
>> > > illustration of some of the tricky dynamics here for a funder:
>> > >
>> > > - It's possible that if there were a great deal more attention giving
>> to
>> >
>> > > treating/preventing aging, we would have some promising treatments.
>> So in a
>> > > broad sense it's possible that aging is underinvested in.
>> > > - A lot of the best basic biology research isn't clearly pointing
>> toward
>> >
>> > > one treatment/condition or another; it's about understanding the
>> > > fundamentals of how organisms operate. So having an interest in
>> treating
>> > > aging, as opposed to cancer, might not have a major impact on which
>> > > projects one funds, if one's main goal is to fund outstanding basic
>> biology
>> > > research.
>> > > - Perhaps because of the lack of emphasis on treating aging (or
>> perhaps
>> >
>> > > because it's simply too difficult of a problem), there don't seem to
>> be
>> > > promising findings in the "Valley of Death" relevant to aging; the few
>> > > promising leads have been explored.
>> > > - So even if, in a broad sense, there is too little attention given to
>> >
>> > > this problem, knowing this doesn't necessarily yield a clear
>> direction for
>> > > a relatively small-scale funder of biomedical research.
>> > >
>> > > Best,
>> > > Holden
>> > >
>> > > On Wed, Oct 10, 2012 at 11:55 PM, Vipul Naik <vipul@...>wrote:
>> > >
>> > > > **
>> >
>> > > >
>> > > >
>> > > > Holden,
>> > > >
>> > > > First, I think that this is an excellent document. I checked for a
>> > > > number of things that I had heard about (Breakout Labs, John
>> > > > Ioannidis, Cochrane Collaboration) and they're all there in your
>> > > > document.
>> > > >
>> > > > The one thing that's not explicitly mentioned: longevity and life
>> > > > extension research. At least prima facie, this seems like something
>> > > > that should be more important than individual disease research, and
>> it
>> > > > seems like a classic "Valley of Death" case (pun unintended, but
>> > > > noted) -- T1 stage to use your terminology. I think the SENS website
>> > > > http://www.sens.org would be a good starting point for one of the
>> (to
>> > > > me promising) approaches to life extension. I recall from past
>> > > > conversations that you were aware of SENS, so this is not new to
>> you,
>> > > > but I think that longevity should be included as part of any
>> > > > discussion of biomedical research and given separate consideration
>> > > > given that it has a much lower status than research into specific
>> > > > conditions such as cancer, dementia, etc. You may ultimately
>> conclude
>> > > > that not enough can be done in this area, but I think it should be
>> > > > part of your preliminary stuff. [btw, the United States has a
>> National
>> > > > Institute of Aging, but it's much lower-status than most of the
>> other
>> > > > grantmakers mentioned here].
>> > > >
>> > > > Vipul
>> > > >
>> > > > * Quoting Holden Karnofsky who at 2012-10-10 23:08:42+0000 (Wed)
>> wrote
>> > > >
>> > > > > Note to the research list: we're now considering reopening our
>> > > > > investigation of the world of biomedical research. We've started
>> and
>> > > > > stopped a couple of times in this area before; this time I
>> decided to
>> > > > start
>> > > > > with a conversation with Dario Amodei, a longtime GiveWell
>> follower and
>> > > > > personal friend who is currently a biology postdoc at Stanford.
>> My goal
>> > > > > with the conversation was just to get some basic context and start
>> > > > putting
>> > > > > together a framework for thinking about the issue, not to use him
>> as an
>> > > > > authoritative source, and the notes below should be read in that
>> spirit.
>> > > > >
>> > > > >
>> > > > > This email has two sections:
>> > > > >
>> > > > >
>> > > > > 1. Notes that I emailed out internally after my conversation with
>> Dario,
>> > > > > slightly edited
>> > > > >
>> > > > > 2. Some more context on the history of our work on biomedical
>> research
>> > > > and
>> > > > > why we think it's appropriate to investigate this field (this was
>> a
>> > > > > response to a question following my original email, along the
>> lines of
>> > > > "Why
>> > > > > are you looking into biomedical research now, given that's an
>> area with a
>> > > > > lot of buzz and funding from wealthy donors, and how does this
>> work
>> > > > relate
>> > > > > to the 'meta-research' work?")
>> > > > >
>> > > > >
>> > > > >
>> > > > > *--*
>> > > > >
>> > > > > *
>> > > > > *
>> > > > >
>> > > > > *1. Notes that I emailed out internally after my conversation
>> with Dario,
>> > > > > slightly edited*
>> > > > >
>> > > > > *
>> > > >
>> > > > > *
>> > > > >
>> > > > > I've done some preliminary work trying to figure out what it
>> would look
>> > > > > like to explore biomedical sciences as funding area. This mostly
>> > > > consisted
>> > > > > of a 3-hour conversation with Dario (recording is available),
>> reading two
>> > > > > papers he sent and a few I found while Googling, and prior
>> knowledge. I'm
>> > > > > including Dario in all emails related to this stuff, as an
>> informal
>> > > > advisor.
>> > > > >
>> > > > > * *
>> > > > >
>> > > > > *My picture of "what the biomedical research world roughly looks
>> like"
>> > > > *(this
>> > > >
>> > > > > is mostly from talking with Dario + prior knowledge)
>> > > > >
>> > > > > · *Academic biology* studies how organisms work and develops
>> tools to
>> > > >
>> > > > > observe and manipulate the building blocks of organisms.
>> > > > >
>> > > > > o The vast bulk of the funding - and the most prestigious funding
>> - comes
>> > > > > from the NIH.
>> > > > >
>> > > > > o There is also funding from what I've heard called "foundations"
>> -
>> > > > groups
>> > > > > like the American Cancer Society and American Heart Association -
>> which
>> > > > > function very similarly to the NIH, in that they tend to hire
>> people with
>> > > > > strong academic credentials and those people judge the merits of
>> grant
>> > > > > proposals.
>> > > > >
>> > > > > o Both the NIH and "foundations" tend to be formally partitioned
>> by
>> > > > > disease, but much of the work done by academic biologists is
>> potentially
>> > > > > relevant to many diseases. A researcher seeking NIH funding may
>> apply to
>> > > > > several different NIH "study sections," though only one at a time
>> (a list
>> > > > > of "study sections" is at
>> > > > >
>> http://public.csr.nih.gov/StudySections/Standing/Pages/default.aspx);
>> > > > > whether s/he gets funding is going to depend more on the academic
>> merits
>> > > > of
>> > > > > the researcher & work than anything else.
>> > > > >
>> > > > > o There are basic definitions of "academic merit " that generally
>> shape
>> > > > > the whole ecosystem: the people handing out money are selected by
>> these
>> > > > > criteria and use these criteria, the people doing the research
>> know that
>> > > > > these criteria shape their career prospects, etc.
>> > > > >
>> > > > > · *Private startups* investigate promising ideas for new
>> > > >
>> > > > > treatments/diagnostics/devices. They may often take the form of a
>> biology
>> > > > > professor spinning off a biotech startup (run by former postdocs)
>> that
>> > > > > raises venture capital, based on the research the professor did.
>> They
>> > > > take
>> > > > > basic knowledge about how the body works (for example, protein X
>> is
>> > > > crucial
>> > > > > for medical condition Y) and do the necessary testing to find a
>> promising
>> > > > > treatment/diagnostic/device (for example, testing a lot of
>> compounds on
>> > > > > animals until they find one that affects protein X).
>> > > > >
>> > > > > · *Big pharma/biotech companies *are best positioned to deal with
>> the
>> > > >
>> > > > > extremely expensive process of conducting clinical trials and
>> getting FDA
>> > > > > approval. Acquisition by one of these is the most common form of
>> exit for
>> > > > > startups.
>> > > > >
>> > > > > · *Academic medicine, epidemiology and other fields* also do work
>> > > >
>> > > > > relevant to medicine, including studying questions whose main
>> relevance
>> > > > is
>> > > > > to medical practice and public health programs: how effective is
>> > > > > treatment/practice X in situation Y, how cost-effective is it,
>> etc.
>> > > > > Sometimes they will hit on commercializable insights (for
>> example, a new
>> > > > > kind of device) as well.
>> > > > >
>> > > > > · *Translational research* is a broad term referring to a bridge
>> between
>> > > >
>> > > > > academic research and treatments/practices. It can include (
>> > > > >
>> > > >
>> http://medicalcenter.osu.edu/research/translational_research/pages/index.aspx
>> > > > > ):
>> > > > >
>> > > > > o T1 - going from basic science (usually academic biology, I
>> presume) to
>> > > > a
>> > > > > new treatment/practice. This includes research that helps go from
>> an
>> > > > > academic biology insight to a private startup.
>> > > > >
>> > > > > o T2 - I think this is basically what Cochrane does - going from
>> academic
>> > > > > medicine/epidemiology (a bunch of studies on what
>> treatments/practices
>> > > > are
>> > > > > effective) to the development of guidelines that actually affect
>> > > > practice.
>> > > > >
>> > > > > o T3, T4 - research on how to actually change practice (as
>> opposed to
>> > > > > setting the guidelines that are a "target" for practice) and get
>> better
>> > > > > real-world results.
>> > > > >
>> > > > > *
>> > > > > *
>> > > > >
>> > > > > *Potential "big opportunity to do good" #1: translational
>> research and
>> > > > the
>> > > >
>> > > > > "Valley of Death" (this comes from prior knowledge, googling
>> "Valley of
>> > > > > Death" and reading a bunch of the stuff I found, and checking out
>> the
>> > > > > FasterCures website again)*
>> > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > The "Valley of Death" seems to mostly refer to a shortage of T1.
>> The idea
>> > > > > is that there are a lot of cases where there's an academic
>> insight that's
>> > > > > potentially valuable in coming up with a new treatment, but to
>> get it to
>> > > > > the point where it's attractive from a for-profit perspective,
>> you need
>> > > > to
>> > > > > do a lot of stuff that academics don't have a reason to do. "For
>> example,
>> > > > > an upstream finding that a given protein is differentially
>> expressed in
>> > > > > individuals with a particular disease may suggest that the
>> protein merits
>> > > > > further investigation. However, much more work (especially
>> medicinal
>> > > > > chemistry) is necessary to determine how good a target the
>> protein really
>> > > > > is and whether a marketable drug candidate that affects the
>> activity of
>> > > > the
>> > > > > protein is likely to be developed." (
>> > > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf)
>> > > > >
>> > > > >
>> > > > >
>> > > > > There are claims that this sort of work is massively underfunded
>> (by the
>> > > > > people we've spoken to who talked about the "Valley of Death";
>> also in
>> > > > >
>> > > >
>> http://www.fastercures.org/documents/file/Valley%20of%20Death%20-%20Translational%20Research
>> > > > (1).pdf).
>> > > > > However, funding isn't the only issue. The other issue is that
>> > > > > "pharmaceutical firms that hold libraries of potentially useful
>> small
>> > > > > molecules as trade secrets, making them largely off limits to ...
>> > > > academic
>> > > > > scientists" (
>> > > > https://www.law.upenn.edu/institutes/ctic/papers/200708/Rai.pdf).
>> > > > > In other words, there is some room for new models of
>> collaboration that
>> > > > > lead to better communication and information sharing between
>> academia and
>> > > > > industry (or between industry and industry).
>> > > > >
>> > > > >
>> > > > > Both the Myelin Repair Foundation and Michael Milken's work on
>> prostate
>> > > > > cancer have been pointed to as examples of innovative
>> collaborations that
>> > > > > deal with some of the information sharing problems. Milken's
>> model:
>> > > > > "drastically cutting the wait time for grant money, to flood the
>> field
>> > > > with
>> > > > > fast cash, to fund therapy-driven ideas rather than basic
>> science, to
>> > > > hold
>> > > > > researchers he funds accountable for results, and to demand
>> collaboration
>> > > > > across disciplines and among institutions, private industry, and
>> > > > academia."
>> > > > > (
>> > > > >
>> > > >
>> http://money.cnn.com/magazines/fortune/fortune_archive/2004/11/29/8192713/index.htm
>> > > > )
>> > > > > Myelin Repair Foundation sounds broadly similar (
>> > > > >
>> > > >
>> http://opinionator.blogs.nytimes.com/2011/05/02/helping-new-drugs-out-of-academias-valley-of-death/
>> > > > ).
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > More broadly:
>> > > > >
>> > > > > · FasterCures (also a Michael Milken production) looks like it's
>> focused
>> > > > > on the broad mission of "more research with a shorter timeline to
>> > > > > treatments," with a heavy sub-focus on the Valley of Death. In
>> addition
>> > > > to
>> > > > > its conference and philanthropic advisory service, it advocates
>> for FDA
>> > > > > improvements (presumably to speed the approval process),
>> advocates for
>> > > > the
>> > > > > NIH to put more funding into translational research (there have
>> > > > definitely
>> > > > > been a lot of new initiatives at the NIH focused on this stuff in
>> the
>> > > > past
>> > > > > few ~decade), promotes "innovative financing mechanisms" for
>> bridging the
>> > > > > Valley of Death (these include Peter Thiel's Breakout Labs; I
>> think they
>> > > > > occupy a conceptually similar space to "social enterprise
>> investment"
>> > > > > though they tend to be structured more as grants and less as
>> > > > > double-bottom-line investments), and works on getting patient
>> data opened
>> > > > > to researchers. The only program of theirs I haven't mentioned is
>> TRAIN;
>> > > > I
>> > > > > can't (easily) figure out what this is.
>> > > > >
>> > > > > · John Ioannidis stated to us that all translational research is
>> > > > > underfunded, not just T1. (The context we talked to him in, of
>> course,
>> > > > was
>> > > > > T2.)
>> > > > >
>> > > > >
>> > > > > These issues seem to have quite a bit of buzz. There are some
>> really
>> > > > stark
>> > > > > #'s out there: even as R&D investment has gone way up over the
>> past 50
>> > > > > years, the # of new drugs has stayed roughly constant at around
>> 20 a
>> > > > year.
>> > > > >
>> > > > >
>> > > > > Dario sent a really interesting paper on this topic. It argues:
>> (a) the #
>> > > > > of NMEs (new medical entities, i.e., drugs) per year has mostly
>> remained
>> > > > > constant-with-noise (no trend); (b) all of the big companies seem
>> to have
>> > > > > produced NMEs at a very steady pace, even as they've changed
>> size, though
>> > > > > different companies do have different rates of NME creation; (c)
>> when it
>> > > > > comes to mergers & acquisitions, "1+1=1": companies that acquire
>> other
>> > > > > companies just keep up the same NME pace; (d) over time, the # of
>> large
>> > > > > companies has shrunk (due to mergers) and the # of small
>> companies has
>> > > > > risen, and the share of NMEs attributed to small companies has
>> gone from
>> > > > > ~30% to ~70%; (e) the cost per NME has gone up over time and is
>> now in
>> > > > the
>> > > > > neighborhood of $1 billion; (f) the % of NMEs that are
>> "blockbusters"
>> > > > (huge
>> > > > > profits) has been steady around 20%, despite intensifying efforts
>> on the
>> > > > > part of pharma to fund only potential blockbusters.
>> > > > >
>> > > > >
>> > > > >
>> > > > > There are also claims - such as by Derek Lowe, a blogger Dario
>> pointed me
>> > > > > to - that the "Valley of Death" is an overrated concept: there is
>> a high
>> > > > > ratio of academic discoveries to useful drugs, but this is just
>> because a
>> > > > > lot of stuff turns out not to work, not because we have a
>> shortage of
>> > > > > doable translational research. The paper above sort of takes this
>> view,
>> > > > > essentially arguing that nothing seems to raise NME production;
>> we'll
>> > > > need
>> > > > > something really radical to make any difference.
>> > > > >
>> > > > >
>> > > > >
>> > > > > The case studies in the FasterCures paper on this are
>> interesting. They
>> > > > > imply that there were some pretty low-hanging fruit in the T1
>> domain. It
>> > > > > might be interesting to talk to Fastercures and see whether they
>> can help
>> > > > > identify "Valley of Death" opportunities that are slam-dunkish.
>> > > > >
>> > > > >
>> > > > > *Potential "big opportunity to do good" #2: inefficiencies in
>> academic
>> > > > > biology*
>> > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > All of the above seems pretty distinct from the question of how
>> to make
>> > > > > academic biology better at doing its job of understanding the
>> body. This
>> > > > is
>> > > > > the question that Dario and I focused on.
>> > > > >
>> > > > >
>> > > > >
>> > > > > Dario painted a picture in which most of academia plays by the
>> same set
>> > > > of
>> > > > > rules, making it very hard for people to do things that break
>> those rules
>> > > > > (for example, academics are expected to publish a lot; it's risky
>> to work
>> > > > > on a "blue sky" or highly ambitious project). Thus, for anyone
>> who wants
>> > > > a
>> > > > > career in academia, a couple of years working on a moonshot
>> project are
>> > > > > risky; in addition, being on bad terms with a small number of
>> people can
>> > > > > damage a career (since there is often a small set of people that
>> makes a
>> > > > > large proportion of the career-impacting decisions for a given
>> area, and
>> > > > it
>> > > > > can be hard to escape this set of people without changing research
>> > > > > interests significantly). The bad news is that this isn't
>> particularly
>> > > > easy
>> > > > > to fix: you can offer funding for blue-sky projects, but (a) a
>> lot of
>> > > > > academics basically train themselves to play by the rules, and
>> won't
>> > > > > necessarily have thought about "what should be studied if these
>> rules
>> > > > > didn't apply," and more importantly, (b) there are many incentives
>> > > > pushing
>> > > > > academics towards playing by certain rules; funding is only one
>> of those
>> > > > > mechanisms (there's also tenure, peer review, etc) so changing
>> that one
>> > > > > incentive won't always change behavior. Dario says that he might
>> hesitate
>> > > > > to work on a particular blue-sky project that he thinks is
>> interesting,
>> > > > > even if he got funding for it, for these reasons.
>> > > > >
>> > > > >
>> > > > > Despite this, there are some funders who push the boundaries.
>> There are
>> > > > > medical centers that don't require teaching and do more ambitious
>> work.
>> > > > > There's the McKnight foundation, which funded some of the
>> pioneering work
>> > > > > on optical control of neurons for which funding might have been
>> difficult
>> > > > > to obtain by traditional routes. There's the HHMI Janelia Farms
>> campus,
>> > > > > which Dario thinks is the most promising thing out there in terms
>> of a
>> > > > > model - academics who go there get guaranteed (and generous)
>> funding for
>> > > > 6
>> > > > > years, which frees them up to take much bigger risks. (I saw a
>> study
>> > > > > claiming that the work coming out of Janelia Farms has a much
>> more skewed
>> > > > > citation count distribution, implying bigger risks - i.e., fewer
>> papers
>> > > > > with decent #'s of citations but more "blockbuster" papers with
>> lots of
>> > > > > citations). But Janelia Farms is only in one sub-field; comparable
>> > > > > institutions don't exist for other fields (as far as Dario
>> knows)..
>> > > > >
>> > > > >
>> > > > >
>> > > > > Dario's gut is that one of the best things a funder could do
>> would be to
>> > > > > work toward creating a large institute that largely "plays by its
>> own
>> > > > > rules," encouraging more ambitious work and providing enough
>> security and
>> > > > > sheer volume of dollars/researchers as to establish a sort of
>> "parallel
>> > > > > system" to academia - thus becoming a place that could provide
>> viable and
>> > > > > reliable career options for people interested in playing by
>> different
>> > > > > rules. I'm also interested in the idea of trying to advocate for
>> changes
>> > > > in
>> > > > > the rules, as we've discussed in the context of meta-research.
>> > > > >
>> > > > >
>> > > > >
>> > > > > There's also the other stuff like data/code sharing.
>> > > > >
>> > > > >
>> > > > >
>> > > > > *Another option: give $ to the NIH*
>> > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > The NIH is by far the largest, most prestigious, most respected
>> funder in
>> > > > > academic biology, and most of academia runs on criteria that
>> mirror the
>> > > > > NIH's. I would guess that an unrestricted check to the NIH would
>> get
>> > > > > allocated in a pretty sector-agnostic way. It seems like this is
>> a giving
>> > > > > option that is pretty nontrivial to beat. Anyone we consider for
>> funding
>> > > > > ought to be able to explain why they're better at allocating the
>> funds
>> > > > than
>> > > > > the NIH. We haven't talked to the NIH about whether it would
>> accept these
>> > > > > sorts of donations.
>> > > > >
>> > > > >
>> > > > > With the context I now have, FasterCures strikes me as pretty
>> > > > > promising/interesting too, though their "room for more funding"
>> situation
>> > > > > isn't clear (w/o talking to them). But they also seem
>> sector-agnostic
>> > > > > within biomedical research, while pushing a very specific theory
>> of
>> > > > change
>> > > > > that may or may not be valid.
>> > > > >
>> > > > >
>> > > > >
>> > > > > *Next steps*
>> > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > I think understanding this world would be a major undertaking. I
>> think
>> > > > we'd
>> > > > > have to be ready to put in a lot of work and to be open to ways of
>> > > > bridging
>> > > > > "funder-expert gap" that we haven't tried before. For example:
>> > > > >
>> > > > > · Dario suggested that I read a freshman biology textbook. He
>> said he
>> > > > > realizes that the time cost could be very large - something like
>> 100
>> > > > hours
>> > > > > - but that without doing so, I'm going to be lacking too much
>> context on
>> > > > > why biomedical research works the way it does. He thinks that
>> reading a
>> > > > > basic textbook would get me to the point of strongly diminishing
>> marginal
>> > > > > returns.
>> > > > >
>> > > > > · Also in the category of "personally picking up rudimentary
>> subject
>> > > > > matter knowledge," I thought it might be worth trying to follow
>> the
>> > > > > development of a particular drug from start to finish - Gleevec
>> (the
>> > > > > "miracle drug") would be a good candidate. The goal would be to
>> > > > understand
>> > > > > each stage of insight leading to new investigation, and where the
>> funding
>> > > > > came from at each stage.
>> > > > >
>> > > > > · I think hiring Dario or someone like him would make a lot of
>> sense.
>> > > > > I've thought about whether we should be hiring "subject matter
>> experts"
>> > > > in
>> > > > > other areas, such as global health, but in my view the need is
>> clearer
>> > > > here
>> > > > > than in any other area. One of the things I don't love about
>> hiring an
>> > > > > expert in a given field, at this stage of our research, is that
>> we could
>> > > > > quickly decide that we're just not interested enough in the field
>> in
>> > > > > question ... but someone with the right kind of technical
>> knowledge &
>> > > > > experience would be so far ahead of us in evaluating *any* area of
>> > > > biology
>> > > > > research that it seems like a good idea. (JTBC, I'm also actively
>> > > > thinking
>> > > > > about whether it would make sense to hire experts in other fields
>> ...)
>> > > > >
>> > > > > · Talking to major funders and potentially co-funding with them is
>> > > > > probably essential. Important groups to talk to would include NIH
>> (by far
>> > > > > the most important; we've already talked to them a bit), the
>> colloquium
>> > > > of
>> > > > > groups like the American Cancer Society (it has a name; I forget
>> the
>> > > > name),
>> > > > > FasterCures, Wellcome, and potentially some funders with
>> > > > > unorthodox/buzzed-about approaches: Myelin Repair Foundation,
>> McKnight
>> > > > > Foundation, and maybe another private foundation or two (I
>> emailed Dario
>> > > > > all my notes on major foundations that do biomedical research so
>> he could
>> > > > > send thoughts on whether any look interesting aside from
>> McKnight).
>> > > > >
>> > > > >
>> > > > > My next plan had been to talk to academics (Dario had good
>> suggestions
>> > > > > about how to approach them), but with all the potential work to
>> be done
>> > > > on
>> > > > > gaining basic context, I'm not sure that's the right next step.
>> But it's
>> > > > > also a possible step.
>> > > > >
>> > > > >
>> > > > >
>> > > > > *2. Some more context on the history of our work on biomedical
>> research
>> > > > and
>> > > >
>> > > > > why we think it's appropriate to investigate this field (this was
>> a
>> > > > > response to a question following my original email, along the
>> lines of
>> > > > "Why
>> > > > > are you looking into biomedical research now, given that's an
>> area with a
>> > > > > lot of buzz and funding from wealthy donors, and how does this
>> work
>> > > > relate
>> > > > > to the 'meta-research' work?")*
>> > > > >
>> > > > >
>> > > > > - We've always had "disease research" as a cause we wanted to
>> > > >
>> > > > > investigate. We've never had a good reason for not investigating
>> it other
>> > > > > than that investigating it seems difficult. We've tried more than
>> once to
>> > > > > investigate it, and it's ended up falling by the wayside because
>> of how
>> > > > big
>> > > > > an undertaking it is.
>> > > > > - It's true that there is a huge amount of funding and buzz in
>> this
>> > > >
>> > > > > area. But it also seems quite possible that there isn't nearly
>> enough; in
>> > > > > fact this seems like a possible point of consensus between all
>> the people
>> > > > > concerned about the "Valley of Death." This potential good
>> accomplished
>> > > > via
>> > > > > biomedical research appears unlimited. What I consider to be the
>> mark of
>> > > > a
>> > > > > bad sector is "tons of buzz/$ and mediocre returns," but the
>> latter part
>> > > > > doesn't apply to biomed; in fact biomedical research is arguably
>> one of
>> > > > > philanthropy's and even humanity's top success stories.
>> > > > > - There are also indications that despite all the buzz and
>> funding,
>> > > >
>> > > > > there are still opportunities to do things differently and shake
>> things
>> > > > up.
>> > > > > There is more than one case where an outsider (Milken, MRF)
>> basically
>> > > > came
>> > > > > in and did things very differently and now even experts in the
>> field seem
>> > > > > to credit them with positive change. There's also a good
>> explanation for
>> > > > > why this might be the case: while there are a ton of people and
>> dollars,
>> > > > > they largely seem to play by one
>> self-reinforcing/network-effect-prone
>> > > > set
>> > > > > of rules, implying high returns to disrupting that equilibrium.
>> > > > > - So, we've always wanted to get into this area. There was a
>> period
>> > > >
>> > > > > where I was presenting meta-research as our best entry point into
>> this
>> > > > > field: my vision was that we would talk to academics about what
>> > > > systematic
>> > > > > failings there were and what funding opportunities these implied,
>> and
>> > > > that
>> > > > > would be as good a way as any to get acclimated in biomed. But
>> this
>> > > > period
>> > > > > wasn't particularly long - the May blog post on Labs priority
>> causes
>> > > > lists
>> > > > > scientific research as a promising area distinct from
>> meta-research.
>> > > > > - I set up the call with Dario without having a clear idea of
>> whether I
>> > > >
>> > > > > wanted to approach biomed from a "meta-research" angle or another
>> angle.
>> > > > > After the call and other investigations described here, I got a
>> clearer
>> > > > > idea of what I think is the best path forward.
>> > > > >
>> > > > > Bottom line - I think it's important to build an understanding of
>> > > > > biomedical research, and that we should take the best path to
>> doing so
>> > > > > whether or not that dovetails with the meta-research work (likely
>> it will
>> > > > > dovetail some but not 100%).
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>> ------------------------------------
>>
>> This is the research mailing list of GiveWell (www.givewell.net).
>>  Emails sent over this list represent the informal thoughts and notes of
>> staff members and other participants.  They do NOT represent official
>> positions of GiveWell.Yahoo! Groups Links
>>
>>
>>
>>
>

Hello Susheela,

Yes, I expected that some people here would be interested in discussing these topics, even if Holden't focus is currently more on whom best to fund to get the job done.

Your first question is one that gives rise to a big outreach dilemma, as it happens. A major reason why the SENS approach (turning back the aging clock with regenerative medicine, rather than slowing the clock down) is so attractive is that it buys so much time to figure out what to do next, i.e. how to make the therapies a notch better so that they can address what you term the next frontier. If we can rejuvenate 60-year-olds well enough that they won't be biologically 60 again until they're chronologically 90 - which is what I believe the "first-generation" SENS therapies will deliver - then we have all those 30 years to develop second-generation therapies that deliver a further 30 years of postponement, and so on. From what we know about the nature of the molecular and cellular damage that these therapies will repair, we can say with great confidence (or so I claim!) that these further improvements will overwhelmingly be in the form of relatively minor extensions of/variations on the initial therapies, which implies that it is vanishingly unlikely that progress will be so slow that we fail to stay one step ahead of the problem. (I have termed the necessary rate of improvement of the therapies "longevity escape velocity".) So, what this means is that for practical purposes we can regard the initial SENS therapies as delivering, with high probability, an indefinite postponement of age-related ill-health. The outreach dilemma is that objectively this is great, since the gain would absolutely be in terms of health costs as well as healthy longevity, but emotionally it ignites all kinds of knee-jerk reactions concerning the nature of a post-aging world, such as overpopulation risks, pensions crises, dictators living forever etc etc, which in my experience are powerful disincentives to supporting this work despite the acknowledgement that (a) health is good and (b) old people are people too, which in my view is pretty much all one needs to acknowledge in order to want aging to be eliminated.

On your second question, for sure there are issues here that cause some people to hesitate. Ultimately the question breaks down (I believe) into the following categories:

Positive:
1 how valuable is the anticipated health gain over the period when someone would otherwise be in impaired health?
2 how valuable is the anticipated additional [all healthy] healthy longevity gain?
3 how valuable are the anticipated gains in quality of life resulting indirectly from expecting to live a lot longer?

Negative:
4 how valuable are the lives that might not exist if we curtailed the birth rate more than we otherwise might?
5 how valuable are the aspects of today's life that might be compromised by the expectation of much longer lives?

My own view is that (1) and 2) are definitely as big as any other humanitarian mission, that (3) could be huge too (for example we might try a lot harder to combat climate change, end war etc), that (4) is unclear even if we regard future people as important, because we're curtailing the birth rate so fast anyway, and that (5) is a figment of Luddite imagination. But others may disagree! In particular, there seems to be a fairly widespread notion that (chronologically) old people really are less deserving of medical care than the young - and I refer here to intrinsic merits, independent of today's problem that it's in practice harder to give the elderly as much additional healthy life as the young. Whether this is a legitimate view or one that we must work against in the same way that people worked against the acceptance of slavery is a matter for debate.

Cheers, Aubrey

On 15 Oct 2012, at 04:05, Susheela Peres da Costa wrote:

> sorry, realise a sentence wasn't finished, amended below
> 
> On 15 October 2012 12:43, Susheela Peres da Costa <susheela.peresdacosta@...> wrote:
> Aubrey, I would love to hear more about two things you raise as worthwhile questions.
>  
> 1) The distinction between longevity on the one hand and the postponement of age-related ill health on the other (I understand the different definitions, but am interested in thoughts on the likely implications of each for the other, in a real-world population-wide context - for instance, would better health in one's 70s-80s result in viewing the demise of health in 90s-100s as the next frontier, essentially buying a net longevity gain rather than a net health-costs gain?)
>  
> 2) In the context of any implications for longevity, whether / why "the medical control of aging [is] a hugely valuable mission?" relative to other uses of philanthropic funds for medical research. 

Hi Aubrey,

Thanks for the detailed answer to Holden's question about the differences
between what the NIA is doing and what the SENS Foundation is doing.

If you have a moment, I have a follow-up question: can you give a quick
summary of the main differences between what the SENS Foundation is doing
and what the Buck Institute is doing, and the difference between funding
one of those vs. the other?

Thanks,
   Brian Skinner



On Mon, Oct 15, 2012 at 11:39 AM, Aubrey de Grey <aubrey@...> wrote:

> **
>
>
> Hello Susheela,
>
> Yes, I expected that some people here would be interested in discussing
> these topics, even if Holden't focus is currently more on whom best to fund
> to get the job done.
>
> Your first question is one that gives rise to a big outreach dilemma, as
> it happens. A major reason why the SENS approach (turning back the aging
> clock with regenerative medicine, rather than slowing the clock down) is so
> attractive is that it buys so much time to figure out what to do next, i.e.
> how to make the therapies a notch better so that they can address what you
> term the next frontier. If we can rejuvenate 60-year-olds well enough that
> they won't be biologically 60 again until they're chronologically 90 -
> which is what I believe the "first-generation" SENS therapies will deliver
> - then we have all those 30 years to develop second-generation therapies
> that deliver a further 30 years of postponement, and so on. From what we
> know about the nature of the molecular and cellular damage that these
> therapies will repair, we can say with great confidence (or so I claim!)
> that these further improvements will overwhelmingly be in the form of
> relatively minor extensions of/variations on the initial therapies, which
> implies that it is vanishingly unlikely that progress will be so slow that
> we fail to stay one step ahead of the problem. (I have termed the necessary
> rate of improvement of the therapies "longevity escape velocity".) So, what
> this means is that for practical purposes we can regard the initial SENS
> therapies as delivering, with high probability, an indefinite postponement
> of age-related ill-health. The outreach dilemma is that objectively this is
> great, since the gain would absolutely be in terms of health costs as well
> as healthy longevity, but emotionally it ignites all kinds of knee-jerk
> reactions concerning the nature of a post-aging world, such as
> overpopulation risks, pensions crises, dictators living forever etc etc,
> which in my experience are powerful disincentives to supporting this work
> despite the acknowledgement that (a) health is good and (b) old people are
> people too, which in my view is pretty much all o ne needs to acknowledge
> in order to want aging to be eliminated.
>
> On your second question, for sure there are issues here that cause some
> people to hesitate. Ultimately the question breaks down (I believe) into
> the following categories:
>
> Positive:
> 1 how valuable is the anticipated health gain over the period when someone
> would otherwise be in impaired health?
> 2 how valuable is the anticipated additional [all healthy] healthy
> longevity gain?
> 3 how valuable are the anticipated gains in quality of life resulting
> indirectly from expecting to live a lot longer?
>
> Negative:
> 4 how valuable are the lives that might not exist if we curtailed the
> birth rate more than we otherwise might?
> 5 how valuable are the aspects of today's life that might be compromised
> by the expectation of much longer lives?
>
> My own view is that (1) and 2) are definitely as big as any other
> humanitarian mission, that (3) could be huge too (for example we might try
> a lot harder to combat climate change, end war etc), that (4) is unclear
> even if we regard future people as important, because we're curtailing the
> birth rate so fast anyway, and that (5) is a figment of Luddite
> imagination. But others may disagree! In particular, there seems to be a
> fairly widespread notion that (chronologically) old people really are less
> deserving of medical care than the young - and I refer here to intrinsic
> merits, independent of today's problem that it's in practice harder to give
> the elderly as much additional healthy life as the young. Whether this is a
> legitimate view or one that we must work against in the same way that
> people worked against the acceptance of slavery is a matter for debate.
>
> Cheers, Aubrey
>
>
> On 15 Oct 2012, at 04:05, Susheela Peres da Costa wrote:
>
> > sorry, realise a sentence wasn't finished, amended below
> >
> > On 15 October 2012 12:43, Susheela Peres da Costa <
> susheela.peresdacosta@...> wrote:
> > Aubrey, I would love to hear more about two things you raise as
> worthwhile questions.
> >
> > 1) The distinction between longevity on the one hand and the
> postponement of age-related ill health on the other (I understand the
> different definitions, but am interested in thoughts on the likely
> implications of each for the other, in a real-world population-wide context
> - for instance, would better health in one's 70s-80s result in viewing the
> demise of health in 90s-100s as the next frontier, essentially buying a net
> longevity gain rather than a net health-costs gain?)
> >
> > 2) In the context of any implications for longevity, whether / why "the
> medical control of aging [is] a hugely valuable mission?" relative to other
> uses of philanthropic funds for medical research.
>
>  
>

Hello Brian,

Thanks for this question. In brief, the main differences between SENS Foundation and the Buck are quite similar to those between SENS Foundation and the NIA, because their funding sources and mechanisms are largely the same. The Buck was founded with a substantial endowment, but that is only a minor source of its scientists' research funds: most of it is from the same sources that university labs use, namely the NIA and other public or philanthropic sources (such as the California Institute of Regenerative Medicine or the Ellison Medical Foundation) that evaluate grant applications by methods that have the various shortcomings I described earlier. The Buck scientists produce excellent work, no question, but ultimately their research priorities are constrained to be no different than other conventionally-funded scientists in universities. In particular, they can only pay lip service to the idea of truly translational research that takes us closer to bringing human aging under genuine medical control.

Cheers, Aubrey

On 30 Oct 2012, at 00:45, Brian Douglas Skinner wrote:

> Hi Aubrey,
> 
> 
> Thanks for the detailed answer to Holden's question about the differences between what the NIA is doing and what the SENS Foundation is doing. 
> 
> If you have a moment, I have a follow-up question: can you give a quick summary of the main differences between what the SENS Foundation is doing and what the Buck Institute is doing, and the difference between funding one of those vs. the other?
> 
> Thanks,
>    Brian Skinner
> 
> 
> 
> On Mon, Oct 15, 2012 at 11:39 AM, Aubrey de Grey <aubrey@...> wrote:
>  
> Hello Susheela,
> 
> Yes, I expected that some people here would be interested in discussing these topics, even if Holden't focus is currently more on whom best to fund to get the job done.
> 
> Your first question is one that gives rise to a big outreach dilemma, as it happens. A major reason why the SENS approach (turning back the aging clock with regenerative medicine, rather than slowing the clock down) is so attractive is that it buys so much time to figure out what to do next, i.e. how to make the therapies a notch better so that they can address what you term the next frontier. If we can rejuvenate 60-year-olds well enough that they won't be biologically 60 again until they're chronologically 90 - which is what I believe the "first-generation" SENS therapies will deliver - then we have all those 30 years to develop second-generation therapies that deliver a further 30 years of postponement, and so on. From what we know about the nature of the molecular and cellular damage that these therapies will repair, we can say with great confidence (or so I claim!) that these further improvements will overwhelmingly be in the form of relatively minor extensions of/variations on the initial therapies, which implies that it is vanishingly unlikely that progress will be so slow that we fail to stay one step ahead of the problem. (I have termed the necessary rate of improvement of the therapies "longevity escape velocity".) So, what this means is that for practical purposes we can regard the initial SENS therapies as delivering, with high probability, an indefinite postponement of age-related ill-health. The outreach dilemma is that objectively this is great, since the gain would absolutely be in terms of health costs as well as healthy longevity, but emotionally it ignites all kinds of knee-jerk reactions concerning the nature of a post-aging world, such as overpopulation risks, pensions crises, dictators living forever etc etc, which in my experience are powerful disincentives to supporting this work despite the acknowledgement that (a) health is good and (b) old people are people too, which in my view is pretty much all o ne needs to acknowledge in order to want aging to be eliminated.
> 
> On your second question, for sure there are issues here that cause some people to hesitate. Ultimately the question breaks down (I believe) into the following categories:
> 
> Positive:
> 1 how valuable is the anticipated health gain over the period when someone would otherwise be in impaired health?
> 2 how valuable is the anticipated additional [all healthy] healthy longevity gain?
> 3 how valuable are the anticipated gains in quality of life resulting indirectly from expecting to live a lot longer?
> 
> Negative:
> 4 how valuable are the lives that might not exist if we curtailed the birth rate more than we otherwise might?
> 5 how valuable are the aspects of today's life that might be compromised by the expectation of much longer lives?
> 
> My own view is that (1) and 2) are definitely as big as any other humanitarian mission, that (3) could be huge too (for example we might try a lot harder to combat climate change, end war etc), that (4) is unclear even if we regard future people as important, because we're curtailing the birth rate so fast anyway, and that (5) is a figment of Luddite imagination. But others may disagree! In particular, there seems to be a fairly widespread notion that (chronologically) old people really are less deserving of medical care than the young - and I refer here to intrinsic merits, independent of today's problem that it's in practice harder to give the elderly as much additional healthy life as the young. Whether this is a legitimate view or one that we must work against in the same way that people worked against the acceptance of slavery is a matter for debate.
> 
> Cheers, Aubrey
> 
> 
> 
> On 15 Oct 2012, at 04:05, Susheela Peres da Costa wrote:
> 
> > sorry, realise a sentence wasn't finished, amended below
> > 
> > On 15 October 2012 12:43, Susheela Peres da Costa <susheela.peresdacosta@...> wrote:
> > Aubrey, I would love to hear more about two things you raise as worthwhile questions.
> > 
> > 1) The distinction between longevity on the one hand and the postponement of age-related ill health on the other (I understand the different definitions, but am interested in thoughts on the likely implications of each for the other, in a real-world population-wide context - for instance, would better health in one's 70s-80s result in viewing the demise of health in 90s-100s as the next frontier, essentially buying a net longevity gain rather than a net health-costs gain?)
> > 
> > 2) In the context of any implications for longevity, whether / why "the medical control of aging [is] a hugely valuable mission?" relative to other uses of philanthropic funds for medical research. 
> 
> 
> 
> 
> 

Hi Aubrey,

Thanks for the quick response. That gives me a better understanding for
where the Buck Institute fits in the landscape of organizations.

Best,
   Brian


On Tue, Oct 30, 2012 at 2:20 AM, Aubrey de Grey <aubrey@sens.org> wrote:

> Hello Brian,
>
> Thanks for this question. In brief, the main differences between SENS
> Foundation and the Buck are quite similar to those between SENS Foundation
> and the NIA, because their funding sources and mechanisms are largely the
> same. The Buck was founded with a substantial endowment, but that is only a
> minor source of its scientists' research funds: most of it is from the same
> sources that university labs use, namely the NIA and other public or
> philanthropic sources (such as the California Institute of Regenerative
> Medicine or the Ellison Medical Foundation) that evaluate grant
> applications by methods that have the various shortcomings I described
> earlier. The Buck scientists produce excellent work, no question, but
> ultimately their research priorities are constrained to be no different
> than other conventionally-funded scientists in universities. In particular,
> they can only pay lip service to the idea of truly translational research
> that takes us closer to bringing human aging under genuine medical control.
>
> Cheers, Aubrey
>
> On 30 Oct 2012, at 00:45, Brian Douglas Skinner wrote:
>
> > Hi Aubrey,
> >
> >
> > Thanks for the detailed answer to Holden's question about the
> differences between what the NIA is doing and what the SENS Foundation is
> doing.
> >
> > If you have a moment, I have a follow-up question: can you give a quick
> summary of the main differences between what the SENS Foundation is doing
> and what the Buck Institute is doing, and the difference between funding
> one of those vs. the other?
> >
> > Thanks,
> >    Brian Skinner
> >
> >
> >
> > On Mon, Oct 15, 2012 at 11:39 AM, Aubrey de Grey <aubrey@...>
> wrote:
> >
> > Hello Susheela,
> >
> > Yes, I expected that some people here would be interested in discussing
> these topics, even if Holden't focus is currently more on whom best to fund
> to get the job done.
> >
> > Your first question is one that gives rise to a big outreach dilemma, as
> it happens. A major reason why the SENS approach (turning back the aging
> clock with regenerative medicine, rather than slowing the clock down) is so
> attractive is that it buys so much time to figure out what to do next, i.e.
> how to make the therapies a notch better so that they can address what you
> term the next frontier. If we can rejuvenate 60-year-olds well enough that
> they won't be biologically 60 again until they're chronologically 90 -
> which is what I believe the "first-generation" SENS therapies will deliver
> - then we have all those 30 years to develop second-generation therapies
> that deliver a further 30 years of postponement, and so on. From what we
> know about the nature of the molecular and cellular damage that these
> therapies will repair, we can say with great confidence (or so I claim!)
> that these further improvements will overwhelmingly be in the form of
> relatively minor extensions of/variations on the initial therapies, which
> implies that it is vanishingly unlikely that progress will be so slow that
> we fail to stay one step ahead of the problem. (I have termed the necessary
> rate of improvement of the therapies "longevity escape velocity".) So, what
> this means is that for practical purposes we can regard the initial SENS
> therapies as delivering, with high probability, an indefinite postponement
> of age-related ill-health. The outreach dilemma is that objectively this is
> great, since the gain would absolutely be in terms of health costs as well
> as healthy longevity, but emotionally it ignites all kinds of knee-jerk
> reactions concerning the nature of a post-aging world, such as
> overpopulation risks, pensions crises, dictators living forever etc etc,
> which in my experience are powerful disincentives to supporting this work
> despite the acknowledgement that (a) health is good and (b) old people are
> people too, which in my view is pretty much all o ne needs to acknowledge
> in order to want aging to be eliminated.
> >
> > On your second question, for sure there are issues here that cause some
> people to hesitate. Ultimately the question breaks down (I believe) into
> the following categories:
> >
> > Positive:
> > 1 how valuable is the anticipated health gain over the period when
> someone would otherwise be in impaired health?
> > 2 how valuable is the anticipated additional [all healthy] healthy
> longevity gain?
> > 3 how valuable are the anticipated gains in quality of life resulting
> indirectly from expecting to live a lot longer?
> >
> > Negative:
> > 4 how valuable are the lives that might not exist if we curtailed the
> birth rate more than we otherwise might?
> > 5 how valuable are the aspects of today's life that might be compromised
> by the expectation of much longer lives?
> >
> > My own view is that (1) and 2) are definitely as big as any other
> humanitarian mission, that (3) could be huge too (for example we might try
> a lot harder to combat climate change, end war etc), that (4) is unclear
> even if we regard future people as important, because we're curtailing the
> birth rate so fast anyway, and that (5) is a figment of Luddite
> imagination. But others may disagree! In particular, there seems to be a
> fairly widespread notion that (chronologically) old people really are less
> deserving of medical care than the young - and I refer here to intrinsic
> merits, independent of today's problem that it's in practice harder to give
> the elderly as much additional healthy life as the young. Whether this is a
> legitimate view or one that we must work against in the same way that
> people worked against the acceptance of slavery is a matter for debate.
> >
> > Cheers, Aubrey
> >
> >
> >
> > On 15 Oct 2012, at 04:05, Susheela Peres da Costa wrote:
> >
> > > sorry, realise a sentence wasn't finished, amended below
> > >
> > > On 15 October 2012 12:43, Susheela Peres da Costa <
> susheela.peresdacosta@...> wrote:
> > > Aubrey, I would love to hear more about two things you raise as
> worthwhile questions.
> > >
> > > 1) The distinction between longevity on the one hand and the
> postponement of age-related ill health on the other (I understand the
> different definitions, but am interested in thoughts on the likely
> implications of each for the other, in a real-world population-wide context
> - for instance, would better health in one's 70s-80s result in viewing the
> demise of health in 90s-100s as the next frontier, essentially buying a net
> longevity gain rather than a net health-costs gain?)
> > >
> > > 2) In the context of any implications for longevity, whether / why
> "the medical control of aging [is] a hugely valuable mission?" relative to
> other uses of philanthropic funds for medical research.
> >
> >
> >
> >
> > 
>
>

In GiveWell's recent update about SCI, they
write<http://www.givewell.org/international/top-charities/schistosomiasis-control-initiative/updates/November-2012>
:

In one report, SCI stated that it had spent $27,000 on the DRC program on
> laboratory material.... In its spending report for the first eight months
> of 2012, SCI reported that it had spent £1,329 (about $2,100) in DRC....
> When we asked SCI to clarify, ...SCI replied, "a typo – there should have
> been a 7 – $27,100".... However, the figure SCI had provided was in pounds,
> which we converted to dollars, so a missing ‘7’ could not account for the
> discrepancy.
>

I found this particular mistake rather concerning -- it made me wonder SCI
was being intentionally dishonest or unusually careless. Either way, it's
the kind of thing that would make me very concerned about naming it a top
charity until I got to the bottom of it.

The risks strike me as asymmetric here. If it was an honest mistake by SCI
and GiveWell didn't recommend them, then some money would instead be given
to charities that GiveWell expects to be either slightly more or less
effective than SCI. But if there is a serious underlying problem with SCI,
and GiveWell does recommend them, it'd be a major scandal. Scandals receive
orders of magnitude more press than normal events, which means such a
scandal could deal a huge blow to GiveWell's reputation, and possibly the
reputation of optimal philanthropy in general.

Given my prior, I suppose the most likely explanation is still that SCI is
a somewhat-shambolic operation (an impression reinforced by the other
mistakes you identified) and they carelessly confabulated an explanation
for the discrepancy. But the combination of errors and communication issues
is also a pattern commonly caused by people who are trying to cover
something up. As a donor, I'd like to see GiveWell put some resources into
figuring out which is the case before recommending them.

Perhaps the person who gave the figure in £s had received it in $s from
someone else, and this latter person had made the typo?

--

Please excuse brevity; this message has been sent from my mobile phone.

Holly Morgan | Executive Director | The Life You Can Save
thelifeyoucansave.com | holly.morgan@ thelifeyoucansave.com
+44 7910 094 247 | Skype: hollyrebeccamorgan

The Life You Can Save is part of the Centre for Effective Altruism, a
charitable company limited by guarantee registered in England and Wales,
number 7962181.

Centre for Effective Altruism, Oxford Uehiro Centre for Practical Ethics,
Littlegate House, St Ebbes Street, Oxford OX1 1PT, UK
On 14 Nov 2012 19:47, "Aaron Swartz" <me@...> wrote:

> **
>
>
> In GiveWell's recent update about SCI, they write<http://www.givewell.org/international/top-charities/schistosomiasis-control-initiative/updates/November-2012>
> :
>
> In one report, SCI stated that it had spent $27,000 on the DRC program on
>> laboratory material.... In its spending report for the first eight months
>> of 2012, SCI reported that it had spent £1,329 (about $2,100) in DRC....
>> When we asked SCI to clarify, ...SCI replied, "a typo – there should have
>> been a 7 – $27,100".... However, the figure SCI had provided was in pounds,
>> which we converted to dollars, so a missing ‘7’ could not account for the
>> discrepancy.
>>
>
> I found this particular mistake rather concerning -- it made me wonder SCI
> was being intentionally dishonest or unusually careless. Either way, it's
> the kind of thing that would make me very concerned about naming it a top
> charity until I got to the bottom of it.
>
> The risks strike me as asymmetric here. If it was an honest mistake by SCI
> and GiveWell didn't recommend them, then some money would instead be given
> to charities that GiveWell expects to be either slightly more or less
> effective than SCI. But if there is a serious underlying problem with SCI,
> and GiveWell does recommend them, it'd be a major scandal. Scandals receive
> orders of magnitude more press than normal events, which means such a
> scandal could deal a huge blow to GiveWell's reputation, and possibly the
> reputation of optimal philanthropy in general.
>
> Given my prior, I suppose the most likely explanation is still that SCI is
> a somewhat-shambolic operation (an impression reinforced by the other
> mistakes you identified) and they carelessly confabulated an explanation
> for the discrepancy. But the combination of errors and communication issues
> is also a pattern commonly caused by people who are trying to cover
> something up. As a donor, I'd like to see GiveWell put some resources into
> figuring out which is the case before recommending them.
>  
>

I realize this mailing list isn't for sharing content from subscribers, but I thought this article published by GOOD Magazine on Monday is especially relevant to the discussions this group has around cash transfers. New Incentives is a fascinating approach connecting donors to conditional cash transfers (the most rigorously evaluated development tool, as you all well know) that is definitely worth paying attention to alongside GiveWell's recommendation and discussion of GiveDirectly and unconditional cash transfers.

http://www.good.is/posts/if-microfinancing-creates-a-cycle-of-debt-is-handing-out-cash-a-better-option

I had wondered if there is reluctance to communicate with GiveWell  
because GiveWell may have developed a reputation for being harsh on  
charities.  I'm not saying they shouldn't be, but I wonder if there is  
an evitable natural tension given the nature of what GiveWell does  
that creates problems.



Quoting Holly Morgan <hollyrebeccamorgan@...>:

> Perhaps the person who gave the figure in £s had received it in $s from
> someone else, and this latter person had made the typo?
>
> --
>
> Please excuse brevity; this message has been sent from my mobile phone.
>
> Holly Morgan | Executive Director | The Life You Can Save
> thelifeyoucansave.com | holly.morgan@ thelifeyoucansave.com
> +44 7910 094 247 | Skype: hollyrebeccamorgan
>
> The Life You Can Save is part of the Centre for Effective Altruism, a
> charitable company limited by guarantee registered in England and Wales,
> number 7962181.
>
> Centre for Effective Altruism, Oxford Uehiro Centre for Practical Ethics,
> Littlegate House, St Ebbes Street, Oxford OX1 1PT, UK
> On 14 Nov 2012 19:47, "Aaron Swartz" <me@...> wrote:
>
>> **
>>
>>
>> In GiveWell's recent update about SCI, they  
>> write<http://www.givewell.org/international/top-charities/schistosomiasis-control-initiative/updates/November-2012>
>> :
>>
>> In one report, SCI stated that it had spent $27,000 on the DRC program on
>>> laboratory material.... In its spending report for the first eight months
>>> of 2012, SCI reported that it had spent £1,329 (about $2,100) in DRC....
>>> When we asked SCI to clarify, ...SCI replied, "a typo – there should have
>>> been a 7 – $27,100".... However, the figure SCI had provided was in pounds,
>>> which we converted to dollars, so a missing ‘7’ could not account for the
>>> discrepancy.
>>>
>>
>> I found this particular mistake rather concerning -- it made me wonder SCI
>> was being intentionally dishonest or unusually careless. Either way, it's
>> the kind of thing that would make me very concerned about naming it a top
>> charity until I got to the bottom of it.
>>
>> The risks strike me as asymmetric here. If it was an honest mistake by SCI
>> and GiveWell didn't recommend them, then some money would instead be given
>> to charities that GiveWell expects to be either slightly more or less
>> effective than SCI. But if there is a serious underlying problem with SCI,
>> and GiveWell does recommend them, it'd be a major scandal. Scandals receive
>> orders of magnitude more press than normal events, which means such a
>> scandal could deal a huge blow to GiveWell's reputation, and possibly the
>> reputation of optimal philanthropy in general.
>>
>> Given my prior, I suppose the most likely explanation is still that SCI is
>> a somewhat-shambolic operation (an impression reinforced by the other
>> mistakes you identified) and they carelessly confabulated an explanation
>> for the discrepancy. But the combination of errors and communication issues
>> is also a pattern commonly caused by people who are trying to cover
>> something up. As a donor, I'd like to see GiveWell put some resources into
>> figuring out which is the case before recommending them.
>>
>>
>



Ronald Noble, Ph.D.
University of Pennsylvania

A similar thing occurs with public companies in immature markets becoming
very emotional about stockbrokers, often completely shutting them out after
a 'sell' recommendation. If the general level of competance in charities is
much lower (most are a 'sell'), we only want to buy a couple, and charities
aren't used to external evaluation, it's easy to see how they could have a
similar response.

NB: I have no reason to suppose this applies to SCI, who seemed quite fond
of GiveWell when I spoke to Alan.

On Wed, Nov 14, 2012 at 6:14 PM, <rnoble@...> wrote:

> **
>
>
>
>
> I had wondered if there is reluctance to communicate with GiveWell
> because GiveWell may have developed a reputation for being harsh on
> charities. I'm not saying they shouldn't be, but I wonder if there is
> an evitable natural tension given the nature of what GiveWell does
> that creates problems.
>
>
> Quoting Holly Morgan <hollyrebeccamorgan@...>:
>
> > Perhaps the person who gave the figure in £s had received it in $s from
> > someone else, and this latter person had made the typo?
> >
> > --
> >
> > Please excuse brevity; this message has been sent from my mobile phone.
> >
> > Holly Morgan | Executive Director | The Life You Can Save
> > thelifeyoucansave.com | holly.morgan@ thelifeyoucansave.com
> > +44 7910 094 247 | Skype: hollyrebeccamorgan
> >
> > The Life You Can Save is part of the Centre for Effective Altruism, a
> > charitable company limited by guarantee registered in England and Wales,
> > number 7962181.
> >
> > Centre for Effective Altruism, Oxford Uehiro Centre for Practical Ethics,
> > Littlegate House, St Ebbes Street, Oxford OX1 1PT, UK
> > On 14 Nov 2012 19:47, "Aaron Swartz" <me@aaronsw.com> wrote:
> >
> >> **
>
> >>
> >>
> >> In GiveWell's recent update about SCI, they
> >> write<
> http://www.givewell.org/international/top-charities/schistosomiasis-control-initiative/updates/November-2012
> >
>
> >> :
> >>
> >> In one report, SCI stated that it had spent $27,000 on the DRC program
> on
> >>> laboratory material.... In its spending report for the first eight
> months
> >>> of 2012, SCI reported that it had spent £1,329 (about $2,100) in
> DRC....
> >>> When we asked SCI to clarify, ...SCI replied, "a typo – there should
> have
> >>> been a 7 – $27,100".... However, the figure SCI had provided was in
> pounds,
> >>> which we converted to dollars, so a missing ‘7’ could not account for
> the
> >>> discrepancy.
> >>>
> >>
> >> I found this particular mistake rather concerning -- it made me wonder
> SCI
> >> was being intentionally dishonest or unusually careless. Either way,
> it's
> >> the kind of thing that would make me very concerned about naming it a
> top
> >> charity until I got to the bottom of it.
> >>
> >> The risks strike me as asymmetric here. If it was an honest mistake by
> SCI
> >> and GiveWell didn't recommend them, then some money would instead be
> given
> >> to charities that GiveWell expects to be either slightly more or less
> >> effective than SCI. But if there is a serious underlying problem with
> SCI,
> >> and GiveWell does recommend them, it'd be a major scandal. Scandals
> receive
> >> orders of magnitude more press than normal events, which means such a
> >> scandal could deal a huge blow to GiveWell's reputation, and possibly
> the
> >> reputation of optimal philanthropy in general.
> >>
> >> Given my prior, I suppose the most likely explanation is still that SCI
> is
> >> a somewhat-shambolic operation (an impression reinforced by the other
> >> mistakes you identified) and they carelessly confabulated an explanation
> >> for the discrepancy. But the combination of errors and communication
> issues
> >> is also a pattern commonly caused by people who are trying to cover
> >> something up. As a donor, I'd like to see GiveWell put some resources
> into
> >> figuring out which is the case before recommending them.
> >>
> >>
> >
>
> Ronald Noble, Ph.D.
> University of Pennsylvania
>
>  
>

Hi Holly,

SCI provided the spending figure to us in pounds (£1,329), and we converted
these to the dollar figure that we then asked SCI about ($2,100). The
document they sent us is
http://www.givewell.org/files/DWDA%202009/SCI/SCI%20expenditures%20Jan%20-%20Aug%202012.xls
(as
noted in the document, we added lines 21 and 23 after later communication
with SCI).

The spending figure for DRC did turn out to be close to $27,000, but we do
not believe that the reason we received the £1,329 figure at first was due
to a typo. SCI later told us about a second outlay of funds to DRC that
brought the total up to £16,701.

Since SCI is a London-based charity, I think it is unlikely that they
converted from pounds to dollars back to pounds before reporting the figure
to us.

Best,
Natalie

On Wed, Nov 14, 2012 at 2:21 PM, Holly Morgan
<hollyrebeccamorgan@...>wrote:

> **
>
>
> Perhaps the person who gave the figure in £s had received it in $s from
> someone else, and this latter person had made the typo?
>
> --
>
> Please excuse brevity; this message has been sent from my mobile phone.
>
> Holly Morgan | Executive Director | The Life You Can Save
> thelifeyoucansave.com | holly.morgan@ thelifeyoucansave.com
> +44 7910 094 247 | Skype: hollyrebeccamorgan
>
> The Life You Can Save is part of the Centre for Effective Altruism, a
> charitable company limited by guarantee registered in England and Wales,
> number 7962181.
>
> Centre for Effective Altruism, Oxford Uehiro Centre for Practical Ethics,
> Littlegate House, St Ebbes Street, Oxford OX1 1PT, UK
> On 14 Nov 2012 19:47, "Aaron Swartz" <me@...> wrote:
>
>> **
>>
>>
>> In GiveWell's recent update about SCI, they write<http://www.givewell.org/international/top-charities/schistosomiasis-control-initiative/updates/November-2012>
>> :
>>
>>  In one report, SCI stated that it had spent $27,000 on the DRC program
>>> on laboratory material.... In its spending report for the first eight
>>> months of 2012, SCI reported that it had spent £1,329 (about $2,100) in
>>> DRC.... When we asked SCI to clarify, ...SCI replied, "a typo – there
>>> should have been a 7 – $27,100".... However, the figure SCI had provided
>>> was in pounds, which we converted to dollars, so a missing ‘7’ could not
>>> account for the discrepancy.
>>>
>>
>> I found this particular mistake rather concerning -- it made me wonder
>> SCI was being intentionally dishonest or unusually careless. Either way,
>> it's the kind of thing that would make me very concerned about naming it a
>> top charity until I got to the bottom of it.
>>
>> The risks strike me as asymmetric here. If it was an honest mistake by
>> SCI and GiveWell didn't recommend them, then some money would instead be
>> given to charities that GiveWell expects to be either slightly more or less
>> effective than SCI. But if there is a serious underlying problem with SCI,
>> and GiveWell does recommend them, it'd be a major scandal. Scandals receive
>> orders of magnitude more press than normal events, which means such a
>> scandal could deal a huge blow to GiveWell's reputation, and possibly the
>> reputation of optimal philanthropy in general.
>>
>> Given my prior, I suppose the most likely explanation is still that SCI
>> is a somewhat-shambolic operation (an impression reinforced by the other
>> mistakes you identified) and they carelessly confabulated an explanation
>> for the discrepancy. But the combination of errors and communication issues
>> is also a pattern commonly caused by people who are trying to cover
>> something up. As a donor, I'd like to see GiveWell put some resources into
>> figuring out which is the case before recommending them.
>>
>>    
>

Ah fair enough, thanks Natalie

--

Please excuse brevity; this message has been sent from my mobile phone.

Holly Morgan | Executive Director | The Life You Can Save
thelifeyoucansave.com | holly.morgan@ thelifeyoucansave.com
+44 7910 094 247 | Skype: hollyrebeccamorgan

The Life You Can Save is part of the Centre for Effective Altruism, a
charitable company limited by guarantee registered in England and Wales,
number 7962181.

Centre for Effective Altruism, Oxford Uehiro Centre for Practical Ethics,
Littlegate House, St Ebbes Street, Oxford OX1 1PT, UK
On 15 Nov 2012 20:05, "Natalie Stone Crispin" <natalie@...> wrote:

> **
>
>
> Hi Holly,
>
> SCI provided the spending figure to us in pounds (£1,329), and we
> converted these to the dollar figure that we then asked SCI about ($2,100).
> The document they sent us is
> http://www.givewell.org/files/DWDA%202009/SCI/SCI%20expenditures%20Jan%20-%20Aug%202012.xls (as
> noted in the document, we added lines 21 and 23 after later communication
> with SCI).
>
> The spending figure for DRC did turn out to be close to $27,000, but we do
> not believe that the reason we received the £1,329 figure at first was due
> to a typo. SCI later told us about a second outlay of funds to DRC that
> brought the total up to £16,701.
>
> Since SCI is a London-based charity, I think it is unlikely that they
> converted from pounds to dollars back to pounds before reporting the figure
> to us.
>
> Best,
> Natalie
>
> On Wed, Nov 14, 2012 at 2:21 PM, Holly Morgan <
> hollyrebeccamorgan@...> wrote:
>
>> **
>>
>>
>> Perhaps the person who gave the figure in £s had received it in $s from
>> someone else, and this latter person had made the typo?
>>
>> --
>>
>> Please excuse brevity; this message has been sent from my mobile phone.
>>
>> Holly Morgan | Executive Director | The Life You Can Save
>> thelifeyoucansave.com | holly.morgan@ thelifeyoucansave.com
>> +44 7910 094 247 | Skype: hollyrebeccamorgan
>>
>> The Life You Can Save is part of the Centre for Effective Altruism, a
>> charitable company limited by guarantee registered in England and Wales,
>> number 7962181.
>>
>> Centre for Effective Altruism, Oxford Uehiro Centre for Practical Ethics,
>> Littlegate House, St Ebbes Street, Oxford OX1 1PT, UK
>> On 14 Nov 2012 19:47, "Aaron Swartz" <me@...> wrote:
>>
>>> **
>>>
>>>
>>> In GiveWell's recent update about SCI, they write<http://www.givewell.org/international/top-charities/schistosomiasis-control-initiative/updates/November-2012>
>>> :
>>>
>>>  In one report, SCI stated that it had spent $27,000 on the DRC program
>>>> on laboratory material.... In its spending report for the first eight
>>>> months of 2012, SCI reported that it had spent £1,329 (about $2,100) in
>>>> DRC.... When we asked SCI to clarify, ...SCI replied, "a typo – there
>>>> should have been a 7 – $27,100".... However, the figure SCI had provided
>>>> was in pounds, which we converted to dollars, so a missing ‘7’ could not
>>>> account for the discrepancy.
>>>>
>>>
>>> I found this particular mistake rather concerning -- it made me wonder
>>> SCI was being intentionally dishonest or unusually careless. Either way,
>>> it's the kind of thing that would make me very concerned about naming it a
>>> top charity until I got to the bottom of it.
>>>
>>> The risks strike me as asymmetric here. If it was an honest mistake by
>>> SCI and GiveWell didn't recommend them, then some money would instead be
>>> given to charities that GiveWell expects to be either slightly more or less
>>> effective than SCI. But if there is a serious underlying problem with SCI,
>>> and GiveWell does recommend them, it'd be a major scandal. Scandals receive
>>> orders of magnitude more press than normal events, which means such a
>>> scandal could deal a huge blow to GiveWell's reputation, and possibly the
>>> reputation of optimal philanthropy in general.
>>>
>>> Given my prior, I suppose the most likely explanation is still that SCI
>>> is a somewhat-shambolic operation (an impression reinforced by the other
>>> mistakes you identified) and they carelessly confabulated an explanation
>>> for the discrepancy. But the combination of errors and communication issues
>>> is also a pattern commonly caused by people who are trying to cover
>>> something up. As a donor, I'd like to see GiveWell put some resources into
>>> figuring out which is the case before recommending them.
>>>
>>>
>  
>

Hi all, thanks for the thoughts. A few responses:

1. We really appreciate Aaron's raising this concern. In general, we
encourage people to raise any issues they see with the content we publish
on top charities.

2. Regarding Ron's and Ben's points, I don't think this concern applies to
SCI. SCI is one of our top-rated charities and we have communicated closely
with it throughout the year. I also don't think that "reluctance to
communicate" would be a good way to explain inaccurate communications.

(On the more general point of whether charities are reluctant to engage
with us, I think this is sometimes the case, but has become less true over
time and generally will become less true as (and if) our "money moved"
rises. I don't believe that we're missing out on opportunities to recommend
proven, cost-effective, scalable charities due to this dynamic, though I
could be wrong.)

3. We believe it is legitimate to be concerned about the incident Aaron
mentioned, but we place substantially less weight on this incident than
Aaron does. Our guess is that the person we corresponded with saw this as a
minor detail and took their best guess as to what accounted for the
discrepancy, without confirming that this was correct. While not ideal,
this is a long way from purposeful deception about core issues, something
we've never had any reason to suspect. We did ultimately nail down all of
the figures in question.

4. We don't want to be risk-averse in our charity recommendations. If we
started recommending charities with lower "expected good accomplished per
marginal dollar donated," on the basis that they had a lower chance of
embarrassing us from a PR perspective, we'd no longer be delivering on our
commitment to recommend the best giving opportunities we can (irrespective
of how these recommendations make GiveWell look, now or in the future).
That, I think, would ultimately be worse for our brand than the kind of
incident Aaron fears, given the audience we have.

Best,
Holden




On Thu, Nov 15, 2012 at 3:04 PM, Natalie Stone Crispin <natalie@...
> wrote:

> **
>
>
> Hi Holly,
>
> SCI provided the spending figure to us in pounds (£1,329), and we
> converted these to the dollar figure that we then asked SCI about ($2,100).
> The document they sent us is
> http://www.givewell.org/files/DWDA%202009/SCI/SCI%20expenditures%20Jan%20-%20Aug%202012.xls (as
> noted in the document, we added lines 21 and 23 after later communication
> with SCI).
>
> The spending figure for DRC did turn out to be close to $27,000, but we do
> not believe that the reason we received the £1,329 figure at first was due
> to a typo. SCI later told us about a second outlay of funds to DRC that
> brought the total up to £16,701.
>
> Since SCI is a London-based charity, I think it is unlikely that they
> converted from pounds to dollars back to pounds before reporting the figure
> to us.
>
> Best,
> Natalie
>
> On Wed, Nov 14, 2012 at 2:21 PM, Holly Morgan <
> hollyrebeccamorgan@...> wrote:
>
>> **
>>
>>
>> Perhaps the person who gave the figure in £s had received it in $s from
>> someone else, and this latter person had made the typo?
>>
>> --
>>
>> Please excuse brevity; this message has been sent from my mobile phone.
>>
>> Holly Morgan | Executive Director | The Life You Can Save
>> thelifeyoucansave.com | holly.morgan@ thelifeyoucansave.com
>> +44 7910 094 247 | Skype: hollyrebeccamorgan
>>
>> The Life You Can Save is part of the Centre for Effective Altruism, a
>> charitable company limited by guarantee registered in England and Wales,
>> number 7962181.
>>
>> Centre for Effective Altruism, Oxford Uehiro Centre for Practical Ethics,
>> Littlegate House, St Ebbes Street, Oxford OX1 1PT, UK
>> On 14 Nov 2012 19:47, "Aaron Swartz" <me@...> wrote:
>>
>>> **
>>>
>>>
>>> In GiveWell's recent update about SCI, they write<http://www.givewell.org/international/top-charities/schistosomiasis-control-initiative/updates/November-2012>
>>> :
>>>
>>>  In one report, SCI stated that it had spent $27,000 on the DRC program
>>>> on laboratory material.... In its spending report for the first eight
>>>> months of 2012, SCI reported that it had spent £1,329 (about $2,100) in
>>>> DRC.... When we asked SCI to clarify, ...SCI replied, "a typo – there
>>>> should have been a 7 – $27,100".... However, the figure SCI had provided
>>>> was in pounds, which we converted to dollars, so a missing ‘7’ could not
>>>> account for the discrepancy.
>>>>
>>>
>>> I found this particular mistake rather concerning -- it made me wonder
>>> SCI was being intentionally dishonest or unusually careless. Either way,
>>> it's the kind of thing that would make me very concerned about naming it a
>>> top charity until I got to the bottom of it.
>>>
>>> The risks strike me as asymmetric here. If it was an honest mistake by
>>> SCI and GiveWell didn't recommend them, then some money would instead be
>>> given to charities that GiveWell expects to be either slightly more or less
>>> effective than SCI. But if there is a serious underlying problem with SCI,
>>> and GiveWell does recommend them, it'd be a major scandal. Scandals receive
>>> orders of magnitude more press than normal events, which means such a
>>> scandal could deal a huge blow to GiveWell's reputation, and possibly the
>>> reputation of optimal philanthropy in general.
>>>
>>> Given my prior, I suppose the most likely explanation is still that SCI
>>> is a somewhat-shambolic operation (an impression reinforced by the other
>>> mistakes you identified) and they carelessly confabulated an explanation
>>> for the discrepancy. But the combination of errors and communication issues
>>> is also a pattern commonly caused by people who are trying to cover
>>> something up. As a donor, I'd like to see GiveWell put some resources into
>>> figuring out which is the case before recommending them.
>>>
>>>
>  
>

A donor recently emailed us asking about the changes in our
cost-effectiveness estimates for AMF. The donor pointed out that our
estimate had changed and whether this was the result of changes in
conditions on the ground that changed the actual cost-effectiveness of AMF,
or from the appearance of new evidence that just changed our estimate. This
is a question we think a number of people might have, so we wanted to share
our response:

The changes in our cost-effectiveness figures for AMF are due entirely to
revisions in our calculation, rather than either (a) changing conditions on
the ground; or (b) evidence that was not available at the time of our
initial review becoming available. The shift from an estimate of roughly
$1,600 per life saved to roughly $2,300 per life saved is the result of a
single change to our calculation, described in the penultimate section of
this blog post:
http://blog.givewell.org/2012/10/18/revisiting-the-case-for-insecticide-treated-nets-itns/
(see
"All-cause childhood mortality has declined; what does this mean for ITN
distribution?"). Essentially, the studies we use to estimate the mortality
benefits of bednets were conducted in the mid-1990s; since then, child
mortality has declined by about 30%. The only change to our estimate was
switching from assuming that bednets saved a certain absolute proportion of
children covered to assuming that bednets avert a certain proportion of
child mortality.

We explain this in a little more depth in the current version of our
intervention
report on bednet
distributions<http://www.givewell.org/international/technical/programs/insecticide-treated-nets#HowcosteffectiveisLLINdistribution>
:

We include a simple adjustment to account for the fact that child mortality
rates are lower today than they were at the time of the studies on ITNs.76
This adjustment assumes that ITNs avert the same proportion of under-5
deaths that they averted at the time of the studies. This could be
incorrect.

   - The deaths averted by ITNs may be the same deaths that could be
   averted by, for example, vitamin A supplementation (as noted above, many of
   the deaths averted by ITNs are not specifically attributable to malaria).
   So perhaps other improvements in general health are independently averting
   all the deaths that ITNs could avert in their absence; under this model
   it’s possible that ITNs don’t avert any deaths at all.
   - On the flipside, there may be increasing returns to improved general
   health: perhaps there are children who previously (at the time of the
   studies) would have been in such poor health that they would have died even
   with ITNs, but now can have their deaths averted by ITNs.


Best,
Alexander


-- 
Alexander Berger
Research Analyst
www.GiveWell.org

We thought this exchange about our shift away from emphasizing the
"standouts" list might be of some general interest:

Vipul Naik wrote to us, asking:

Dear GiveWell folk,

I've been thinking a bit about the change you introduced this year
regarding discontinuing the list of standout charities and
domain-specific top recommendations on your main page and
concentrating all your energies on your top three recommendations. I'm
wondering how this affects the incentives for charities to cooperate
with you in the review process.

One possibility that occurred to me is that charities will be more
likely to view cooperating with GiveWell as a high-upfront-cost,
very-low-probability-of-reward endeavor, which may make them reluctant
to cooperate with you. In the past, making it to a list of standout
charities or domain-specific top recommendations might have been
sufficient incentive to cooperate with GiveWell. With the new system,
there are few immediate payoffs for a charity unless it's convinced it
can make it to the top list.

I don't know if that's a good thing or a bad thing. On the plus side,
it means that charities that aren't really a good fit for GiveWell
would self-select out of getting reviewed. On the other hand, some
charities that hold promise for the long term but are unlikely to get
a high rating at present may be put off by the process, so you may not
get an opportunity to follow their progress from the time of inception
to the time that they may be ripe for getting a good rating.

Of course, it's possible that I am wrong about this and that
charities' incentive to cooperate is not much affected by the changes
in your recommendation style.

One small change I could think of, however, might be worth
considering. In your reviews of charities that you decide not to
recommend, you might add two tiny sections:

(1) What kind of new evidence would change your mind and make you
re-investigate the charity?

(2) With the existing evidence, are there other reasonable priors
(normative or descriptive) that people may have that make the charity
worth donating to?

The presence of these sections could help make your reviews more
useful to people who want to gain knowledge of a charity and are
already strongly considering donating to it, while requiring minimum
additional effort on your part. It also gives charities some idea of
the kind of things they'd need to do in order to convince you of their
utility. With both of these, charities have more to gain from getting
reviewed by you and linking to your reviews of them.

Vipul

We responded:

Hi Vipul,

We don't expect to see big changes in the kinds of participation we're
getting from charities as a result of this switch. Although it does
obviously decrease the number of organizations receiving significant funds
as a result of GiveWell's recommendations, we don't think it materially
impacts the expected value of a recommendation (since we don't think it
will have much impact on the total money moved), and we also doubt that
charities considering our process are doing complex expected value
calculations. More importantly, we've observed a substantial increase in
our access due to our growing influence, and we expect that to dwarf the
impact of this change. (It's also worth noting that we only *want* engagement
from promising charities, and we think that such charities are more likely
to see that they have a shot at the top spot.)

Nonetheless, we think both of your specific proposals are worth considering:

   1. For most organizations we look at, we don't see realistic short term
   steps that would lead us to a recommendation, but in the past we've tried
   to communicate privately with organizations where that has been the case
   (e.g. GiveDirectly, Cochrane) and have seen some success; we'll think about
   trying to make that feedback more public, but if we do it's likely to be
   for a small subset of organizations.
   2. Our reviews of most charities we recommend receive very little
   traffic, so we're hesitant to spend much more time on them. That said,
   we're considering trying to do more to foreground organizations that we
   haven't investigated as deeply but believe may be as good as some of our
   recommended organizations, depending on donor values or preferences.

Thanks for the question and happy holidays!

Best,
Alexander


-- 
Alexander Berger
Research Analyst
www.GiveWell.org

Hi guys,
On a related note, I'm getting ready to donate to GW's recommended
charities and have been thinking a bit about VillageReach. In your July 26
blog post<http://blog.givewell.org/2012/07/26/rethinking-villagereachs-pilot-project/>,
you stated that while your view of the effectiveness of VR's pilot project
has changed, "we continue to view VillageReach as a highly transparent
“learning organization” (capable of conducting its activities in a way that
can lead to learning). Over the past few years, VillageReach has provided
us with the source data behind its evaluations enabling us to do our own in
depth analysis and draw our own conclusions. That work has contributed to
our own growing ability to evaluate impact evaluations and determine the
level of reliance that can be placed on them. *We will be talking with
VillageReach about how more funding could contribute to more
experimentation and learning, and we will likely be interested in
recommending such funding – to encourage such outstanding transparency and
accountability, and learn more in the future*."

Later on in the post, you write:

"Groups like VillageReach are creating a new dialogue around charitable
giving, and it’s important to us that this type of behavior is supported.
We want to encourage VillageReach and other groups to share information
about how their programs are going, and we want to continue to see more
experimentation and learning. So, we are seriously considering recommending
donations to VillageReach, not despite the struggles it’s had but because
it’s had these struggles and is being honest about them."

These sentiments are very nice and make a lot of sense to me, but it
doesn't seem like the plan to recommend funding (even on a limited basis)
to reward VillageReach's honesty and experimentation ever materialized. As
Vipul notes, the laser-like focus on the top 3 charities does make it seem
like VR has effectively been "de-funded" by GiveWell. I feel that the need
to support experimentation and learning is important, and the lack of
follow-through here sends the message, whether intentionally or not, that
VR was too honest with a funder and now is suffering for it. Your thoughts?

Thanks,
Ian


On Thu, Dec 27, 2012 at 7:52 PM, Alexander Berger <alexander.is@...>wrote:

> **
>
>
> We thought this exchange about our shift away from emphasizing the
> "standouts" list might be of some general interest:
>
> Vipul Naik wrote to us, asking:
>
> Dear GiveWell folk,
>
> I've been thinking a bit about the change you introduced this year
> regarding discontinuing the list of standout charities and
> domain-specific top recommendations on your main page and
> concentrating all your energies on your top three recommendations. I'm
> wondering how this affects the incentives for charities to cooperate
> with you in the review process.
>
> One possibility that occurred to me is that charities will be more
> likely to view cooperating with GiveWell as a high-upfront-cost,
> very-low-probability-of-reward endeavor, which may make them reluctant
> to cooperate with you. In the past, making it to a list of standout
> charities or domain-specific top recommendations might have been
> sufficient incentive to cooperate with GiveWell. With the new system,
> there are few immediate payoffs for a charity unless it's convinced it
> can make it to the top list.
>
> I don't know if that's a good thing or a bad thing. On the plus side,
> it means that charities that aren't really a good fit for GiveWell
> would self-select out of getting reviewed. On the other hand, some
> charities that hold promise for the long term but are unlikely to get
> a high rating at present may be put off by the process, so you may not
> get an opportunity to follow their progress from the time of inception
> to the time that they may be ripe for getting a good rating.
>
> Of course, it's possible that I am wrong about this and that
> charities' incentive to cooperate is not much affected by the changes
> in your recommendation style.
>
> One small change I could think of, however, might be worth
> considering. In your reviews of charities that you decide not to
> recommend, you might add two tiny sections:
>
> (1) What kind of new evidence would change your mind and make you
> re-investigate the charity?
>
> (2) With the existing evidence, are there other reasonable priors
> (normative or descriptive) that people may have that make the charity
> worth donating to?
>
> The presence of these sections could help make your reviews more
> useful to people who want to gain knowledge of a charity and are
> already strongly considering donating to it, while requiring minimum
> additional effort on your part. It also gives charities some idea of
> the kind of things they'd need to do in order to convince you of their
> utility. With both of these, charities have more to gain from getting
> reviewed by you and linking to your reviews of them.
>
> Vipul
>
> We responded:
>
> Hi Vipul,
>
> We don't expect to see big changes in the kinds of participation we're
> getting from charities as a result of this switch. Although it does
> obviously decrease the number of organizations receiving significant funds
> as a result of GiveWell's recommendations, we don't think it materially
> impacts the expected value of a recommendation (since we don't think it
> will have much impact on the total money moved), and we also doubt that
> charities considering our process are doing complex expected value
> calculations. More importantly, we've observed a substantial increase in
> our access due to our growing influence, and we expect that to dwarf the
> impact of this change. (It's also worth noting that we only *want* engagement
> from promising charities, and we think that such charities are more likely
> to see that they have a shot at the top spot.)
>
> Nonetheless, we think both of your specific proposals are worth
> considering:
>
>    1. For most organizations we look at, we don't see realistic short
>    term steps that would lead us to a recommendation, but in the past we've
>    tried to communicate privately with organizations where that has been the
>    case (e.g. GiveDirectly, Cochrane) and have seen some success; we'll think
>    about trying to make that feedback more public, but if we do it's likely to
>    be for a small subset of organizations.
>    2. Our reviews of most charities we recommend receive very little
>    traffic, so we're hesitant to spend much more time on them. That said,
>    we're considering trying to do more to foreground organizations that we
>    haven't investigated as deeply but believe may be as good as some of our
>    recommended organizations, depending on donor values or preferences.
>
> Thanks for the question and happy holidays!
>
> Best,
> Alexander
>
>
> --
> Alexander Berger
> Research Analyst
> www.GiveWell.org
>
>  
>

Hi Ian,

We have continued to follow VillageReach and did consider potentially
recommending funding for them for the sake of "transparency incentives."

I don't think it's accurate to say that VillageReach is suffering because
it was too honest with a funder. The only thing VillageReach coud have done
to maintain its rating was to continue being transparent with us. If it had
withheld information, we wouldn't have recommended it. Instead, because it
continues to be transparent, we continue to publish information about it,
and some (albeit a small proportion) of our donors continue to support it.
The fact that these donors maintained their support reduces the problem
you're bringing up.

It's also worth noting that VillageReach received over $2 million as a
result of GiveWell's recommendation. I'd guess that the message most
non-profits would takeaway were they to consider VillageReach's case is
that by being transparent VillageReach gained significant funding that it
otherwise would not have. VillageReach has also been reasonably successful
raising funds for other initiatives, so there's no evidence that its
transparency has harmed it.

Best,
Elie


On Sun, Dec 30, 2012 at 11:48 AM, Ian David Moss <ian.moss@...>wrote:

> **
>
>
> Hi guys,
> On a related note, I'm getting ready to donate to GW's recommended
> charities and have been thinking a bit about VillageReach. In your July
> 26 blog post<http://blog.givewell.org/2012/07/26/rethinking-villagereachs-pilot-project/>,
> you stated that while your view of the effectiveness of VR's pilot project
> has changed, "we continue to view VillageReach as a highly transparent
> “learning organization” (capable of conducting its activities in a way that
> can lead to learning). Over the past few years, VillageReach has provided
> us with the source data behind its evaluations enabling us to do our own in
> depth analysis and draw our own conclusions. That work has contributed to
> our own growing ability to evaluate impact evaluations and determine the
> level of reliance that can be placed on them. *We will be talking with
> VillageReach about how more funding could contribute to more
> experimentation and learning, and we will likely be interested in
> recommending such funding – to encourage such outstanding transparency and
> accountability, and learn more in the future*."
>
> Later on in the post, you write:
>
> "Groups like VillageReach are creating a new dialogue around charitable
> giving, and it’s important to us that this type of behavior is supported.
> We want to encourage VillageReach and other groups to share information
> about how their programs are going, and we want to continue to see more
> experimentation and learning. So, we are seriously considering recommending
> donations to VillageReach, not despite the struggles it’s had but because
> it’s had these struggles and is being honest about them."
>
> These sentiments are very nice and make a lot of sense to me, but it
> doesn't seem like the plan to recommend funding (even on a limited basis)
> to reward VillageReach's honesty and experimentation ever materialized. As
> Vipul notes, the laser-like focus on the top 3 charities does make it seem
> like VR has effectively been "de-funded" by GiveWell. I feel that the need
> to support experimentation and learning is important, and the lack of
> follow-through here sends the message, whether intentionally or not, that
> VR was too honest with a funder and now is suffering for it. Your thoughts?
>
> Thanks,
> Ian
>
>
> On Thu, Dec 27, 2012 at 7:52 PM, Alexander Berger <alexander.is@...>wrote:
>
>> **
>>
>>
>> We thought this exchange about our shift away from emphasizing the
>> "standouts" list might be of some general interest:
>>
>> Vipul Naik wrote to us, asking:
>>
>> Dear GiveWell folk,
>>
>> I've been thinking a bit about the change you introduced this year
>> regarding discontinuing the list of standout charities and
>> domain-specific top recommendations on your main page and
>> concentrating all your energies on your top three recommendations. I'm
>> wondering how this affects the incentives for charities to cooperate
>> with you in the review process.
>>
>> One possibility that occurred to me is that charities will be more
>> likely to view cooperating with GiveWell as a high-upfront-cost,
>> very-low-probability-of-reward endeavor, which may make them reluctant
>> to cooperate with you. In the past, making it to a list of standout
>> charities or domain-specific top recommendations might have been
>> sufficient incentive to cooperate with GiveWell. With the new system,
>> there are few immediate payoffs for a charity unless it's convinced it
>> can make it to the top list.
>>
>> I don't know if that's a good thing or a bad thing. On the plus side,
>> it means that charities that aren't really a good fit for GiveWell
>> would self-select out of getting reviewed. On the other hand, some
>> charities that hold promise for the long term but are unlikely to get
>> a high rating at present may be put off by the process, so you may not
>> get an opportunity to follow their progress from the time of inception
>> to the time that they may be ripe for getting a good rating.
>>
>> Of course, it's possible that I am wrong about this and that
>> charities' incentive to cooperate is not much affected by the changes
>> in your recommendation style.
>>
>> One small change I could think of, however, might be worth
>> considering. In your reviews of charities that you decide not to
>> recommend, you might add two tiny sections:
>>
>> (1) What kind of new evidence would change your mind and make you
>> re-investigate the charity?
>>
>> (2) With the existing evidence, are there other reasonable priors
>> (normative or descriptive) that people may have that make the charity
>> worth donating to?
>>
>> The presence of these sections could help make your reviews more
>> useful to people who want to gain knowledge of a charity and are
>> already strongly considering donating to it, while requiring minimum
>> additional effort on your part. It also gives charities some idea of
>> the kind of things they'd need to do in order to convince you of their
>> utility. With both of these, charities have more to gain from getting
>> reviewed by you and linking to your reviews of them.
>>
>> Vipul
>>
>> We responded:
>>
>> Hi Vipul,
>>
>> We don't expect to see big changes in the kinds of participation we're
>> getting from charities as a result of this switch. Although it does
>> obviously decrease the number of organizations receiving significant funds
>> as a result of GiveWell's recommendations, we don't think it materially
>> impacts the expected value of a recommendation (since we don't think it
>> will have much impact on the total money moved), and we also doubt that
>> charities considering our process are doing complex expected value
>> calculations. More importantly, we've observed a substantial increase in
>> our access due to our growing influence, and we expect that to dwarf the
>> impact of this change. (It's also worth noting that we only *want* engagement
>> from promising charities, and we think that such charities are more likely
>> to see that they have a shot at the top spot.)
>>
>> Nonetheless, we think both of your specific proposals are worth
>> considering:
>>
>>    1. For most organizations we look at, we don't see realistic short
>>    term steps that would lead us to a recommendation, but in the past we've
>>    tried to communicate privately with organizations where that has been the
>>    case (e.g. GiveDirectly, Cochrane) and have seen some success; we'll think
>>    about trying to make that feedback more public, but if we do it's likely to
>>    be for a small subset of organizations.
>>    2. Our reviews of most charities we recommend receive very little
>>    traffic, so we're hesitant to spend much more time on them. That said,
>>    we're considering trying to do more to foreground organizations that we
>>    haven't investigated as deeply but believe may be as good as some of our
>>    recommended organizations, depending on donor values or preferences.
>>
>> Thanks for the question and happy holidays!
>>
>> Best,
>> Alexander
>>
>>
>> --
>> Alexander Berger
>> Research Analyst
>> www.GiveWell.org
>>
>>
>  
>

Thanks, Elie, I appreciate the response. I am curious, though, about why
you decided against recommending for the sake of transparency. Was there
new information about VillageReach specifically that caused you to
re-evaluate, or was it more that the logic you cite below made you feel
that transparency incentives, in general, were not a good enough reason to
recommend funding?

On Sun, Dec 30, 2012 at 10:28 PM, Elie Hassenfeld <elie@...> wrote:

> **
>
>
> Hi Ian,
>
> We have continued to follow VillageReach and did consider potentially
> recommending funding for them for the sake of "transparency incentives."
>
> I don't think it's accurate to say that VillageReach is suffering because
> it was too honest with a funder. The only thing VillageReach coud have done
> to maintain its rating was to continue being transparent with us. If it had
> withheld information, we wouldn't have recommended it. Instead, because it
> continues to be transparent, we continue to publish information about it,
> and some (albeit a small proportion) of our donors continue to support it.
> The fact that these donors maintained their support reduces the problem
> you're bringing up.
>
> It's also worth noting that VillageReach received over $2 million as a
> result of GiveWell's recommendation. I'd guess that the message most
> non-profits would takeaway were they to consider VillageReach's case is
> that by being transparent VillageReach gained significant funding that it
> otherwise would not have. VillageReach has also been reasonably successful
> raising funds for other initiatives, so there's no evidence that its
> transparency has harmed it.
>
> Best,
> Elie
>
>
> On Sun, Dec 30, 2012 at 11:48 AM, Ian David Moss <ian.moss@...>wrote:
>
>> **
>>
>>
>> Hi guys,
>> On a related note, I'm getting ready to donate to GW's recommended
>> charities and have been thinking a bit about VillageReach. In your July
>> 26 blog post<http://blog.givewell.org/2012/07/26/rethinking-villagereachs-pilot-project/>,
>> you stated that while your view of the effectiveness of VR's pilot project
>> has changed, "we continue to view VillageReach as a highly transparent
>> “learning organization” (capable of conducting its activities in a way that
>> can lead to learning). Over the past few years, VillageReach has provided
>> us with the source data behind its evaluations enabling us to do our own in
>> depth analysis and draw our own conclusions. That work has contributed to
>> our own growing ability to evaluate impact evaluations and determine the
>> level of reliance that can be placed on them. *We will be talking with
>> VillageReach about how more funding could contribute to more
>> experimentation and learning, and we will likely be interested in
>> recommending such funding – to encourage such outstanding transparency and
>> accountability, and learn more in the future*."
>>
>> Later on in the post, you write:
>>
>> "Groups like VillageReach are creating a new dialogue around charitable
>> giving, and it’s important to us that this type of behavior is supported.
>> We want to encourage VillageReach and other groups to share information
>> about how their programs are going, and we want to continue to see more
>> experimentation and learning. So, we are seriously considering recommending
>> donations to VillageReach, not despite the struggles it’s had but because
>> it’s had these struggles and is being honest about them."
>>
>> These sentiments are very nice and make a lot of sense to me, but it
>> doesn't seem like the plan to recommend funding (even on a limited basis)
>> to reward VillageReach's honesty and experimentation ever materialized. As
>> Vipul notes, the laser-like focus on the top 3 charities does make it seem
>> like VR has effectively been "de-funded" by GiveWell. I feel that the need
>> to support experimentation and learning is important, and the lack of
>> follow-through here sends the message, whether intentionally or not, that
>> VR was too honest with a funder and now is suffering for it. Your thoughts?
>>
>> Thanks,
>> Ian
>>
>>
>> On Thu, Dec 27, 2012 at 7:52 PM, Alexander Berger <alexander.is@...
>> > wrote:
>>
>>> **
>>>
>>>
>>> We thought this exchange about our shift away from emphasizing the
>>> "standouts" list might be of some general interest:
>>>
>>> Vipul Naik wrote to us, asking:
>>>
>>> Dear GiveWell folk,
>>>
>>> I've been thinking a bit about the change you introduced this year
>>> regarding discontinuing the list of standout charities and
>>> domain-specific top recommendations on your main page and
>>> concentrating all your energies on your top three recommendations. I'm
>>> wondering how this affects the incentives for charities to cooperate
>>> with you in the review process.
>>>
>>> One possibility that occurred to me is that charities will be more
>>> likely to view cooperating with GiveWell as a high-upfront-cost,
>>> very-low-probability-of-reward endeavor, which may make them reluctant
>>> to cooperate with you. In the past, making it to a list of standout
>>> charities or domain-specific top recommendations might have been
>>> sufficient incentive to cooperate with GiveWell. With the new system,
>>> there are few immediate payoffs for a charity unless it's convinced it
>>> can make it to the top list.
>>>
>>> I don't know if that's a good thing or a bad thing. On the plus side,
>>> it means that charities that aren't really a good fit for GiveWell
>>> would self-select out of getting reviewed. On the other hand, some
>>> charities that hold promise for the long term but are unlikely to get
>>> a high rating at present may be put off by the process, so you may not
>>> get an opportunity to follow their progress from the time of inception
>>> to the time that they may be ripe for getting a good rating.
>>>
>>> Of course, it's possible that I am wrong about this and that
>>> charities' incentive to cooperate is not much affected by the changes
>>> in your recommendation style.
>>>
>>> One small change I could think of, however, might be worth
>>> considering. In your reviews of charities that you decide not to
>>> recommend, you might add two tiny sections:
>>>
>>> (1) What kind of new evidence would change your mind and make you
>>> re-investigate the charity?
>>>
>>> (2) With the existing evidence, are there other reasonable priors
>>> (normative or descriptive) that people may have that make the charity
>>> worth donating to?
>>>
>>> The presence of these sections could help make your reviews more
>>> useful to people who want to gain knowledge of a charity and are
>>> already strongly considering donating to it, while requiring minimum
>>> additional effort on your part. It also gives charities some idea of
>>> the kind of things they'd need to do in order to convince you of their
>>> utility. With both of these, charities have more to gain from getting
>>> reviewed by you and linking to your reviews of them.
>>>
>>> Vipul
>>>
>>> We responded:
>>>
>>> Hi Vipul,
>>>
>>> We don't expect to see big changes in the kinds of participation we're
>>> getting from charities as a result of this switch. Although it does
>>> obviously decrease the number of organizations receiving significant funds
>>> as a result of GiveWell's recommendations, we don't think it materially
>>> impacts the expected value of a recommendation (since we don't think it
>>> will have much impact on the total money moved), and we also doubt that
>>> charities considering our process are doing complex expected value
>>> calculations. More importantly, we've observed a substantial increase in
>>> our access due to our growing influence, and we expect that to dwarf the
>>> impact of this change. (It's also worth noting that we only *want* engagement
>>> from promising charities, and we think that such charities are more likely
>>> to see that they have a shot at the top spot.)
>>>
>>> Nonetheless, we think both of your specific proposals are worth
>>> considering:
>>>
>>>    1. For most organizations we look at, we don't see realistic short
>>>    term steps that would lead us to a recommendation, but in the past we've
>>>    tried to communicate privately with organizations where that has been the
>>>    case (e.g. GiveDirectly, Cochrane) and have seen some success; we'll think
>>>    about trying to make that feedback more public, but if we do it's likely to
>>>    be for a small subset of organizations.
>>>    2. Our reviews of most charities we recommend receive very little
>>>    traffic, so we're hesitant to spend much more time on them. That said,
>>>    we're considering trying to do more to foreground organizations that we
>>>    haven't investigated as deeply but believe may be as good as some of our
>>>    recommended organizations, depending on donor values or preferences.
>>>
>>> Thanks for the question and happy holidays!
>>>
>>> Best,
>>> Alexander
>>>
>>>
>>> --
>>> Alexander Berger
>>> Research Analyst
>>> www.GiveWell.org
>>>
>>>
>>
>  
>

Hi Ian,

It was more that transparency incentives in general were not a good enough
reason to recommend funding.

-Elie


On Sun, Dec 30, 2012 at 11:15 PM, Ian David Moss <mossinator@...>wrote:

> **
>
>
> Thanks, Elie, I appreciate the response. I am curious, though, about why
> you decided against recommending for the sake of transparency. Was there
> new information about VillageReach specifically that caused you to
> re-evaluate, or was it more that the logic you cite below made you feel
> that transparency incentives, in general, were not a good enough reason to
> recommend funding?
>
>
> On Sun, Dec 30, 2012 at 10:28 PM, Elie Hassenfeld <elie@...>wrote:
>
>> **
>>
>>
>> Hi Ian,
>>
>> We have continued to follow VillageReach and did consider potentially
>> recommending funding for them for the sake of "transparency incentives."
>>
>> I don't think it's accurate to say that VillageReach is suffering because
>> it was too honest with a funder. The only thing VillageReach coud have done
>> to maintain its rating was to continue being transparent with us. If it had
>> withheld information, we wouldn't have recommended it. Instead, because it
>> continues to be transparent, we continue to publish information about it,
>> and some (albeit a small proportion) of our donors continue to support it.
>> The fact that these donors maintained their support reduces the problem
>> you're bringing up.
>>
>> It's also worth noting that VillageReach received over $2 million as a
>> result of GiveWell's recommendation. I'd guess that the message most
>> non-profits would takeaway were they to consider VillageReach's case is
>> that by being transparent VillageReach gained significant funding that it
>> otherwise would not have. VillageReach has also been reasonably successful
>> raising funds for other initiatives, so there's no evidence that its
>> transparency has harmed it.
>>
>> Best,
>> Elie
>>
>>
>> On Sun, Dec 30, 2012 at 11:48 AM, Ian David Moss <ian.moss@...>wrote:
>>
>>> **
>>>
>>>
>>> Hi guys,
>>> On a related note, I'm getting ready to donate to GW's recommended
>>> charities and have been thinking a bit about VillageReach. In your July
>>> 26 blog post<http://blog.givewell.org/2012/07/26/rethinking-villagereachs-pilot-project/>,
>>> you stated that while your view of the effectiveness of VR's pilot project
>>> has changed, "we continue to view VillageReach as a highly transparent
>>> “learning organization” (capable of conducting its activities in a way that
>>> can lead to learning). Over the past few years, VillageReach has provided
>>> us with the source data behind its evaluations enabling us to do our own in
>>> depth analysis and draw our own conclusions. That work has contributed to
>>> our own growing ability to evaluate impact evaluations and determine the
>>> level of reliance that can be placed on them. *We will be talking with
>>> VillageReach about how more funding could contribute to more
>>> experimentation and learning, and we will likely be interested in
>>> recommending such funding – to encourage such outstanding transparency and
>>> accountability, and learn more in the future*."
>>>
>>> Later on in the post, you write:
>>>
>>> "Groups like VillageReach are creating a new dialogue around charitable
>>> giving, and it’s important to us that this type of behavior is supported.
>>> We want to encourage VillageReach and other groups to share information
>>> about how their programs are going, and we want to continue to see more
>>> experimentation and learning. So, we are seriously considering recommending
>>> donations to VillageReach, not despite the struggles it’s had but because
>>> it’s had these struggles and is being honest about them."
>>>
>>> These sentiments are very nice and make a lot of sense to me, but it
>>> doesn't seem like the plan to recommend funding (even on a limited basis)
>>> to reward VillageReach's honesty and experimentation ever materialized. As
>>> Vipul notes, the laser-like focus on the top 3 charities does make it seem
>>> like VR has effectively been "de-funded" by GiveWell. I feel that the need
>>> to support experimentation and learning is important, and the lack of
>>> follow-through here sends the message, whether intentionally or not, that
>>> VR was too honest with a funder and now is suffering for it. Your thoughts?
>>>
>>> Thanks,
>>> Ian
>>>
>>>
>>> On Thu, Dec 27, 2012 at 7:52 PM, Alexander Berger <
>>> alexander.is@gmail.com> wrote:
>>>
>>>> **
>>>>
>>>>
>>>> We thought this exchange about our shift away from emphasizing the
>>>> "standouts" list might be of some general interest:
>>>>
>>>> Vipul Naik wrote to us, asking:
>>>>
>>>> Dear GiveWell folk,
>>>>
>>>> I've been thinking a bit about the change you introduced this year
>>>> regarding discontinuing the list of standout charities and
>>>> domain-specific top recommendations on your main page and
>>>> concentrating all your energies on your top three recommendations. I'm
>>>> wondering how this affects the incentives for charities to cooperate
>>>> with you in the review process.
>>>>
>>>> One possibility that occurred to me is that charities will be more
>>>> likely to view cooperating with GiveWell as a high-upfront-cost,
>>>> very-low-probability-of-reward endeavor, which may make them reluctant
>>>> to cooperate with you. In the past, making it to a list of standout
>>>> charities or domain-specific top recommendations might have been
>>>> sufficient incentive to cooperate with GiveWell. With the new system,
>>>> there are few immediate payoffs for a charity unless it's convinced it
>>>> can make it to the top list.
>>>>
>>>> I don't know if that's a good thing or a bad thing. On the plus side,
>>>> it means that charities that aren't really a good fit for GiveWell
>>>> would self-select out of getting reviewed. On the other hand, some
>>>> charities that hold promise for the long term but are unlikely to get
>>>> a high rating at present may be put off by the process, so you may not
>>>> get an opportunity to follow their progress from the time of inception
>>>> to the time that they may be ripe for getting a good rating.
>>>>
>>>> Of course, it's possible that I am wrong about this and that
>>>> charities' incentive to cooperate is not much affected by the changes
>>>> in your recommendation style.
>>>>
>>>> One small change I could think of, however, might be worth
>>>> considering. In your reviews of charities that you decide not to
>>>> recommend, you might add two tiny sections:
>>>>
>>>> (1) What kind of new evidence would change your mind and make you
>>>> re-investigate the charity?
>>>>
>>>> (2) With the existing evidence, are there other reasonable priors
>>>> (normative or descriptive) that people may have that make the charity
>>>> worth donating to?
>>>>
>>>> The presence of these sections could help make your reviews more
>>>> useful to people who want to gain knowledge of a charity and are
>>>> already strongly considering donating to it, while requiring minimum
>>>> additional effort on your part. It also gives charities some idea of
>>>> the kind of things they'd need to do in order to convince you of their
>>>> utility. With both of these, charities have more to gain from getting
>>>> reviewed by you and linking to your reviews of them.
>>>>
>>>> Vipul
>>>>
>>>> We responded:
>>>>
>>>> Hi Vipul,
>>>>
>>>> We don't expect to see big changes in the kinds of participation we're
>>>> getting from charities as a result of this switch. Although it does
>>>> obviously decrease the number of organizations receiving significant funds
>>>> as a result of GiveWell's recommendations, we don't think it materially
>>>> impacts the expected value of a recommendation (since we don't think it
>>>> will have much impact on the total money moved), and we also doubt that
>>>> charities considering our process are doing complex expected value
>>>> calculations. More importantly, we've observed a substantial increase in
>>>> our access due to our growing influence, and we expect that to dwarf the
>>>> impact of this change. (It's also worth noting that we only *want* engagement
>>>> from promising charities, and we think that such charities are more likely
>>>> to see that they have a shot at the top spot.)
>>>>
>>>> Nonetheless, we think both of your specific proposals are worth
>>>> considering:
>>>>
>>>>    1. For most organizations we look at, we don't see realistic short
>>>>    term steps that would lead us to a recommendation, but in the past we've
>>>>    tried to communicate privately with organizations where that has been the
>>>>    case (e.g. GiveDirectly, Cochrane) and have seen some success; we'll think
>>>>    about trying to make that feedback more public, but if we do it's likely to
>>>>    be for a small subset of organizations.
>>>>    2. Our reviews of most charities we recommend receive very little
>>>>    traffic, so we're hesitant to spend much more time on them. That said,
>>>>    we're considering trying to do more to foreground organizations that we
>>>>    haven't investigated as deeply but believe may be as good as some of our
>>>>    recommended organizations, depending on donor values or preferences.
>>>>
>>>> Thanks for the question and happy holidays!
>>>>
>>>> Best,
>>>> Alexander
>>>>
>>>>
>>>> --
>>>> Alexander Berger
>>>> Research Analyst
>>>> www.GiveWell.org
>>>>
>>>>
>>>
>>
>  
>