[ICT4D Principles] Does your project use real time data? Is ti data driven? Tell us


#1

My firm, Sonjara, has been supporting sector programs with technology, especially around data and Performance Management. An example is our role in supporting a usaid Global Health EPCMD program called Accelerate (acceleratorbehaviors.org).

Part of the goal of this project is to collect and use both open and internal data to support the evidence around behavior change as a framework to improve health outcomes for mothers and children. We’re trying to help usaid missions use data around behavior change to improve outcomes.

As we have discovered so far “real-time data” is generally not real time, but rather more frequently than every 5 years. Even Senegal with its continuous DHS approach only gets data annually. Right now the issue seems to be around data collection systems which only collect data or at least share data on periodic bases. And too much data is still being captured in paper formats and only digitized way up the data lifecycle at which point it’s fully aggregated and has lost a lot of its context.

This isn’t really a technology problem, as there are lots of tools that can be used. Rather it seems to be more of a technology infrastructure, data collection culture, and data collection infrastructure challenge.

Because the data is primarily being captured to support senior-level reporting requirements, it often does not serve the people who are capturing it. There have been multiple reports of local staff having two sets of data books; one for their own needs and another to report to management.

An alternative approach would be to focus on making Performance Management tools at the local level serve local needs with a side benefit that that some of data is shared upwards. Then you’ll start seeing data driven development.

I would love to hear more about projects that have managed to address this issue.

Siobhan Green
Sonjara, Inc
Sio@sonjara.com
Office: 571 297 6383 ext 101
Cell: 703 981 9982
Skype : sewgreen

··· -------- Original message -------- From: Mark Roest Date: 05/12/2016 12:32 AM (GMT-05:00) To: Shyamadas Banerji Cc: Robert Kirkpatrick , Samir Doshi , Glen Burnett , Merrick Schaefer , ICT4D Principles , Michael Bamberger Subject: Re: [ICT4D Principles] Does your project use real time data? Is ti data driven? Tell us.

Hello Robert,

Perhaps the best way to achieve what you want is to organize data into a Geographic Information System knowledgebase, organized at the ecosystem and eco-region level (using the Terrestrial Eco-regions files held by World Wildlife Fund). Thus, information shows up in the eco-regions and ecosystems for which it is likely to be relevant. If input by every person in each region is afforded, considered, and celebrated, and ways are found to support their proposals and base actions on their analyses, communities will accept it as a useful, respectful tool.

I gave a talk at the 5th International Symposium on Digital Earth (ISDE5, UC Berkeley, July 2007) on collaboration between the community of science and communities on the ground, based on common interests in specific ecosystem problems. The scientist gets access to extremely fine-grained data, and if she supports communication on the subject with a large sample or class of the people involved, it will add up to big data. The community gets access to cross-cutting research results and interpretation, and, potentially, exposure to a very wide range of experiences from many parts of the world. If the community is also afforded access to databases such as introduced above, and supply chain and distribution channel software is integrated into the db, and if not already present, concepts of cooperatives are introduced, it becomes possible for communities to design their own production and business ecosystems, based on the surplus capacity of their natural ecosystems.

Whether there is sufficient engagement for a single ecosystem to generate ‘big data’ or not, when you look at results from 867 eco-regions and thousands of ecosystems, and start looking for environmental, cultural, craft and technological commonalities, using them to empower people in each ecosystem, and assisting those who decide to use the db as a tool, you will rapidly wind up working with big data.

Regards,

Mark Roest
International Development
SeaWave Battery

Co-founder and Research Director
DesignEarth.net

On Wed, May 11, 2016 at 7:49 PM, Shyamadas Banerji <banerji1@comcast.netmailto:banerji1@comcast.net> wrote:
Dear Robert,
Many thanks for your interest in the question I raised. First, I should mention that I am delighted to learn of your data analytics initiative. If I can assist in any way , kindly let me know. I look forward to reading Michael’s draft reports.
As someone who has undertaken evaluations of development programs and projects for over two decades at the World Bank , ADB and other agencies, I have been troubled that evaluators rely mainly on experience and judgments in conducting evaluations, including conducting impact evaluations, taking into consideration experimental trials and methods. The standard DAC criteria used in program and project evaluations -relevance, efficiency, efficacy, outcome and impact -are usually not well defined or quantified a priori at appraisal , so that project evaluators assign a subjective rating during ex-post evaluation which can be erroneous. The log frame of the project may not be logical or evidence based. The indicators used to assess each of the evaluation criteria may fail to reflect the achievements taking into account broader sectoral and country experience . And since these indicators are specified by the project formulators at appraisal , weak or faulty indicators may constrain the ex-post evaluation exercise.
Another issue is the the Theory of Change (TOC) which supports the logical framework and the conceptual underpinning of the project. Recently, I have seen quite a few bird nests of TOC diagrams which contain many assumptions about the logical process of change and transformation which the project is trying to achieve without providing any empirical evidence about how the change actually happens, the risks and why the proposed intervention is the best or optimal method of achieving the outcomes desired. Applications of Program Theory suffer from similar deficiencies which I can elaborate on.
My interest is to explore how one could use big data analytics/artificial intelligence/machine learning to strengthen the evidence base of project formulation and evaluation in different sectors or development issues. I would like to know how one could strengthen the formulation of TOC for projects in different problem areas based on actual field evidence from learning and implementation of a wide cross section of completed projects . This would lead to a deeper understanding of change pathways and risk variables at the society/community/institutional levels depending on the level of project intervention and how monitoring and mitigating actions can be built into the project to achieve desired goals. Various development agencies could contribute to providing data for building the data base. The data base would provide a full picture of real life experiences on how development problems in various areas have been defined and addressed and with what results and reasons for the outcomes and the lessons.
I am aware that the evaluation departments some development agencies have built data banks of projects they have supported and have undertaken large meta studies of project interventions in specific areas to learn some generalizable lessons. My experience looking at these meta studies is that they are not particularly useful for designing new specific project interventions and that the lessons of earlier projects are inadequately reflected in new projects.
I am thinking of the value of developing a large multi country , multi agency data base which can be queried for evidence to help with project design, implementation and evaluation. Can one build a data bank which can use an IBM Watson type approach of artificial intelligence to learn from past project interventions in different areas to design better future interventions? The feedback loop between evaluation results and new project designs can then be finally closed.
This is just a quick response to your question and I hope it will stimulate some thinking and generate some more focused ideas.
Kind regards,
Shyamadas

On May 11, 2016, at 5:06 PM, Robert Kirkpatrick <kirkpatrick@unglobalpulse.orgmailto:kirkpatrick@unglobalpulse.org> wrote:

Dear Shyamdas,

We are working with Michael Bamberger (copied) on finalizing what we believe will be the seminal report (actually a pair of reports) on applications of analytics to development project evaluation. These reports will be available this summer. Happy to chat further on this topic. Would love to know more about your interests here!

Warm regards,

Robert

Robert Kirkpatrick
Director
United Nations Global Pulse
Executive Office of the Secretary-General
+1 650.796.5709tel:%2B1%20650.796.5709
kirkpatrick@un.orgmailto:kirkpatrick@un.org
@rgkirkpatrick

The Data Revolution in Action: 20 Case Studies by Global Pulse:
http://unglobalpulse.org/blog/big-data-development-action-global-pulse-project-serieshttp://unglobalpulse.org/blog/big-data-development-action-global-pulse-project-series

On Wed, May 11, 2016 at 3:23 PM, Shyamadas Banerji <banerji1@comcast.netmailto:banerji1@comcast.net> wrote:
Dear Samir,
Please keep me in the information loop. I am particularly interested in the application of big data analytics to ex-post and impact evaluation of development projects. Real time data probably has limited use in project evaluation since one evaluates projects for outcomes after a delay. But it would have use in project monitoring where the results need to be continuously measured such as in pollution abatement projects. If you or any one else in your group is working on big data applications in impact evaluations of development projects , I would be interested in learning about the initiative. many thanks.
Best,
Shyamadas Banerji
sbanerji@alum.mit.edumailto:sbanerji@alum.mit.edu

On May 11, 2016, at 9:45 AM, Samir Doshi <sdoshi@usaid.govmailto:sdoshi@usaid.gov> wrote:

Glen,

We will be publishing a literature review on the current knowledge base of real-time data and the intersection with data informed decision making / adaptive management.

The survey responses will be compiled and then scoped for a comparative case study report that will include empirical investigation by a research consortium of how select cases are implementing RTD systems for best practice and purpose. This report will be published at the end of the year.

Kind regards,
Samir


Samir K. Doshi, Ph.D. | Senior Scientist
U.S. Global Development Lab | US Agency for International Development
E: sdoshi@usaid.govmailto:sdoshi@usaid.gov | P: +1.202.712.1380tel:%2B1.202.712.1380 | W: USAID.gov/GlobalDevLabhttp://www.usaid.gov/GlobalDevLab

On Tue, May 10, 2016 at 5:29 PM, Glen Burnett <Glen.Burnett@practicalaction.org.ukmailto:Glen.Burnett@practicalaction.org.uk> wrote:
Merrick, will you be publishing results of this review? It would be great to see what you learn.

Glen Burnett

From: ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com [mailto:ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com] On Behalf Of Merrick Schaefer
Sent: Tuesday, May 10, 2016 5:12 PM
To: ICT4D Principles
Subject: [ICT4D Principles] Does your project use real time data? Is ti data driven? Tell us.

Hey Everyone,

USAID, the mSTAR project, and a research consortium led by Institute of Development Studies and including the Overseas Development Institute, Reboot, and Feedback Labs are looking for case study examples of projects that are using real-time data in development work.

We want to better understand how real-time data initiatives work and how they contribute to better strategic and operational decision-making, and ultimately how they might be used to enhance and support stronger adaptive managements approaches. We want to build the evidence base in this increasingly important but under-analysed area, and use this to development tools and frameworks that can inform future police, practice and research.

Please do take a few minutes to share your examples of projects that utilize real-time data for improved decision-making by filling out the form in the link below.

The best five contributions will receive a copy of “Time to Listen,” by Dayna Brown, Mary B. Anderson, & Isabella Jean. To contribute more than one case study, please reuse this form and submit your cases by 5/17/2016 in order to be considered and to win a copy of “Time to Listen.”

Your help and support are much appreciated! Also, please forward along to others that could contribute.

Best,

Merrick

You received this message because you are subscribed to the Google Groups “ICT4D Principles” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ict4d-principles+unsubscribe@googlegroups.commailto:ict4d-principles+unsubscribe@googlegroups.com.
To post to this group, send email to ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[http://bit.ly/1hl4Btr]http://www.practicalaction.org/

Practical Action uses technology to challenge poverty, working with poor women and men around the world.www.practicalaction.orghttp://www.practicalaction.org/

[http://bit.ly/17osJ5h]http://www.facebook.com/practicalaction [http://bit.ly/12Eshft] http://www.twitter.com/practicalaction [http://bit.ly/17osOpB] http://www.youtube.com/practicalaction [http://bit.ly/Z5rh4c] http://www.flickr.com/photos/practicalaction [http://bit.ly/XpFdWm] http://www.linkedin.com/groups/Practical-Action-2428887 [http://bit.ly/YestUa] http://practicalaction.org/blogs [http://bit.ly/XpFkRI] http://practicalaction.org/get-our-e-newsletter Support our work today [Donate online] http://practicalaction.org/signature

Practical Action, The Schumacher Centre, Bourton on Dunsmore, Rugby, Warwickshire, UK, CV23 9QZ
Practical Action is a charity registered in England and Wales, and a company limited by guarantee in England no. 871954
VAT Reg No. 880992476 | Reg Charity No. 247257. [http://bit.ly/17BommD]


You received this message because you are subscribed to the Google Groups “ICT4D Principles” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ict4d-principles+unsubscribe@googlegroups.commailto:ict4d-principles+unsubscribe@googlegroups.com.
To post to this group, send email to ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “ICT4D Principles” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ict4d-principles+unsubscribe@googlegroups.commailto:ict4d-principles+unsubscribe@googlegroups.com.
To post to this group, send email to ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “ICT4D Principles” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ict4d-principles+unsubscribe@googlegroups.commailto:ict4d-principles+unsubscribe@googlegroups.com.
To post to this group, send email to ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “ICT4D Principles” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ict4d-principles+unsubscribe@googlegroups.commailto:ict4d-principles+unsubscribe@googlegroups.com.
To post to this group, send email to ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


You received this message because you are subscribed to the Google Groups “ICT4D Principles” group.
To unsubscribe from this group and stop receiving emails from it, send an email to ict4d-principles+unsubscribe@googlegroups.commailto:ict4d-principles+unsubscribe@googlegroups.com.
To post to this group, send email to ict4d-principles@googlegroups.commailto:ict4d-principles@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.