i	
  
	
  CGIAR Review 2012 ACTIVITY	
  1	
  
	
  IFAD –AGRINATURA-EEIG Institutional Contract  
 
 FINAL REPORT 	
  
	
  Review	
  of	
  CGIAR	
  Research	
  Programme	
  7:	
  Climate	
  Change,	
  Agriculture	
  and	
  Food	
  Security	
  TEAM	
  6	
  Guy	
  Jobbins	
  and	
  Didier	
  Pillot	
  January	
  2013	
   	
  
	
   ii	
  
Table	
  of	
  Contents	
  Executive	
  Summary	
  .........................................................................................................................................	
  iii	
  1.	
  Introduction	
  ....................................................................................................................................................	
  1	
  2	
  Review	
  Criteria	
  ...............................................................................................................................................	
  2	
  2.1	
  Relevance	
  ..................................................................................................................................................	
  2	
  2.2.	
  Efficiency	
  ..................................................................................................................................................	
  5	
  2.3	
  Effectiveness	
  ............................................................................................................................................	
  6	
  2.3.1	
  General	
  provisions	
  for	
  monitoring	
  and	
  evaluation	
  ........................................................	
  6	
  2.3.2	
  Analysis	
  of	
  Logical	
  Framework	
  and	
  Baselines	
  for	
  Impact	
  Assessment	
  .................	
  9	
  2.4.	
  Practical	
  implementation	
  of	
  the	
  CRPs	
  .......................................................................................	
  12	
  2.5.	
  Synergy	
  &	
  collective	
  action	
  ............................................................................................................	
  12	
  2.6.	
  Internationally	
  or	
  regional	
  public	
  goods	
  character	
  of	
  research	
  ....................................	
  13	
  3	
  Strategic	
  issues	
  for	
  EC	
  &	
  CGIAR	
  .............................................................................................................	
  13	
  3.1	
  Lessons	
  regarding	
  the	
  CGIAR	
  reform	
  process	
  ........................................................................	
  13	
  3.1.1	
  Organisational	
  cohesiveness	
  and	
  partnerships	
  .............................................................	
  14	
  3.1.2	
  Science	
  quality	
  versus	
  development	
  outcomes	
  .............................................................	
  15	
  3.2	
  Monitoring	
  and	
  Evaluation	
  at	
  the	
  Consortium	
  Level	
  ...........................................................	
  16	
  3.3	
  Impact,	
  sustainability	
  &	
  visibility	
  of	
  EC	
  co-­‐financing	
  ..........................................................	
  17	
  4.	
  Conclusions	
  ....................................................................................................................................................	
  19	
  5.	
  Recommendations	
  ......................................................................................................................................	
  21	
  Annexes	
  ................................................................................................................................................................	
  24	
  A1.	
  Review	
  Terms	
  of	
  Reference	
  ............................................................................................................	
  24	
  A2.	
  Feed	
  back	
  on	
  the	
  review	
  process	
  and	
  terms	
  of	
  reference	
  .................................................	
  28	
  A3.	
  Curricula	
  Vitae	
  of	
  Consultants	
  .......................................................................................................	
  28	
  A4.	
  Maps	
  ..........................................................................................................................................................	
  31	
  A5.	
  Review	
  Agenda	
  .....................................................................................................................................	
  32	
  A6	
  Literature	
  &	
  documentation	
  consulted	
  ......................................................................................	
  35	
  A7	
  Other	
  technical	
  annexes	
  ....................................................................................................................	
  37	
  A8	
  Response	
  of	
  CCAFS	
  Coordinating	
  Unit	
  to	
  the	
  Draft	
  Report	
  ...............................................	
  38	
  A9	
  DAC	
  Summary	
  ........................................................................................................................................	
  41	
  	
  Cover Photo: P. Casier (CGIAR) (Creative Commons License). ‘Pictures are from meeting with innovative farmer Joel Yiri, Jirapa Village in Ghana. He started developing his plot of land by using manure from his pigs when he realized the soil had become infertile. He is also serious about keeping a record of all input costs and revenues, meaning that he now can track looses and change crops and production techniques accordingly. Joel Yiri is also experiencing a change in rainfall patters, the rain start later every year, and stops earlier, mixed with periods of drought. View Joel Yiri talk about his farming techniques and his hopes for the future in the video ‘Farmer Testimonials’. Read also the accompanying blog story on the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS) blog "We should not farm anymore like our grandfathers did".’ 	
  
	
   iii	
  
Executive	
  Summary	
  1. This	
  review	
  was	
  commissioned	
  to	
  evaluate	
  European	
  Commission	
  (EC)	
  contributions	
  to	
  the	
  CGIAR	
  Research	
  Programme	
  (CRP)	
  on	
  Climate	
  Change,	
  Agriculture	
  and	
  Food	
  Security	
  (CCAFS)	
  in	
  2011.	
  EC	
  contributions	
  were	
  €2.17	
  million,	
  whilst	
  CCAFS	
  total	
  budget	
  was	
  USD	
  62	
  million.	
  (Page	
  1)	
  2. The	
  review	
  is	
  framed	
  around	
  evaluation	
  of	
  processes	
  and	
  tools	
  in	
  the	
  context	
  of	
  the	
  current	
  CGIAR	
  reform	
  process	
  rather	
  than	
  observed	
  impact	
  of	
  a	
  specific	
  project.	
  In	
  particular	
  the	
  review	
  was	
  intended	
  to	
  focus	
  on	
  programme	
  provisions	
  for	
  monitoring	
  and	
  evaluation,	
  the	
  progress	
  of	
  the	
  CGIAR	
  reform	
  process,	
  and	
  the	
  role	
  of	
  EC	
  funding	
  within	
  a	
  CRP.	
  The	
  review	
  focuses	
  on	
  the	
  West	
  African	
  region,	
  and	
  was	
  based	
  on	
  documentary	
  analysis,	
  face	
  to	
  face	
  and	
  telephone	
  interviews,	
  and	
  visits	
  to	
  the	
  CCAFS	
  Coordination	
  Unit	
  in	
  Copenhagen	
  and	
  field	
  site	
  at	
  Lawra-­‐Jirapa,	
  Ghana	
  (Page	
  1	
  -­‐	
  2).	
  The	
  importance	
  of	
  local	
  institutional	
  and	
  ecological	
  specificities	
  in	
  adaptation,	
  and	
  the	
  different	
  stages	
  of	
  development	
  at	
  pilot	
  sites,	
  findings	
  directly	
  related	
  to	
  the	
  field	
  site	
  visit	
  may	
  not	
  be	
  representative	
  of	
  the	
  other	
  regional	
  programmes.	
  However,	
  given	
  the	
  review’s	
  focus	
  on	
  program	
  level	
  activities,	
  the	
  team	
  believes	
  this	
  would	
  not	
  have	
  affected	
  the	
  final	
  conclusions	
  or	
  the	
  recommendations.	
  3. Established	
  as	
  a	
  Challenge	
  Programme	
  in	
  2009,	
  CCAFS	
  has	
  been	
  fast-­‐tracked	
  for	
  development	
  as	
  a	
  CRP.	
  It	
  is	
  organized	
  around	
  three	
  Regional	
  Programmes	
  and	
  four	
  Thematic	
  Programmes,	
  although	
  the	
  majority	
  of	
  funding	
  supports	
  157	
  activities	
  led	
  by	
  CGIAR	
  Centres	
  (Page	
  1),	
  most	
  of	
  which	
  pre-­‐existed	
  the	
  CRP.	
  4. Due	
  to	
  its	
  origins	
  in	
  partnership	
  between	
  the	
  CGIAR	
  and	
  the	
  Earth	
  Systems	
  Science	
  Partnership	
  (ESSP),	
  CCAFS	
  has	
  a	
  unique	
  structure	
  and	
  level	
  of	
  participation	
  by	
  non-­‐CGIAR	
  research	
  institutions.	
  Aspects	
  of	
  this	
  include	
  the	
  Independent	
  Science	
  Panel,	
  the	
  CCAFS	
  Coordination	
  Unit	
  partly	
  being	
  hosted	
  at	
  University	
  of	
  Copenhagen,	
  and	
  the	
  participation	
  of	
  three	
  universities	
  as	
  Thematic	
  Programme	
  Leaders.	
  (Page	
  1).	
  5. CCAFS’	
  Coordination	
  Unit	
  (CU)	
  is	
  partly	
  based	
  at	
  the	
  University	
  of	
  Copenhagen,	
  partly	
  at	
  CIAT	
  (S	
  America),	
  whilst	
  some	
  Thematic	
  Program	
  Leaders	
  are	
  based	
  at	
  Leeds,	
  Vermont,	
  and	
  Columbia	
  Universities.	
  CGIAR	
  institutes	
  host	
  Regional	
  and	
  other	
  Theme	
  Leaders,	
  including	
  ICRISAT	
  (W	
  Africa),	
  ILRI	
  (E	
  Africa),	
  and	
  IWMI	
  (S	
  Asia).	
  The	
  Theme	
  Leaders	
  and	
  the	
  Regional	
  Program	
  Leader	
  of	
  S	
  Asia,	
  along	
  with	
  the	
  Program	
  Director,	
  comprise	
  the	
  Program	
  Management	
  Committee	
  (PMC).	
  (Page	
  1)	
  6. CCAFS	
  has	
  in	
  place	
  processes	
  to	
  ensure	
  the	
  relevance	
  of	
  Thematic	
  and	
  Regional	
  Programme	
  activities	
  at	
  local,	
  national	
  and	
  regional	
  levels,	
  as	
  well	
  as	
  positioning	
  itself	
  in	
  terms	
  of	
  strategic	
  research.	
  However,	
  there	
  are	
  opportunities	
  to	
  improve	
  the	
  alignment	
  of	
  Window	
  3	
  and	
  bilaterally	
  funded	
  Centre	
  activities	
  with	
  strategic	
  priorities,	
  strengthen	
  focus	
  on	
  comparatively	
  under-­‐resourced	
  areas	
  within	
  the	
  portfolio,	
  and	
  capacitate	
  CCAFS	
  sites	
  teams	
  who	
  will	
  be	
  main	
  interlocutors	
  with	
  communities.	
  (Page	
  2	
  –	
  4;	
  Recommendations	
  1	
  –	
  6)	
  7. Despite	
  the	
  challenges	
  of	
  rapid	
  growth,	
  CCAFS	
  has	
  improved	
  on	
  its	
  past	
  financial	
  performance.	
  However,	
  the	
  programme	
  faced	
  delays	
  in	
  contracting,	
  confirmation	
  of	
  budgets,	
  and	
  receipt	
  of	
  funds	
  due	
  to	
  uncertainties	
  surrounding	
  the	
  CGIAR	
  reform	
  process	
  and	
  a	
  diversity	
  of	
  management	
  systems	
  employed	
  across	
  the	
  CGIAR.	
  This	
  increased	
  transaction	
  costs	
  for	
  the	
  programme,	
  and	
  led	
  to	
  two	
  non-­‐CGIAR	
  Thematic	
  Programme	
  Leaders	
  to	
  hand	
  back	
  financial	
  administration	
  to	
  CGIAR	
  Centres.	
  Although	
  the	
  financial	
  situation	
  is	
  improving,	
  
	
   iv	
  
more	
  can	
  be	
  done	
  at	
  the	
  CGIAR	
  level	
  to	
  streamline	
  administration	
  and	
  financial	
  management.	
  (Pages	
  5	
  –	
  6;	
  Recommendations	
  7	
  –	
  11)	
  8. CCAFS	
  has	
  a	
  range	
  of	
  processes	
  in	
  place	
  to	
  ensure	
  effective	
  programme	
  delivery,	
  including	
  a	
  novel	
  governance	
  institution	
  in	
  the	
  form	
  of	
  the	
  ISP.	
  The	
  development	
  of	
  M&E	
  tools	
  is	
  a	
  significant	
  aspect	
  of	
  the	
  CCAFS	
  work	
  plan,	
  although	
  a	
  focus	
  on	
  quantitative	
  techniques	
  should	
  be	
  balanced	
  by	
  qualitative	
  methods	
  which	
  can	
  strengthen	
  understanding	
  of	
  adaptive	
  decision-­‐making	
  processes.	
  Similarly,	
  acceleration	
  of	
  activities	
  at	
  CCAFS	
  sites	
  to	
  ensure	
  the	
  priorities	
  of	
  farmers	
  are	
  addressed	
  in	
  research	
  should	
  be	
  a	
  priority.	
  (Pages	
  6	
  –	
  9,	
  11;	
  Recommendations	
  12	
  –	
  18)	
  9. The	
  programme	
  LogFrame	
  is	
  internally	
  consistent,	
  with	
  a	
  logical	
  hierarchical	
  structure.	
  Indicators	
  can	
  be	
  substantially	
  improved	
  in	
  terms	
  of	
  specificity,	
  measurability,	
  and	
  achievability,	
  particularly	
  if	
  the	
  LogFrame	
  is	
  to	
  act	
  as	
  a	
  framework	
  for	
  accountability	
  as	
  well	
  as	
  programme	
  planning.	
  (Pages	
  9	
  –	
  11;	
  Recommendation	
  19)	
  10. The	
  development	
  of	
  longer-­‐term	
  workplans	
  and	
  contracts	
  for	
  CCAFS	
  sites	
  would	
  reduce	
  transaction	
  costs	
  and	
  strengthen	
  integration	
  between	
  Regional	
  and	
  Thematic	
  Programmes.	
  This	
  would	
  also	
  reduce	
  burdens	
  on	
  participating	
  country	
  partners.	
  (Page	
  12;	
  Recommendation	
  20)	
  11. At	
  this	
  stage	
  there	
  are	
  positive	
  signs	
  of	
  engagement	
  between	
  CCAFS	
  and	
  other	
  CRPs,	
  and	
  this	
  may	
  be	
  an	
  area	
  for	
  a	
  specific	
  future	
  evaluation	
  exercise	
  (Page	
  13;	
  Recommendation	
  21)	
  12. The	
  review	
  has	
  indicated	
  the	
  need	
  for	
  increased	
  harmonisation	
  between	
  Centres	
  and	
  CRPs	
  of	
  policies	
  and	
  procedures,	
  although	
  with	
  sufficient	
  flexibility	
  to	
  account	
  for	
  specific	
  needs.	
  The	
  location	
  of	
  CRPs	
  within	
  host	
  Centres	
  may	
  limit	
  the	
  potential	
  for	
  enhanced	
  performance	
  across	
  the	
  CGIAR,	
  and	
  locating	
  the	
  CRPs	
  under	
  the	
  Consortium	
  Office	
  (CO)	
  could	
  improve	
  performance,	
  and	
  the	
  CO	
  can	
  also	
  do	
  more	
  to	
  incentivise	
  participation	
  of	
  non-­‐CGIAR	
  research	
  institutions	
  in	
  CRPs.	
  The	
  Review	
  Team	
  also	
  noted	
  the	
  potential	
  tension	
  between	
  an	
  increased	
  focus	
  on	
  achieving	
  development	
  impact	
  and	
  the	
  CGIAR’s	
  mandate	
  for	
  high	
  quality	
  research.	
  (Pages	
  13	
  –	
  16;	
  Recommendations	
  22	
  –	
  27)	
  13. The	
  development	
  of	
  the	
  CGIAR’s	
  Strategy	
  and	
  Results	
  Framework	
  to	
  prioritise	
  the	
  allocation	
  of	
  research	
  resources	
  towards	
  development	
  outcomes	
  is	
  welcomed.	
  However,	
  the	
  Review	
  Team	
  has	
  deep	
  concerns	
  that	
  attempting	
  to	
  rigorously	
  address	
  issues	
  of	
  quantitative	
  impact	
  assessment	
  will	
  divert	
  resources	
  and	
  intellectual	
  energy	
  of	
  programme	
  staff	
  from	
  substantive	
  research	
  priorities.	
  The	
  Review	
  Team	
  is	
  also	
  mindful	
  of	
  the	
  potential	
  tension	
  between	
  the	
  innovative,	
  risky	
  nature	
  of	
  research	
  and	
  the	
  tendency	
  of	
  quantitative	
  accountability	
  frameworks	
  to	
  incentivize	
  low	
  risk	
  behaviour.	
  (Pages	
  16	
  –	
  17;	
  Recommendations	
  28	
  –	
  30)	
  14. Current	
  EC/IFAD	
  funding	
  arrangements	
  do	
  not	
  have	
  a	
  significant	
  impact	
  on	
  CCAFS	
  programming	
  directions.	
  However,	
  delays	
  in	
  disbursing	
  funds	
  do	
  add	
  to	
  the	
  programme’s	
  financial	
  uncertainty	
  and	
  pose	
  a	
  reputational	
  risk	
  to	
  EC/IFAD.	
  If	
  the	
  EC	
  wishes	
  to	
  increase	
  influence	
  on	
  CCAFS	
  programming,	
  it	
  should	
  consider	
  other	
  means	
  such	
  as	
  participation	
  in	
  stakeholder	
  meetings,	
  the	
  provision	
  of	
  parallel	
  support	
  to	
  EU	
  research	
  institutions	
  to	
  encourage	
  their	
  participation	
  in	
  CRPs,	
  or	
  backstopping	
  support	
  to	
  the	
  Consortium	
  Office	
  to	
  facilitate	
  partnership	
  mechanisms.	
  	
  (Pages	
  17	
  –	
  19;	
  Recommendations	
  31	
  –	
  34)	
  15. CCAFS	
  is	
  a	
  strategic	
  programme	
  for	
  the	
  EC,	
  IFAD	
  and	
  the	
  CGIAR.	
  Despite	
  challenges	
  in	
  the	
  first	
  year	
  of	
  operation	
  as	
  a	
  CRP	
  and	
  some	
  remaining	
  scientific	
  
	
   v	
  
and	
  managerial	
  challenges,	
  significant	
  progress	
  has	
  been	
  made.	
  The	
  programme	
  is	
  on	
  course	
  and	
  is	
  managed	
  by	
  highly	
  competent	
  professional	
  staff.	
  The	
  EC	
  should	
  maintain	
  current	
  levels	
  of	
  funding,	
  and	
  consider	
  increased	
  financial	
  support	
  in	
  line	
  with	
  CCAFS’	
  capacity	
  to	
  absorb	
  additional	
  funding.	
  	
  
	
   1	
  
1.	
  Introduction	
  This	
  report	
  presents	
  a	
  review	
  of	
  the	
  EC/IFAD	
  contribution	
  to	
  the	
  CGIAR	
  Research	
  Program	
  (CRP)	
  “Climate	
  Change,	
  Agriculture	
  and	
  Food	
  Security”	
  (CCAFS),	
  also	
  known	
  as	
  CRP7.	
  The	
  review	
  was	
  conducted	
  during	
  August	
  and	
  September	
  2012,	
  based	
  on	
  visits	
  to	
  CCAFS	
  headquarters	
  in	
  Copenhagen	
  and	
  field	
  sites	
  in	
  Lawra-­‐Jirapa,	
  Ghana,	
  telephone	
  interviews,	
  and	
  written	
  documentation.	
  Full	
  lists	
  of	
  persons	
  interviewed	
  and	
  literature	
  reviewed	
  are	
  given	
  in	
  Annex	
  3.	
  	
  CCAFS	
  CCAFS	
  began	
  as	
  a	
  Challenge	
  Programme	
  in	
  2009,	
  a	
  collaboration	
  between	
  the	
  CGIAR	
  and	
  the	
  Earth	
  Systems	
  Science	
  Partnership	
  (ESSP).	
  The	
  nature	
  of	
  the	
  collaboration	
  led	
  to	
  a	
  unique	
  structure	
  and	
  level	
  of	
  participation	
  by	
  non-­‐CGIAR	
  organisations.	
  CCAFS	
  is	
  organized	
  around	
  three	
  Regional	
  Programmes	
  (West	
  Africa,	
  East	
  Africa,	
  and	
  South	
  Asia)	
  and	
  four	
  Thematic	
  Programmes:	
  Adaptation	
  to	
  Progressive	
  Climate	
  Change	
  (Theme	
  1),	
  Adaptation	
  through	
  Managing	
  Climate	
  Risk	
  (Theme	
  2),	
  Pro-­‐poor	
  Climate	
  Mitigation	
  (Theme	
  3),	
  and	
  Integration	
  for	
  Decision	
  Making	
  (Theme	
  4).	
  CCAFS’	
  progamme	
  Coordination	
  Unit	
  (CU)	
  is	
  partly	
  based	
  at	
  the	
  University	
  of	
  Copenhagen,	
  partly	
  at	
  CIAT	
  (S	
  America),	
  whilst	
  some	
  Thematic	
  Program	
  Leaders	
  are	
  based	
  at	
  Leeds,	
  Vermont,	
  and	
  Columbia	
  Universities.	
  CGIAR	
  institutes	
  host	
  Regional	
  and	
  other	
  Theme	
  Leaders,	
  including	
  ICRISAT	
  (W	
  Africa),	
  ILRI	
  (E	
  Africa),	
  and	
  IWMI	
  (S	
  Asia).	
  The	
  Theme	
  Leaders	
  and	
  the	
  Regional	
  Program	
  Leader	
  of	
  S	
  Asia,	
  along	
  with	
  the	
  Program	
  Director,	
  comprise	
  the	
  Programme	
  Management	
  Committee	
  (PMC).	
  	
  In	
  February	
  2011	
  CCAFS	
  was	
  approved	
  as	
  a	
  Consortium	
  Research	
  Program	
  (CRP),	
  with	
  a	
  total	
  project	
  budget	
  of	
  USD	
  392.5	
  million	
  for	
  Phase	
  1	
  (2011-­‐15).	
  CCAFS	
  is	
  now	
  the	
  focus	
  for	
  collaboration	
  on	
  climate	
  change	
  by	
  all	
  15	
  CGIAR	
  Centres.	
  Whereas	
  most	
  CRPs	
  were	
  established	
  under	
  a	
  lead	
  Centre	
  from	
  their	
  inception,	
  CRP7	
  was	
  instead	
  designed	
  by	
  the	
  CCAFS	
  PMC	
  and	
  the	
  previous	
  Scientific	
  Steering	
  Committee	
  for	
  the	
  Challenge	
  Program,	
  and	
  then	
  CIAT	
  won	
  a	
  competition	
  to	
  host	
  it.	
  	
  CCAFS	
  grew	
  from	
  USD	
  14	
  million	
  as	
  a	
  Challenge	
  Programme	
  in	
  2010	
  to	
  USD	
  62	
  million	
  as	
  a	
  CRP	
  in	
  2011.The	
  core	
  Challenge	
  Programme	
  research	
  activities	
  continued	
  in	
  the	
  Thematic	
  and	
  Regional	
  Programmes,	
  with	
  2011	
  budgets	
  remaining	
  roughly	
  stable	
  at	
  USD	
  15	
  million.	
  However,	
  transition	
  to	
  a	
  CRP	
  added	
  a	
  third	
  category	
  of	
  programming,	
  the	
  CGIAR	
  Centre-­‐led	
  activities,	
  which	
  amounted	
  to	
  USD	
  42	
  million	
  in	
  2011,	
  all	
  of	
  which	
  is	
  expected	
  to	
  contributed	
  to	
  one	
  or	
  more	
  of	
  CCAFS’	
  four	
  themes.	
  Approximately	
  USD	
  17.7	
  million	
  of	
  these	
  activities	
  were	
  supported	
  by	
  bilateral	
  funding,	
  which	
  recipient	
  Centres	
  then	
  ‘contribute’	
  to	
  CCAFS.	
  The	
  remaining	
  USD	
  24.7million,	
  funded	
  through	
  Windows	
  1&	
  2,	
  was	
  allocated	
  by	
  CCAFS	
  to	
  support	
  Centre-­‐led	
  activities	
  on	
  a	
  competitive	
  basis.	
  The	
  greatly	
  increased	
  scale	
  and	
  operations	
  of	
  CCAFS	
  has	
  created	
  new	
  challenges	
  for	
  the	
  management	
  team,	
  especially	
  as	
  the	
  scaling	
  up	
  has	
  occurred	
  in	
  the	
  context	
  of	
  the	
  CGIAR	
  reform	
  process.	
  	
  These	
  factors	
  –	
  CCAFS	
  prior	
  existence	
  as	
  a	
  Challenge	
  Program,	
  the	
  strategic	
  participation	
  of	
  non-­‐CGIAR	
  research	
  institutions,	
  and	
  the	
  competitive	
  process	
  to	
  select	
  the	
  host	
  of	
  CRP7,	
  make	
  CCAFS	
  unlike	
  other	
  CRPs.	
  This	
  gives	
  insights	
  into	
  the	
  CG	
  reform	
  processes	
  precisely	
  because	
  it	
  is	
  non-­‐typical.	
  	
  	
  The	
  Review	
  	
  This	
  review	
  was	
  commissioned	
  to	
  evaluate	
  EC/IFAD	
  contributions	
  to	
  the	
  CCAFS	
  CRP	
  in	
  2011.	
  This	
  was	
  budgeted	
  at	
  €2.17	
  million	
  valued,	
  down	
  from	
  €4.9	
  million	
  in	
  2010	
  under	
  the	
  Challenge	
  Programme.	
  As	
  CCAFS	
  began	
  relatively	
  recently,	
  the	
  review	
  is	
  framed	
  around	
  evaluation	
  of	
  processes	
  and	
  tools	
  rather	
  than	
  observed	
  impact.	
  In	
  particular	
  the	
  review	
  was	
  intended	
  to	
  focus	
  on	
  the	
  adequacy	
  of	
  the	
  program	
  monitoring	
  and	
  
	
   2	
  
evaluation	
  framework,	
  assessing	
  the	
  quality	
  of	
  indicators	
  and	
  baselines,	
  proposed	
  system-­‐wide	
  CGIAR	
  M&E	
  criteria,	
  the	
  progress	
  of	
  the	
  CGIAR	
  reform	
  process,	
  and	
  the	
  role	
  of	
  EC	
  funding	
  within	
  CCAFS.	
  Terms	
  of	
  Reference	
  for	
  the	
  review	
  can	
  be	
  found	
  in	
  Annex	
  1.	
  This	
  review	
  has	
  limited	
  scope	
  due	
  to	
  restricted	
  time	
  and	
  the	
  focus	
  on	
  the	
  West	
  African	
  Regional	
  Programme.	
  Activities	
  in	
  Asia	
  and	
  East	
  Africa	
  were	
  not	
  reviewed,	
  nor	
  was	
  any	
  Centre-­‐led	
  activity,	
  and	
  coverage	
  of	
  the	
  Thematic	
  Programmes	
  was	
  light.	
  As	
  a	
  result	
  the	
  review	
  is	
  necessarily	
  a	
  snapshot	
  rather	
  than	
  a	
  comprehensive	
  assessment.	
  In	
  addition,	
  the	
  planned	
  inclusion	
  of	
  a	
  third	
  review	
  team	
  member	
  specialised	
  in	
  partnership	
  evaluation	
  did	
  not	
  come	
  to	
  pass.	
  	
  The	
  standardized	
  reporting	
  template	
  does	
  not	
  entirely	
  equate	
  with	
  these	
  review	
  objectives.	
  Section	
  2	
  presents	
  evidence,	
  conclusions	
  and	
  recommendations	
  from	
  the	
  review	
  of	
  CCAFS	
  in	
  a	
  logical	
  manner	
  within	
  the	
  standard	
  reporting	
  template.	
  Section	
  3	
  presents	
  reflections	
  on	
  strategic	
  issues:	
  the	
  CGIAR	
  reform	
  process,	
  Consortium	
  Office	
  plans	
  for	
  M&E,	
  and	
  EC/IFAD	
  engagement	
  with	
  CRPs.	
  Section	
  4	
  offers	
  conclusions	
  and	
  recommendations.	
  Recommendations	
  presented	
  in	
  Section	
  4	
  are	
  numerically	
  cross-­‐referenced	
  in	
  the	
  main	
  text	
  so:	
  (R1)	
  2	
  Review	
  Criteria	
  2.1	
  Relevance	
  Programme	
  relevance	
  CCAFS	
  is	
  structured	
  around	
  both	
  thematic	
  and	
  regional	
  programs.	
  CCAFS’	
  thematic	
  foci	
  are	
  logical	
  and	
  relevant.	
  Progressive	
  adaptation	
  to	
  climate	
  change	
  (Theme	
  1)	
  is	
  a	
  significant	
  long	
  term	
  challenge.	
  CCAFS	
  is	
  initially	
  focusing	
  on	
  the	
  year	
  2030.	
  This	
  is	
  a	
  year	
  not	
  too	
  distant	
  to	
  be	
  irrelevant	
  to	
  policymakers,	
  yet	
  far	
  enough	
  that	
  significant	
  climatic	
  changes	
  can	
  be	
  expected.	
  Adaptation	
  to	
  climate	
  risk	
  (Theme	
  2)	
  is	
  an	
  immediate	
  need,	
  with	
  increased	
  climate	
  variability	
  already	
  affecting	
  rain-­‐fed	
  agriculture,	
  particularly	
  in	
  the	
  Sahel.	
  Mitigation	
  in	
  smallholding	
  agriculture	
  (Theme	
  3)	
  is	
  an	
  attempt	
  to	
  link	
  small	
  farmers	
  with	
  the	
  international	
  carbon	
  trade	
  and	
  other	
  incentive	
  systems	
  for	
  low	
  carbon	
  development.	
  This	
  is	
  significant	
  both	
  as	
  a	
  means	
  of	
  reducing	
  emissions	
  from	
  agriculture,	
  and	
  also	
  for	
  its	
  potential	
  to	
  diversify	
  farmers’	
  incomes.	
  Theme	
  4	
  is	
  cross-­‐cutting,	
  and	
  focuses	
  on	
  linking	
  knowledge	
  with	
  action,	
  developing	
  tools,	
  and	
  developing	
  policy	
  platforms,	
  all	
  of	
  which	
  are	
  crucial	
  elements	
  of	
  adaptive	
  capacity.	
  The	
  three	
  regions	
  initially	
  selected	
  for	
  the	
  Challenge	
  Programme	
  are	
  highly	
  relevant	
  to	
  Themes	
  1	
  and	
  2.	
  West	
  and	
  East	
  Africa	
  and	
  South	
  Asia	
  are	
  climate	
  vulnerability	
  hotspots	
  due	
  to	
  the	
  low	
  adaptive	
  capacity,	
  reliance	
  on	
  rainfed	
  agriculture,	
  and	
  the	
  impacts	
  of	
  climate	
  change	
  and	
  increased	
  climate	
  variability.	
  Within	
  regions,	
  sites	
  were	
  identified	
  in	
  consultation	
  with	
  regional	
  and	
  national	
  partners	
  based	
  on	
  poverty	
  indices,	
  ensuring	
  a	
  representative	
  sample	
  of	
  agroecosystems,	
  and	
  building	
  on	
  previous	
  CGIAR	
  research.	
  	
  An	
  extensive	
  process	
  to	
  identify	
  two	
  additional	
  areas	
  for	
  expansion	
  in	
  2012	
  has	
  selected	
  South	
  East	
  Asia	
  and	
  Southern	
  Central	
  America/Northern	
  South	
  America.	
  These	
  add	
  regions	
  offering	
  improved	
  opportunities	
  for	
  work	
  in	
  Theme	
  3.	
  The	
  process	
  of	
  identification	
  was	
  consultative	
  and	
  transparent,	
  including	
  inputs	
  from	
  all	
  CGIAR	
  Centres	
  and	
  the	
  Independent	
  Science	
  Panel	
  (ISP).	
  	
  During	
  the	
  programme	
  design	
  process,	
  a	
  stakeholder	
  validation	
  process	
  launched	
  in	
  Nairobi	
  during	
  May	
  2010	
  to	
  engage	
  with	
  researchers,	
  farmers	
  representatives,	
  civil	
  society,	
  policymakers,	
  and	
  private	
  sector	
  agri-­‐businesses.	
  	
  Since	
  the	
  launch	
  of	
  CCAFS	
  in	
  2011,	
  regional	
  program	
  leaders	
  have	
  continued	
  to	
  consult	
  with	
  regional	
  and	
  national	
  actors	
  in	
  order	
  to	
  ensure	
  CCAFS	
  activities	
  are	
  aligned	
  with	
  national	
  needs	
  and	
  priorities.	
  
	
   3	
  
The	
  process	
  in	
  E	
  Africa	
  is	
  documented	
  on	
  CCAFS	
  website1,	
  and	
  similar	
  processes	
  have	
  also	
  occurred	
  in	
  W	
  Africa.	
  	
  Thematic	
  balance	
  Approximately	
  two	
  third	
  of	
  CCAFS’	
  budget	
  is	
  managed	
  through	
  Centre-­‐led	
  activities,	
  either	
  through	
  bilateral	
  funding	
  (much	
  of	
  it	
  inherited	
  from	
  pre-­‐CCAFS	
  arrangements)	
  or	
  competitively	
  granted	
  Window	
  1&2	
  funds	
  (see	
  Section	
  1).	
  An	
  evaluation	
  by	
  CCAFS	
  Theme	
  Leaders	
  of	
  157	
  Centre-­‐led	
  activities	
  supported	
  judged	
  that	
  for	
  2012,	
  60%	
  of	
  these	
  activities	
  were	
  highly	
  strategic	
  to	
  the	
  CCAFS	
  declared	
  objectives,	
  29%	
  were	
  partially	
  strategic,	
  and	
  11%	
  were	
  not	
  strategic.	
  Partially	
  and	
  non-­‐strategic	
  activities	
  included	
  numerous	
  studies	
  looking	
  at	
  coping	
  mechanisms	
  and	
  vulnerability	
  lacking	
  forward-­‐looking	
  components	
  and	
  weakly	
  connected	
  to	
  impact	
  pathways.	
  Some	
  Centres	
  were	
  making	
  more	
  strategic	
  contributions	
  than	
  others,	
  with	
  the	
  best	
  performing	
  having	
  89%	
  of	
  activities	
  with	
  high	
  strategic	
  value	
  and	
  the	
  worst	
  having	
  75%	
  of	
  activities	
  with	
  no	
  strategic	
  value.	
  	
  Additionally,	
  as	
  a	
  consequence	
  of	
  the	
  programme	
  budget	
  being	
  allocated	
  in	
  response	
  to	
  proposals	
  from	
  Centres,	
  funding	
  and	
  activities	
  are	
  not	
  distributed	
  evenly	
  amongst	
  themes.	
  For	
  example,	
  in	
  2011,	
  USD	
  21.2	
  million	
  was	
  allocated	
  to	
  Theme	
  1	
  (Progressive	
  Adaptation)	
  by	
  comparison	
  to	
  USD	
  8.3	
  million	
  for	
  Theme	
  2	
  (Climate	
  Risk	
  Management).	
  Traditional	
  strengths	
  of	
  CGIAR	
  Centres	
  in	
  crop	
  breeding	
  and	
  technology	
  development	
  makes	
  Theme	
  1	
  more	
  accessible,	
  but	
  Theme	
  2	
  addresses	
  institutional	
  innovations	
  which	
  are	
  immediate	
  priorities	
  for	
  communities	
  and	
  policy	
  makers2.	
  	
  As	
  the	
  focus	
  of	
  CCAFS	
  is	
  on	
  technologies	
  and	
  institutions	
  to	
  face	
  changing	
  climatic	
  conditions	
  over	
  the	
  next	
  20	
  years,	
  a	
  direct	
  impact	
  on	
  food	
  security	
  now	
  is	
  not	
  -­‐and	
  should	
  not	
  be	
  -­‐	
  a	
  primary	
  research	
  objective.	
  However,	
  both	
  the	
  regional	
  and	
  thematic	
  programmes	
  address	
  most	
  technical	
  and	
  economic	
  –	
  in	
  short	
  the	
  productivity	
  -­‐	
  aspects	
  of	
  food	
  security	
  in	
  the	
  future.	
  The	
  non-­‐productivity	
  aspects	
  of	
  food	
  security	
  currently	
  less	
  well	
  addressed,	
  especially	
  with	
  regard	
  to	
  nutrition,	
  but	
  this	
  may	
  be	
  corrected	
  when	
  the	
  programme	
  grows	
  in	
  size.	
  An	
  attempt	
  should	
  however	
  be	
  made	
  to	
  strengthen	
  these	
  objectives	
  for	
  in	
  the	
  program	
  plan	
  and	
  LogFrame.	
  The	
  level	
  of	
  ambition	
  will	
  need	
  to	
  be	
  related	
  to	
  an	
  achievement	
  strategy	
  that	
  both	
  draws	
  on	
  outside	
  expertise	
  and	
  strengthens	
  capability	
  within	
  the	
  CGIAR	
  on	
  food	
  security.	
  (R1)	
  CCAFS	
  CU	
  reported	
  that	
  they	
  were	
  trying	
  to	
  establish	
  ways	
  of	
  improving	
  alignment	
  of	
  Centre	
  activities	
  with	
  the	
  program	
  plan,	
  with	
  some	
  Centres	
  being	
  more	
  responsive	
  to	
  feedback	
  than	
  others.	
  The	
  strategic	
  value	
  evaluation	
  exercise	
  will	
  be	
  repeated	
  next	
  year,	
  with	
  Theme	
  and	
  Regional	
  Leaders	
  and	
  the	
  ISP	
  contributing	
  to	
  the	
  evaluation.	
  They	
  hope	
  to	
  move	
  a	
  model	
  whereby	
  projects	
  well	
  aligned	
  with	
  CCAFS	
  priorities,	
  including	
  gender	
  and	
  communication,	
  could	
  receive	
  top-­‐up	
  funding	
  from	
  W1	
  and	
  W2.	
  (R2,	
  R3,	
  &	
  R4)	
  Local	
  relevance	
  At	
  CCAFS	
  sites,	
  participatory	
  approaches	
  have	
  been	
  used	
  to	
  engage	
  with	
  communities.	
  The	
  review	
  mission	
  to	
  the	
  site	
  of	
  Lawra-­‐Jirapa	
  in	
  Ghana	
  found	
  that	
  the	
  development	
  of	
  a	
  local	
  project	
  support	
  unit	
  (PSU),	
  comprised	
  of	
  researchers,	
  agricultural	
  extension	
  officers,	
  NGOs,	
  village	
  chiefs	
  and	
  assemblymen,	
  had	
  made	
  a	
  positive	
  contribution	
  to	
  building	
  trust	
  relations	
  and	
  interactions	
  with	
  the	
  community.	
  Execution	
  of	
  baseline	
  studies	
  and	
  participatory	
  dialogues	
  including	
  gender-­‐differentiated	
  vulnerability	
  assessment	
  had	
  developed	
  a	
  list	
  of	
  key	
  issues	
  for	
  the	
  research	
  program	
  to	
  address.	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  1http://ccafs.cgiar.org/blog/responding-­‐research-­‐needs-­‐climate-­‐change-­‐and-­‐agriculture-­‐east-­‐africa	
  2	
  For	
  example,	
  Objective	
  1.3	
  (species	
  and	
  genetic	
  diversity	
  for	
  climate	
  change)	
  was	
  allocated	
  USD	
  7.6	
  million,	
  by	
  contrast	
  to	
  Objective	
  2.2	
  (Managing	
  climate	
  risk	
  through	
  food	
  delivery,	
  trade	
  and	
  crisis	
  response)	
  which	
  was	
  allocated	
  USD	
  0.7	
  million.	
  
	
   4	
  
It	
  was	
  notable,	
  however,	
  that	
  the	
  needs	
  stressed	
  by	
  the	
  community	
  and	
  PSU	
  during	
  our	
  visit	
  reflected	
  current	
  concerns	
  with	
  agricultural	
  development	
  and	
  food	
  security,	
  rather	
  than	
  those	
  informed	
  by	
  future	
  climate	
  scenarios.	
  The	
  Lawra-­‐Jirapa	
  site	
  is	
  not	
  the	
  most	
  advanced	
  in	
  W	
  Africa,	
  and	
  the	
  review	
  mission	
  took	
  place	
  shortly	
  before	
  a	
  planned	
  visit	
  by	
  farmers	
  to	
  a	
  site	
  in	
  Burkina	
  Faso	
  identified	
  as	
  a	
  climate	
  analogue	
  for	
  Lawra-­‐Jirapa	
  in	
  the	
  year	
  20303.	
  This	
  visit,	
  a	
  key	
  part	
  of	
  the	
  project	
  methodology,	
  should	
  make	
  a	
  significant	
  difference	
  to	
  the	
  ability	
  of	
  local	
  communities	
  to	
  frame	
  knowledge	
  needs	
  in	
  terms	
  of	
  climate	
  change.	
  Despite	
  these	
  caveats,	
  the	
  review	
  mission	
  did	
  underline	
  the	
  difficulty	
  of	
  making	
  research	
  focused	
  on	
  progressive	
  adaptation	
  to	
  climate	
  change	
  responsive	
  to	
  community-­‐defined	
  needs,	
  which	
  will	
  inevitably	
  stress	
  immediate	
  priorities.	
  CCAFS	
  should	
  capacitate	
  the	
  country	
  core	
  team	
  and	
  PSU	
  on	
  climate	
  change,	
  particularly	
  as	
  they	
  are	
  the	
  main	
  interlocutors	
  with	
  the	
  community.	
  For	
  example,	
  in	
  response	
  to	
  dialogue	
  with	
  the	
  community,	
  the	
  PSU	
  had	
  identified	
  tree	
  planting	
  as	
  a	
  means	
  of	
  building	
  engagement	
  as	
  well	
  as	
  diversifying	
  agriculture.	
  The	
  PSU	
  identified	
  a	
  number	
  of	
  species	
  from	
  which	
  farmers	
  chose,	
  and	
  to	
  date	
  1646	
  mango,	
  tamarind,	
  cashew	
  and	
  mahogany	
  saplings	
  have	
  been	
  provided	
  to	
  farmers	
  and	
  planted.	
  However,	
  it	
  had	
  apparently	
  not	
  occurred	
  to	
  the	
  PSU	
  to	
  seek	
  advice	
  from	
  CCAFS	
  on	
  which	
  trees	
  would	
  be	
  appropriate	
  under	
  future	
  climate	
  scenarios.	
  (Such	
  advice	
  should	
  be	
  coming	
  online	
  through	
  the	
  development	
  of	
  tools	
  under	
  Theme	
  4	
  (see,	
  e.g.,	
  Section	
  3.1.2),	
  but	
  these	
  are	
  not	
  yet	
  available.	
  	
  Similarly,	
  capacitated	
  PSU	
  members	
  and	
  country	
  researchers	
  should	
  be	
  able	
  to	
  identify	
  areas	
  for	
  complementary	
  research	
  to	
  foresee	
  the	
  systemic	
  transformations	
  of	
  livelihoods	
  and	
  mitigate	
  against	
  potentially	
  negative	
  outcomes	
  of	
  agricultural	
  adaptations.	
  In	
  Lawra-­‐Jirapa,	
  water	
  conservation	
  and	
  storage	
  for	
  dry	
  season	
  agriculture	
  and	
  intensification	
  of	
  the	
  lowlands	
  has	
  been	
  identified	
  as	
  one	
  potential	
  adaptation.	
  Due	
  to	
  the	
  close	
  linkages	
  between	
  water	
  management	
  regimes	
  and	
  vector	
  borne	
  disease,	
  any	
  actions	
  in	
  this	
  direction	
  should	
  be	
  accompanied	
  by	
  research	
  on	
  the	
  effects	
  of	
  climate	
  and	
  land	
  use	
  change	
  on	
  human	
  and	
  livestock	
  health.	
  	
  The	
  role	
  of	
  different	
  Centres	
  in	
  site	
  activities	
  was	
  not	
  yet	
  obvious.	
  With	
  the	
  identification	
  of	
  new,	
  climate	
  resilient	
  cropping	
  systems	
  with	
  potential	
  roles	
  in	
  mitigation	
  remaining	
  to	
  be	
  done,	
  Centres	
  with	
  experience	
  in	
  these	
  issues4	
  can	
  obviously	
  play	
  a	
  greater	
  role.	
  Similar	
  remarks	
  could	
  be	
  made	
  about	
  two	
  other	
  challenges	
  identified	
  in	
  the	
  initial	
  baseline	
  and	
  participatory	
  appraisal;	
  (i)	
  the	
  question	
  of	
  the	
  regeneration	
  of	
  the	
  tree	
  population	
  of	
  the	
  bush,	
  specially	
  with	
  species	
  having	
  a	
  positive	
  impact	
  on	
  soils	
  fertility	
  and	
  water	
  holding	
  capacity	
  (such	
  as	
  Acacia	
  albida)	
  and	
  (ii)	
  the	
  adaptation	
  of	
  the	
  whole	
  cropping	
  systems	
  (beyond	
  variety)5	
  to	
  the	
  shortening	
  of	
  cropping	
  cycles	
  of	
  the	
  food	
  crops	
  (cereals	
  and	
  legumes	
  mostly).	
  	
  Increased	
  alignment	
  between	
  the	
  local	
  teams,	
  regional	
  and	
  thematic	
  program	
  teams,	
  and	
  Centre-­‐led	
  activities	
  at	
  CCAFS	
  sites	
  should	
  be	
  an	
  area	
  of	
  focus	
  for	
  CCAFS	
  in	
  the	
  coming	
  period.	
  CCAFS	
  could	
  help	
  accelerate	
  this	
  integration	
  by	
  capacitating	
  local	
  team	
  members,	
  and	
  training	
  them	
  to	
  recognize	
  potential	
  climate	
  relevant	
  and	
  research	
  questions	
  issues	
  for	
  input	
  from	
  CCAFS,	
  and	
  thereby	
  strengthen	
  the	
  programme’s	
  local	
  relevance.	
  (R5	
  &	
  R6)	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  3	
  The	
  visit	
  is	
  part	
  of	
  the	
  Theme	
  1	
  activity	
  “Climate	
  analogues:	
  Finding	
  tomorrow’s	
  agriculture	
  today”	
  (managed	
  by	
  CIAT),	
  which	
  has	
  developed	
  a	
  tool	
  connecting	
  sites	
  with	
  statistically	
  similar	
  climates,	
  across	
  both	
  space	
  and	
  time.	
  Farmer	
  exchanges	
  are	
  then	
  used	
  as	
  an	
  educational	
  tool,	
  giving	
  farmers	
  from	
  benchmark	
  sites	
  the	
  opportunity	
  to	
  envision	
  and	
  plan	
  future	
  agriculture.	
  	
  4	
  In	
  the	
  case	
  of	
  the	
  lowlands	
  and	
  inland	
  valleys	
  in	
  West	
  Africa,	
  see	
  particularly	
  AfricaRice.	
  5	
  Plant	
  density,	
  mode	
  of	
  sowing,	
  fertilisation	
  
	
   5	
  
2.2.	
  Efficiency	
  The	
  timing	
  of	
  this	
  review	
  meant	
  that	
  we	
  did	
  not	
  assess	
  results	
  stemming	
  from	
  activities.	
  However,	
  we	
  did	
  examine	
  some	
  issues	
  related	
  to	
  efficiency	
  in	
  terms	
  of	
  financial	
  management.	
  	
  Under-­‐spending	
  In	
  2011	
  CCAFS	
  executed	
  96%	
  of	
  its	
  €2.17	
  million	
  EC/IFAD	
  budget,	
  with	
  remainder	
  set	
  for	
  expenditure	
  by	
  August	
  2012.	
  Total	
  under-­‐spending	
  for	
  CCAFS	
  was	
  USD	
  5.7	
  million,	
  or	
  9%.	
  Centre-­‐led	
  activities	
  (through	
  Windows	
  1&2	
  and	
  bilateral)	
  underspent	
  by	
  USD	
  2.5	
  million	
  (6%),	
  whilst	
  the	
  Regional	
  and	
  Thematic	
  Programmes	
  underspent	
  by	
  USD	
  2.7	
  million	
  (18%).	
  This	
  is	
  an	
  improvement	
  from	
  the	
  level	
  of	
  37%	
  under-­‐spending	
  during	
  the	
  Challenge	
  Programme	
  in	
  2010.	
  The	
  2011	
  level	
  of	
  under-­‐spending	
  is	
  thought	
  to	
  be	
  due	
  to	
  challenges	
  in	
  contracting,	
  receiving	
  and	
  disbursing	
  funds	
  (see	
  below),	
  and	
  other	
  management	
  challenges	
  incurred	
  by	
  the	
  transition	
  to	
  the	
  CRP	
  structure	
  and	
  the	
  CGIAR	
  reform	
  process.	
  As	
  these	
  management	
  challenges	
  are	
  overcome,	
  the	
  activities	
  –	
  and	
  hence	
  spending	
  –	
  of	
  CCAFS’	
  Regional	
  and	
  Thematic	
  Programmes	
  should	
  expand	
  to	
  fill	
  their	
  substantive	
  objectives.	
  Contracting	
  challenges	
  and	
  funding	
  uncertainty	
  The	
  contract	
  between	
  the	
  Fund	
  Council	
  and	
  the	
  Consortium	
  Office	
  was	
  not	
  signed	
  until	
  May	
  2011,	
  and	
  the	
  Program	
  Implementation	
  Agreement	
  between	
  the	
  Consortium	
  Office	
  and	
  CIAT	
  was	
  not	
  signed	
  until	
  July	
  2011.	
  Further	
  legal	
  questions	
  meant	
  that	
  contracts	
  between	
  CIAT	
  and	
  participating	
  Centres	
  were	
  not	
  finalized	
  until	
  December	
  2011.	
  Now	
  that	
  contracts	
  are	
  in	
  place	
  there	
  are	
  no	
  reported	
  problems	
  with	
  CIAT	
  disbursing	
  funds	
  once	
  they	
  have	
  been	
  received	
  from	
  the	
  Consortium	
  Office.	
  However,	
  CCAFS	
  has	
  experienced	
  delays	
  in	
  receiving	
  confirmation	
  of	
  Window	
  1	
  and	
  Window	
  2	
  budgets,	
  with	
  consequent	
  challenges	
  in	
  programme	
  management.	
  As	
  an	
  extreme	
  example,	
  the	
  2011	
  CCAFS	
  budget	
  for	
  Windows	
  1	
  and	
  2	
  was	
  not	
  confirmed	
  until	
  December	
  2011.	
  Further,	
  CCAFS	
  did	
  not	
  receive	
  confirmation	
  until	
  March	
  2012	
  as	
  to	
  how	
  funds	
  carried	
  over	
  from	
  2011	
  into	
  2012	
  would	
  affect	
  the	
  2012	
  budget	
  allocation.	
  However,	
  it	
  is	
  likely	
  that	
  delays	
  in	
  2011	
  were	
  at	
  least	
  in	
  part	
  due	
  to	
  challenges	
  in	
  implementing	
  a	
  new	
  funding	
  system	
  across	
  the	
  CGIAR.	
  CCAFS	
  CU	
  reported	
  signs	
  of	
  improvement	
  in	
  2012,	
  with	
  the	
  projected	
  budget	
  for	
  2012	
  advised	
  in	
  December	
  2011	
  and	
  confirmed	
  in	
  March	
  2012.	
  However,	
  there	
  is	
  room	
  for	
  further	
  improvement	
  and	
  confirmation	
  of	
  budgets	
  before	
  the	
  start	
  of	
  the	
  programming	
  year	
  would	
  be	
  optimal.	
  (R7)	
  CCAFS	
  also	
  experienced	
  delays	
  in	
  receiving	
  funds	
  from	
  the	
  Consortium	
  Office.	
  By	
  August	
  2011	
  only	
  30%	
  of	
  Window	
  1	
  and	
  2	
  funds	
  had	
  been	
  received,	
  and	
  by	
  December	
  only	
  89%.	
  Again,	
  there	
  are	
  signs	
  of	
  improvement	
  in	
  2012,	
  although	
  disbursements	
  are	
  still	
  arriving	
  late.	
  CCAFS	
  had	
  received	
  just	
  38%	
  of	
  2012	
  funds	
  by	
  August	
  24	
  2012.	
  	
  In	
  part	
  this	
  is	
  thought	
  to	
  be	
  due	
  to	
  late	
  arrival	
  of	
  donor	
  funds	
  to	
  the	
  Consortium	
  Office.	
  (R8)	
  Due	
  to	
  these	
  delays	
  in	
  finalizing	
  contracts	
  and	
  budgets	
  and	
  disbursing	
  funds,	
  pre-­‐financing	
  has	
  been	
  required	
  from	
  Centres	
  and	
  organisations	
  participating	
  in	
  CCAFS.	
  In	
  2011,	
  most	
  Centres	
  accepted	
  the	
  risk	
  of	
  pre-­‐financing,	
  although	
  some	
  remained	
  reluctant	
  to	
  pre-­‐finance.	
  This	
  contributed	
  to	
  programme	
  under-­‐spending	
  in	
  2011.	
  One	
  of	
  the	
  results	
  of	
  these	
  uncertainties,	
  and	
  also	
  due	
  to	
  the	
  poor	
  adaptation	
  of	
  the	
  management	
  and	
  reporting	
  systems	
  to	
  non	
  CGIAR	
  partners,	
  Columbia	
  University	
  handed	
  back	
  financial	
  administration	
  to	
  CIAT,	
  and	
  Vermont	
  handed	
  financial	
  administration	
  to	
  ICRAF.	
  
	
   6	
  
Indirect	
  costs	
  As	
  a	
  Challenge	
  Program,	
  CCAFS	
  established	
  a	
  policy	
  fixing	
  indirect	
  costs	
  at	
  20%	
  for	
  partners,	
  5%	
  for	
  pass	
  through	
  funds,	
  and	
  15%	
  for	
  third-­‐party	
  recipients	
  of	
  funds.	
  This	
  was	
  an	
  attempt	
  to	
  prevent	
  spiraling	
  indirect	
  costs	
  as	
  funds	
  move	
  through	
  the	
  partnership,	
  and	
  avoid	
  the	
  high	
  indirect	
  costs	
  of	
  the	
  University	
  partners	
  leading	
  Themes	
  1,	
  2	
  and	
  3.	
  	
  CCAFS	
  CU	
  anticipate	
  a	
  new	
  policy	
  from	
  the	
  Consortium	
  Office,	
  defining	
  a	
  single	
  flat	
  rate	
  for	
  indirect	
  costs.	
  Under	
  such	
  a	
  policy	
  the	
  host	
  of	
  a	
  Regional	
  Programme,	
  such	
  as	
  ICRISAT,	
  would	
  charge	
  a	
  full	
  rate	
  for	
  funds	
  they	
  then	
  disburse	
  to	
  a	
  regional	
  or	
  county	
  partner.	
  Indirect	
  costs	
  will	
  therefore	
  increase	
  over	
  time	
  as	
  old	
  contracts	
  expire	
  and	
  are	
  replaced	
  by	
  updated	
  agreements,	
  reducing	
  overall	
  financial	
  efficiency.	
  (R10	
  &	
  R11)	
  2.3	
  Effectiveness	
  The	
  framing	
  of	
  this	
  review	
  meant	
  that	
  effectiveness	
  is	
  discussed	
  in	
  terms	
  of	
  monitoring	
  and	
  evaluation	
  processes	
  to	
  ensure	
  effective	
  program	
  delivery.	
  CCAFS	
  employs	
  a	
  number	
  of	
  tools	
  and	
  approaches	
  in	
  monitoring	
  and	
  evaluating	
  and	
  program	
  management.	
  This	
  section	
  gives	
  an	
  overview	
  of	
  general	
  provisions	
  for	
  monitoring	
  and	
  evaluation,	
  and	
  then	
  examines	
  the	
  programme	
  LogFrame	
  and	
  baseline	
  surveys	
  in	
  more	
  detail.	
  2.3.1	
  General	
  provisions	
  for	
  monitoring	
  and	
  evaluation	
  The	
  complexity	
  of	
  CCAFS,	
  and	
  the	
  different	
  requirements	
  and	
  audiences	
  for	
  monitoring	
  and	
  evaluation,	
  makes	
  the	
  development	
  of	
  a	
  single	
  integrated	
  framework	
  extremely	
  challenging	
  and	
  most	
  likely	
  unfeasible.	
  The	
  Monitoring	
  and	
  Evaluation	
  Strategy	
  of	
  CCAFS	
  distinguishes	
  between	
  three	
  separate	
  domains	
  for	
  evaluative	
  thinking:	
  priority	
  setting	
  and	
  strategic	
  foresight;	
  ongoing	
  planning,	
  monitoring	
  and	
  evaluation;	
  and	
  impact	
  assessment.	
  Each	
  domain	
  has	
  a	
  set	
  of	
  planned	
  activities	
  and	
  mechanisms	
  that	
  the	
  programme	
  is	
  in	
  the	
  process	
  of	
  implementing.	
  Additionally,	
  CCAFS	
  also	
  benefits	
  from	
  the	
  governance	
  role	
  of	
  the	
  Independent	
  Science	
  Panel,	
  which	
  has	
  an	
  important	
  role	
  in	
  agenda	
  setting	
  and	
  evaluation.	
  	
  Governance	
  The	
  Independent	
  Science	
  Panel	
  (ISP)	
  of	
  CCAFS	
  plays	
  an	
  important	
  governance	
  role,	
  which	
  includes	
  reviewing	
  scientific	
  programming	
  and	
  priority	
  setting.	
  This	
  includes	
  periodic	
  sampling	
  of	
  research	
  proposals	
  and	
  outputs	
  to	
  evaluate	
  quality.	
  In	
  2011	
  the	
  ISP	
  expanded	
  in	
  numbers	
  and	
  diversity,	
  and	
  now	
  has	
  representation	
  from	
  the	
  research,	
  the	
  private	
  sector,	
  and	
  developing	
  country	
  organisations.	
  No	
  CGIAR	
  representatives	
  or	
  CCAFS	
  donors	
  sit	
  on	
  the	
  board,	
  although	
  there	
  are	
  ex-­‐officio	
  members	
  from	
  CIAT,	
  ESSP,	
  and	
  the	
  CCAFS	
  Director.	
  The	
  ISP	
  also	
  has	
  the	
  authority	
  to	
  commission	
  external	
  reviews	
  focused	
  on	
  programming	
  issues.	
  Such	
  a	
  review	
  is	
  planned	
  for	
  January	
  2013	
  to	
  look	
  at	
  the	
  governance	
  of	
  CCAFS.	
  As	
  lead	
  Centre,	
  the	
  CIAT	
  Board	
  also	
  has	
  the	
  authority	
  to	
  commission	
  external	
  reviews	
  focused	
  on	
  legal,	
  financial,	
  and	
  administrative	
  aspects.	
  (R12)	
  Foresight	
  and	
  priority	
  setting	
  With	
  climate	
  change	
  as	
  a	
  central	
  focus	
  for	
  programming,	
  CCAFS	
  has	
  taken	
  seriously	
  the	
  challenge	
  of	
  developing	
  foresight	
  approaches	
  to	
  inform	
  programming	
  decisions.	
  As	
  noted	
  in	
  Section	
  2.1,	
  the	
  CCAFS	
  programme	
  plan	
  has	
  been	
  framed	
  through	
  consultation	
  with	
  key	
  internal	
  and	
  external	
  stakeholders	
  at	
  different	
  levels,	
  effectively	
  relying	
  on	
  expert	
  judgement	
  to	
  identify	
  priorities.	
  In	
  the	
  future,	
  CCAFS	
  expects	
  to	
  strengthen	
  priority	
  setting	
  through	
  ongoing	
  work	
  under	
  Theme	
  4	
  to	
  develop	
  climate	
  and	
  futures	
  
	
   7	
  
scenarios	
  that	
  will	
  identify	
  potential	
  vulnerabilities,	
  development	
  pathways,	
  and	
  evaluate	
  potential	
  cost/benefit	
  tradeoffs	
  in	
  different	
  adaptation	
  and	
  mitigation	
  options.	
  	
  Outputs	
  of	
  such	
  ex	
  ante	
  impact	
  assessment	
  (EAIA)	
  tools	
  are	
  expected	
  to	
  feed	
  directly	
  in	
  to	
  priority	
  setting	
  for	
  research	
  in	
  Themes	
  1	
  to	
  3,	
  as	
  well	
  as	
  contributing	
  to	
  international	
  public	
  goods	
  as	
  research	
  outputs.	
  The	
  suite	
  of	
  tools	
  being	
  applied,	
  and	
  in	
  some	
  cases	
  developed,	
  include	
  global	
  integrated	
  assessment	
  partial	
  equilibrium	
  models,	
  semi-­‐quantitative	
  static	
  methods,	
  quantitative	
  static	
  tools	
  based	
  on	
  economic	
  surplus	
  methods,	
  and	
  modelling	
  tools	
  at	
  household,	
  crop	
  and	
  livestock	
  models.	
  Although	
  developing	
  such	
  EAIA	
  tools	
  and	
  approaches	
  is	
  innovative	
  and	
  will	
  help	
  drive	
  programme	
  priority-­‐setting,	
  they	
  have	
  limitations	
  (see	
  below).	
  However,	
  the	
  first	
  results	
  from	
  EAIA	
  tools	
  are	
  not	
  anticipated	
  to	
  make	
  significant	
  contributions	
  to	
  priority	
  setting	
  until	
  after	
  Year	
  3	
  (2013).	
  In	
  the	
  interim,	
  priority	
  setting	
  through	
  stocktaking	
  and	
  expert	
  consultation	
  has	
  continued	
  to	
  be	
  used	
  to	
  determining	
  strategic	
  aspects	
  of	
  programme	
  growth.	
  This	
  has	
  included	
  the	
  selection	
  of	
  new	
  programming	
  regions	
  in	
  SE	
  Asia	
  and	
  the	
  Americas.	
  Another	
  example	
  has	
  been	
  the	
  prioritization	
  of	
  agronomic	
  and	
  modelling	
  research	
  into	
  climate-­‐resilient	
  and	
  under-­‐research	
  crops,	
  identified	
  through	
  meetings	
  between	
  CGIAR	
  Centres	
  and	
  national	
  breeders.	
  	
  Progress	
  at	
  field	
  sites	
  no	
  doubt	
  varies	
  considerably.	
  The	
  Review	
  Team’s	
  mission	
  found	
  that,	
  in	
  Ghana,	
  there	
  had	
  been	
  progress	
  in	
  implementing	
  forecasting	
  approaches	
  for	
  local	
  priority	
  setting.	
  Community	
  consultation	
  and	
  vulnerability	
  studies	
  had	
  been	
  conducted,	
  climate	
  analogous	
  sites	
  had	
  been	
  identified,	
  and	
  farmer	
  exchange	
  visits	
  were	
  planned,	
  from	
  which	
  participatory	
  dialogue	
  on	
  priority	
  setting	
  for	
  progressive	
  adaptation	
  is	
  expected	
  to	
  follow.	
  In	
  the	
  opinion	
  of	
  the	
  Review	
  Team,	
  the	
  planned	
  process	
  is	
  sound	
  and	
  logical,	
  albeit	
  inevitably	
  subject	
  to	
  the	
  tempo	
  of	
  the	
  participatory	
  processes.	
  It	
  has	
  now	
  to	
  lead	
  to	
  the	
  definition	
  of	
  real	
  research	
  questions	
  that	
  could	
  be	
  the	
  basis	
  for	
  an	
  innovative	
  production	
  of	
  knowledge.	
  (see	
  also	
  Section	
  3.1.2).	
  The	
  Review	
  Team	
  accepts	
  that	
  challenges	
  with	
  administration,	
  finances,	
  and	
  the	
  rapid	
  evolution	
  of	
  the	
  programme	
  have	
  been	
  largely	
  responsible	
  for	
  delays.	
  The	
  pace	
  of	
  participatory	
  approaches	
  cannot	
  be	
  forced,	
  and	
  monitoring	
  and	
  evaluation	
  needs	
  to	
  be	
  flexible	
  due	
  to	
  the	
  importance	
  of	
  negotiating	
  process	
  and	
  outcomes.	
  However,	
  CCAFS	
  should	
  strengthen	
  internal	
  processes	
  to	
  ensure	
  that	
  challenges	
  in	
  programme	
  management	
  are	
  not	
  contributing	
  to	
  additional	
  delays	
  in	
  participatory	
  processes.	
  (R14)	
  Ongoing	
  program	
  management,	
  monitoring	
  and	
  evaluation,	
  and	
  learning	
  CCAFS	
  is	
  faced	
  with	
  a	
  series	
  of	
  challenges	
  related	
  both	
  to	
  domains	
  of	
  activities	
  (research	
  into	
  uncertain	
  global	
  change)	
  and	
  to	
  institutional	
  change	
  (the	
  ongoing	
  CGIAR	
  reform	
  process).	
  As	
  a	
  response,	
  the	
  CCAFS	
  CU	
  has	
  sensibly	
  adopted	
  an	
  adaptive	
  management	
  approach	
  based	
  on	
  learning	
  partnerships,	
  and	
  ‘learning	
  loops’	
  between	
  CCAFS	
  staff,	
  partners	
  and	
  stakeholders.	
  	
  The	
  programme	
  Logical	
  Framework	
  (LogFrame)	
  serves	
  as	
  the	
  principal	
  planning	
  tool	
  at	
  an	
  operational	
  level	
  (see	
  Section	
  2.3.2	
  for	
  assessment	
  of	
  the	
  LogFrame).	
  Activities	
  proposed	
  by	
  Centres	
  are	
  assessed	
  by	
  Regional	
  and	
  Thematic	
  Leaders	
  and	
  related	
  to	
  the	
  LogFrame	
  against	
  yearly	
  milestones.	
  Similar	
  activity	
  plans	
  are	
  developed	
  for	
  Regional	
  and	
  Thematic	
  Progammes.	
  Indicators	
  on	
  the	
  LogFrame	
  therefore	
  become	
  critical	
  parts	
  of	
  the	
  decision	
  making	
  process	
  for	
  allocating	
  funding.	
  In	
  response	
  to	
  emerging	
  priorities	
  and	
  consultation	
  the	
  LogFrame	
  has	
  been	
  updated	
  and	
  is	
  intended	
  to	
  be	
  a	
  flexible	
  document,	
  with	
  a	
  regular	
  update	
  planned	
  each	
  year.	
  	
  However,	
  complex	
  planning	
  and	
  decision-­‐making	
  judgments	
  are	
  not	
  entirely	
  bounded	
  by	
  the	
  dimensions	
  captured	
  in	
  the	
  LogFrame.	
  Decision-­‐making	
  on	
  programming	
  is	
  also	
  informed	
  by	
  issues	
  such	
  as	
  evidence	
  of	
  partnership	
  quality	
  or	
  the	
  quality	
  of	
  proposed	
  work	
  on	
  gender	
  and	
  social	
  differentiation.	
  As	
  CCAFS	
  now	
  effectively	
  controls	
  a	
  large	
  proportion	
  of	
  funds	
  within	
  the	
  CGIAR	
  on	
  climate	
  change,	
  it	
  must	
  ensure	
  that	
  the	
  criteria	
  
	
   8	
  
used	
  to	
  evaluate	
  proposed	
  work	
  are	
  made	
  clear	
  and	
  transparent	
  to	
  all	
  Centres	
  and	
  proponents.	
  This	
  will	
  be	
  particularly	
  important	
  for	
  CCAFS	
  to	
  avoid	
  reputation	
  risk	
  as	
  it	
  begins	
  to	
  be	
  more	
  selective	
  and	
  focused	
  in	
  awarding	
  Window	
  1	
  and	
  2	
  funds	
  in	
  the	
  future	
  (R3).	
  	
  Aside	
  from	
  the	
  LogFrame,	
  other	
  ‘learning	
  loop’	
  processes	
  employed	
  include	
  twice	
  yearly	
  meetings	
  of	
  the	
  ISP,	
  twice	
  yearly	
  physical	
  meetings	
  of	
  the	
  Programme	
  Management	
  Committee	
  (PMC),	
  annual	
  progress	
  reporting	
  against	
  indicators	
  by	
  activity,	
  theme	
  and	
  region,	
  annual	
  reporting	
  to	
  the	
  CIAT	
  Board	
  and	
  EC/IFAD,	
  monthly	
  teleconferences	
  between	
  the	
  PMC	
  members,	
  yearly	
  meetings	
  between	
  PMC	
  members	
  and	
  contact	
  points	
  in	
  each	
  Centre,	
  annual	
  meetings	
  with	
  key	
  stakeholder	
  meetings	
  in	
  each	
  region,	
  and	
  quality	
  control	
  of	
  research	
  outputs.	
  In	
  addition,	
  each	
  year	
  the	
  core	
  CCAFS	
  team	
  meets	
  for	
  a	
  facilitated	
  reflection	
  exercise	
  to	
  identify	
  programme	
  risks	
  and	
  opportunities.	
  	
  In	
  such	
  a	
  complex	
  research	
  program,	
  a	
  key	
  concern	
  is	
  information	
  management,	
  and	
  in	
  particular	
  how	
  CCAFS	
  ensures	
  evidence	
  from	
  individual	
  activities	
  is	
  contributing	
  towards	
  top	
  level	
  research	
  questions,	
  and	
  that	
  results	
  are	
  synthesized	
  appropriately.	
  This	
  is	
  particularly	
  problematic	
  given	
  the	
  lack	
  of	
  standardized	
  research	
  methods,	
  and	
  the	
  number	
  of	
  Centre-­‐led	
  activities	
  from	
  Window	
  2	
  and	
  3	
  funds.	
  The	
  ISP	
  has	
  expressed	
  concerns	
  on	
  this	
  issue,	
  and	
  integration	
  of	
  research	
  activities	
  was	
  also	
  identified	
  as	
  a	
  program	
  risk	
  during	
  a	
  reflection	
  exercise	
  by	
  CCAFS	
  core	
  staff.	
  	
  In	
  principle,	
  CCAFS	
  Head	
  of	
  Research	
  is	
  responsible	
  for	
  synthesis	
  across	
  the	
  programme	
  and	
  global	
  policy	
  engagement,	
  whilst	
  Theme	
  &	
  Regional	
  leaders	
  are	
  responsible	
  for	
  managing	
  research	
  linked	
  to	
  policy	
  processes.	
  CCAFS	
  is	
  promoting	
  the	
  model	
  of	
  a	
  ‘climate	
  smart	
  village’	
  to	
  foster	
  integration	
  across	
  food	
  security,	
  adaptation	
  and	
  mitigation.	
  However,	
  the	
  majority	
  of	
  current	
  CCAFS	
  funds	
  are	
  in	
  Centre-­‐led	
  activities	
  not	
  located	
  at	
  CCAFS	
  sites.	
  Therefore	
  there	
  is	
  an	
  important	
  need	
  to	
  develop	
  ways	
  to	
  integrate	
  these	
  activities	
  and	
  their	
  findings	
  in	
  to	
  a	
  functional	
  analytical	
  framework.	
  (R15)	
  The	
  Review	
  Team	
  is	
  also	
  concerned	
  that	
  the	
  extant	
  planning	
  cycle	
  of	
  proposal	
  writing	
  and	
  allocation	
  could	
  lead	
  to	
  an	
  emphasis	
  on	
  activities	
  rather	
  than	
  medium	
  term	
  research	
  objectives.	
  The	
  program	
  framework	
  is	
  populated	
  by	
  strategic	
  documents	
  and	
  annual	
  documents,	
  but	
  with	
  little	
  bridging	
  material.	
  No	
  proposal	
  was	
  reviewed	
  that	
  articulated	
  how	
  –	
  for	
  example	
  –	
  a	
  three-­‐year	
  initiative	
  would	
  contribute	
  to	
  strategic	
  research	
  questions.	
  Given	
  CCAFS	
  current	
  workload,	
  the	
  Review	
  Team	
  feels	
  it	
  would	
  be	
  counter-­‐productive	
  to	
  recommend	
  a	
  further	
  level	
  of	
  planning	
  activity.	
  However,	
  if	
  CCAFS	
  could	
  encourage	
  the	
  submission	
  of	
  multi-­‐year	
  proposals,	
  with	
  modular,	
  single	
  year	
  activity	
  plans	
  and	
  milestones,	
  then	
  that	
  could	
  help	
  bridge	
  the	
  gap	
  in	
  ensuring	
  annual	
  research	
  activities	
  are	
  strategic	
  (R16)	
  .	
  Research	
  quality	
  is	
  another	
  area	
  of	
  concern.	
  Operational	
  responsibility	
  for	
  research	
  quality	
  lies	
  with	
  activity	
  leaders,	
  or	
  intellectual	
  leaders	
  within	
  themes	
  and	
  regional	
  programmes.	
  The	
  ISP	
  also	
  periodically	
  samples	
  programming	
  to	
  ascertain	
  quality.	
  CCAFS’	
  internal	
  reflection	
  process	
  for	
  2011/12	
  identified	
  the	
  production	
  of	
  quality	
  science	
  as	
  a	
  critical	
  success	
  factor,	
  and	
  CCAFS	
  CU	
  takes	
  the	
  issue	
  seriously.	
  However,	
  thematic	
  leaders	
  and	
  the	
  ISP	
  have	
  noted	
  that	
  performance	
  needs	
  to	
  improve,	
  and	
  that	
  contributions	
  from	
  Centres	
  are	
  not	
  of	
  sufficient	
  quality.	
  	
  The	
  partnership	
  contracts	
  between	
  CCAFS	
  and	
  participating	
  Centres	
  contain	
  clauses	
  allowing	
  CCAFS	
  to	
  reduce	
  funding	
  due	
  to	
  poor	
  quality	
  of	
  outputs.	
  However,	
  CCAFS	
  could	
  feel	
  that	
  the	
  potential	
  repercussions	
  of	
  invoking	
  these	
  clauses	
  outweigh	
  the	
  benefits	
  unless	
  they	
  are	
  confident	
  of	
  the	
  support	
  of	
  the	
  Consortium	
  Office	
  and	
  donors.	
  Another	
  perspective	
  could	
  be	
  that	
  the	
  focus	
  of	
  CCAFS	
  requires	
  innovation	
  in	
  the	
  methods,	
  interests	
  and	
  capacities	
  of	
  the	
  Centres,	
  yet	
  Centres	
  have	
  not	
  yet	
  been	
  able	
  to	
  innovate	
  sufficiently.	
  A	
  combination	
  of	
  withdrawing	
  funding	
  from	
  the	
  worst	
  performers	
  whilst	
  
	
   9	
  
offering	
  top-­‐up	
  funding	
  to	
  build	
  capacity	
  in	
  innovative	
  areas	
  within	
  willing	
  Centres	
  could	
  be	
  a	
  productive	
  strategy	
  (R17)	
  .	
  Monitoring	
  and	
  evaluation	
  tools	
  are	
  also	
  being	
  used	
  to	
  track	
  progress	
  with	
  community	
  engagement	
  at	
  CCAFS	
  sites.	
  Implementation	
  of	
  participatory	
  M&E	
  toolkit	
  is	
  being	
  trialed	
  in	
  W	
  Africa	
  with	
  reportedly	
  positive	
  results	
  in	
  Burkina	
  Faso	
  and	
  Mali.	
  One	
  important	
  aspect	
  of	
  such	
  approaches	
  is	
  that	
  they	
  provide	
  feedback	
  on	
  the	
  development	
  and	
  quality	
  of	
  partnerships.	
  Currently,	
  CCAFS	
  intends	
  these	
  tools	
  for	
  use	
  in	
  gauging	
  community	
  engagement	
  and	
  for	
  evaluation	
  at	
  CCAFS	
  sites,	
  rather	
  than	
  for	
  programme	
  level	
  evaluation.	
  However,	
  given	
  the	
  importance	
  of	
  the	
  partnership	
  approach	
  to	
  CCAFS,	
  the	
  programme	
  may	
  wish	
  to	
  develop	
  a	
  complementary	
  system	
  evaluating	
  progress	
  in	
  the	
  development	
  of	
  CCAFS’	
  global	
  partnerships	
  (R18)	
  .	
  2.3.2	
  Analysis	
  of	
  Logical	
  Framework	
  and	
  Baselines	
  for	
  Impact	
  Assessment	
  The	
  CCAFS	
  Consolidated	
  LogFrame	
  2012-­‐2015	
  is	
  a	
  44	
  page	
  document	
  organized	
  by	
  Theme,	
  Objective,	
  Outcome,	
  Output	
  and	
  Milestone.	
  Rather	
  than	
  attempt	
  to	
  analyse	
  the	
  whole	
  document,	
  the	
  Review	
  Team	
  has	
  sampled	
  specific	
  areas	
  to	
  examine	
  the	
  consistency	
  of	
  the	
  LogFrame	
  and	
  relevance	
  of	
  indicators.	
  Logical	
  Framework	
  Consistency	
  Objective	
  2.3	
  was	
  selected	
  to	
  assess	
  consistency	
  in	
  the	
  LogFrame	
  (see	
  Annex	
  5).	
  Within	
  Theme	
  2	
  (Managing	
  Climate	
  Risk),	
  Objective	
  3	
  sets	
  out	
  to	
  provide	
  enhanced	
  predictions	
  of	
  climate	
  impacts	
  on	
  agriculture,	
  and	
  enhanced	
  provision	
  of	
  climate	
  information	
  and	
  services.	
  The	
  Outcome	
  Statement	
  is	
  to	
  have	
  “enhanced	
  uptake	
  and	
  use	
  of	
  improved	
  climate	
  information	
  products	
  and	
  services,	
  and	
  of	
  information	
  about	
  agricultural	
  production	
  and	
  biological	
  threats,	
  by	
  resource- poor	
  farmers,	
  particularly	
  vulnerable	
  groups	
  and	
  women,	
  in	
  at	
  least	
  12	
  countries”.	
  The	
  Outcome	
  follows	
  logically	
  from	
  the	
  Objective.	
  However,	
  it	
  is	
  not	
  evident	
  whether	
  the	
  Outcome	
  Statement	
  target	
  (12	
  countries)	
  is	
  related	
  to	
  achievements	
  by	
  the	
  year	
  2015	
  or	
  the	
  end	
  of	
  programme.	
   The	
  Outcome	
  is	
  supported	
  by	
  two	
  Outputs.	
  The	
  first	
  (2.3.1)	
  focuses	
  on	
  enhanced	
  tools	
  and	
  platforms	
  for	
  developing	
  climate	
  information	
  and	
  forecasts	
  and	
  agricultural	
  vulnerability	
  models.	
  The	
  second	
  (2.3.2)	
  focuses	
  on	
  institutional	
  arrangements	
  and	
  communication	
  processes	
  for	
  enhancing	
  climate	
  information	
  services,	
  particularly	
  to	
  reach	
  marginalized	
  farmers	
  and	
  women.	
  These	
  twin	
  sets	
  of	
  Outputs	
  –	
  information	
  production	
  and	
  sharing	
  -­‐	
  are	
  both	
  consistent	
  with	
  the	
  Outcome,	
  and	
  appropriately	
  differentiate	
  sets	
  of	
  research	
  questions,	
  activities,	
  and	
  implicated	
  partners.	
  	
  Outputs	
  are	
  supported	
  by	
  four	
  annual	
  milestones	
  and	
  targets	
  (2012	
  to	
  2015),	
  each	
  accompanied	
  by	
  performance	
  indicators,	
  means	
  of	
  verification,	
  and	
  implicated	
  partners.	
  Each	
  milestone	
  covers	
  different	
  activities	
  to	
  be	
  delivered	
  within	
  a	
  year.	
  For	
  example,	
  milestones	
  under	
  Output	
  2.3.1	
  (product	
  development)	
  include	
  annual	
  activities	
  on	
  improving	
  both	
  seasonal	
  climate	
  forecasts	
  and	
  crop	
  forecasts.	
  In	
  general	
  the	
  components	
  proposed	
  milestones	
  flow	
  logically	
  from	
  year	
  to	
  year.	
  As	
  an	
  example,	
  crop	
  forecasting	
  tools	
  are	
  designed	
  in	
  year	
  1,	
  developed	
  and	
  tested	
  in	
  year	
  2,	
  documented	
  and	
  training	
  provided	
  in	
  year	
  3,	
  and	
  operational	
  in	
  year	
  4.	
  However,	
  due	
  the	
  mix	
  of	
  different	
  activities	
  within	
  each	
  milestone,	
  the	
  sequential	
  step	
  for	
  each	
  strand	
  of	
  activity	
  can	
  be	
  hard	
  to	
  follow	
  in	
  this	
  format.	
  Linkages	
  between	
  milestones	
  –	
  and	
  Outputs	
  –	
  are	
  left	
  implicit,	
  such	
  as	
  the	
  connection	
  between	
  the	
  identification	
  of	
  climate	
  information	
  user	
  needs	
  (2.3.2	
  2012)	
  and	
  the	
  design	
  of	
  forescasting	
  tools	
  (2.3.1	
  2012).	
  Qualitatively,	
  milestone	
  targets	
  and	
  performance	
  indicators	
  relate	
  to	
  Outputs.	
  Quantitatively,	
  however,	
  it	
  is	
  not	
  clear	
  how	
  the	
  targets	
  relate	
  to	
  each	
  other	
  or	
  to	
  the	
  Outcome	
  Statement.	
  For	
  example,	
  by	
  2015	
  two	
  enhanced	
  climate	
  tools	
  are	
  planned	
  to	
  be	
  operational	
  in	
  two	
  to	
  four	
  countries	
  or	
  two	
  regional	
  institutions,	
  whilst	
  enhanced	
  climate	
  services	
  will	
  have	
  been	
  evaluated	
  in	
  ten	
  countries,	
  
	
   10	
  
and	
  training	
  on	
  climate	
  services	
  held	
  in	
  twelve	
  countries.	
  By	
  contrast,	
  the	
  Outcome	
  Statement	
  calls	
  for	
  enhanced	
  information	
  and	
  services	
  in	
  twelve	
  countries,	
  which	
  will	
  not	
  be	
  met	
  by	
  the	
  stated	
  milestone	
  targets.	
  These	
  ambiguities	
  mean	
  that	
  CCAFS	
  could	
  be	
  ill	
  served	
  in	
  an	
  evaluation	
  process	
  based	
  on	
  the	
  LogFrame.	
  Objective	
  2.3	
  as	
  framed	
  in	
  the	
  LogFrame	
  is	
  consistent	
  with	
  the	
  West	
  African	
  Impact	
  Pathway	
  Diagram	
  for	
  the	
  same	
  objective	
  given	
  in	
  the	
  CCAFS	
  Programme	
  Plan	
  2011	
  (see	
  Annex	
  X).	
  Aside	
  from	
  the	
  point	
  raised	
  above	
  regarding	
  relationships	
  between	
  quantitative	
  indicators,	
  the	
  flow	
  between	
  the	
  reviewed	
  LogFrame	
  elements	
  is	
  consistent	
  and	
  logical.	
  There	
  appears	
  to	
  be	
  a	
  good	
  level	
  of	
  consistency	
  between	
  the	
  Objectives	
  and	
  activities	
  within	
  Theme	
  2.	
  Potential	
  interactions	
  with	
  other	
  Themes	
  are	
  not	
  articulated.	
  	
  SMART	
  indicators	
  of	
  outputs	
  To	
  review	
  indicators	
  in	
  more	
  breadth,	
  those	
  for	
  Output	
  4.3(Frameworks	
  for	
  Policy	
  Analysis)	
  and	
  3.1(Agricultural	
  Development	
  Pathways)	
  were	
  also	
  included	
  in	
  the	
  assessment.	
  The	
  concept	
  of	
  SMART	
  (Specific,	
  Measurable,	
  Achievable,	
  Relevant,	
  and	
  Time-­‐bound)	
  indicators	
  was	
  used	
  as	
  a	
  framework.	
  	
  Specificity	
  and	
  measurability	
  were	
  weaknesses	
  of	
  most	
  performance	
  indicators	
  reviewed.	
  For	
  example,	
  milestone	
  2.3.2	
  proposes	
  ‘training	
  offered’	
  and	
  ‘capacity	
  built’,	
  but	
  does	
  not	
  specific	
  who	
  will	
  be	
  trained,	
  or	
  how	
  capacity	
  might	
  be	
  evaluated.	
  Milestone	
  4.3.2	
  proposes	
  ‘reports	
  written	
  and	
  disseminated’	
  and	
  milestone	
  4.3.3	
  proposes	
  ‘activities	
  held’	
  without	
  defining	
  their	
  number,	
  type,	
  or	
  audience,	
  or	
  how	
  their	
  impact	
  might	
  be	
  measured.	
  	
  The	
  indicators	
  of	
  Objective	
  3.1	
  are	
  generally,	
  although	
  not	
  consistently,	
  more	
  clearly	
  defined,	
  noting	
  in	
  places	
  the	
  specific	
  subjects	
  of	
  papers	
  to	
  be	
  written,	
  and	
  numbers	
  of	
  decision	
  makers	
  to	
  attend	
  workshops,	
  and	
  referring	
  to	
  workshop	
  reports	
  as	
  means	
  of	
  verification.	
  (R19)	
  Indicator	
  achievability	
  is	
  harder	
  to	
  assess,	
  particularly	
  given	
  the	
  critical	
  sequential	
  links	
  between	
  some	
  milestones,	
  and	
  the	
  difficulties	
  of	
  delivery	
  through	
  engagement	
  in	
  complex	
  partnerships.	
  CCAFS	
  CU	
  have	
  acknowledged	
  that	
  in	
  some	
  cases	
  targets	
  can	
  be	
  considered	
  aspirational	
  rather	
  than	
  calculated.	
  In	
  some	
  cases,	
  low	
  specificity	
  and	
  measurability	
  adds	
  to	
  the	
  difficulty	
  of	
  assessing	
  achievability.	
  For	
  example,	
  if	
  a	
  forecast	
  system	
  is	
  proposed	
  to	
  be	
  operational	
  in	
  two	
  regions	
  by	
  2015	
  (milestone	
  2.3.1),	
  does	
  that	
  mean	
  regional	
  institutions	
  have	
  the	
  tools	
  to	
  produce	
  seasonal	
  forecasts,	
  that	
  they	
  are	
  producing	
  seasonal	
  forecasts,	
  or	
  that	
  they	
  are	
  being	
  produced	
  and	
  shared	
  with	
  national	
  partners?	
  Different	
  aspects	
  of	
  institutional	
  capacity	
  beyond	
  the	
  provision	
  of	
  tools	
  might	
  have	
  effects	
  here,	
  so	
  precision	
  is	
  important	
  for	
  accountability	
  purposes.	
  (R19)	
  Relevance	
  of	
  indicators	
  is	
  generally	
  high,	
  with	
  most	
  focused	
  on	
  the	
  production	
  or	
  sharing	
  of	
  knowledge	
  outputs	
  that	
  support	
  the	
  Outcome	
  Statement.	
  Timeliness	
  is	
  also	
  consistently	
  clear,	
  with	
  each	
  indicator	
  set	
  against	
  a	
  particular	
  year.	
  However,	
  in	
  the	
  future	
  CCAFS	
  may	
  wish	
  to	
  consider	
  whether	
  how	
  to	
  reflect	
  inherent	
  delays	
  in	
  research	
  processes,	
  which	
  see	
  publications	
  from	
  activities	
  materialize	
  in	
  later	
  years.	
  	
  Impact	
  assessment	
  –	
  Baselines	
  	
  CCAFS	
  takes	
  impact	
  assessment	
  seriously,	
  but	
  faces	
  considerable	
  challenges	
  in	
  implementation.	
  The	
  programme	
  has	
  undertaken	
  a	
  series	
  of	
  baseline	
  surveys	
  at	
  all	
  CCAFS	
  sites,	
  and	
  activities	
  under	
  Theme	
  4	
  are	
  also	
  developing	
  ex	
  post	
  impact	
  assessment	
  (EPIA)	
  tools	
  that	
  will	
  draw	
  on	
  a	
  common	
  set	
  of	
  baseline	
  agricultural,	
  socioeconomic,	
  and	
  biogeophysical	
  indicators	
  at	
  each	
  CCAFS	
  site,	
  supplemented	
  by	
  regional	
  and	
  global	
  indicators	
  to	
  capture	
  cross-­‐scale	
  impacts.	
  CCAFS	
  intends	
  to	
  evaluate	
  performance	
  against	
  ten-­‐year	
  outcomes,	
  and	
  three-­‐year	
  performance	
  indicators	
  identified	
  in	
  the	
  LogFrame.	
  However,	
  the	
  team	
  is	
  still	
  attempting	
  to	
  operationalize	
  an	
  impact	
  assessment	
  strategy	
  on	
  the	
  ground.	
  Challenges	
  include	
  the	
  identification	
  of	
  key	
  indicators,	
  and	
  how	
  
	
   11	
  
–	
  or	
  if	
  -­‐	
  to	
  differentiate	
  between	
  the	
  impacts	
  of	
  different	
  CRPs,	
  other	
  initiatives	
  operating	
  in	
  the	
  same	
  area6,	
  and	
  transformations	
  of	
  the	
  ecological	
  and	
  economic	
  context.	
  	
  To	
  address	
  sub-­‐goal	
  1,	
  current	
  mitigation	
  and	
  adaptation	
  behaviours	
  and	
  institutions	
  have	
  been	
  characterized	
  by	
  3	
  baseline	
  surveys	
  at	
  all	
  15	
  CCAFS	
  sites,	
  covering	
  household,	
  village	
  and	
  organisational	
  levels.	
  	
  The	
  household	
  survey	
  was	
  completed	
  in	
  2011.	
  It	
  is	
  a	
  rather	
  classic	
  exercise,	
  with	
  interviews	
  of	
  more	
  than	
  6000	
  households.	
  This	
  was	
  conceptualized	
  by	
  CCAFS	
  in	
  conjunction	
  with	
  partners,	
  and	
  Reading	
  University	
  Statistics	
  Services	
  group	
  helped	
  design	
  the	
  sample	
  regime,	
  which	
  includes	
  resampling	
  every	
  five	
  years	
  to	
  assess	
  changes.	
  The	
  household	
  survey	
  establishes	
  indicators	
  (e.g.	
  of	
  production	
  factors,	
  wealth/assets,	
  innovation/adaptation,	
  access	
  to	
  information)	
  that	
  can	
  be	
  disaggregated	
  by	
  household	
  characteristics	
  including	
  gender.	
  This	
  will	
  allow	
  CCAFS	
  to	
  explore	
  gender	
  and	
  other	
  social	
  and	
  economic	
  differences	
  in	
  target	
  populations	
  (for	
  example,	
  in	
  household	
  and	
  farm	
  structure,	
  access	
  to	
  information	
  and	
  to	
  resources).	
  	
  However,	
  such	
  surveys	
  do	
  not	
  capture	
  the	
  interactions	
  between	
  criteria,	
  nor	
  the	
  diversity	
  of	
  systems	
  resulting	
  from	
  the	
  allocation	
  of	
  production	
  factors.	
  	
  Neither	
  do	
  they	
  help	
  to	
  understand	
  the	
  behaviours	
  and	
  the	
  decision	
  making	
  of	
  different	
  stakeholder	
  groups,	
  nor	
  to	
  track	
  how	
  these	
  change	
  over	
  time	
  in	
  response	
  inter	
  alia	
  to	
  CCAFS	
  activities	
  and	
  research	
  findings.	
  So	
  although	
  the	
  quantitative	
  household	
  surveys	
  allow	
  correlation	
  between	
  individual	
  criteria	
  and	
  help	
  formulate	
  research	
  assumptions,	
  they	
  do	
  not	
  necessarily	
  contribute	
  to	
  knowledge.	
  CCAFS	
  reports	
  that	
  they	
  intend	
  to	
  develop	
  qualitative	
  tools,	
  such	
  as	
  impact	
  pathway	
  assessments,	
  farming	
  systems	
  assessments,	
  and	
  commodity	
  chains	
  appraisals,	
  to	
  provide	
  information	
  complementing	
  the	
  quantitative	
  indicators.	
  The	
  Review	
  Team	
  encourages	
  CCAFS	
  to	
  make	
  full	
  use	
  of	
  these	
  tools	
  to	
  analyse	
  the	
  mechanisms	
  which	
  generate	
  the	
  quantitative	
  results	
  captured	
  in	
  the	
  models,	
  and	
  to	
  draw	
  plausible	
  narratives	
  and	
  systemic	
  analysis	
  about	
  the	
  benefits	
  of	
  CCAFS	
  outputs	
  to	
  different	
  groups	
  of	
  stakeholders.	
  The	
  Review	
  Team	
  also	
  suggests	
  that	
  such	
  in-­‐depth	
  analyses	
  of	
  CCAFS	
  site’	
  farming	
  systems	
  be	
  implemented	
  in	
  collaboration	
  with	
  post	
  graduate	
  students	
  from	
  local7	
  or	
  international	
  universities.	
  It	
  is	
  important	
  to	
  also	
  note	
  that	
  the	
  baseline	
  survey	
  was	
  designed	
  to	
  make	
  comparisons	
  across	
  sites	
  and	
  regions,	
  rather	
  than	
  as	
  a	
  formal	
  ex-­‐post	
  impact	
  assessment	
  tool.	
  With	
  good	
  statistical	
  reasons	
  to	
  avoid	
  the	
  use	
  of	
  control	
  sites,	
  different	
  activities	
  in	
  different	
  sites	
  mean	
  that	
  observed	
  changes	
  cannot	
  necessarily	
  be	
  attributed	
  to	
  interventions.	
  This	
  is	
  one	
  reason	
  for	
  CCAFS	
  adopting	
  alternate	
  ex	
  ante	
  and	
  ex	
  post	
  impact	
  assessments	
  to	
  judge	
  impact	
  of	
  actions.	
  As	
  a	
  consequence,	
  the	
  baseline	
  surveys	
  will	
  be	
  more	
  effective	
  in	
  assessing	
  the	
  contribution	
  of	
  CCAFS,	
  rather	
  than	
  assessing	
  attributional	
  impact.	
  Baselines	
  would	
  therefore	
  need	
  to	
  be	
  supplemented	
  by	
  targeted	
  impact	
  assessment	
  studies	
  to	
  capture	
  attribution,	
  as	
  laid	
  out	
  in	
  the	
  CCAFS	
  Programme	
  Plan.	
  (R13)	
  The	
  village	
  and	
  organisational	
  surveys	
  have	
  been	
  completed	
  in	
  the	
  first	
  quarter	
  of	
  2012.	
  Survey	
  manuals,	
  data	
  sets	
  and	
  site	
  reports	
  are	
  publicly	
  available	
  through	
  the	
  CCAFS	
  website	
  (ccafs.cgiar.org/resources/baseline-­‐surveys),	
  mostly	
  based	
  on	
  a	
  participatory	
  rural	
  appraisal	
  working	
  with	
  self-­‐help	
  groups	
  of	
  female	
  and	
  male	
  farmers.	
  The	
  results	
  are	
  more	
  qualitative,	
  more	
  helpful	
  for	
  generating	
  appropriate	
  assumptions	
  about	
  the	
  problems	
  raised	
  and	
  the	
  research	
  challenges	
  at	
  a	
  subset	
  of	
  the	
  CCAFS	
  sites.	
  	
  National	
  and	
  global	
  policy	
  baselines	
  (to	
  address	
  sub-­‐goal	
  2)	
  have	
  been	
  addressed	
  through	
  policy	
  analyses	
  in	
  each	
  of	
  the	
  three	
  regions,	
  as	
  well	
  as	
  ongoing	
  tracking	
  of	
  the	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  6	
  The	
  benchmark	
  site	
  of	
  Lawra-­‐Jirapa,	
  Ghana,	
  for	
  example,	
  overlaps	
  with	
  benchmark	
  sites	
  for	
  CRPs	
  1.1	
  and	
  5.	
  Indeed,	
  pre-­‐existing	
  activities	
  and	
  partnerships	
  were	
  one	
  criteria	
  for	
  CCAFS	
  benchmark	
  site	
  selection.	
  7	
  such	
  as	
  University	
  of	
  Development	
  Studies	
  in	
  Tamale	
  for	
  Ghana	
  
	
   12	
  
UNFCCC	
  process	
  by	
  the	
  CCAFS	
  Coordinating	
  Unit.	
  To	
  measure	
  progress	
  towards	
  the	
  achievement	
  of	
  the	
  sub-­‐goals,	
  3-­‐year	
  and	
  10-­‐year	
  quantitative	
  indicators	
  have	
  been	
  identified	
  for	
  each	
  Output	
  associated	
  with	
  each	
  of	
  the	
  twelve	
  Objectives	
  of	
  CCAFS.	
  	
  2.4.	
  Practical	
  implementation	
  of	
  the	
  CRPs	
  As	
  this	
  exercise	
  concentrates	
  on	
  a	
  CRP,	
  the	
  review	
  team	
  has	
  chosen	
  to	
  examine	
  this	
  criterion	
  in	
  terms	
  of	
  how	
  the	
  CRP	
  is	
  being	
  operationalized	
  in	
  the	
  Lawra-­‐Jirapa	
  CCAFS	
  site	
  in	
  Ghana	
  in	
  terms	
  of	
  engagement	
  with	
  local	
  partners.	
  	
  The	
  above	
  mentioned	
  challenges	
  in	
  programme	
  management	
  (Section	
  1)	
  and	
  financial	
  management	
  (Section	
  2.2)	
  have	
  also	
  had	
  some	
  consequences	
  for	
  the	
  implementation	
  of	
  Regional	
  Programme	
  Activities.	
  In	
  preparation	
  for	
  the	
  field	
  mission	
  Ghana,	
  seven	
  sub-­‐partner	
  agreements	
  between	
  ICRISAT	
  (host	
  of	
  the	
  Regional	
  Programme)	
  and	
  the	
  Savannah	
  Agricultural	
  Research	
  Institute	
  (SARI)	
  of	
  the	
  Council	
  for	
  Scientific	
  and	
  Industrial	
  Research	
  (CSIR),	
  Ghana	
  (country	
  coordinator	
  for	
  CCAFS)	
  were	
  reviewed.	
  They	
  ranged	
  in	
  value	
  from	
  14	
  500	
  USD	
  to	
  32	
  800	
  USD,	
  and	
  in	
  duration	
  from	
  2	
  weeks	
  to	
  five	
  months,	
  with	
  the	
  majority	
  being	
  of	
  less	
  than	
  5	
  weeks	
  duration	
  and	
  25	
  000	
  USD	
  in	
  value.	
  The	
  earliest	
  was	
  dated	
  June	
  2011,	
  and	
  five	
  were	
  dated	
  December	
  2011	
  or	
  later.	
  The	
  Regional	
  Programme	
  Leader	
  estimated	
  that	
  80%	
  of	
  contracting	
  in	
  West	
  Africa	
  during	
  2011	
  had	
  taken	
  place	
  after	
  September	
  2011	
  due	
  to	
  challenges	
  with	
  the	
  disbursement	
  of	
  funds	
  and	
  prefinancing.	
  	
  The	
  relatively	
  small	
  sums	
  and	
  short	
  durations	
  involved	
  appear	
  to	
  be	
  partly	
  a	
  reflection	
  of	
  CCAFS	
  managing	
  risk	
  and	
  assessing	
  the	
  absorptive	
  capacity	
  in	
  a	
  new	
  partner.	
  However,	
  they	
  have	
  incurred	
  significant	
  transaction	
  costs	
  for	
  the	
  Regional	
  Programme	
  Leader.	
  The	
  Ghanaian	
  Country	
  Coordinator	
  also	
  expressed	
  his	
  frustration	
  with	
  short-­‐term	
  notice	
  given	
  of	
  some	
  assignments,	
  and	
  limited	
  information	
  about	
  the	
  long	
  term	
  strategy	
  and	
  intentions	
  of	
  CCAFS,	
  although	
  he	
  noted	
  that	
  the	
  situation	
  was	
  improving.	
  (R20)	
  	
  2.5.	
  Synergy	
  &	
  collective	
  action	
  Integration	
  and	
  coordination	
  between	
  CRPs	
  At	
  this	
  relatively	
  early	
  stage	
  there	
  appears	
  to	
  be	
  a	
  good	
  level	
  of	
  engagement	
  between	
  CCAFS	
  and	
  other	
  CRPs.	
  In	
  the	
  case	
  of	
  Ghana,	
  the	
  presence	
  of	
  ICRISAT	
  research	
  activities	
  and	
  partners	
  in	
  Wa	
  was	
  a	
  contributing	
  factor	
  in	
  selection	
  of	
  the	
  Lawra-­‐Jirapa	
  CCAFS	
  site.	
  Lawra-­‐Jirapa	
  is	
  also	
  a	
  CCAFS	
  site	
  for	
  CRP	
  1.1,	
  providing	
  opportunities	
  for	
  synergies	
  between	
  the	
  two	
  CRPs	
  whilst	
  creating	
  some	
  potential	
  challenges	
  for	
  impact	
  evaluation.	
  CRP	
  5	
  also	
  has	
  activities	
  in	
  the	
  Wa	
  District,	
  with	
  the	
  potential	
  for	
  increased	
  collaboration	
  in	
  the	
  future.	
  	
  There	
  is	
  early	
  but	
  positive	
  collaboration	
  between	
  CCAFS	
  and	
  CRPs	
  focused	
  on	
  genetic	
  improvement.	
  CCAFS	
  has	
  undertaken	
  to	
  provide	
  climate	
  projections,	
  assessments	
  and	
  tools	
  to	
  advise	
  on	
  breeding	
  strategies	
  to	
  be	
  utilised	
  by	
  CRP3.	
  CCAFS	
  researchers	
  are	
  convening	
  a	
  series	
  of	
  meetings	
  between	
  Centres	
  and	
  national	
  crop	
  breeders,	
  identifying	
  major	
  threats	
  for	
  the	
  year	
  2030,	
  such	
  as	
  heat	
  stress	
  and	
  drought	
  tolerance.	
  CCAFS	
  is	
  also	
  receiving	
  feedback	
  from	
  the	
  crop	
  improvement	
  programmes	
  to	
  frame	
  criteria	
  and	
  knowledge	
  needs.	
  Rice,	
  beans,	
  sorghum	
  &	
  banana	
  have	
  been	
  selected	
  as	
  the	
  main	
  crops	
  to	
  focus	
  on	
  in	
  the	
  near	
  term,	
  due	
  to	
  their	
  biological	
  diversity,	
  the	
  range	
  of	
  extant	
  physiological	
  and	
  modelling	
  knowledge,	
  their	
  importance	
  for	
  the	
  CCAFS	
  regions,	
  and	
  researcher	
  and	
  stakeholder	
  interest.	
  (R21)	
  
	
   13	
  
2.6.	
  Internationally	
  or	
  regional	
  public	
  goods	
  character	
  of	
  research	
  Currently,	
  the	
  public	
  goods	
  character	
  of	
  the	
  CCAFS	
  results	
  does	
  not	
  raise	
  concerns.	
  The	
  outputs	
  and	
  communications	
  of	
  the	
  programme	
  in	
  2011	
  included	
  65	
  peer-­‐reviewed	
  journal	
  articles,	
  3	
  books,	
  24	
  book	
  chapters,	
  22	
  policy	
  briefs,	
  18	
  working	
  papers,	
  and	
  39	
  conference	
  proceedings.	
  A	
  viable	
  communication	
  and	
  knowledge	
  sharing	
  strategy	
  has	
  been	
  developed,	
  partner	
  agreements	
  contain	
  appropriate	
  clauses	
  regarding	
  intellectual	
  property	
  rights,	
  budgets	
  are	
  available	
  to	
  support	
  open	
  access	
  publication,	
  and	
  results	
  are	
  published	
  on	
  the	
  CCAFS	
  website	
  with	
  very	
  short	
  delays.	
  This	
  is	
  exemplified	
  by	
  the	
  publication	
  of	
  the	
  baseline	
  survey	
  data	
  within	
  three	
  months	
  of	
  completion.	
  	
  CCAFS	
  has	
  also	
  set	
  up	
  global	
  public-­‐access	
  databases,	
  such	
  as	
  AgTrials,	
  where	
  over	
  800	
  sets	
  of	
  crop	
  trial	
  data	
  from	
  the	
  CGIAR	
  and	
  partners	
  are	
  collated.However,	
  it	
  has	
  sometimes	
  been	
  difficult	
  to	
  obtain	
  results	
  for	
  these	
  databases	
  before	
  they	
  have	
  been	
  made	
  available	
  through	
  academic	
  publication.	
  	
  3	
  Strategic	
  issues	
  for	
  EC	
  &	
  CGIAR	
  The	
  Consortium	
  Board’s	
  decision	
  to	
  adopt	
  CRPs	
  as	
  the	
  primary	
  instrument	
  for	
  CGIAR	
  research	
  signals	
  several	
  fundamental	
  changes	
  in	
  the	
  way	
  the	
  system’s	
  research	
  is	
  planned,	
  approved	
  and	
  implemented:	
  Ø First,	
  the	
  CGIAR	
  should	
  progressively	
  function	
  as	
  a	
  single	
  institution,	
  with	
  its	
  Centres	
  collaborating	
  in	
  pursuit	
  of	
  shared	
  goals	
  and	
  objectives.	
  Ø Second	
  is	
  the	
  adoption	
  of	
  an	
  agricultural-­‐research-­‐for-­‐development	
  approach	
  throughout	
  the	
  CGIAR	
  system.	
  This	
  means	
  that	
  all	
  research	
  priorities	
  and	
  activities	
  should	
  be	
  guided	
  by	
  their	
  potential	
  contributions	
  to	
  the	
  four	
  system-­‐wide	
  outcomes.	
  	
  Ø Finally,	
  by	
  structuring	
  research	
  activities	
  into	
  CRPs,	
  the	
  Centres	
  and	
  their	
  partners	
  should	
  integrate	
  their	
  work	
  into	
  large	
  and	
  ambitious	
  programs	
  driven	
  by	
  their	
  potential	
  impact	
  on	
  development.	
  3.1	
  Lessons	
  regarding	
  the	
  CGIAR	
  reform	
  process	
  As	
  an	
  early,	
  fast-­‐tracked	
  CRP,	
  CCAFS	
  has	
  been	
  at	
  the	
  forefront	
  of	
  a	
  new	
  process	
  for	
  programme	
  development	
  and	
  implementation.	
  This	
  leading	
  position	
  has	
  enabled	
  CCAFS	
  to	
  make	
  significant	
  contributions	
  to	
  the	
  evolution	
  of	
  the	
  reform	
  process.	
  However,	
  this	
  has	
  generated	
  some	
  additional	
  difficulties.	
  	
  In	
  the	
  absence	
  of	
  standardised	
  systems	
  across	
  the	
  CGIAR,	
  CCAFS	
  has	
  invested	
  considerable	
  resources	
  to	
  develop	
  the	
  tools	
  required	
  for	
  managing	
  the	
  CRP:	
  including	
  modalities	
  for	
  planning,	
  financial	
  management,	
  and	
  internal	
  and	
  external	
  communications	
  and	
  reporting.	
  CCAFS	
  has	
  therefore	
  developed	
  management	
  processes	
  suiting	
  its	
  needs,	
  certainly	
  contributing	
  to	
  the	
  successful	
  implementation	
  of	
  the	
  programme’s	
  first	
  year.	
  However,	
  as	
  systems	
  standardise,	
  CCAFS	
  may	
  be	
  required	
  to	
  revise	
  practices	
  and	
  processes.	
  	
  A	
  second	
  difficulty	
  has	
  been	
  that	
  CCAFS	
  is	
  regularly	
  referred	
  to	
  as	
  an	
  example	
  of	
  positive	
  reform,	
  and	
  its	
  innovations	
  suggested	
  for	
  adoption	
  by	
  other	
  CRPs.	
  This	
  has	
  created	
  pressure	
  to	
  maintain	
  performance,	
  and	
  additional	
  time	
  demands	
  in	
  responding	
  to	
  requests	
  and	
  sharing	
  experiences.	
  The	
  raised	
  profile	
  of	
  CCAFS	
  could	
  also	
  proves	
  limiting	
  to	
  other	
  CRPs	
  if	
  they	
  are	
  expected	
  to	
  follow	
  CCAFS’	
  lead	
  on	
  these	
  issues.	
  CCAFS	
  has	
  also	
  faced	
  challenges	
  in	
  negotiating	
  the	
  diversity	
  of	
  management	
  systems	
  in	
  place	
  across	
  the	
  CGIAR	
  system.	
  These	
  have	
  related	
  to,	
  inter	
  alia,	
  governance	
  processes,	
  management	
  between	
  Centres	
  and	
  partners,	
  knowledge	
  sharing,	
  reporting	
  and	
  human	
  
	
   14	
  
resources.	
  For	
  example,	
  CCAFS	
  reported	
  that	
  Science	
  Officers	
  for	
  Regional	
  Programmes	
  were	
  not	
  recruited	
  until	
  the	
  end	
  of	
  2011	
  partly	
  due	
  differences	
  in	
  HR	
  policies	
  across	
  Centres	
  and	
  the	
  need	
  to	
  make	
  fixed	
  cost	
  salary	
  commitments	
  before	
  budgets	
  were	
  confirmed.	
  Such	
  challenges	
  are	
  cross-­‐cutting,	
  and	
  need	
  to	
  be	
  addressed	
  across	
  the	
  CGIAR	
  	
  as	
  soon	
  as	
  possible,	
  although	
  a	
  number	
  of	
  them	
  will	
  only	
  find	
  solutions	
  in	
  the	
  longer	
  term.	
  	
  More	
  broadly,	
  it	
  seems	
  that	
  uncertainties	
  resulting	
  from	
  the	
  reform	
  process	
  have	
  affected	
  morale	
  across	
  the	
  CGIAR.	
  Some	
  staff	
  have	
  no	
  doubt	
  responded	
  to	
  the	
  reform	
  with	
  considerable	
  energy	
  and	
  creativity.	
  However,	
  interviewees	
  reported	
  that	
  Centre	
  staff	
  turn-­‐over	
  has	
  been	
  high	
  over	
  the	
  last	
  two	
  years,	
  with	
  some	
  Centres	
  more	
  affected	
  than	
  others,	
  and	
  that	
  low	
  morale	
  had	
  reduced	
  the	
  staff	
  capacity	
  to	
  engage	
  creatively	
  and	
  innovatively	
  with	
  CRPs.	
  	
  	
  The	
  CGIAR’s	
  reform	
  is	
  a	
  change	
  process	
  that	
  will	
  not	
  be	
  finalised	
  in	
  two	
  years.	
  The	
  general	
  framework	
  looks	
  clearer	
  now,	
  but	
  implementation	
  still	
  faces	
  numerous	
  challenges	
  (R22).	
  3.1.1	
  Organisational	
  cohesiveness	
  and	
  partnerships	
  Harmonisation	
  Section	
  2.2	
  reported	
  on	
  the	
  challenges	
  CCAFS	
  faced	
  due	
  to	
  the	
  lack	
  of	
  harmonized	
  management	
  systems	
  across	
  the	
  CGIAR.	
  CU	
  staff	
  reported	
  that	
  CCAFS	
  Centre	
  Contact	
  Points	
  had	
  complained	
  of	
  increased	
  transaction	
  costs	
  under	
  the	
  new	
  CRP	
  system,	
  partly	
  because	
  scientists	
  are	
  required	
  to	
  fulfil	
  both	
  Centre	
  and	
  CRP	
  planning	
  and	
  reporting	
  procedures.	
  As	
  management	
  procedures	
  vary	
  from	
  Centre	
  to	
  Centre,	
  transaction	
  costs	
  have	
  been	
  high	
  as	
  CRP	
  and	
  Centre	
  staff	
  have	
  attempted	
  to	
  negotiate	
  multiple	
  systems,	
  often	
  resulting	
  in	
  ad	
  hoc	
  solutions.	
  The	
  CO	
  has	
  now	
  organised	
  working	
  groups	
  to	
  revise	
  CGIAR	
  financial	
  guidelines	
  pre-­‐dating	
  the	
  reform,	
  and	
  in	
  principle	
  this	
  should	
  result	
  in	
  greater	
  harmonisation	
  in	
  practices.	
  	
  	
  There	
  are	
  also	
  needs	
  for	
  harmonisation	
  in	
  other	
  areas	
  of	
  management,	
  including	
  intellectual	
  property	
  rights,	
  research	
  ethics8,	
  and	
  knowledge	
  management.	
  Human	
  resources	
  management	
  was	
  highlighted	
  as	
  an	
  area	
  of	
  concern,	
  with	
  policies	
  regarding	
  contracts	
  and	
  benefits	
  varying	
  across	
  Centres,	
  particularly	
  for	
  local	
  staff.	
  With	
  increasing	
  collaboration	
  between	
  different	
  Centres	
  through	
  CRPs,	
  this	
  situation	
  is	
  likely	
  to	
  become	
  untenable.	
  Potential	
  consequences	
  include	
  reduced	
  morale,	
  and	
  staff	
  attempting	
  to	
  move	
  to	
  different	
  Centres	
  to	
  attain	
  improved	
  benefits.	
  This	
  being	
  said,	
  strict	
  standardisation	
  of	
  all	
  procedures	
  would	
  likely	
  lead	
  to	
  increased	
  bureaucracy	
  and	
  transaction	
  costs.	
  CRPs	
  are	
  large	
  units	
  and	
  sufficiently	
  different	
  from	
  one	
  another	
  that	
  management	
  procedures	
  should	
  be	
  adapted	
  to	
  their	
  institutional	
  and	
  organisational	
  setting.	
  We	
  hence	
  recommend	
  a	
  part	
  of	
  flexibility	
  in	
  this	
  regard.	
  (R23)	
  Enhancing	
  system	
  wide	
  productivity	
  	
  The	
  decision	
  to	
  locate	
  CRPs	
  under	
  Host	
  Centres	
  is	
  likely	
  to	
  limit	
  the	
  potential	
  of	
  the	
  CGIAR	
  reform	
  to	
  enhance	
  performance	
  through	
  collaboration	
  and	
  competition.	
  CCAFS	
  was	
  in	
  the	
  position	
  to	
  invite	
  bids	
  from	
  potential	
  hosts	
  (see	
  Section	
  1),	
  but	
  other	
  CRPs	
  were	
  allocated	
  to	
  host	
  Centres.	
  	
  CCAFS	
  report	
  that	
  the	
  Director	
  General	
  of	
  CIAT	
  has	
  been	
  extremely	
  supportive.	
  However,	
  for	
  other	
  CRPs,	
  the	
  governance	
  structures	
  in	
  place	
  reduce	
  autonomy	
  to	
  pursue	
  their	
  objectives	
  rather	
  than	
  the	
  objectives	
  of	
  the	
  hosting	
  Centre.	
  For	
  example,	
  CRPs	
  could	
  be	
  pressured	
  to	
  allocate	
  more	
  Window	
  1	
  &	
  2	
  funding	
  to	
  the	
  hosting	
  Centre.	
  This	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  8	
  Not	
  every	
  Centre	
  has	
  a	
  research	
  ethics	
  committee	
  -­‐	
  most	
  have	
  policies	
  on	
  prior	
  informed	
  consent,	
  but	
  possibly	
  not	
  as	
  rigorous	
  as	
  in	
  universities.	
  
	
   15	
  
has	
  the	
  potential	
  to	
  lead	
  to	
  “tit	
  for	
  tat”	
  conflict	
  between	
  Centres	
  which	
  host	
  CRPs,	
  and	
  marginalise	
  those	
  that	
  do	
  not.	
  At	
  a	
  system	
  level,	
  separating	
  service	
  providers	
  (the	
  Centres)	
  from	
  programming	
  funds	
  (the	
  CRPs)	
  would	
  create	
  a	
  more	
  effective	
  internal	
  market	
  within	
  the	
  CGIAR.	
  It	
  could	
  also	
  contribute	
  to	
  increasing	
  opportunities	
  for	
  participation	
  of	
  non-­‐CGIAR	
  expertise	
  in	
  the	
  CRPs.	
  (R24)	
  Enhancing	
  participation	
  of	
  non-­‐CGIAR	
  expertise	
  One	
  intention	
  of	
  both	
  CCAFS	
  as	
  a	
  partnership	
  with	
  ESSP	
  and	
  the	
  CGIAR	
  reform	
  process	
  was	
  to	
  enhance	
  collaboration	
  with	
  non-­‐CGIAR	
  research	
  institutions.	
  	
  Strong	
  participation	
  of	
  non-­‐CGIAR	
  institutions	
  has	
  made	
  CCAFS	
  a	
  outlier	
  amongst	
  CRPs.	
  The	
  leadership	
  of	
  Thematic	
  Programmes	
  by	
  non-­‐CGIAR	
  institutions	
  and	
  the	
  Independent	
  Science	
  Panel’s	
  role	
  have	
  also	
  been	
  subject	
  to	
  some	
  criticism	
  within	
  the	
  CGIAR.	
  However,	
  the	
  Review	
  Team	
  considers	
  their	
  involvement	
  to	
  be	
  highly	
  valuable.	
  This	
  is	
  particularly	
  the	
  case	
  given	
  the	
  expertise	
  brought	
  to	
  bear	
  on	
  climate	
  change,	
  an	
  area	
  in	
  which	
  the	
  CGIAR	
  does	
  not	
  have	
  competitive	
  advantage.	
  There	
  are	
  other	
  areas	
  related	
  to	
  the	
  CGIAR	
  SLOs,	
  such	
  as	
  non-­‐productivity	
  aspects	
  of	
  food	
  security,	
  where	
  CGIAR	
  do	
  not	
  necessarily	
  have	
  competitive	
  advantage.	
  	
  However,	
  at	
  present	
  this	
  potential	
  is	
  not	
  maximised.	
  The	
  CGIAR	
  could	
  benefit	
  more	
  from	
  external	
  expertise	
  solicited	
  through	
  competitive	
  processes	
  for	
  Window	
  1	
  and	
  2	
  funds	
  managed	
  by	
  CRPs.	
  However,	
  to	
  date	
  such	
  engagement	
  has	
  been	
  limited.	
  	
  For	
  the	
  non-­‐CGIAR	
  partners	
  of	
  CRPs	
  the	
  funding	
  uncertainty	
  can	
  be	
  seen	
  as	
  an	
  acute	
  problem,	
  partly	
  due	
  to	
  their	
  more	
  limited	
  cash	
  on	
  hand,	
  but	
  also	
  due	
  to	
  different	
  accounting	
  practices	
  which	
  make	
  prefinancing	
  impossible,	
  and	
  to	
  financial	
  reporting	
  procedures	
  little	
  to	
  adapted	
  to	
  non	
  CGIAR	
  institutions.	
  Issues	
  of	
  financial	
  management	
  could	
  therefore	
  have	
  a	
  somewhat	
  negative	
  effect	
  on	
  the	
  concept	
  of	
  CRPs	
  as	
  platforms	
  for	
  partnership	
  between	
  the	
  CGIAR,	
  universities,	
  and	
  other	
  science	
  organisations.	
  If	
  an	
  intention	
  of	
  the	
  CGIAR	
  reform	
  process	
  is	
  to	
  open	
  up	
  collaboration	
  between	
  CGIAR	
  Centres	
  and	
  other	
  stakeholders,	
  this	
  is	
  an	
  area	
  that	
  should	
  be	
  addressed	
  by	
  the	
  Consortium	
  Office	
  (R9)	
  Conversely,	
  more	
  can	
  be	
  done	
  to	
  realize	
  the	
  potential	
  for	
  CCAFS	
  and	
  other	
  CRPs	
  to	
  mobilise	
  and	
  focus	
  resources	
  from	
  other	
  sources.	
  For	
  example,	
  with	
  significant	
  sources	
  available	
  to	
  the	
  global	
  change	
  research	
  community,	
  CCAFS	
  should	
  be	
  in	
  a	
  position	
  to	
  engage	
  with	
  recipients	
  and	
  shape	
  their	
  research.	
  Options	
  might	
  include	
  offering	
  top-­‐up	
  funding	
  to	
  universities	
  receiving	
  research	
  grants	
  that	
  align	
  with	
  CCAFS	
  priorities.	
  	
  Current	
  management	
  systems	
  do	
  not	
  incentivise	
  such	
  behaviour	
  from	
  CRPs.	
  For	
  example,	
  if	
  a	
  university	
  brought	
  government	
  funds	
  to	
  participate	
  in	
  CCAFS,	
  these	
  leveraged	
  funds	
  would	
  not	
  be	
  reflected	
  CCAFS	
  financial	
  or	
  results	
  reporting	
  framework.	
  Mechanisms	
  relating	
  leveraged	
  funds	
  to	
  allocations	
  of	
  Window	
  1	
  &	
  2	
  funding	
  could	
  be	
  a	
  step	
  to	
  incentivising	
  these	
  approaches	
  (R25).	
  3.1.2	
  Science	
  quality	
  versus	
  development	
  outcomes	
  As	
  noted	
  in	
  Section	
  2.3.2	
  there	
  is	
  a	
  potential	
  tension	
  between	
  the	
  CGIAR’s	
  role	
  as	
  a	
  research	
  institution	
  and	
  the	
  increased	
  focus	
  from	
  donors	
  on	
  achieving	
  development	
  impact.	
  	
  1. As	
  even	
  ‘failed’	
  research	
  can	
  lead	
  to	
  breakthroughs,	
  short-­‐	
  or	
  medium-­‐term	
  impact	
  should	
  not	
  be	
  the	
  dominant	
  criteria	
  for	
  evaluation.	
  	
  	
  2. The	
  knowledge	
  produced	
  at	
  CCAFS	
  sites	
  is	
  scientific	
  only	
  if	
  it	
  increases	
  the	
  state	
  of	
  the	
  art	
  and	
  the	
  body	
  of	
  theories	
  in	
  the	
  disciplines	
  involved.	
  However	
  successful	
  innovation	
  in	
  field	
  sites	
  could	
  be	
  supported	
  through	
  empirical	
  work	
  rather	
  than	
  true	
  science,	
  and	
  in	
  some	
  cases	
  this	
  might	
  be	
  more	
  efficient	
  in	
  terms	
  
	
   16	
  
of	
  responding	
  to	
  local	
  needs	
  in	
  a	
  timely	
  fashion.	
  However,	
  the	
  consequence	
  would	
  be	
  local	
  impact	
  without	
  significant	
  disciplinary	
  contributions.	
  (R26)	
  3. CCAFS	
  has	
  started	
  to	
  address	
  this	
  issue	
  through	
  the	
  concept	
  of	
  “climate	
  smart	
  villages”.	
  However,	
  integration	
  of	
  the	
  top-­‐down	
  Thematic	
  program	
  research	
  and	
  bottom-­‐up	
  participatory	
  action	
  research	
  will	
  be	
  one	
  of	
  CCAFS’	
  main	
  scientific	
  challenges	
  to	
  be	
  addressed	
  in	
  the	
  near	
  future.	
  This	
  problem	
  is	
  compounded	
  by	
  the	
  challenges	
  of	
  integrating	
  scientific	
  results	
  from	
  the	
  assemblage	
  of	
  relatively	
  independent	
  Centre-­‐led	
  activities,	
  using	
  different	
  methods	
  at	
  different	
  sites.	
  For	
  CCAFS,	
  these	
  issues	
  this	
  will	
  require	
  close	
  attention	
  from	
  the	
  ISP.	
  However,	
  donors	
  and	
  the	
  CGIAR	
  as	
  a	
  whole	
  should	
  evaluate	
  how	
  other	
  CRPs	
  are	
  managing	
  scientific	
  integration,	
  quality	
  and	
  contribution	
  at	
  the	
  programme	
  level.	
  (R27)	
  4. The	
  encouragement	
  of	
  participatory	
  and	
  demand	
  led	
  research	
  should	
  not	
  lead	
  to	
  the	
  exclusion	
  of	
  more	
  traditional	
  models	
  of	
  investigation	
  led	
  by	
  scientific	
  agendas.	
  This	
  is	
  especially	
  valid	
  in	
  the	
  case	
  of	
  CCAFS,	
  where	
  demand	
  driven	
  research	
  is	
  likely	
  to	
  focus	
  on	
  immediate	
  needs	
  such	
  as	
  climate	
  risk	
  management	
  rather	
  than	
  progressive	
  adaptation	
  to	
  states	
  30	
  years	
  in	
  the	
  future.	
  	
  A	
  CRP	
  such	
  as	
  CCAFS	
  should	
  therefore	
  draw	
  on	
  a	
  range	
  of	
  scientific	
  approaches,	
  methods	
  and	
  techniques,	
  according	
  to	
  their	
  respective	
  merits.	
  Participatory	
  action	
  research	
  is	
  one	
  such	
  approach,	
  but	
  it	
  should	
  be	
  able	
  to	
  contribute	
  to	
  strategic,	
  not	
  just	
  locally	
  specific,	
  knowledge	
  production.	
  Conversely,	
  whilst	
  top-­‐down	
  research	
  should	
  be	
  oriented	
  to	
  address	
  research	
  needs	
  identified	
  through	
  participatory	
  processes	
  where	
  appropriate,	
  there	
  must	
  also	
  be	
  space	
  for	
  research	
  led	
  by	
  scientific	
  agendas.	
  In	
  the	
  Review	
  Team’s	
  opinion,	
  CCAFS	
  PMC	
  are	
  well	
  aware	
  of	
  these	
  issues	
  and	
  is	
  either	
  taking	
  appropriate	
  steps	
  or	
  does	
  not	
  anticipate	
  immediate	
  problems.	
  For	
  example,	
  incentive	
  structures	
  for	
  CGIAR	
  researchers	
  focus	
  on	
  quality	
  science	
  leading	
  to	
  peer-­‐reviewed	
  publications,	
  favouring	
  traditional	
  research	
  over	
  participatory	
  approaches.	
  Indeed,	
  to	
  an	
  extent	
  a	
  new	
  focus	
  on	
  development	
  impact	
  in	
  programming	
  prioritisation	
  should	
  incentivise	
  a	
  more	
  development	
  responsive	
  and	
  externally-­‐oriented	
  CGIAR.	
  However,	
  donors	
  need	
  to	
  be	
  aware	
  of	
  the	
  risks	
  to	
  quality	
  research	
  that	
  arise	
  from	
  a	
  focus	
  on	
  accountability	
  in	
  terms	
  of	
  attributional	
  development	
  impact	
  (see	
  Section	
  3.2).	
  	
  3.2	
  Monitoring	
  and	
  Evaluation	
  at	
  the	
  Consortium	
  Level	
  The	
  development	
  of	
  system-­‐wide	
  M&E	
  frameworks	
  and	
  procedures	
  is	
  an	
  important	
  component	
  of	
  the	
  CGIAR	
  reform	
  process.	
  The	
  Strategy	
  and	
  Results	
  Framework	
  (SRF)	
  is	
  intended	
  to	
  support	
  prioritisation	
  in	
  resource	
  allocation,	
  performance	
  management,	
  and	
  reduce	
  M&E	
  burdens	
  by	
  centralising	
  accountability.	
  	
  At	
  this	
  stage	
  in	
  the	
  process,	
  Intermediate	
  Development	
  Outcomes	
  (IDOs)	
  are	
  being	
  developed	
  as	
  a	
  means	
  of	
  bridging	
  CRP	
  outcomes	
  and	
  objectives	
  with	
  the	
  four	
  System	
  Level	
  Outcomes	
  (SLOs)	
  of	
  the	
  SRF9.	
  IDOs	
  are	
  being	
  developed	
  both	
  at	
  a	
  system	
  level	
  (“top-­‐down”)	
  and	
  for	
  each	
  CRP	
  (“bottom-­‐up”),	
  with	
  an	
  expectation	
  that	
  they	
  will	
  be	
  used	
  to	
  support	
  resource	
  allocation	
  across	
  the	
  CGIAR	
  in	
  2014.	
  There	
  is	
  an	
  expectation	
  that	
  IDOs	
  will	
  be	
  quantitative	
  on	
  focused	
  on	
  evaluating	
  the	
  impact	
  of	
  CGIAR	
  research.	
  	
  An	
  increased	
  emphasis	
  on	
  quantitative	
  impact	
  assessment	
  may	
  require	
  CCAFS	
  to	
  redesign	
  its	
  tools	
  and	
  approaches.	
  CCAFS	
  is	
  aware	
  of	
  the	
  importance	
  of	
  complying	
  with	
  any	
  CGIAR	
  protocols	
  that	
  emerge,	
  and	
  has	
  already	
  begun	
  work	
  to	
  identify	
  IDOs	
  for	
  the	
  SRF.	
  CCAFS	
  CU	
  reported	
  that	
  the	
  process	
  of	
  developing	
  IDOs	
  had	
  focused	
  minds,	
  and	
  will	
  provide	
  the	
  mechanism	
  to	
  send	
  strong	
  messages	
  to	
  Centres	
  that	
  future	
  activities	
  proposed	
  for	
  funding	
  through	
  CCAFS	
  will	
  need	
  to	
  be	
  strategically	
  oriented.	
  Theme	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
   	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  9	
  The	
  four	
  SLOs	
  are:	
  reduced	
  rural	
  poverty,	
  improved	
  food	
  security,	
  improved	
  nutrition	
  and	
  health,	
  and	
  sustainably	
  managed	
  natural	
  resources	
  
	
   17	
  
Leaders	
  have	
  already	
  begun	
  identifying	
  Centre-­‐led	
  activities	
  weakly	
  connected	
  to	
  impact	
  pathways	
  (see	
  Section	
  2.1).	
  	
  If	
  the	
  resulting	
  CGIAR	
  framework	
  is	
  dominated	
  by	
  a	
  concern	
  with	
  quantitative	
  attributional	
  impact	
  assessment,	
  however,	
  CCAFS	
  is	
  likely	
  to	
  find	
  compliance	
  extremely	
  challenging.	
  For	
  example,	
  assessing	
  outcomes	
  of	
  climate	
  risk	
  management	
  (Theme	
  2)	
  can	
  be	
  more	
  readily	
  assessed	
  than	
  evaluating	
  outcomes	
  of	
  progressive	
  adaptation	
  to	
  future	
  environmental	
  change	
  (Theme	
  1).	
  The	
  engagement	
  of	
  CCAFS	
  with	
  external	
  partners,	
  and	
  areas	
  where	
  causal	
  chains	
  from	
  research	
  to	
  development	
  outcome	
  is	
  weak,	
  pose	
  different	
  challenges	
  for	
  questions	
  of	
  attribution	
  versus	
  contribution	
  evaluation.	
  Additionally,	
  responding	
  quantitative	
  impact	
  indicators	
  on	
  an	
  annual	
  basis,	
  as	
  some	
  have	
  suggested,	
  would	
  significantly	
  stress	
  programme	
  resources.	
  The	
  Review	
  Team	
  therefore	
  has	
  deep	
  concerns	
  that	
  attempting	
  to	
  completely	
  and	
  rigorously	
  address	
  issues	
  of	
  quantitative	
  impact	
  assessment	
  will	
  divert	
  valuable	
  intellectual	
  energy	
  away	
  from	
  substantive	
  research	
  priorities.	
  	
  Whilst	
  the	
  increased	
  concern	
  with	
  aligning	
  CGIAR	
  with	
  development	
  needs	
  and	
  priorities	
  is	
  welcome,	
  CCAFS	
  remains	
  a	
  research	
  programme.	
  CCAFS	
  has	
  demonstrated	
  how	
  the	
  logical	
  framework	
  can	
  be	
  effectively	
  used	
  as	
  a	
  tool	
  for	
  priority	
  setting	
  and	
  resource	
  allocation.	
  However,	
  attempting	
  to	
  use	
  logical	
  frameworks	
  to	
  assess	
  the	
  attributional	
  impact	
  of	
  research	
  can	
  result	
  in	
  claims	
  that	
  are	
  of	
  tenuous	
  validity,	
  or	
  even	
  disingenuous.	
  Both	
  donors	
  and	
  the	
  Consortium	
  Office	
  should	
  be	
  mindful	
  also	
  of	
  potential	
  tensions	
  between	
  the	
  innovative,	
  risky	
  nature	
  of	
  research	
  and	
  the	
  tendency	
  of	
  quantitative	
  accountability	
  frameworks	
  to	
  incentivize	
  low	
  risk	
  behaviour.	
  	
  Ideally,	
  the	
  CGIAR	
  would	
  adopt	
  an	
  M&E	
  framework	
  supporting	
  evaluative	
  programme	
  learning	
  rather	
  than	
  focused	
  on	
  accountability	
  and	
  assessment	
  of	
  attributional	
  impact.	
  However,	
  if	
  the	
  CGIAR	
  does	
  choose	
  to	
  focus	
  on	
  attributional	
  impact,	
  as	
  the	
  SRF	
  evolves,	
  increasing	
  emphasis	
  should	
  be	
  placed	
  on	
  the	
  evaluation	
  of	
  quality	
  research	
  as	
  the	
  principle	
  means	
  of	
  achieving	
  development	
  impact.	
  (R28,	
  R29	
  &	
  R30)	
  	
  3.3	
  Impact,	
  sustainability	
  &	
  visibility	
  of	
  EC	
  co-­‐financing	
  To	
  recap,	
  in	
  2011	
  EC/IFAD	
  support	
  to	
  CCAFS	
  was	
  USD	
  2.8	
  million,	
  or	
  nearly	
  5%	
  of	
  of	
  CCAFS’	
  total	
  budget	
  of	
  USD	
  62	
  million.	
  However,	
  although	
  CCAFS	
  receives	
  a	
  relatively	
  small	
  proportion	
  of	
  its	
  budget	
  from	
  the	
  EC,	
  the	
  programme	
  is	
  of	
  major	
  interest	
  for	
  the	
  Commission.	
  Europe’s	
  political	
  positions	
  on	
  climate	
  change	
  justify	
  commitments	
  to	
  influencing	
  the	
  international	
  community	
  through	
  research,	
  and	
  CCAFS	
  is	
  one	
  potential	
  vehicle.	
  Impact	
  This	
  review	
  does	
  not	
  focus	
  on	
  the	
  impact	
  of	
  EC	
  funding	
  on	
  beneficiaries,	
  but	
  does	
  offer	
  a	
  perspective	
  on	
  the	
  impact	
  of	
  EC	
  funding	
  on	
  CCAFS.	
  The	
  EC	
  is	
  the	
  only	
  donor	
  to	
  geographically	
  and	
  thematically	
  earmark	
  its	
  funding	
  for	
  CCAFS,	
  and	
  in	
  the	
  past	
  changes	
  in	
  priorities	
  led	
  to	
  some	
  additional,	
  albeit	
  relatively	
  minor,	
  transaction	
  costs	
  in	
  programme	
  planning.	
  CCAFS	
  has	
  also	
  been	
  obliged	
  to	
  prepare	
  annual	
  proposals	
  and	
  reports	
  to	
  EC/IFAD,	
  rather	
  than	
  operate	
  under	
  longer-­‐term	
  commitments.	
  	
  Current	
  arrangements	
  therefore	
  incur	
  increased	
  transaction	
  costs	
  and	
  uncertainty	
  for	
  the	
  CCAFS	
  management	
  team.	
  However,	
  the	
  arrangements	
  appear	
  to	
  have	
  no	
  substantial	
  impact	
  on	
  the	
  directions	
  of	
  CCAFS	
  programming.	
  Due	
  to	
  the	
  relatively	
  small	
  size	
  of	
  EC	
  funding	
  within	
  the	
  whole	
  CCAFS	
  envelope,	
  the	
  PMC	
  is	
  able	
  to	
  redistribute	
  other	
  sources	
  of	
  funding	
  to	
  balance	
  against	
  changing	
  EC	
  priorities,	
  thereby	
  conserving	
  planned	
  budget	
  allocations	
  and	
  programming.	
  	
  The	
  request	
  for	
  a	
  3-­‐year	
  proposal	
  for	
  2012-­‐14	
  may	
  indicate	
  that	
  the	
  EC	
  is	
  prepared	
  to	
  make	
  longer-­‐term	
  commitments,	
  which	
  would	
  reduce	
  CCAFS	
  transaction	
  costs.	
  
	
   18	
  
However,	
  if	
  the	
  EC/IFAD	
  wishes	
  to	
  have	
  more	
  impact	
  on	
  programming	
  directions,	
  then	
  they	
  would	
  be	
  better	
  served	
  by	
  abandoning	
  geographic	
  and	
  thematic	
  earmarking	
  and	
  engaging	
  CCAFS	
  through	
  other	
  means	
  (R31)	
  .	
  Strengthening	
  EC	
  Impact	
  on	
  CCAFS	
  Programming	
  and	
  the	
  CGIAR	
  If	
  the	
  EC	
  wishes	
  to	
  have	
  more	
  influence	
  on	
  CCAFS	
  programming,	
  it	
  is	
  likely	
  to	
  be	
  more	
  effective	
  to	
  fund	
  the	
  whole	
  programme	
  package	
  but	
  engage	
  through	
  different	
  avenues.	
  Three	
  possibilities	
  are	
  proposed	
  below.	
  	
  i)	
  Participation	
  in	
  a	
  stakeholder	
  forum	
  The	
  Independent	
  Science	
  Panel	
  has	
  a	
  strategic	
  guiding	
  role.	
  A	
  donor	
  representative	
  would	
  negate	
  the	
  ISP’s	
  independence,	
  so	
  alternatives	
  should	
  be	
  explored	
  in	
  terms	
  of	
  promoting	
  dialogue.	
  An	
  option	
  would	
  be	
  to	
  suggest	
  that	
  CCAFS	
  establish	
  a	
  “stakeholder	
  forum“	
  where	
  donors,	
  civil	
  society	
  organisations,	
  and	
  representatives	
  of	
  international	
  and	
  regional	
  organisations	
  have	
  opportunities	
  to	
  interact	
  with	
  CCAFS.	
  This	
  might	
  involve	
  joint	
  meetings	
  in	
  situ	
  on	
  the	
  back	
  of	
  ISP	
  meetings,	
  showcasing	
  national	
  partners,	
  with	
  field	
  visit	
  and	
  media	
  presentations/discussion.	
  This	
  could	
  open	
  opportunities	
  for	
  dialogue	
  on	
  programming	
  without	
  making	
  CCAFS	
  directly	
  beholden	
  to	
  donors	
  or	
  increasing	
  accountability	
  burdens	
  (R32)	
  .	
  ii)	
  Organising	
  the	
  scientific	
  contribution	
  of	
  European	
  ARD	
  A	
  third	
  option	
  would	
  be	
  to	
  organise	
  the	
  EU	
  scientific	
  and	
  academic	
  contribution	
  to	
  CCAFS	
  and	
  other	
  CRPs.	
  As	
  mentioned	
  above	
  (Section	
  3.1),	
  the	
  current	
  participation	
  of	
  Advanced	
  Research	
  Institutes	
  and	
  European	
  universities	
  in	
  CCAFS	
  are	
  below	
  initial	
  expectations.	
  Difficulties	
  in	
  the	
  organisation	
  and	
  management	
  of	
  cooperation	
  explain	
  part	
  of	
  the	
  situation,	
  but	
  another	
  part	
  lies	
  within	
  the	
  universities	
  and	
  research	
  institutes	
  themselves.	
  	
  In	
  many	
  subject	
  areas,	
  their	
  potential	
  or	
  capacity	
  is	
  dispersed	
  across	
  too	
  many	
  different	
  institutions	
  to	
  allow	
  a	
  global	
  and	
  balanced	
  negotiation	
  with	
  the	
  CGIAR	
  or	
  individual	
  CRPs.	
  The	
  archetypical	
  example	
  is	
  the	
  contribution	
  of	
  tens	
  of	
  university-­‐based	
  PhD	
  students,	
  doing	
  their	
  research	
  in	
  connection	
  with	
  a	
  Centre	
  and	
  providing	
  considerable	
  added	
  value,	
  but	
  are	
  too	
  dispersed	
  to	
  generate	
  joint	
  programming	
  between	
  the	
  host	
  center	
  (or	
  CRP)	
  and	
  the	
  said	
  Universities.	
  The	
  University	
  of	
  Copenhagen	
  has	
  shown	
  the	
  way	
  by	
  offering	
  globally	
  the	
  participation	
  of	
  a	
  few	
  PhD	
  students	
  to	
  measure	
  the	
  potential	
  of	
  C	
  sequestration	
  in	
  a	
  number	
  of	
  cropping	
  systems.	
  This	
  kind	
  of	
  agreement	
  could	
  be	
  scaled	
  out,	
  provided	
  the	
  universities	
  are	
  able	
  to	
  collaboratively	
  organise	
  themselves	
  to	
  offer	
  a	
  critical	
  mass	
  of	
  research	
  and	
  education	
  options,	
  and	
  to	
  negotiate	
  jointly	
  with	
  the	
  CGIAR	
  and	
  CRPs.	
  An	
  option	
  could	
  be	
  to	
  explore	
  opportunities	
  for	
  financial	
  incentives	
  to	
  increase	
  participation	
  of	
  European	
  scientists	
  in	
  the	
  CRPs.	
  This	
  might	
  involve	
  calls	
  for	
  proposals	
  funded	
  under	
  the	
  framework	
  Programme	
  for	
  Research	
  and	
  Development	
  ,	
  or	
  by	
  the	
  Food	
  Security	
  Thematic	
  Programme	
  to	
  offer	
  direct	
  funding	
  to	
  EC	
  research	
  institutions	
  which	
  would	
  develop	
  a	
  strategic	
  contribution	
  to	
  a	
  CRP,	
  with	
  the	
  possibility	
  of	
  supplementary	
  top-­‐up	
  funding	
  from	
  the	
  CRP	
  for	
  strategic	
  research.	
  	
  iii)	
  Supporting	
  the	
  CO	
  to	
  organise	
  collaboration	
  between	
  Centres	
  As	
  mentioned	
  before,	
  the	
  Consortium	
  Office	
  faces	
  considerable	
  challenges	
  in	
  organising	
  the	
  new	
  system.	
  The	
  EC	
  may	
  be	
  able	
  to	
  offer	
  useful	
  support	
  to	
  the	
  central	
  functions	
  of	
  the	
  CO,	
  especially	
  to	
  develop	
  partnerships	
  with	
  non	
  conventional	
  CGIAR	
  stakeholders.	
  The	
  staff	
  and	
  resources	
  of	
  the	
  CO	
  are	
  currently	
  too	
  weak	
  to	
  adequately	
  regulate	
  the	
  
	
   19	
  
global	
  system,	
  and	
  additional	
  human	
  resources	
  offered	
  by	
  the	
  Commission	
  could	
  be	
  welcomed.	
  (R33)	
  Funding	
  Issues	
  Current	
  EC/IFAD	
  funding	
  to	
  CCAFS	
  can	
  be	
  considered	
  sustainable,	
  as	
  it	
  comprises	
  a	
  relatively	
  small	
  proportion	
  of	
  total	
  CRP	
  funding,	
  and	
  its	
  loss	
  would	
  not	
  be	
  critical.	
  However,	
  current	
  arrangements	
  for	
  administration	
  of	
  EC	
  funds	
  do	
  have	
  transaction	
  costs	
  for	
  CCAFS:	
  for	
  example	
  a	
  specific	
  cost	
  centre	
  is	
  needed	
  to	
  manage	
  EC	
  funds,	
  and	
  separate	
  annual	
  proposals	
  and	
  reports	
  need	
  to	
  be	
  written.	
  More	
  serious	
  are	
  the	
  consequences	
  from	
  delays	
  in	
  contracting	
  and	
  disbursing	
  funds.	
  The	
  EC	
  contract	
  for	
  2011	
  was	
  only	
  signed	
  in	
  August	
  2012.	
  For	
  example,	
  due	
  to	
  exchange	
  rate	
  fluctuations	
  between	
  the	
  USD	
  and	
  Euro	
  since	
  the	
  end	
  of	
  2011,	
  the	
  potential	
  costs	
  of	
  this	
  delay	
  to	
  CCAFS	
  amount	
  to	
  USD	
  500	
  000.	
  As	
  another	
  example,	
  the	
  University	
  of	
  Copenhagen	
  is	
  awaiting	
  €500	
  000	
  from	
  a	
  2009-­‐2010	
  EC	
  grant	
  to	
  the	
  Challenge	
  Program.	
  The	
  funds	
  should	
  have	
  been	
  received	
  within	
  months	
  of	
  the	
  final	
  report	
  being	
  submitted	
  to	
  IFAD	
  in	
  April	
  2011.	
  These	
  delays	
  add	
  to	
  financial	
  uncertainty	
  and	
  the	
  transaction	
  costs	
  of	
  CCAFS	
  management.	
  Their	
  consequences	
  demand	
  time	
  and	
  attention	
  by	
  the	
  ISP,	
  CCAFS	
  CU	
  staff,	
  and	
  Centre	
  staff	
  in	
  discussion	
  and	
  administration	
  disproportionate	
  to	
  the	
  amount	
  of	
  money	
  involved.	
  (R34)	
  Visibility	
  Due	
  to	
  contractual	
  agreement,	
  the	
  EC	
  and	
  IFAD	
  logos	
  appear	
  on	
  almost	
  all	
  outputs.	
  In	
  terms	
  of	
  visibility	
  per	
  dollar,	
  the	
  EC	
  does	
  well	
  in	
  comparison	
  to	
  other	
  donors.	
  By	
  contrast,	
  DfID	
  contributes	
  11.4	
  million	
  USD	
  to	
  CCAFS	
  through	
  Window	
  2	
  and	
  does	
  not	
  receive	
  a	
  specific	
  mention	
  on	
  products.	
  In	
  global	
  forums	
  –	
  such	
  as	
  CCAFS-­‐run	
  events	
  at	
  UNFCCC	
  COPs	
  –	
  EC	
  visibility	
  has	
  been	
  high.	
  The	
  EC	
  logo	
  is	
  prominent,	
  EC	
  representatives	
  have	
  been	
  involved,	
  and	
  their	
  reported	
  feedback	
  to	
  CCAFS	
  indicates	
  they	
  have	
  appreciated	
  the	
  consequent	
  exposure.	
  The	
  political	
  sensitivity	
  of	
  climate	
  change	
  negotiations	
  mean	
  that	
  the	
  current	
  level	
  of	
  visibility	
  is	
  appropriate,	
  as	
  negotiating	
  positions	
  amongst	
  CCAFS	
  stakeholders	
  vary.	
  Too	
  close	
  an	
  alignment	
  with	
  the	
  EC	
  could	
  either	
  risk	
  CCAFS	
  relations	
  with	
  other	
  stakeholders,	
  or	
  potentially	
  embarrass	
  the	
  EC.	
  	
  4.	
  Conclusions	
  	
  Established	
  as	
  a	
  Challenge	
  Programme	
  before	
  it	
  became	
  a	
  CRP,	
  with	
  a	
  secretariat	
  based	
  outside	
  a	
  CGIAR	
  Centre,	
  CCAFS	
  has	
  a	
  unique	
  structure.	
  Challenges	
  in	
  institutional	
  arrangements	
  with	
  non-­‐CGIAR	
  research	
  institutions	
  during	
  the	
  transition	
  to	
  the	
  new	
  CRP	
  have	
  highlighted	
  some	
  obstacles	
  that	
  the	
  CGIAR	
  reform	
  process	
  still	
  needs	
  to	
  address.	
  	
  With	
  its	
  three	
  Regional	
  and	
  four	
  Thematic	
  Programmes,	
  the	
  structure	
  of	
  CCAFS	
  is	
  consistent	
  and	
  relevant.	
  However	
  the	
  majority	
  of	
  funding	
  supports	
  Centre-­‐led	
  activities	
  through	
  Windows	
  2	
  &	
  3,	
  and	
  there	
  are	
  some	
  challenges	
  with	
  their	
  alignment	
  to	
  CCAFS’	
  strategic	
  priorities.	
  After	
  18	
  months	
  of	
  existence,	
  CCAFS	
  has	
  in	
  place	
  processes	
  to	
  ensure	
  the	
  relevance	
  of	
  Thematic	
  and	
  Regional	
  Programme	
  activities	
  at	
  local,	
  national	
  and	
  regional	
  levels,	
  as	
  well	
  as	
  positioning	
  itself	
  in	
  terms	
  of	
  strategic	
  research.	
  However,	
  there	
  are	
  opportunities	
  to	
  improve,	
  strengthen	
  focus	
  on	
  comparatively	
  under-­‐resourced	
  areas	
  within	
  the	
  portfolio,	
  and	
  capacitate	
  CCAFS	
  sites	
  teams	
  who	
  will	
  be	
  main	
  interlocutors	
  with	
  communities.	
  The	
  right	
  balance	
  has	
  still	
  to	
  be	
  found	
  between	
  research	
  on	
  the	
  sites	
  governed	
  by	
  development	
  impact	
  and	
  thematic	
  research	
  guided	
  by	
  the	
  scientific	
  challenges.	
  An	
  increased	
  focus	
  on	
  achieving	
  development	
  impact	
  on	
  the	
  ground,	
  
	
   20	
  
respecting	
  the	
  rhythm	
  of	
  Participatory	
  Action	
  Research	
  may	
  challenge	
  the	
  CGIAR’s	
  mandate	
  for	
  high	
  quality	
  research	
  which	
  can	
  demand	
  to	
  exert	
  independently	
  of	
  any	
  immediate	
  stakeholders	
  supported	
  need.	
  This	
  is	
  especially	
  true	
  in	
  the	
  case	
  of	
  Climate	
  Change,	
  a	
  topic	
  for	
  which	
  the	
  priorities	
  should	
  be	
  assessed	
  along	
  the	
  economic	
  and	
  social	
  needs	
  in	
  20-­‐50	
  years	
  time.	
  The	
  CO	
  and	
  FC	
  should	
  also	
  be	
  mindful	
  of	
  potential	
  tensions	
  between	
  the	
  innovative,	
  risky	
  nature	
  of	
  research	
  and	
  the	
  tendency	
  of	
  quantitative	
  accountability	
  frameworks	
  (such	
  as	
  those	
  proposed	
  for	
  the	
  whole	
  CGIAR	
  system)	
  to	
  incentivize	
  low	
  risk	
  behaviour,	
  and	
  potentially,	
  therefore,	
  low	
  quality	
  research.	
  Harmonisation	
  of	
  procedures	
  within	
  the	
  CGIAR	
  system	
  should	
  be	
  considered	
  with	
  care,	
  with	
  the	
  overall	
  objective	
  of	
  reducing	
  transaction	
  costs	
  and	
  with	
  sufficient	
  flexibility	
  to	
  account	
  for	
  specificities	
  of	
  each	
  programme.	
  The	
  location	
  of	
  CRPs	
  within	
  host	
  Centres	
  may	
  limit	
  the	
  potential	
  for	
  enhanced	
  performance	
  across	
  the	
  CGIAR,	
  and	
  locating	
  the	
  CRPs	
  under	
  the	
  Consortium	
  Office	
  will	
  probably	
  have	
  to	
  be	
  considered	
  in	
  the	
  future.	
  The	
  CO	
  and	
  FC	
  can	
  also	
  do	
  more	
  to	
  incentivise	
  participation	
  of	
  non-­‐CGIAR	
  research	
  institutions	
  in	
  CRPs.	
  Finally,	
  current	
  EC/IFAD	
  funding	
  arrangements	
  do	
  not	
  have	
  a	
  significant	
  influence	
  on	
  CCAFS’	
  programming.	
  If	
  the	
  EC	
  wishes	
  to	
  increase	
  this	
  influence,	
  it	
  should	
  consider	
  other	
  means	
  than	
  earmarking	
  its	
  contribution	
  to	
  specific	
  themes	
  or	
  regions.	
  It	
  is	
  suggested	
  to	
  establish	
  and	
  participate	
  to	
  a	
  “stakeholder	
  forum”	
  where	
  donors,	
  civil	
  society	
  organisations,	
  and	
  representatives	
  of	
  international	
  and	
  regional	
  organisations	
  would	
  have	
  opportunities	
  to	
  interact	
  with	
  CCAFS.	
  EC	
  could	
  also	
  directly	
  encourage	
  EU	
  research	
  institutions	
  to	
  develop	
  programmes	
  within	
  the	
  CCAFS	
  framework.	
  It	
  could	
  also	
  provide	
  backstopping	
  support	
  to	
  the	
  Consortium	
  Office	
  to	
  facilitate	
  partnership	
  mechanisms,	
  including	
  those	
  of	
  non	
  traditional	
  CGIAR	
  partners.	
  Finally,	
  CCAFS	
  is	
  a	
  strategic	
  programme	
  for	
  the	
  EC,	
  IFAD	
  and	
  the	
  CGIAR.	
  Despite	
  challenges	
  in	
  the	
  first	
  year	
  of	
  operation	
  as	
  a	
  CRP	
  and	
  some	
  remaining	
  scientific	
  and	
  managerial	
  challenges,	
  significant	
  progress	
  has	
  been	
  made.	
  The	
  programme	
  is	
  on	
  course	
  and	
  is	
  managed	
  by	
  highly	
  competent	
  professional	
  staff.	
  The	
  EC	
  maintain	
  current	
  levels	
  of	
  funding,	
  and	
  consider	
  increased	
  financial	
  support,	
  with	
  multiple	
  year	
  commitments,	
  inline	
  with	
  CCAFS’	
  capacity	
  to	
  absorb	
  additional	
  funding.	
  However,	
  delays	
  in	
  disbursing	
  funds,	
  as	
  those	
  faced	
  in	
  2011,	
  should	
  be	
  strongly	
  avoided.	
  	
   	
  
	
   21	
  
5.	
  Recommendations	
  Relevance	
  CCAFS	
  has	
  in	
  place	
  processes	
  to	
  ensure	
  relevance	
  of	
  Thematic	
  and	
  Regional	
  Program	
  activities	
  at	
  local,	
  national	
  and	
  regional	
  levels,	
  as	
  well	
  as	
  positioning	
  itself	
  in	
  terms	
  of	
  strategic	
  research.	
  However,	
  there	
  are	
  opportunities	
  to	
  improve	
  the	
  alignment	
  of	
  Centre	
  activities,	
  strengthen	
  focus	
  on	
  comparatively	
  under-­‐resourced	
  areas	
  within	
  the	
  portfolio,	
  and	
  capacitate	
  CCAFS	
  sites.	
  R1. More	
  focus	
  should	
  be	
  placed	
  in	
  Theme	
  2	
  and	
  on	
  non-­‐productivity	
  dimensions	
  of	
  food	
  security,	
  and	
  this	
  may	
  involve	
  recruiting	
  appropriate	
  staff	
  to	
  increase	
  the	
  capacity	
  and	
  capability	
  of	
  Centres,	
  or	
  bringing	
  in	
  outside	
  suppliers	
  of	
  research.	
  R2. In	
  coming	
  years	
  CCAFS	
  should	
  be	
  prepared	
  to	
  reduce	
  non-­‐strategic	
  Centre-­‐led	
  research	
  in	
  its	
  portfolio.	
  	
  R3. This	
  would	
  need	
  to	
  be	
  done	
  in	
  an	
  objective	
  and	
  transparent	
  manner,	
  clearly	
  communicating	
  expectations	
  in	
  advance,	
  sharing	
  evaluation	
  criteria,	
  and	
  focusing	
  on	
  activities	
  with	
  potential	
  for	
  most	
  improvement.	
  	
  R4. The	
  Consortium	
  Office	
  should	
  prepare	
  to	
  support	
  a	
  CGIAR	
  system-­‐wide	
  policy	
  incentivising	
  Centres	
  to	
  focus	
  on	
  CRP	
  priorities,	
  and	
  to	
  support	
  CRPs	
  implementing	
  it.	
  R5. Increased	
  alignment	
  between	
  the	
  Thematic	
  and	
  Regional	
  Programmes	
  and	
  Centre-­‐led	
  activities	
  at	
  CCAFS	
  sites	
  should	
  be	
  an	
  area	
  of	
  focus	
  over	
  the	
  next	
  period.	
  R6. CCAFS	
  should	
  strengthen	
  the	
  climate	
  change	
  knowledge	
  of	
  CCAFS	
  site	
  teams.	
  Efficiency	
  Despite	
  a	
  challenging	
  year,	
  CCAFS	
  has	
  improved	
  on	
  its	
  financial	
  performance	
  from	
  2010.	
  The	
  majority	
  of	
  challenges	
  experienced	
  have	
  not	
  been	
  within	
  the	
  control	
  of	
  CCAFS	
  staff,	
  and	
  they	
  have	
  done	
  well	
  to	
  manage	
  financial	
  risks.	
  	
  	
  R7. CO	
  should	
  try	
  to	
  move	
  financial	
  planning	
  earlier	
  in	
  the	
  year	
  and	
  provide	
  confirmation	
  of	
  next	
  financial	
  year’s	
  budget	
  to	
  CRPs	
  before	
  commencement	
  of	
  the	
  financial	
  year	
  R8. Donors	
  should	
  ensure	
  that	
  funds	
  are	
  with	
  the	
  CO	
  to	
  allow	
  disbursement	
  to	
  CRPs	
  a	
  quarter	
  in	
  advance	
  R9. Reform	
  at	
  the	
  level	
  of	
  the	
  Consortium	
  Office	
  and	
  Fund	
  Council	
  would	
  be	
  needed	
  to	
  make	
  financial	
  arrangements	
  more	
  accessible	
  to	
  non-­‐CGIAR	
  partners.	
  R10. Financial	
  efficiency	
  of	
  CRPs	
  would	
  be	
  improved	
  if	
  they	
  were	
  able	
  to	
  negotiate	
  lower	
  rates	
  with	
  Centres,	
  or	
  to	
  preferentially	
  select	
  Centres	
  based	
  on	
  lower	
  indirect	
  costs.	
  	
  R11. The	
  Consortium	
  Office	
  should	
  make	
  provisions	
  for	
  indirect	
  cost	
  charges	
  on	
  pass-­‐through	
  funds,	
  particularly	
  on	
  long-­‐term	
  multiple	
  year	
  commitments,	
  to	
  be	
  lower	
  than	
  indirect	
  cost	
  charges	
  for	
  operations.	
  Effectiveness	
  CCAFS	
  has	
  a	
  range	
  of	
  processes	
  in	
  place	
  to	
  ensure	
  effective	
  programme	
  delivery.	
  The	
  development	
  of	
  M&E	
  tools	
  is	
  a	
  significant	
  aspect	
  of	
  the	
  CCAFS	
  workplan,	
  although	
  quantitative	
  techniques	
  need	
  to	
  be	
  balanced	
  by	
  qualitative	
  methods.	
  The	
  programme	
  LogFrame	
  is	
  internally	
  consistent.	
  Indicators	
  can	
  be	
  improved	
  in	
  terms	
  of	
  specificity	
  and	
  measurability.	
  	
  
	
   22	
  
R12. The	
  positive	
  role	
  of	
  the	
  ISP	
  should	
  be	
  considered	
  as	
  a	
  model	
  for	
  other	
  CRPs.	
  	
  R13. Quantitative	
  impact	
  assessment	
  tool	
  should	
  be	
  complemented	
  by	
  qualitative	
  assessments	
  examining	
  farming	
  systems	
  and	
  decision-­‐making.	
  	
  R14. CCAFS	
  should	
  strengthen	
  internal	
  processes	
  for	
  planning	
  and	
  integrating	
  activities	
  to	
  ensure	
  that	
  challenges	
  in	
  programme	
  management	
  are	
  not	
  contributing	
  to	
  additional	
  delays	
  in	
  participatory	
  processes	
  at	
  CCAFS	
  sites.	
  R15. CCAFS	
  needs	
  to	
  focus	
  on	
  approaches	
  for	
  integrating	
  research	
  results	
  from	
  multiple	
  activities,	
  partners	
  and	
  sites	
  R16. CCAFS	
  should	
  consider	
  accepting	
  multi-­‐year	
  project	
  proposals	
  to	
  ensure	
  that	
  annual	
  research	
  activities	
  are	
  strategic.	
  	
  R17. CCAFS	
  should	
  screen	
  Centre-­‐led	
  proposals	
  more	
  rigorously,	
  rewarding	
  strategic	
  research	
  with	
  increased	
  funding	
  and	
  withdrawing	
  funding	
  from	
  non-­‐strategic	
  research.	
  R18. CCAFS	
  should	
  consider	
  implementing	
  a	
  monitoring	
  and	
  evaluation	
  system	
  tracking	
  progress	
  in	
  partnership	
  development.	
  R19. The	
  specificity,	
  measurability	
  and	
  achievability	
  of	
  logframe	
  indicators	
  should	
  be	
  improved	
  if	
  the	
  document	
  is	
  to	
  be	
  used	
  for	
  accountability	
  purposes.	
  Practical	
  Implementation	
  of	
  the	
  CRPs	
  The	
  development	
  of	
  longer-­‐term	
  workplans	
  and	
  contracts	
  for	
  CCAFS	
  sites	
  would	
  reduce	
  transaction	
  costs	
  and	
  strengthen	
  integration	
  between	
  Regional	
  and	
  Thematic	
  Programmes.	
  This	
  would	
  also	
  reduce	
  burdens	
  on	
  participating	
  country	
  partners.	
  R20. CCAFS	
  should	
  move	
  towards	
  longer	
  term	
  work-­‐planning	
  and	
  contracting	
  at	
  each	
  site.	
  Synergy	
  and	
  collective	
  action	
  At	
  this	
  stage	
  there	
  are	
  positive	
  signs	
  of	
  engagement	
  between	
  CRPs.	
  	
  R21. In	
  the	
  future,	
  the	
  Consortium	
  Office	
  and/or	
  donors	
  may	
  wish	
  to	
  map	
  and	
  evaluate	
  synergies	
  between	
  CRPs.	
  Lessons	
  regarding	
  the	
  CGIAR	
  reform	
  process	
  There	
  is	
  a	
  need	
  for	
  increased,	
  but	
  flexible,	
  harmonisation	
  between	
  Centres	
  and	
  CRPs	
  of	
  policies	
  and	
  procedures.	
  Situating	
  CRPs	
  within	
  host	
  Centres	
  may	
  limit	
  the	
  potential	
  for	
  enhanced	
  performance	
  across	
  the	
  CGIAR.	
  More	
  can	
  be	
  done	
  to	
  incentivise	
  participation	
  of	
  non-­‐CGIAR	
  research	
  institutions	
  in	
  CRPs.	
  Finally,	
  there	
  is	
  a	
  potential	
  tension	
  between	
  an	
  increased	
  focus	
  on	
  achieving	
  development	
  impact	
  and	
  supporting	
  high	
  quality	
  research,	
  although	
  as	
  yet	
  there	
  are	
  no	
  indications	
  this	
  is	
  a	
  problem	
  in	
  CCAFS.	
  R22. The	
  capacity	
  of	
  the	
  Consortium	
  Office	
  to	
  generate	
  harmonisation	
  and	
  manage	
  the	
  reform	
  process	
  should	
  be	
  reinforced.	
  R23. Over	
  the	
  medium	
  term	
  (2-­‐3	
  years)	
  the	
  CO	
  should	
  map	
  policy	
  areas	
  requiring	
  harmonization	
  and	
  institute	
  reviews	
  across	
  Centres	
  with	
  a	
  view	
  to	
  harmonise,	
  where	
  possible.	
  	
  R24. Over	
  a	
  longer	
  term	
  period,	
  as	
  the	
  reform	
  process	
  stabilizes,	
  the	
  CGIAR	
  may	
  wish	
  to	
  consider	
  reorienting	
  governance	
  towards	
  a	
  matrix	
  management	
  style	
  with	
  CRP	
  directors	
  reporting	
  directly	
  to	
  the	
  CO	
  rather	
  than	
  to	
  Centre	
  DGs.	
  	
  R25. Institutionalise	
  mechanisms	
  within	
  the	
  Consortium	
  Office	
  and	
  CRPs	
  for	
  strategic	
  engagement	
  with	
  non-­‐CGIAR	
  research	
  institutions.	
  
	
   23	
  
R26. After	
  3-­‐4	
  years	
  the	
  ISP	
  should	
  commission	
  a	
  review	
  examining	
  the	
  role	
  of	
  participatory	
  action	
  research	
  approaches	
  to	
  climate	
  change	
  adaptation	
  and	
  mitigation,	
  specifically	
  addressing	
  scientific	
  outputs.	
  R27. CO	
  to	
  establish	
  working	
  group	
  on	
  the	
  conditions	
  for	
  production	
  of	
  international	
  science	
  in	
  the	
  context	
  of	
  field-­‐based	
  action	
  research.	
  Monitoring	
  and	
  Evaluation	
  at	
  the	
  Consortium	
  Level	
  	
  Improved	
  prioritisation	
  of	
  research	
  and	
  resource	
  allocation	
  towards	
  development	
  outcomes	
  is	
  welcomed.	
  However,	
  the	
  Review	
  Team	
  has	
  deep	
  concerns	
  that	
  a	
  focus	
  on	
  making	
  research	
  accountable	
  for	
  development	
  impact	
  is	
  misplaced,	
  and	
  that	
  the	
  CGIAR	
  would	
  be	
  better	
  served	
  by	
  M&E	
  frameworks	
  focused	
  on	
  evaluative	
  learning.	
  The	
  Review	
  Team	
  is	
  also	
  mindful	
  of	
  the	
  potential	
  tension	
  between	
  the	
  nature	
  of	
  research	
  and	
  the	
  tendency	
  of	
  accountability	
  frameworks	
  to	
  incentivize	
  low	
  risk	
  behaviour.	
  R28. The	
  final	
  CGIAR	
  SRF	
  and	
  M&E	
  strategy	
  should	
  incentivize	
  innovative,	
  collaborative	
  and	
  high	
  quality	
  research,	
  which	
  allows	
  for	
  the	
  possibility	
  of	
  failure.	
  R29. The	
  CO	
  should	
  manage	
  expectations	
  and	
  ambitions	
  for	
  the	
  ability	
  to	
  rigorously	
  assess	
  the	
  development	
  impact	
  of	
  research.	
  A	
  focus	
  on	
  evaluative	
  learning	
  rather	
  than	
  accountability	
  would	
  support	
  better	
  programme	
  performance.	
  	
  R30. As	
  the	
  SRF	
  evolves,	
  increasing	
  emphasis	
  should	
  be	
  placed	
  on	
  the	
  evaluation	
  of	
  quality	
  research	
  as	
  the	
  vehicle	
  for	
  achieving	
  development	
  impact.	
  Impact,	
  sustainability	
  and	
  visibility	
  of	
  EC/IFAD	
  funding	
  Current	
  EC/IFAD	
  funding	
  arrangements	
  do	
  not	
  have	
  a	
  significant	
  impact	
  on	
  CCAFS	
  programming	
  directions.	
  However,	
  delays	
  in	
  disbursing	
  funds	
  do	
  add	
  to	
  the	
  programme’s	
  financial	
  uncertainty	
  and	
  pose	
  a	
  reputational	
  risk	
  to	
  EC/IFAD	
  	
  R31. Switching	
  to	
  multi-­‐year	
  commitments	
  would	
  reduce	
  the	
  transaction	
  costs	
  of	
  CCAFS,	
  as	
  would	
  abandoning	
  thematic	
  and	
  geographic	
  earmarking.	
  R32. If	
  the	
  EC/IFAD	
  wish	
  to	
  have	
  stronger	
  influence	
  on	
  CCAFS	
  programming	
  then	
  they	
  should	
  consider	
  advocating	
  a	
  stakeholder	
  forum	
  meeting	
  once	
  per	
  year	
  alongside	
  ISP	
  meetings,	
  including	
  site	
  visits	
  and	
  substantive	
  presentations	
  and	
  discussions	
  with	
  CCAFS	
  partners.	
  R33. Other	
  options	
  for	
  increasing	
  collaborations	
  with	
  EU	
  research	
  include	
  supporting	
  participation	
  of	
  EU	
  knowledge	
  centres	
  in	
  CRPs,	
  and	
  providing	
  backstopping	
  support	
  to	
  the	
  CO	
  to	
  better	
  accommodate	
  such	
  participation.	
  R34. EC/IFAD	
  should	
  streamline	
  contracting	
  and	
  disbursement	
  procedures	
  to	
  reduce	
  transaction	
  costs	
  and	
  financial	
  uncertainty	
  of	
  CCAFS,	
  and	
  reduce	
  reputational	
  risk	
  to	
  the	
  EC	
  and	
  IFAD.	
  
	
   24	
  
Annexes	
  A1.	
  Review	
  Terms	
  of	
  Reference	
  a) Methodology The second approach, which is more appropriate to new programmes or projects within the CRP framework is based on assessing the adequacy of the logframe and results monitoring framework (RMF) so that progress can be properly captured in annual reporting. As standard indicators are still being developed for some CRPs this objective seeks to draw lessons from existing logframes, contribute to their improvement where necessary and to support the CGIAR Consortium in developing a common reporting framework that will allow progress to be measured across CRPs.       Particular issues that should be addressed under approach 2 are:  • Commenting on the progress of the CGIAR reform process; • Using, assessing and validating the new CGIAR M&E criteria and procedures (as far as this is practical) (see Annexes 2 and 3– Consortium level Monitoring Principles and ); • Assessing the quality of the CRPs M&E provisions, including to what extent the CRPs are implementing them, the quality of their monitoring plans, and the likelihood that they will provide a transparent, credible and rigorous frame for assessing progress in delivering results (milestones, outputs and outcomes); • Assessing the quality and consistency of logframes, indicators and baseline studies, and recommend improvements or further quantification where appropriate; • Commenting on how the EC-funded work fits into the larger planning framework of the CRP. 	
  b) Review criteria to be utilised for each selected project of the cluster  i) Relevance: the relevance of a project relates primarily to its design and concerns the extent to which its stated objectives correctly address the identified problems and real needs at two points in time: when the project was designed and at the time of review. This concerns the identification of real (as distinct from perceived) problems or needs and of the correct beneficiaries, and how well the project’s initial design addressed them, as well as other project design features.  ii) Efficiency: The efficiency criterion concerns how well the various activities transformed the available resources into the intended results (sometimes referred to as outputs), in terms of quantity, quality and timeliness. A key question it asks is "were things done right?" and thereby also addresses value-for-money, that is whether similar results could have been achieved more by other means at lower cost in the same time. It includes a focus on the quality of the research from various points of view: scientific, technical, social, environmental, ethical, financial - including possible IPR issues, and policy, etc. It also explores the quality of the day-to day-management.  iii) Effectiveness: the effectiveness criterion concerns how far the project’s results were used or their potential benefits were realised - in other words, whether they achieved the project purpose. The key question is what difference the project made in practice, as measured by how far the intended beneficiaries really benefited from the products or services it made available.   
	
   25	
  
iv) Practical implementation of the CRPs: a critical review of the CRPs within this cluster should show whether (if and how) they have influenced the project design.   v) Synergy and collective action: An analysis of synergies and collective action will be conducted to show the links among selected projects with reference to CRP clusters both at project level (links with any similar projects even beyond the sampled projects) and globally across the sampled projects and Centres.  vi) International or regional public goods character of the research undertaken: are the research activities that are undertaken in the selected projects resulting in the production of important international or regional public goods, as opposed to national or local public or private goods benefiting only a single country or part of a country?   vii) Impact, sustainability and visibility of the EC co-financing: these important issues relate to the longer-term effect of the projects on beneficiaries.   Details of issues to be considered under each criterion are presented in Annex 1.  Performance rating:  Review teams should include in their assessments an overall performance rating for each of  the above seven review criteria, on the basis of the following scale:  ð highly satisfactory: fully according to plan or better ; ð satisfactory: on balance according to plan, positive aspects outweighing negative aspects ; ð less than satisfactory: not sufficiently according to plan, taking account of the evolving context; a few positive aspects, but outweighed by negative aspects; ð highly unsatisfactory: seriously deficient, very few or no positive aspects.  Each rating should be stated as part of the conclusions for each of the three criteria.  The review exercise requires an examination of project outputs in terms of reports and technical papers. Therefore the review teams should also examine the quality of such reports.    4.   Reporting  a) Reports and presentations: for each selected project, a debriefing of the draft report must be presented to the EC, in order to finalise the review report.   b) Language: English  c) Date of delivery: draft report within 30 days of the completion of the mission, final report within 10 days of the reception of the comments from the EC (due within 60 days after reception of the draft report)  d) Number of copies required: 3 copies of the draft reports and 3 copies of the final reports  e) The main text of a review report should not exceed 20 pages per project reviewed, plus Annexes, plus an Executive Summary of no more than 2 pages with fully cross-referenced findings and recommendations.  f) The main sections of the review report for each selected project will be as follows :  
	
   26	
  
1- Executive Summary: a tightly drafted, to the point and free-standing Executive Summary is an essential component. It should be short, no more than two pages. It should focus mainly on the key purpose or issues of the review, outline the main analytical points, and clearly indicate the main conclusions, lessons learned and specific recommendations. Cross-references should be made to the corresponding page or paragraph numbers in the main text that follows.  2- Main text: the main text should start with an introduction describing, first, the project to be reviewed and, second, the review objectives. The body or core of the report should follow the eight review criteria mentioned above, describing the facts and interpreting or analysing them in accordance with the key questions pertinent to each criterion.  3- Conclusions and recommendations: these should be the subject of a separate final chapter. Wherever possible, for each key conclusion there should be a corresponding recommendation. The key points of the conclusions will vary in nature but will often cover aspects of the key review criteria (including performance ratings - see above), that is: ð Relevance : whether the design of the project was originally, and still is, sound as regards targeting the real needs and problems of the right beneficiaries; ð Efficiency : whether the same results could have been achieved at lower costs; or whether there might have been different, more appropriate ways of achieving the same results; ð Effectiveness : whether the planned benefits were in fact received, whether the beneficiaries’ behavioural patterns changed, whether neglect of cross-cutting issues affected the achievement of the project purpose; ð Practical implementation of CRP cluster: whether they have influenced project design,  is the cluster adequately addressed by the projects and the CGIAR; ð Synergy and collective action : whether the cluster is adequately addressed by the Centres taken together, with collective action and synergy, avoiding overlap and duplication and gaps in the research; ð IPG/RPG character of the research : whether the outputs and outcomes of the research are truly IPG/RPG with important spillover effects and economies of scale and scope; ð Impact, sustainability and visibility of the EC co-financing: these important issues relate to the longer-term effect of the projects on beneficiaries.  Recommendations should be as realistic, operational and pragmatic as possible; that is, they should take careful account of the circumstances currently prevailing in the context of the project, and of the resources available to implement them. They could concern policy, organisational and operational aspects. Recommendations should also possibly contribute to the design of the future CGIAR Strategic Results Framework and CGIAR Research Programmes.  4- Annexes: the report should include the following annexes:  ð The Terms of Reference of the review ð The names of the evaluators and their companies (CVs should be shown, but summarised and limited to one page per person) ð Map of project area implementation, ð Calendar of visit and list of persons/organisations consulted ð Literature and documentation consulted ð Other technical annexes (e.g. statistical analyses) ð 1-page DAC summary per project  
	
   27	
  
The draft report will be sent for comments to the relevant Centre by each team of reviewers before the debriefing in Brussels or Rome. The reviewers will be responsible for addressing the Centre’s comments on the draft report if appropriate. The responses of the Centres should be attached to the reports as an annex.   The final reports will be published and posted as appropriate on the EuropeAid Web. If necessary, an additional confidential report may be submitted to the EC for its consideration. 	
  	
   	
  
	
   28	
  
A2.	
  Feed	
  back	
  on	
  the	
  review	
  process	
  and	
  the	
  terms	
  of	
  reference	
  	
  As	
  a	
  former	
  Challenge	
  Programme	
  that	
  pre-­‐existed	
  the	
  CRPs,	
  CCAFS	
  was	
  fast-­‐tracked	
  for	
  development	
   as	
   a	
   CRP.	
   The	
   timing	
   of	
   the	
   review	
   afforded	
   an	
   opportunity	
   to	
   examine	
  CCAFS	
   in	
   the	
   context	
   of	
   the	
  on-­‐going	
  CGIAR	
   reform	
  process,	
   and	
   indeed	
   to	
  use	
   it	
   as	
   a	
  lens	
  to	
  understand	
  some	
  facets	
  of	
  that	
  reform.	
  As	
  a	
  consequence,	
  a	
  number	
  of	
  the	
  review	
  team’s	
  recommendations	
  go	
  beyond	
  the	
  classic	
  purview	
  of	
  an	
  evaluation.	
  	
  The	
  review	
  team	
  had	
  a	
  number	
  of	
  internal	
  discussions	
  about	
  how	
  to	
  best	
  reconcile	
  the	
  format	
  of	
  the	
  monitoring	
  exercise	
  with	
  the	
  requirements	
  of	
  program	
  evaluation	
  and	
  the	
  opportunity	
  to	
  present	
  wider	
   findings	
  on	
  the	
  reform	
  process.	
   It	
  was	
  clear	
  to	
  us	
  that	
  of	
  the	
   approaches given in the terms of references, the second was more appropriate to the review of CCAFS. However, the format and reporting guidelines of the second approach are more appropriate to project than program monitoring. They also remain focused on outcomes rather than assessment of internal processes likely to lead to outcomes, and therefore are not well adapted to assessment of projects and programs early in their lifecycle.   Anticipating that reviews of CRPs are likely to be more common in the future, we strongly recommend that EC/IFAD develop a specific approach for CRP monitoring whilst retaining the current Approach 1 for project monitoring. Given the scale and complexities of CRPs, and particularly the challenges in relating local impacts with program-level characteristics, CRP monitoring should focus on internal processes. These are suggested to include governance, organisational relationships between CG centres and between CG and non-CG partners, scientific coordination, budgetary decision-making and allocations, monitoring and evaluation processes, and management mechanisms in place to ensure progress towards objectives. A reorientation towards these review objectives would also provide clearer entry-points for relating CRP performance to the CGIAR reform process.  The review team also found that the framing of the review around one region (West Africa) with limited time imposed some constraints on their ability to comprehensively cover CCAFS’ thematic programmes and assess global performance. As CCAFS currently has three regions and a relatively young scientific program this is not deemed a major loss in the current review, but it is an issue that will be problematic in the future as CRPs expand geographically and mature in scientific results.  The review team therefore recommends that EC/IFAD consider commissioning reviews of fewer CRPs but in more depth. Visits to participating centres and partners will be important for garnering perspectives on these issues, whilst missions to field sites should be focused on relationships with local partners and practical implementation of scientific approaches on the ground rather than local development impacts and outcomes. Reviews of the total scientific output of a CRP are likely to be unfeasible. Approaches might include the selection of a single thematic program, or to focus entirely on issues of scientific coordination and research management.  The review team also notes that EC/IFAD might commission CRP level reviews inline with the above suggestions whilst simultaneously commissioning separate reviews of centre-led activities under that CRP using the current Approach 1. This would provide both top-down and bottom-up perspectives on the challenges of integrating projects within CRPs and the CGIAR reform process.  	
   	
  
	
   29	
  
A3.	
  Curricula	
  Vitae	
  of	
  Consultants	
  Dr	
  Guy	
  Jobbins	
  5	
  Ibn	
  el	
  Nabii,	
  Zemalek,	
  Cairo,	
  Egypt	
  Email:	
  guy@tayridge.com	
  Tel	
  :	
  +20	
  100	
  391	
  7027,	
  +	
  44	
  778	
  647	
  1325	
  Specific	
  Expertise	
  Guy	
  Jobbins	
  is	
  a	
  climate	
  change	
  adaptation	
  specialist	
  with	
  a	
  background	
  in	
  ecosystem	
  sciences	
  (B.Sc.	
  Leeds,	
  M.Sc.	
  Warwick)	
  and	
  public	
  policy	
  (Ph.D.	
  Birmigham).	
  His	
  career	
  has	
  focused	
  on	
  natural	
  resource	
  management	
  in	
  the	
  Middle	
  East	
  and	
  Africa.	
  Between	
  2006	
  and	
  2012	
  he	
  was	
  a	
  Senior	
  Programme	
  Officer	
  with	
  the	
  International	
  Development	
  Research	
  Centre’s	
  Climate	
  Change	
  Adaptation	
  in	
  Africa	
  Program,	
  developing	
  programming	
  and	
  projects	
  on	
  water	
  policy,	
  coastal	
  change,	
  agriculture,	
  and	
  ecosystem	
  approaches	
  to	
  human	
  health	
  in	
  the	
  context	
  of	
  global	
  change.	
  Recently	
  he	
  has	
  worked	
  as	
  a	
  consultant	
  with	
  the	
  GIZ	
  program	
  “Adaptation	
  to	
  Climate	
  Change	
  in	
  the	
  Water	
  Sector	
  of	
  the	
  MENA	
  Region”	
  and	
  the	
  Alexandria	
  Research	
  Centre	
  for	
  Adaptation.	
  	
  Profile	
  • Experienced	
  rural	
  development	
  and	
  climate	
  change	
  adaptation	
  specialist	
  • 13	
  years	
  experience	
  living	
  and	
  working	
  in	
  Africa	
  and	
  the	
  Middle	
  East	
  	
  • Accomplished	
  manager	
  and	
  team	
  builder	
  • Project	
  and	
  program	
  management,	
  development	
  and	
  evaluation	
  • Languages:	
  Native	
  English,	
  Intermediate	
  French	
  and	
  Arabic	
  Experience	
  Record:	
  2012	
  -­‐	
  	
   	
   Independent	
  Consultant.	
  Egypt	
  2006-­‐2012	
   Senior	
  Program	
  Officer,	
  International	
  Development	
  Research	
  Centre,	
  Egypt	
  2005-­‐2006	
   Independent	
  Consultant.	
  Egypt	
  2004-­‐2005	
   Environmental	
  Advisor.	
  SEAM	
  Programme,	
  Egypt	
  	
  2003-­‐2004	
   Interim	
  Director.	
  Sinai	
  Environmental	
  Centre,	
  Egypt	
  	
  2001-­‐2003	
   Post-­‐Doctoral	
  Research	
  Fellow.	
  University	
  College	
  London,	
  UK	
  	
  1998-­‐2001	
   Field	
  Researcher.	
  MECO	
  Project,	
  University	
  of	
  Birmingham,	
  UK	
  	
  1997-­‐1998	
   Researcher.	
  Ras	
  Mohamed	
  National	
  Park,	
  Egypt	
  	
  1996-­‐1997	
   Advisor.	
  River	
  Ocean	
  Research	
  &	
  Education,	
  UK	
  	
  1996-­‐1997	
   Marine	
  Policy	
  Analyst.	
  UNED-­‐UK,	
  UK	
  	
  Country	
  Experience	
  In	
  depth:	
  Egypt,	
  Morocco,	
  Tunisia,	
  Kenya,	
  Malta,	
  United	
  Kingdom,	
  United	
  States,	
  Canada,	
  Jordan,	
  Lebanon	
  Familiarity:	
  Uganda,	
  Senegal,	
  Guinea,	
  Ghana,	
  Cote	
  d’Ivoire,	
  Cape	
  Verde,	
  Sao	
  Tome	
  &	
  Principe,	
  South	
  Africa,	
  Sudan,	
  Algeria,	
  Libya,	
  Syria,	
  Norway,	
  France	
  	
  
	
   30	
  
Dr.	
  Didier	
  PILLOT	
  	
  AGREENIUM	
  147	
  rue	
  de	
  l’Université	
  -­‐750007	
  -­‐	
  PARIS	
  Phone:	
  +33	
  6	
  64	
  00	
  50	
  98	
  	
  email:	
  pillot@supagro.inra.fr	
  Specific	
  Expertise	
  :	
  Didier	
  Pillot	
  is	
  an	
  Agronomist	
  (MSc,	
  Paris,	
  France)	
  and	
  Agricultural	
  economist	
  (PhD,	
  McGill,	
  Canada).	
  He	
  has	
  had	
  an	
  international	
  career,	
  mostly	
  devoted	
  to	
  Agricultural	
  Development	
  in	
  the	
  tropics	
  and	
  subtropics	
  for	
  various	
  International	
  organisations	
  with	
  a	
  specific	
  focus	
  in	
  the	
  Caribbean,	
  Eastern	
  African	
  and	
  South	
  East	
  Asia.	
  He	
  directed	
  GRET,	
  the	
  most	
  important	
  French	
  NGO	
  in	
  the	
  domain	
  of	
  international	
  development	
  from	
  1996	
  to	
  2002,	
  where	
  he	
  started	
  to	
  build	
  a	
  number	
  of	
  European	
  projects,	
  in	
  research	
  and	
  development.	
  In	
  2002,	
  he	
  joined	
  Agricultural	
  Higher	
  Education	
  as	
  deputy	
  director	
  of	
  the	
  Center	
  for	
  Post	
  Graduate	
  Studies	
  in	
  Tropical	
  and	
  Subtropical	
  Agriculture	
  (CNEARC	
  as	
  the	
  French	
  acronym),	
  and	
  extended	
  his	
  international	
  experience	
  to	
  education	
  and	
  training	
  engineering.	
  From	
  2007	
  to	
  2010,	
  he	
  launched	
  the	
  Department	
  of	
  International	
  Affairs	
  of	
  the	
  newly	
  born	
  Montpellier	
  SupAgro.	
  He	
  is	
  now	
  sharing	
  his	
  time	
  between	
  Agreenium,	
  the	
  new	
  French	
  Consortium	
  for	
  Research	
  and	
  Education	
  in	
  Agricultural	
  Sciences,	
  Montpellier	
  SupAgro	
  and	
  a	
  number	
  of	
  European	
  positions	
  in	
  research	
  and	
  education	
  in	
  Agriculture.	
  Since	
  2006,	
  Didier	
  Pillot	
  has	
  coordinated	
  Agris	
  Mundus,	
  an	
  Erasmus	
  Mundus	
  Master	
  Course.	
  He	
  has	
  also	
  been	
  Board	
  Member	
  of	
  Agtrain,	
  an	
  Erasmus	
  Mundus	
  Joint	
  Doctorate	
  since	
  2010.	
  MEMBERSHIP	
  IN	
  SCIENTIFIC	
  OR	
  PROFESSIONAL	
  ORGANISATIONS:	
  § President,	
  Agrinatura	
  -­‐	
  European	
  Alliance	
  on	
  Agricultural	
  research	
  for	
  development	
  –	
  (European	
  Universities	
  and	
  Research	
  Centers	
  “tropically	
  oriented”);	
  § Member	
  of	
  the	
  Scientific	
  Committee	
  of	
  the	
  Institute	
  for	
  the	
  Tropics	
  and	
  Subtropics	
  (ITS),	
  Agricultural	
  University	
  of	
  Prague.	
  § Board	
  Member	
  Global	
  Initiative	
  in	
  Horticultural	
  Sciences	
  (GlobalHort)	
  § Steering	
  Committee	
  Member	
  of	
  European	
  Forum	
  for	
  Agricultural	
  Research	
  for	
  Development	
  (EFARD)	
  Relevant	
  books	
  and	
  manuals	
  :	
  1.	
  Pillot,	
  D.	
  (2008).	
  Rizières	
  et	
  jardins	
  du	
  Cambodge.	
  Les	
  enjeux	
  du	
  développement	
  agricole,	
  Karthala,	
  Paris,	
  522	
  p.	
  2.	
  Pillot	
  D.,	
  Lauga	
  Sallenave	
  C.,	
  Gautier	
  D..	
  (2002).	
  Haies	
  et	
  bocages	
  en	
  milieu	
  tropical	
  d'altitude	
  :	
  des	
  pratiques	
  au	
  projet	
  ,-­‐	
  Editions	
  GRET,	
  Coll	
  Le	
  point	
  sur,	
  Paris,	
  239	
  p.	
  3.	
  Pillot	
  D.	
  et	
  Bellande,	
  A.,	
  ed.	
  (1993)	
  -­‐	
  Paysans,	
  systèmes	
  et	
  crises;	
  travaux	
  sur	
  l'agraire	
  haïtien	
  –	
  3	
  volumes,	
  ed.	
  Université	
  Antilles	
  Guyane,	
  Pointe-­‐à-­‐Pitre,	
  365,	
  298	
  et	
  476	
  p.	
  4.	
  Bonnefoy,	
  M.	
  et	
  Pillot,	
  D.	
  (1991).	
  Manuel	
  d’agronomie	
  tropicale	
  appliquée	
  à	
  l’agriculture	
  haïtienne,	
  Editions	
  GRET/FAMV,	
  Paris,	
  490	
  p.	
  5.	
  Pillot,	
  D.	
  (1995).	
  Bringing	
  down	
  the	
  barriers;	
  francophone	
  and	
  anglophone	
  approaches	
  in	
  system	
  oriented	
  agricultural	
  research	
  and	
  rural	
  development,	
  Rural	
  Extension	
  Bulletin	
  no	
  7,	
  University	
  of	
  Reading.	
   	
  
	
   31	
  
A4.	
  Maps	
  
a)	
  CCAFS	
  CCAFS	
  Site	
  Locations	
  
	
  	
  b)	
  West	
  Africa	
  CCAFS	
  Sites	
  
	
  Note:	
  No	
  maps	
  were	
  available	
  showing	
  locations	
  of	
  all	
  program	
  activities,	
  including	
  e.g.	
  Centre-­‐led	
  activities.	
  These	
  maps	
  indicate	
  CCAFS	
  benchmark	
  sites	
   	
  
	
   32	
  
A5.	
  Review	
  Agenda	
  a)	
  Review	
  Timeline	
  	
  Mission	
  to	
  Copenhagen	
  22	
  –	
  26	
  August	
  1012	
  Wednesday	
  22	
  August	
  	
  1400	
  	
   	
   Guy	
  Jobbins	
  arrives	
  in	
  Copenhagen	
  1500	
  -­‐	
  1730	
   Introductory	
  meeting	
  at	
  CCAFS	
  CU:	
  strategic	
  overview	
  of	
  programme	
  and	
  orientation	
  of	
  review	
  exercise	
  Thursday	
  23	
  August	
  0900	
  -­‐	
  1100	
   Programme	
  design:	
  partnerships,	
  site	
  selection,	
  and	
  research	
  questions	
  1100	
  –	
  1200	
   Knowledge	
  management	
  and	
  communications	
  1200	
  –	
  1300	
  	
   Lunch,	
  Didier	
  Pillot	
  arrives	
  1300	
  –	
  1600	
   Monitoring	
  and	
  evaluation	
  1600	
  –	
  1630	
   Debrief	
  	
  Location:	
  Hotel	
  1830	
  –	
  1900	
   Meeting	
  with	
  Henning	
  H.	
  Jensen,	
  University	
  of	
  Aarhus	
  Friday	
  24	
  August	
  0900	
  –	
  1030	
   Programme	
  management	
  and	
  coordination	
  1030	
  –	
  1200	
   Financial	
  management	
  1200	
  –	
  1300	
   Lunch	
  1300	
  –	
  1400	
   Planning	
  Ghana	
  field	
  visit,	
  linkages	
  and	
  integration	
  between	
  programme	
  and	
  site	
  activities	
  1400	
  –	
  1500	
   Monitoring	
  and	
  evaluation:	
  scientific	
  quality	
  control	
  of	
  science	
  	
  1500	
  –	
  1600	
  	
   Impact,	
  sustainability	
  and	
  visibility	
  of	
  EC	
  financing	
  1600	
  –	
  1630	
   Debrief	
  Saturday	
  25	
  August	
  Departure	
  of	
  Review	
  Team	
  Mission	
  to	
  Ghana,	
  September	
  22	
  –	
  October	
  1	
  2012	
  Saturday	
  22	
  September	
  	
  Guy	
  Jobbins	
  arrives	
  in	
  Accra	
  	
  Sunday	
  23	
  September	
  Didier	
  Pillot	
  arrives	
  in	
  Accra	
  Monday	
  24	
  September	
  0930	
  –	
  1030	
   Meeting	
  at	
  Directorate	
  of	
  Crop	
  Services,	
  Ministry	
  of	
  Food	
  and	
  Agriculture	
  1100	
  –	
  1230	
   Meeting	
  at	
  Forum	
  for	
  Agricultural	
  Research	
  in	
  Africa	
  1300	
  –	
  1700	
   Review	
  Team	
  meeting	
  Tuesday	
  25	
  September	
  0715	
  –	
  0815	
   Review	
  Team	
  fly	
  from	
  Accra	
  to	
  Tamale	
  0930	
  –	
  1100	
   Meeting	
  at	
  Livestock	
  Research	
  Institute	
  	
  1100	
  –	
  1130	
   Courtesy	
  visit	
  to	
  Director	
  of	
  Savannah	
  Agricultural	
  Research	
  Institute	
  1130	
  –	
  1200	
   Lunch	
  1200	
  –	
  1800	
   Review	
  Team	
  drive	
  to	
  Wa,	
  Upper	
  West	
  Region	
  
	
   33	
  
1800	
  –	
  1830	
   Introductory	
  meeting	
  at	
  Wa	
  Field	
  Station	
  Wednesday	
  26	
  September	
  0830	
  –	
  1200	
   Overview	
  of	
  CCAFS	
  activities	
  at	
  the	
  Lawra-­‐Jirapa	
  CCAFS	
  site	
  1200	
  –	
  1230	
  	
  	
   Lunch	
  1230	
  –	
  1330	
  	
   Travel	
  to	
  Jirapa	
  	
  1330	
  –	
  1600	
  	
   Meeting	
  with	
  Lawra-­‐Jirapa	
  Project	
  Support	
  Unit	
  1600	
  –	
  1700	
  	
   Return	
  to	
  Wa	
  Thursday	
  27	
  September	
  0830	
  –	
  1000	
   Travel	
  to	
  Doggoh	
  Village,	
  CCAFS	
  site	
  1000	
  –	
  1300	
  	
   Farm	
  visits	
  1300	
  –	
  1500	
  	
   Village	
  meeting	
  in	
  Doggoh,	
  question	
  and	
  answer	
  session	
  1500	
  –	
  1630	
   Return	
  to	
  Wa	
  1900	
  –	
  2030	
   Debrief	
  with	
  Robert	
  Zougmore	
  and	
  Dr	
  Jesse	
  Naab	
  	
  Friday	
  28	
  September	
  0830	
  –	
  0930	
   Debrief	
  with	
  with	
  Robert	
  Zougmore	
  and	
  Dr	
  Jesse	
  Naab	
  0930	
  –	
  1600	
   Drive	
  to	
  Tamale	
  Saturday	
  29	
  September	
  0845	
  –	
  1200	
   Review	
  Team	
  fly	
  back	
  to	
  Accra	
  Didier	
  Pillot	
  departs	
  Accra	
  Sunday	
  30	
  September	
  1400	
  –	
  1600	
   Meeting	
  at	
  Noguchi	
  Memorial	
  Institute	
  Monday	
  1	
  October	
  Guy	
  Jobbins	
  departs	
  Accra	
  b)	
  List	
  of	
  People	
  and	
  Organisations	
  Consulted	
  CCAFS	
  Coordination	
  Unit,	
  Copenhagen	
  Bruce	
  Campbell,	
  CCAFS	
  Director	
  Sonja	
  Vermeulen,	
  Head	
  of	
  Research	
  Misha	
  Wolsgaard-­‐Iversen	
  Program	
  Manager	
  Gloria	
  Cecilia	
  Rengifo,	
  Senior	
  Manager,	
  Finance,	
  Contracts	
  &	
  Liaison	
  Torben	
  Mandrup	
  Timmermann,	
  Head	
  of	
  Program	
  Coordination	
  and	
  Communications	
  Vanessa	
  Meadu,	
  Manager,	
  Communications	
  and	
  Knowledge	
  Management	
  University	
  of	
  Aarhus	
  Henning	
  H.	
  Jensen	
  CCAFS	
  Programme	
  Leaders	
  Phil	
  Thornton,	
  ILRI,	
  Theme	
  4	
  Programme	
  Leader	
  (via	
  Skype)	
  Robert	
  Zougmore,	
  ICRISAT,	
  West	
  African	
  Regional	
  Programme	
  Leader	
  (in	
  Ghana)	
  CCAFS	
  Independent	
  Science	
  Panel	
  Thomas	
  Rosswall,	
  Chair	
  (meeting	
  in	
  London)	
  Directorate	
  of	
  Crop	
  Services,	
  Ministry	
  of	
  Food	
  and	
  Agriculture,	
  Accra	
  Mr	
  Delali	
  Kofi,	
  Director	
  and	
  Ministry	
  Climate	
  Change	
  Focal	
  Point	
  
	
   34	
  
Forum	
  for	
  Agricultural	
  Research	
  in	
  Africa,	
  Accra	
  Dr	
  Ramadjita	
  Tabo,	
  Deputy	
  Executive	
  Director	
  Prof	
  Wale	
  Adekunle,	
  Director	
  of	
  Partnerships	
  and	
  Strategic	
  Alliances	
  Dr	
  Emmanuel	
  Tambi,	
  Director	
  of	
  Advocacy	
  and	
  Policy	
  Solomon	
  Bangali,	
  Sub-­‐Saharan	
  Africa	
  Challenge	
  Programme	
  Animal	
  Research	
  Institute,	
  CSIR,	
  Tamale	
  	
  	
  Dr	
  Karbo	
  Naaminong,	
  Director	
  Savannah	
  Agricultural	
  Research	
  Institute,	
  Tamale	
  Dr.	
  Stephen	
  Nutsugah,	
  Director	
  University	
  of	
  Development	
  Studies	
  (UDS),	
  Tamale	
  Dr	
  Francis	
  Obeng,	
  Department	
  of	
  Extension	
  and	
  Communication	
  Savannah	
  Agricultural	
  Research	
  Institute,	
  Wa	
  Dr.	
  Jesse	
  Naab,	
  Senior	
  Research	
  Scientist	
  and	
  Head	
  of	
  Station.	
  CCAFS	
  Country	
  Team	
  Leader.	
  Anslem	
  Nyuir	
  CCAFS	
  Project	
  Support	
  Unit,	
  Lawra-­‐Jirapa	
  Yowataafaa	
  Mwinapuo,	
  Chief	
  of	
  Bompari	
  Village	
  James	
  Vuuro,	
  Ministry	
  of	
  Food	
  and	
  Agriculture,	
  Lawra	
  District	
  Damiay	
  Tampuari,	
  Ministry	
  of	
  Food	
  and	
  Agriculture,	
  Lawra	
  District	
  Bellingfaa	
  Francis,	
  representative	
  of	
  the	
  Chief	
  of	
  Doggoh	
  Village	
  Kuuriaah	
  Richard,	
  Mission	
  of	
  Hope	
  International	
  Lofko	
  Erex	
  John,	
  Mission	
  of	
  Hope	
  International	
  Linus	
  Kabo-­‐Bak,	
  LAGERD	
  NGO	
  Hon.	
  Salia	
  Stephen,	
  Assembly	
  Member	
  for	
  Doggoh	
  Mavis	
  Derigubah,	
  Ministry	
  of	
  Food	
  and	
  Agriculture,	
  Jirapa	
  District	
  Joel	
  Yuri	
  Song,	
  Agricultural	
  Input	
  Dealer	
  Basilide	
  Babasignae,	
  Forestry	
  Commission	
  Allansah	
  Kuzie,	
  Ministry	
  of	
  Food	
  and	
  Agriculture,	
  Jirapa	
  District	
  Doggoh	
  Village	
  Farmers	
  and	
  citizens	
  Noguchi	
  Memorial	
  Institute	
  for	
  Medical	
  Reserach,	
  Accra	
  Prof	
  Michael	
  Wilson,	
  Deputy	
  Director	
  	
  	
  	
  	
  
	
   35	
  
A6	
  Literature	
  &	
  documentation	
  consulted	
  
Management	
  Documents	
  CGIAR	
  Challenge	
  Program	
  on	
  Climate	
  Change,	
  Agriculture	
  and	
  Food	
  Security,	
  CCAFS	
  (2009)	
  Overhead	
  Policy,	
  CCAFS	
  (2009)	
  Report	
  from	
  CCAFS	
  Planning	
  Workshop,	
  May	
  5-­‐7,	
  2010	
  	
  Nairobi,	
  Kenya	
  Program	
  Plan:	
  CGIAR	
  Research	
  Program	
  on	
  Climate	
  Change,	
  Agriculture	
  and	
  Food	
  Security	
  	
  (CCAFS,	
  2011)	
  Final	
  comments	
  from	
  the	
  Independent	
  Science	
  &	
  Partnership	
  Council	
  (ISPC)	
  on	
  the	
  CCAFS	
  proposal	
  Response	
  to	
  ISPC	
  comments	
  CCAFS	
  Annual	
  Report	
  2010	
  Financial	
  Statements	
  2010	
  Consolidated	
  Logframe	
  of	
  Activities,	
  2012-­‐2015	
  Full	
  report	
  to	
  CGIAR	
  Consortium	
  2011	
  	
   Reports	
  from	
  CCAFS	
  Themes	
  and	
  Regions	
  	
   Reports	
  from	
  CGIAR	
  centers	
  	
   Financial	
  Statements	
  2011	
  	
   CCAFS	
  Report	
  to	
  EU/IFAD	
  2011	
  	
   CCAFS	
  Financial	
  Report	
  to	
  EU	
  2011	
  Full	
  list	
  of	
  2011	
  publications	
  Groundwork	
  for	
  Success:	
  CCAFS	
  Annual	
  Report	
  2011	
  	
  	
  CCAFS	
  Reports	
  CCAFS	
  Report	
  No.	
  5,	
  Mapping	
  Hotspots	
  of	
  Climate	
  Change	
  and	
  Food	
  Insecurity	
  in	
  the	
  Global	
  Tropics,	
  2011,	
  Polly	
  Ericksen,	
  Philip	
  Thornton,	
  An	
  Notenbaert,	
  Laura	
  Cramer,	
  Peter	
  Jones,	
  Mario	
  Herrero	
  CCAFS	
  Working	
  Papers	
  	
  Working	
  Paper	
  no.	
  19:	
  Participatory	
  gender-­‐sensitive	
  approaches	
  for	
  addressing	
  key	
  climate	
  change-­‐related	
  research	
  issues:	
  evidence	
  from	
  Bangladesh,	
  Ghana	
  and	
  Uganda,	
  2012,	
  Moushumi	
  Chaudhury,	
  Patti	
  Kristjanson,	
  Florence	
  Kyagazze,	
  Jesse	
  Naab	
  and	
  Sharmind	
  Neelormi	
  Working	
  Paper	
  no.	
  17:	
  Using	
  a	
  gender	
  lens	
  to	
  explore	
  farmers'	
  adaptation	
  options	
  in	
  the	
  face	
  of	
  climate	
  change:	
  results	
  of	
  a	
  pilot	
  study	
  in	
  Ghana,	
  2012,	
  Jesse	
  B.	
  Naab	
  and	
  Helen	
  Koranteng	
  Working	
  Paper	
  no.	
  4:	
  The	
  State	
  of	
  Climate	
  Information	
  Services	
  for	
  Agriculture	
  and	
  Food	
  Security	
  in	
  West	
  African	
  Countries,	
  2011,	
  Mohammed	
  Kadi,	
  Leonard	
  Njogu	
  Njau,	
  John	
  Mwikya,	
  Andre	
  Kamga	
  Tools,	
  Maps	
  and	
  Data	
  CCAFS	
  baseline	
  survey	
  data	
  and	
  materials	
  
	
   36	
  
Household	
  Survey:	
  Lawra-­‐Jirapa,	
  Ghana	
  Global	
  Summary	
  of	
  Baseline	
  Household	
  Survey	
  Results,	
  2012	
  Adaptation	
  and	
  Mitigation	
  Knowledge	
  Network	
  Initial	
  Sites	
  in	
  the	
  CCAFS	
  Regions:	
  Eastern	
  Africa,	
  West	
  Africa	
  and	
  Indo-­‐Gangetic	
  Plains.	
  Version	
  2,	
  2011.	
  Wiebke	
  Förch,	
  Patti	
  Kristjanson,	
  Philip	
  Thornton.	
  Maps	
  by	
  Jusper	
  Kiplimo)	
  	
  Project	
  Reports	
  Regional	
  Scenarios	
  for	
  Food	
  Security,	
  Environments	
  and	
  Livelihoods	
  in	
  West	
  Africa:	
  Workshop	
  Report,	
  Dakar,	
  Senegal.	
  2-­‐4	
  November	
  2011	
  Journal	
  Articles	
   	
  Review	
  of	
  seasonal	
  climate	
  forecasting	
  for	
  agriculture	
  in	
  Sub-­‐Saharan	
  Africa,	
  James	
  Hansen,	
  Simon	
  J	
  Mason,	
  Liqiang	
  Sun,	
  Arame	
  Tall,	
  Experimental	
  Agriculture,	
  47:2,	
  205-­‐204,	
  2011.	
  DOI:	
  10.1017/S0014479710000876	
  Other	
  Documents	
  Climate	
  change	
  adaptation	
  and	
  mitigation	
  in	
  agriculture	
  Status	
  &	
  Trends	
  in	
  Ghana,	
  2011,	
  Priti	
  Narasimhan	
  CCAFS1	
  Village	
  Baseline	
  Study	
  –	
  Implementation	
  Manual,	
  2011,	
  W.	
  Förch,	
  C.	
  Barahona,	
  P.	
  Kristjanson,	
  P.	
  Thornton,	
  J.	
  Mango,	
  L.	
  Onyango,	
  F.	
  Noor	
   
	
   37	
  
A7	
  Other	
  technical	
  annexes	
  	
  a) Logframe	
  entry	
  for	
  Objective	
  2.3	
  Objective	
  2.3:	
  Support	
  risk	
  management	
  through	
  enhanced	
  prediction	
  of	
  climate	
  impacts	
  on	
  agriculture,	
  and	
  enhanced	
  climate	
  information	
  and	
  services	
  
	
  
	
  	
  	
  
	
   38	
  
A8	
  Response	
  of	
  CCAFS	
  Coordinating	
  Unit	
  to	
  the	
  Draft	
  Report	
  Brief	
  response	
  on	
  recommendations	
  from	
  Evaluation	
  Report	
  The	
  Coordinating	
  Unit	
  members	
  have	
  looked	
  at	
  the	
  draft	
  report	
  and	
  have	
  some	
  initial	
  reactions	
  to	
  the	
  recommendations.	
  We	
  should	
  note,	
  however,	
  that	
  these	
  are	
  just	
  the	
  initial	
  reactions	
  and	
  the	
  formal	
  process	
  to	
  discuss	
  the	
  recommendations	
  and	
  take	
  actions	
  on	
  them	
  will	
  involve	
  the	
  full	
  Program	
  Management	
  Committee	
  (PMC).	
  In	
  addition	
  the	
  final	
  report	
  will	
  be	
  circulated	
  to	
  the	
  Independent	
  Science	
  Panel	
  (ISP)	
  and	
  CIAT	
  management,	
  together	
  with	
  the	
  PMC	
  proposed	
  response	
  to	
  the	
  recommendations.	
  The	
  ISP	
  will	
  then	
  make	
  decisions	
  about	
  strategic	
  and	
  other	
  programmatic	
  changes	
  that	
  may	
  be	
  needed	
  as	
  a	
  result	
  of	
  the	
  recommendations,	
  while	
  CIAT	
  management	
  will	
  deliberate	
  on	
  changes	
  required	
  in	
  relation	
  to	
  its	
  administrative,	
  financial	
  and	
  legal	
  responsibilities.	
  	
  	
  In	
  general,	
  the	
  recommendations	
  appear	
  fair	
  and	
  provide	
  ideas	
  for	
  positive	
  improvement	
  of	
  CCAFS.	
  	
  For	
  a	
  number	
  of	
  the	
  recommendations	
  we	
  can	
  move	
  very	
  quickly	
  to	
  implement	
  them	
  (e.g.	
  R16,	
  R17,	
  R20)	
  as	
  some	
  moves	
  in	
  that	
  direction	
  have	
  already	
  been	
  taken.	
  	
  For	
  some	
  recommendations	
  action	
  may	
  be	
  better	
  at	
  the	
  consortium	
  level	
  than	
  at	
  the	
  CCAFS	
  level	
  (e.g.	
  R18)	
  and	
  we	
  would	
  discuss	
  with	
  the	
  Consortium	
  about	
  such	
  possibilities.	
  Our	
  initial	
  query	
  on	
  R19	
  is	
  whether	
  the	
  milestone	
  level	
  is	
  the	
  appropriate	
  level	
  to	
  invest	
  M&E	
  efforts.	
  Should	
  CCAFS	
  rather	
  put	
  its	
  limited	
  resources	
  into	
  monitoring	
  achievement	
  of	
  outcomes	
  (at	
  the	
  IDO	
  level)	
  rather	
  than	
  adding	
  a	
  huge	
  amount	
  of	
  quantification	
  and	
  reporting	
  demands	
  on	
  Centres	
  at	
  a	
  lower	
  level?	
  	
  The	
  outcome	
  approach	
  would	
  be	
  more	
  in	
  keeping	
  with	
  R28.	
  We	
  think	
  we	
  need	
  time	
  to	
  consider	
  R32,	
  as	
  we	
  are	
  wondering	
  whether	
  creating	
  a	
  new	
  stratum	
  of	
  annual	
  meetings,	
  perhaps	
  the	
  EC	
  could	
  participate	
  in	
  the	
  existing	
  meeting	
  cycle,	
  such	
  as	
  the	
  annual	
  science	
  meeting	
  (since	
  the	
  intention	
  of	
  this	
  recommendation	
  is	
  technical	
  exchange,	
  not	
  donor	
  oversight).	
  	
  We	
  have	
  classified	
  the	
  draft	
  recommendations	
  as	
  to	
  the	
  follow	
  up	
  required.	
  Discussion	
  at	
  PMC	
  and	
  ISP/CIAT	
  management	
  R1.	
   More	
  focus	
  should	
  be	
  placed	
  in	
  Theme	
  2	
  and	
  on	
  non-­‐productivity	
  dimensions	
  of	
  food	
  security,	
  and	
  this	
  may	
  involve	
  recruiting	
  appropriate	
  staff	
  to	
  increase	
  the	
  capacity	
  and	
  capability	
  of	
  Centres,	
  or	
  bringing	
  in	
  outside	
  suppliers	
  of	
  research.	
  R2.	
   In	
  coming	
  years	
  CCAFS	
  should	
  be	
  prepared	
  to	
  reward	
  strategic	
  Centre-­‐led	
  research	
  with	
  top-­‐up	
  funds,	
  and	
  to	
  reduce	
  non-­‐strategic	
  Centre-­‐led	
  research	
  in	
  its	
  portfolio.	
  	
  R3.	
   This	
  would	
  need	
  to	
  be	
  done	
  in	
  an	
  objective	
  and	
  transparent	
  manner,	
  clearly	
  communicating	
  expectations	
  in	
  advance,	
  sharing	
  evaluation	
  criteria,	
  and	
  focusing	
  on	
  activities	
  with	
  potential	
  for	
  most	
  improvement.	
  	
  R5.	
   Increased	
  alignment	
  between	
  the	
  Thematic	
  and	
  Regional	
  Programmes	
  and	
  Centre-­‐led	
  activities	
  at	
  CCAFS	
  sites	
  should	
  be	
  an	
  area	
  of	
  focus	
  over	
  the	
  next	
  period.	
  R6.	
   CCAFS	
  should	
  strengthen	
  the	
  climate	
  change	
  knowledge	
  of	
  CCAFS	
  site	
  teams.	
  R12.	
   The	
  positive	
  role	
  of	
  the	
  ISP	
  should	
  be	
  considered	
  as	
  a	
  model	
  for	
  other	
  CRPs.	
  	
  R13.	
   Quantitative	
  impact	
  assessment	
  tool	
  should	
  be	
  complemented	
  by	
  qualitative	
  assessments	
  examining	
  farming	
  systems	
  and	
  decision-­‐making.	
  	
  
	
   39	
  
R14.	
   CCAFS	
  should	
  strengthen	
  internal	
  processes	
  to	
  ensure	
  that	
  challenges	
  in	
  programme	
  management	
  are	
  not	
  contributing	
  to	
  additional	
  delays	
  in	
  participatory	
  processes	
  at	
  benchmarkCCAFS	
  sites.	
  R15.	
   CCAFS	
  needs	
  to	
  focus	
  on	
  approaches	
  for	
  integrating	
  research	
  results	
  from	
  multiple	
  activities,	
  partners	
  and	
  sites	
  R16.	
   CCAFS	
  should	
  consider	
  accepting	
  multi-­‐year	
  project	
  proposals	
  to	
  ensure	
  that	
  annual	
  research	
  activities	
  are	
  strategic.	
  	
  R17.	
   CCAFS	
  should	
  screen	
  Centre-­‐led	
  proposals	
  more	
  rigorously,	
  rewarding	
  strategic	
  research	
  with	
  increased	
  funding	
  and	
  withdrawing	
  funding	
  from	
  non-­‐strategic	
  research.	
  R18.	
   CCAFS	
  should	
  consider	
  implementing	
  a	
  monitoring	
  and	
  evaluation	
  system	
  tracking	
  progress	
  in	
  partnership	
  development.	
  R19.	
   The	
  specificity,	
  measurability	
  and	
  achievability	
  of	
  logframe	
  indicators	
  should	
  be	
  improved.	
  R20.	
   CCAFS	
  should	
  move	
  towards	
  longer	
  term	
  work-­‐planning	
  and	
  contracting	
  at	
  each	
  site.	
  R25.	
   Institutionalise	
  mechanisms	
  within	
  the	
  Consortium	
  Office	
  and	
  CRPs	
  for	
  strategic	
  engagement	
  with	
  non-­‐CGIAR	
  research	
  institutions.	
  R26.	
   After	
  3-­‐4	
  years	
  the	
  ISP	
  should	
  commission	
  a	
  review	
  examining	
  the	
  role	
  of	
  participatory	
  action	
  research	
  approaches	
  to	
  climate	
  change	
  adaptation	
  and	
  mitigation,	
  specifically	
  addressing	
  scientific	
  outputs.	
  R27.	
   CO	
  to	
  establish	
  working	
  group	
  on	
  the	
  conditions	
  for	
  production	
  of	
  international	
  science	
  in	
  the	
  framework	
  of	
  the	
  field	
  based	
  researches	
  (NOTE:	
  	
  CCAFS	
  can	
  also	
  give	
  thought	
  to	
  this)	
  For	
  follow	
  up	
  by	
  Consortium	
  Office	
  R4.	
   The	
  Consortium	
  Office	
  should	
  prepare	
  to	
  support	
  a	
  CGIAR	
  system-­‐wide	
  policy	
  incentivising	
  Centres	
  to	
  focus	
  on	
  CRP	
  priorities,	
  and	
  to	
  support	
  CRPs	
  implementing	
  it.	
  R7.	
   CO	
  should	
  try	
  to	
  move	
  financial	
  planning	
  earlier	
  in	
  the	
  year	
  and	
  provide	
  confirmation	
  of	
  next	
  financial	
  year’s	
  budget	
  to	
  CRPs	
  before	
  commencement	
  of	
  the	
  financial	
  year	
  R9.	
   Reform	
  at	
  the	
  level	
  of	
  the	
  Consortium	
  Office	
  and	
  Funding	
  Council	
  would	
  be	
  needed	
  to	
  make	
  financial	
  arrangements	
  more	
  accessible	
  to	
  non-­‐CGIAR	
  partners.	
  R10.	
   Financial	
  efficiency	
  of	
  CRPs	
  would	
  be	
  improved	
  if	
  they	
  were	
  able	
  to	
  negotiate	
  lower	
  rates	
  with	
  Centres,	
  or	
  to	
  preferentially	
  select	
  Centres	
  based	
  on	
  lower	
  indirect	
  costs.	
  	
  R11.	
   The	
  Consortium	
  Office	
  should	
  make	
  provisions	
  for	
  indirect	
  cost	
  charges	
  on	
  pass-­‐through	
  funds,	
  particularly	
  on	
  long-­‐term	
  multiple	
  year	
  commitments,	
  to	
  be	
  lower	
  than	
  indirect	
  cost	
  charges	
  for	
  operations.	
  R22.	
   The	
  capacity	
  of	
  the	
  Consortium	
  Office	
  to	
  generate	
  harmonisation	
  and	
  manage	
  the	
  reform	
  process	
  should	
  be	
  reinforced.	
  R23.	
   Over	
  the	
  medium	
  term	
  (2-­‐3	
  years)	
  the	
  CO	
  should	
  map	
  policy	
  areas	
  requiring	
  harmonization	
  and	
  institute	
  reviews	
  across	
  Centres	
  with	
  a	
  view	
  to	
  harmonise,	
  where	
  possible.	
  	
  R24.	
   Over	
  a	
  longer	
  term	
  period,	
  as	
  the	
  reform	
  process	
  stabilizes,	
  the	
  CGIAR	
  may	
  wish	
  to	
  consider	
  reorienting	
  governance	
  towards	
  a	
  matrix	
  management	
  style	
  with	
  CRP	
  directors	
  reporting	
  directly	
  to	
  the	
  CO	
  rather	
  than	
  to	
  Centre	
  DGs.	
  	
  
	
   40	
  
R25.	
   Institutionalise	
  mechanisms	
  within	
  the	
  Consortium	
  Office	
  and	
  CRPs	
  for	
  strategic	
  engagement	
  with	
  non-­‐CGIAR	
  research	
  institutions.	
  R27.	
   CO	
  to	
  establish	
  working	
  group	
  on	
  the	
  conditions	
  for	
  production	
  of	
  international	
  science	
  in	
  the	
  framework	
  of	
  the	
  field	
  based	
  researches	
  R28.	
   The	
  final	
  CGIAR	
  SRF	
  and	
  M&E	
  strategy	
  should	
  incentivize	
  innovative,	
  collaborative	
  and	
  high	
  quality	
  research,	
  which	
  allows	
  for	
  the	
  possibility	
  of	
  failure.	
  R29.	
   The	
  CO	
  should	
  manage	
  expectations	
  and	
  ambitions	
  for	
  the	
  ability	
  to	
  rigorously	
  assess	
  the	
  development	
  impact	
  of	
  research.	
  A	
  focus	
  on	
  evaluative	
  learning	
  rather	
  than	
  accountability	
  would	
  support	
  better	
  programme	
  performance.	
  	
  R30.	
   As	
  the	
  SRF	
  evolves,	
  increasing	
  emphasis	
  should	
  be	
  placed	
  on	
  the	
  evaluation	
  of	
  quality	
  research	
  as	
  the	
  vehicle	
  for	
  achieving	
  development	
  impact.	
  For	
  follow	
  up	
  by	
  Fund	
  Council,	
  donors,	
  EU	
  R8.	
   Donors	
  should	
  ensure	
  that	
  funds	
  are	
  with	
  the	
  CO	
  to	
  allow	
  disbursement	
  to	
  CRPs	
  a	
  quarter	
  in	
  advance	
  R9.	
   Reform	
  at	
  the	
  level	
  of	
  the	
  Consortium	
  Office	
  and	
  Funding	
  Council	
  would	
  be	
  needed	
  to	
  make	
  financial	
  arrangements	
  more	
  accessible	
  to	
  non-­‐CGIAR	
  partners.	
  R21.	
   In	
  the	
  future,	
  the	
  Consortium	
  Office	
  and/or	
  donors	
  may	
  wish	
  to	
  map	
  and	
  evaluate	
  synergies	
  between	
  CRPs.	
  	
  R31.	
   Switching	
  to	
  multi-­‐year	
  commitments	
  would	
  reduce	
  the	
  transaction	
  costs	
  of	
  CCAFS,	
  as	
  would	
  abandoning	
  thematic	
  and	
  geographic	
  earmarking.	
  R32.	
   If	
  the	
  EC/IFAD	
  wish	
  to	
  have	
  stronger	
  influence	
  on	
  CCAFS	
  programming	
  then	
  they	
  should	
  consider	
  advocating	
  a	
  stakeholder	
  forum	
  meeting	
  once	
  per	
  year	
  alongside	
  ISP	
  meetings,	
  including	
  site	
  visits	
  and	
  substantive	
  presentations	
  and	
  discussions	
  with	
  CCAFS	
  partners.	
  R33.	
   Other	
  options	
  for	
  increasing	
  collaborations	
  with	
  EU	
  research	
  include	
  supporting	
  participation	
  of	
  EU	
  knowledge	
  centres	
  in	
  CRPs,	
  and	
  providing	
  backstopping	
  support	
  to	
  the	
  CO	
  to	
  better	
  accommodate	
  such	
  participation.	
  R34.	
   EC/IFAD	
  should	
  streamline	
  contracting	
  and	
  disbursement	
  procedures	
  to	
  reduce	
  transaction	
  costs	
  and	
  financial	
  uncertainty	
  of	
  CCAFS,	
  and	
  reduce	
  reputational	
  risk	
  to	
  the	
  EC	
  and	
  IFAD.	
  	
  	
  	
  	
   	
  
	
   41	
  
A9	
  DAC	
  Summary	
  	
  Evaluation	
  title	
  	
  2012	
  Review	
  of	
  CGIAR	
  projects	
  funded	
  by	
  the	
  EC	
  in	
  2010	
  and	
  2011	
  Subject	
  of	
  the	
  evaluation	
  CGIAR	
  Research	
  Programme	
  7:	
  Climate	
  Change,	
  Agriculture	
  and	
  Food	
  Security	
  Evaluation	
  description	
  This	
  review	
  was	
  conducted	
  as	
  part	
  of	
  an	
  EC	
  exercise	
  to	
  monitor	
  progress	
  with	
  CGIAR	
  projects	
  and	
  assess	
  orientations	
  for	
  future	
  funding.	
  Given	
  the	
  recent	
  initiation	
  of	
  CRP7,	
  this	
  review	
  focused	
  on	
  provisions	
  for	
  monitoring	
  and	
  evaluation,	
  the	
  CGIAR	
  reform	
  process,	
  and	
  the	
  role	
  of	
  EC	
  supported	
  work	
  within	
  the	
  larger	
  scope	
  of	
  the	
  CRP.	
  	
  Main	
  findings	
  CCAFS	
  is	
  a	
  strategic	
  programme	
  for	
  the	
  EC,	
  IFAD	
  and	
  the	
  CGIAR.	
  Established	
  as	
  a	
  Challenge	
  Programme	
  in	
  2009,	
  CCAFS	
  has	
  been	
  fast-­‐tracked	
  for	
  development	
  as	
  a	
  CRP.	
  Despite	
  challenges	
  in	
  the	
  first	
  year	
  of	
  operation	
  as	
  a	
  CRP	
  and	
  some	
  remaining	
  scientific	
  and	
  managerial	
  challenges,	
  significant	
  progress	
  has	
  been	
  made.	
  Three	
  Regional	
  Programmes	
  and	
  four	
  Thematic	
  Programmes	
  activities	
  are	
  in	
  place	
  at	
  local,	
  national	
  and	
  regional	
  levels.	
  CCAFS	
  has	
  however	
  faced	
  delays	
  in	
  contracting,	
  confirmation	
  of	
  budgets,	
  and	
  receipt	
  of	
  funds	
  due	
  to	
  uncertainties	
  surrounding	
  the	
  CGIAR	
  reform	
  process	
  and	
  a	
  diversity	
  of	
  management	
  systems	
  employed	
  across	
  the	
  CGIAR.	
  Although	
  the	
  financial	
  situation	
  is	
  improving,	
  more	
  can	
  be	
  done	
  at	
  the	
  CGIAR	
  level	
  to	
  streamline	
  administration	
  and	
  financial	
  management.	
  	
  The	
  development	
  of	
  the	
  CGIAR’s	
  Strategy	
  and	
  Results	
  Framework	
  to	
  prioritise	
  the	
  allocation	
  of	
  research	
  resources	
  towards	
  development	
  outcomes	
  is	
  welcomed.	
  However,	
  the	
  Review	
  Team	
  has	
  deep	
  concerns	
  that	
  attempting	
  to	
  rigorously	
  address	
  issues	
  of	
  quantitative	
  impact	
  assessment	
  will	
  divert	
  resources	
  and	
  intellectual	
  energy	
  of	
  programme	
  staff	
  from	
  substantive	
  research	
  priorities.	
  	
  Recommendations	
  The	
  review	
  has	
  indicated	
  the	
  need	
  for	
  increased	
  harmonisation	
  between	
  centres	
  and	
  CRPs	
  of	
  policies	
  and	
  procedures,	
  although	
  with	
  sufficient	
  flexibility	
  to	
  account	
  for	
  specific	
  needs.	
  The	
  location	
  of	
  CRPs	
  within	
  host	
  centres	
  may	
  limit	
  the	
  potential	
  for	
  enhanced	
  performance	
  across	
  the	
  CGIAR,	
  and	
  locating	
  the	
  CRPs	
  under	
  the	
  Consortium	
  Office	
  (CO)	
  could	
  improve	
  performance,	
  and	
  the	
  CO	
  can	
  also	
  do	
  more	
  to	
  incentivise	
  participation	
  of	
  non-­‐CGIAR	
  research	
  institutions	
  in	
  CRPs.	
  The	
  Review	
  Team	
  also	
  noted	
  the	
  potential	
  tension	
  between	
  an	
  increased	
  focus	
  on	
  achieving	
  development	
  impact	
  and	
  the	
  CGIAR’s	
  mandate	
  for	
  high	
  quality	
  research.	
  	
  Current	
  EC/IFAD	
  funding	
  arrangements	
  do	
  not	
  have	
  a	
  significant	
  impact	
  on	
  CCAFS	
  programming	
  directions.	
  If	
  the	
  EC	
  wishes	
  to	
  increase	
  influence	
  on	
  CCAFS	
  programming,	
  they	
  should	
  consider	
  other	
  means	
  such	
  as	
  participation	
  in	
  stakeholder	
  meetings,	
  the	
  provision	
  of	
  parallel	
  support	
  to	
  EU	
  research	
  institutions,	
  or	
  backstopping	
  support	
  to	
  the	
  Consortium	
  Office.	
  The	
  EC	
  should	
  maintain	
  current	
  levels	
  of	
  funding,	
  and	
  consider	
  increased	
  financial	
  support	
  inline	
  with	
  CCAFS’	
  capacity	
  to	
  absorb	
  additional	
  funding.	
  	
  	
  	
  	
  
	
   42	
  
	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  	
  
	
  	
  	
  For	
  further	
  information	
  on	
  AGRINATURA	
  Association:	
  	
  	
  AGRINATURA	
  Association	
  Secretariat	
  Czech	
  University	
  of	
  Life	
  Sciences	
  Prague	
  Kamýcká	
  129,	
  165	
  21	
  Prague	
  6	
  	
  Czech	
  Republic	
  Phone:	
  +420	
  224	
  382	
  011	
  Fax:	
  +420	
  224	
  382	
  012	
  secretariat@agrinatura.eu	
  http://www.agrinatura.eu/	
  	
  
	
  	
  	
  For	
  further	
  information	
  on	
  AGRINATURA-­‐EEIG:	
  	
  	
  	
  AGRINATURA-­‐EEIG	
  Secretariat	
  42	
  rue	
  Scheffer	
  F-­‐75116	
  PARIS	
  FRANCE	
  Fax:	
  +33.1.53.70.21.56	
  secretariat@agrinatura-­‐eeig.eu	
  http://www.agrinatura.eu/