Business Development Services (BDS) Forum

Monitoring and Evaluation (M+E) of BDS

Hollow-block forms,
Cameroon

Website Statistics

M+E for Projects and Partner Organizations  [top]

Extensive Monitoring and Evaluation Systems
for local Institutions?

International experts and development projects often tend to put their high-quality M+E system on local partner organisations. Thus, in the past we have seen a growing culture of indicator-based planning and evaluation systems not only for international projects but also for local institutions. Often we can see a nice gallery of 47 indicators with detailed figures of expected results (in percentage and absolute figures) - for local institutions! And on top of that, most of the planning documents lack to indicate the sources of verification and the persons responsible for data collection.

But what problems do such monitoring and evaluation systems with these indicators encounter? International projects may have the necessary funds and staff for data collection and analysis, but not local institutions. They latter can only follow so long as they are pushed and assisted by international projects; but after project's end there will be no sustainability - the stipulated M+E system will immediately come to its end.

Everybody is speaking about Impact Evaluation -
what about Activity Monitoring?  [top]
Another problem in the recent M+E discussion is that emphasis is made exclusively on "Impact evaluation". "Activity monitoring" is too boring for a "scientific-based" M+E discussion. But in our understanding activity monitoring is the basis of impact evaluation: How can one evaluate the performance and quality of a programme if one does not know what and how much has been done? Don't believe that activity monitoring is that simple - we have seen many national institutions and even international projects without any activity monitoring and systematic data collection.

Activity monitoring in business development service delivery should at least capture the following data: number and type of businesses involved (by business-sector, region and gender), number and type of services provided (by gender), number and type of trainings provided (by gender) and partner organisations involved. If we got these basic data, then we can proceed with impact monitoring. In most cases you have to engage a local M+E officer who assures systematic and permanent data collection. Projects without an M+E officer are regularly in mess!

Project Monitoring and Evaluation  [top]
The M+E system of the former GTZ MSE Development Project was a simple one - not from the Harvard University, but efficient: we developed a system of permanent activity monitoring and data collection and engaged a permanent M+E officer. Impact monitoring was done periodically, once per year, using an independent local consultant. Questionnaires focussing on impacts measured by parameters such as income generation, increased turnover and increased employment on the basis of initially defined indicators. All data have been gathered in one single M+E folder (see pictures left side) organised by separators presenting the project indicators to be achieved (details see M+E documents below).

In addition to that it should be assured that an indicator based M+E system defines right in the beginning the sources of verification and the responsible persons for data collection:

    (Example from the SNV Value Chain Programme "Support to Business Associations and
    their Access to Markets - BOAM" www.business-ethiopia.com.

... but projects often postpone filling the last three columns with the effect that nice initial planning documents are waiting for nothing to be checked (M+E overview for download: m-and-e-overview.doc). More details see documents for M+E.

M+E of Partner Organizations  [top]
If indicator-based M+E systems are appropriate for international development projects with the necessary resources, we do not impose this project M+E systems on our partner organisations. Organize periodical meetings and discussions with the help of the below mentioned SWOT analysis and you will get the minimum necessary for impact evaluation with partner organisations. Convince them to open an office folder for data collection where all activities and services will be documented in a centralized manner (see approaches on Organizational Development and the Folder method on www.bds-ethiopia.net/lesson-od.html)

In group discussion, put cards on the SWOT table. In the final report, cards are classified, discussed and comments made including recommendations.You can do one general SWOT exercise for your entire structure or/and sub-structure-specific SWOTs in sub-groups to go more into detail.  

Checklist for Monitoring and Evaluation  [top]

you should be ready with the design of your M+E system in the first six project months.

• Are your planning documents complete?

  • Is your initial planning document (offer) available for project staff?

  • Are your planning documents ok or do they have to be chnaged?

  • Do you have a Plan of Operations including title, introduction, project planning matrix, time table sheets of results, activities and indicators?

  • Do you have yearly activity plans?

• Did you design an M+E system?

  • Do you think, you can do M+E (data collection and data analysing) yourself or do you have an M+E officer?

  • Do you have job descriptions for your M+E staff?

  • Did you design and fill an M+E overview table on how to collect data to prove the respective indicators (where? who? when? how?)

  • Did you establish an M+E office folder centraliszing all M+E data in one folder (folder with seperators presenting the indicators; behind each indicator the respective documents should be filed)

  • Did you establish an M+E office box for big documents?

  • Is the M+E staff regularly updating these M+E tools?

• Data collection

  • Based on the initial planning and project requirements, did you define the needs for data collection?

    • Number and type of supported businesses, number and type of support activities;

    • Number and type of partner organizations supported, number and type of support activities;

    • Number and type of networking and information services.

    • Number and type of policy and strategy issues discussed.

  • Are these data analysed and reported?

• Activity and Impact Monitoring

  • Do you assure permanent activity monitoring by your project staff?

  • Do you assure periodical impact studies with an external independent local consultant?

    • Terms written, questionnaires designed on the basis of the defined impact indicators.

    • Consultants engaged

    • Study discussed and analysed.

  • If the expected impact did not arise, did you take the necessary steps to change your intervention?

  • Are your M+E system and your data collection of activities and impact ready whenever necessary for the next external evaluation?

 
M+E Documents for download  [top]

Dieter Gagel, Heidelberg 2006
Product description 5: Start and run a simple but efficient Monitoring and Evaluation System  product-m+e.doc 65 KB

Dieter Gagel, Addis Ababa, 2005
Monitoring and Evaluation of BDS, Organizational Development and Networking. Our M+E System m-and-e-system.doc

Dieter Gagel, Addis Ababa 2005
M+E Overview Table m-and-e-overview.doc

MSE Development Project, Addis Ababa 2004
Report on activity monitoring of BDS, CEFE entrepreneurship training, organizational development and networking 
activity-monitoring-8-10-04.doc

Mulatu Zerihun, freelance consultant, Addis Ababa 2004
Impact Study of MSE development project activities on BDS, CEFE entrepreneurship training, organizational development and networking  impact-final-19-10-04.doc

Dieter Gagel, for SNV-BOAM, Addis Ababa 6/2005

Implementation of a Monitoring and Evaluation System for Value Chain Development m+e-report-7-05.pdf  500 KB

Complete manual for SWOT analysis: SWOT-Analysis.doc 116 K

Lessons learnt on Monitoring and Evaluation (M+E)  [top]

• No real impact monitoring is possible, when there is no activity monitoring. How can you know how was the quality of your services if you don’t know who has been served and what and how much you have done?

• Activity monitoring data for BDS have to be collected permanently: number and type of enterprises supported by region, gender, size. Type and number of services (including trainings) planned and achieved.

• Impact monitoring can be done periodically, e.g. once per year, by an independent local consultant and by sample interviews of businesses supported.

• Impact is: Business has improved in terms of increased turnover, income, increased employment, better security. Impact is not: activities have been achieved.

• Indicator based development projects need an M+E overview table on how the data for the project indicators will be collected: Indicator – Source of verification – Timeframe – Responsible for data collection – Already existing documents – Actors involved.

• Impact monitoring of international projects needs a permanent local M+E officer, if not you will be in mess!

• Complexe M+E systems of international projects cannot be “downsized” for local partners. M+E for local partners must be developed based on their limited M+E capacity: simple activity plan, activity reports and SWOT analysis for simple impact monitoring.

[top]

New BDS Portal in Senegal www.senegal-entreprises.net .
One of the biggest national BDS portals world wide!

Value Chain Development in Ethiopia www.business-ethiopia.com

Blacksmith, The Gambia