(back to the Documentation)

Synthesis Multi Actor Multi Criteria Analysis Methodology

(Based on: Macharis, C. De Witte, A. Festraets, T. Ampe, J, 2007, "The multi-actor, multi-criteria analysis methodology (MAMCA) for the evaluation of transport projects : theory and practice", submitted for the Journal of Advanced Transportation)


1. Evaluation methods used in transport projects

2. The multi stakeholder multi criteria analysis evaluation framework

3. Examples of the successful use of the MAMCA methodology



1. Evaluation methods used in transport projects

Several evaluation methods can be employed for the evaluation of transport related projects. Five common used evaluation methods can be identified, namely the private investment analysis (PIA), the cost effectiveness analysis (CEA), economic effects analysis (EEA), the social cost benefit analysis (SKBA) and the multi criteria decision analysis (MCDA).

The private investment analysis (PIA) or private cost-benefit analysis takes the pure financial cost and benefits of the project into account. It is being executed from the point of view of the private or public investor and does not take more broad objectives into account.

The cost-effectiveness analysis (CEA) looks at the effectiveness of the measure in terms of the costs that government puts in. The CEA has thus a one criterion - one actor perspective. It looks at the effectiveness with regard to one specific goal.

The economic-effect analysis (EEA) or regional economic impact study (REIS) looks at the projects' impact on added value, employment and fiscal revenue. Input-output tables are used and indirect effects are captured through the use of multiplicators. The EEA is specifically designed for mapping of the government perspective and takes only three criteria of this particular stakeholder into account.

In conclusion we can state that the private investment analysis (PIA), the cost effectiveness analysis (CEA) and economic effects analysis (EEA) are not interesting if we want to include stakeholders into the analysis.

The social cost benefit analysis (SCBA) is grounded in welfare theory. It takes a wider societal perspective and in this sense it can also include the external costs of transport into the analysis. It is mainly used when there are only a few possible alternatives to be examined. A discount rate is used to calculate the net present value and the internal rate of return of the project. All the costs and benefits have to be expressed in monetary terms. Some of the external effects of transport are difficult to assess and translate in monetary terms. The SCBA is based on the compensation criterion. The introduction of a stakeholder analysis in a SCBA is in principle possible if the costs and benefits are structured according to the stakeholders. So for each stakeholder the costs and benefits would be listed and calculated. However, in the end of the process the costs of one stakeholder can be compensated by the benefits of another. The redistribution effects are not clearly coming out of such an analysis. This problem can be avoided by creating an end table per stakeholder so as to get a cost benefit analysis of the project per stakeholder. The problem of monetarisation will however exclude many more subjective or qualitative costs or benefits from the analysis.

The multi criteria decision analysis (MCDA) provides a framework to evaluate different transport options on several criteria. The concept of stakeholders was first introduced in MCDA by Banville et al. (1998) (1). As denoted by them the multi criteria analysis is useful for the introduction of the stakeholder concept. In their paper, a first framework for the introduction of the concept of stakeholders is introduced. They argue that certainly in the first three stages of a multi criteria analysis the concept of stakeholders can enrich the analysis, but they do not include the stakeholders within the methodology further on. The multi actor multi criteria analysis on the other hand does incorporate stakeholders in all stages of the analysis.

2. The multi stakeholder multi criteria analysis evaluation framework

The methodology consists of seven steps (see figure below). The first step is the definition of the problem and the identification of the alternatives (step 1). The various relevant stakeholders are then identified as well as their key objectives (step 2). Second, these objectives are translated into criteria and then given a relative importance (weights) (step 3). For each criterion, one or more indicators are constructed (e.g., direct quantitative indicators such as money spent, number of lives saved, reductions in CO2 emissions achieved, etc. or scores on an ordinal indicator such as high/medium/low for criteria with values that are difficult to express in quantitative terms, etc.) (step 4). The measurement method for each indicator is also made explicit (e.g. willingness to pay, quantitative scores based on macroscopic computer simulation, etc.). This permits the measurement of each alternative performance in terms of its contribution to the objectives of specific stakeholder groups. Steps 1 to 4 can be considered as mainly analytical, and they precede the "overall analysis", which takes into account the objectives of all stakeholder groups simultaneously and is more "synthetic" in nature. Here, an evaluation matrix is constructed aggregating each alternative contribution to the objectives of all stakeholders (step 5). The MCDA yields a ranking of the various alternatives and gives the strong and weak points of the proposed alternatives (step 6). The stability of this ranking can be assessed through a sensitivity analysis. The last stage of the methodology (step 7) includes the actual implementation.

Graph on the MAMCA methodology

Source : Multi actor multi criteria analysis (MAMCA), Macharis 2004

2.1 Define alternatives

The first stage of the methodology consists of identifying and classifying the possible alternatives submitted for evaluation. These alternatives can take different forms according to the problem situation. They can be different technological solutions, possible future scenarios together with a base scenario, different policy measures, long term strategic options, etc. There should be minimum two alternatives to be compared. If not, a social cost benefit analysis might prove to be a better method for the problem.

2.2 Stakeholder analysis

In step 2 the stakeholders are identified. Stakeholders are people who have an interest, financial or otherwise, in the consequences of any decisions taken. An in depth understanding of each stakeholder group's objectives is critical in order to appropriately assess the different alternatives. Stakeholder analysis should be viewed as an aid to properly identify the range of stakeholders to be consulted and whose views should be taken into account in the evaluation process. Once identified they might also give new ideas on the alternatives that have to be taken into account.

2.3 Define criteria and weights

The choice and definition of evaluation criteria are based primarily on the identified stakeholder objectives and the purposes of the alternatives considered. A hierarchical decision tree can be set up. Several methods for determining the weights have been developed. The weights of each criterion represent the importance that the stakeholder allocates to the considered criterion. In practice the pair-wise comparison procedure proves to be very interesting for this purpose. The relative priorities of each element in the hierarchy are determined by comparing all the elements of the lower level in pairs against the criteria with which a causal relationship exists. The applied multi actor multi criteria analysis method and software (see step 6) allow an interactive process with the stakeholders in order to perform sensitivity analysis.

2.4 Criteria, indicators and measurement methods

In this stage, the previously identified stakeholder criteria are "operationalized" by constructing indicators (also called metrics or variables) that can be used to measure whether, or to what extent, an alternative contributes to each individual criterion. Indicators provide a "scale" against which a project's contribution to the criteria can be judged. Indicators are usually, but not always, quantitative in nature. More than one indicator may be required to measure a project's contribution to a criterion and indicators themselves may measure contributions to multiple criteria.

2.5 Overall analysis and ranking

The MCDA method used to assess the different strategic alternatives can be any MCDA-method. Most of the cases discussed below are analysed with the Analytical Hierarchical Process (AHP). This method, described by Saaty, allows to build a hierarchical tree and to work with pair wise comparisons. The consistency of the different pair wise comparisons as well as the overall consistency of the whole decision procedure can easily be tested in AHP that can handle both quantitative and qualitative data, the latter being very important for transport evaluations. Certain criteria in transport concern ecological impact or road safety issues. These criteria are difficult to quantify. Moreover, the method is relatively simple and transparent to decision makers and to the public. The method does not act like a black box since the decision makers and the stakeholders can easily trace the way in which a synthesis was achieved.

2.6 Results

The multi criteria analysis developed in the previous step eventually leads to a classification of the proposed alternatives. A sensitivity analysis is in this stage performed in order to see if the result changes when the weights are changed. More important than the ranking, the multi criteria analysis allows to reveal the critical stakeholders and their criteria. The multi actor multi criteria analysis provides a comparison of different strategic alternatives and supports the decision maker in making his final decision by pointing out for each stakeholder which elements have a clearly positive or a clearly negative impact on the sustainability of the considered alternatives.

2.7 Implementation

When the decision is taken, steps have to be taken to implement the chosen alternative by creating deployment schemes. This implementation process can be complemented by a cost benefit analysis for well defined projects.

3. Examples of the successful use of the MAMCA methodology


3.1 Location analysis for intermodal terminals

The MAMCA methodology was first applied to evaluate the location of intermodal terminals (Macharis, 2000 and Macharis, 2004). The LAMBIT-model, Location Analysis Model for Belgian Intermodal Terminals, provided the framework for the decision making process on the location of new intermodal terminals. In a preliminary phase the traffic potential of the terminal projects is determined. In order to have a sustainable terminal, the traffic potential in the surrounding area of the terminal must be large enough to support it. Furthermore, the impact of the new projects on the market area of the existing terminals must be analysed. A network model allowed the determination of the traffic potential and the impact on the existing terminals. In the risk analysis, the proposed locations were screened according to pre-determined standards (large enough for an intermodal terminal, grants by the local district, ability to get permissions, etc.). In the next phase a more comprehensive evaluation of a discrete set of terminal projects was applied. The criteria used in this evaluation represent the aims of the parties involved, namely the users of the terminal, the operators/investors and the community as a whole.

3.2 Brussels garbage by water

In the BRUGARWAT case study (Brussels Garbage by Water) the possible modal shift of waste transport in the Brussels region was analysed (Macharis and Boel, 2004). A social cost benefit analysis was done in iteration with a multi actor multi criteria analysis. The social cost benefit analysis made it possible to get an idea of the social desirability of the project, while the multi criteria analysis was used for the choice between the several possible types of package units that can be used. The alternatives were the ISO 20', 30' and 40' containers that compact the waste, ISO 40' open top containers and MSTS containers (multi service transport system) or bulk transport. For the garbage operator (Net Brussel) the operational results (containers, ships, terminals, savings compared to road transport) were important. For the local community the effects on visual intrusion, noise, smell and congestion were considered. For the community as a whole the impact of the modal shift on accidents, global congestion, global noise, pollution and climate change were taken into account.

3.3 Port of Brussels

In the framework of the master plan of the Port of Brussels (Dooms, Macharis and Verbeke, 2004) the methodology was used in two types of applications. A first type of application was for a location analysis and planning for a separate port site (i.e. the site of Carcoke and Béco). In the mini master plan Carcoke the possible destinations of the site were compared to each other. The strategic alternatives were here an European distribution centre (EDC), value added logistics (VAL) or recycling. The second type of application was for the long term strategic planning of the whole port area. For each port area the possible strategic development options were compared. This consisted of a pro-active and a status-quo scenario. Depending on the area different stakeholders were included in the analysis. Four main stakeholders are important in the context of port planning: government, local community, port authority and potential port users. A main stakeholder can be unbundled in several sub-categories with their own specific criteria (e.g. local community can be unbundled in tourists, residents, adjacent non-port firms and organisations) if the characteristics of a zone necessitate this approach. The definition of criteria for each stakeholder follows the approach followed for the definition of stakeholders: the criteria depend on the purpose, i.e. on the characteristics of each zone. This is very relevant for stakeholders, such as government and the local community, as their objectives often change throughout the port area. For example, in some port zones government objectives will be oriented towards economic development, whereas other port zones will be considered suitable for housing development and recreation. The objectives of the port authority and the port companies are much more stable, although there can be variations depending on port zone, but not as intense as for government or local community stakeholders. Another reason for this difference is that the port authority can be considered as 'identical' or 'univocal' over the whole port area, whereas the identity of local community stakeholders and sometimes even government (e.g. municipalities) can change depending on the considered port zone. The last step of the methodology consisted in checking if the different strategic options proposed in each port zone were consistent with each other.

3.4 HST-teminal in Brussels

The introduction of a new HST-terminal in Brussels (Meeus, Macharis and Dooms, 2004) was analysed also according to the MAMCA methodology. Seven possible alternatives were proposed and compared. The stakeholders here were the railway operator NMBS, the government, the local community and the users.

3.5 Extension of DHL at Brussels Airport

A very interesting case was the evaluation of the possible extension of DHL at Brussels Airport (Verbeke, Macharis, Dooms and S'Jegers, 2004). The decision consisted in choosing between a pan-European consolidation strategy with Brussels Airport as a superhub, a West-European expansion strategy with Brussels Airport as one of the multihubs or the further development of DHL in an external superhub, for example in Leipzig. The stakeholders in this case were DHL, the airport operator (Brussels Airport Company), the government and the local community. Interesting in this case from a methodological point of view - next to the fact that the methodology highlighted very well the difficult decision making the government was involved in - was the introduction of time horizons into the analysis. With a 2012 time horizon, the global preference was for the multihub expansion whereas when the time horizon moved to 2023 the global preference changed to the superhub. This is due to the constraints of the Brussels Airport Company and also DHL itself to quickly grow in terms of capacity. In a longer timeframe, this does not pose any problems anymore. The government had to provide a legal institutional framework with horizon 2023 that could secure a long term growth of the activity, especially after 2012. If not, the hub-activities of DHL would be relocated to another airport. In the short run, towards 2012, the multi criteria analysis showed that Brussels Airport had to be protected as a node in a multi-hub network of the company DHL.

  1. Banville, C., Landry, M., Marte,l J-M. and Boulaire, C. (1998) "A stakeholder approach to MCDA", System Research 15: 15-32.
  2. Dooms, M., Macharis, C., Verbeke A., ism Ecorys, BGDA and COOPARCH-RU (2004) Masterplan van de Haven van Brussel: Interimrapport 5, Haven van Brussel, Brussel.
  3. Dooms, M. and Macharis, C. (2003) "A framework for sustainable port planning in inland ports: a multistakeholder approach", The 43th European Congress of the Regional Science Association (ERSA), Young Scientist Sessions, University of Jyvaskyla, 27-30 August 2003.
  4. Macharis, C. and Boel, B. (2004) "BRUGARWAT: Brussels Garbage by water", Vervoerslogistieke werkdagen, 4 en 5 november, Hoeven, Nederland, In Ruijgrok, C. and Rodenburg, R. (eds) Bijdragen vervoerslogistieke werkdagen.
  5. Macharis, C. (2000) Strategische modellering voor intermodale terminals. Socio-economische evaluatie van de locatie van binnenvaart/weg terminals in Vlaanderen, Doctoraal proefschrift, Vrije Universiteit Brussel, Brussel.
  6. Macharis, C. (2004) "The optimal location of an intermodal bargeterminal", In: Beuthe, M., Himanen, V., Reggiani, A. and Zamparini, L. (eds) Transport Developments and Innovations in an Evolving World.
  7. Meeus, T., Macharis, C. and Dooms, M. (2004) Inplanting van tweede HST-terminal te Brussel: scenario Brussel-Nationaal moet verder worden onderzocht, De Lloyd. Universiteit en Transport.
  8. Verbeke, A., Macharis, C., Dooms, M. and S'Jegers, R. (2004) Economische impactstudie van de uitbreiding van de hubactiviteiten van het koeriersbedrijf DHL op de luchthaven van Zaventem (Brussel Nationaal; Brussels Airport). EINDRAPPORT, studie in opdracht van de provincie Vlaams Brabant.