Leonardo ENERGY’s Impact

Assessing the Impact of the Leonardo ENERGY Initiative

Entries Tagged as 'evaluation'

Macro indicators relevant to our campaigns now published on-line

July 23rd, 2008 · No Comments · evaluation, impact

A selection of European and World macro indicators relevant to our campaigns is now published on-line.

To assist the development of campaign impact assessment, the indicators have been posted here:

http://www.leonardo-energy.org/planning/?p=801

This is only a selection and other indicators exist. However, these are the key indicators most in use today.

I would be delighted to gather more for the team, on request.

[Read more →]

Tags:

Surveying users

February 28th, 2008 · 5 Comments · evaluation, impact, reception analysis, tools

We need to decide whether we want to survey few users in-depth, on many user groups on a variety of issues in a shallow way.The latter appears more appropriate considering the rich content of LE. In this case, a simple attitudinal measurement instrument is needed. Ref [1] divides groups in 5 categories of receptivity: hostile, neutral, uninterested, uninformed and supportive. Each group requires a different approach:

  • hostile: find areas of agreement, use solid science, phrase proposals in value terms. The goal is to divert negative activism into neutrality.
  • neutral: spell out benefits, focus on the downside of not accepting a proposal, discuss alternatives. Keep it simple. The goal is the convert neutrals into supporters.
  • uninterested. these people are informed, but simply don’t care. Appeal to their self-interest.
  • uninformed: lack information. Establish credibility. Keep it simple. Find an emotional link (stories, anecdotes, case studies, …).
  • supportive: keep them on your side. Recharge them, remind them of the stakes, help them with arguments against opponents.

The criteria of a dream case is a positive answer to 5 questions. However, in the real world, such cases do not even need selling – they fly by themselves. So it’s a matter of approaching the ideal as much as possible:

  1. is the case logical, and consistent with facts and experience?
  2. does it favourably address the interests of people who have to decide?
  3. does it eliminate or neutralise competing alternatives?
  4. does it recognise and deal with politics?
  5. is it endorsed by objective, authoritative 3rd parties?

[1] Harvard Business Essentials, Power, Influence and Persuasion, HBS Publishing 2005

Blogged with Flock

[Read more →]

Evaluating programmes

February 26th, 2008 · 1 Comment · evaluation, impact

Bill’s 4-step method to evaluate technology may also be suitable for programmes:

  1. Establish that there is a copper potential to be influenced.
  2. Develop a conceptual model how the programme influences this potential.
  3. Collect evidence that the programme works as designed.
  4. Verify the return. Benchmark activities. Check quality.

A programme statement along above lines does not need to be long. In fact, shorter statements reveal crystallised thinking. For example:

  1. Technical solutions are emerging to assist the elderly and mobility-impaired to live comfortable and independently at home. These require a significant upgrade in the electrical installation.
  2. This market needs custom solutions for target groups sharing a pathology. The programme will work at 3 levels: (1) mapping user needs through market research, partnering with consumer organisations, (2) develop training packages on ‘smart installations’ for contractors in partnership with their federation and (3) develop cost-benefit analysis for governmental organisations to provide financial support with a net benefit on social security budgets.
  3. In the short run, this programme will be measured on its intelligence gathered, and outputs produced, e.g. user research reports, training courses published, training courses delivered, cost-benefit analysis case studies, … In the medium run, the emergence of solution package will be monitored, with their market share. Copper IoU of various systems will be assessed. A final output to be monitored is government funding and soft regulation promoting this sector.
  4. …

Blogged with Flock

[Read more →]

Marketing Measurement Webcast series

February 18th, 2008 · No Comments · evaluation, impact, tools

Better Management has an excellent series of 6 webcasts so far on Marketing Measurement:
Marketing Measurement Today – BetterManagement.com

[Read more →]

When to measure

February 9th, 2008 · No Comments · evaluation

When you can measure something in numbers, then you know something about it.

Lord Kelvin

Starting a new campaign, it’s almost impossible to make the business case, since you don’t know enough. But to know enough about the market, you need to start a campaign (or do some sterile market research), for which we need a business case …

What ways out do we have?

  • Be lenient on the business case in the early stages
  • Run a pilot project
  • Accept a percentage of budget to be used for ‘exploratory activities’
  • Never start anything new

[Read more →]

Tags:

Sense of scoring

February 6th, 2008 · 2 Comments · evaluation, tools

When does scoring make sense. Suppose you’re an account manager for a geographical region, and have 100 prospects to develop into customers. What you’re likely to do is to define a number of criteria relevant for you to prioritize these accounts. Factors under consideration might be:

  • do you do business with this client in other regions?
  • growth potential
  • size
  • interesting account (business may be low, but the client helps you to push the boundary)
  • …

You’ll do a first ‘quick & dirty’ ranking, that you revisit in the beginning frequently to go through a number of iterations, to come up with a system that sounds like a plan. In the beginning, nonsensical results are likely to come up, which you’ll quickly spot and fine tune. In the self-contained universe of an account manager or small team, this approach makes a lot of sense.

Now suppose that you want to centralise this approach accross account managers, to allocate resources efficiently. Immediately, it gets very political. Every account manager will want criteria in there on which his accounts score well, and you’ll end up with an amalgamate of criteria. Next, gaming will enter into the scoring. If you’re team of account managers scores, you may see ‘reciprocal scoring’. And nonsensical results are not sorted out quickly anymore – they are convenient for anybody who wants to bypass the system, which is only a matter of time.

So lead scoring, campaign scoring, project scoring, which makes a lot of sense for individual portfolio’s, loose their value when used at a too aggregate level.

Blogged with Flock

[Read more →]