Leonardo ENERGY’s Impact

Assessing the Impact of the Leonardo ENERGY Initiative

Macro indicators relevant to our campaigns now published on-line

July 23rd, 2008 by · No Comments · evaluation, impact

A selection of European and World macro indicators relevant to our campaigns is now published on-line.

To assist the development of campaign impact assessment, the indicators have been posted here:

http://www.leonardo-energy.org/planning/?p=801

This is only a selection and other indicators exist. However, these are the key indicators most in use today.

I would be delighted to gather more for the team, on request.

Tags:

Tonnage impact per trainee

July 9th, 2008 by · No Comments · impact

Another approach for impact is to measure tonnage per person reached. Historically, we’ve not done this as it produced very high dollar/tonne campaigns, and hence needs very high replication factors for such programme to work.

However, with the recent advances in web-based communication, the subsequent factor X cost reduction in delivering messages, our mastering of these technologies and the traceability it offers, it may be time to rethink.

Hereby a message from Jonathan which I’d like to log here. It summarises our current knowledge on the subject.

[Read more →]

Tags: ·

e-learning – the science

April 3rd, 2008 by · No Comments · reception analysis

This excellent book [1] on e-learning summarises the state of knowledge in the field from a scientific perspective. A few highlights (including a few surprises):

  1. The market for professional training is approaching 60 B$ annually in the US. About a third a delivered by technology. Over the past 5 years, e-learning has grown 200% at the expense of classroom learning.
  2. The first media comparison study dates from 1947. A classroom was divided in 3 groups, with one group given a lecture, the second watching a movie and the third reading through a text. Subsequent tests showed no significant difference. Hundreds of media comparisons have been conducted meanwhile, confirming that the medium has no influence. Quality of instruction (and interest of the student) is a much more important factor. (our own Cracow experiment confirms this finding)
  3. The contiguity principle: Integrating text & graphics is much more effective than separating text in captions.
  4. The multimedia principle: using words combined with graphics is much more effective than using words alone.Interestingly, it appears that a sequence of static graphics is more effective than animations. Unless there is a compelling reason, best practice is the use of static sequences rather than animations.
  5. The modality principle: words as audio narration to graphics are more effective than on-screen text. We only have a visual and and auditory channel to process information. Using both channels in parallel increases processing capacity.
  6. The redundancy principle: explaining visuals with words using both audio and text hurts learning. (we’ve all been exasperated at times by presenters reading their slides). Use one or the other.
  7. The coherence principle: adding interesting but loosely related materials hurts the learning process.
  8. The personalisation principle: a conversational style and virtual coaches improves learning.
  9. The segmenting principle: manage complexity by breaking a lesson into parts.

[Read more →]

4 type of communications – 4 campaign types

March 20th, 2008 by · No Comments · impact, reception analysis, technical communication, tools

Communication can serve 4 purposes:

  1. inform – raise awareness
  2. interpret – raise understanding
  3. convince – support readers to develop an opinion or correct a misunderstanding
  4. behaviour change

Obviously, the ultimate goal is behaviour change, this is largely an outcome beyond our control. Most communication are inputs designed for #1-3, with the ultimate purpose of achieving number 4.

This list may be of interest to think about our marketing communications. In essence, it reiterates the point of an earlier post on measuring impact from a different perspective.

The key question is the level of resources we choose to deploy towards a given objective. This needs to be combined with a benchmarking approach on cost effectiveness and quality of the action. As an impact assessment methodology, I sense there is something fundamentally different, but cannot yet put my finger on it. To be continued.

Blogged with the Flock Browser

If you know your return, it could have been more

March 3rd, 2008 by · No Comments · impact

In physics the Heisenberg uncertainty principle rules. It states that it is impossible to measure the impulse of a particle if you know its exact place, and vice versa.
In marketing communication, there is also an uncertainty principle: the maximum return of your communication action is impossible to measure. Because once you make the return measurable, it loses value.

Perhaps this is not quite so much the case if your market enables a streamlined communication. If you’re selling diapers, you can measure your market share before and after your communication action, and this will allow you to make a fairly good estimate of your return.

But if a chaotic communication model is to be preferred (see my previous post), things aren’t that easy. The paradox is that the more you want to make the return measurable, the more you will have to streamline your communication and the less effective your communication will be.
[Read more →]

The benefits of soft power

March 1st, 2008 by · 1 Comment · branding

In this debate, a track we’ve ignored so far is the branding and positioning of our organisation to influence the markets we operate in. We’re upstream, remote from our markets and the use of our product is quite dispersed. Therefore, consideration on this aspect is important.

A relevant article worth a read is this paper from HBS Working Knowledge on The Benefits of Soft Power.

At Leonardo ENERGY, we have certainly noticed a difference from being one of the top 100,000 websites in the world. It makes some things easier, and avoids lots of explanations.

Blogged with Flock

Surveying users

February 28th, 2008 by · 5 Comments · evaluation, impact, reception analysis, tools

We need to decide whether we want to survey few users in-depth, on many user groups on a variety of issues in a shallow way.The latter appears more appropriate considering the rich content of LE. In this case, a simple attitudinal measurement instrument is needed. Ref [1] divides groups in 5 categories of receptivity: hostile, neutral, uninterested, uninformed and supportive. Each group requires a different approach:

  • hostile: find areas of agreement, use solid science, phrase proposals in value terms. The goal is to divert negative activism into neutrality.
  • neutral: spell out benefits, focus on the downside of not accepting a proposal, discuss alternatives. Keep it simple. The goal is the convert neutrals into supporters.
  • uninterested. these people are informed, but simply don’t care. Appeal to their self-interest.
  • uninformed: lack information. Establish credibility. Keep it simple. Find an emotional link (stories, anecdotes, case studies, …).
  • supportive: keep them on your side. Recharge them, remind them of the stakes, help them with arguments against opponents.

The criteria of a dream case is a positive answer to 5 questions. However, in the real world, such cases do not even need selling – they fly by themselves. So it’s a matter of approaching the ideal as much as possible:

  1. is the case logical, and consistent with facts and experience?
  2. does it favourably address the interests of people who have to decide?
  3. does it eliminate or neutralise competing alternatives?
  4. does it recognise and deal with politics?
  5. is it endorsed by objective, authoritative 3rd parties?

[1] Harvard Business Essentials, Power, Influence and Persuasion, HBS Publishing 2005

Blogged with Flock

Measuring LE programme effectiveness [II]

February 27th, 2008 by · 2 Comments · Uncategorized

In my last post we said:- to understand the effectiveness we’re having we need to be clear about what we expect to achieve from communicat9ing with multiple target groups about multiple LE programmes- to get some idea of how effective the outputs are we need to find mechanism to track the conversion of “mega-metrics” into behavioural / attitudinal change.- to do that practically we need to set up a panel- form that a workable hypothesis on how all this functions can be drawn – a model for as many different “niche” markets as we deem necessary that explain|: group x visited y times and, as a consequence, did the following. The second of these posts builds on that and presents the process we might follow [and this chimes with what Bill Howe was proposing]. So the process could look like this:
[Read more →]

Evaluating programmes

February 26th, 2008 by · 1 Comment · evaluation, impact

Bill’s 4-step method to evaluate technology may also be suitable for programmes:

  1. Establish that there is a copper potential to be influenced.
  2. Develop a conceptual model how the programme influences this potential.
  3. Collect evidence that the programme works as designed.
  4. Verify the return. Benchmark activities. Check quality.

A programme statement along above lines does not need to be long. In fact, shorter statements reveal crystallised thinking. For example:

  1. Technical solutions are emerging to assist the elderly and mobility-impaired to live comfortable and independently at home. These require a significant upgrade in the electrical installation.
  2. This market needs custom solutions for target groups sharing a pathology. The programme will work at 3 levels: (1) mapping user needs through market research, partnering with consumer organisations, (2) develop training packages on ‘smart installations’ for contractors in partnership with their federation and (3) develop cost-benefit analysis for governmental organisations to provide financial support with a net benefit on social security budgets.
  3. In the short run, this programme will be measured on its intelligence gathered, and outputs produced, e.g. user research reports, training courses published, training courses delivered, cost-benefit analysis case studies, … In the medium run, the emergence of solution package will be monitored, with their market share. Copper IoU of various systems will be assessed. A final output to be monitored is government funding and soft regulation promoting this sector.

Blogged with Flock

Measuring Leonardo ENERGY programme impact – a way forward

February 26th, 2008 by · 1 Comment · Uncategorized

“Trying to parse web data was like taking a drink out of a fire hose” – Bob Ivens

The purpose of this note is to arrive at a workable method for Leonardo ENERGY to understand:- how our digital communication is working – how interacts with off line communications- what the consequences for increasing demand for copper are as a result of what we do. LE’s content is extensive and covers a wide range of subjects and is directed at many, quite varied target receivers. Ideally what we need to know is how these various contacts use the information they take from Leonardo ENERGY at various levels and what they do with that information once they get it. This graphic illustrates the flow we need to understand:
[Read more →]