Fronteers Spring conference

Closing Keynote: the Business Case for WPO — Kristian Sköld

This is the closing keynote after three sets of talks on Visual Performance, Accessible Performance and Technical Performance delivered on April 1, 2016 at Fronteers Spring Conference in Amsterdam

Watch high quality video on Vimeo
Download audio (MP3, 32MB)

Occasionally, the stakeholders at your client need some convincing to have you go the extra mile for performance. That means making a business case for web performance optimization. In his closing keynote, Kristian Sköld will help you do this using data-based analysis and real-life examples of his large-scale clients.

Transcript

(moderator) And our closing keynote comes from a company that does lots of performance testing, lots of performance optimization, there's a theme running through the day.

You'll spot it.

Kristian Skold works at SOASTA as a solution strategy director for the Amir region.

Is that correct? Yes.

Nailed it.

And has a talk for us which is hopefully going to help us understand how we can fight for the opportunity to do some of these things and how there is a real justification and argument for doing so.

So it's your final talk of the day.

Please make him very welcome.

Kristian Seosta-- Skold.

Thank you.

Thank you.

OK.

Cool.

First question, who thinks that web performance stuff that you learned today was useful and good? OK.

Great.

Who has been using these kind of things or similar things in projects so far trying to speed things up? A few, most of them.

That's great.

How many of you had to struggle to get the time to do it? Yeah.

And I heard that from the discussion early on, if you don't get the buy in from exec to do these kind of things, just do them anyway.

And that's absolutely correct.

Now, the problem is, when it comes to these kind of topics, they can be a bit-- if you talk to people who are not quite as technical-- they can be a bit geeky.

And I don't see the use with it.

What's the bottom line of it? I don't care if the code is fine as long as it works and as long as I get my application out there, I can make some money with it.

So I'm following a bit of Carl's thing.

I'm trying to get a bit of capitalism into this.

And not because I don't want you to be good internet citizens and do good things just for the sake of doing good things.

But I want to show something where you maybe maybe-- you might be able to actually get a better buy-in to not just do it for their own sake but actually to get a good sponsorship from most of your decision makers.

Now, before I do that, because most of the other talks have been really fast-paced and lot of condensed information-- which I really love that point.

And I like speed, especially on the internet.

But I do like to slow things down a bit.

So let me start off with a little video that's completely off topic and it buys me two minutes of not having to talk.

And it's actually something-- some of you might already know this-- it's Formula 1 racing, or what it used to be in the '50s.

[VIDEO PLAYBACK] But Holland comes in for a pit stop.

Time to refuel and change tires.

(kristian skold) A little bit of history here.

A little more-- (kristian skold) And what I love about it is sort of the high tech they're using, this-- the agility of the team itself, getting things up and running.

Holland stays in his seat, anxious to get away.

(kristian skold) When he gets a drink in between-- so that's first tire off.

[SOUND OF CARS ZOOMING BY] Yeah.

Should get the windshield cleaned as well.

[SOUND OF CARS ZOOMING BY] To be honest, I don't know.

I hope not.

So that's the second tire off.

The '50s were not quite as stressful as today's world so that's fine.

[SOUND OF CARS ZOOMING BY] Yeah.

Just bang a little bit.

The tires are changed at last.

A crew man polishes the windshield-- (kristian skold) Get the spit off and then off he goes.

---just 67 seconds after he stopped.

(kristian skold) 67 seconds.

That's great.

[END VIDEO PLAYBACK] Now, times have changed a bit.

So let's take a look at a more modern version of this.

[VIDEO PLAYBACK] Team is slightly bigger.

[SOUND OF ELECTRIC LUG WRENCH] Everyone gets ready.

[SOUND OF CARS RACING BY] Just watch carefully.

Yeah.

Don't blink.

Oh, they missed it there.

OK.

So that's four tires, [END VIDEO PLAYBACK] In just over 2.8

seconds something.

Now, why do you think there is such a big change in those last 63 years? (audience) He didn't get a drink.

That's true.

He didn't get a drink probably because he's got his drinking bag somewhere there and can sip it straight away.

But why is it so different now? (audience) MTV.

MTV Yeah.

(audience) Money.

Like that.

Exactly.

Money.

It's not because they can do it.

I mean, to be honest, the first one was pretty amusing, right? So why not do that one? It's money.

The first-- the winner of the first race got a price money of $56-- $57,000.

Now let me just cheat.

The winning team of the 2013 series got $98 million.

That's a slight change.

So the speeding up the tire change by a factor of 20, that makes sense.

And again, it's not because they want to be good citizens.

I'm not saying that Formula 1 racing is not good citizenship.

I don't see it, I don't understand it.

But it's all about the money.

[MUSIC - JESSIE J, "PRICE TAG"] Yeah, it is.

It is about the money.

And everything that you do here is also in one way or other about money.

Even if you work for-- and I-- big hoorays for that-- even if you work for an NGO, at the end, you're trying to get a service out and you need some kind of back up on that.

You need some backing money for it.

If you're working for a retailer and someone who tries to sell something, someone who tries to put news out there so they get a lot of traffic going for the ads, whatever it is, there should always be a business case behind it.

Now the question is, what is that business case? And why do you need to know about it? Well, first of all, web performance is important because speed sells.

If speed sells, revenue goes up, the cash flow goes up.

If that happens, you can invest more, the IT budget will grow eventually, and that will then eventually also pay your salary.

So if you can't connect those dots, and your team's-- not just your team but any team within your organization-- can't connect those dots, you're missing out on a huge opportunity to actually be able to do more of this fun stuff, not under the covers and be this sort of unsung hero, but actually do something that is a great value for the company and sell it to those decision makers that it is a great value.

To do that, you need to have a business case.

So what is the performance business case? What is your performance business case and what is a general performance business case? We saw some great numbers earlier from Carl.

Those are not the newest numbers, and it's sometimes not that easy to get the newest numbers.

Actually, most of those numbers that you see out there are some, well, five, seven, eight 10 years old.

Because there aren't that many studies out there.

There are some of them.

Famous examples like Wal-Mart, always quoted.

Every second they made their site faster, 2% higher conversion rate.

Now, question.

Who knows-- first question, who knows what conversion rate is? Great.

Who knows what the conversion rate of your site is? Ah.

OK.

That makes it a bit difficult to build a business case if you don't know it, right? So you know whether you make things faster but you don't know the conversation rate.

So who does know that one? Well, some other team.

You need to collaborate somehow.

Other example, Mozilla.

Different kind of use case.

Not so much the conversion of buying something, now it's conversion of downloading stuff.

But a huge gain just by making things faster.

So there are a lot of business cases out there in different studies, and there's a great summary Tammy Everts-- I'm privileged to work with-- she wrote a book.

It's coming out now I think or it has come out.

I have three copies of a pre-release which I'm happy to hand out.

I mean, I could pack three in my bag.

And actually, I'm happy to give out more of those signed by Tammy if you want to to anyone who is able to give me a good business case from their-- from their experience out of their projects they've done.

Don't have to do it publicly, can come to me later on.

Because I'm always eager to hear those apart from all the stories that I get when I work with clients.

Because it's, to me, over the last years, it's been-- refreshing is not the right word.

What's the opposite of refreshing? To talk to a lot of the IT or technical teams and they have no way of building a business case of what they're actually doing.

That's not good if you actually want to progress in your career and if you want to-- if you want to get some buy-in to do your different projects.

Now, there are also some cases where sites speed up like mad.

They've done a really good job in speeding things up.

And they try to build a business case around it.

Like, this is an example where the load time actually went up-- went down from 9.5

seconds to 5.5 seconds.

So about a 42% better performance at the point.

And the business impact was zilch.

There was no change in the user behavior.

There was no change in the number of ads they delivered.

Not statistically significant, let's put it that way.

And obviously, the whole team is like, what the-- you got to be kidding me.

We've put all this work in and now everything is faster.

All the stuff that we read in studies and we went to Velocity Conference, it's just bollocks? No.

That doesn't make sense.

Well, the problem in this case was they didn't really look at the right timer.

So they looked at DOM content loaded out of Google Analytics.

And Google Analytics documents what they put into this time or what this time includes, and you might be able to spot what the problem is.

Style sheets and images and subframes may not be finished in this timer.

Now, if you have a new site with lots of ads and iframes and style sheets and all that stuff, this timer is completely useless in regards to user behavior.

It's not a useless timer for a technical analysis.

Don't get me wrong.

All this time is that you have available in the navigation timings, and there's a whole lot of them, they have a good use case but you need to use them in the right moment.

And the one timer, or the timers that you need to build a good business case, are the ones that are user experience relevant.

Now, the DNS look up might be user experience relevant if it just fails and is really bad.

But in normal circumstances, the correlation between the DNS look up and the how fast can I move around it, it's not that significant.

Other timers like the above the fold time or the time to first, eh, whatever your site is doing-- time to first tweet, time to first-- to display the hero image, time to login screen popping up or login dialogue popping up-- those are much more important.

And you need to define which of the timers are important for your specific use case.

Like, we had a client-- again, sort of a new site-- and they figured out that the time to first scroll was an important timer for them.

So it's hard to see when you have lots of ads on there which one-- when is everything load it? They are loading ads synchronously, so is that a problem or not? Should we be doing it, should we not be doing it? How do we know if a user actually started to engaging with the content that we have? And the timer that they used was time to first scroll.

So the soon-- someone starts scrolling down, actually starts reading the article, that's a good indication to that they're starting to engage with site.

And that one had a very good correlation to the business success.

So in that case, ads delivered or session length, number of pages people click on.

So what about-- what about the business success metrics in your organization? What is it that you're trying to achieve in your-- with your application that you're working on? The code change that you did, the great animation using transitions or not, everything that you're working on, what is the actual business meaning behind it and who defines those? Well, it really depends on which part of the teams you're talking to.

And typically I would divide them into these kind of four groups.

And depending on how your organization is built, it might be merging a bit-- so you've got dev ops and all the different ones.

But let's assume we have these kind of four different building blocks-- big building blocks-- of an organization.

Line of business might be the product owner or product manager or, in German, we call them [GERMAN].

So anyone who sort of decides what the click path should be and what we should be selling and tries to optimize those kind of things.

Not from a visual point of view, just from the use case point of view.

Then also we have marketing.

Who loves marketing? Huh.

[LAUGHTER] Yeah.

Let's flip this around.

Who doesn't like marketing? I just want to check that you're still awake.

OK.

Good.

And it's-- that's bad.

Marketing is-- and not the-- the ads that get fired, but your marketing team is great.

And they're your biggest assets when it comes to this topic.

Because it's so easy to persuade marketing how important performance is if you put the right figures into it.

To do that you need to speak their language.

Don't start with, I think we switched to [INAUDIBLE] instead gzip that will help.

No.

Just don't.

But if you start using their vocabulary, if you start using the custom acquisition costs, what's your pay per click campaign budget, what is your cost per lead, how do you-- so all these different words that they use within their KPI's.

Same with line of business.

Do you know the KPI's of line of business? Of your line of business? If you don't, ask.

Because it helps you to communicate with those teams.

And if you know what language they're using, you can translate whatever you trying to achieve into their language.

That's much easier than trying to teach them your language.

Because to be honest, our tech language is much more complicated than what they do.

And once you've done that, then you can start building the blocks of winning them over to the light side from the dark side.

Marketing-- marketing campaigns, they're not cheap, especially for big brands.

They spend a lot of money on that.

Smaller brands still spend a lot of money from their budget.

And they-- very often they outsource microsites or their whole thing to agencies, you get the result back.

They pay a huge fortune on pay per click campaigns buying traffic to come onto their sites-- the Google ads, the banners, etc.

At the end, they do an analysis.

Was this campaign successful or not? And, yeah, it was pretty good.

It was really worth the investment.

I had a look at one of those campaigns.

The-- the landing site for a mobile campaign-- and I name that a mobile campaign because I found the link-- a sponsored link on Google-- while I was Googling on my iPhone.

I clicked on it.

It took a while for the site to load.

Why? 25 mg of images on a responsive site.

None compressed.

So just running that through a very simple-- not even going into the fancy and really cool stuff that Toby does, just simple compression of the images, cut off 21 megabytes.

I'm pretty sure they were very happy with the result of that campaign because a few weeks-- or two months later-- it said thank you for this great campaign and all your contribution and everything.

I thought, you don't even know what you missed out on.

Now, if you start using-- looking at those kind of projects they are running, and you start collecting the right data, you could actually back all the things up that you want to change based on a good business case.

First of all, you need to collect the data.

So how fast are we now? I do a change.

How fast am I then? What is the business metric that I want to compare it to? Let's say, conversion rates or number of downloads.

Or a very simple factor in marketing, just a bounce rate.

That's the easiest one to track and it's one that speaks very-- is easy to explain.

So if you have these kind of numbers-- let's say you've got a bounce rate-- a load time of 4.3

seconds, and that's a bounce rate of 38% with that load time.

Now, the problem is, with 4.3 seconds-- and you

communicate that to your marketing team-- the mind when you say-- when you give one number and that's the average-- the mind things of that number as a very slim kind of distribution.

That's just how we work.

So, yeah, 4.3 is pretty good.

There might be a few people who have three seconds maybe some who have five or six seconds, but we're pretty good in that one.

So why should that change if I can change the load time by 300 milliseconds, 500 milliseconds? Why should that be such a big change in the bounce rate? Because to be honest, I don't-- I don't even notice it.

Yes, I'm a bit older, but if the site loads 100 milliseconds faster, I'm not going to react differently.

So it's not about those 100 milliseconds around the average or around the median.

It's about the whole distribution of the load times.

How many of you have a site that loads just about this fast as the Melbourne 2013, so about two and-- two and a half, three seconds? Or you think it loads about that one? OK.

How many of you have a site that looks like the 1950s, 67 seconds? How many of you have no idea how fast your site loads? Because a number of hands didn't go up.

So the ones that said, you have no idea what the site loads, that's cool.

All you others, you're wrong.

You do have-- you might have a site that loads within three seconds, but your site also loads within 67 seconds.

Because it all depends on where you are as a user, what kind of connection you have, what kind of browser you have, what's happening on the site.

So even if the median is around 4.3 seconds,

the distribution is always a shape similar to this.

A lognormal distribution.

So you have a high amount of people within a, hopefully, narrow time frame.

And then you have a long tail.

And this long tail is very important to not neglect because that's where people really bail out because they're frustrated.

If you take such a data set-- and you can very easily collect that data set from your data as well-- if you take that such a data set and you correlate it with a bounce rate-- so basically like a multi-variant testing.

If someone had a session length or a load time of two seconds, how many percent of those bounced? What about five seconds, how many percent bounced? What about 10 seconds, how many bounced? And you lay this one over.

Now all of sudden, even for marketing-- I hope for you as well-- but even for marketing guys, it becomes apparent how the speed influences the user behavior.

And you can overlay this same thing with session length, conversion rates, number of downloads, whatever your business metric is that people are looking at.

Once you've done that, then it's much easier to talk about, well, if I shave this off, if I make this change, if I introduce HTP2, if I compress those images a bit more, then I will hit a lot of people that are on the far right side from you will move over.

And if they move over far enough they will not bounce anymore.

Now, this shift is not-- it's not a linear shift.

It's something that looks more like this.

So you're shifting over the mountain a bit.

The actual median low time might just shift over by 200 milliseconds or so.

But the core thing is that on the far right hand side, you are capturing many more people.

Just like with the accessibility topic.

The overall percentage is highly underestimated with-- by a lot of people who need an accessible website.

They think they can just dismiss it.

The same thing is with slow performance in a fairly fast website.

They just dismiss it.

A few years back-- and I believe it's like three years back-- the 20% of the German internet connected people still had a dial up connection.

And people don't think about that.

So try and make your marketing team, for example, or the line of business team, open up your site and throttle their bandwidth or introduce another 200 milliseconds, 500 milliseconds latency and ask them if this is still a good experience.

And then take one of your competitors site maybe, where you know it's better, or take Google where you definitely know it's better, and let them search that one.

So why are we driving around on the web with our hand brakes on? Can you just give me some time to improve this and I can show, you with the right correlation, what the actual positive business impact will be.

And so far I've never met a team that doesn't jump on this.

We had a-- we had a workshop, a client, where the IT team had never really talked a lot with the marketing team.

And we wanted to assess how we can improve the whole culture within the organization to focus more about this topic of web performance.

What we did was to interview marketing, line of business, development, and operations, and ask them what drives them, what is their problem.

And when we came to this point of their different campaigns, do you know how much money you're losing on your campaigns-- or rather, not losing on the campaigns but how much money you're missing out on in your campaigns because you still haven't agreed upon compressing those images? Your IT team has said they wanted to compress those images a long time ago, they never got the time to actually do it.

Now, if you would compress those images, your site will speed up with x% which will lower your bounce rate by y% which will increase the effect-- effectiveness of your marketing budget and overall you will probably gain about 10% of your marketing budget.

Now, part of the IT-- marketing team didn't really get it.

The main thing is, the head of marketing-- the CMO-- he got it.

And what he said in that meeting was, well, if you can save 10% of my marketing budget based on these optimizations, I'm happy to give IT half a million in funding to do it.

You could-- you should have seen the eyes of the IT head of operations on that one.

Because it's easy math as long as you take a look at the numbers.

Now, to take a look at the numbers you need to collect them.

And the beauty is with performance data, it's very easy.

There are a lot of open source ways to do it.

You can use Google Analytics.

Most of the analytics tools give you some insights into those.

You can use Open Source Boomerang to collect the data.

Then when you have the data, analyze it.

If you don't have the time to do it, and you have data analysts in your company, please become their friends because they're your best help you can get.

And typically they love this kind of stuff because it's new data set they can play around with and find correlations.

And so far we haven't found-- we have not found a business metric that doesn't have a strong correlation to the performance aspect.

Once you've done that, you collect a bit of the budgetary issues that people have and you tie it to this equation.

And you will get the buy-in.

And you will not be the guy whose [MUMBLING] he wants to make things faster again, I don't know, let's get that hero image out.

You will actually become a hero at that point because you're shifting the whole organization into a better business case.

So with that, I would say my closing notes would be-- and then I'm happy to get any kind of things thrown at me because I did introduce a very foul topic which is money.

Sorry about that.

With that just do stay artists in your-- in the things that you're doing.

Make sure that you love what you're doing.

Make sure that you try to make the world a better place with what you're doing.

Have fun with it.

Follow the best practices.

Try to sneak under the covers if you have to.

But make a conscious decision about whether you want to be the poor artist, the unknown one, or whether you want to become someone-- I'm not saying that you need to love that guy-- but someone who still creates art and was very successful with it.

If it's a career building move that you're trying to do, again, you don't have to love his art, but the concept of bringing money into the play was certainly a good career move for him.

Decide which side you want to be on.

Or whether you want to help someone else, maybe, to get onto the right side.

It doesn't need to be yourself.

It could be someone else.

So, yeah.

That's basically my short talk about, guys, look at the business cases.

Become business analysts, become business consultants, not just technical consultants.

Because then you will be able to make your company change.

The other teams in your company don't have the data or they don't have the expertise to fathom how much is possible in optimizing things.

You have.

And you need to help them to understand this.

Once you've done that, you will have a great movement in your whole organization to actually have investment in the topic that you hopefully love and that's why you came here and paid some money for it.

And hopefully you'll come back again to some other events when it comes to web performance.

Thank you.

[APPLAUSE]

Post a comment