Ryan Dawley 0:00:26.8:

Hey everyone, thanks for joining today. We've got Hayk Baghdasaryan here with us. I've been working with Hayk since I first came to Anaplan back in November. Hayk helped us stand up, I think, one of the most complete, connected planning instances that there is out there. A ton of value in what we want to talk about today is measuring success via ROI. Hayk, could you share with us your background and how you first came to work with Anaplan?

 

Hayk Baghdasaryan 0:00:50.8:

Yes. Hello everyone, Hayk Baghdasaryan, I'm with Anaplan for a bit more than seven years. I started my career with Anaplan in Diageo, they had huge CoE there, and then around four years ago, I was invited to help this large pet nutrition company to improve their planning and forecasting. I've tried different roles, model builder, architect, Anaplan implementation lead, service delivery lead, all related to Anaplan. When I joined the company, they were undergoing digital transformation process across all of the different parts of the business, and my responsibility was to improve how business does the planning and forecasting part. The business was overall in good shape, the volume sales was increasing, they had good market share, but they had a lot of challenges. For example, they had high volume on deal, which was also increasing. This is the amount of promo sales compared to regular sales, and there was increasing cost of goods, so the margin was declining.

 

Hayk Baghdasaryan 0:02:00.3:

The largest issue the company had was this high inventory level, and the reason of that was they had inaccurate demand forecast. Historically, there were several cases when the company didn't produce enough to meet the surge in demand, and there were some cases where they put manufacturing capacity to the wrong SKUs, and they had missed sales opportunities. Since then, they've come up with the strategy of keeping high inventory levels so that they don't miss any sales opportunities. This worked, but this is also a very expensive solution to that problem. Then demand planning also is a source for almost every other part of the business planning, and if we click to the net slide, you can see that the demand planning feeds almost every part of this honeycomb with data. Almost each P&L line depends on how much are you going to sell. It was obvious that this is the first thing we should need to fix, considering the potential cost of implementation and all of the benefits we would have with this solution, so this is where we started.

 

Ryan Dawley 0:03:19.0:

Perfect, so we're talking about those benefits and measuring those benefits, and what was the requirement from the business as it relates to measuring the potential impact of success? Was capturing ROI one of the business requirements?

 

Hayk Baghdasaryan 0:03:32.8:

Yes, so with this digital transformation, we're teaching business to make data-driven decisions, and this is how we were doing it as well. All the IT initiatives were going through assessment in terms of metrics, ROI, other things, so you have to understand what are the metrics, how do you measure them? You have to put financial part of that in place so that you can choose which initiatives go first, which is the next, and what's the impact of your potential project.

 

Ryan Dawley 0:04:07.8:

Let's talk about that a little bit, how does ROI influence your decisions, whether that be to decide which Anaplan use case to go after next, or how to prioritize Anaplan versus other projects that you may potentially be looking at?

 

Hayk Baghdasaryan 0:04:23.5:

The simple answer is that highest ROI products go first, but then you may find yourself with highest ROI product, but in order to build that product, you would need some data, which is not within Anaplan, it's coming from other sources. It may come from [?SAP 0:04:43.2] or other TPM systems. Since you have this highest ROI product but you don't have the reliable data there, you could prioritize upstream projects so that they deliver faster and they feed Anaplan with data. You could build Anaplan, it could work, but without having data and hierarchies and everything in place, Anaplan would fail without data. This is how we do it, and there is another thing, the project can have highest ROI but then there might be some strategic decisions. For example, we had cash forecasting case. This didn't have the highest ROI, but there were high currency fluctuations, and there were interest rates increasing a lot, so our CFO took the decision that the cash forecasting is essential. It's a strategic decision that we should have a visibility of our cash forecast, and that's why we started building it first. Each time is different. You rely on ROI, but you would like to make sure that you can deliver that ROI.

 

Ryan Dawley 0:05:55.1:

Do you measure ROI for all of the Anaplan use cases you're considering?

 

Hayk Baghdasaryan 0:05:59.4:

Yes, and we don't even start Anaplan implementation unless we have this business justification case. We prepare the Word document together with business, where we put all of the goals and objectives, proposed solution, roles and responsibilities, and the most important, metrics and ROI. This document describes what are we going to achieve, how, and how do we measure the result, and we prepare this together with business. This also helps everyone to be on the same page, like Anaplan team, stakeholders, sponsors, everyone speaks in the language of metrics and finance. We put this document in place, and only after we have everything in place, all of the metrics and ROI, do we start delivering the project.

 

Ryan Dawley 0:06:51.6:

I've seen these grids looking at the potential ROI, they're fascinating, and you see some really high numbers. What was the most challenging aspect of calculating Anaplan's ROI?

 

Hayk Baghdasaryan 0:07:04.8:

I would say the most challenging part is when you try to convert these metrics into real finance part. Say you have, I don't know, five per cent improvement into your forecast, but how does it affect the finance? How much money did you save, or did you gain in extra volume of sales? It's similar to advertisement, for example. You definitely know there is an effect from advertisement and promotion, but how do you calculate what is the impact? To do that, we calculate the ROI. What we do is that we come up with these metrics, we try to convert these metrics into finance. We say, for example, each percent of improvement in sales forecast accuracy allows us to decrease the inventory level by X per cent, and this way we try to come up with the finances.

 

Ryan Dawley 0:07:59.6:

What I think is unique about how you measure ROI, and we've discussed this previously, is you utilize risk avoidance as a datapoint. Can you talk a little bit about how that works, maybe give us an example of where Anaplan helped avoid a financial risk?

 

Hayk Baghdasaryan 0:08:15.6:

Yes, I'll give you my favorite example, and this was a very interesting case. We've implemented COGS Management product, this is the product which helps forecast the prime forecast, which is the forecast for raw materials and package price. The process is involving different departments. We had our buyers commercial team, who are updating the price for each component, then we have recipes coming from SAP, and this is actually a huge amount of data, considering the size of the company and the number of SKUs. You have thousands of SKUs, each has recipes, a lot of components. This way we could build very accurate Prime COGS forecast, you could see recipe price for each SKU, and then we've delivered this product just about a month later. There was a disaster in our supply chain, so we've lost a lot of supplies and a lot of raw materials. We had to find some local ones who can give you less volume but you have to find a lot of them, and then you have to keep up with the quality of product, you have to keep the nutrition of your SKUs, there are some restrictions in terms of quality.

 

Hayk Baghdasaryan 0:09:39.2:

The R&D team, they are doing these recipe changes, and they used to do it previously, just change some suppliers, some minor things and recipe, you do some optimization, but they've done this around several times a month because this is a complex project. After that, we would need to do it ten or twenty times each week because all of the suppliers are changing, you have different raw materials, which are just disappearing from your supply chain, you have to incorporate some alternatives. We've managed to build a product for this R&D team in just a month because we already had all of the necessary data and people inside Anaplan. What we did, we built several pages for this R&D team, and they could scenario plan their recipes in live. They don't have to go manually in Teams or just organize a meeting with commercial team asking, 'Okay, I want to change this component, and how much would it cost? What would the cost of the recipe? How much of this component is available on the market?' Commercial team pulls everything in Anaplan, R&D team comes and just replaces all of the SKUs in different scenarios. It compares all of the scenarios and chooses the best recipe, and that's it.

 

Hayk Baghdasaryan 0:11:04.6:

This is how we avoided this major risk, because if you don't have a certain component, your manufacturing process stops.

 

Ryan Dawley 0:11:13.1:

That makes sense. What is the highest ROI that you've calculated for your projects, and is there a rhyme or reason to where you tend to see the highest ROI within the business?

 

Hayk Baghdasaryan 0:11:25.8:

Our highest ROI comes from trade expenditure model and it's more than 2000 per cent. The number may sound crazy, but if you considered the budget, if you know all of the details, it makes sense because the company is huge and it spends around 35 per cent of gross income into customer trade investment, so the budget is massive. All of this budget is tracked and managed in literally hundreds of Excels because each customer has its own trade terms, there are Excels from older periods which you should also use to calculate these financial accruals. It's a very manual process with errors, issues, and not accurate digitalization. Even if you improve this process, the forecast of the trade expenditure, let's say by one per cent, it gives you huge value. Also, considering the internal team, we have internal team, and we implement all of the solutions with our own resources, so the cost of implementation is also low. That's how you get 2000 per cent because you gain visibility all over your trade investment. You can make better decisions earlier.

 

Hayk Baghdasaryan 0:12:45.0:

You could see for example that you have savings for one customer, and it means that you can reallocate your resources, your investment, and put it into another customer. The same for the brands, you could see that this brand has overspent on trade investment, why is it so, we should stop this and maybe reallocate resources.

 

Ryan Dawley 0:13:04.0:

Having that internal CoE is kind of the rising tide, it's going to lift all of the ROI across the board. Typically, companies from my experience, they're using ROI prospectively to develop a business case to get a project approved. I'm not seeing as much customers going back later and saying, 'Okay, what is the real ROI that we're realizing from this project?' How did you do it to track real-time ROI once the project has already gone live post-implementation?

 

Hayk Baghdasaryan 0:13:30.0:

Before the project starts, we put all of the metrics, and we try to do our best to continue measuring the benefits across the year. We do a major check-up once a year, when we just check up on our portfolio for the next year, but then we put all of the metrics in place, we try to build far better reports which are available for the whole company. We try to do it as transparently as possible so everyone can see what are the metrics, how are we improving. We also use [?Hyper-K 0:14:07.7] data to utilize information about the product health, both performance, utilization, etc., and you could see for example that the usage for certain pages dropped, and this may be the reason, because there are new people who need training, or maybe there are changes in business processes, and you need to change Anaplan as well to meet this new business process so that we can react in time.

 

Ryan Dawley 0:14:37.2:

It sounds like there's almost a positive feedback loop where you're able to measure the real ROI and that helps you have better calculations going into your projected ROI. Let me ask you, are there any other benefits or metrics that you track and measure outside of ROI?

 

Hayk Baghdasaryan 0:14:52.9:

Yes, definitely. We measure [?FTR list 0:14:55.2], FTR savings. We measure planning cycles, planning frequency, number of planning units per person, and each of them is unique for different use cases. For example, we have our innovation planning, which used to have a very long planning cycle. Whenever you start the planning process and then - but by the time you have the product on the shelf, there is a huge time gap. All of the data you have collected at the beginning of the process is already outdated, so it was important to make the planning cycle shorter. With this COGS example, the process was so laborious and it required so much involvement from a lot of people, that it was run only twice or three times a year. The full bottom-up forecast was run only twice a year, but because all of your own material prices are increasing, it was very important to start tracking this more often. There we have a metric which is called planning frequency and we managed to bring this to each assembly cycle, so basically update all of the forecasts, each assembly cycle, full bottom-up.

Hayk Baghdasaryan 0:16:07.7:

Then there is FTR list. It's important that you save time for your end users as well, because those are very smart people who are spending time on writing complex Excel formulas instead of making decisions and trying to improve the forecasting process. Another interesting metric is number of planning units per person, so our operations are getting complex, we incorporate more direct national account clients, you have more innovation products running. Either you have to hire more people, which is also challenging, or you have to make the process efficient so that one person could support more customers, more products, and longer planning horizon.

 

Ryan Dawley 0:16:53.8:

I think that closes the loop on an earlier comment. ROI is important, but it's one piece of context, it's one data point, and you've got others that you're measuring as well. Before we get to questions, last question from me would be what advice would you give to other companies looking to measure and use ROI effectively in their decision-making processes?

 

Hayk Baghdasaryan 0:17:13.8:

I would say you should definitely have metrics in place. You should understand first what's the business problem, then how do you measure the progress of improvement in that business problem, and having the metrics and ongoing tracking of those metrics. This is very important, but I would say this is not the only decision point. As we discussed, there are several other factors that the whole organization should understand what are the priorities at this moment, but other than that, ROI is the most important decision point, because you could influence also upstream process. We had a case when we didn't have the right tool and data upstream to feed into Anaplan but with a case with higher ROI, so we could influence other departments to react and support us with data.

 

Ryan Dawley 0:18:09.7:

I see that every day with the internal referrals, the internal conversations we have, you're able to bring up some of those ROI and it really captures people's attention, and they say, 'How do we do some of this? We're not seeing ROIs at that high of a level, we're very interested in moving forward.' I think it's been a real game-changer. Let's move to maybe audience Q&A at this point.

 

Hayk Baghdasaryan 0:18:29.7:

Yes.

 

Ryan Dawley 0:18:33.1:

Any questions from the audience?

 

Audience 0:18:40.6:

For companies who are not tracking their ROI right now, or are beginning to track, what metrics would you recommend to start with, or how to get started with their process formally?

 

 

Hayk Baghdasaryan 0:18:54.8:

We started right away, so we didn't start any project without having this ROI calculation, and I think this is easy to start, because whenever you have any demand, you'll ask business for metrics and for measurements of the financial effect and the implication of the use case, otherwise you don't understand what you want to achieve. We had cases when there were small initiatives which started without calculations and ROI in place, and they ended up with building some nice reports, color coding, and making some small automations, but then you don't understand where should this go, what's the real business problem we're trying to solve. I would say don't start unless you have everything in place.

 

Audience 0:19:46.3:

I'm hearing all this talk about quantifiable ROI and before we embark on a new model or doing a new project, we need to guesstimate or hypothesize what the ROI is, but everyone has a plan until they get punched in the face. Was there a scenario in which you went into a new model or something you were going to build felt really solid and the ROI or what we're resolving for, and then got punched in the face and were surprised at what actually ended up occurring? Was there anything like that where you felt really confident in what you had hypothesized but something different or unexpected happened good or bad?

 

Hayk Baghdasaryan 0:20:25.0:

Yes, we had a case, and this is where we change our strategy so we started looking not only to ROI. We had a case where we were very confident to ROI, and this was true, but we've built a project, and we've launched it, and then it turned out that we don't have reliable data in time. Each time there is an issue here, there, which makes the forecast not accurate. There is a SKU missing, there is mismatched data integrations, the data integration part fails, and you have the right tool to improve the metrics to get the ROI but you don't have the right data. This is where we started to ask ourselves the question, do I have the reliable date in time, and start the project only after that.

 

Ryan Dawley 0:21:17.9:

Also, outside of that context, I think Hayk's motto to me often has been, and uses it internally with his team, is to fail fast. You can calculate that ROI, but also to the extent that you've deployed as many use cases as the pet nutrition group had, you can really afford to get in there and fail fast or succeed.

 

Hayk Baghdasaryan 0:21:37.3:

Yes, definitely, because we had internal team and all of the work has been done with our internal resources, we could just try and fail quickly, because whenever you are onboarding a new project, you don't have to learn about specifics of the organization, you don't have to transfer knowledge on the hierarchy, etc. We have a lot of products, which were just NVP, we tried, in two months we've done small NVP, we've just saw that it doesn't give that much of a value already, so just put it away and start the new one.

 

Ryan Dawley 0:22:17.3:

Any other Mike Tyson quotes disguised as questions?

 

Audience 0:22:20.7:

Not a Mike Tyson quote. Question for you, you're talking about trade investment, when you look at your whole honeycomb, how much of it, and I lead revenue growth management, so how much of it is connected - when I think about price modelling and portfolio, optimization, and assortment, how much of it, if any of it, is connected with your trade investment?

 

Hayk Baghdasaryan 0:22:41.9:

Trade investment is connected to our demand planning tool, to TPM, and to revenue management tool. This was an enabler for the completely new process in the organization where we started looking at profit share between customer and between our own company, so this enabled us to have the visibility of that profit share on the SKU-end customer level. It also connects to, I think, COGS model, and definitely FP&A.

 

Audience 0:23:15.7:

A quick question on - our team has to solve from multiple business use cases when we're making choices on what changes we're going to make in our Anaplan forecasting model. How do you deal with trying to discreetly determine the attribution for the return when you have multiple things that could have effects on each other? It could be multiplicative, it could be subtractive. How did you deal with that?

 

Hayk Baghdasaryan 0:23:46.0:

I hope I got the question right. We tried to keep the main metrics, and we tried to not overcomplicate the measurements. Of course, there are a lot of different factors. In most cases, those are even increasing the potential income, the potential revenue, because the more Anaplan use cases you have, the easier it is to build the next one, and we actually don't try to calculate that impact, because this value of connecting tools together, it provides additional ROI, but it's very difficult to calculate. We don't go into that much detail, so we try to stick to the main business problem, measure it, and calculate the effect of that.

 

Ryan Dawley 0:24:37.8:

That is something that we've discussed, that holy grail of you've got ROI related to a use case, how do you find additional ROI when you tie in the planning, when you connect the planning together.

 

Hayk Baghdasaryan 0:24:48.9:

Yes, so for example, the case where we've built this product which allowed us to have what-if scenarios for recipe, this wouldn't have been possible if we didn't have our COGS model in place, because otherwise we would spend a year-and-a-half to build that capability for the business, but since we have already a lot of data, all of the necessary data, all of the necessary people in place, we could just put some calculations, do nice pages, build up the business process and that's it. In a month, we had a nice project.

 

Audience 0:25:29.1:

Hi. My question is about using or obtaining the PlanIQ. I want to understand how you would calculate ROI for that decision, whether your company wants to use some other models, software tools that they are already using, stay there, or acquire the PlanIQ in addition to Anaplan and say, 'Okay, what are the pros and cons of doing that?' Thank you.

 

Hayk Baghdasaryan 0:26:05.5:

I think the tool doesn't matter, because we were doing the same process for all parts of the digital initiatives, whether it's going to paperless, doing better planning, or digitalizing the manufacturing process, all parts work a similar way. You put metrics, you try to understand the financial impact on this, and when we were assessing the plan IQ and other alternative optimization and AI AML solutions, what we were doing is that we were running several months of retrospective analysis and trying to understand which model is accurate. This way we could choose the best one which performed the best. You know the impact of each percentage improvement on the finance, and you just improve that amount of improvement on the impact, and you get the final result.

 

Unknown Speaker 0:27:04.4:

Fantastic. Thank you, Hayk, for your insight and sharing your story. Our session will begin at 2:30, but when you have a chance, speak to Hayk afterwards and see how many brands he represents, and all of these brands are probably found in all of the pet stores that you frequent. I know, I have a little dog, so I got to pet shops, and I say, 'Wow, all of these are coming from our customer,' so ask him who the brands are, because they're very global, and it's kind of like we had Netflix, and now we have a major brand here that are using Anaplan for their planning. Thank you for joining us today, and please come back in a little bit for our next section.

 

Hayk Baghdasaryan 0:27:44.3:

Thanks [over speaking 0:27:44.5].

 

Ryan Dawley 0:27:45.6:

Thanks everyone.

 

Unknown Speaker 0:27:45.8:

Thank you.

 

Hayk Baghdasaryan 0:27:46.7:

Thank you.