Stress testing and the insurance business cycle

In the second part of this InsuranceERM roundtable held in association with QRM, risk and actuarial executives discuss how they deal with stress testing requests and results, incorporating it into the business cycle, and their frustrations and expectations regarding stress testing

Responding to requests

Charles Richard: One problem with any scenario analysis or stress testing analysis is, as soon as you run one scenario, someone either likes it or says, 'Can you just change these two things and come back and give me an answer tomorrow?'  If that takes nine months, it becomes not very useful.

Perry Thomas: Yes, but then you also provide some sensitivities, and you can just tweak it that little bit.

Richard Charlton: That is where it becomes a bit of an art form. It's possible to do 'rough' scenario analyses off the back of your suite of hard-coded scenario and sensitivity runs. You can tinker around the edges, without running the full models again.

Chris Cundy: How do you respond to ad-hoc requests for stress testing?

Peter Telford: There is a trade-off between speed and accuracy, as in so many things. If you ask me a question, by tomorrow I can probably tell you whether the answer is positive or negative. A week from now, I can give you a ballpark figure. But if you want it down to the nearest percentage point, then it is going to take longer. 

Actually, methods typically evolve the opposite way; you start by building the 'Death Star' model, highly engineered to give you a brilliant answer in three months, but then the business finds that it needs answers quicker than that for some purposes, and so you devise parallel estimation techniques.

Charles Richard: It relates to your approach, as well. If you have a bottom-up approach, it is going to be a much bigger process than if you generated a top-down approach.

Jason Domoney: With a multinational business, you tend to have to do a lot more bottom-up, and it does take a long time to get an answer.

Proxy models

Vinaya Sharma: When there is a request for stress testing, it sounds like in most companies it goes back to line one, they do their work and it comes back to line two for review and analysis. What we are seeing in some companies in the US is that line two has their own model that substantially represents the entire organisation, and can provide quick, ballpark answers. Are you seeing that in your firms?

Perry Thomas: Yes. We all have our spreadsheets.

Peter Shepherd: From a stress test point of view, the difficult bit is defining and calibrating the model; the running is relatively straightforward. We report most of our stresses every week.

Richard Charlton: We have developed daily or weekly solvency monitoring tools, which are more approximate by their nature. We might start to use some of those tools for 'quick and dirty' stress and scenario testing, as they become better developed.

Communicating stress tests with the board

Chris Cundy: What are the issues around communicating a stress test to different groups of people?

Richard Charlton: The biggest challenge is the communication with the board, to make sure it is sufficiently engaged and understand the risks of the business. The expectation on the board has certainly increased over the past year or two and that has caused people from a variety of backgrounds – not necessarily who have spent their careers in insurance – to start questioning some of the assumptions and scenarios. It means you get a much more diverse set of questions and challenges than you would normally get.

Jason Domoney: We have found that as well; we have clinical doctors on our boards, and they will ask different questions from the accountants and actuaries. It does make you think about the totality of your scenarios and whether you are covering all your risks and whether they are representative of the business.

Charles Richard: That is one of the beauties of scenario-based stress testing. If you have a board that is made up of diverse backgrounds like medical doctors, for example, how do you explain TVaR or 95% quantile to them? Stress testing can bring a greater level of understanding amongst people who are not experienced in our business.

Peter Shepherd: Yes, and it helps to articulate it in terms of things that we think might happen in the next five to ten years, and things we think are very unlikely to happen, rather than a '1-in-10' or a '1-in-200' year event.

Peter Telford: The narrative is also very valuable not just for the board to buy into, but because you have to get from a narrative to a set of parameters, and then there is a further stage where the people running the model almost certainly have to add fine detail to those parameters to get the model to behave. If at each stage you can go back to the narrative, you stand a much better chance of getting the detail consistent with what your intentions were.

Peter Shepherd: If you can do it in the context of a business plan, such as via the ORSA, all the better. If a board member looks at the plans and says, 'That is a great plan,' and then you say, 'Actually, a reasonable outcome will look like this,' that gets their attention a great deal more.

Richard Charlton: Board members also tend to like to see some back testing relative to what you are churning out from the model. There is an element of, 'Okay, you have gone away and built various statistical distributions, but how does it relate to the real world?' This prompts very useful sense checks.

Peter Shepherd: Around the internal model development, the board was interested when we looked back at the prior year and ran that through the model, telling everybody on each individual module what probability of that outcome was. We had quite a wide range, and that is one of the potential shortcomings of these models, is that they can be calibrated very much around extreme scenarios and can potentially be less useful at the lower percentiles.

The business planning cycle

Chris Cundy: How does stress testing fit within your business planning cycle?

Peter Shepherd: The ORSA is probably the biggest element for us in terms of looking at the business plan and stressing it in different ways, under an economic model or the standard formula.

Richard Charlton: There is an interesting question in terms of timing. Our ORSA report typically gets signed off by the board in November time, but stress and scenario testing around the business plan might be done in January/February time.

Perry Thomas: We are taking some steps to make that a more continuous process. There is something on stress testing in every board meeting, I am sure.

Frustrations with stress testing

Chris Cundy: Overall, what would you say your biggest frustrations are with the process of stress testing a business plan? 

David Leach: The number of different iterations of the business plan; that always seems to be the challenge for many firms. You can iterate through tens of different versions of the plan, which does not seem particularly reasonable.

Peter Telford: At some point, you have to let the shutter drop on the risk function assessing the business plan. If the numbers are subsequently tweaked, then you can sweep that up in the next ORSA report – unless the tweak is such as to change your conclusions, in which case, you have to redo that assessment anyway.

Richard Charlton: For me, it is getting the timings of the stress and scenario testing across the year to tie up with the decision-making across the business, when the business plans are being locked down.

Peter Shepherd: I am working on a framework for more regularly assessing what our stress tests and scenarios are, and the triggers and points at which we should check those.

Karina Lo Dico: Being a small firm, we are earlier in the development cycle and still building the models to meet the Solvency II and Canadian requirements. Trying to apply stresses to these models is cumbersome right now.

Jason Domoney: Our biggest challenge is around prioritising the business plan, versus stress testing the business plan. The same group of people do everything and they will tend not to do the stress testing until they are comfortable the plan is close to final.

Future expectations

Chris Cundy: How do you expect stress testing to evolve?

Peter Shepherd: One thing that feels different to me, coming from a banking environment, is the level of prescription around stress testing and scenario running is lower in the insurance environment. I would expect that the insurance regulators are going require more specific stress testing over time.

Perry Thomas: Insurance stress testing might be less prescriptive, but the flipside is a bank would never go into the level of detail that we do. You get into assumptions that the banks just do not deal with.

David Leach: Internal models have got so complex, but stress and scenario testing is a way of cutting through some of that complexity and helping the regulator as much as the board to understand how things are working. I would be surprised if there was not a tendency to do more stress and scenario testing.

Richard Charlton: I expect there will be requirements to do more over the next few years.  Hopefully it will reach a plateau, but doing more does not mean necessarily using up a lot more resources. You need to get better at having approximate tools, so we do not double the number of scenarios and double the amount of effort to do them.

Part one of this roundtable can be read here.