One research method produces comparable answers across companies and periods

Comparing companies is only useful when the same question means the same thing each time.

An investment research team may evaluate several companies, factors and periods in one study. If the method changes from one cell to the next, the final comparison can look precise while measuring different things. The team needed a way to lock the method before the research expanded across the full comparison.

We delivered a research workspace that applies a defined method across a company-by-factor-by-period matrix. Peer comparison and backtesting use explicit aggregation rules. The outcome is a set of answers that can be compared on a common basis and reviewed when the team needs to understand how a result was produced.

Client background

Our client is a large institutional investor with an investment research team that combines qualitative judgement and quantitative comparison. The team needs flexibility to ask meaningful questions, but consistency to know whether the answer changes because the company changed or because the method changed.

The workspace supports live frontier research. It is designed around the team's research lifecycle, from creating a factor and defining the question to validating the result and publishing a report.

Business challenge

A research matrix creates a hidden consistency risk. The more companies, factors and dates involved, the easier it is for a source rule, interpretation or calculation to drift. A later reviewer then has to ask whether the results are truly comparable.

The team needed to control several points:

• the research question and method had to be explicit before execution;
• specialist work had to stay inside the approved plan;
• factor definitions had to be managed through creation, revision and validation;
• peer comparison and backtesting had to follow known aggregation rules;
• and the evidence behind every cell had to remain available for review.

The requirement was not a single impressive result. It was a process that remained coherent as the analysis expanded.

Implementation

We created a lead analyst role that owns the question and method. Bounded specialist roles complete the research tasks required by that plan. The plan can be revised deliberately, but the system keeps the revision in the execution record instead of silently replacing the original method.

The workspace evaluates the full company-by-factor-by-period matrix against the method that was defined. It manages the lifecycle of factors, so a change to a factor can be reviewed as a change in the research design rather than appearing as an unexplained difference in the output.

Peer comparison and backtesting are aggregated deterministically. That means the combination rule is explicit and repeatable. A model may help complete bounded research tasks, but it does not decide how the final values should be combined merely because its prose sounds persuasive.

Each execution retains sources, evidence, provenance, methodology, model configuration and caveats. The team can therefore return to a result and inspect the conditions under which it was produced.

Why it was difficult

Consistency cannot be achieved by flattening every research question into one template. The system had to provide a stable method while preserving the judgement needed to define factors and interpret evidence.

The second challenge was scale inside the workflow. A matrix can become large enough that manual checking is unrealistic, but automation without a locked method only makes inconsistency faster. The workspace had to make the method the control point.

Value delivered

• One locked methodology can be applied across companies, factors and periods.
• Factor creation, revision and validation have a managed lifecycle.
• Peer comparison and backtesting follow explicit, deterministic aggregation rules.
• Specialist research stays inside the plan set by the lead analyst role.
• Evidence and provenance remain available for each completed result.
• The team can inspect the method before relying on the comparison.

The result is comparability with a visible basis. When a company or period looks different, the team has a better chance of determining whether the difference comes from the evidence or from an unrecorded change in method.

What this means for your business

If several analysts, teams or models produce answers that need to be compared, consistency should be designed into the workflow. Lock the question, define the method, record revisions and make the aggregation rule explicit before the final report is written.

This applies to credit analysis, portfolio review, operational benchmarking and any evidence-based process where a comparable answer matters more than a fast paragraph.

Related: Applied research · Data and analytics · IFAs and wealth managers