Understanding Periodic Research Impact Analysis

Overview

Factbase measures research quality across multiple time periods to reveal trends, momentum, and sustained excellence. This article focuses specifically on research publication impact metrics. Factbase also incorporates other intelligence streams (actors, assets, and strategic capability indicators) which use different temporal methodologies.

Understanding how we calculate and interpret periodic research impact is essential for accurate analysis of national and institutional research capability.

What is Periodic Research Impact Analysis?

Periodic research impact analysis assesses research quality over defined time windows (3-year, 5-year, 10-year, and 20-year periods) to answer strategic questions:

• Is research quality improving or declining?
• Does an entity show recent momentum or rely on historical reputation?
• Is excellence sustained over multiple research cycles or episodic?
• How does current performance compare to generational contribution?

The key insight: A single time window only shows performance at one point. Multiple windows reveal trajectory.

Time Windows Explained

Standard Periods

Factbase calculates research impact across four standard time windows:

Year-to-Date (YTD) Inclusion

Current year data is included in all periods but flagged for recency:

• Papers from 2025 have limited citation accumulation
• TMCM calculations account for this through year-specific normalization
• YTD data enables real-time monitoring but is less stable than complete years

Example: A query run in June 2025 analyzing the 5Y period includes:

• Complete years: 2020, 2021, 2022, 2023, 2024
• Partial year: January–June 2025

Calculation Methodology: Annual TMCM Averaging

The Core Principle

Factbase calculates research quality by averaging annual TMCM values, not by aggregating all papers across the period.

This is a critical methodological distinction that affects interpretation.

How It Works

Step 1: Calculate TMCM for each year individually

For each year in the period:

Year_TMCM = (Sum of paper TMCMs × fractional credits) ÷ (Sum of fractional credits)

**Example – Country X in Artificial Intelligence**:

| Year | Fractional papers | Total citations | Median citations | Annual TMCM |
|------|-------------------|-----------------|------------------|-------------|
| 2020 | 247.3 | 8,450 | 8 | 3.2× |
| 2021 | 289.6 | 9,120 | 9 | 3.0× |
| 2022 | 312.8 | 10,850 | 7 | 3.4× |
| 2023 | 356.2 | 11,200 | 6 | 3.6× |
| 2024 | 398.5 | 8,920 | 4 | 2.8× |

**Step 2**: Average the annual TMCMs

Period_TMCM = (Sum of annual TMCMs) ÷ (Number of years)

5Y TMCM = (3.2 + 3.0 + 3.4 + 3.6 + 2.8) ÷ 5 = 3.2×

**Interpretation**: On average across 2020–2024, Country X's AI research received **3.2× the median citations** for papers in the same topic and year.

---

## Why Annual Averaging Matters

### What Annual TMCM Averaging Measures

✅ **Average quality performance** across the period

✅ **Year-normalized comparison** (fair despite citation accumulation differences)

✅ **Consistent quality** or variability over time

✅ **Trend direction** (improving, stable, or declining)

### What It Does NOT Measure

❌ **Total volume** of papers across the period (use fractional paper count for this)

❌ **Total citations** accumulated across the period (use citation count for this)

❌ **Aggregate impact** of all papers combined (use citation share for this)

### Why Not Just Count All Papers?

**Alternative (NOT used)**: Aggregate all papers across 2020–2024 into one large pool and calculate TMCM

**Problem with aggregation**:

Papers from 2020: 5 years of citation accumulation Papers from 2024: 1 year of citation accumulation

Even with year-normalized TMCM, older years dominate aggregate statistics because they have more papers AND more mature citation patterns.

**Annual averaging solves this**:
- Each year gets equal weight regardless of paper volume
- 2020 and 2024 both contribute one annual TMCM value
- Fair representation of quality across the entire period

---

## Interpreting Multi-Period Analysis

### Reading Trend Patterns

**Example – Country X in Quantum Computing**:

| Window | Period | TMCM | Interpretation |
|--------|--------|------|----------------|
| 3Y | 2022-2024, 2025 YTD | 3.8× | Recent performance |
| 5Y | 2020-2024, 2025 YTD | 3.4× | Current cycle |
| 10Y | 2015-2024, 2025 YTD | 3.1× | Sustained capability |
| 20Y | 2005-2024, 2025 YTD | 2.4× | Generational contribution |

**Analysis**:

**Trend: Improving quality** (2.4× → 3.1× → 3.4× → 3.8×)
- Recent research (3Y) significantly stronger than historical average (20Y)
- Sustained improvement across all windows
- Momentum building in recent years

**What this tells you**:
- ✅ Country X is **accelerating** in quantum computing research quality
- ✅ Recent investments/programs appear to be paying off
- ✅ Not just resting on historical reputation

**What this does NOT tell you**:
- ❌ Whether paper volume is increasing or decreasing (need separate volume metrics)
- ❌ Whether Country X leads globally (need comparative rankings)
- ❌ Whether citations are international or domestic (need citation diversity analysis)

### Alternative Pattern: Declining Quality

**Example – Country Y in Hypersonics**:

| Window | TMCM |
|--------|------|
| 3Y | 2.1× |
| 5Y | 2.5× |
| 10Y | 2.9× |
| 20Y | 3.2× |

**Analysis**:

**Trend: Declining quality** (3.2× → 2.9× → 2.5× → 2.1×)
- Historical strength (20Y) not maintained in recent years
- Consistent decline across all windows
- Recent performance well below historical average

**Possible interpretations**:
- Research programme maturity (diminishing returns)
- Brain drain or talent migration
- Funding reductions or strategic de-prioritization
- Increased global competition raising the bar

### Stable Performance Pattern

**Example – Country Z in Biotechnology**:

| Window | TMCM |
|--------|------|
| 3Y | 2.7× |
| 5Y | 2.8× |
| 10Y | 2.7× |
| 20Y | 2.6× |

**Analysis**:

**Trend: Stable excellence** (consistent 2.6-2.8× across all periods)
- Sustained high quality over two decades
- No significant improvement or decline
- Reliable, consistent research capability

---

## Common Misinterpretations to Avoid

### ❌ Mistake 1: Treating Period TMCM as Aggregate Citations

**Wrong interpretation**:
> "Country X has 5Y TMCM of 3.0×, therefore its 5,000 papers collectively received 15,000 citations (3.0 × 5,000)"

**Correct interpretation**:
> "Country X's papers from 2020-2024 received, on average across those years, 3.0× the median citations for papers in the same topic and year"

**Why it matters**: TMCM is a **quality metric**, not a **citation multiplier** for volume calculations.

### ❌ Mistake 2: Assuming More Papers = Higher TMCM

**Wrong assumption**:
> "Country X published 10,000 papers (10Y) vs 5,000 papers (5Y), so 10Y TMCM should be higher"

**Reality**: 
- TMCM measures **average quality per paper**, not total volume
- More papers with lower average quality can decrease TMCM
- Volume and quality are independent dimensions

**Example**:

Country X 5Y: 5,000 papers, TMCM = 3.5× (high selectivity) Country X 10Y: 10,000 papers, TMCM = 3.0× (includes earlier, lower-quality work)

Interpretation: Recent 5 years are higher quality than the full 10-year period

### ❌ Mistake 3: Comparing Periods from Different Entities Without Volume Context

**Incomplete analysis**:
> "Country A has 5Y TMCM = 4.0×, Country B has 5Y TMCM = 2.0×, therefore Country A has twice the capability"

**Complete analysis**:

Country A: 5Y TMCM = 4.0×, Paper share = 2% (high quality, low volume) Country B: 5Y TMCM = 2.0×, Paper share = 35% (good quality, massive volume)

Interpretation: Country B likely has greater overall capability despite lower TMCM because it combines scale with above-median quality.

**Always combine TMCM with volume metrics** (paper count, paper share, citation share).

### ❌ Mistake 4: Ignoring Year-to-Date Volatility

**Premature conclusion**:
> "Country X's 3Y TMCM dropped from 3.8× (Q1 2025) to 2.9× (Q2 2025), their quality is declining!"

**Reality**:
- Q2 2025 includes more 2025 YTD papers with limited citations
- Annual TMCM for incomplete years is volatile
- Wait for complete year before drawing conclusions

**Best practice**: Compare complete-year periods (e.g., 2020-2023 vs 2021-2024) for stable trend analysis.

---

## Combining Period Analysis with Other Metrics

### Complete Research Profile

**Periodic TMCM** should be analyzed alongside:

**Volume metrics**:
- Fractional paper count (how much research output?)
- Paper share (what % of global output?)
- Growth rate (increasing or decreasing production?)

**Impact metrics**:
- Citation count (total influence?)
- Citation share (what % of global citations?)
- Top 1%/10% papers (breakthrough research?)

**Quality metrics**:
- Median TMCM (typical paper quality?)
- TMCM distribution (consistent or variable?)
- TMCM-Int (international recognition vs self-citation?)

**Example – Complete 5Y Profile**:

COUNTRY X – ARTIFICIAL INTELLIGENCE – 5Y (2020-2024, 2025 YTD)

VOLUME Fractional papers: 2,847.3 Paper share: 33.7% (rank #1 globally) Annual growth: +8.2% per year

QUALITY (PERIODIC TMCM) 5Y TMCM: 3.4× (rank #3 globally) 3Y TMCM: 3.8× (improving trend) 10Y TMCM: 3.1× (sustained excellence)

IMPACT Total citations: 94,520 Citation share: 42.1% (rank #1 globally) Top 1% papers: 8.2% of output

TRAJECTORY Improving quality (3Y > 5Y > 10Y) Increasing volume (+8.2% annual growth) Sustained excellence over decade

ASSESSMENT: Dominant leader with accelerating momentum

---

## Technical Specifications

### Annual TMCM Calculation

For each year Y in period P:

1. For each paper published in year Y: Paper_TMCM = (Citations to paper) ÷ (Median citations for topic-year)
2. For entity E (country/institution): Year_TMCM[Y] = Σ(Paper_TMCM × Fractional_credit) ÷ Σ(Fractional_credit)
3. Period TMCM: Period_TMCM[P] = (Σ Year_TMCM for all Y in P) ÷ (Number of years in P)

### Handling Incomplete Years

**Current year (YTD)**:
- Included in all period calculations
- Flagged with "YTD" designation
- Lower citation counts expected (normalized by year-specific median)
- May cause volatility in 3Y window

**Missing years**:
- If entity has zero papers in a year, that year excluded from average
- Prevents artificial deflation from zero-paper years

### Update Frequency

**Annual recalculation**:
- As calendar year completes, YTD becomes complete year
- Medians recalculated with full year's data
- Period definitions shift forward (e.g., 5Y becomes 2021-2025 instead of 2020-2024)

**Quarterly updates**:
- YTD data refreshed
- Citations updated for all papers
- Annual TMCMs recalculated with latest citation counts

---

## Frequently Asked Questions

### Q: Why use annual averaging instead of weighted averaging by paper count?

**A**: Annual averaging gives **equal weight to each year** regardless of publication volume.

**Alternative approach** (weighted):

Weighted TMCM = Σ(Year_TMCM × Papers_in_year) ÷ Σ(Papers_in_year)

**Problem**: Years with more papers dominate the metric, obscuring quality trends.

**Example**:

2020: 100 papers, TMCM = 2.0× 2024: 500 papers, TMCM = 4.0×

Annual average: (2.0 + 4.0) ÷ 2 = 3.0× (shows improvement) Weighted average: (2.0×100 + 4.0×500) ÷ 600 = 3.7× (over-emphasizes recent volume)

Annual averaging reveals **quality trajectory** more clearly.

### Q: How do I compare periodic TMCM across different window sizes?

**A**: Different windows measure different things:

- **3Y > 5Y > 10Y** → Quality improving (recent work stronger)
- **3Y < 5Y < 10Y** → Quality declining (historical work stronger)
- **3Y ≈ 5Y ≈ 10Y** → Stable quality (consistent performance)

**Don't directly compare values** (e.g., "3Y TMCM is 3.8 but 10Y is 3.1, which is the 'right' one?")

**Both are correct** – they measure different things:
- 3Y = current momentum
- 10Y = sustained capability

### Q: Can periodic TMCM go down even if quality is improving?

**A**: Yes, if **older high-quality years roll out of the window**.

**Example**:

2015: Exceptional year, TMCM = 5.0× 2016-2024: Good years, TMCM = 3.0× average 2025: Good year, TMCM = 3.0×

10Y TMCM (2015-2024): includes exceptional 2015 = higher average 10Y TMCM (2016-2025): excludes 2015 = lower average

Quality didn't decline – the exceptional year aged out.

**Watch for**: Step changes when exceptional years enter/exit windows.

### Q: How does YTD affect period calculations?

**A**: YTD years have:
- Fewer papers (incomplete publication year)
- Lower citation counts (less time to accumulate)
- Year-normalized TMCM (compared to YTD median, not full-year median)

**Effect on periods**:
- **3Y**: Most affected (YTD is 1 of 3-4 years)
- **5Y**: Moderate effect (YTD is 1 of 5-6 years)
- **10Y/20Y**: Minimal effect (YTD is small fraction)

**Best practice**: Monitor trends using complete years (e.g., compare 2019-2023 to 2020-2024).

---

## Summary

**Periodic research impact analysis** measures research quality across multiple time windows (3Y, 5Y, 10Y, 20Y) by **averaging annual TMCM values**.

**Key principles**:
- **Annual TMCM averaging** ensures fair comparison across years with different citation accumulation times
- **Multiple windows reveal trajectory** – improving, stable, or declining quality
- **Not a volume measure** – TMCM shows quality per paper, not total output or citations
- **Combine with volume metrics** – paper count, paper share, and citation share for complete picture

**Interpretation guidance**:
- 3Y → Recent momentum
- 5Y → Current capability (standard assessment)
- 10Y → Sustained excellence
- 20Y → Generational authority

**This is one component of Factbase's research intelligence.** For comprehensive capability assessment, combine periodic impact analysis with volume trends, excellence indicators, and non-research intelligence (actors, assets, strategic indicators).

---

*For technical specifications on TMCM calculation, see the TMCM methodology documentation.*

*For information on other Factbase temporal analysis methods (actor trajectories, asset timelines), see the relevant documentation sections.*