Sensitivity analysis: what the tornado chart can and cannot tell you

Why the dollar-swing bars are hard to interpret, and six other ways to ask which uncertainties matter

← Back to the tornado chart Dollar-swing definition (docs)

The Advanced Model page ends with a tornado chart: one bar per input, showing how much the mean cost differs between draws where that input is in its top 10% and draws where it is in its bottom 10%. It is easy to read and, as a first pass, useful. It is also easy to over-read. Several of the inputs move together by construction, the 10% cut is arbitrary, the mean is pulled around by the right tail, and the question the chart answers (“what do costs look like in the worlds where this input is high?”) is not the question people usually have in mind (“what happens to costs if this input changes?”). Neither is the question that matters for research prioritization (“would learning this change what we would do?”).

This page works through those problems with live numbers and then runs the alternatives on the same scenario. Everything below is computed from the same 30,000-draw simulation the main page uses, except the Shapley table, which is precomputed offline for the default scenario, and the intervention and structure checks (Sections 3.4 and 4.1), which re-run the engine with one setting changed. The methods and text were drafted with AI assistance and have not been independently reviewed; the code is in sensitivity-methods.mjs with tests, and we welcome corrections via Hypothes.is annotations.

1. What a bar on the tornado chart is

For each input \(x\), the chart sorts the 30,000 draws by \(x\), takes the top 10% and the bottom 10% (3,000 draws each), and reports the difference in mean unit cost between the two groups:

\[\text{Swing}(x) = \overline{\text{cost}}\,\big|\,x \in \text{top 10\%} \;-\; \overline{\text{cost}}\,\big|\,x \in \text{bottom 10\%}.\]

That is all. There is no re-simulation, no holding anything else fixed, and no model of how \(x\) enters the cost equation. It is a summary of the joint distribution the simulation happened to produce.

2. Why the bars are hard to interpret

2.1 The inputs move together by construction

The chart lists inputs side by side as if each were an independent dial. In the engine they are not:

Coupling in the engine What it does to the bars
Process mode → density and media-use multiplier. Each draw first picks fed-batch, perfusion or continuous, then draws density and the multiplier from that mode’s range (fed-batch 1–6 g dry/L with multiplier 1–2; perfusion 6–30 g dry/L, 1–5; continuous 10–40 g dry/L, 0.5–3; i.e. 5–30, 30–150 and 50–200 g wet/L at 20% solids). The top density decile is almost entirely continuous and perfusion draws; the bottom decile is fed-batch. The density bar is partly a process-mode bar, and it is entangled with the multiplier bar.
Maturity → adoption, financing and equipment. One latent maturity draw shifts the hydrolysate and cheap-GF adoption probabilities, the supplemental-protein regime, WACC and the custom-reactor share. The maturity bar bundles all of those channels. It overlaps with the Media $/L, GF price, GF quantity and WACC bars, which already carry the regime effects it induces.
Regime switches inside the mixtures. “Media $/L” mixes a hydrolysate regime and a pharma-grade regime; “GF price” and “GF quantity” mix a cheap and an expensive regime, and both switch together. The GF price and GF quantity bars share the same coin flip, so they overlap heavily. Each mixture bar is a regime effect plus a within-regime effect.
Deterministic identities. Fresh media L/kg = 1000 / density × multiplier; GF cost = quantity × price. Removed from the chart for this reason, but the parents still co-determine cost in a multiplicative way.

The consequence is that a bar answers “what do costs look like in the worlds where this input is high?” and those worlds differ from the low-input worlds in other respects too. Bars cannot be added, and no bar isolates “the effect of” its input. The rank correlations among the displayed inputs, computed from the current draws, show the couplings directly:

Spearman rank correlations between the displayed inputs across the 30,000 draws. Red: move together; blue: move in opposite directions. Correlations near zero mean the engine samples the pair independently in this scenario; they say nothing about whether the pair is really independent.

2.2 Some uncertainties are probably correlated in ways the model does not encode

The couplings above are the ones the engine builds in. There are others it leaves out. Media cost per litre and achievable cell density are plausibly linked (denser cultures need richer, more expensive media, at least in fed-batch systems), but the engine samples them independently within a process mode, so the joint distribution the chart summarizes puts weight on combinations an expert might rule out — very cheap media with very high density, for example. The chart cannot detect this; it only summarizes the draws it is given. The beliefs form asks for media cost per kilogram of biomass, rather than $/L and density separately, partly so that respondents can integrate this interaction themselves. The Limits page discusses it further.

2.3 “Top 10% versus bottom 10%” and “mean cost” are choices

Nothing in the method fixes the 10% cut or the mean. Comparing the top and bottom 5% gives bigger bars (more extreme worlds); comparing the top and bottom halves gives smaller ones. The mean cost is dominated by the right tail (in the default scenario the mean is about $570 against a median of $289 per kg dry cell mass, engine 2026-11-09.1), so a bar mostly reports what happens in the expensive tail. If the question is whether cultivated chicken can get under $25/kg, the relevant statistic is a threshold probability, and that ranks the inputs differently. Try it:

Bar colour gives the sign (red: high input goes with higher cost or a lower chance of clearing the threshold; green: the opposite). Black whiskers are ±2 Monte Carlo standard errors. For the threshold statistics, green means a high value of the input goes with a higher probability of cheap cultivated meat.

2.4 Small bars are noise

Each bar is a difference between two sample means of 3,000 draws with a standard deviation around $1,090 per kg dry cell mass, so its standard error is roughly $25–50/kg. In the default scenario (engine 2026-11-09.1) the utilization, WACC and asset-life bars (about $20–30/kg, about one standard error) are noise, and the plant-capacity bar (about $53/kg, two standard errors) is at best marginally distinguishable from zero; the chart on the main page nonetheless draws and labels them. The whiskers above show the same problem for other statistics.

2.5 Two different questions: learning an input versus setting it

The chart, and every statistic in this section, conditions on the input: it compares worlds in which the input happens to be high with worlds in which it happens to be low. Under the model’s dependence structure, that is the right way to answer “if we learned this input’s value, what would we expect cost to be?”, because learning that density is high also tells you the process is probably continuous, and the conditional statistic carries that along.

It is the wrong way to answer “if this input were changed while everything else stayed as it is, what would happen to cost?”, which is what an engineer improving a process or a funder backing a technology usually means. That needs an intervention: re-run the simulation with the input pinned. When inputs are independent, the two answers coincide; here they do not (Section 3.5 shows the gap for the inputs the engine lets us pin). Neither answer is more correct in general; they are answers to different questions, and the chart only gives the first.

2.6 None of this is research value

The September review makes the point directly: a large bar can have low information value if every plausible value of the input supports the same decision, and a small bar can matter if the decision is on a knife edge. Deciding what to study requires specifying a choice (fund a medium improvement, obtain a scale-up measurement, support deployment) and asking whether plausible evidence would change it. None of the statistics on this page do that. They describe the model’s uncertainty; they do not price it.

3. Alternatives, run on this scenario

3.1 Conditional-mean profiles: where in the range the cost moves

Instead of two tails, look at all ten deciles. For each input, the red line is the mean cost within each decile of that input, the dashed blue line is the median, and the band is the 10th–90th percentile of cost within the decile. Steep, straight profiles are what the tornado bar assumes; curved profiles show that the swing comes from one end of the range; flat profiles with wide bands are inputs the bar ranks by noise.

Log scale on cost. Deciles are equal-count bins of the input’s own distribution (decile 1 is the lowest 10%, decile 10 the highest), so the horizontal axis is not linear in the input’s units.

3.2 Expected uncertainty after learning one input

This is the closest thing to the question “how much would our uncertainty shrink if we pinned this input down?” Split the draws into 30 equal-count bins of the input, measure the spread of cost within each bin, and average. The share of variance removed, \(\eta^2 = 1 - \mathbb{E}[\operatorname{Var}(\text{cost} \mid x)]/\operatorname{Var}(\text{cost})\), is the first-order Sobol index when inputs are independent; under dependence it also counts the information the input carries about correlated inputs, so the shares can add up to more than 100%. The 80% interval width is the more robust companion, because the variance of a heavy-tailed cost is driven by a few extreme draws.

3.3 Rank regression: partial adjustment for co-movement

Regress the rank of cost on the ranks of all displayed inputs at once. The standardized coefficients (SRRC) describe each input’s monotone association with cost holding the other listed inputs at their ranks, which is a partial correction for the overlap problem: an input whose association runs entirely through other listed inputs gets a coefficient near zero. The \(R^2\) says how much of the rank variation the listed inputs explain additively; the remainder is unlisted inputs (other variable costs, reactor and plant-factor draws, supplemental proteins, fixed costs) plus interactions and non-monotone effects. It is still a description of the joint sample, not an intervention.

3.4 Interventions: pin one input and re-run

For the inputs the engine exposes as ranges, we can set the input to a single value and re-simulate with the same seed, so every other draw is unchanged. The set-points are the mean of the input within the bottom and top 10% tails, which makes the intervention comparable with the chart’s conditional contrast. The difference between the two columns is the dependence structure at work.

In the default scenario the density intervention is larger than the conditional contrast. Setting density to the bottom-tail value for every draw, including perfusion draws with high media-use multipliers, produces media volumes the engine never generates on its own (low density with a fed-batch multiplier is what the conditional bottom tail contains). So the intervention breaks a coupling that is there for a physical reason, and its answer is not “the effect of density” either. When inputs are structurally linked, the honest statement is that the effect of one input is only defined relative to what is assumed about the others.

3.5 Shapley effects: a variance split that respects dependence

Sobol indices split the output variance into shares attributable to each input, but the classical decomposition requires independent inputs, which we do not have. Shapley effects (Owen 2014; Song, Nelson and Staum 2016; Iooss and Prieur 2019) are the standard answer to that problem. They borrow the Shapley value from cooperative game theory: treat each input as a player, define the “value” of any group of inputs as the variance they jointly explain, \(c(u) = \operatorname{Var}(Y) - \mathbb{E}[\operatorname{Var}(Y \mid X_u)]\), and give each input the average of its marginal contribution over all orders in which the inputs could be added. The result is a set of shares that sum to the explained variance even when inputs are dependent, with each input’s share lying between its first-order and total-effect Sobol index. So the answer to “is this relevant here?” is yes: it is precisely the method built for dependent inputs like ours.

Two practical limits. First, computing \(\mathbb{E}[\operatorname{Var}(Y \mid X_u)]\) for every subset \(u\) needs conditional distributions of the inputs, which the engine does not expose; we use the given-data estimator of Broto, Bachoc and Depecker (2020), which approximates the conditional variance by the variance of cost among each draw’s nearest neighbours in the subset’s coordinates. That estimator degrades quickly as the number of inputs grows, because neighbourhoods in many dimensions are wide: with all ten displayed inputs it returns negative shares and explains only about half the variance. It behaves well for the five inputs that carry nearly all the variance (six, including growth-factor quantity, before that quantity was tied to medium use on November 9, 2026), on log cost, with two independent subsamples agreeing closely. Second, the shares depend on how the inputs are carved up. “Media $/L including the hydrolysate regime” is one input here; split it into the regime switch and the within-regime price and the attribution changes.

Reproduce with node scripts/sensitivity-shapley.mjs, which writes sensitivity-shapley-2026-09.json with the full-sample statistics for every input alongside the runs.

3.6 What is still not done

  • Sobol indices on independent primitive inputs. The engine draws about twenty-five independent primitives (the mode-selection uniform, the Beta noise on each adoption probability, the within-regime lognormals, the reactor, plant-factor and fixed-cost draws, and so on). A classical Sobol decomposition on those would be exact and cheap with the pick-freeze scheme, but the engine would need to accept the primitives as inputs, and most primitives are not quantities anyone would set out to research (“the uniform draw that picks the process mode” is not a research target). Attributing to the displayed, aggregated inputs is more meaningful and less clean; that trade-off is inherent, not an implementation gap.
  • Multi-factor dependence. All of the above takes the single latent maturity factor as given. The structural comparison on the main page shows what independent maturity channels do to the headline numbers; rerunning this page’s statistics under that setting is one click away via the Advanced page link.
  • Value of information. The decision-relevant version of this analysis (expected value of perfect or sample information for a specified choice) needs a decision and a loss function. We have neither on the site yet.

4. What the cheap draws look like

This section was drafted with AI assistance in October 2026 and has not yet been reviewed by a person.

The sections above ask which inputs move cost. This one turns the question around: take the draws that come in under a cost threshold and see how they differ from the draws as a whole. If nearly every cheap draw has high cell density, density is something that has to be true for cheap cultivated meat in this model. Like the tornado chart, this conditions rather than intervenes: it shows what the cheap draws have in common, not what would make a real plant cheap, and it inherits every assumption the engine makes.

What the thresholds mean here. The model’s headline output is the cost per kg of dry cell mass at the factory gate, in a mix of the source studies’ dollar years. Since October 29, 2026 the thresholds apply to a kg of hybrid product: 25% cultivated cells, counted at the water content of chicken meat (27.5% solids), and 75% plant-based ingredients at $3/kg. Products will not be made entirely of cells; 25% is a convention, not a consensus figure. Because the filler alone costs $2.25 per kg of product, the hybrid thresholds sit far up the dry-cost axis: $5/kg hybrid is about $40 per kg dry cell mass (close to the old $10/kg all-cell meat-equivalent line, $36 dry), $10/kg about $113 and $25/kg about $331. A draw under $5/kg is “very cheap biomass”, not parity with chicken. Since 5 November 2026 conventional chicken is compared at its US wholesale price, the same point in the supply chain as the model’s cost: about $3.50/kg for boneless breast in 2026 dollars (range $2.6–4.6), which a hybrid product reaches only if cells cost under about $18 per kg dry (comparison with conventional chicken). At the defaults (engine 2026-11-09.1) none of the 30,000 draws gets there; the cheapest costs about $3.84 per kg of hybrid product. The cautions under the main results set this out.

The figures in this paragraph and in Section 4.1 are for engine 2026-11-09.1 (seed 42, 30,000 draws), which includes the provisional nutrient floor, recovery losses, failed runs and seed-train capital added on November 7, 2026, pending engineering review (physical consistency), and the growth-factor dose linked to medium use since November 9 (correction).

In the default scenario only 0.14% of draws (42 of 30,000) come in under $5/kg of hybrid product, too few for reliable figures; 12.0% (3,614) come in under $10 and 55.6% under $25. Compared with all draws, the under-$10 draws use less than half as much fresh medium per kilogram of dry cell mass (median 94 against 206 litres), because they have higher cell density (21 against 13 g dry/L), a lower media-use multiplier (1.33 against 1.66) and cheaper medium ($0.36 against $0.61 per litre). Most use hydrolysates (92% against 73%) and cheap recombinant growth factors (76% against 48%). Plant capacity, utilization and the cost of capital barely differ between the two groups. The cheap draws are cheap because medium and growth factors become small, not because of scale or financing. Their mean cost is $86 per kg dry, of which medium is $42, and capital and fixed operating costs ($11 and $18) make up about a third. The 42 under-$5 draws show the same pattern more sharply: just over a quarter of the typical medium use (57 against 206 litres), nearly double the density, all on hydrolysates, and capital plus fixed costs about two fifths of a $35 mean. (Engine 2026-11-07.2, before the growth-factor correction: 41 draws under $5, 11.6% under $10 and 58.2% under $25.)

Before November 7 (engine 2026-10-29.1) 1.0% of draws were under $5, 23.6% under $10 and 67.2% under $25; the under-$5 draws then used about a fifth of the typical medium. The nutrient floor removed most of those very low-medium draws.

4.1 Structure check: how much depends on the medium rule

The model’s default derives fresh medium per kilogram of cells from density: litres per kg = 1000 / density × media-use multiplier. So any draw with high density automatically uses little medium (since November 7, 2026, no less than the nutrient floor allows), and since medium is the largest cost component, high density lowers cost twice, once through reactor size and once through medium. Two independent reviews dispute the medium half of that. The September scientific review and an outside audit (Report 5 in the review response) both argue that medium per kg is set by how much of each nutrient the cells consume per kg of growth (nutrient yield), which is roughly the same whatever density the culture reaches; concentrating a culture does not undo the medium it has already used. Under that view, density mainly sets reactor size.

The Advanced page has an alternative rule that draws fresh medium per kg directly (default range 40–300 litres per kg dry cell mass, 5th–95th percentile, i.e. 8–60 per wet kg at 20% solids; an illustrative stress range rather than a sourced estimate) and leaves density to set reactor volume and capital cost only. A second alternative (October 2026) derives medium per kg from nutrient yield, the grams of dry cell mass the nutrients in a litre of medium support: litres per kg dry = 1000 / yield (default yield range 3–30 g dry cells per litre, 5th–95th percentile, 3–25 before November 7, 2026; about 33–330 litres per kg dry). Since November 7 the default density rule also uses this yield as a floor (nutrient_floor), so row (a) is the density rule with the floor. Density then affects only reactor size. The table runs the current scenario under each rule with the same seed, so every other input is identical draw for draw. Row (c) also adds forming, $0.5–2.5 per kg of hybrid product since November 7, 2026 (an unsourced $2–15 per kg of cells before), which is outside the cell-mass estimate and which the default leaves out; it changes the hybrid-product thresholds but not the cost per kg dry.

Engine 2026-11-09.1, default scenario, thresholds on the hybrid product. Since the nutrient floor, the default rule is the most expensive of the three: it can never use less medium than the yield rule, and it keeps the density rule’s heavy high-use tail (median 206 litres per kg dry and 90th percentile 647, against 109 and 239 under the direct rule and 106 and 261 under the yield rule). So the alternatives now give more cheap draws, not fewer: 0.35% under $5/kg and 23.3% under $10 for the direct rule, 0.48% and 25.0% for the yield rule, against 0.14% and 12.0% by default; their medians are about $184 and $182 per kg dry against $289. Holding every other input fixed, 28 of the 42 default under-$5 draws and 1,164 of the 3,614 under-$10 draws (32%) rise above the threshold under the direct rule, and none do under the yield rule. Adding forming ($0.5–2.5 per kg of hybrid product) cuts the under-$10 share to 15.2% with the direct rule and to 7.3% with the default rule, and leaves almost no draws under $5. Because the growth-factor dose now follows medium use, every rule’s medium figure also moves its growth-factor cost.

Before the floor (engine 2026-10-29.1), the picture was the reverse: the very cheap tail rested mainly on the assumption that dense cultures use proportionally less medium (247 of the 306 under-$5 draws rose above $5 under the direct rule). The floor removed most of that dependence. What remains is the question of how much medium low-density, high-multiplier processes need, which drives the default’s expensive tail. Anyone quoting a threshold share should still report it alongside the direct-rule or yield-rule figure. These figures should be rerun after engine changes.

Before October 26, 2026 these thresholds applied to wet biomass, where 2.7% of draws were under $10 per wet kg and 27.7% under $25; those figures are reproduced exactly by the wet cost basis. From October 26 to 29 they applied to an all-cell chicken-meat equivalent (dry × 27.5%), where 0.7% of draws were under $10 and 15.8% under $25.

5. Summary

Method Question it answers Handles the engine’s dependence? Unit Where
Tornado (tail contrast, mean) In worlds where \(x\) is high, how much higher is mean cost than in worlds where it is low? Describes the joint sample; bars overlap and cannot be added $/kg Main page; Section 2.3 with other tails and statistics
Threshold contrast Same, for P(cost below a target) Same percentage points Section 2.3
Conditional profiles How does cost change across the whole range of \(x\)? Same $/kg by decile Section 3.1
Expected uncertainty after learning \(x\) (η², interval width) If we learned \(x\), how much narrower would the cost distribution be? Yes, in the “learning” sense: counts information carried by correlated inputs share; $/kg Section 3.2
Rank regression (SRRC) Monotone association with cost, adjusting for the other listed inputs Partially, among listed inputs SD units Section 3.3
Intervention contrast If \(x\) were set to a value, everything else unchanged, what would cost be? Breaks the dependence by design; only for pinnable inputs $/kg Section 3.4
Shapley effects How should the variance be split among inputs so the shares add up? Yes, by construction; estimator is the weak point share of variance Section 3.5 (default scenario, offline)
Threshold profile What do the draws under $10, $25 or $50/kg have in common? Describes the joint sample, conditioned on the outcome input medians, shares, $/kg Section 4
Medium-rule structure check How much of the cheap tail rests on density setting medium use? Compares model structures with paired draws probability, $/kg Section 4.1
Sobol on primitives, value of information Exact variance split; decision relevance — — Not implemented

The practical reading for this model, in the default scenario (engine 2026-11-09.1): cell density, growth-factor price and media $/L dominate every measure; the media-use multiplier comes next (growth-factor quantity, a separate input until November 9, now follows medium use, so its effect is carried by the density and multiplier bars); and the financing, scale, timing and utilization inputs explain almost none of the variance (η² under 1%). With 30,000 draws, production time, and at the cost thresholds plant capacity and WACC, show small swings that are distinguishable from zero, but they are an order of magnitude below the leading inputs. The methods disagree about the ordering and size of the middle group, which is the honest result: the middle group’s ranking depends on which question you ask.

Estimators: sensitivity-methods.mjs; tests: tests/sensitivity-methods.test.mjs; offline run: scripts/sensitivity-shapley.mjs. References: Owen, A. B. (2014) “Sobol’ indices and Shapley value,” SIAM/ASA J. Uncertainty Quantification; Song, E., Nelson, B. L. and Staum, J. (2016) “Shapley effects for global sensitivity analysis,” SIAM/ASA JUQ; Iooss, B. and Prieur, C. (2019) “Shapley effects for sensitivity analysis with correlated inputs,” Int. J. Uncertainty Quantification; Broto, B., Bachoc, F. and Depecker, M. (2020) “Variance reduction for estimation of Shapley effects and adaptation to unknown input distribution,” SIAM/ASA JUQ; Saltelli et al. (2008) Global Sensitivity Analysis: The Primer.