Theory of Change (Full)
Our complete theory of change showing how The Unjournal's tools, incentives, and platforms lead to better research evaluation,
improved impactful research, better-informed decision makers, and ultimately positive outcomes for global priorities.
flowchart TD
classDef Direct fill:#faefaf,stroke:#b8a000
classDef Multi fill:#bdf2c9,stroke:#2d8a4e
classDef Integrate fill:#99bdf0,stroke:#2563eb
classDef Systemic fill:#f0b6c4,stroke:#be185d
A0["UJ tools, incentives, platforms, emphasis"]:::Multi --enable--> A["More efficient rsch. feedback/eval."]:::Multi
A0 --"we will"--- FocusEA["Focus on impactful rsch."]:::Direct --> EAR
A --> EAR["+ evaluation of EA-aligned rsch."]:::Direct
EAR --> EARig["+ Rigor of EA-aligned rsch."]:::Direct
EARig --> A1["Impactful research improved"]:::Multi
EAR --- OF["+ 'Open science'"]:::Direct --> EARO["More replicable, reliable"]:::Direct --> A1
A0 --> PayAcad["Academics paid to eval. impactful rsch."]:::Integrate --> AcadAware["+ Awareness/interest in impactful rsch."]:::Integrate --> D1["+ Acad. focus on global priorities"]:::Integrate
PayAcad --> A1
EARig --"Stronger work"--> AcadAware
A --> D["UJ Attractive to researchers"]
A --> A1
A --> A2["Decisionmakers better informed"]
A0 --"Building"--> B["Reliable predictors of rsch. 'success'"]:::Systemic --> B2["UJ-style Eval. replaces Trad. journals"]:::Systemic
B --> A2
D:::Systemic --> B2
B1["UJ attractive to gatekeepers"]:::Systemic --> D
D --> D1 --> A1
B --> B1 --> B2
B2 --> B3["Rsch. eval. more efficient"]:::Systemic
B3 --> RSXreliable["Acad. rsch. more useful/reliable"]:::Systemic
RSXreliable --"Some impactful"--> A1
B2 --> B4["Open access"]:::Systemic
B4 --> A2
A1 --> A2:::Direct
A0 --"Incentivize"--> B6["Readable formats"]:::Systemic --> A2:::Multi --> E1["Better policy choices"]:::Multi --> O["+ Survival & flourishing"]:::Multi
B6 --> RSXreliable
Direct path (Yellow)
Multi-step outcomes (Green)
Academic integration (Blue)
Systemic change (Pink)
Main Paths (Simplified)
A simplified view focusing on the most direct path from Unjournal's activities to improved research and better-informed decision makers.
graph TD
classDef Direct fill:#faefaf,stroke:#b8a000
classDef Multi fill:#bdf2c9,stroke:#2d8a4e
classDef Integrate fill:#99bdf0,stroke:#2563eb
A0["UJ tools, incentives, platforms"]:::Multi --"we will"--> FocusEA["Focus on impactful research"]:::Direct
FocusEA --> EAR["+ Evaluation of EA-aligned research"]:::Direct
EAR --> EARig["+ Rigor of EA-aligned research"]:::Direct
EAR --- OF["+ Open science"]:::Direct --> EARO["More replicable, reliable"]:::Direct
EARO --> A1["Impactful research improved"]:::Multi
EARig --> A1
EARig --"Stronger work"--> AcadAware["+ Awareness in impactful research"]:::Integrate
A0 --> PayAcad["Academics paid to evaluate"]:::Integrate --> AcadAware
AcadAware --> D1["+ Academic focus on global priorities"]:::Integrate
D1 --> A1
A1 --> A2["Decision makers better informed"]:::Direct
A2 --> E1["Better assessment of priorities"]:::Multi
E1 --> O["+ Survival & flourishing"]:::Multi
Evaluation Workflow (Simplified)
The core evaluation process: from research suggestion through prioritization, evaluation, author response, and publication with DOIs.
graph TD
S["RESEARCH SUGGESTED, identified, or submitted"]
S --> MP["UJ teams PRIORITIZE"]
MP --> |Low vote| DP[Deprioritize]
MP --> AA["Seek authors' permission/engagement"]
AA --> |"'No' from junior authors"| DP
AA --> EM["EVAL. MANAGER assigned, contacts EVALUATORS"]
EM --> |5+ weeks| EVC["2-3 evaluators complete reports & ratings"]
EVC --> |2+ weeks| ARE["Authors respond"]
ARE -.->|EM considers| EMS["EM's summary"]
EVC -.->|EM considers| EMS
EMS --> UJO["UJ publishes output w/ DOIs"]
ARE --> UJO
EVC --> UJO
style S fill:#e0f2fe,stroke:#0284c7
style UJO fill:#dcfce7,stroke:#16a34a
style DP fill:#fef2f2,stroke:#dc2626
Evaluation Workflow (Detailed)
The complete evaluation workflow showing all steps, decision points, and information flows between authors, managers, and evaluators.
graph TD
S["Author submits work"] --> MP[Managers prioritize work]
NBER["Work enters 'prestige' archive"] --> MP
MP --> WE[Work to evaluate]
WE ..- |May add| BEN["'Bespoke Evaluation Notes'"]
MS["Managers select & contact authors"] --> AP[Author: OK]
AP --> WE
MS --> AN[Author: NO] --> DE[Don't evaluate]
WE -->|Managers select| EM(("Evaluation Manager"))
EM .-> |May add tips| BEN
EM -->|Selects & contacts| UJEV(UJ evaluators)
UJEV --> |Accepts| EA((Evaluator))
BEN ..-|Incorporated| UJT[Template & guidelines]
EA --> |Scheduled| EVC[Evaluators complete evals]
EVC --> EC[\Evaluation content\]
EVC --> EVR[\Evaluation ratings\]
UJT --> |Shared with| EA
EC -->|Passed anonymously| EVPA[Shared with Author]
EVR --> EVPA
EVPA .-> ARE[\Authors' response\]
EVC .->|EV chooses|EVA[Anonymity]
EVC .->|EV chooses|EVS[Sign evaluation]
ARE .->|EM considers| EMS[\EM's summary\]
EVC ..- |EM considers| EMS
EMS --> UJO[/Published output w/ DOIs/]
EC --> UJO
ARE --> UJO
EVR --> |Reported in| UJO
EVS .-> |EV name| UJO
UJO --> UJPUB[Publicize & track]
style UJO fill:#dcfce7,stroke:#16a34a
Research Prioritization Framework
How we prioritize research for evaluation based on prestige, global decision-relevance, influence, and methodological quality.
graph TD
subgraph Prestigious["Prestigious & Fully-baked Work"]
A[Prestigious Work]
A -->|Global-Decision Relevant| B[PRIORITIZE]
A -->|Not Quite Relevant| C[Consider UJ Priorities]
C --> C1[Submitted Work]
C --> C2[Transparent Code/Data]
C --> C3[Supports Impactful Work]
C1 --> D[Potentially Prioritize]
C2 --> D
C3 --> D
end
subgraph LessPrestigious["Less Prestigious Work"]
E[Less Prestigious Work]
E -->|Influential| F[PRIORITIZE]
F --> F1[Academic Stream]
F --> F2[Policy Stream]
E -->|Less Influential but Relevant| G[Consider]
G --> G1[Methodological Strength]
G --> G2[Clarity and Logic]
G1 -->|Strong| H[Potentially Prioritize]
G2 -->|Strong| H
end
style B fill:#dcfce7,stroke:#16a34a
style F fill:#dcfce7,stroke:#16a34a
style D fill:#fef9c3,stroke:#ca8a04
style H fill:#fef9c3,stroke:#ca8a04