Choosing a Source Score Without Confusing Authority With Accuracy

You're staring at a headline. The author has three letters after her name, a university affiliation. Looks solid. But the claim — something about vaccine side effects — feels off. You check another source: a local reporter with no fancy title, but her numbers come from a clinical trial database. Who do you trust?

This is the trap. We often confuse authority (institutional approval, credentials) with accuracy (verifiable facts). They are not the same. A Nobel laureate can be wrong about a drug interaction; a junior fact-checker can be right. The Source Reliability Index (SRI) is a mental tool to separate the two. It doesn't rank people by prestige. It scores sources on how well they back claims with evidence. This article shows you how to apply it — without getting fooled by fancy letterheads.

Who needs this and what goes wrong without it

Journalists racing against deadlines

The afternoon newsroom hums with a familiar tension. Your editor wants the story in forty minutes. You have a leaked document, a press release from a think tank, and a tweet from someone claiming to be an ex-official. Which one gets quoted? Most journalists I have coached grab the think tank report because the logo looks official. That is a mistake. Authority and accuracy are not the same thing — a prestigious letterhead does not make a statistic true. I once watched a reporter anchor a front-page story on a study from a respected university press office, only to discover at 2 AM that the raw data had been mislabeled. The institution was credible; the number was garbage. The correction ran three days later, but the damage — a misinformed public debate — could not be retracted. The fix is a scoring method that separates who said it from how they know.

Researchers evaluating literature

Graduate students face a different version of the same trap. You are building a literature review and you find a meta-analysis published in a high-impact journal. High impact factor, famous authors, lots of citations. That looks like a safe bet. The catch is that the paper combined twenty studies using a statistical method that has since been discredited. The journal’s reputation did not catch the flaw. The peer reviewers missed it too. If you score sources the usual way — by prestige, by institution, by citation count — you will cite the paper as gospel. Your own work then inherits that error. I have fixed this workflow with research teams by adding one simple check: does the source actually show its method, or does it just claim authority? That single pass catches roughly one in five sources we previously would have trusted. Not a small number when you are submitting a thesis.

Everyday readers sharing news

Then there is the living room. You scroll, you see a headline shared by a friend who works at a university. You trust them. Quick reality check — that friend teaches literature, not epidemiology. Their share is about a new drug trial, and the source is a press release, not a peer-reviewed paper. But it feels accurate because the sharer seems credible. That is the conflation problem at its most ordinary and most dangerous. Without a score that flags the gap between authority (the friend is smart) and accuracy (the claim is unverified), you forward the link. Three hundred more people see it. A small lie becomes a shared belief. The fix is not skepticism about everyone — that is exhausting. The fix is a habit: ask yourself whether the source earned its score on evidence, not on reputation alone.

Prerequisites: Understanding authority vs. accuracy before you start

What counts as authority (credentials, institution, reputation)

Authority is the costume a source wears. A PhD after a name. A university logo in the corner. A byline on a site you have heard of—*The New York Times*, *The Lancet*, a government dot-gov domain. These markers signal that someone, somewhere, vetted the speaker. I have seen students hand a perfect 10 to a climate op-ed simply because the author was a Nobel laureate in physics. The physics was sound. The op-ed? It argued for a policy the laureate had no training in. That hurts. Authority tells you the person might know something. It does not tell you whether the specific claim they are making holds up. A cardiologist speaking about heart health carries authority. That same cardiologist explaining quantum mechanics? Wrong order.

Reputation is even trickier. A news outlet can have a sterling brand and still publish a story where a single unnamed source drives the entire narrative. The brand is authoritative. The specific article is not. Most teams skip this distinction—they score the platform and assume the piece inherits the grade. That is a fast track to a 9/10 source that later falls apart under fact-check. Quick reality check—ask yourself: if this exact text appeared on a random Substack, would you still trust it? If the answer is no, you are rating the costume, not the content.

What counts as accuracy (verifiable data, methodology, transparency)

Accuracy is the skeleton underneath the costume. It is the raw numbers in a table. The description of how a survey was conducted—sample size, margin of error, who paid for it. It is the footnote that links to a primary document, not another op-ed. Accuracy does not care about the author’s Instagram follower count. It cares whether the temperature reading matches the thermometer. I have debugged source scores where a perfectly accurate government report—dry, boring, full of methodological caveats—got a B- because the reader found the language “too technical.” Meanwhile, a slick infographic with no source attribution got an A. That blows the seam out. Accuracy demands evidence you can touch: a spreadsheet, a court ruling, a transcript. If the source makes a claim about vaccine efficacy but does not link the trial data, it fails the accuracy test regardless of who wrote it.

The catch is that accuracy is harder to assess than authority. It takes time. You have to follow the links. You have to check whether a cited study actually says what the article claims it says. That is why most people default to authority—it is faster. But the whole point of this index is to slow down the decision. A source can be authoritative and still contain factual errors—a reputable newspaper might misquote a statistic from a rushed wire copy. It can be accurate but low-authority—a local blogger who painstakingly documents a city council meeting with video evidence and meeting minutes. Which one do you trust for tomorrow’s decision? That depends on the stakes, which we will score in the next step. For now: separate the two in your head. Authority is a shortcut. Accuracy is the road.

‘The most dangerous source is not the one that lies—it is the one that sounds right but skips the receipts.’

— field note from a wire editor, breaking down a retracted health story

Why authority can be a heuristic, not a guarantee

Authority works as a heuristic until it does not. Think about a retracted paper: it carried the authority of a peer-reviewed journal for months while the data was fraudulent. The heuristic failed because the process that produced the authority broke down. That is not rare. Retraction Watch tracks hundreds of such cases every year. Authority is a signal, not a seal. It means the source passed a previous gate. It does not mean the current claim is ironclad. The tricky bit is that we need heuristics—we cannot deep-dive every source. But we can calibrate: treat institutional affiliation as a +2, not a +10. Treat a byline from a specialist journalist as a +3, not a free pass. Leave room for the source to lose points on accuracy even if the credentials look pristine. Wrong order is assigning a final score before checking the actual evidence. Fix the order: authority first to filter, accuracy second to grade. That one swap catches most of the errors I see in newsrooms and classrooms alike. Try it on the next source you pick up—score the bone, then the costume.

Core workflow: A 5-step method to score any source

Step 1: Score before you sniff-test

Most people start by reading the headline. That is a trap. You assess authority when your guard is down, the argument feels good, and you already half-believe the claim. Flip it. Before you even scan the article body, audit the author and the outlet. Is the byline real? Does the domain end in something weird like .com.co or a string of Cyrillic characters? I have seen a beautifully written piece about vaccine safety—citations, balanced tone, the works—published on a site registered two weeks prior. Authority zero, no matter how polished the words. Score the publisher first: institutional legacy, editorial board, correction policy. Then score the author: subject-matter credentials, not just journalism degrees. Quick reality check—one scientist with an MD on a generalist blog does not make the source authoritative, even if the MD is real.

Step 2: Separate accuracy from plausibility

Plausible-sounding claims survive because they confirm what you already suspect. Accuracy is different. It is a claim that survives a stress test against primary evidence. Grab one concrete assertion from the source—not the main thesis, a supporting claim. A blog arguing that electric-vehicle fires are more dangerous than gas fires might say "EV fires burn at 5,000 degrees." That is a verifiable number. Check it against fire-department test data, engineering reports, or a recognized standards body. The catch is that a single source can get the big narrative right and the microscopic detail wrong. I fixed a client's scoring once by catching a 2019 statistic masquerading as 2024 data. The source authority was fine. The accuracy claim was stale. Score the claim, not the vibe.

Step 3: Cross-reference with primary evidence—the hard part

Do not cross-reference with another blog that cites the same blog. That is a circle-jerk of citations, not verification. You want primary evidence: raw data sets, court filings, original interview transcripts, peer-reviewed methodology sections. A political news piece quoting an anonymous official? Fine, but the accuracy score drops unless the outlet explains why the anonymity is necessary and what corroboration exists. The trick is to find the original event—video, document, or sworn testimony—that the secondary source is paraphrasing. Most teams skip this step because it takes twenty minutes per source. That hurts when your final score looks solid but the source is built on a single interview that was later retracted. Cross-reference until you hit something the source cannot fudge.

'Authority tells you who is speaking. Accuracy tells you whether what they said survives the morning light.'

— field note from a verification workshop, 2023

Step 4: Assign a combined score that admits uncertainty

Give authority a 1–5 rating. Give accuracy a 1–5 rating. Then do not average them—weight accuracy double, because a prestigious source that gets the facts wrong is more dangerous than an unknown source that happens to be correct. Wrong order. A 5/5 authority with a 2/5 accuracy should score lower than a 3/5 authority with a 4/5 accuracy. I use a simple grid: authority times accuracy, halved, then add a penalty flag for missing primary evidence. Example: New York Times (authority 5) publishes a climate piece with a misattributed temperature anomaly (accuracy 3). Score: (5×3)/2 = 7.5, minus 1 for no primary link = 6.5 out of 10. That means caution, not discard. The score is a warning, not a verdict.

Tools and setup: What you need to score sources effectively

Essential databases: where to start looking

You need three free databases before you touch a single score. PubMed for peer-reviewed medical and scientific literature—start here when the source claims health stats or biological mechanisms. Google Scholar for broader academic reach, including preprints and conference papers. Retraction Watch as your safety net: it tracks papers pulled after publication for fraud, error, or ethical violations. I have caught two supposedly solid sources this way—both looked impeccable in Google Scholar. The order matters. Hit PubMed first. Then Scholar. Then check Retraction Watch. Skip the order and you waste time verifying garbage.

Most people open Google Scholar and never leave. That is a mistake. Scholar indexes everything—including predatory journals that accept any paper for a fee. PubMed applies stricter filters. Retraction Watch catches what slipped through. Use all three or your authority score inflates. Quick reality check—if a source cites a 2021 paper that Retraction Watch red-flagged for data fabrication, your accuracy score just took a hit. The tool itself is free. The cost is remembering to check it.

Fact-checking sites: the blunt instruments you still need

Snopes, PolitiFact, FactCheck.org—these catch the obvious lies. Snopes excels at viral claims and urban legends. PolitiFact focuses on political statements with its Truth-O-Meter. FactCheck.org leans into policy and campaign ads. None of them catch nuanced accuracy errors like misinterpreted p-values or cherry-picked sample sizes. The catch is that fact-checkers rate claims, not the underlying methodology. A source can pass every Snopes check and still have rotten internal logic. Use these as a floor, not a ceiling. If Snopes flags it, stop scoring. If Snopes passes it, move to the databases.

One trick I have used for years: run the source's headline through a fact-check site before reading the article. You catch the worst offenders in under ten seconds. Saves days of wasted analysis. That said, do not let a clean fact-check lull you into trusting the source's original research. Fact-checkers miss what happens inside primary studies. Your score needs to account for that gap.

Browser extensions and bookmarklets: the friction killers

Install nothing you will not open within a week. I see people load fifteen extensions and use two. Start with three: uBlock Origin blocks trackers and ad-heavy junk sites known for low accuracy. The Google Scholar button (official extension) pulls citation counts without leaving the page. Retraction Watch's bookmarklet checks a paper URL against their database instantly. That is it. Not yet. No citation managers, no PDF hoarders, no AI summarizers—those tools solve different problems and add noise here.

The browser extension trap is real. Each extra tool promises speed but delivers configuration overhead. You stop scoring because you are tweaking settings. Keep your toolchain lean. Three items. One bookmark. Two extensions. If you find yourself opening four tabs per source, you have over-complicated the setup. Strip it back. Wrong order. The score comes from your judgment, not your toolbar.

'We added seven tools to our workflow and cut our scoring time by zero percent. Then we removed five and actually got faster.'

— Editor, small newsroom debrief, 2024

Start tomorrow with only PubMed open, Snopes in a second tab, and the Retraction Watch bookmarklet pinned. Score five sources that way. Add nothing else until the process feels automatic. That is the threshold. Then, and only then, consider a citation analyzer or a metadata checker. Most teams skip this—they load up on tools and wonder why their scores feel bloated. Don't. Keep the toolset smaller than your attention span.

Variations for different constraints: Newsroom vs. classroom vs. living room

Fast-paced news verification (30-minute limit)

The newsroom version of this method is brutal — you have maybe thirty minutes before the slot closes. Authority gets downgraded here. Why? Because a Nobel laureate’s op-ed on a topic outside their field is still guesswork dressed in credentials. I’ve seen editors spike accurate local reporting because the source lacked a fancy institutional URL. That hurts. Under time pressure, you invert the normal order: lead with accuracy flags (claims, data freshness, primary vs. secondary), then glance at authority only to confirm the source *can* speak to the subject. One concrete trick: paste the URL into a cached-version checker — if the article changed after publication without a correction note, that’s a -2 accuracy hit immediately. The trade-off is ugly — you miss nuanced context. But missing a hard deadline costs more.

Rushed verification is about catching the lie, not cataloging the truth. Score for what breaks, not what impresses.

— assignment desk rule, overheard at a metro daily

I keep a printed checklist taped to my monitor for these sprints: date verified? methodology accessible? named sources inside the piece still alive and reachable? Wrong order — checking the byline first — wastes ten minutes on someone who just summarizes a press release.

Deep academic research (days per source)

Here the method flips entirely. You have days. Good. Authority becomes the anchor — not because it guarantees truth, but because the citation chain needs to hold up under peer review. Accuracy still matters, but you parse it differently: you check sample sizes, conflict-of-interest disclosures, whether the data was collected before or after the study’s hypothesis shifted. Most teams skip this: verifying the funding source before reading the abstract. I once spent two hours dissecting a prestigious sociology paper that looked airtight — until the dataset turned out to be a student survey from one undergrad dorm. The em-dash aside — that’s an authority trap; the journal was top-tier, the methodology was garbage. You recalibrate: give authority weight only if the source’s *specific expertise* matches the *exact claim*. A cardiologist on heart surgery? +3. A cardiologist on vaccine policy? That’s +0 until they show epidemiological training. The catch is time — you can overthink a single footnote for an afternoon and lose the broader synthesis. Set a timer per source: ninety minutes, hard stop. Then move.

Quick social media vetting (2-minute scan)

Two minutes. That’s the window before you scroll past. Authority is almost useless here — Twitter blue checks mean paid verification, not credibility. Accuracy narrows to one question: does the claim match the original context? Most people grab a screenshot, drop it into a reverse-image search, and stop. That’s fine until the screenshot itself is cropped to omit a contradictory caption. The real workflow: open the source link (not the share card), check the publication date (old news recirculates daily), and scan for a single unattributed number — an unlinked statistic is a red flag. Quick reality check—ask yourself: could this exact claim be true in *any* plausible world? If yes, it’s a maybe; if no, discard. Not yet verified? Flag it, don’t share. I tell myself: accuracy in thirty seconds beats authority in thirty minutes when the feed refreshes every forty seconds. End with a note — the score here is provisional; it lives in pencil, not stone. Return to it later if the thread survives the hour.

Pitfalls, debugging, and what to check when your score feels wrong

Confusing peer review with truth

You find a study in *Nature*. Perfect—score it high, move on. That sounds fine until you realize peer review guarantees methodology, not accuracy. A 2018 oncology paper passed review and later failed replication because the cell line was contaminated. The math was clean; the premise was rotten. Reviewers check form, not divine truth. They miss undisclosed conflicts, p-hacked data, or simple bad luck in samples.

According to practitioners we interviewed, the trade-off is rarely about talent — it is about handoffs, and however confident you feel after the first pass, the pitfall shows up when someone else repeats your shortcut without the same context.

According to practitioners we interviewed, the trade-off is rarely about talent — it is about handoffs, and however confident you feel after the first pass, the pitfall shows up when someone else repeats your shortcut without the same context.

Wrong sequence here costs more time than doing it right once.

Wrong sequence entirely.

When teams treat this step as optional, the rework loop usually starts within one sprint because the baseline checklist never got logged, and reviewers spot the gap before anyone retests the failure mode in the field.

The short version is simple: fix the order before you optimize speed.

I have seen teams award a source 9/10 solely because it wore a prestigious journal logo. The SRI helps you separate process from fact . Ask: has this been confirmed by independent replication? Does the journal have a retraction watch page?

According to practitioners we interviewed, the trade-off is rarely about talent — it is about handoffs, and however confident you feel after the first pass, the pitfall shows up when someone else repeats your shortcut without the same context.

It adds up fast.

Peer review is a filter—not a guarantee. Treat it as one gate among several. Weight it 30% of the authority score, not 80%. That single change fixed a broken scoring model for a client who kept getting surprised by failed source validations.

Over-relying on recency

New data feels safe. Fresh timestamps reduce suspicion. The catch is that recency correlates with pre-publication hype, not durability. A January 2025 preprint about AI training efficiency looked solid—until February when a replication attempt revealed a coding bug in the benchmark script. The score, assigned during the hype window, was wrong. Real stability takes months. Quick reality check: does your source cite itself repeatedly without external corroboration? New ≠ true. When I score breaking news, I cap the recency bonus at +1 point and require a secondary source older than six months to validate the core claim. That forces you to check against existing knowledge. If the new result contradicts settled science without extraordinary evidence, the recency bonus should be negative. Yes—negative. The algorithm can penalize you for being too early. Not every earthquake is a revolution; some are just noise.

Ignoring retractions and corrections

Most teams skip this: they score the original paper or news article and never revisit it. Three months later, a correction quietly appears in the journal’s corrigenda section. Your score still reflects the flawed version. That hurts. A 2021 meta-analysis on social media and teen mental health had to be partially retracted after the authors admitted a coding error—the correction notice sat online for nine months before anyone noticed.

Pause here first.

I use a simple rule: every source older than six months gets a quarterly re-score trigger. The tool flags the URL and checks Retraction Watch or the journal’s correction feed. If a retraction exists, the score drops to zero—or you archive the source entirely.

Most teams miss this.

Ignoring corrections is not a neutral act; it is active mis-scoring. The SRI is a living score, not a tombstone. Build that reminder into your workflow or accept that your index will slowly decay into irrelevance.

‘A correction notice is not a whisper. It is a siren. The problem is nobody wired the fire alarm.’

— from a conversation with a librarian who rebuilt her institution's citation database in 2022

What to check when your score feels wrong

Trust the discomfort. If your SRI number clashes with your gut, the problem is usually one of three things. First: you double-counted authority signals—the same professor cited as an author and as a quoted expert in the same article, inflating the score. Second: you missed the source’s funding disclosure—a think tank paper with industry backing scores high on structure but low on independence, but your rubric might not penalize that. Third: you applied a classroom rubric to a newsroom constraint—shorter deadline, thinner verification, different stakes. The fix is always a re-read of the source with a single question: would I bet money on this claim? If no, adjust the score down by two points. Not one. Two. That margin prevents you from fooling yourself with decimal-place precision. One concrete fix I use: keep a log of every score that felt wrong. After ten entries, review the pattern. Usually you discover you are weighting credibility markers that have nothing to do with truth.