Work in progress · every country on the board

Operation Atlas

Every country's freight forwarders, on one board, in three layers: names first, then addresses proven on the company's own site, then the Monday campaign that never writes to anyone twice. This page is the machine and the plan, read live from the board as it stands right now.

1 · How it is done

Three layers, three commands, one board

A country moves up a ladder that is computed from data and never typed: nothing on file →names known → websites known → reachable → drafted →mailed. Each layer is a separate operation with its own command, its own guards and its own refusals, and each hands a stricter list to the next. Names are cheap and come from the networks; addresses are expensive and come only from the company's own website; sending happens only on an explicit word, and only once.

L1
/ffscout <country>
Names onto the board
Reads
JCtrans public profiles, the WCA directory, PPL, the FMC registry, FIATA and every national association's own roster, and the partner networks' member pages.
Proves
That a company exists under one identity — one key across every source, diacritics folded, non-Latin scripts kept — and belongs to a country that resolved through the alias table.
Refuses
A corrupt location. A guessed field. A directory's claimed member count in place of the roster it actually publishes. Any email address at all — that is the next layer's job.
L2
/email-scout <country>
Addresses proven on the company's own site
Reads
The company's own website, in parallel workers, with no model and no search engine in the loop. Then DNS, to see whether the domain accepts mail at all.
Proves
That an address belongs to the site it was found on (same registrable domain), that the domain has MX, that the run itself was trustworthy (a control sample decides), and that the company is not already held by another record.
Refuses
gmail and other-domain finds, guessed URLs, overwriting a human's work, and a broken network dressed as a result. Blank beats wrong.
L3
/outreach <country>
The Monday campaign, on his word
Reads
Only the cold list — everyone we may write to minus everyone anyone has ever written to — for the one country he names, plus the analytics behind every claim in the copy.
Proves
The never-mail-twice guarantee, by prove-guarantee.mjs, before staging; the 12-hunt pre-send audit; and every address again at the moment of sending.
Refuses
To transmit without the word GO. A held jurisdiction. Our own group, our suppliers, the associations we scraped, a placeholder address, a same-domain twin. Any address that has left the sendable list since approval.
supabase/migrations/20260901142000_the_board_knows_who_is_ours.sql · lines 46–58How one company is counted once, whichever sources hold it — domain first, key second, per-row last.
46agg as (47  select code,48         count(distinct ck)                                as known_names,49         count(distinct ck) filter (where has_site)        as with_website,50         count(distinct ck) filter (where has_email)       as with_email,51         count(distinct ck) filter (where src = 'jctrans') as jctrans_names,52         count(distinct ck) filter (where src = 'wca')     as wca_names,53         count(distinct ck) filter (where src = 'ppl')     as ppl_names,54         count(distinct ck) filter (where src = 'us_oti')  as oti_names,55         count(distinct ck) filter (where src = 'assoc')   as assoc_names,56         count(distinct ck) filter (where src = 'agents')  as onboarded57    from pool58   group by code
supabase/migrations/20260901142000_the_board_knows_who_is_ours.sql · lines 106–113How a country's stage is graded. Nobody types a level; the data decides.
106case107  when coalesce(d.mailed, 0) + coalesce(x.n, 0) > 0             then 5108  when coalesce(d.drafted, 0) > 0                               then 4109  when coalesce(s.n, 0) > 0 or coalesce(g.with_email, 0) > 0    then 3110  when coalesce(g.with_website, 0) > 0                          then 2111  when coalesce(g.known_names, 0) > 0                           then 1112  else 0113end                                                         as ladder,
2 · The machinery

16 agents, and how they hand off

Each one is a program in the repository with a single job, a single output and something it refuses to do. They run without a model in the loop, so a second run costs nothing; judgement was spent once, when they were written. The status on each card is checked against the file on disk at build time, not declared.

L1 · names/ffscout
01

Scout

names from the networks
live
scripts/scout/jctrans.mjs
scripts/import-jctrans.mjs · scripts/import-wca.mjs · scripts/import-ppl.mjs · scripts/import-us-oti.mjs · scripts/seed-country-aliases.mjs
Reads
JCtrans public company profiles through their own sitemap (robots honoured, unhurried); the WCA directory through a logged-in session, never in parallel; PPL; the FMC licence registry.
Writes
jctrans_members · wca_members · ppl_members · us_oti — the raw pools, one row per published profile.
Refuses
A row whose own location field is corrupt (the Niue trap) never counts. A country label that does not resolve through country_aliases stays visibly unresolved rather than guessed.
Hands to
Key Keeper, then the Board Keeper.
02

Harvester

association registers
live
scripts/import-assoc.mjs
scripts/scout/atlas-fiata-run.sh · scripts/onboard-assoc.mjs · scripts/repair-fiata-rows.mjs
Reads
FIATA’s world register, then each national association’s own roster — CIFFA, AAACI, SPEDLOGSWISS, MIFFA, JLA, NAFL, EIFFA and the rest — as the association printed it.
Writes
assoc_members — one table for every register, so a new source never needs a schema change. The published roster is the evidence and is stored verbatim.
Refuses
Guessing an absent field. A re-import is an on-conflict no-op, so nothing is ever counted twice by running it twice.
Hands to
Key Keeper.
03

Network Cartographer

partner-network rosters
live
scripts/probe-directory-networks.mjs
scripts/import-assoc.mjs (net-* sources) · resources/ff-agent-scout-pipeline/networks/net_harvest_*.py
Reads
The public member rosters of the partner networks — OLO Family, GT Global, IFN, FNC, SCN, PCN, WWPC, XLProjects, WLA, IASA and the others — plus a card per network.
Writes
assoc_members as net-<slug>; a card that keeps member_count_claimed and member_count_harvested apart on purpose.
Refuses
Quoting a claim as a holding — a directory banner counts offices or ambitions, a roster counts members. A wall (login, no roster, names served as images) is recorded as a wall and not re-spent on.
Hands to
Key Keeper.
04

Key Keeper

one identity per company
live
supabase/migrations/20260903203000_the_key_keeps_letters_it_cannot_spell.sql
Reads
Every company name from every source, in whatever script it was published.
Writes
wz_company_key() — folds Latin diacritics so Akıner and AKINER agree, strips the legal suffixes, and for a name with no Latin letters keeps the Arabic, Cyrillic or Chinese as written. Indexed on agents, jctrans_members, wca_members, assoc_members.
Refuses
Transliteration — a spelling the company never published, which two harvesters would invent differently. And the empty key: it once cost a whole Jordanian register.
Hands to
Board Keeper and Deduplicator, who both dedupe on it.
L2 · addresses/email-scout
05

Site Walker

the company’s own site
live
scripts/scout/email-scout.mjs
scripts/scout/atlas-email-run.mjs · scripts/scout/site-discovery.mjs · scripts/scout/contact-names.mjs
Reads
Only the company’s own website — home and contact pages — in parallel workers, four waves (agents, us_oti, wca, jctrans), one country at a time through country_aliases. Cloudflare-obfuscated addresses are decoded; [at] and (dot) are unfolded.
Writes
JSONL on disk, never the database. Resumable: a key already on disk is skipped on restart.
Refuses
Search engines — the enrichment that once matched a Turkish forwarder to an observatory stays quarantined. Guessed URLs. Anything not published by the company itself.
Hands to
Reachability Control, then the Domain Prover.
06

Reachability Control

network or country?
live
scripts/scout/email-scout.mjs
Reads
When most fetches in a run fail: a fresh sample of sites from OTHER countries, same code path, same moment.
Writes
The verdict on the run — trustworthy or not — in the log, loudly.
Refuses
Letting a hard country look like a broken network, or a broken network look like a result. Control fails too: exit non-zero, nothing kept. Control healthy: the country is genuinely hard, result kept, coverage declared partial. Unmeasurable control: untrusted.
Hands to
Domain Prover — or it stops the run.
07

Domain Prover

same-domain or nothing
live
scripts/import-scouted-emails.mjs
Reads
Each candidate address against the registrable domain of the site it was found on (co.uk and com.vn aware).
Writes
Only the same-domain best candidate, only into an empty field. Human work and prior evidence are never overwritten.
Refuses
gmail, yahoo and other-domain finds — recorded as evidence, never promoted. Blank beats wrong, enforced in code.
Hands to
MX Gate.
08

MX Gate

does the domain take mail
live
scripts/onboard-jctrans.mjs
public.is_mailable() in the database
Reads
The address’s domain through DNS MX, cached per domain, five-second deadline on every lookup.
Writes
A no-MX address is demoted to no_email; the company stays on file without it.
Refuses
Treating a DNS timeout as a dead mailbox — slow DNS is counted and kept, not deleted. A hang is worse than a crash, so every call has a deadline.
Hands to
Deduplicator.
09

Deduplicator

never twice, proved
live
scripts/prove-guarantee.mjs
scripts/onboard-jctrans.mjs guards 2, 3, 3b
Reads
v_sendable and v_sendable_cold after every write to agents.
Writes
A verdict, exit code non-zero when it fails. 12 checks: one row per address, one company per row, no doubled top-level domain, no do_not_contact record, the sender reads nothing else, no already-mailed company on the cold list, every sent draft stamped, no held jurisdiction.
Refuses
An address already held by another agent — the company is created as no_email instead. A guarantee proved on the wrong list (it was, once; the cold list is now proved too).
Hands to
Auditor.
L3 · the send/outreach
10

Auditor

hunts, nightly, read-only
live
scripts/audit-sendable.mjs
Reads
One snapshot of the sendable views, proved against count(*) first so a partial pull cannot look complete.
Writes
Nothing. 12 hunts — software vendors, rival networks, associations and ports, banks and carriers, HR inboxes, same-domain twins, our own group, placeholder addresses and more — print the suppression SQL and never execute it.
Refuses
Silence. A finding fails the run until a human adds it to the ALLOW list with a reason and a date; that list is the only way out.
Hands to
Sanctions Warden.
11

Sanctions Warden

held jurisdictions
live
scripts/lib/sanctions.mjs
Reads
One list of held jurisdictions, shared by every onboarder and by the guarantee’s last check — a rule only one reader enforces is not a rule.
Writes
The hold note on the company record; the record itself is kept because its association publishes it.
Refuses
Any address in a held jurisdiction reaching a sending list before the compliance decision is made by a person, not a script.
Hands to
Campaign Stager.
12

Campaign Stager

Monday, prepared and asked
live
scripts/outreach/monday-prep.mjs
scripts/outreach/stage-usa-intro.mjs · scripts/outreach/stage-india-touch1.mjs · scripts/outreach/stage-vietnam-rates.mjs
Reads
v_sendable_cold for the country he names; analytics_enquiries for every claim the copy makes.
Writes
outreach_drafts, draft until a named person approves; the Monday prep summary — counts, guards, one line to him.
Refuses
Firing on its own. A country nobody named. A claim the analytics cannot support. Network CTAs to anyone outside that network.
Hands to
Sender.
13

Sender

transmits on his word only
live
scripts/outreach/send.mjs
scripts/outreach/send-gmail.mjs · scripts/outreach/record-bounces.mjs · scripts/outreach/attribute-replies.mjs
Reads
Approved drafts only, and it re-proves every address against v_sendable at the moment of sending.
Writes
The send record and agent_state.first_emailed_at; a unique index in the database makes a second send of the same touch impossible.
Refuses
Everything without an explicit GO. Bursts — the throttle is the deliverability budget. It is the only code in the project that can put a message on the wire, and it ships pointing at nothing.
Hands to
Ledger; replies land on My Inbox and the Lead Desk.
Always onreads every lane
14

Board Keeper

the ladder, per country
live
supabase/migrations/20260901142000_the_board_knows_who_is_ours.sql
Reads
All six name sources, unioned and deduplicated on domain first, company key second.
Writes
v_country_ladder — every country graded L0 to L5 from data, never typed — and mv_country_ladder, its materialised copy, refreshed after every import. The Directory and this page read the copy; the view is the fallback.
Refuses
Counting one company twice across sources. Calling a country reachable on a raw register address that suppression would refuse.
Hands to
Every page. Next: a materialised copy refreshed after each import — the union is a full scan by construction.
15

Dossier Writer

one brief per company
in build
scripts/company-briefs.mjs
scripts/country-briefs.mjs
Reads
Every table for one company, and for the partner dossiers the company’s own site.
Writes
corpus_docs — one generated dossier per company and country, which the Ask desk cites. The partner dossiers for Canada and Egypt exist as documents; a page for them does not yet.
Refuses
Hand edits. A wrong figure is fixed in the table and the dossier regenerated.
Hands to
The Ask desk today; the Meetings desk when it exists.
16

Ledger

every run, its result
live
scripts/scout/atlas-email-run.mjs
scripts/time-views.mjs · scripts/atlas-onboard-all.sh
Reads
Every run’s exit code, seconds and verdict; the milliseconds of every view a page runs, before and after a change.
Writes
scouted / degraded / never-scouted per country; data/timings.jsonl; a per-country onboarding ledger that skips what is done and records what failed.
Refuses
"Run complete" over silent holes — exits non-zero if any country failed. A skipped country is not a done country; a loaded job is not a working job.
Hands to
The next session, which resumes from it.
3 · The guards, as written

What the machine actually runs

Cut from the repository at build time, line numbers included. If a line moves, the build breaks rather than the page drifting. Each of these exists because the failure it prevents has already happened once here.

scripts/scout/email-scout.mjs · lines 355–379Reachability Control. A country that is hard to reach from Dubai must not read as a broken network, and a broken network must never read as a result — so a control sample from other countries decides which it is.
355/* A high failure rate has two completely different causes and the old check356 * could not tell them apart: our network is broken, or this country is simply357 * hard to reach from here. Measured 3 Sep 2026 through this same code path,358 * at the same moment: Chinese sites failed 72%, everything else failed 30%.359 * So China's rate is a fact about China. Failing the run on it would have360 * thrown away 420 addresses that were correctly found.361 *362 * The control decides. If a fresh sample of sites OUTSIDE this run's country363 * also fails, the machine is the problem and the result is not trustworthy.364 * If the control is healthy, the country is genuinely hard: say so loudly,365 * keep the result, and exit 0. Silence is what we are avoiding, not failure. */366if (doneN > 20 && netFail / doneN > 0.5) {367  const rate = Math.round((100 * netFail) / doneN);368  console.error(`⚠️  ${rate}% of fetches failed. Running a control sample to find out whether that is the network or the country…`);369  const control = await controlReachability();370  if (control === null) {371    console.error('🔴 could not run the control — treating a high failure rate as untrustworthy, as before.');372    process.exit(1);373  }374  if (control > 50) {375    console.error(`🔴 the control failed too (${control}%) — this is a network problem, not a result. Not trusting it.`);376    process.exit(1);377  }378  console.error(`✅ control failed only ${control}% — the ${rate}% is this country, not our network. Result kept; treat the coverage as genuinely partial.`);379}
supabase/migrations/20260903203000_the_key_keeps_letters_it_cannot_spell.sql · lines 132–144Key Keeper. The identity key used to erase every letter it could not spell, so an Arabic-named register collapsed onto one empty key. The fallback keeps the name as published.
132                   '[^a-z0-9]+', '', 'g'),133                 '(incorporated|international|limited|company|holdings|group|inc|llc|ltd|corp|co|sa|sarl|bv|gmbh|srl|spa|pte|pvt|plc|ag|as|nv)+$',134                 '')135             ), ''),136           -- The name normalised to nothing. Keep every letter and digit in137           -- whatever script it was written in; drop only spacing and138           -- punctuation. Reached by 155 rows today.139           regexp_replace(lower(coalesce(t, '')), '[^[:alnum:]]+', '', 'g')140         )141$function$;142 143comment on function public.wz_company_key(text) is144  'Company identity key. Folds Latin diacritics; for names with no Latin letters at all, falls back to the name stripped of spacing and punctuation rather than to the empty string (3 Sep 2026 — 189 Jordanian companies were lost to the empty key).';
scripts/audit-sendable.mjs · lines 422–434Auditor, one of 12 hunts. The pre-send audit found WorldZone about to cold-pitch itself; this hunt runs every night so it cannot happen again.
422{423  key: 'own-group',424  title: 'Hunt 7 · our own group (WorldZone, Thunga, WayTrans, XB Brothers, Godavari, Arise, Winlines, Ozone, Youngitute)',425  cost: 'we cold-pitch ourselves — and the FIATA suppression of worldzone.ae proved the other entry paths let a second record through.',426  reason: 'This is our own group (WorldZone / Thunga / WayTrans / XB Brothers / Godavari / Arise / Winlines / Ozone / Youngitute). Never a cold-outreach target.',427  find: (rows) => rows.flatMap((r) => {428    const n = hit(OWN_GROUP_NAME, r.company);429    if (n) return [{ row: r, why: `name "${n}"` }];430    const d = hit(OWN_GROUP_DOMAIN, emailDomain(r.email), r.domain, r.group_domain, registrable(r.website), r.agent_id);431    if (d) return [{ row: r, why: `domain "${d}"` }];432    return [];433  }),434},
scripts/prove-guarantee.mjs · lines 170–190Deduplicator, checks of 12. The guarantee was once proved on the wrong list; the cold list — the only list a first touch may read — is now proved directly, and so is the bookkeeping it depends on.
170/* ---- 7 · nobody we have already mailed is on the cold list ---------------- */171// The cold view filters on agent_state.first_emailed_at. If a send ever lands172// without that stamp, the company silently returns to the first-touch pool and173// receives the same introduction twice — the exact failure this file exists to174// prevent, and one that check 1 cannot see because the address is distinct.175const g = await one(`176  select count(*)::int n from v_sendable_cold c177   where c.agent_id in (select agent_id from outreach_drafts where status = 'sent')`);178check('No already-mailed company is on the cold (first-touch) list',179  g.n === 0,180  `${g.n} companies with a sent draft are still offered by v_sendable_cold.`,181  'a company receives our cold introduction a second time.');182 183const h = await one(`184  select count(*)::int n from outreach_drafts d185    left join agent_state st on st.agent_id = d.agent_id186   where d.status = 'sent' and st.first_emailed_at is null`);187check('Every sent draft stamped agent_state.first_emailed_at',188  h.n === 0,189  `${h.n} sent drafts have no first_emailed_at on their agent.`,190  'the cold list forgets a send and re-offers the company next Monday.');
scripts/onboard-jctrans.mjs · lines 94–112MX Gate. A domain that accepts no mail is not an address — but a slow DNS answer is not a dead mailbox, so a timeout is counted and kept rather than deleted.
94/* ---- the MX gate: an address whose domain accepts no mail is not an95        address (verify_contact.py's check, wired in at the promotion point).96        Cached per domain; every lookup gets a deadline — a hang is worse97        than a crash. On timeout the address is kept and counted: DNS being98        slow is not evidence the mailbox is dead. ------------------------- */99const mxCache = new Map();100async function hasMx(domain) {101  if (mxCache.has(domain)) return mxCache.get(domain);102  let ok;103  try {104    const mx = await Promise.race([105      resolveMx(domain),106      new Promise((_, rej) => setTimeout(() => rej(new Error('timeout')), 5000)),107    ]);108    ok = Array.isArray(mx) && mx.length > 0;109  } catch (e) { ok = e.message === 'timeout' ? 'unknown' : false; }110  mxCache.set(domain, ok);111  return ok;112}
scripts/lib/sanctions.mjs · lines 8–20Sanctions Warden. One definition, read by every onboarder and by the guarantee, so no new import path can quietly disagree with it.
8 * WHAT THIS IS. Not a legal opinion — a hold. The company records are kept9 * (their national associations and the networks publish them openly, and the10 * board should show the truth about a market), but they never reach a sending11 * list until Nisarg makes the compliance call. WorldZone banks in USD, is12 * pushing into the United States, and its CEO is at a JCtrans US conference:13 * mailing Iran or Syria is his decision and his bank's, not a script's.14 *15 * IR/SY/KP/CU are comprehensively sanctioned by the US. RU/BY are sectoral16 * rather than comprehensive — a UAE company may lawfully trade with them —17 * so those two are a commercial and reputational judgment more than a legal18 * one. They are held here for the same reason: he should choose, not inherit.19 */20export const SANCTIONED = new Set(['IR', 'SY', 'KP', 'CU', 'BY', 'RU']);
4 · The board

Every country

Read from mv_country_ladder — the materialised copy of v_country_ladder the Directory reads too, rebuilt after every import (since 4 Sep: 0.2 s instead of 9 s). Below reachable nobody in that country can be written to yet, so the row says Research in progress instead of a code.

All countries on the board
Loading the board…
CountryRegionStageNames knownWith websiteWith emailOn fileSendableNamed peopleMailed
5 · The register layer

Networks and associations, as harvested

Every partner network and national association we have read sits in one table, assoc_members, one source per roster. These are the rows we hold, not the counts the directories claim — the two are kept apart on purpose, because a banner counts offices or ambitions and a roster counts members.

6 · The next months

Where this goes

Each item below is a plan that already exists in writing, with a script, a page or a decision behind it. Nothing here is a wish.

Now

September 2026

  • JCtrans USA Regional Conference, Las Vegas, 8–9 September. The CEO and the Saudi head in the room with the working list, the must-meet asks and the FMC-licensed overlap on /jctrans.
  • Mondays: one country, his choice. The monday_outreach stage counts, runs the guards and asks at 15:00 IST; only an explicit GO transmits, and only through send.mjs.
  • China's own run. The largest pool in the harvest was kept out of the default campaign on purpose. The scout has walked the Chinese sites and proved addresses on them; promotion through the four guards and the MX gate is the next step, then the guarantee.
  • WCA, targeted. Before each Monday, the WCA detail pass for that week's country only — one human login, then the harvester — for named office contacts with titles. The list-only rows wait for it.
  • Done 4 SepThe board materialised. v_country_ladder is a six-table union and a full scan by construction; mv_country_ladder is its copy, refreshed after every import. Measured through the browser with the same token: 9.4 s → 0.2 s; /directory 22 s → 3 s.
  • Done 4 SepThe Meetings desk. One room per partner call: the company profile first, the artifact shelf, and a Present mode so a screen share shows the software and nothing else. Open it.
  • A private remote for the repository. Decided 2 September; the pre-flight has passed; it runs on his word.
Q4

October – December 2026

  • Named decision-makers per company. The four facts of a complete record are person, email, website, membership. The contact-names pass reads a name only where it sits beside a leadership title on the company's own site — no title, no row — across every reachable company.
  • Per-company dossiers, in the software. Canada and Egypt are written as documents; every reachable market follows, generated from the tables so a dossier never goes stale, and surfaced on a page rather than in a file.
  • The register layer completes. The strongest national registers still un-harvested go in through the same importer — one table, no schema change per source.
  • The always-on runner. Long runs move off the laptop to the MSI Cyborg under WSL2, so a closed lid stops killing a night's work; the ledger is what lets a run resume there.
  • BAM Rome, 1–4 November. Nisarg and Naga Prasad; the tiered shortlist and dossiers on /rome. Companies marked "meet in Rome, do not cold-email" stay suppressed, and the audit checks that the suppression cannot be defeated by a spelling.
  • Outcome detection. Quoted, won and lost read from the mail itself, with the CONFIRMED marker as the won signal, so the board can carry what happened after the send, not only that it was sent.
  • MAILMIND. Categorisation of the desk's mail — after the taxonomy is frozen, and nothing touches a mailbox before that.
2027

First half

  • Every country with names reaches reachable. The standing email-scout campaign works the biggest untapped pools first, derived from the board, until every market that has names has proven addresses.
  • The dates season. The booking window that opens at Rome and runs into January, ahead of Ramadan on 8 February 2027 — the lane study on /dates is built for it.
  • Every Monday, a country; every reply, a room. The campaign machine, the inbox and the Meetings desk as one loop: introduced, not interrogated; a meeting's logistics chased the moment a reply agrees to one.
  • The board keeps its promises. Nothing typed, nothing summed twice, nothing sent twice — the same guards, on a larger board.