Economist-style deep analysis for motions, propositions, and committee-report articles#461
Economist-style deep analysis for motions, propositions, and committee-report articles#461
Conversation
Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…xpand domain detection Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
github-actions
bot
added
deployment
There was a problem hiding this comment.
Pull request overview
This PR upgrades the TypeScript article generators in scripts/data-transformers.ts to produce more substantive, domain-aware “deep analysis” content (Economist-style), with better document-type inference when doktyp is missing from raw MCP/API data.
Changes:
- Injects
impliedDoktypinto deep-analysis/significance generation so motions/reports/propositions get type-specific analysis even whendoc.doktypis absent. - Cleans up
policySignificanceTouchesacross 14 languages to avoid redundant procedural boilerplate. - Expands
detectPolicyDomains()keyword coverage and adds new analytical sections (thematic grouping, opposition strategy, cross-committee and government priority signals), with corresponding test additions.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
scripts/data-transformers.ts |
Adds/rewires deep analysis generation, improves domain detection, and introduces thematic/strategy narrative sections in motions/propositions/committee reports content. |
tests/data-transformers.test.ts |
Extends unit tests to assert domain-specific analysis, implied doktyp behavior, thematic grouping, and removal of old boilerplate. |
| // ── PRIMARY: policy domain significance derived from document content ──── | ||
| const significance = generatePolicySignificance(doc, lang); | ||
| parts.push(`<strong>${escapeHtml(String(L(lang, 'whatThisMeans')))}:</strong> ${significance}`); |
There was a problem hiding this comment.
generateDocumentIntelligenceAnalysis() calls generatePolicySignificance(doc, lang) without passing an implied document type. If doc.doktyp / doc.documentType is absent (common in raw MCP responses), this reintroduces the same "default case" problem this PR fixes elsewhere. Pass the docType argument through as impliedDoktyp (e.g. 'mot'/'bet'/'prop') so type-specific analysis is preserved.
There was a problem hiding this comment.
Fixed in aa4ddb3 — generateDocumentIntelligenceAnalysis now passes its local docType variable as impliedDoktyp: generatePolicySignificance(doc, lang, docType).
| const allDomains = new Set<string>(); | ||
| const enrichedCount = docs.filter(d => d.contentFetched).length; | ||
| docs.forEach(doc => { | ||
| const sig = generatePolicySignificance(doc, lang); | ||
| const genericVal = L(lang, 'policySignificanceGeneric'); | ||
| if (sig !== genericVal) allDomains.add(sig); | ||
| }); | ||
| if (allDomains.size > 0) { | ||
| const policyContextVal = L(lang, 'policyContext'); | ||
| content += ` <li>${escapeHtml(String(policyContextVal))}: ${escapeHtml(Array.from(allDomains).slice(0, 4).join('; '))}</li>\n`; | ||
| } |
There was a problem hiding this comment.
In generateGenericContent() key takeaways, allDomains is populated with the full output of generatePolicySignificance() (now multiple sentences of analysis), but it is later rendered as "Policy context". This makes the summary bullet potentially very long and repetitive, and the variable name no longer matches what’s stored. Consider collecting just the domain labels from detectPolicyDomains() (or truncating to the first sentence) for this list instead of the full analysis text.
There was a problem hiding this comment.
Fixed in aa4ddb3 — replaced generatePolicySignificance(doc, lang) with detectPolicyDomains(doc, lang).forEach(d => allDomains.add(d)) so the "Policy context" bullet shows short domain labels only (e.g., fiscal policy; defence and security policy), not multi-sentence analysis text.
tests/data-transformers.test.ts
Outdated
| }); | ||
|
|
||
| describe('Domain-specific policy analysis (getDomainSpecificAnalysis)', () => { | ||
| it('should include substantive fiscal motion analysis — not just generic boilerplate', () => { |
There was a problem hiding this comment.
Test name says "fiscal motion analysis" but the fixture uses committee reports (reports: [{..., organ: 'FiU'}]). Renaming the test to reflect the actual document type would make failures easier to interpret.
| it('should include substantive fiscal motion analysis — not just generic boilerplate', () => { | |
| it('should include substantive fiscal committee report analysis — not just generic boilerplate', () => { |
There was a problem hiding this comment.
Fixed in aa4ddb3 — renamed to should include substantive fiscal committee report analysis — not just generic boilerplate.
scripts/data-transformers.ts
Outdated
| // Identify primary policy domain(s) for the most-active party using detectPolicyDomains | ||
| // Use English keys for domain detection to get consistent results for display | ||
| const topDomainSet = new Set<string>(); | ||
| topMotions.forEach(m => { | ||
| detectPolicyDomains(m, 'en').forEach(d => topDomainSet.add(d)); | ||
| }); | ||
| const topDomains = Array.from(topDomainSet).slice(0, 2); | ||
|
|
||
| const isSv = lang === 'sv'; | ||
| const count = topMotions.length; | ||
| let text = ''; | ||
|
|
||
| if (isSv) { | ||
| const domainList = topDomains.join(' och '); | ||
| text = `<strong>${escapeHtml(topParty)}</strong> är mest aktiv med ${count} motion${count !== 1 ? 'er' : ''}`; | ||
| if (domainList) text += `, med fokus på ${escapeHtml(domainList)}`; | ||
| text += '.'; |
There was a problem hiding this comment.
In generateOppositionStrategySection(), domain detection is hard-coded to English via detectPolicyDomains(m, 'en'), but the result is later embedded into Swedish prose when lang === 'sv'. This will produce mixed-language output like "fokus på fiscal policy" in Swedish articles. Use detectPolicyDomains(m, lang) (or map English domain keys back to localized display strings) so the displayed domain list matches the requested language.
There was a problem hiding this comment.
Fixed in aa4ddb3 — changed detectPolicyDomains(m, 'en') to detectPolicyDomains(m, lang) so domain names in the opposition strategy section match the article language.
| const domains = detectPolicyDomains(motion, lang); | ||
| const theme = domains[0] || (lang === 'sv' ? 'Övriga frågor' : 'General matters'); | ||
| if (!byTheme[theme]) byTheme[theme] = []; | ||
| byTheme[theme].push(motion); |
There was a problem hiding this comment.
The fallback theme label for motions is hard-coded as "General matters" for any non-Swedish language. In non-English languages (fr/de/ja/…), this will leak an English heading into localized pages. Prefer a localized label via CONTENT_LABELS (new key) or reuse an existing localized label to avoid English-only output.
There was a problem hiding this comment.
Fixed in aa4ddb3 — added generalMatters key to ContentLabelSet and all 14 CONTENT_LABELS language entries (e.g. sv: 'Övriga frågor', de: 'Allgemeine Angelegenheiten', ja: '一般事項', etc.). The fallback now uses String(L(lang, 'generalMatters')) instead of the hard-coded English string.
scripts/data-transformers.ts
Outdated
| function getDomainSpecificAnalysis(primaryDomain: string, doktyp: string, lang: Language | string): string { | ||
| const isSv = lang === 'sv'; | ||
| type LangPair = { en: Record<string, string>; sv: Record<string, string> }; | ||
| const analyses: Record<string, LangPair> = { | ||
| 'fiscal policy': { | ||
| en: { | ||
| mot: 'Fiscal policy motions directly challenge the government\'s budget assumptions and signal opposition readiness to contest tax and spending priorities.', | ||
| bet: 'The Finance Committee\'s position on fiscal matters is usually decisive — the chamber almost always follows its recommendation on budgetary questions.', | ||
| default: 'Government fiscal proposals must clear rigorous Finance Committee scrutiny and align with Sweden\'s fiscal surplus rule, making the committee\'s verdict pivotal.' | ||
| }, | ||
| sv: { | ||
| mot: 'Finanspolitiska motioner utmanar direkt regeringens budgetantaganden och signalerar oppositionens beredskap att bestrida skatte- och utgiftsprioriteringar.', | ||
| bet: 'Finansutskottets ståndpunkt i finanspolitiska frågor är i regel avgörande – kammaren följer nästan alltid utskottets rekommendation.', | ||
| default: 'Regeringens finanspolitiska förslag måste klara finansutskottets granskning och harmonisera med överskottsmålet för att nå bifall.' | ||
| } | ||
| } else { | ||
| if (doktyp === 'mot') { | ||
| implication = 'This motion signals a policy position; passage requires government or majority support.'; | ||
| } else if (doktyp === 'bet') { | ||
| implication = 'The committee vote will be determinative for the legislation\'s path forward.'; | ||
| } else { | ||
| implication = 'Parliamentary committee review and chamber debate will shape the final outcome.'; | ||
| }, | ||
| 'defence and security policy': { | ||
| en: { | ||
| mot: 'Defence motions carry heightened strategic significance following Sweden\'s NATO accession, pressing the government on long-term security commitments.', | ||
| bet: 'Committee reports on defence shape Sweden\'s military posture and NATO integration trajectory — decisions here have multi-decade consequences.', | ||
| default: 'Defence proposals engage Sweden\'s NATO obligations and cross-party consensus-building mechanisms for national security legislation.' | ||
| }, | ||
| sv: { | ||
| mot: 'Försvarsrelaterade motioner har förhöjd strategisk betydelse efter Sveriges NATO-inträde och pressar regeringen om långsiktiga säkerhetsåtaganden.', | ||
| bet: 'Utskottsbetänkanden om försvar formar Sveriges militära inriktning och NATO-integration – besluten har konsekvenser i decennier.', | ||
| default: 'Försvarspropositioner engagerar Sveriges NATO-förpliktelser och mekanismer för brett partistöd inom säkerhetspolitiken.' | ||
| } | ||
| }, | ||
| 'environmental and climate policy': { | ||
| en: { | ||
| mot: 'Climate motions reflect growing parliamentary pressure for faster decarbonisation, often targeting specific industries or the pace of policy implementation.', | ||
| bet: 'The Environment Committee\'s recommendations balance climate ambition against economic competitiveness — its position sets the legislative baseline.', | ||
| default: 'Environmental proposals must navigate competing interests from industry, regional governments, and EU climate commitments, making parliamentary support critical.' | ||
| }, | ||
| sv: { | ||
| mot: 'Klimatmotioner speglar växande parlamentariskt tryck för snabbare koldioxidminskning och riktar sig ofta mot specifika branscher.', | ||
| bet: 'Miljöutskottet väger klimatambition mot ekonomisk konkurrenskraft – dess rekommendation sätter lagstiftningens utgångspunkt.', | ||
| default: 'Miljöförslag måste navigera konkurrerande intressen från industrin, regionerna och EU:s klimatåtaganden.' | ||
| } | ||
| }, | ||
| 'healthcare policy': { | ||
| en: { | ||
| mot: 'Healthcare motions typically target gaps in regional service delivery, pressing for national minimum standards, additional funding, or new patient rights.', | ||
| bet: 'Social Affairs Committee reports on healthcare set the framework for Sweden\'s regionally delivered but nationally financed health system.', | ||
| default: 'Healthcare proposals require coordination between national government, regional councils, and professional bodies — a complexity that shapes the legislative timeline.' | ||
| }, | ||
| sv: { | ||
| mot: 'Hälso- och sjukvårdsmotioner riktar sig typiskt mot brister i regionala tjänster och driver på för nationella miniminivåer eller nya patienträttigheter.', | ||
| bet: 'Socialutskottets betänkanden om hälso- och sjukvård sätter ramarna för det regionalt levererade men nationellt finansierade hälsosystemet.', | ||
| default: 'Hälso- och sjukvårdspropositioner kräver samordning mellan stat, regioner och professioner – en komplexitet som formar lagstiftningens tidslinje.' | ||
| } | ||
| }, | ||
| 'migration policy': { | ||
| en: { | ||
| mot: 'Migration motions reflect one of Sweden\'s most contested policy areas, with parties divided on asylum rules, integration requirements, and deportation procedures.', | ||
| bet: 'The Social Insurance Committee\'s migration reports navigate Sweden\'s EU law obligations and UN Refugee Convention commitments alongside domestic political pressures.', | ||
| default: 'Migration proposals must balance EU regulatory obligations with national political imperatives, making cross-party support essential for durable legislation.' | ||
| }, | ||
| sv: { | ||
| mot: 'Migrationsmotioner speglar ett av Sveriges mest omtvistade politikområden, med partier delade om asylregler, integrationskrav och återvändanderutiner.', | ||
| bet: 'Socialförsäkringsutskottets migrationsbetänkanden navigerar Sveriges åtaganden enligt EU-rätten och FN:s flyktingkonvention.', | ||
| default: 'Migrationspropositioner måste balansera EU-rättsliga förpliktelser med nationella politiska imperativ.' | ||
| } | ||
| }, | ||
| 'EU and foreign affairs': { | ||
| en: { | ||
| mot: 'EU and foreign affairs motions signal parliamentary expectations for government negotiating positions — influential despite executive prerogative in external relations.', | ||
| bet: 'The Foreign Affairs Committee\'s reports on EU matters reflect Sweden\'s positioning within the bloc and may bind future negotiating postures.', | ||
| default: 'EU and foreign affairs proposals engage Sweden\'s treaty obligations and often require coordination with European partners before domestic enactment.' | ||
| }, | ||
| sv: { | ||
| mot: 'EU- och utrikespolitiska motioner signalerar parlamentets förväntningar på regeringens förhandlingspositioner.', | ||
| bet: 'Utrikesutskottets betänkanden om EU-frågor speglar Sveriges positionering inom unionen och kan binda framtida förhandlingslinjer.', | ||
| default: 'EU- och utrikespropositioner engagerar Sveriges fördragsförpliktelser och kräver samordning med europeiska partner.' | ||
| } | ||
| }, | ||
| 'justice policy': { | ||
| en: { | ||
| mot: 'Justice motions address crime, sentencing, and policing — areas with high public salience where opposition parties frequently press for tougher or more targeted measures.', | ||
| bet: 'The Justice Committee shapes the criminal law framework; its reports on sentencing and policing directly affect prosecution practice and enforcement priorities.', | ||
| default: 'Justice proposals balance rule-of-law principles, human rights obligations, and public safety demands — requiring careful drafting to withstand constitutional scrutiny.' | ||
| }, | ||
| sv: { | ||
| mot: 'Rättsliga motioner rör brott, straff och polis – frågor med hög allmän relevans där oppositionen ofta driver på för hårdare åtgärder.', | ||
| bet: 'Justitieutskottet formar den straffrättsliga ramen; dess betänkanden om straffsatser och polisverksamhet påverkar direkt åklagarnas praxis.', | ||
| default: 'Rättsliga propositioner balanserar rättsstatsprinciper, mänskliga rättigheter och allmän säkerhet.' | ||
| } | ||
| }, | ||
| 'labour market policy': { | ||
| en: { | ||
| mot: 'Labour market motions engage sensitive negotiations between employers, unions, and the state — every motion sends a signal to Sweden\'s social partners.', | ||
| bet: 'The Labour Committee\'s reports on workplace legislation must navigate collective bargaining autonomy while setting minimum statutory floors.', | ||
| default: 'Labour market proposals enter an arena where tripartite negotiation shapes the final legislative outcome as much as parliamentary votes.' | ||
| }, | ||
| sv: { | ||
| mot: 'Arbetsmarknadsmotioner engagerar känsliga förhandlingar mellan arbetsgivare, fackförbund och stat – varje motion signalerar till parterna.', | ||
| bet: 'Arbetsmarknadsutskottets betänkanden om arbetsplatslagar måste navigera kollektivavtalens självständighet.', | ||
| default: 'Arbetsmarknadspropositioner träder in i en arena där trepartsförhandlingar formar det slutliga lagstiftningsresultatet.' | ||
| } | ||
| }, | ||
| 'housing policy': { | ||
| en: { | ||
| mot: 'Housing motions reflect structural tension between demand for affordable homes and constraints of planning law, rent regulation, and construction cost pressures.', | ||
| bet: 'The Civil Affairs Committee\'s housing reports address one of Sweden\'s most persistent policy challenges, where committee decisions unlock or block major regulatory change.', | ||
| default: 'Housing proposals must reconcile competing interests from municipalities, property owners, tenants, and developers — a coalition rarely achieved quickly.' | ||
| }, | ||
| sv: { | ||
| mot: 'Bostadsmotioner speglar strukturell spänning mellan efterfrågan på prisvärda bostäder och begränsningarna i plan- och hyreslagstiftning.', | ||
| bet: 'Civilutskottets bostadsbetänkanden hanterar en av Sveriges mest ihållande politiska utmaningar.', | ||
| default: 'Bostadspropositioner måste balansera konkurrerande intressen från kommuner, fastighetsägare, hyresgäster och byggföretag.' | ||
| } | ||
| }, | ||
| 'transport policy': { | ||
| en: { | ||
| mot: 'Transport motions address infrastructure investment, road safety, and public transit — areas where regional and national interests frequently diverge.', | ||
| bet: 'The Transport Committee\'s reports guide Sweden\'s national infrastructure planning cycle, directly affecting long-term investment priorities.', | ||
| default: 'Transport proposals engage the national infrastructure budget, regional equity, and climate transition targets — all must be balanced in committee deliberation.' | ||
| }, | ||
| sv: { | ||
| mot: 'Transportmotioner rör infrastrukturinvesteringar, trafiksäkerhet och kollektivtrafik – frågor där regionala och nationella intressen ofta divergerar.', | ||
| bet: 'Trafikutskottets betänkanden vägleder Sveriges nationella infrastrukturplanering och påverkar direkt långsiktiga investeringsprioriteringar.', | ||
| default: 'Transportpropositioner engagerar den nationella infrastrukturbudgeten, regional jämlikhet och klimatomställningsmål.' | ||
| } | ||
| }, | ||
| 'trade and industry policy': { | ||
| en: { | ||
| mot: 'Industry and trade motions often target competitiveness, innovation, or trade agreements — signalling party positions ahead of EU-level or bilateral negotiations.', | ||
| bet: 'The Committee on Industry and Trade shapes Sweden\'s business environment through reports that set conditions for investment, innovation, and exports.', | ||
| default: 'Industry and trade proposals engage international commitments, EU single-market rules, and domestic competitiveness imperatives simultaneously.' | ||
| }, | ||
| sv: { | ||
| mot: 'Näringspolitiska motioner riktar sig ofta mot konkurrenskraft, innovation eller handelsavtal och signalerar partipositioner inför förhandlingar.', | ||
| bet: 'Näringsutskottets betänkanden formar Sveriges affärsmiljö och sätter villkoren för investeringar och export.', | ||
| default: 'Näringspolitiska propositioner engagerar internationella åtaganden, EU:s inre marknadsregler och inhemsk konkurrenskraft.' | ||
| } | ||
| }, | ||
| 'education policy': { | ||
| en: { | ||
| mot: 'Education motions reflect deep disagreements on school standards, teacher pay, and the role of independent schools — one of Sweden\'s most contested domestic debates.', | ||
| bet: 'The Education Committee\'s reports directly shape curriculum standards, funding formulas, and school regulation — decisions with long generational consequences.', | ||
| default: 'Education proposals must balance national curriculum standards with municipal delivery autonomy and the contested role of private providers in the Swedish school system.' | ||
| }, | ||
| sv: { | ||
| mot: 'Utbildningsmotioner speglar djupa meningsskiljaktigheter om skolstandard, lärarlöner och friskolornas roll.', | ||
| bet: 'Utbildningsutskottets betänkanden formar direkt läroplaner, finansieringsmodeller och skolreglering.', | ||
| default: 'Utbildningspropositioner måste balansera nationella läroplaner med kommunalt leveransansvar och de privata aktörernas omstridda roll.' | ||
| } | ||
| } | ||
| return `Touches on ${escapeHtml(domains)}. ${implication}`; | ||
| }; | ||
|
|
||
| // Normalise display-language domain string to English for lookup | ||
| const enDomainMap: Record<string, string> = { | ||
| 'finanspolitik': 'fiscal policy', | ||
| 'försvars- och säkerhetspolitik': 'defence and security policy', | ||
| 'miljö- och klimatpolitik': 'environmental and climate policy', | ||
| 'utbildningspolitik': 'education policy', | ||
| 'hälso- och sjukvårdspolitik': 'healthcare policy', | ||
| 'migrationspolitik': 'migration policy', | ||
| 'EU- och utrikespolitik': 'EU and foreign affairs', | ||
| 'rättspolitik': 'justice policy', | ||
| 'arbetsmarknadspolitik': 'labour market policy', | ||
| 'bostadspolitik': 'housing policy', | ||
| 'transportpolitik': 'transport policy', | ||
| 'näringspolitik': 'trade and industry policy' | ||
| }; | ||
|
|
||
| const lookupKey = enDomainMap[primaryDomain] ?? primaryDomain; | ||
| const entry = analyses[lookupKey]; | ||
| if (!entry) return ''; |
There was a problem hiding this comment.
getDomainSpecificAnalysis() allocates the large analyses object and enDomainMap on every call. Since this function is invoked per-document, this adds unnecessary overhead and makes the function harder to scan. Move these structures to module-level constants (or memoize) so they are created once and reused.
There was a problem hiding this comment.
Fixed in aa4ddb3 — extracted DOMAIN_ANALYSES and EN_DOMAIN_MAP as module-level const declarations. The function body now only does lookups into these constants; the large object is allocated once on module load, not on each call.
|
@copilot apply changes based on the comments in this thread |
…-level constants Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
…e-report articles Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
scripts/data-transformers.ts
Outdated
There was a problem hiding this comment.
In generatePropositionsContent(), domainCount is computed from Object.keys(byCommittee).length, which counts referred committees, not policy domains. This makes the policyImplicationsContext text (“touch on N policy domains/areas”) inaccurate. Compute a unique set of domains via detectPolicyDomains() across propositions for domainCount, and keep byCommittee solely for the “government priority signal” committee ranking.
There was a problem hiding this comment.
Fixed in 46d5ab6. domainCount now comes from the union of detectPolicyDomains() across all propositions, while byCommittee is kept exclusively for the government priority signal committee ranking:
// Count unique policy domains across all propositions
const allPropDomains = new Set<string>();
propositions.forEach(p => detectPolicyDomains(p, 'en').forEach(d => allPropDomains.add(d)));
const domainCount = Math.max(allPropDomains.size, 1);
// Group by referred committee — only used for priority signal below
const byCommittee: Record<string, number> = {};Added a test that verifies 3 propositions all sent to FiU (1 committee) but covering climate/defence/fiscal keywords do not produce "1 policy domain" in the output.
|
@copilot apply changes based on the comments in this thread |
…eratePropositionsContent Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Done — commit 46d5ab6 fixes |
scripts/data-transformers.ts
Outdated
| propositions.forEach(p => detectPolicyDomains(p, 'en').forEach(d => allPropDomains.add(d))); | ||
| const domainCount = Math.max(allPropDomains.size, 1); |
There was a problem hiding this comment.
domainCount is forced to be at least 1 via Math.max(allPropDomains.size, 1), which will misreport the number of detected policy domains when none are detected (it will say “1 policy domain”). Consider allowing 0 (and letting policyImplicationsContext render “0”) or special-casing the 0-domain case with a different message. Also, this loop hard-codes 'en' when calling detectPolicyDomains; using lang would keep the implementation consistent with the rest of the generators.
| propositions.forEach(p => detectPolicyDomains(p, 'en').forEach(d => allPropDomains.add(d))); | |
| const domainCount = Math.max(allPropDomains.size, 1); | |
| propositions.forEach(p => detectPolicyDomains(p, lang).forEach(d => allPropDomains.add(d))); | |
| const domainCount = allPropDomains.size; |
There was a problem hiding this comment.
Fixed in 96eb786 — detectPolicyDomains(p, lang) (was 'en'), and Math.max removed so domainCount = allPropDomains.size directly (renders "0" accurately when no domains are detected).
tests/data-transformers.test.ts
Outdated
| it('should include substantive climate motion analysis', () => { | ||
| const content = generateArticleContent({ | ||
| reports: [{ titel: 'Klimatpolitik', organ: 'MJU', url: 'https://example.com/1', dok_id: 'MJU1' }] | ||
| } as MockArticlePayload, 'committee-reports', 'en') as string; | ||
|
|
There was a problem hiding this comment.
Test name says "climate motion analysis" but the fixture uses reports and the committee-reports article type. Renaming the test to reflect that it’s a committee report (or changing the fixture to a motion) will make failures easier to interpret.
There was a problem hiding this comment.
Fixed in 96eb786 — renamed to should include substantive climate committee report analysis.
|
@copilot apply changes based on the comments in this thread and rebase with main branch and resolve beloe merge issues This branch has conflicts that must be resolved |
…y, test renames/fixes Co-authored-by: pethers <1726836+pethers@users.noreply.github.com>
Done in 96eb786 — merged |
Signed-off-by: James Pether Sörling <pethers@users.noreply.github.com>
🔍 Lighthouse Performance Audit
📥 Download full Lighthouse report Budget Compliance: Performance budgets enforced via |
| if (themeCount > 1) { | ||
| // Thematic sections: group motions by policy area | ||
| content += `\n <h2>${L(lang, 'thematicAnalysis')}</h2>\n`; | ||
| Object.entries(byTheme).forEach(([theme, themeMotions]) => { | ||
| content += `\n <h3>${escapeHtml(theme)} (${themeMotions.length})</h3>\n`; | ||
| themeMotions.forEach(motion => { content += renderMotionEntry(motion, lang); }); | ||
| }); | ||
| } else { | ||
| // Single theme or no detection: flat list | ||
| motions.forEach(motion => { content += renderMotionEntry(motion, lang); }); |
There was a problem hiding this comment.
In generateMotionsContent(), the function only returns content inside the else branch (themeCount <= 1). When themeCount > 1 (the new thematic grouping case), execution falls through to the end of the function without returning, which will yield undefined and break motions articles. Move the return content; outside the if/else (or return from both branches) and remove the leftover branch-local code so the thematic path returns correctly.
| // Group motions by parent proposition to eliminate repetitive section headers | ||
| const { grouped, independent } = groupMotionsByProposition(motions); | ||
|
|
||
| if (grouped.size > 0) { | ||
| const responsesLabel = lang === 'sv' ? 'Svar på propositioner' |
There was a problem hiding this comment.
generateMotionsContent() still contains the old proposition-based grouping flow (hardcoded per-language responsesLabel / indepLabel, groupMotionsByProposition, and the inline renderMotion helper). This conflicts with the new thematic grouping (and will duplicate/contradict rendered motion entries when the else branch runs). Remove this legacy block and rely on renderMotionEntry() + thematic grouping (and localized labels via L(lang, key)) so only one consistent rendering path exists.
| const isSwedishSource = !!(doc.titel && !doc.title); | ||
| const passageHtml = isSwedishSource | ||
| ? `<span data-translate="true" lang="sv">${escapeHtml(passage)}</span>` | ||
| : escapeHtml(passage); |
There was a problem hiding this comment.
generateDeepPolicyAnalysis() manually emits <span data-translate="true" lang="sv">…</span> for Swedish-source passages regardless of target language. This bypasses svSpan()'s established contract (data-translate only in lang === 'sv') and can trigger translation/quality validation failures in non-Swedish articles. Use svSpan(escapeHtml(passage), lang) for Swedish-source passages instead of hardcoding the span.
| function generateOppositionStrategySection(motions: RawDocument[], lang: Language | string): string { | ||
| const byParty: Record<string, RawDocument[]> = {}; | ||
| motions.forEach(m => { | ||
| const party = m.parti || 'other'; |
There was a problem hiding this comment.
generateOppositionStrategySection() groups motions by m.parti without treating the 'Unknown' sentinel as missing. This can produce output like <strong>Unknown</strong> leads opposition activity… even though generateMotionsContent already maps Unknown to other. Normalize the party key here the same way (e.g., map missing/Unknown to other and exclude it from the ranked list).
| const party = m.parti || 'other'; | |
| const rawParty = typeof m.parti === 'string' ? m.parti.trim() : ''; | |
| const party = !rawParty || rawParty.toLowerCase() === 'unknown' ? 'other' : rawParty; |
| content += `\n <h2>${L(lang, 'thematicAnalysis')}</h2>\n`; | ||
| Object.entries(byTheme).forEach(([theme, themeMotions]) => { | ||
| content += `\n <h3>${escapeHtml(theme)} (${themeMotions.length})</h3>\n`; | ||
| themeMotions.forEach(motion => { content += renderMotionEntry(motion, lang); }); |
There was a problem hiding this comment.
Within the new thematic grouping, theme headings are rendered as <h3>, but each motion entry also renders its title as <h3>. This breaks heading hierarchy (two sibling h3 levels for nested content) and is confusing for screen readers. Consider rendering motion titles as <h4> when inside a themed section (or bump theme headings to <h2>/<h3> and keep entries one level deeper).
| themeMotions.forEach(motion => { content += renderMotionEntry(motion, lang); }); | |
| themeMotions.forEach(motion => { | |
| // Demote motion entry headings one level when inside a themed section | |
| const entryHtml = renderMotionEntry(motion, lang); | |
| const demotedHtml = entryHtml | |
| .replace(/<h3(\b[^>]*)?>/g, '<h4$1>') | |
| .replace(/<\/h3>/g, '</h4>'); | |
| content += demotedHtml; | |
| }); |
detectPolicyDomains(doc, lang)helper — Set-based domain extractionDOMAIN_ANALYSESmodule-level constant — 12-domain × 3-type matrix allocated oncegeneratePolicySignificance(doc, lang, impliedDoktyp?)— domain ID only +getDomainSpecificAnalysisgenerateDeepPolicyAnalysis(doc, lang, impliedDoktyp?)— fullText passage withimpliedDoktypgenerateOppositionStrategySection(motions, lang)— usesdetectPolicyDomains(m, lang)(not hard-coded'en')generateMotionsContent()— thematic grouping withL(lang, 'generalMatters')localized fallbackgeneratePropositionsContent()—domainCountfromdetectPolicyDomains(p, lang)(not committee count); noMath.max(..., 1)per reviewgenerateCommitteeContent()— cross-committee analysis;generateDeepPolicyAnalysiswith implied'bet'generateDocumentIntelligenceAnalysis()— passesdocTypeasimpliedDoktypgenerateGenericContent()policy bullet — domain labels only viadetectPolicyDomains()renderMotionEntry— usessvSpan()(not hardcodeddata-translate="true") + improved Unknown sentinel handlinggeneralMatterskey toContentLabelSetand all 14 languageCONTENT_LABELSentriesorigin/main— resolvedscripts/data-transformers.tsconflict (keep thematic grouping, adoptsvSpanfrom main); resolvedsitemap.xmlconflict (take main's timestamps)data-translatecount test — change tolang='sv'(svSpan addsdata-translateonly for Swedish articles)groupMotionsByPropositiontests — reflect thematic grouping behaviourOriginal prompt
This section details on the original issue you should resolve
<issue_title>Upgrade article content from document lists to Economist-style deep analysis</issue_title>
<issue_description>## 📋 Issue Type
Feature / Content Quality
🎯 Objective
Transform motion, proposition, and committee-report articles from document link lists into Economist-style deep analysis articles. Currently, the TypeScript-generated articles (via
generate-news-enhanced.ts) produce a flat list of documents with generic "Why It Matters" boilerplate, while the agentic workflow prompts explicitly require deep analytical content.📊 Current State
In
scripts/data-transformers.ts, the content generators produce:The "Why It Matters" text is always the same template:
"Touches on {policy area} policy. This motion signals a policy position; passage requires government or majority support."— zero actual analysis.Contrast: The agentic workflow prompts require "Opposition Strategy section analysing which parties are most active and why", "Coalition Dynamics section", and analytical ledes.
🚀 Desired State
Each article type should include:
For Motions:
For Propositions:
For Committee Reports:
🔧 Implementation Approach
Target file:
scripts/data-transformers.ts(generateMotionsContent, generatePropositionsContent, generateCommitteeContent)generateMotionsContent()(~line 1953): Add thematic grouping by proposition reference, party activity breakdown, strategic analysis sectiongeneratePropositionsContent()(~line 1872): Add government priority analysis, policy impact sectionsgenerateCommitteeContent()(~line 1769): Add cross-committee analysis, decision implicationsgenerateDeepPolicyAnalysis(doc)that generates substantive analysis based on document metadatagenerateOppositionStrategySection(motions)helper for party dynamics✅ Acceptance Criteria
📚 References
scripts/data-transformers.tslines 1769-2052 (content generators).github/skills/editorial-standards/SKILL.md— Economist-style standards.github/workflows/news-motions.md— "Each article MUST contain real analysis, not just a list"news/2026-02-opposition-motions-en.html(older, well-analyzed article)news/2026-02-22-opposition-motions-en.html(newer, list-only article)🤖 Recommended Agent
Copilot coding agent — enhance content generators in data-transformers.ts</issue_description>
<agent_instructions>Focus ONLY on the content generation functions in scripts/data-transformers.ts: generateMotionsContent (~line 1953), generatePropositionsContent (~line 1872), and generateCommitteeContent (~line 1769). Enhance these to produce Economist-style analysis: group documents by policy theme, add opposition strategy sections, replace generic 'Why It Matters' boilerplate with substantive analysis. Add helper functions like generateDeepPolicyAnalysis() and generateOppositionStrategySection(). Look at the generatePolicySignificance() function and make it produce real analytical content. Update tests in tests/data-transformers.test.ts. Do NOT modify mcp-client.ts, generate-news-enhanced.ts, title generation code, or any workflow files.</agent_instructions>
Comments on the Issue (you are @copilot in this section)
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.