Every pipeline — what it produces, where the data comes from, when it last ran, and how to refresh it. Refresh buttons only work when running locally.
| Trigger | Refreshes | UTC | Sydney / AEST * | New York / EST * |
|---|---|---|---|---|
| Post US close | Full refresh (all AI + scores)(L1 → L2 → L3 → L4) | 21:15 | 7:15 AM | 5:15 PM |
| Post ASX open + 30 min | Yahoo-only prices(L2 + L4) | 00:30 | 10:30 AM | 8:30 PM |
| Post ASX close + 15 min | Yahoo-only prices(L2 + L4) | 06:15 | 4:15 PM | 2:15 AM |
Weekdays only · no weekend runs · manual trigger via GitHub Actions workflow_dispatch
* Times adjust automatically for daylight saving. View current timezones here
GitHub Actions scheduled runs can be delayed during periods of high load. Delays of 5–30 minutes are typical; occasionally longer. GitHub docs
Refreshes prices, returns, momentum/dip scores, RSI, and valuation metrics from Yahoo Finance for both GICS ETFs and personal holdings. Skips all AI costs — no Finnhub, no Google News, no Claude calls. Existing AI movement summaries are preserved.
.venv/bin/python -m pipeline.refresh_yahooScores all 25 GICS industry groups on their 6–12 month fundamental outlook (0–100). C1 Analyst Consensus (40%): yfinance analyst recommendations + EPS revisions for the top 5 constituents — also stores buy% and EPS revision%. C2 Structural Tailwinds (35%): one Claude Sonnet call scores all groups simultaneously using 8 universal FRED macro series + Finnhub macro news — also stores tailwinds, headwinds, and cyclical/structural flag. C3 Expert Sentiment (25%): Claude Haiku reads Finnhub news headlines for each group over the last 14 days.
.venv/bin/python -m pipeline.layer1.refreshmarket_contextindustry_group_scores — 25 rowsmarket_context — 1 rowCross-asset daily snapshot. CNN Fear & Greed score drives the speedometer. Six expandable tiles cover US equity, global equity, rates, commodities, FX, and crypto — 24 instruments total via yfinance (batch download). Claude Haiku generates a 2–3 sentence AI narrative. All data upserted to the market_pulse table as a single row (id=1).
.venv/bin/python -m pipeline.layer2.market_pulseDownloads 1 year of daily close prices for the primary ETF of each of the 25 GICS industry groups via yfinance. Computes four return windows (1D, 5D, 1M, 3M), 52-week high/low, % from 52w high, 21-day annualised volatility, RSI (14-day), forward P/E, and PEG ratio. Blends returns into a weighted momentum score — 1D×15% + 5D×20% + 1M×40% + 3M×25% — scaled 0–100. Also generates a 1–2 sentence AI movement summary for each group: Finnhub headlines for the primary ETF plus top 3 GICS constituents are fetched first; if no headlines are found, Google News RSS is queried as a fallback.
.venv/bin/python -m pipeline.layer2.momentummarket_pulse — 1 rowmomentum_scores — 25 rowsopportunity_scores — Dip Score column — refreshed by ③ Opportunity Signal belowCombines Layer 1 macro scores with Layer 2 price returns to compute an Opportunity Score for each of the 25 GICS industry groups. The Dip Signal inverts recent price momentum (bigger pullback = higher signal). Final score = Macro×50% + Dip×50%. A Panic Bonus (+15 pts) fires when a group drops ≥5% in a single day and its macro score is ≥60. Claude Sonnet generates a 3-sentence rationale for the top 5 and bottom 5 groups. Also feeds the Dip Score column on ② What's Moving.
.venv/bin/python -m pipeline.layer3.opportunityAI-generated sector verdicts. Step 1: gathers all Layer 1 + Layer 2 data, fresh FRED indicators, and Google News RSS macro headlines. Step 2: Claude Sonnet identifies 2-3 dominant macro themes and maps scenarios. Step 3: Claude Sonnet produces three verdict buckets — buy opportunities, buying-at-the-top risks, and sectors to avoid — each citing specific data points (returns, RSI, P/E, PEG, EPS revisions).
.venv/bin/python -m pipeline.layer3.macro_scenariosopportunity_scores — 25 rowsmacro_scenarios — 1 rowDownloads up to 6 months of daily close prices for every ticker in the watchlist via yfinance. Computes four return windows (1D, 5D, 1M, 3M) and scores each ticker using the same weighted momentum formula as Layer 2 (1D×15% + 5D×20% + 1M×40% + 3M×25%) and the same dip signal formula as Layer 3 (1D×20% + 5D×50% + 1M×30%). Both formulas are sourced from the shared pipeline.utils.scoring module. US and ASX tickers are supported. Also generates a 1–2 sentence AI movement summary per ticker: US tickers use Finnhub headlines, ASX tickers use yfinance news — if either returns nothing, Google News RSS is queried as a fallback using the company name (ASX queries append 'ASX' for disambiguation). All headlines are passed to Claude Haiku in a single batched call.
.venv/bin/python -m pipeline.layer4.ticker_momentumRe-generates movement_summary for all watchlist tickers without re-downloading price history. Reads existing scores from the DB, fetches fresh headlines per ticker (Finnhub → yfinance → Google News RSS fallback), calls Claude Haiku, and updates only the movement_summary column. Run this when you want to refresh AI summaries without running the full price/score pipeline (which takes much longer due to yfinance downloads).
.venv/bin/python -m pipeline.layer4.refresh_summarieswatchlist_tickers — 53 rowsticker_scores — 53 rows