Supply Chain · Machine Learning · Agentic AI
FlowForge: Agentic Supply Chain Decision Intelligence
End-to-end supply-chain decision system combining reproducible Kedro ML pipelines, leakage-safe late-delivery modeling, deterministic business analytics, and LangChain-based decision support.
- Role
- Personal technical project
- Date
- 2026
Technologies
Context
Context
FlowForge is a supply-chain decision-intelligence prototype that predicts late-delivery risk when an order is created, converts that probability into a revenue-weighted prioritization score, and uses a language model only to explain deterministic analytical results. Prediction and decision support are separate: Python computes probabilities, counts, sales, exposure, and rankings; the large language model (LLM) does not calculate those numerical values.
Snapshot
Final chronological test metrics
Order-item rows
180,519
Raw DataCo extract
Unique orders
65,752
After Kedro aggregation
Test slice
9,863
Final chronological test orders
Test ROC-AUC
0.7490
Ranking quality, not accuracy
Test Brier Score
0.1967
Final chronological test
Late rate (test)
≈ 55.1%
Observed prevalence
Mean P(late) (test)
≈ 56.0%
Mean predicted probability
Problem
Problem
Late-delivery risk has to be estimated at order creation, without leaking post-shipment information into the model.
A raw probability is not an operational ranking: high-value orders and high-risk orders are not the same thing.
A language model should not invent counts, rates, or exposure figures. Those have to come from deterministic tools.
Scope
System scope
Kedro pipeline on the DataCo order dataset
Leakage-safe features at order-creation time
Chronological 70 / 15 / 15 train / validation / test split
Logistic Regression late-delivery probability model
Revenue-weighted exposure ranking
Deterministic analytics tools orchestrated by LangChain + OpenAI
Streamlit Control Tower with Flow View and operational brief
Architecture
Data pipeline
- 01
Raw data
180,519 order-item rows
- 02
Kedro
Aggregation + validation
- 03
Order table
65,752 order-level records
- 04
Leakage-safe features
At order creation
- 05
Temporal split
70% train / 15% validation / 15% test
- 06
Logistic Regression
Binary late-delivery model
- 07
P(late delivery)
Order-level probability
- 08
Business exposure
Probability × net sales
- 09
Deterministic tools
Counts, rates, rankings
- 10
LangChain + OpenAI
Tool selection only
- 11
Operational brief
Headline, summary, recommendations, metrics
Methods
Approach & methods
Aggregated 180,519 order-item rows into 65,752 unique orders in a Kedro pipeline, then applied a leakage-safe feature set and a chronological 70% / 15% / 15% split.
Excluded post-shipment variables from predictive features, including Delivery Status, shipping date, Days for shipping (real), delay_days, and Late_delivery_risk, because using actual shipping duration to predict whether an order will eventually be late would leak future information.
Trained a binary Logistic Regression baseline (StandardScaler on numeric fields, OneHotEncoder(handle_unknown="ignore") on categoricals, L-BFGS, L2 regularization) as a transparent predictive model, not a causal or formally probability-calibrated one.
Converted predicted late-delivery probability into expected_revenue_exposure = probability × net sales. This is used only as a prioritization proxy, not as expected accounting loss, insurance loss, margin, or causal economic impact.
Exposed four deterministic tools (network_summary, risk_hotspots, priority_orders, compare_shipping_modes) through LangChain to OpenAI. Three fixed actions return a structured brief: Analyze Hotspots, Prioritize Interventions, and Compare Shipping Modes. Dashboard filters change the visual analytics; the agent always reads the scored operational test period.
Model
Final chronological test: classification scores
ROC-AUC
0.7490
Not accuracy
Brier Score
0.1967
Accuracy
0.6869
Precision
0.7653
Recall
0.6221
F1
0.6863
Contributions
Contributions
Built the Kedro pipeline from 180,519 order-item rows to 65,752 order-level records, with Parquet intermediates and pandas.
Implemented leakage-safe order-creation features and a chronological train / validation / test split (final test slice: 9,863 orders).
Fit a Logistic Regression baseline and reported final chronological test metrics (ROC-AUC 0.7490, Brier Score 0.1967).
Shipped a Streamlit Control Tower: KPI cards, geographic store-coordinate map, Flow View Sankey, shipping-mode and regional tables, and a LangChain operational brief over deterministic tools.
Product
Control Tower
The Streamlit Control Tower combines historical operational analytics with a decision brief generated from the latest scored test period.
Dashboard filters (market, shipping mode, customer segment) affect the visual analytics. The agent itself always reads the scored operational test period. The language model only summarizes deterministic analytical outputs. It does not compute the KPIs.
Product
Flow View
An SVG Sankey summarizes how orders move from Market to Shipping Mode to actual Delivery Outcome. Band width represents order count; it is an analytical flow visualization, not a physical transportation route map.
Product
Operational analytics
Shipping modes are compared on order volume, actual late rate, scheduled versus actual shipping time, average delay, and gross sales. These are descriptive comparisons and do not imply that a shipping mode causes delays.
Regions can be ranked by observed late-delivery rate within the active dashboard filters. Rankings are observational, not causal.
Agent
Decision support
Deterministic Python analytics feed LangChain tool orchestration, then OpenAI, then a structured Operational Brief. OpenAI does not calculate numerical metrics.
Approved tools: network_summary, risk_hotspots, priority_orders, compare_shipping_modes. Three fixed actions are exposed: Analyze Hotspots, Prioritize Interventions, Compare Shipping Modes. There is no arbitrary Python, unrestricted chat over the raw dataset, or free-form code execution.
The brief contains headline, summary, recommendations, and supporting_metrics. Deterministic tools are the source of numerical truth; the language model selects approved tools and converts their outputs into an executive brief.
Limitations
Limitations
The source is a historical DataCo extract, not a live operational feed.
Geographic points are recorded store coordinates, not shipment routes.
expected_revenue_exposure is a ranking proxy, not expected loss, cost, or margin impact.
Logistic Regression is an intentional transparent baseline, not a state-of-the-art or formally probability-calibrated model, and it is not a causal model of shipping mode or region.
The system does not do inventory, capacity, or routing optimization.
Lessons
Lessons learned
Leakage-safe features at order creation matter more than model complexity on this dataset.
Deterministic tools are the source of numerical truth; the language model only selects approved tools and writes the brief.