Generating & Comparing Combinations

Section: Vendor Rates & Combinations

Overview

The combination engine exists because the client's procurement rule is explicit: never select the cheapest individual rate per charge head in isolation. Ocean freight, inland haulage, procurement cost and every other head interact — a cheap freight vendor paired with an expensive origin location can result in a higher landed cost than a slightly more expensive freight vendor paired with a closer port. The only correct unit of comparison is the complete landed cost for a specific set of rate picks across all heads and a specific port of loading.

To enforce this, the engine computes the landed cost for every valid permutation of vendor rates across charge heads and origin ports. Each permutation runs the same locked calculation sequence that the costing itself uses, producing numbers that are in exact parity with the Excel model the client originally built. The results are stored as ranked Export Cost Combination documents and the cheapest is auto-applied.

Step 1 — Pulling Vendor Rates (Prerequisite)

Before combinations can be generated, submitted Supplier Quotation rates must be snapshotted into the costing's raw_rates child table via Pull Vendor Rates. This step is separate and is not documented here, but two things about it are relevant to understanding combination generation:

• Each raw-rate row captures: charge_item, item_code, port_of_loading, port_of_discharge, container_type, rate, currency, supplier, and an is_expired flag.
• A rate with a blank port_of_loading is a wildcard — it applies to any origin port.

Step 2 — Generate Combinations

Triggered from the Combinations section on the Export Costing form via the Generate Combinations button (calls CombinationEngineMixin.generate_combinations).

2.1 Pre-flight

The engine runs these checks before any computation:

1. Raw rates must exist; if self.raw_rates is empty, it throws.
2. Any currently selected combination is deselected (clear_existing_selection) — regenerating always replaces the previous result.
3. FX rows are ensured for every currency that appears in the raw rates (ensure_exchange_rate_rows_for_currencies). This is done once on the parent costing and persisted so that the transient clones inherit the same FX rates; the eventual selection save reads those same rows.
4. The parent costing is saved.

2.2 Grouping Rates by Head

rates_by_head = group_rates_by_head()

A "head" is the tuple (charge_item, item_code). This means rice procurement and cashew procurement are separate heads, even if both use the same charge item category. Each raw-rate row is assigned to exactly one head.

During grouping, two filters are applied:

2.3 Identifying Variable Heads

variable_heads = ordered_variable_heads(rates_by_head)

A head is "variable" if it has at least one raw-rate row after filtering. The order follows the costing's charges table order, which makes the combination layout deterministic and reproducible.

Fixed charges (those with no corresponding raw-rate rows) are not combinatorially varied; they remain at whatever rate is already in the charge row.

2.4 Determining Origin Ports

ports = origin_ports()

The set of origin ports is derived from the distinct port_of_loading values across all raw_rates rows (including expired ones — port discovery is not filtered). The costing's own port_of_loading is added if it is not already present. If every rate is a wildcard (blank loading port), the engine falls back to the costing's own port as the single combination space.

2.5 Building Candidates per Port

For each origin port, the engine calls compatible_rates_for_port:

def compatible_rates_for_port(port, variable_heads, rates_by_head) -> list | None:
    rates_per_head = []
    for head in variable_heads:
        compatible = [
            rate for rate in rates_by_head[head]
            if not rate.port_of_loading or rate.port_of_loading == port
        ]
        if not compatible:
            return None   # port is skipped entirely
        rates_per_head.append(compatible)
    return rates_per_head

The critical rule: if any variable head has zero compatible rates for a given port, that entire port is skipped. Falling back to the manual charge-row rate would silently hide the gap and produce a misleading comparison. A port only participates if every variable head has at least one rate for it (where a blank-port rate qualifies as compatible with any port).

The total combination count is:

total_count = Σ (over ports) ∏ (over heads for that port) len(compatible_rates_for_head)

2.6 Max-Combinations Cap Guard

def guard_combination_cap(self, total_count: int) -> None:
    settings = get_export_settings()
    if total_count > settings.max_combinations:
        frappe.throw(...)

Before any clone is computed, the total count is checked against Export Settings.max_combinations. If the cap would be exceeded, the engine throws with a message naming the actual count, the cap, and the remediation options (prune expired/duplicate rates, or raise the cap in Export Settings).

Note: The cap is checked against the actual producible count after port pruning (ports with no compatible rate for some head are already excluded from the total). If you have 4 heads each with 5 vendors across 3 fully-compatible ports, that is 5⁴ × 3 = 1,875 combinations.

2.7 Computing Ranked Clones

Stale (non-selected) combinations from a previous generation run are hard-deleted before new ones are inserted.

For each port, itertools.product(*rates_per_head) produces every permutation of rate picks across heads. Each permutation is a tuple of raw-rate rows — one pick per variable head.

For each pick tuple, compute_combination_clone runs:

def compute_combination_clone(self, port, picks):
    clone = frappe.copy_doc(self)
    clone.port_of_loading = port
    clone.reset_quoted_price()
    apply_picks_to_charges(clone, picks)
    clone.set_exchange_rate_buffers()
    clone.calculate_charges()
    clone.calculate_per_mt_costs()
    clone.set_quoted_price()
    clone.calculate_destination_block()
    return clone

Key points:

• frappe.copy_doc produces an in-memory transient copy — it is never saved to the database.
• The cached quoted price is reset on the clone so set_quoted_price computes a price specific to this combination's cost, not a prior selection's.
• apply_picks_to_charges writes the chosen rate, currency, supplier, qty basis, and (if the source is a Supplier Quotation) the SQ link onto the relevant charge rows of the clone.
• The calculation sequence (calculate_charges → calculate_per_mt_costs → set_quoted_price → calculate_destination_block) is identical to what the costing runs on save — this is the Excel-parity guarantee. No separate formula; no approximation.
• The sequence explicitly avoids calling validate() to prevent any database writes or throws from mid-loop validation side-effects.

All computed (port, picks, clone) tuples are sorted ascending by (landed_cost_per_bag, cif_total). The cheapest landed cost per bag is rank 1.

2.8 Persisting as Export Cost Combination Docs

Each sorted entry becomes one Export Cost Combination document, inserted in rank order. The document stores:

The lines child table is the audit record of which rate was picked for each head in this combination. The converted_amount_inr on each line comes from the clone's charge row amount_inr, so the breakdown is in a single currency (INR) for comparability.

2.9 Auto-Select Best

After all combinations are persisted, auto_select_best calls select_combination on the rank-1 document. This means the costing is left with the cheapest combination already applied — the user can see real numbers immediately without having to manually select.

The return value of generate_combinations is {"count": total_count, "best": best_name}.

Step 3 — Combination Comparison Report

The Combination Comparison query report (combination_comparison.py) shows all Export Cost Combination documents for a costing in a single ranked table.

3.1 Filters

Vendor and charge-item filters work by querying Export Cost Combination Line first and then filtering the parent combination documents to those that contain a matching line. This is the correct pattern because the filters live on the child table, not the parent.

3.2 Columns

Why FOB / Freight / Insurance are separate columns: The CFR−FOB and CIF−CFR breakdowns let the user see exactly where the cost difference between two combinations comes from. A combination that looks expensive at FOB might be cheaper at CIF because it uses a more direct shipping lane. The columns make that visible without opening each combination document.

The combinations are fetched ordered by rank asc, so the table is always cheapest-first.

Step 4 — Selecting Another Combination

To override the auto-selected best, the user opens the desired Export Cost Combination document and clicks Select This Combination, which calls select_combination(combination).

What select_combination Does

1. Fetches the combination and its parent costing.
2. Marks any other currently-selected combination as is_selected = 0, status = "Generated" (directly via frappe.db.set_value — no full doc save needed).
3. Writes the combination's lines back into the costing's charges table: rate, currency, supplier, qty basis, and SQ link per charge head.
4. Sets costing.port_of_loading to the combination's origin port.
5. Ensures FX rows exist for the combination's currencies.
6. Resets the costing's cached quoted price so set_quoted_price recomputes it fresh for this combination.
7. Advances costing status to "Costing Selected" (unless it is already "Won" or "Lost").
8. Freezes the backing raw-rate snapshot rows whose (source_name, charge_item, item_code, port_of_loading) keys match the picked lines. Frozen rows survive a re-pull, preserving the audit trail for the selected rates.
9. Saves the costing — the full calculation sequence reruns on the real document.
10. Marks the combination as is_selected = 1, status = "Selected" and saves it.

Parity Check

After the costing saves and recalculates, two assertions run:

PARITY_TOLERANCE_INR = 1.0  # ₹1 on totals that run into crores

if abs(flt(costing.cif_total) - flt(selected.cif_total)) > PARITY_TOLERANCE_INR:
    frappe.throw("Parity check failed: CIF diverged ...")

if abs(flt(costing.quoted_price_per_mt) - flt(selected.quoted_price_per_mt)) > PARITY_TOLERANCE_INR:
    frappe.throw("Parity check failed: quoted price/MT diverged ...")

The tolerance is ₹1.00 on totals that run into crores. If the real costing recomputes a CIF or quoted-price-per-MT that differs by more than ₹1 from what the combination stored at generation time, it means an input changed between generation and selection (an FX row was edited, a charge row was modified, etc.) and the engine throws, requiring regeneration.

Gotcha: Do not relax PARITY_TOLERANCE_INR. The comment in the source says exactly this. A silent drift here means the combination table shows one set of numbers and the actual costing ships different ones to the customer.

Regenerating

Clicking Generate Combinations again:

1. Deselects and unfreezes the currently selected combination (clear_existing_selection → deselect_combination).
2. Hard-deletes all non-selected Export Cost Combination and their Export Cost Combination Line rows.
3. Rebuilds from the current state of raw_rates.
4. Re-ranks and persists new combinations.
5. Auto-selects the new cheapest (rank 1).
6. The new selection's select_combination resets the quoted price and the per-MT figure, so the costing reflects the new best.

The costing status returns to "Rates Received" during deselection (if it was "Costing Selected") and advances back to "Costing Selected" once the new best is auto-applied.

Design Decisions Worth Knowing

Why transient clones and not just summing the picked rates? The landed-cost calculation involves FX conversion, qty-basis normalisation, and a multi-stage charge accumulation (EXW → FOB → CFR → CIF → destination). Reproducing that inline would duplicate logic and drift over time. Running the same methods on an in-memory copy of the document guarantees parity without maintenance overhead.

Why is a missing head fatal for a port rather than falling back to the manual rate? Because the manual rate is whatever was last saved — it might be from a prior supplier, a prior port, or a placeholder. Including it would make the comparison meaningless: the user would see a "combination" that is partially vendor-priced and partially manual, and would not be able to tell which is which.

Why does ordered_variable_heads follow the charges table order? So that itertools.product always produces picks in the same order for a given costing. This makes the combination numbers (and lines rows) deterministic and human-readable when you open a combination document.

Why freeze raw-rate rows on selection rather than on generation? The raw-rate snapshot is a mutable staging area — vendors can re-quote and the user can re-pull. Freezing only the rows that feed the selected combination preserves the flexibility to pull updated rates and regenerate while protecting the audit trail for the combination that was actually used.