Progressive model builder for large enterprise data platforms

The system

An enterprise platform generated complex display models from discovered source systems. The original implementation handled first-time creation, but repeated scans could not safely incorporate new objects without risking existing user configuration.

What had to be solved

A full rebuild was technically simpler but functionally wrong. It could overwrite manual ordering, user-created groups and established layer structure. The service was also large enough that adding update behavior directly would make future changes harder to test and reason about.

Non-negotiable requirements

• Existing objects had to be matched across scans using stable business identifiers.
• Manual ordering and user-created layers or groups had to remain untouched.
• A genuinely missing baseline could create a new model; a failed read had to stop safely.
• Current first-run behavior had to remain compatible while the implementation was decomposed.

What I implemented

• Designed distinct CREATE and UPDATE execution paths instead of forcing both scenarios through a rebuild.
• Extracted classification, table filtering, layer discovery and key-matching rules into focused Java components.
• Implemented stable matching using source-model and atomic-origin identifiers.
• Added progressive placement rules that append newly discovered objects without reordering curated content.
• Reduced the main implementation from roughly 2,000 lines to about 1,500 while adding new behavior.

Engineering outcome

• Newly discovered objects can be added during subsequent runs without destroying manual configuration.
• The orchestration code is smaller, clearer and divided into independently testable responsibilities.
• The design supports future classification and ordering rules without extending a single monolithic service.