Floemdata
AI

AI Model Management: Production Versioning, Rollback

Why version an AI model in production and be able to roll it back? We explain how to keep model, data and code versions together.

Floemdata · · 3 min read
A version graph: a main line of five versions with a purple branch that merges back and one highlighted live version

When a model is updated it may answer the same input differently. In production that becomes the question of why yesterday's report and today's report do not match. Model management means knowing which version ran when, with which data and on which code, and being able to go back to an older one when needed.

What should be versioned?

  • The model itself: the weights, or the model version on the provider's side.
  • Training or tuning data: which dated snapshot of data did it learn from?
  • Code and input definitions: features, prompt texts, transformation code.
  • Parameters: thresholds and settings such as temperature.
  • Evaluation result: which test the version passed and with what score.

A rollback strategy

  • Keep the previous version deployable at all times.
  • Run the new version in the shadow first (alongside, with results unused) and compare it with the old one.
  • Write down in advance how a rollback is done and who decides.
  • If a provider changes the model on its side, keep a check that will notice it (a fixed example set).

Monitoring

While a model is in production its performance can degrade over time; the input data changes or a business rule drifts. Measure regularly on the same example set and record the result. When performance drops, look first at version changes and then at data changes.

Where Floemdata stands

Floemdata Analytics does not host a machine learning model today; model management is described here as general knowledge, not a product feature (not verified). An AI assistant is on the roadmap. What the product versions are the agent version and the signed extraction catalog, not models.

To get started, write down the version, the settings and a check example set for each AI tool you use. Questions: info@floemdata.com.

← All guides